An image processing method, apparatus, electronic device and medium

Abstract

The present disclosure relates to an image processing method, apparatus, electronic device, and medium, and relates to the fields of image processing, satellite image processing technology, etc. The method includes: obtaining an image to be processed, where the image to be processed is an image that needs to be color-adjusted; dividing the image to be processed into blocks according to N rows and M columns to obtain respective block images, determining the color deviation values of the respective block images to obtain respective color deviation values, where N and M are positive integers; based on the color deviation values corresponding to the respective block images, performing color adjustment on the image to be processed by means of histogram matching, and performing an elimination operation on the splicing edges of the color-adjusted image to obtain a grayscale image; adjusting the size of the grayscale image, and inputting the grayscale image with the adjusted size into a style transfer neural network to perform style transfer to obtain a target image. The present disclosure realizes efficient and automated image color adjustment and improves the efficiency.

Description

Technical Field

[0001] The present disclosure relates to the technical fields of image processing, satellite image processing, etc. Specifically, the present disclosure relates to an image processing method, apparatus, electronic device, and medium. Background Art

[0002] As a training system, a flight simulator has extremely high requirements for the authenticity and continuity of terrain images. In the prior art, satellite images are generally used to provide terrain image resources for the flight simulator. However, there is an obvious problem of splicing traces in the existing satellite image resources. This is because the satellite takes pictures of the ground at different times and in different seasons. Due to reasons such as lighting conditions and vegetation changes, there are large differences in tone and content among the images. Simply splicing these images will result in discontinuous terrain and affect the pilot's judgment.

[0003] In the prior art, in order to solve the problem of obvious splicing traces in satellite images, the following several technical solutions are generally adopted for processing:

[0004] 1. Manual PS processing method: Engineers use image processing software (such as Photoshop, abbreviated as PS) and other software to perform overall color adjustment on satellite images, and use tools such as layer fusion and stamp to repair the splicing edges and areas with large color differences, so as to make the images more coordinated and unified.

[0005] 2. Image selection and synthesis method: By comparing multiple satellite picture sources, select the pictures with better quality in the same area for comprehensive processing.

[0006] 3. Histogram matching method: Using histogram matching technology, adjust the histogram characteristics of the image in the area to be processed to be the same as that of the target area Figure 1 so as to achieve color correction and unification.

[0007] However, the above methods of the prior art have the following problems:

[0008] 1. Although the method of manual post-PS processing can improve the image effect, it has a large workload, low efficiency, and may also change the original terrain information.

[0009] 2. The image selection and synthesis method requires a lot of manual comparison and analysis work, has a large workload, low efficiency, and cannot completely eliminate the color difference and splicing traces between images. Different sources need to be purchased separately, increasing the cost.

[0010] 3. The histogram matching method has high requirements for region selection and determination of the target histogram, and there are also certain limitations in the processing effect.

[0011] Therefore, how to efficiently eliminate the splicing traces of satellite images and maintain the accuracy of terrain information has become an urgent problem to be solved. Summary of the Invention

[0012] Embodiments of the present disclosure provide an image processing method, apparatus, electronic device, and medium, which achieve efficient and automated image color adjustment for satellite images, solve the problem of obvious splicing traces in satellite images, and improve the processing efficiency.

[0013] On the one hand, embodiments of the present disclosure provide an image processing method, which includes:

[0014] Obtain an image to be processed, where the image to be processed is an image that needs to be color-adjusted;

[0015] Divide the image to be processed into blocks according to N rows and M columns to obtain each block image, and determine the color deviation value of each block image to obtain each color deviation value, where N and M are positive integers;

[0016] Based on the color deviation values corresponding to each block image, perform color adjustment on the image to be processed by means of histogram matching, and perform an elimination operation on the splicing edge of the image to be processed after color adjustment to obtain a grayscale image;

[0017] Adjust the size of the grayscale image, and input the grayscale image after size adjustment into a style transfer neural network for style transfer to obtain a target image.

[0018] In an optional embodiment, the adjusting the size of the grayscale image, inputting the grayscale image after size adjustment into a style transfer neural network for style transfer to obtain a target image includes:

[0019] Shrink the grayscale image according to a first size to obtain a grayscale image of the first size, where the image size of the grayscale image is an image size not supported by the image processing device, the first size is an image size supported by the image processing device, and the image processing device is a device for processing the image to be processed;

[0020] Obtain a target style image, input the target style image and the grayscale image of the first size into a style transfer neural network, and perform style transfer on the grayscale image of the first size based on the target style image to obtain a first colored image, where the image size of the target style image is the same as that of the grayscale image of the first size;

[0021] Repeat the following operations until a preset end condition is met:

[0022] Divide the first colored image into four parts according to a second size to obtain four second colored images, where the first size is twice the second size;

[0023] For any second-colored image, enlarge the second-colored image according to the first size to obtain a second-colored image of the first size, and denote the second-colored image of the first size as the third-colored image;

[0024] Intercept the image area corresponding to the area where the second-colored image is located in the above grayscale image as the target image area, and determine whether the image size of the above target image area is the first size. If not, reduce the above target image area to the first size. If so, do not perform the operation of reducing the size of the above target image area;

[0025] Use the above third-colored image as the target style image, use the target image area of the first size as the content image, and perform style transfer on the above target image area to obtain a fourth-colored image;

[0026] Until the fourth-colored images corresponding to each second-colored image are obtained, and use each fourth-colored image as a new first-colored image;

[0027] Determine whether the preset end condition is satisfied. If not, perform the above operations on each new first-colored image. If satisfied, splice the new first-colored images to obtain the above target image, where the above preset end condition is not to perform the operation of reducing the size of the above target image area and obtaining the new first-colored images.

[0028] In an alternative embodiment, the above color adjustment of the image to be processed by histogram matching based on the above color deviation values corresponding to each of the above divided images includes:

[0029] Perform a clustering operation on the above color deviation values corresponding to each of the above divided images to obtain the largest clustering set;

[0030] Determine the mean value of the color deviation values in the above largest clustering set to obtain the average color deviation value;

[0031] Determine the color deviation value with the smallest absolute difference from the average color deviation value among the above color deviation values to obtain the target color deviation value;

[0032] Determine the target divided image corresponding to the above target color deviation value from each of the above divided images;

[0033] Statistically analyze the histogram of the above target divided image, and perform histogram matching on each of the above divided images other than the above target divided image based on the histogram of the above target divided image to obtain each of the other divided images after histogram matching;

[0034] Stitch the other block images after matching the above target block image with the histogram to obtain a fused image, where the fused image is an image obtained by performing color adjustment on the image to be processed.

[0035] In an alternative embodiment, performing an elimination operation on the stitching edges of the image after the color adjustment to obtain a grayscale image includes:

[0036] Create a transfer image with all pixels being 1, where the transfer image has the same image size as the fused image;

[0037] For any two block images to be processed, determine the brightness ratio of the two block images to be processed, and assign values to the transfer image based on the brightness ratio to obtain a weight image; where any two of the above block images to be processed are any two images among the other block images after matching the target block image with the histogram;

[0038] Multiply the fused image by the weight image to obtain a multiplied image;

[0039] Perform grayscale conversion on the multiplied image to obtain the grayscale image.

[0040] In an alternative embodiment, the image to be processed is a satellite image to be processed, and the image size of the satellite image to be processed is an image size not supported by the image processing device, where the image processing device is a device for processing the satellite image to be processed.

[0041] On the one hand, an embodiment of the present disclosure provides an image processing apparatus, which includes:

[0042] An acquisition module, configured to acquire an image to be processed, where the image to be processed is an image that needs to be color-adjusted;

[0043] A determination module, configured to divide the image to be processed into blocks according to N rows and M columns to obtain each block image, and determine the color deviation value of each block image to obtain each color deviation value, where N and M are positive integers;

[0044] A first processing module, configured to perform color adjustment on the image to be processed by means of histogram matching based on the color deviation values corresponding to each block image, and perform an elimination operation on the stitching edges of the image to be processed after the color adjustment to obtain a grayscale image;

[0045] A second processing module, configured to adjust the size of the grayscale image, and input the grayscale image with the adjusted size into a style transfer neural network for style transfer to obtain a target image.

[0046] In an alternative embodiment, the above-mentioned second processing module is specifically configured to:

[0047] Reduce the above grayscale image according to a first size to obtain a grayscale image of the first size, where the image size of the above grayscale image is an image size not supported by the image processing device, the above first size is an image size supported by the image processing device, and the above image processing device is a device for processing the above image to be processed;

[0048] Obtain a target style image, input the above target style image and the grayscale image of the first size into a style transfer neural network, and perform style transfer on the grayscale image of the first size based on the above target style image to obtain a first colored image, where the image size of the above target style image is the same as that of the grayscale image of the first size;

[0049] Repeat the following operations until a preset end condition is met:

[0050] Divide the above first colored image into four parts according to a second size to obtain four second colored images, where the above first size is twice the above second size;

[0051] For any one of the second colored images, enlarge the second colored image according to the first size to obtain a second colored image of the first size, and denote the second colored image of the first size as a third colored image;

[0052] Intercept the image area corresponding to the area where the second colored image is located in the above grayscale image as a target image area, and determine whether the image size of the above target image area is the first size. If not, reduce the above target image area to the first size. If so, do not perform the operation of reducing the size of the above target image area;

[0053] Use the above third colored image as the target style image, use the target image area of the first size as the content image, and perform style transfer on the above target image area to obtain a fourth colored image;

[0054] Until the fourth colored images corresponding to each of the second colored images are obtained, and use each of the fourth colored images as a new first colored image;

[0055] Determine whether the preset end condition is met. If not, perform the above operations on each new first colored image. If so, splice the new first colored images to obtain the above target image, where the above preset end condition is not to perform the operation of reducing the size of the above target image area and obtaining the new first colored images.

[0056] In an alternative embodiment, the above-mentioned first processing module is specifically configured to:

[0057] Perform a clustering operation on the above color deviation values corresponding to each of the above segmented images to obtain the largest clustering set;

[0058] Determine the mean value of the color deviation values in the above largest clustering set to obtain the average color deviation value;

[0059] Determine the color deviation value with the smallest difference from the absolute value difference between the above average color deviation value among the above color deviation values to obtain the target color deviation value;

[0060] Determine the target segmented image corresponding to the above target color deviation value from each of the above segmented images;

[0061] Statistically analyze the histogram of the above target segmented image, and perform histogram matching on each of the above segmented images other than the above target segmented image based on the histogram of the above target segmented image to obtain each of the other segmented images after histogram matching;

[0062] Stitch the above target segmented image and each of the above other segmented images after histogram matching to obtain a fused image, where the above fused image is an image obtained by adjusting the color of the above image to be processed.

[0063] In an optional embodiment, the above first processing module is specifically configured to:

[0064] Create a transit image with all pixels being 1, where the above transit image has the same image size as the above fused image;

[0065] For any two images to be processed, determine the brightness ratio of the two images to be processed, and assign values to the above transit image based on the brightness ratio to obtain a weight image; where any two of the above images to be processed are any two images among the above target segmented image and each of the above other segmented images after histogram matching;

[0066] Multiply the above fused image by the above weight image to obtain a multiplied image;

[0067] Convert the above multiplied image to grayscale to obtain the above grayscale image.

[0068] In an optional embodiment, the above image to be processed is a satellite image to be processed, and the image size of the satellite image to be processed is an image size not supported by the image processing device, where the above image processing device is a device for processing the satellite image to be processed.

[0069] On the one hand, embodiments of the present disclosure provide an electronic device, which includes a processor and a memory. The processor and the memory are interconnected. The memory is used to store a computer program. The processor is configured to execute the method provided by any possible implementation of the above image processing method when calling the above computer program.

[0070] On the one hand, embodiments of the present disclosure provide a computer-readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the method provided by any possible implementation of the above image processing method.

[0071] On the one hand, embodiments of the present disclosure provide a computer program product or a computer program, which includes computer instructions. The computer instructions are stored in a computer-readable storage medium. The processor of the electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided by any possible implementation of the above image processing method.

[0072] The beneficial effects brought by the technical solution provided in this application are as follows: Obtain an image to be processed, where the image to be processed is an image that needs to be color-adjusted. Divide the image to be processed into blocks according to the specified N rows and M columns, and each block image after division can be obtained, and determine the respective color offset values corresponding to each block image to obtain each color offset value, where N and M are positive integers. Based on the obtained color offset values, perform color adjustment on the image to be processed by means of histogram matching, and eliminate the splicing edges of the image to be processed after color adjustment to obtain a grayscale image. Adjust the size of the grayscale image, and input the grayscale image with the adjusted size into a style transfer neural network for style transfer to obtain a target image. In the prior art, although the method of manual post-PS processing can improve the image effect, the workload is large, and at the same time, the original terrain information may be changed. In the embodiments of the present disclosure, through an automated satellite image color adjustment method, not only can the splicing traces be eliminated, but also the accuracy of the terrain information can be maintained, meeting the needs of flight simulation training. Through an automated satellite image processing method, not only can the existing image quality problems be solved, but also the processing efficiency can be greatly improved, providing a more realistic and reliable terrain environment for flight simulation training. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following will briefly introduce the drawings required for description in the embodiments of the present disclosure.

[0074] Figure 1 It is a schematic flowchart of an image processing method provided by an embodiment of the present disclosure;

[0075] Figure 2 Schematic diagram of a to-be-processed image with different illuminations in a regular area provided by an embodiment of the present disclosure;

[0076] Figure 3 Schematic diagram of a to-be-processed image with different times in an irregular area provided by an embodiment of the present disclosure;

[0077] Figure 4 Schematic diagram of a to-be-processed image with inconsistent overall image illumination and other images provided by an embodiment of the present disclosure;

[0078] Figure 5 Schematic diagram of a to-be-processed image with cloud shadows in an irregular area provided by an embodiment of the present disclosure;

[0079] Figure 6 Schematic diagram of a to-be-processed image with the original image in an irregular area already fused provided by an embodiment of the present disclosure;

[0080] Figure 7 Schematic diagram of a grayscale image provided by an embodiment of the present disclosure;

[0081] Figure 8 Schematic diagram of a target image provided by an embodiment of the present disclosure;

[0082] Figure 9 Schematic diagram of the structure of an image processing device provided by an embodiment of the present disclosure;

[0083] Figure 10 Schematic diagram of the structure of an electronic device for an image processing method provided by an embodiment of the present disclosure. Detailed implementation manners

[0084] The embodiments of the present disclosure will be described below with reference to the accompanying drawings in the present disclosure. It should be understood that the embodiments described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present disclosure, and do not constitute limitations on the technical solutions of the embodiments of the present disclosure.

[0085] Those skilled in the art can understand that, unless specifically stated otherwise, the singular forms "a", "an", "the above", and "the" used herein may also include the plural forms. It should be further understood that the terms "include" and "comprise" used in the embodiments of the present disclosure mean that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements, and / or components, but do not exclude the implementation of other features, information, data, steps, operations, elements, components, and / or their combinations, etc. supported by the technical field of the present disclosure. It should be understood that when we say an element is "connected" or "coupled" to another element, the one element can be directly connected or coupled to the other element, or it can mean that the one element and the other element establish a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used here can include wireless connection or wireless coupling. The term "and / or" used here indicates at least one of the items defined by the term, for example, "A and / or B" or "A, B" indicates being implemented as "A", or being implemented as "B", or being implemented as "A and B".

[0086] To make the objectives, technical solutions, and advantages of the present disclosure clearer, the embodiments of the present disclosure will be further described in detail below with reference to the accompanying drawings.

[0087] See Figure 1 , Figure 1 is a schematic flowchart of an image processing method provided by an embodiment of the present disclosure. As Figure 1 shown, the image processing method provided by the present disclosure includes the following steps:

[0088] Step S101: Obtain an image to be processed, where the image to be processed is an image that needs to be color-adjusted;

[0089] Step S102: Divide the image to be processed into blocks according to N rows and M columns to obtain each block image, and determine the color deviation value of each block image to obtain each color deviation value, where N and M are positive integers;

[0090] Step S103: Based on the color deviation value corresponding to each block image, adjust the color of the image to be processed by means of histogram matching, and perform an elimination operation on the splicing edge of the image to be processed after color adjustment to obtain a grayscale image;

[0091] Step S104: Adjust the size of the grayscale image, and input the grayscale image with the adjusted size into a style transfer neural network for style transfer to obtain a target image.

[0092] Optionally, the image processing method in the embodiments of the present disclosure can be executed by an image processing device. Among them, the image processing device can be a server, which can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server or server cluster that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The image processing device can also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a laptop computer, a digital broadcast receiver, a MID (Mobile Internet Devices), a PDA (Personal Digital Assistant), a desktop computer, a vehicle-mounted terminal (such as a vehicle-mounted navigation terminal), a smart speaker, a smart watch, etc.

[0093] The network used by the above image processing method during image processing can include, but is not limited to: wired networks and wireless networks. Among them, the wired network includes: local area networks, metropolitan area networks, and wide area networks, and the wireless network includes: Bluetooth, Wi-Fi, and other networks that implement wireless communication.

[0094] Optionally, in the embodiments of the present disclosure, the above image processing method specifically includes the following steps:

[0095] Obtain an image to be processed. Among them, in an optional embodiment, the image to be processed is a satellite image to be processed, and the image size of the satellite image to be processed is an image size not supported by the image processing device, where the image processing device is a device for processing the satellite image to be processed.

[0096] Specifically, the image to be processed belongs to a large-scale image that needs to be color-adjusted and is a satellite image. Large-scale can be understood as a situation where the resolution of the satellite image to be processed is very large, such as an image with a size of 16384*16384. It can be understood that the embodiments of the present disclosure do not limit the size of the satellite image. The embodiments of the present disclosure are applicable to the situation where there is only one source image in a geographical area and the satellite image needs to be color-adjusted.

[0097] Optionally, the images to be processed in the embodiments of the present disclosure can be classified into the following types: different illuminations in regular regions ( Figure 2 ), different times in irregular regions ( Figure 3 ), inconsistent illumination of the whole image with other images ( Figure 4 ), cloud shadows in irregular regions ( Figure 5 ), and the original image in irregular regions has been fused ( Figure 6).

[0098] Among them, Figure 2 is a schematic diagram of an image to be processed with different illuminations in a regular area provided by an embodiment of the present disclosure. As Figure 2 shown, Figure 2 there are two regular areas in it. The two areas are photographed by the satellite in different illumination environments within a relatively short time (such as within a few days), and then spliced together. The content of this image only has differences in the main color, and there is no significant difference in the content. Among them, the regular area refers to a simple concave-convex polygon that can be formed by a small number of points.

[0099] Figure 3 is a schematic diagram of an image to be processed with different times in an irregular area provided by an embodiment of the present disclosure. As Figure 3 shown, Figure 3 is photographed by the satellite in an approximately similar illumination environment but at a relatively long time (such as more than a few weeks). The spliced area is irregular. The regular area refers to a simple concave-convex polygon that can be formed by a small number of points. Among them, the irregular area refers to an area that cannot be selected by a simple polygon.

[0100] Figure 4 is a schematic diagram of an image to be processed with inconsistent illumination of the whole image with other images provided by an embodiment of the present disclosure. As Figure 4 shown, Figure 4 the whole image of [[ID]] is an image photographed by the satellite under non-ideal illumination environment, season, time and other conditions. There is no obvious hue difference in this image, and the whole image has a unified hue, and the whole image needs to be color-adjusted.

[0101] Figure 5 is a schematic diagram of an image to be processed with cloud shadows in an irregular area provided by an embodiment of the present disclosure. As Figure 5 shown, Figure 5 is that there is a thin cloud under the satellite during shooting, and it shows an irregular area with shadows or an irregular area that turns white on the satellite image.

[0102] Figure 6 is a schematic diagram of an image to be processed with the original image of an irregular area already fused provided by an embodiment of the present disclosure. As Figure 6 shown, Figure 6 is an image that has been modified and adjusted by others when the above several situations are obtained.

[0103] Optionally, after obtaining the image to be processed in the above manner, the image to be processed can be block-processed. For example, it can be blocked into N rows and M columns to obtain each block image (if the image to be processed is a satellite image to be processed, this block image can also be called a satellite image block or a satellite image piece). Then, calculate the color deviation value corresponding to each block image to obtain each color deviation value, where N and M are positive integers. It should be noted that the color deviation value refers to the difference between the formula calculated by the computer and the target standard. In image processing, color deviation means that the color of the image is different from the original hue, usually caused by improper exposure time, negative film, flash, etc. The calculation of the color deviation value usually involves color comparison and difference calculation. In computer calculation, the color deviation value refers to the difference between the formula calculated by the computer under a certain light source and the target standard. The smaller the value, the higher the accuracy. Among them, the calculation of the color deviation value is a prior art, and the embodiments of the present disclosure will not be introduced in detail here.

[0104] Optionally, based on the color deviation values corresponding to each block image, the image to be processed is color-adjusted by means of histogram matching. Optionally, the image to be processed can be color-adjusted in the following manner: Based on the color deviation values corresponding to each block image, the maximum color deviation value set corresponding to each color deviation value (corresponding to the maximum clustering set in the text) is determined by clustering, and the mean value of each color deviation value in the maximum color deviation value set is calculated. Select the block image whose color deviation value is closest to this mean value and denote it as the target block image. Perform histogram matching on each of the other block images except the target block image with the target block image. Through this histogram matching method, the above-mentioned image to be processed is color-adjusted to obtain the image to be processed with corrected colors.

[0105] Optionally, the splicing edge of the image to be processed after color adjustment can be eliminated by the iterative color fusion method to obtain a grayscale image. This method applies a gradient color weight at the splicing edge to achieve natural transition and eliminate obvious splicing traces, which can greatly improve the visual effect of the image. For details, see the following description.

[0106] Optionally, the image size of the grayscale image obtained in the above manner is adjusted to a certain extent to obtain a grayscale image with an adjusted size that meets the requirements. If the image size of the grayscale image meets the requirements, the operation of size adjustment is not performed. In practical applications, the grayscale image corresponding to the image to be processed is a large-scale image. Due to the GPU video memory and computing power limitations of the image processing device, the image processing device cannot support processing this large-scale grayscale image. Therefore, the size of this grayscale image needs to be adjusted. Assuming that the image size of this grayscale image is an image of 16384*16384, the image size of this grayscale image can be adjusted to 2048*2048. It can be understood that the embodiments of the present disclosure do not limit the image size of the grayscale image after size adjustment.

[0107] Then, the obtained grayscale image with adjusted size is input into the style transfer neural network, and style transfer is performed on the grayscale image with adjusted size to obtain a target image that meets the target style. For the detailed process, please refer to the following description.

[0108] Through the embodiments of the present disclosure, in the prior art, manual post - processing with PS is used to handle stitching traces. Although this method can improve the image effect, it has a large workload and may also change the original terrain information. Compared with the prior art, the embodiments of the present disclosure make full use of the advantages of deep learning, realize the full automation and intelligence of image processing, and greatly improve the quality and efficiency of image processing. Moreover, the embodiments of the present disclosure can perform well in different processing tasks such as different illuminations in regular areas, different times in irregular areas, inconsistent illuminations of the whole map with other maps, cloud shadows in irregular areas, and the original image of the irregular area has been fused. The processing results have a unified style and meet the actual usage requirements.

[0109] To more clearly illustrate the process of color - adjusting the image to be processed, in an optional embodiment, the above - mentioned color - adjusting the image to be processed by histogram matching based on the respective color deviation values corresponding to each of the above - mentioned sub - images includes:

[0110] Performing a clustering operation on the respective color deviation values corresponding to each of the above - mentioned sub - images to obtain the largest clustering set;

[0111] Determining the mean value of the color deviation values in the above - mentioned largest clustering set to obtain the average color deviation value;

[0112] Determining the color deviation value with the smallest absolute - value difference from the average color deviation value among the above - mentioned color deviation values to obtain the target color deviation value;

[0113] Determining the target sub - image corresponding to the above - mentioned target color deviation value from each of the above - mentioned sub - images;

[0114] Counting the histogram of the above - mentioned target sub - image, and performing histogram matching on each of the other sub - images except the above - mentioned target sub - image among the above - mentioned sub - images based on the histogram of the above - mentioned target sub - image to obtain the other sub - images after histogram matching;

[0115] Stitching the above - mentioned target sub - image and the other sub - images after histogram matching to obtain a fused image, where the above - mentioned fused image is the image after color - adjusting the above - mentioned image to be processed.

[0116] Optionally, the embodiments of the present disclosure combine an example to elaborate in detail on how to color - adjust the image to be processed, that is, how to determine the fused image. The specific process is as follows:

[0117] Step 1: Divide the image to be processed (i.e., the original image), which is a satellite image to be processed. The resolution of the original image is 16384*16384 (including but not limited to this size). Divide the original image into sub-images of N rows and M columns (i.e., satellite image blocks), denoted as Pij (1 <= i <= N, 1 <= j <= M).

[0118] Step 2: Calculate the color deviation value of each sub-image, that is, calculate the color deviation value of image Pij respectively to obtain each color deviation value, denoted as dij.

[0119] Step 3: Cluster each color deviation value using the k-means clustering algorithm. The number of clusters can be set to 3 to obtain the largest cluster set. It can be understood that in the embodiments of the present disclosure, no limitation is imposed on the number of clusters.

[0120] Among them, the k-means clustering algorithm is an iterative clustering analysis algorithm. Its steps are as follows: First, divide the data (i.e., each color deviation value) into K groups, then randomly select K objects (i.e., randomly select K color deviation values) as the initial cluster centers, and then calculate the distance between each object and each seed cluster center, and assign each object to the cluster center closest to it. The cluster center and the objects assigned to it represent a cluster. Each time a sample is assigned, the cluster center of the cluster will be recalculated based on the existing objects in the cluster. This process will continue to repeat until a certain termination condition is met. The termination condition can be that no (or the minimum number of) objects are reassigned to different clusters, no (or the minimum number of) cluster centers change anymore, and the sum of squared errors is locally minimized.

[0121] It can be understood that in the embodiments of the present disclosure, no limitation is imposed on which clustering method is used. In practical applications, other methods such as partitioning methods, hierarchical methods, density-based methods, grid-based methods, model-based methods, transitive closure methods, Boolean matrix methods, direct clustering methods, correlation analysis clustering, and statistical-based clustering methods can also be used.

[0122] Step 4: Calculate the mean of each color deviation value in the largest cluster set to obtain the average color deviation value da.

[0123] Step 5: Compare the absolute value differences between all dij and da, and select the Pij corresponding to the dij with the smallest absolute value difference from the average color deviation value da, denoted as Pa (corresponding to the target sub-image);

[0124] Step 6: Statistically analyze the histogram of Pa, perform histogram matching on other Pij (i.e., each sub-block image except the target sub-block image), obtain the matched image Qij (i.e., each sub-block image after histogram matching), and splice Pa and the matched image Qij together by rows and columns to obtain image Q, which is the fused image.

[0125] Through the embodiments of the present disclosure, based on the color deviation values of each sub-block image, the hue of the largest area is statistically analyzed by means of clustering, and then color adjustment is performed by means of histogram matching based on this hue, which can make the overall hue of the obtained fused image more unified and harmonious, achieve a unified hue effect, eliminate obvious splicing traces, and can greatly improve the visual effect of the image.

[0126] Optionally, after obtaining the fused image, in order to make the edges of the fused image more natural and smooth, the fused image can also be iteratively edge-fused. In an optional embodiment, the above-mentioned elimination operation on the splicing edges of the image after the above-mentioned color adjustment to obtain a grayscale image includes:

[0127] Create a transfer image with all pixels being 1, where the size of the transfer image is the same as that of the fused image;

[0128] For any two sub-block images to be processed, determine the brightness ratio of the two sub-block images to be processed, and assign values to the transfer image based on this brightness ratio to obtain a weight image; where any two of the above-mentioned sub-block images to be processed are any two images among the target sub-block image and the other sub-block images after histogram matching;

[0129] Multiply the above-mentioned fused image by the above-mentioned weight image to obtain a multiplied image;

[0130] Convert the above-mentioned multiplied image to grayscale to obtain the above-mentioned grayscale image.

[0131] Optionally, in combination with the method for determining the fused image described above, the embodiments of the present disclosure provide a method for determining a grayscale image, and the specific process is as follows:

[0132] Step 1: Create a new image D (i.e., the above-mentioned intermediate image) with all pixel values being 1 and a resolution of 16384 * 16384. The resolution of this image D is the same as that of the fused image Q. It can be understood that the resolution of image D is not limited in the embodiments of the present disclosure. Calculate the brightness ratio between the image Qij and the corresponding adjacent image. The specific method is as follows: Taking any two adjacent images as an example, assume the two adjacent images are Qa and Qb (corresponding to any two to-be-processed sub-block images mentioned above). Assume the coordinates of a certain pixel on the boundary of image Qa are (xe, ye). First, calculate the average value of all pixels within a side length of 100 pixels centered on this pixel and belonging to Qa, denoted as AverP. Calculate the average value of all pixels within a side length of 100 pixels centered on this pixel and belonging to the image Qb adjacent to Qa, denoted as AverN. The brightness ratio is R = AverP / AverN, then assign the value of point (xe, ye) in image D to R. After all pixel points in image D are assigned values, a weight image is obtained.

[0133] Step 3: Traverse the weight image D. If there is a calculated brightness ratio (not 1) within the 3 * 3 area near a certain pixel point of the weight image D, then the brightness ratio value of this point is the average of the brightness ratios (not 1) existing within the 3 * 3 area nearby, otherwise it is 1. Repeat Step 3 1000 times. Finally, a gray-scale brightness weight transition band with a width of 1000 can be obtained.

[0134] Step 4: Multiply the image Q (i.e., the fused image) by the weight image D to obtain the multiplied image E. Among them, after multiplying based on the weight transition band corresponding to the weight image D and the image Q, the gray-scale gap between the adjacent contact boundaries of the image Q (i.e., any image Qij and the corresponding adjacent image, that is, any two to-be-processed sub-block images mentioned above) can be largely eliminated, achieving smooth transition of the image. This step can be selected according to the situation. If the gap is too small, it can be not executed. If the gap is too large, this step is adopted. The size of the gap can be specified in advance according to requirements.

[0135] Step 5: Convert the multiplied image E into a gray-scale image to obtain image F (i.e., the gray-scale image), as Figure 7 shown, which is a schematic diagram of the gray-scale image obtained through the embodiments of the present disclosure.

[0136] Through the embodiments of the present disclosure, by adopting the above iterative color fusion method and applying a gradient color weight at the stitching edge, the stitching edge of the fused image can achieve smooth transition, effectively eliminating the gray-scale gap at the stitching edge, improving the accuracy of image processing, and improving the image quality.

[0137] In an alternative embodiment, the above-mentioned resizing of the gray-scale image, inputting the resized gray-scale image into a style transfer neural network for style transfer to obtain a target image, includes:

[0138] Reduce the above grayscale image to the first size to obtain a grayscale image of the first size, where the image size of the above grayscale image is an image size not supported by the image processing device, the above first size is an image size supported by the above image processing device, and the above image processing device is a device for processing the above image to be processed;

[0139] Obtain a target style image, input the above target style image and the grayscale image of the first size into a style transfer neural network, and perform style transfer on the grayscale image of the first size based on the above target style image to obtain a first colored image, where the image sizes of the above target style image and the grayscale image of the first size are the same;

[0140] Repeat the following operations until a preset end condition is met:

[0141] Divide the above first colored image into four parts according to the second size to obtain four second colored images, where the above first size is twice the above second size;

[0142] For any one of the second colored images, enlarge the second colored image to the first size to obtain a second colored image of the first size, and denote the second colored image of the first size as the third colored image;

[0143] Intercept the image area corresponding to the area where the second colored image is located in the above grayscale image as the target image area, and determine whether the image size of the above target image area is the first size. If not, reduce the above target image area to the first size. If so, do not perform the operation of reducing the size of the above target image area;

[0144] Use the above third colored image as the target style image, use the target image area of the first size as the content image, and perform style transfer on the target image area to obtain a fourth colored image;

[0145] Until the fourth colored images corresponding to each of the second colored images are obtained, and use each of the fourth colored images as the new first colored images;

[0146] Determine whether the preset end condition is met. If not, perform the above operations on each new first colored image. If so, splice the new first colored images to obtain the above target image, where the preset end condition is not to perform the operation of reducing the size of the above target image area and obtaining the new first colored images.

[0147] Optionally, in the embodiments of the present disclosure, style transfer is performed through a style transfer neural network. The main idea is to adopt an exponential hierarchical method. Using the style transfer neural network, style transfer is performed on a high-resolution grayscale image through a low-resolution colored map to gradually improve the accuracy. Among them, in practical applications, the style transfer neural network can also be optimized through an optimization algorithm (Limited-memory BFGS, abbreviated as LBFGS). Among them, LBFGS is an optimizer for the style transfer neural network. The specific steps are as follows:

[0148] Step 1: Perform an operation to reduce the size of the image F (i.e., the above grayscale image). The image size of this image F is 16384*16384, which is reduced to 2048*2048 (i.e., the above first size). Denote the image F after reducing the size as F1 (i.e., the grayscale image of the above first size). Among them, the purpose of performing the operation to reduce the size is as follows: First, due to the limitations of the style transfer algorithm, the graphics card cannot support an image as large as 16384*16384, and the video memory can support at most an image of 2048*2048 for operation. Second, it is to meet the input requirements of the style transfer neural network.

[0149] Then, perform style transfer on the image F1 and the target style image to obtain a colored map G with a size of 2048*2048 (corresponding to the above first colored map). Among them, the image size of the above target style image is the same as that of the above grayscale image. The selection criterion of the above target style image can be determined according to actual needs and is an artificially selected ideal color style of the satellite image. For example, the color style of the above target style image can be that most of it is green forest, with a part of towns and a part of loess, etc. This criterion is related to the geographical location being processed. For example, if the satellite image being photographed is a desert area, a satellite image with a large area of yellow can be selected as the target style image.

[0150] Perform the following operations until the preset end condition is met. Among them, the preset end condition is not to perform the operation of reducing the size of the target image area and to obtain new first colored maps:

[0151] Step 2: Divide the colored map G (corresponding to the above first colored map) into four parts according to the size of 1024*1024 (i.e., the above second size), and denote them as G1 (x: 0-1024, y: 0-1024), G2 (x: 1024-2048, y: 0-1024), G3 (x: 0-1024, y: 1024-2048), and G4 (x: 1024-2048, y: 1024-2048). Among them, G1, G2, G3, and G4 correspond to the above four second colored maps.

[0152] Taking G1 among G1, G2, G3, and G4 as an example, G1 is enlarged to 2048*2048 to obtain G1 with a size of 2048*2048 (the second colored image corresponding to the above first size, that is, the third colored image), and G1 with a size of 2048*2048 (i.e., the third colored image) is used as the target style image. The area corresponding to the image content of image F and G1 (x: 0 - 8192, y: 0 - 8192) is intercepted, and the intercepted area is used as the target image area. It is judged whether the image size of the target image area is the first size. If not, the target image area is reduced to the first size. If so, the operation of reducing the size of the target image area is not performed on the target image area. It can be known that at this time, the size of the target image area does not meet the first size, that is, the image size does not meet 2048*2048, so the target image area is reduced to the first size of 2048*2048. The target image area with the first size (i.e., 2048*2048) is used as the content image for style transfer, and G1 with a size of 2048*2048 (i.e., the third colored image) is used as the target style image for style transfer to obtain a colored image H1 with a size of 2048*2048 (i.e., the above fourth colored image). Using the same method to process G2, G3, and G4, the corresponding colored images H2, H3, and H4 can be obtained, which are not elaborated in this embodiment of the present disclosure.

[0153] Finally, four fourth colored images H1, H2, H3, and H4 corresponding to the first colored image G can be obtained, and each of the obtained fourth colored images H1, H2, H3, and H4 is used as a new first colored image.

[0154] It is judged whether the preset end condition is met. If not, step 3 is continued to be executed at this time.

[0155] In step 3, using the same method as in step 2 above, each of the first colored images H1... H4 is processed to obtain the corresponding fourth colored images I1, I2... I16. The following takes the colored image H1 as an example for illustration, and the specific process is as follows:

[0156] The colored image H1 (corresponding to the above first colored image) is divided into four parts according to the size of 1024*1024 (i.e., the above second size), and are respectively denoted as I1 (x: 0 - 1024, y: 0 - 1024), I2 (x: 1024 - 2048, y: 0 - 1024), I5 (x: 0 - 1024, y: 1024 - 2048), I6 (x: 1024 - 2048, y: 1024 - 2048), where I1, I2, I5, and I6 correspond to the above four second colored images.

[0157] Taking I1 among I1, I2, I5, and I6 as an example, I1 is enlarged to 2048*2048 to obtain I1 with a size of 2048*2048 (the second colored image corresponding to the above-mentioned first size, that is, the third colored image), and I1 with a size of 2048*2048 (i.e., the third colored image) is used as the target style image. The area corresponding to the image content of image F and I1 (x: 0 - 4096, y: 0 - 4096) is intercepted, and the intercepted area is used as the target image area. It is judged whether the image size of the target image area is the first size. If not, the target image area is reduced to the first size. If so, the operation of reducing the size of the target image area is not performed on the target image area. It can be known that at this time, the size of the target image area does not meet the first size, that is, the image size does not meet 2048*2048, so the target image area is reduced to the first size of 2048*2048. The target image area with the first size (i.e., 2048*2048) is used as the content image for style transfer, and I1 with a size of 2048*2048 (i.e., the third colored image) is used as the target style image for style transfer to obtain a colored image J1 with a size of 2048*2048 (i.e., the above-mentioned fourth colored image). I2, I5, and I6 are processed in the same way to obtain the corresponding colored images J2, J5, and J6. This is not elaborated in this embodiment of the present disclosure.

[0158] Finally, four fourth colored images J1, J2, J5, and J6 corresponding to the first colored image H1 can be obtained, and four fourth colored images J3, J4, J7, and J8 corresponding to the second colored image H2; four fourth colored images J9, J10, J13, and J14 corresponding to the second colored image H3; and four fourth colored images J11, J12, J15, and J16 corresponding to the second colored image H4. Each of the fourth colored images J1, J2, …… J16 is used as a new first colored image.

[0159] It is judged whether the preset end condition is met. If not, step 4 is continued to be executed at this time.

[0160] In step 4, in the same way as in steps 2 and 3 above, each of the first colored images J1, J2, …… J16 is processed to obtain the corresponding fourth colored images L1, L2, …… L64. The following takes the colored image J1 as an example for illustration, and the specific process is as follows:

[0161] Divide the colored image J1 (corresponding to the above-mentioned first colored image) into four parts according to the size of 1024*1024 (i.e., the above-mentioned second size), and denote them as K1 (x: 0-1024, y: 0-1024), K2 (x: 1024-2048, y: 0-1024), K9 (x: 0-1024, y: 1024-2048), K10 (x: 1024-2048, y: 1024-2048), where K1, K2, K9, and K10 correspond to the above-mentioned four second colored images.

[0162] Taking K1 in K1, K2, K9, and K10 as an example, enlarge K1 to 2048*2048 to obtain K1 with a size of 2048*2048 (corresponding to the second colored image of the above-mentioned first size, that is, the third colored image). Take K1 with a size of 2048*2048 (i.e., the third colored image) as the target style image. Cut out the corresponding area (x: 0-2048, y: 0-2048) of the image content of image F and K1, and take the cut-out area as the target image area. Determine whether the image size of the target image area is the first size. If not, reduce the target image area to the first size. If so, do not perform the operation of reducing the size of the target image area. It can be known that at this time, the size of the target image area meets the first size, so there is no need to perform the operation of reducing the size of the target image area. Take the target image area of the first size (i.e., 2048*2048) as the content image for style transfer, and take K1 with a size of 2048*2048 (i.e., the third colored image) as the target style image for style transfer to obtain a colored image L1 with a size of 2048*2048 (i.e., the above-mentioned fourth colored image). Process K2, K9, and K10 in the same way to obtain the corresponding colored images L2, L9, and L10. The embodiments of the present disclosure will not be elaborated here.

[0163] Finally, four fourth colored images L1, L2, L9, and L10 corresponding to the first colored image J1 can be obtained, and four fourth colored images L3, L4, L11, and L12 corresponding to the second colored image J2;...; four fourth colored images L55, L56, L63, and L64 corresponding to the second colored image J16. Take each of the fourth colored images L1, L2,... L64 as the new first colored images.

[0164] Determine whether the preset end condition is satisfied. If satisfied, continue to execute step 5.

[0165] Step 5, piece together the new first colored images L1, L2... L64 to generate a colored image L with a size of (16384*16384). This colored image L is the above-mentioned target image, where, as Figure 8 shown, is a schematic diagram of the target image obtained through the embodiments of the present disclosure.

[0166] It should be noted that image G is a low - resolution colored image, and image F is a high - resolution grayscale image. In step 2, first, one - quarter of image G is enlarged to obtain a blurred 2048 - colored image. Then, the corresponding one - quarter area of image F is reduced in size to 2048, losing 4 - fold precision. The two have the same size, and style transfer is performed. After this operation, a 4096 * 4096 colored image H (H1, H1... H4) is obtained, which equivalently increases the resolution of image G by 2 times. Similarly, in step 3, the corresponding one - eighth area of image F is reduced in size to 2048, losing 2 - fold precision, to obtain an 8192 * 8192 colored image J (J1, J2... J16). In step 4, the corresponding one - sixteenth area of image F is intercepted. At this time, the corresponding one - sixteenth of image F is the area of 2048 * 2048, without losing precision, and style transfer is performed to obtain a 16384 * 16384 colored image L (L1, L2... L64).

[0167] Through the embodiments of the present disclosure, by using a low - resolution colored image for style transfer with a high - resolution grayscale image, the precision can be gradually improved, and the processing precision of style transfer is enhanced.

[0168] The embodiments of the present disclosure provide an image processing apparatus. As Figure 9 shown, the image processing apparatus 10 may include: an acquisition module 101, a determination module 102, a first processing module 103, and a second processing module 104, where:

[0169] The acquisition module 101 is configured to acquire an image to be processed, where the image to be processed is an image that needs to be color - adjusted;

[0170] The determination module 102 is configured to divide the image to be processed into blocks according to N rows and M columns to obtain each block image, and determine the color deviation value of each block image to obtain each color deviation value, where N and M are positive integers;

[0171] The first processing module 103 is configured to perform color adjustment on the image to be processed by means of histogram matching based on the respective color deviation values corresponding to each block image, and perform an elimination operation on the splicing edge of the image to be processed after color adjustment to obtain a grayscale image;

[0172] The second processing module 104 is configured to adjust the size of the grayscale image, input the grayscale image after size adjustment into a style - transfer neural network, and perform style transfer to obtain a target image.

[0173] In an optional embodiment, the second processing module 104 is specifically configured to:

[0174] Reduce the above grayscale image to the first size to obtain a grayscale image of the first size, where the image size of the above grayscale image is an image size not supported by the image processing device, the above first size is an image size supported by the above image processing device, and the above image processing device is a device for processing the above to-be-processed image;

[0175] Obtain a target style image, input the above target style image and the grayscale image of the first size into a style transfer neural network, and perform style transfer on the grayscale image of the first size based on the above target style image to obtain a first colored image, where the image sizes of the above target style image and the grayscale image of the first size are the same;

[0176] Repeat the following operations until a preset end condition is met:

[0177] Divide the above first colored image into four parts according to the second size to obtain four second colored images, where the above first size is twice the above second size;

[0178] For any one of the second colored images, enlarge the second colored image to the first size to obtain a second colored image of the first size, and denote the second colored image of the first size as a third colored image;

[0179] Intercept the image area corresponding to the position of the area where the second colored image is located in the above grayscale image as the target image area, and determine whether the image size of the above target image area is the first size. If not, reduce the above target image area to the first size. If so, do not perform the operation of reducing the size of the above target image area;

[0180] Use the above third colored image as the target style image, use the target image area of the first size as the content image, and perform style transfer on the above target image area to obtain a fourth colored image;

[0181] Until the fourth colored images corresponding to the respective second colored images are obtained, and use the respective fourth colored images as the new respective first colored images;

[0182] Determine whether the preset end condition is met. If not, perform the above operations on each new first colored image. If so, splice the new respective first colored images to obtain the above target image, where the above preset end condition is not to perform the operation of reducing the size of the above target image area and obtaining the new respective first colored images.

[0183] In an optional embodiment, the above first processing module 103 is specifically configured to:

[0184] Perform a clustering operation on the respective color deviation values corresponding to the above divided images to obtain a maximum clustering set;

[0185] Determine the mean of the color deviation values in the above maximum clustering set to obtain the average color deviation value;

[0186] Determine the color deviation value with the smallest difference from the absolute value of the difference between the above average color deviation value among the above color deviation values to obtain the target color deviation value;

[0187] Determine the target block image corresponding to the above target color deviation value from each of the above block images;

[0188] Statistically analyze the histogram of the above target block image, and based on the histogram of the above target block image, perform histogram matching on each of the above block images other than the above target block image to obtain each of the other block images after histogram matching;

[0189] Stitch the above target block image and each of the above other block images after histogram matching to obtain a fused image, where the above fused image is an image obtained by performing color adjustment on the above image to be processed.

[0190] In an alternative embodiment, the above first processing module 103 is specifically configured to:

[0191] Create a transit image with all pixels being 1, where the image size of the above transit image is the same as that of the above fused image;

[0192] For any two images to be processed, determine the brightness ratio of the two images to be processed, and assign values to the above transit image based on the brightness ratio to obtain a weight image; where any two of the above images to be processed are any two images among the above target block image and each of the above other block images after histogram matching;

[0193] Multiply the above fused image by the above weight image to obtain a multiplied image;

[0194] Convert the above multiplied image to grayscale to obtain the above grayscale image.

[0195] In an alternative embodiment, the above image to be processed is a satellite image to be processed, and the image size of the satellite image to be processed is an image size not supported by the image processing device, where the above image processing device is a device for processing the satellite image to be processed.

[0196] Through the embodiments of the present disclosure, through the automated satellite image color adjustment method, not only can the stitching traces be eliminated, but also the accuracy of terrain information can be maintained, meeting the requirements of flight simulation training. Through the automated satellite image processing method, not only can the existing image quality problems be solved, but also the processing efficiency can be greatly improved, providing a more realistic and reliable terrain environment for flight simulation training.

[0197] The device according to an embodiment of the present disclosure can execute the method provided by the embodiment of the present disclosure. Their implementation principles are similar and have corresponding technical effects. The actions performed by each module in the device of each embodiment of the present disclosure correspond to the steps in the method of each embodiment of the present disclosure. For a detailed description of the functions of each module of the device, reference can be specifically made to the description in the corresponding method shown above, and details will not be repeated here.

[0198] An electronic device (computer device / equipment / system) is provided in an embodiment of the present disclosure, including a memory, a processor, and a computer program stored on the memory. The processor executes the above computer program to implement the steps of the method provided by any optional embodiment of the present disclosure.

[0199] In an optional embodiment, an electronic device is provided, as Figure 10 shown Figure 10 The electronic device 1000 shown includes: a processor 1001 and a memory 1003. Among them, the processor 1001 and the memory 1003 are connected, such as connected through a bus 1002. Optionally, the electronic device 1000 may further include a transceiver 1004, and the transceiver 1004 can be used for data interaction between this electronic device and other electronic devices, such as data sending and / or data receiving, etc. It should be noted that in practical applications, the transceiver 1004 is not limited to one, and the structure of the electronic device 1000 does not constitute a limitation to the embodiment of the present disclosure.

[0200] The processor 1001 can be a CPU (Central Processing Unit, central processor), a general-purpose processor, a DSP (Digital Signal Processor, data signal processor), an ASIC (Application Specific Integrated Circuit, application-specific integrated circuit), an FPGA (Field Programmable Gate Array, field programmable gate array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logical blocks, modules, and circuits described in combination with the disclosure content of the present disclosure. The processor 1001 can also be a combination that realizes computing functions, such as a combination including one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0201] The bus 1002 may include a path for transmitting information among the above components. The bus 1002 can be a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, or the like. The bus 1002 can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity of representation, Figure 10 it is only represented by a thick line in Figure 10 , but it does not mean that there is only one bus or one type of bus.

[0202] The memory 1003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or it can also be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, other magnetic storage devices, or any other medium that can be used to carry or store computer programs and can be read by a computer, which is not limited herein.

[0203] The memory 1003 is used to store the computer program for implementing the embodiments of the present disclosure and is controlled by the processor 1001 to execute. The processor 1001 is used to execute the computer program stored in the memory 1003 to implement the steps shown in the foregoing method embodiments.

[0204] The embodiments of the present disclosure provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps and corresponding contents of the foregoing method embodiments can be implemented.

[0205] The embodiments of the present disclosure further provide a computer program product, including a computer program, and when the computer program is executed by a processor, the steps and corresponding contents of the foregoing method embodiments can be implemented.

[0206] It should be understood that although the flowcharts of the embodiments of the present disclosure indicate each operation step by arrows, the execution order of these steps is not limited to the order indicated by the arrows. Unless otherwise clearly stated herein, in some implementation scenarios of the embodiments of the present disclosure, the implementation steps in each flowchart may be executed in other orders according to requirements. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on the actual implementation scenario. Some or all of these sub-steps or stages may be executed at the same time, and each sub-step or stage among these sub-steps or stages may also be executed at different times respectively. In the scenario where the execution times are different, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and the embodiments of the present disclosure do not limit this.

[0207] The above are only optional implementation manners of some implementation scenarios of the present disclosure. It should be noted that for those of ordinary skill in the art in the technical field, without departing from the technical concept of the solution of the present disclosure, adopting other similar implementation means based on the technical idea of the present disclosure also belongs to the protection scope of the embodiments of the present disclosure.

Claims

1. An image processing method, characterized in that, The method includes: Obtain an image to be processed, where the image to be processed is an image that needs color adjustment; Divide the image to be processed into blocks according to N rows and M columns to obtain respective block images, and determine the color deviation values of the respective block images to obtain respective color deviation values, where N and M are positive integers; Based on the respective color deviation values corresponding to the respective block images, perform color adjustment on the image to be processed by means of histogram matching, and perform an elimination operation on the splicing edges of the image to be processed after color adjustment to obtain a grayscale image; Perform the following operations to adjust the size of the grayscale image, input the grayscale image after size adjustment into a style transfer neural network, and perform style transfer to obtain a target image: Shrink the grayscale image according to a first size to obtain a grayscale image of the first size; Obtain a target style image, input the target style image and the grayscale image of the first size into a style transfer neural network, and perform style transfer on the grayscale image of the first size based on the target style image to obtain a first colored image, where the image size of the target style image is the same as that of the grayscale image of the first size; Repeat the following operations until a preset end condition is met: Divide the first colored image into four parts according to a second size to obtain four second colored images, where the first size is twice the second size; For any one of the second colored images, enlarge the second colored image according to the first size to obtain a second colored image of the first size, and denote the second colored image of the first size as a third colored image; Intercept an image area corresponding to the area where the second colored image is located in the grayscale image as a target image area, and determine whether the image size of the target image area is the first size. If not, shrink the target image area to the first size. If so, do not perform the operation of shrinking the size of the target image area; Use the third colored image as the target style image, use the target image area of the first size as the content image, and perform style transfer on the target image area to obtain a fourth colored image; Until fourth colored images corresponding to the respective second colored images are obtained, and use the respective fourth colored images as new respective first colored images; Determine whether the preset end condition is met. If not, perform the above operations on each new first colored image. If so, splice the new respective first colored images to obtain the target image, where the preset end condition is not to perform the operation of shrinking the size of the target image area and new respective first colored images are obtained.

2. The method according to claim 1, wherein The performing color adjustment on the image to be processed by means of histogram matching based on the respective color deviation values corresponding to the respective block images includes: Perform a clustering operation on the respective color deviation values corresponding to the respective block images to obtain a maximum clustering set; Determine the mean value of the respective color deviation values in the maximum clustering set to obtain an average color deviation value; Determine the color deviation value with the smallest absolute difference from the average color deviation value among the respective color deviation values to obtain a target color deviation value; Determine the target sub-block image corresponding to the target color deviation value from each of the sub-block images; Statistically analyze the histogram of the target sub-block image, and based on the histogram of the target sub-block image, perform histogram matching on each of the other sub-block images in the sub-block images except the target sub-block image to obtain the other sub-block images after histogram matching; Stitch the target sub-block image and the other sub-block images after histogram matching to obtain a fused image, where the fused image is an image obtained by performing color adjustment on the image to be processed.

3. The method according to claim 2, wherein The eliminating operation on the stitching edge of the image after color adjustment to obtain a grayscale image includes: Create an intermediate image with all pixels being 1, where the intermediate image has the same image size as the fused image; For any two sub-block images to be processed, determine the brightness ratio of the two sub-block images to be processed, and assign values to the intermediate image based on the brightness ratio to obtain a weight image; where any two of the sub-block images to be processed are any two images among the target sub-block image and the other sub-block images after histogram matching; Multiply the fused image and the weight image to obtain a multiplied image; Convert the multiplied image to grayscale to obtain the grayscale image.

4. The method according to claim 1, wherein The image to be processed is a satellite image to be processed, and the image size of the satellite image to be processed is an image size not supported by the image processing device, where the image processing device is a device for processing the satellite image to be processed.

5. An image processing apparatus, characterized in that, The apparatus includes: An acquisition module for acquiring an image to be processed, where the image to be processed is an image that needs to be color-adjusted; A determination module for dividing the image to be processed into sub-blocks according to N rows and M columns to obtain each sub-block image, and determining the color deviation value of each sub-block image to obtain each color deviation value, where N and M are positive integers; A first processing module for performing color adjustment on the image to be processed by histogram matching based on the color deviation value corresponding to each sub-block image, and performing an eliminating operation on the stitching edge of the image after color adjustment to obtain a grayscale image; A second processing module for performing the following operations to adjust the size of the grayscale image, and inputting the grayscale image with adjusted size into a style transfer neural network for style transfer to obtain a target image: Shrink the grayscale image according to a first size to obtain a grayscale image of the first size; Obtain a target style image, input the target style image and the grayscale image of the first size into the style transfer neural network, and perform style transfer on the grayscale image of the first size based on the target style image to obtain a first colored image, where the image size of the target style image is the same as that of the grayscale image of the first size; Repeat the following operations until a preset end condition is met: Divide the first colored image into four parts according to a second size to obtain four second colored images, where the first size is twice the second size; For any second-colored image, enlarge the second-colored image according to the first size to obtain a second-colored image of the first size, and denote the second-colored image of the first size as the third-colored image; Intercept the image area corresponding to the area where the second-colored image is located in the grayscale image as the target image area, and determine whether the image size of the target image area is the first size. If not, reduce the target image area to the first size. If so, do not perform the operation of reducing the size of the target image area; Use the third-colored image as the target style image, use the target image area of the first size as the content image, and perform style transfer on the target image area to obtain a fourth-colored image; Until the fourth-colored images corresponding to each second-colored image are obtained, and use each fourth-colored image as a new first-colored image; Determine whether a preset end condition is satisfied. If not, perform the above operations on each new first-colored image. If satisfied, splice the new first-colored images to obtain the target image, where the preset end condition is not to perform the operation of reducing the size of the target image area and obtaining the new first-colored images.

6. An electronic device, comprising a memory, a processor, and a computer program stored on the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1-4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1-4.

Images