Image fusion preprocessing method, device, equipment, and storage medium

Abstract

The present application provides a method, apparatus, device, and storage medium for image fusion preprocessing, wherein the image includes a foreground image and a background image to be fused, and the method includes processing the foreground image and the background image to obtain a high-frequency image and a low-frequency image of the foreground image, as well as a low-frequency image of the background image, replacing the low-frequency information of the non-specified area of the low-frequency image of the foreground image with the low-frequency information of the non-specified area corresponding to the low-frequency image of the background image to obtain a replaced low-frequency image, and adding the obtained replaced low-frequency image to the high-frequency image of the foreground image to obtain a preprocessed foreground image. Before the foreground image and the background image are fused, the present application replaces the low-frequency information of the foreground image with the low-frequency information of the background image to avoid an obvious fusion boundary between the foreground image and the background image, thereby achieving a better fusion effect of the foreground image and the background image.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular to a method for image fusion preprocessing, an apparatus for image fusion preprocessing, a device, and a computer-readable storage medium. Background Art

[0002] When performing image fusion, it may be necessary to superimpose a specified area of one image onto a specified area of another image. However, a problem often encountered at this time is that the background colors of the two images are quite different, resulting in an overly obvious boundary between the image fusion and a poor fusion effect. Summary of the Invention

[0003] In view of this, the present application provides a method for image fusion preprocessing, an apparatus for image fusion preprocessing, a device, and a computer-readable storage medium.

[0004] According to a first aspect of an embodiment of the present application, a method for image fusion preprocessing is provided, wherein the image includes a foreground image and a background image to be fused, and the method includes:

[0005] Processing the foreground image and the background image to obtain a high-frequency image and a low-frequency image of the foreground image, and a low-frequency image of the background image;

[0006] Replacing the low-frequency information of the non-designated area of the low-frequency image of the foreground image with the low-frequency information of the non-designated area corresponding to the low-frequency image of the background image to obtain a replaced low-frequency image;

[0007] The obtained replaced low-frequency image is added to the high-frequency image of the foreground image to obtain a preprocessed foreground image.

[0008] According to a second aspect of an embodiment of the present application, a device for image fusion preprocessing is provided, wherein the image includes a foreground image and a background image to be fused, and the device includes:

[0009] an image processing module, configured to process the foreground image and the background image to obtain a high-frequency image and a low-frequency image of the foreground image, and a low-frequency image of the background image;

[0010] an image replacement module, replacing low-frequency information of a non-designated area of the low-frequency image of the foreground image with low-frequency information of a non-designated area corresponding to the low-frequency image of the background image, to obtain a replaced low-frequency image;

[0011] The image operation module is used to add the obtained replaced low-frequency image to the high-frequency image of the foreground image to obtain a preprocessed foreground image.

[0012] According to a third aspect of an embodiment of the present application, there is provided a device, including:

[0013] processor;

[0014] a memory for storing instructions executable by the processor;

[0015] The processor is configured to execute the instructions to implement the method described in any one of the above embodiments.

[0016] According to a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described in any of the above embodiments is implemented.

[0017] The present application processes the foreground image and background image to be fused to obtain a high-frequency image and a low-frequency image of the foreground image, as well as a low-frequency image of the background image, and replaces the low-frequency information of the non-specified area of the low-frequency image of the foreground image with the low-frequency information of the non-specified area corresponding to the low-frequency image of the background image to obtain a replaced low-frequency image, and adds the obtained replaced low-frequency image to the high-frequency image of the foreground image to obtain a preprocessed foreground image. Before fusing the foreground image to be fused with the background image to be fused, the present application preprocesses the foreground image and the background image separately, replaces the low-frequency information of the foreground image with the low-frequency information of the background image, thereby avoiding the problem of obvious fusion boundary between the foreground image and the background image, and achieving a better fusion effect of the foreground image and the background image. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flowchart of an image fusion preprocessing method shown in an exemplary embodiment of the present application.

[0019] Figure 2 This is a flow chart of a method for processing a foreground image based on a Gaussian pyramid, shown as an exemplary embodiment of the present application.

[0020] Figure 3a FIG. 1 is a schematic diagram of a facial image to be subjected to Poisson fusion, shown in an exemplary embodiment of the present application.

[0021] Figure 3b FIG. 1 is a schematic diagram of a facial image to be subjected to Poisson fusion, shown in an exemplary embodiment of the present application.

[0022] Figure 3c This is an exemplary embodiment of the present application. Figure 3a Schematic diagram of the result of marking specific areas of a face image.

[0023] Figure 3d FIG. 1 is a schematic diagram of a mask image shown in an exemplary embodiment of the present application.

[0024] Figure 4 It is a structural diagram of an image fusion preprocessing device shown in an exemplary embodiment of the present application.

[0025] Figure 5 The figure shows a hardware structure diagram of a device where an image fusion preprocessing apparatus is located, as shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0026] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0027] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0028] It should be understood that although the terms first, second, third, etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0029] When performing image fusion, a specific area of one image is typically superimposed on a specific area of another. However, a common problem encountered during image fusion is that the background colors of the two images differ significantly, resulting in a very distinct boundary between the fused images and poor fusion results. This is particularly evident when performing face fusion. When a certain area of one face image is superimposed on the corresponding area of another face image, if the skin color of the two faces differs significantly, the boundary between the two faces will be very obvious, resulting in an unnatural appearance of the fused face.

[0030] In response to the above problems, the present application proposes a method for pre-processing the image to be fused before image fusion based on the high and low frequency information of the image, so as to avoid the problem of too obvious image fusion boundaries. Image fusion usually requires three material images, one foreground image, one background image and one mask image, where the mask image is used to specify which part of the foreground image is to be fused with the background image. The present application takes the example of pre-processing the foreground image and the background image separately before superimposing a part of the foreground image onto the background image. It can be understood that the number of foreground images and background images is not limited to one. When the foreground image and the background image include multiple images, the method of the present application can still be applied. Figure 1 This is a flow chart of a method for image fusion preprocessing shown in an exemplary embodiment of the present application, wherein the image includes a foreground image and a background image to be fused, such as Figure 1 As shown, the method includes the following steps:

[0031] S101, processing the foreground image and the background image to obtain a high-frequency image and a low-frequency image of the foreground image, and a low-frequency image of the background image;

[0032] S102, replacing low-frequency information of a non-designated area of the low-frequency image of the foreground image with low-frequency information of a non-designated area corresponding to the low-frequency image of the background image, to obtain a replaced low-frequency image;

[0033] S103 : Add the obtained replaced low-frequency image to the high-frequency image of the foreground image to obtain a preprocessed foreground image.

[0034] The high-frequency image mentioned in this application can be understood as an image that only includes high-frequency information. The high-frequency image refers to the pixels in the area where the image intensity (brightness / grayscale) changes dramatically in the image to be processed. The low-frequency image can be understood as an image that approximately only includes low-frequency information. The low-frequency information refers to the pixels in the area where the image intensity (brightness / grayscale) changes smoothly in the image to be processed. Taking the human face as an example, the skin color part in the face image can be regarded as low-frequency information, and the boundary between the facial features and the skin color can be regarded as high-frequency information. This application processes the foreground image and the background image to obtain a high-frequency image and a low-frequency image of the foreground image, as well as a high-frequency image and a low-frequency image of the background image.

[0035] In one embodiment, the processing of the foreground image and the background image can be performed based on the Gaussian pyramid. For example, the processing of the foreground image by the Gaussian pyramid is performed. Figure 2 As shown, Figure 2 FIG. 1 is a flow chart of a method for processing a foreground image based on a Gaussian pyramid according to an exemplary embodiment of the present application. The method includes the following steps:

[0036] S201, obtaining a foreground image;

[0037] S202, setting the number of repeated processing times;

[0038] S203, performing low-pass filtering on the foreground image;

[0039] S204, downsampling the foreground image;

[0040] S205, determine whether the preset number of repetitions has been reached, if so, execute S206; otherwise, return to execute S203;

[0041] S206: Output a Gaussian pyramid including images of each layer.

[0042] In this embodiment, the number of repetitions can be set by those skilled in the art based on actual needs. The low-pass filtering and downsampling of the foreground image are performed to remove high-frequency information in the foreground image as much as possible, thereby obtaining a low-frequency image of the foreground image. The low-frequency image constitutes the highest layer image of the Gaussian pyramid. The low-pass filtering is mainly implemented by a low-pass filter. For example, a Gaussian filter can be used for low-pass filtering. For the Gaussian filter used, its Gaussian filter kernel can be linearly related to the alpha coefficient of the scale of the foreground image to be processed. Thus, for images of different scales, an appropriate filter kernel can be used for processing, thereby improving processing efficiency. During the downsampling process, the low-pass filtered image can be subjected to pixel selection in designated rows and columns. For example, only pixels in even rows and even columns can be sampled to form a new reduced image, which is used as the image of a new layer of the Gaussian pyramid. The specific method of downsampling the image can be determined by those skilled in the art based on actual needs and is not limited by this application.

[0043] For the images in each layer of the Gaussian pyramid, since each layer is obtained by low-pass filtering and downsampling the image in the previous layer to form the Gaussian pyramid, each layer of the image lacks some high-frequency information relative to the image in the previous layer. This missing high-frequency information can be recorded so that after processing the low-frequency image of the highest layer of the Gaussian pyramid in S103, the processed low-frequency image can be restored through the Gaussian pyramid to obtain the pre-processed foreground image. Therefore, for each layer of the image in the Gaussian pyramid, it can be upsampled so that the pixel size of the image of this layer is consistent with the adjacent previous layer, and a subtraction operation is performed with the adjacent previous layer image to obtain a differential image between the two layers of images. The differential image is the high-frequency image of the foreground image. When restoring the foreground image, the low-frequency image of the highest layer of the processed Gaussian pyramid can be upsampled to obtain an image with the same pixel size as the previous layer image, and added to the differential image between the two layers of images. The added image is further upsampled and added to the next differential image, and so on. After repeating several times, a preprocessed foreground image is obtained. The preprocessed foreground image contains the high-frequency information and low-frequency information of the foreground image that you want to retain, as well as part of the low-frequency information of the acquired background image.

[0044] It should be understood that processing a foreground image using a Gaussian pyramid to obtain a high-frequency image and a low-frequency image of the foreground image is merely an example presented in this application. This application is not limited to obtaining high-frequency and low-frequency images through Gaussian pyramid processing; all methods of processing an image to obtain low-frequency and high-frequency images are applicable to this application. In some feasible examples, the foreground image can also be processed using a Laplacian pyramid, the principles of which are similar to those of the Gaussian pyramid and will not be described in detail here. Alternatively, if a low-pass filter with a sufficient filter kernel size can be used to obtain a low-frequency image in a single pass, the foreground image can be processed using this low-pass filter, eliminating the need for multiple repetitive steps in the image pyramid.

[0045] In one embodiment, some specific areas in the foreground image may not need to be processed. Therefore, in S102, before processing the foreground image, these specific areas can be marked first, and for the low-frequency image obtained by processing the foreground image, the low-frequency information of areas outside of these designated areas can be replaced. As a specific example, before the step of replacing the low-frequency information of the non-designated areas of the low-frequency image of the foreground image with the low-frequency information of the non-designated areas corresponding to the low-frequency image of the background image, the step may include: constructing a first mask image based on the designated areas, and extracting the low-frequency information of the non-designated areas corresponding to the low-frequency image of the background image through the first mask image. After the low-frequency information of the non-designated areas corresponding to the low-frequency image of the background image is extracted from the low-frequency image of the background image by the first mask image, a replacement image including the low-frequency information of the non-designated areas corresponding to the low-frequency image of the background image can be obtained, and the replacement of the low-frequency information is achieved by superimposing the replacement image on the low-frequency image of the foreground image.

[0046] Among them, the first mask image is mainly used to indicate which part of the low-frequency image of the background image is used to replace the low-frequency information of the non-specified area in the low-frequency image of the foreground image. The first mask image is constructed according to the specified area. A grayscale image with the same size as the foreground image can be established, and the pixels of the area corresponding to the specified area in the foreground image are set to 0, and the pixels of the area corresponding to the non-specified area are set to 255 to obtain the first mask image. The area with pixels of 255 in the first mask image can be regarded as the area of interest, and the area with pixels of 0 can be regarded as the area not to be processed. Through the area of interest of the first mask image, only the low-frequency information of the non-specified area corresponding to the low-frequency image of the background image is copied to obtain a replacement image for replacing the low-frequency information of the non-specified area in the low-frequency image of the foreground image, and the replacement image is superimposed on the low-frequency image of the foreground image to replace the low-frequency information of the non-specified area in the low-frequency image of the foreground image with the low-frequency information of the non-specified area corresponding to the low-frequency image of the background image.

[0047] In one embodiment, before extracting low-frequency information from the non-specified area corresponding to the low-frequency image of the background image, the first mask image constructed based on the specified area can be subjected to erosion and Gaussian blur processing. The erosion and Gaussian blur processing can smooth the boundary between the 0-pixel area and the 255-pixel area in the first mask image, avoiding jagged edges. Furthermore, the pixel transition from the 0-pixel area to the 255-pixel area is smoothed, and the boundary is processed into a natural pixel area transitioning from 0 to 255 pixels, such as the transition area from 0, 5, 20, 60, ..., 255. Thus, when the first mask image is used to extract low-frequency information from the low-frequency image of the background image, the resulting replacement image will not have jagged edges, and the pixel transition will be more natural. When the replacement image is superimposed on the foreground image, the transition between the boundary of the unprocessed specified area in the original foreground image will be more natural.

[0048] In one embodiment, for the low-frequency image obtained by processing the foreground image, a specific area may not be set, and the entire low-frequency image may be processed. In this case, there is no need to consider retaining the pixels in the specific area. Instead, the low-frequency information on the low-frequency image of the entire foreground image can be replaced with the low-frequency information of the low-frequency image of the background image. Therefore, the non-designated area can be the entire low-frequency image. In this case, there is no need to construct a mask to protect the pixels in the specific area of the low-frequency image of the foreground image. The low-frequency image of the foreground image can be directly replaced with the low-frequency image of the background image, and then the low-frequency image of the background image and the high-frequency image of the foreground image are added together to restore the pre-processed foreground image.

[0049] The present application replaces the low-frequency information of the foreground image so that the low-frequency information on the foreground image is replaced by the low-frequency information on the background image. In addition, the present application also processes the mask image used in image fusion. In one embodiment, after determining the area to be fused in the preprocessed foreground image, a second mask image required for image fusion is constructed based on the area to be fused. The size of the second mask image can be consistent with the preprocessed foreground image. The pixels of the area corresponding to the area to be fused on the preprocessed foreground image on the second mask image are set to 255, and the pixels of other areas are set to 0. For the constructed second mask image, it is also necessary to perform corrosion processing and Gaussian blur processing on it so that the boundary between the area with pixels of 255 and the area with pixels of 0 has a natural transition. At the same time, the image pixels of the area to be fused in the preprocessed foreground image are extracted through the processed second mask image to obtain the image to be fused. The image to be fused is used to be superimposed on the specified area on the background image to complete image fusion.

[0050] After extracting the area to be fused from the second mask image to obtain the image to be fused, the background image to be fused can be processed. Alpha blending is performed on the background image based on the pixel values of the region boundary corresponding to the area to be fused on the constructed second mask image. For example, the value of a pixel in the background image can be calculated using the following formula:

[0051]

[0052] In the above formula, src value is the value of the pixel on the processed foreground image, mask value For the pixel value on the constructed second mask image, the dst on the right side of the equation value is the value of the pixel on the background image to be processed, and the dst on the left side of the equation value is the value of the pixel point on the processed background image. Through this formula, any pixel point on the background image to be processed, the pixel point at the corresponding position on the foreground image, and the pixel point at the corresponding position on the second mask image can be selected. According to the values of these three pixels, the pixel value of the processed background image at that position is calculated. Through alpha blending processing, the fusion boundary pixel value of the area where the image to be fused is superimposed on the background image can be made close to the boundary pixel value of the image to be fused, avoiding the problem of too obvious fusion boundary when the image to be fused is superimposed on the background image.

[0053] One practical application scenario of the method of the present application can be its application in Poisson fusion. When using Poisson fusion for face fusion, if the skin color of the faces to be fused is quite different, the fusion boundary will be more obvious and the fusion effect will be poor. The method of the present application can be used to pre-process the face images to be Poisson fused so that the effect of Poisson fusion is better. The method of the present application will be illustrated below by taking the pre-processing of the face images to be Poisson fused as an example.

[0054] Figure 3a 、 Figure 3b is a face image to be Poisson fused as shown in an exemplary embodiment of the present application. Figure 3a 、 Figure 3b As shown, when performing Poisson fusion, it is necessary to superimpose the face area A in face image 3a onto the corresponding face area B in face image 3b. Before performing Poisson fusion, face image 3a and face image 3b are preprocessed separately:

[0055] First, it is necessary to mark specific areas in the facial image 3a, such as the areas of the eyes, eyebrows, mouth and other facial features. Specifically, these areas can be identified and key points can be marked through face recognition, and the area composed of key points can be regarded as a specific area. For the specific method of how to mark key points, you can refer to relevant technical literature, which will not be described in detail in this application. Figure 3c For Figure 3a The result diagram after marking a specific area is as follows: Figure 3c As shown, it includes the marked eyebrow area 1, eye area 2, nose area 3, mouth area 4 and the unmarked non-specific area C.

[0056] After marking the specific area, the face image 3b and the face image 3c can be processed separately to obtain the high-frequency image and low-frequency image of the face image 3c and the low-frequency image of the face image 3b. The specific steps of the processing can be referred to Figure 2 In the process of processing the foreground image based on the Gaussian pyramid, other methods that can obtain high-frequency images and low-frequency images of the image can also be used, such as a method of processing the image based on the Laplacian pyramid.

[0057] For the obtained low-frequency image of facial image 3c, the low-frequency information of the non-specific region C of the low-frequency image needs to be replaced with the low-frequency information of the corresponding non-specific region C in the low-frequency image of facial image 3b. At this time, it is necessary to construct a mask image to protect the low-frequency information of specific regions in the low-frequency image of facial image 3c from being replaced. In other words, it is necessary to protect the low-frequency information of eyebrow region 1, eye region 2, nose region 3, and mouth region 4 in facial image 3c from being replaced. Specifically, a mask image can be constructed based on eyebrow region 1, eye region 2, nose region 3, and mouth region 4. The constructed mask image can also be subjected to erosion processing and Gaussian blur processing. Figure 3d A schematic diagram of a mask image constructed, such as Figure 3d As shown, the size of the mask image is consistent with that of the facial image 3c. The mask image only includes 0 pixels and 255 pixels, wherein the pixels of the areas corresponding to the eyebrow area 1, the eye area 2, the nose area 3, and the mouth area 4 in the facial image 3c are set to 0 pixels, and the pixels of the area corresponding to the non-specific area C in the facial image 3c are set to 255. Through the mask image, the low-frequency information of the area corresponding to area C in the low-frequency image of the facial image 3b can be extracted, and the extracted low-frequency information is superimposed on the low-frequency image of the facial image 3c, so that the low-frequency information of area C on the low-frequency image of the facial image 3c is replaced by the low-frequency information of the corresponding area on the facial image 3b.

[0058] After completing the replacement of the low-frequency information, the low-frequency image of the replaced facial image 3c and the high-frequency image obtained when processing the image 3a are added together to restore the facial image 3a, completing the preprocessing of the facial image 3a. At this time, it is also necessary to process the mask image required for Poisson fusion. For the mask image constructed based on area A in the facial image 3a, the mask image can be corroded and Gaussian blurred to make the edge transition of the image to be fused obtained by extracting the image of area A in the facial image 3a more natural. Before fusing the obtained image to be fused with the facial image 3b, the facial image 3b can also be alphablending processed. The pixels on the facial image 3b can be calculated and optimized in combination with the formula 1 mentioned in the previous embodiment, so that the boundary pixels of the facial image 3b in the fusion transition area are closer to the image to be fused. After completing the processing of the face image 3b, the image to be fused obtained by extracting the image of area A in the face image 3a from the mask image is superimposed on the corresponding area B on the face image 3b, and the fusion boundary is further processed by solving the Poisson equation to complete Poisson fusion.

[0059] The processing of facial images in the above-mentioned process mainly considers replacing the low-frequency information of the facial image to be fused with the low-frequency information of another facial image. In fact, it is to replace the skin color of one facial image to be fused with the skin color of another facial image to be fused, so that the fusion effect of the two facial images will be better when they are fused again. The reason why the low-frequency information of the facial features area is retained when the present application replaces the low-frequency information is mainly because the facial features positions of the two facial images to be fused may not be completely aligned. At this time, if all the low-frequency information including the low-frequency information of the facial features area on the low-frequency image of the image to be fused is replaced, the position of the low-frequency information of the facial features area on the replaced low-frequency image may not be aligned with the position of the high-frequency information of the facial features area on the high-frequency image, resulting in the image obtained after image restoration showing low-frequency information that does not belong to the image. For this reason, it is usually not considered to process the high-frequency image and low-frequency image obtained by processing an image, replace the low-frequency image with other low-frequency images, and then restore the image by adding the replaced low-frequency image to the high-frequency image, because this can easily cause the restored image to introduce other erroneous information. The present application is based on this problem and chooses to retain the low-frequency information of the facial features area without replacing it, which not only achieves the replacement of low-frequency information but also avoids the introduction of erroneous information. In some cases, if the facial features of the two images to be fused are completely aligned, then there is no need to consider the problem of image restoration failure. The low-frequency information of one image to be fused can be directly replaced by the low-frequency information of another image to be fused. Or in some cases, some facial features may be aligned, while some are not. At this time, the low-frequency information of the area where the facial features are aligned can be replaced, while the low-frequency information of the area where the facial features are not aligned can be retained.

[0060] The present application also provides a device for image fusion preprocessing, wherein the image includes a foreground image and a background image to be fused. Figure 4 FIG. 1 is a structural diagram of an image fusion preprocessing device shown in an exemplary embodiment of the present application. Figure 4 As shown, the image fusion preprocessing device 400 includes:

[0061] An image processing module 401 is configured to process the foreground image and the background image to obtain a high-frequency image and a low-frequency image of the foreground image, and a low-frequency image of the background image;

[0062] An image replacement module 402 replaces the low-frequency information of the non-designated area of the low-frequency image of the foreground image with the low-frequency information of the non-designated area corresponding to the low-frequency image of the background image to obtain a replaced low-frequency image;

[0063] The image operation module 403 is configured to add the obtained replaced low-frequency image to the high-frequency image of the foreground image to obtain a pre-processed foreground image.

[0064] The embodiment of the image fusion preprocessing device of the present application can be applied to a device. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of the device in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory and running it. From the hardware level, if Figure 5 The figure shows a hardware structure diagram of the device where the image fusion preprocessing device of this application is located, except Figure 5 In addition to the processor, memory, network interface, and non-volatile memory shown, the device in which the apparatus is located in the embodiment may also include other hardware according to the actual function of the device, which will not be described in detail.

[0065] The non-volatile memory is used to store the processor executable instructions, and the processor is configured to execute the instructions to implement the image fusion preprocessing method described in any one of the above embodiments.

[0066] The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image fusion preprocessing method described in any of the above embodiments.

[0067] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0068] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present application scheme. A person of ordinary skill in the art can understand and implement it without paying any creative work.

[0069] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for image fusion preprocessing, characterized in that: The image includes a foreground image and a background image to be fused, and the method includes: Processing the foreground image and the background image to obtain a high-frequency image and a low-frequency image of the foreground image, and a low-frequency image of the background image; Constructing a first mask image based on the designated area, wherein the first mask image is a grayscale image of the same size as the foreground image, and the area in the first mask image corresponding to the non-designated area in the foreground image is a region of interest; extracting low-frequency information of the non-designated area corresponding to the low-frequency image of the background image through the region of interest; Replacing the low-frequency information of the non-designated area of the low-frequency image of the foreground image with the low-frequency information of the non-designated area corresponding to the low-frequency image of the background image to obtain a replaced low-frequency image; The obtained replaced low-frequency image is added to the high-frequency image of the foreground image to obtain a preprocessed foreground image.

2. The method according to claim 1, characterized in that The foreground image and the background image are face images, the designated area is the facial features area, and the designated area is determined based on key points of the face.

3. The method according to claim 1, characterized in that The processing of the foreground image comprises: Processing the foreground image based on a Gaussian pyramid includes: The foreground image is subjected to low-pass filtering and downsampling processing according to a preset number of repetitions to obtain images of each layer constituting a Gaussian pyramid; wherein the highest layer image is a low-frequency image of the foreground image, and the difference between the images of each two adjacent layers is a high-frequency image of the foreground image.

4. The method according to claim 1, wherein Before extracting the low-frequency information of the non-designated area corresponding to the low-frequency image of the background image, the method further includes: Performing erosion processing and Gaussian blur processing on the first mask image.

5. The method according to claim 1, wherein The method further comprises: Determining a to-be-fused region in the preprocessed foreground image, and constructing a second mask image according to the to-be-fused region; Performing erosion processing and Gaussian blur processing on the second mask image.

6. The method according to claim 5, characterized in that The method further comprises: Perform alpha blending processing on the background image according to the boundary pixel values of the area corresponding to the area to be fused on the second mask image.

7. An image fusion preprocessing device, characterized in that: The image includes a foreground image and a background image to be fused, and the device includes: an image processing module, configured to process the foreground image and the background image to obtain a high-frequency image and a low-frequency image of the foreground image, and a low-frequency image of the background image; The image processing module is further configured to construct a first mask image based on the designated area, wherein the first mask image is a grayscale image of the same size as the foreground image, and an area in the first mask image corresponding to the non-designated area in the foreground image is a region of interest; low-frequency information of the non-designated area corresponding to the low-frequency image of the background image is extracted through the region of interest; an image replacement module, replacing low-frequency information of a non-designated area of the low-frequency image of the foreground image with low-frequency information of a non-designated area corresponding to the low-frequency image of the background image, to obtain a replaced low-frequency image; The image operation module is used to add the obtained replaced low-frequency image to the high-frequency image of the foreground image to obtain a preprocessed foreground image.

8. A device, characterized in that include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images