Image fusion method and device and storage medium

Abstract

The invention relates to an image fusion method and device and a storage medium. The image fusion method comprises the steps of obtaining a first image and a second image; extracting a color parameter value meeting a condition from the first image, and fusing the first image and the second image based on the color parameter value to obtain a fused image; wherein the first image in the fused image is a foreground image, the second image is a background image, and the color of the background image is determined based on the color parameter value. According to the invention, the color of the background image in the fused image is close to the color of the foreground image, the fusion degree is higher, and the fused image is more natural.

Description

Technical Field

[0001] The present disclosure relates to the field of image technology, and in particular to an image fusion method, device and storage medium. Background Art

[0002] At present, in the field of image technology, two images can be fused together. For example, for a product image that needs to be displayed, you can choose any landscape image and overlay the product image on the landscape image to show the product to the user. Summary of the invention

[0003] In order to overcome the problems existing in the related art, the present disclosure provides an image fusion method, device and storage medium.

[0004] According to a first aspect of an embodiment of the present disclosure, there is provided an image fusion method, comprising: acquiring a first image and a second image; extracting a color parameter value that satisfies a condition in the first image, and based on the color parameter value, fusing the first image and the second image to obtain a fused image; wherein, in the fused image, the first image is a foreground image, the second image is a background image, and the color of the background image is determined based on the color parameter value.

[0005] In one embodiment, the first image and the second image are fused based on the color parameter value to obtain a fused image, including: expanding the peripheral area of ​​the first image and filling the peripheral area according to the color parameter value to obtain a third image, the size of the peripheral area satisfies that the size of the third image is consistent with the size of the second image; and fusing the second image and the third image to obtain a fused image.

[0006] In one embodiment, extracting color parameter values ​​that meet the conditions in the first image includes: extracting K groups of color parameter values ​​in the first image in descending order of the area of ​​the regions corresponding to each color, where K is a positive integer and each group of color parameter values ​​corresponds to a color; and determining one or more groups of color parameter values ​​from the K groups of color parameter values ​​as color parameter values ​​that meet the conditions.

[0007] In one embodiment, determining one or more groups of color parameter values ​​from the K groups of color parameter values ​​as color parameter values ​​that meet the conditions includes: determining the color parameter value corresponding to the first non-black and non-white color from the K groups of color parameters in the order of the area corresponding to each color from large to small, as the color parameter value that meets the conditions.

[0008] In one embodiment, the fusing of the first image and the second image to obtain a fused image includes: generating an image mask, wherein the image mask is used to shield a third image during the fusion process; fusing the first image and the second image, and during the fusion process, shielding the third image based on the image mask, and performing color processing on the second image to obtain a fused image.

[0009] According to a second aspect of an embodiment of the present disclosure, an image fusion device is provided, comprising: an acquisition module, used to acquire a first image and a second image; a processing module, used to extract color parameter values ​​that meet a condition in the first image, and based on the color parameter values, fuse the first image and the second image to obtain a fused image; wherein, in the fused image, the first image is a foreground image, the second image is a background image, and the color of the background image is determined based on the color parameter values.

[0010] In one embodiment, the processing module fuses the first image and the second image based on the color parameter value to obtain a fused image in the following manner: expands the peripheral area of ​​the first image and fills the peripheral area according to the color parameter value to obtain a third image, and the size of the peripheral area satisfies that the size of the third image is consistent with the size of the second image; fuses the second image and the third image to obtain a fused image.

[0011] In one embodiment, the processing module extracts color parameter values ​​that meet the conditions in the first image in the following manner: extract K groups of color parameter values ​​in the first image in the order of the area corresponding to each color from large to small, where K is a positive integer and each group of color parameter values ​​corresponds to a color; and determine one or more groups of color parameter values ​​from the K groups of color parameter values ​​as color parameter values ​​that meet the conditions.

[0012] In one embodiment, the processing module determines one or more groups of color parameter values ​​from the K groups of color parameter values ​​as color parameter values ​​that meet the conditions in the following manner: from the K groups of color parameter values, in the order of the area corresponding to each color from large to small, determine the color parameter value corresponding to the first non-black and non-white color as the color parameter value that meets the conditions.

[0013] In one embodiment, the processing module fuses the first image and the second image in the following manner to obtain a fused image: generating an image mask, wherein the image mask is used to shield a third image during the fusion process; fusing the first image and the second image, and during the fusion process, shielding the third image based on the image mask, and performing color processing on the second image to obtain a fused image.

[0014] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a memory for storing instructions; and a processor for calling the instructions stored in the memory to execute the method in the first aspect or any one of the implementations of the first aspect.

[0015] According to a fourth aspect of an embodiment of the present disclosure, a storage medium is provided, in which instructions are stored. When the instructions are executed by a processor, the method in the first aspect or any one of the implementations of the first aspect is executed.

[0016] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: by acquiring the first image and the second image, extracting the color parameter value that meets the condition in the first image, and fusing the first image and the second image based on the color parameter value, the color of the background image in the fused image is made close to the color of the foreground image, so that the degree of fusion is higher and the fused image is more natural.

[0017] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0019] Figure 1 is a schematic diagram showing images before and after fusion according to an exemplary embodiment.

[0020] Figure 2 The figure is a flow chart of an image fusion method according to an exemplary embodiment.

[0021] Figure 3 The figure is a flowchart of an image fusion method according to an exemplary embodiment.

[0022] Figure 4 The figure is a flowchart of a method for determining a color parameter value according to an exemplary embodiment.

[0023] Figure 5 is a schematic diagram of an image mask according to an exemplary embodiment.

[0024] Figure 6 The figure is a flowchart of an image fusion method according to an exemplary embodiment.

[0025] Figure 7 is a schematic diagram showing image fusion according to an exemplary embodiment.

[0026] Figure 8The figure is a block diagram of an image fusion device according to an exemplary embodiment.

[0027] Fig. 9 The figure is a block diagram of an image fusion device according to an exemplary embodiment. DETAILED DESCRIPTION

[0028] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure.

[0029] At present, in the field of image technology, two images can be fused together. For example, for a product image that needs to be displayed, you can choose any landscape image and overlay the product image on the landscape image to show the product to the user.

[0030] In some embodiments, the product image in the above example is the foreground image, and the landscape image is the background image. When the foreground image and the background image are fused, the foreground image can be directly copied and pasted onto the background image, i.e., the copy-paste method. However, due to the inconsistency of the tones between the foreground image and the background image, the fused image often has a poor degree of fusion, and the foreground image has a visual abruptness. Figure 1 FIG. 1 is a schematic diagram showing images before and after fusion according to an exemplary embodiment. Figure 1 As shown in the figure, from left to right are the foreground image, the background image, and the image after the foreground image and the background image are fused. It can be seen that Figure 1 The image after fusion is unnatural.

[0031] Therefore, the present disclosure provides an image fusion method, which obtains a first image and a second image, extracts a color parameter value that satisfies a condition in the first image, and fuses the first image and the second image based on the color parameter value, so that the color of the background image in the fused image is close to the color of the foreground image, so that the fusion degree is higher and the fused image is more natural.

[0032] The image fusion method provided by the present disclosure can be applied to a terminal or a server, etc. Among them, the terminal can also be called a terminal device, a mobile station (MS), a mobile terminal (MT), etc., which is a device that provides voice and / or data connectivity to users. For example, the terminal can be a handheld device with a wireless connection function, a vehicle-mounted device, etc. At present, some examples of terminals are: a smart phone (mobile phone), a pocket personal computer (PPC), a handheld computer, a personal digital assistant (PDA), a laptop, a tablet computer, a wearable device, or a vehicle-mounted device, etc. In addition, when it is a vehicle to everything (V2X) communication system, the terminal device can also be a vehicle-mounted device. It should be understood that the embodiments of the present disclosure do not limit the specific technology and specific device form adopted by the terminal.

[0033] Figure 2 is a flow chart of an image fusion method according to an exemplary embodiment. Figure 2 As shown, the image fusion method includes the following steps.

[0034] In step S11 , a first image and a second image are acquired.

[0035] In some embodiments, a first image and a second image may be obtained. The first image is a foreground image to be fused, that is, an image placed on the top layer after fusion. The second image is a background image to be fused, that is, an image placed on the bottom layer after fusion.

[0036] In some embodiments, the first image may be an image that is smaller in size than the second image. Figure 1 The foreground image size is shown to be smaller than the background image size. Figure 1 In the fused image shown, the connecting portion between the area corresponding to the foreground image and the area corresponding to the background image has an obvious sense of separation, and the fusion effect is unnatural.

[0037] In step S12, color parameter values ​​that meet the conditions are extracted from the first image, and based on the color parameter values, the first image and the second image are fused to obtain a fused image.

[0038] In some embodiments, the color parameter value that meets the conditions in the first image can be extracted, and the first image and the second image can be fused based on the color parameter value. The color parameter value includes the values ​​corresponding to the three primary colors, that is, the values ​​corresponding to red (red, R), green (green, G), and blue (blue, B). Different values ​​of R, G, and B determine the color. For example, if the grayscale level is 256, that is, the value range of R, G, and B is between (0-255), then the color parameter value corresponding to black is (0,0,0), and the values ​​of R, G, and B are all 0. The color parameter value corresponding to white is (255,255,255), and the values ​​of R, G, and B are all 255.

[0039] In some embodiments, in the fused image, the first image is a foreground image and the second image is a background image. The color of the background image is determined based on the extracted color parameter value that meets the condition. The color of the second image can be adjusted based on the color parameter value that meets the condition. For example, a pure color image is generated and fused with the second image. The fused second image can be understood as fused with the color of the first image, and then fused with the first image, which can make the fusion effect more natural. For another example, the background of the first image is filled based on the color parameter value that meets the condition. The first image after the background is filled is fused with the second image.

[0040] The present invention obtains a first image and a second image, extracts a color parameter value that satisfies a condition in the first image, and fuses the first image and the second image based on the color parameter value, so that the color of the background image in the fused image is close to the color of the foreground image, so that the fusion degree is higher and the fused image is more natural.

[0041] In the present disclosure, the color of the first image is used as a reference. For example, the color of the first image can be extracted as a pure color background of the first image. Figure 3 FIG. 1 is a flow chart of an image fusion method according to an exemplary embodiment. Figure 3 As shown, the image fusion method includes the following steps.

[0042] In step S21, the peripheral area of ​​the first image is expanded and filled according to the color parameter value to obtain a third image, and the size of the peripheral area satisfies the size of the third image to be consistent with the size of the second image.

[0043] In some embodiments, the peripheral area of ​​the first image can be expanded, and the peripheral area can be filled according to the extracted color parameter values ​​that meet the conditions to obtain a third image. For example, multiple pixels can be determined in the periphery of the first image, and the color parameter value of each pixel is determined based on the extracted color parameter value that meets the conditions. For example, if the color parameter value that meets the conditions is a group of color parameter values, it can be determined that the color parameter value of each pixel is the same as the group of color parameter values. For another example, if the color parameter value that meets the conditions is a plurality of groups of color parameter values, the average value of the plurality of groups of color parameter values ​​can be calculated as the color parameter value of each pixel. An image containing the first image and a plurality of pixels is used as a third image. For example, a plurality of determined pixels can be drawn around the first image to generate a third image. Among them, the size of the peripheral area satisfies the size of the third image to be consistent with the size of the second image, so as to facilitate the subsequent fusion of the third image and the second image.

[0044] In step S22, the second image and the third image are fused.

[0045] In some embodiments, the second image and the third image may be fused. For example, the second image and the third image may be input into a neural network model, and a fused image may be output. Alternatively, other fusion methods may be used, which are not limited in the present disclosure.

[0046] The present disclosure obtains a third image by expanding the peripheral area of ​​the first image and filling the peripheral area according to the color parameter value. The third image is fused with the second image so that the color of the area corresponding to the second image in the fused image is close to the color of the first image. That is, the fusion effect is more natural.

[0047] In some embodiments, Figure 4 is a flow chart of a method for determining a color parameter value according to an exemplary embodiment. Figure 4 The present disclosure provides a method for determining a color parameter value, comprising the following steps.

[0048] In step S31, K groups of color parameter values ​​are extracted from the first image in the order of the area of ​​the region corresponding to each color from large to small, where K is a positive integer and each group of color parameter values ​​corresponds to a color.

[0049] In some embodiments, there is at least one color in the first image. When there is only one color in the first image, the color parameter value corresponding to the color can be extracted as the color parameter value that meets the condition. When there are multiple colors in the first image, the area occupied by each color can be sorted from large to small, and the color parameter values ​​corresponding to the first K colors can be extracted, that is, K groups of color parameter values. The K groups of color parameter values ​​are used to determine one or more groups of color parameter values ​​as the color parameter values ​​of the first image. Each group of color parameter values ​​includes R, G, and B values ​​corresponding to one color. K groups of color parameter values ​​represent the color parameter values ​​corresponding to K colors. K is a positive integer. For example, the size of K can be preset. K groups of color parameter values ​​can be extracted according to the image main color extraction algorithm of mean shift filtering and flood filling. Of course, K groups of color parameter values ​​can also be extracted according to other feasibility algorithms, which are not limited in the present disclosure.

[0050] In step S32, one or more groups of color parameter values ​​are determined from the K groups of color parameter values ​​as color parameter values ​​that meet the conditions.

[0051] In some embodiments, a group of color parameter values ​​can be determined from K groups of color parameter values ​​as color parameter values ​​that meet the conditions. Multiple groups of color parameter values ​​can also be determined from K groups of color parameter values, and the average values ​​of the multiple groups of color parameter values ​​can be calculated. For example, the R, G, and B values ​​of one group of color parameter values ​​are R1, G1, and B1, respectively, and the R, G, and B values ​​of another group of color parameter values ​​are R2, G2, and B2, respectively. Then, the average values ​​of the two groups of color parameter values ​​are calculated, that is, the average values ​​of R1 and R2, the average values ​​of G2 and G2, and the average values ​​of B1 and B2 are calculated respectively. These three average values ​​are used as color parameter values ​​that meet the conditions.

[0052] In some embodiments, in response to the presence of a color parameter value corresponding to a color with the largest corresponding area in the K groups of color parameter values, the color parameter value corresponding to the color can be used as a color parameter value that meets the conditions. For example, if there is no color parameter value corresponding to a color with the largest area ratio in the K groups of color parameter values, that is, there may be multiple colors corresponding to the same area, and the area is larger than the area corresponding to other colors. It can be understood that there is more than one color with the largest corresponding area. Then, multiple groups of color parameter values ​​can be determined, and the average value of the multiple groups of color parameter values ​​can be calculated. Alternatively, if the area areas corresponding to multiple colors in the first image are slightly different, for example, less than a threshold value, multiple groups of color parameter values ​​corresponding to these multiple colors can be determined. The average value of the multiple groups of color parameter values ​​can be calculated later as the color parameter value of the first image.

[0053] The present disclosure extracts K groups of color parameter values, and determines one or more groups of color parameter values ​​therefrom as the color parameter values ​​of the first image. The K groups of color parameter values ​​are the color parameter values ​​corresponding to the K colors with the largest area ratio in the first image. Therefore, the subsequent fusion effect can be better.

[0054] In some embodiments, a large area of ​​black may cause the image to be underexposed, that is, the image is too dark. A large area of ​​white may cause the image to be overexposed, that is, the image is too bright. An image that is too bright or too dark may result in unclear image details and affect image quality. Therefore, the color parameter value determination method provided by the present disclosure includes: from the K groups of color parameter values, in the order of the area corresponding to each color from large to small, determine the color parameter value corresponding to the first non-black and non-white color as the color parameter value that meets the condition.

[0055] In some embodiments, when determining one or more groups of color parameter values ​​from K groups of color parameter values, black or white color parameter values ​​can be excluded. According to the order of the area corresponding to each color from large to small, when the color parameter value ranked first in the K groups of color parameter values ​​is the color parameter value corresponding to black or white, the color parameter value is discarded. Continue to judge whether the next color parameter value is the color parameter value corresponding to black or white. The color parameter value corresponding to the first non-black and non-white color is determined as the color parameter value corresponding to the first image. For example, when the determined color parameter value is a group of color parameter values, it can be the color parameter value corresponding to the first non-black and non-white color. When the determined color parameter value is a plurality of groups of color parameter values, the plurality of groups of color parameter values ​​can be the color parameter value corresponding to the first non-black and non-white color, the color parameter value corresponding to the second non-black and non-white color, and the color parameter value corresponding to the Nth non-black and non-white color. That is, if the next color adjacent to the first non-black and non-white color is black or white, it is discarded until the next non-black and non-white color is screened and its corresponding color parameter value is used as one of the determined plurality of groups of color parameter values.

[0056] In some embodiments, when extracting K groups of color parameter values, the color parameter values ​​corresponding to black and white may be screened out, that is, it is ensured that there are no color parameter values ​​corresponding to black and white in the extracted K groups of color parameter values.

[0057] The present disclosure determines the first or first N non-black and non-white color parameter values ​​from the K groups of color parameter values, so that the fused image will not be too dark due to a large area of ​​black pixels in the third image, nor will it be too bright due to a large area of ​​white pixels in the third image.

[0058] In some embodiments, when two images are fused, an image mask may be generated, which may cover all or part of the image to be processed through a selected image, graphic or object to control the image processing area or processing process. Figure 5 FIG. 1 is a schematic diagram of an image mask according to an exemplary embodiment. Figure 5 As shown, it is Figure 1 The mask used when fusing the foreground image and the background image. Figure 5 The mask shown indicates that the foreground image is processed, while the background image area outside the foreground image is shielded and not processed. That is, the foreground image and the background image can be quickly fused through the image mask. Figure 5 The area to be processed is the foreground area, that is, the tone of the foreground image is processed to make it closer to the tone of the background image. This step is also called harmonizing the foreground image. However, this method reduces the overall fusion degree and makes the fusion effect relatively unnatural.

[0059] Therefore, the present disclosure provides an image fusion method. Figure 6 is a flowchart of an image fusion method according to an exemplary embodiment. Figure 6 As shown, the following steps are included.

[0060] In step S41, an image mask is generated, and the image mask is used to shield the third image during the fusion process.

[0061] In step S42, the first image and the second image are fused, and in the fusion process, the third image is shielded based on the image mask, and the second image is color processed to obtain a fused image.

[0062] In some embodiments, an image mask may be generated, and the image mask is used to shield the third image during the fusion process, that is, to block the third image so that the third image is not processed and does not participate in the subsequent processing. The first image and the second image are fused, and during the fusion process, the third image is shielded based on the image mask, and the second image is color processed to obtain a fused image. The second image is color processed so that the color of the second image is close to that of the third image, that is, the second image is harmonized to the third image.

[0063] In some embodiments, the second image and the third image may be fused based on the image mask. For example, after the second image and the third image are superimposed, the area outside the superimposed first image may be processed to correct the color tone of the area outside the first image to the color tone of the first image.

[0064] In some embodiments, the third image is input as the foreground, and the second image is input as the background, and input into a pre-trained neural network model to obtain a fused image. The neural network model can be pre-trained in the following manner: obtain training data. The training data includes an image placed at the top layer after fusion and an image placed at the bottom layer after fusion. The image placed at the top layer after fusion is processed as the first image to the third image to obtain first training data. The image placed at the bottom layer after fusion is used as second training data. A training mask corresponding to the training data is generated, and the processed area of ​​the training mask corresponds to the second training data, and the non-processed area corresponds to the first training data. According to the image mask, the first training data and the second training data are fused to obtain a standard image. The first training data is used as the foreground data, and the second training data is used as the background input, and input into an untrained neural network model for training. The parameters of the neural network model are adjusted using the standard image so that the output result of the neural network model is close to the standard image.

[0065] The present invention generates an image mask, shields the third image based on the image mask, and performs color processing on the second image to obtain a fused image, thereby making the fusion degree of the fused image higher and the effect more natural.

[0066] Figure 7 FIG. 1 is a schematic diagram showing an image fusion according to an exemplary embodiment. Figure 7 As shown, from top to bottom are respectively an exemplary first image, a second image, a third image, an image mask, and a fused image. The third image and the second image are fused using the solution of the embodiment of the present disclosure, so that the fused image has a more natural effect.

[0067] Based on the same concept, the embodiment of the present disclosure also provides an image fusion device.

[0068] It is understandable that in order to realize the above functions, the image fusion device provided by the embodiment of the present disclosure includes hardware structures and / or software modules corresponding to the execution of each function. In combination with the units and algorithm steps of each example disclosed in the embodiment of the present disclosure, the embodiment of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present disclosure.

[0069] It should be noted that those skilled in the art can understand that the various implementation methods / embodiments involved in the embodiments of the present disclosure can be used in conjunction with the aforementioned embodiments or can be used independently. Whether used alone or in conjunction with the aforementioned embodiments, the implementation principle is similar. In the implementation of the present disclosure, some embodiments are described in terms of implementation methods used together. Of course, those skilled in the art can understand that such examples are not limitations of the embodiments of the present disclosure.

[0070] Figure 8 FIG. 1 is a block diagram of an image fusion device 100 according to an exemplary embodiment. Figure 8 The device 100 includes: an acquisition module 101 and a processing module 102.

[0071] The acquisition module 101 is used to acquire a first image and a second image. The processing module 102 is used to extract a color parameter value that meets a condition in the first image, and fuse the first image and the second image based on the color parameter value to obtain a fused image. In the fused image, the first image is a foreground image, the second image is a background image, and the color of the background image is determined based on the color parameter value.

[0072] In one embodiment, the processing module 102 fuses the first image and the second image based on the color parameter value to obtain a fused image in the following manner: the peripheral area of ​​the first image is expanded, and the peripheral area is filled according to the color parameter value to obtain a third image, and the size of the peripheral area satisfies that the size of the third image is consistent with the size of the second image. The second image and the third image are fused to obtain a fused image.

[0073] In one embodiment, the processing module 102 extracts the color parameter values ​​satisfying the condition in the first image in the following manner: extract K groups of color parameter values ​​in the first image in the order of the area corresponding to each color from large to small, where K is a positive integer and each group of color parameter values ​​corresponds to a color. From the K groups of color parameter values, determine one or more groups of color parameter values ​​as the color parameter values ​​satisfying the condition.

[0074] In one embodiment, the processing module 102 determines one or more groups of color parameter values ​​from K groups of color parameter values ​​as color parameter values ​​that meet the conditions in the following manner: from the K groups of color parameter values, in the order of the area corresponding to each color from large to small, determine the color parameter value corresponding to the first non-black and non-white color as the color parameter value that meets the conditions.

[0075] In one embodiment, the processing module 102 fuses the first image and the second image in the following manner to obtain a fused image: generating an image mask, the image mask is used to shield the third image during the fusion process. Fusing the first image and the second image, and during the fusion process, shielding the third image based on the image mask, and performing color processing on the second image to obtain a fused image.

[0076] Fig. 9 is a block diagram of an image fusion device 200 according to an exemplary embodiment.

[0077] like Fig. 9 As shown, apparatus 200 may include one or more of the following components: a processing component 202 , a memory 204 , a power component 206 , a multimedia component 208 , an audio component 210 , an input / output (I / O) interface 212 , a sensor component 214 , and a communication component 216 .

[0078] The processing component 202 generally controls the overall operation of the device 200, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 202 may include one or more modules to facilitate interaction between the processing component 202 and other components. For example, the processing component 202 may include a multimedia module to facilitate interaction between the multimedia component 208 and the processing component 202.

[0079] The memory 204 is configured to store various types of data to support operations on the device 200. Examples of such data include instructions for any application or method operating on the device 200, contact data, phone book data, messages, pictures, videos, etc. The memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0080] The power component 206 provides power to the various components of the device 200. The power component 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 200.

[0081] The multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 208 includes a front camera and / or a rear camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0082] The audio component 210 is configured to output and / or input audio signals. For example, the audio component 210 includes a microphone (MIC), and when the device 200 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 204 or sent via the communication component 216. In some embodiments, the audio component 210 also includes a speaker for outputting audio signals.

[0083] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0084] The sensor assembly 214 includes one or more sensors for providing various aspects of the status assessment of the device 200. For example, the sensor assembly 214 can detect the open / closed state of the device 200, the relative positioning of components, such as the display and keypad of the device 200, the sensor assembly 214 can also detect the position change of the device 200 or a component of the device 200, the presence or absence of user contact with the device 200, the orientation or acceleration / deceleration of the device 200 and the temperature change of the device 200. The sensor assembly 214 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 214 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 214 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

[0085] The communication component 216 is configured to facilitate wired or wireless communication between the device 200 and other devices. The device 200 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0086] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0087] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, and the instructions can be executed by the processor 220 of the device 200 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0088] The present invention obtains a first image and a second image, extracts a color parameter value that satisfies a condition in the first image, and fuses the first image and the second image based on the color parameter value, so that the color of the background image in the fused image is close to the color of the foreground image, so that the fusion degree is higher and the fused image is more natural.

[0089] It is to be understood that in the present disclosure, "plurality" refers to two or more than two, and other quantifiers are similar thereto. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. The singular forms "a", "" and "the" are also intended to include plural forms, unless the context clearly indicates other meanings.

[0090] It is further understood that the terms "first", "second", etc. are used to describe various information, but such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other, and do not indicate a specific order or degree of importance. In fact, the expressions "first", "second", etc. can be used interchangeably. For example, without departing from the scope of the present disclosure, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information.

[0091] It is further understood that, although the operations are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be understood as requiring the operations to be performed in the specific order shown or in a serial order, or requiring the execution of all the operations shown to obtain the desired results. In certain environments, multitasking and parallel processing may be advantageous.

[0092] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modifications, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure.

[0093] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

Claims

1. An image fusion method, It is characterized in that include: Acquire a first image and a second image; Extracting color parameter values ​​that meet a condition from the first image, and fusing the first image and the second image based on the color parameter values ​​to obtain a fused image; Among them, in the fused image, the first image is a foreground image, the second image is a background image, and the color of the background image is determined based on the color parameter value.

2. The method according to claim 1, It is characterized in that The step of fusing the first image and the second image based on the color parameter value to obtain a fused image includes: Expanding the peripheral area of ​​the first image and filling the peripheral area according to the color parameter value to obtain a third image, wherein the size of the peripheral area satisfies the size of the third image to be consistent with the size of the second image; The second image and the third image are fused to obtain a fused image.

3. The method according to claim 1 or 2, It is characterized in that The step of extracting a color parameter value satisfying a condition from the first image comprises: Extract K groups of color parameter values ​​in the first image in descending order of the area of ​​the regions corresponding to the colors, where K is a positive integer and each group of color parameter values ​​corresponds to a color; From the K groups of color parameter values, one or more groups of color parameter values ​​are determined as color parameter values ​​that meet the conditions.

4. The method according to claim 3, It is characterized in that The step of determining one or more groups of color parameter values ​​from the K groups of color parameter values ​​as color parameter values ​​that meet the conditions includes: From the K groups of color parameter values, in descending order of the area of ​​the regions corresponding to the colors, the color parameter value corresponding to the first non-black and non-white color is determined as the color parameter value that meets the condition.

5. The method according to claim 1, It is characterized in that The fusing the first image and the second image to obtain a fused image includes: generating an image mask, wherein the image mask is used to shield the third image during the fusion process; The first image and the second image are fused, and during the fusion process, the third image is shielded based on the image mask, and the second image is color processed to obtain a fused image.

6. An image fusion device, It is characterized in that include: An acquisition module, used for acquiring a first image and a second image; a processing module, configured to extract color parameter values ​​satisfying a condition from the first image, and fuse the first image and the second image based on the color parameter values ​​to obtain a fused image; Among them, in the fused image, the first image is a foreground image, the second image is a background image, and the color of the background image is determined based on the color parameter value.

7. The device according to claim 6, It is characterized in that The processing module fuses the first image and the second image based on the color parameter value to obtain a fused image in the following manner: Expanding the peripheral area of ​​the first image and filling the peripheral area according to the color parameter value to obtain a third image, wherein the size of the peripheral area satisfies the size of the third image to be consistent with the size of the second image; The second image and the third image are fused to obtain a fused image.

8. The device according to claim 6 or 7, It is characterized in that The processing module extracts the color parameter value that meets the condition from the first image in the following manner: Extract K groups of color parameter values ​​in the first image in descending order of the area of ​​the regions corresponding to the colors, where K is a positive integer and each group of color parameter values ​​corresponds to a color; From the K groups of color parameter values, one or more groups of color parameter values ​​are determined as color parameter values ​​that meet the conditions.

9. The device according to claim 8, It is characterized in that The processing module determines one or more groups of color parameter values ​​from the K groups of color parameter values ​​as color parameter values ​​that meet the conditions in the following manner: From the K groups of color parameter values, in descending order of the area of ​​the regions corresponding to the colors, the color parameter value corresponding to the first non-black and non-white color is determined as the color parameter value that meets the condition.

10. The device according to claim 6, It is characterized in that The processing module fuses the first image and the second image in the following manner to obtain a fused image: generating an image mask, wherein the image mask is used to shield the third image during the fusion process; The first image and the second image are fused, and during the fusion process, the third image is shielded based on the image mask, and the second image is color processed to obtain a fused image.

11. An electronic device, It is characterized in that include: A memory for storing instructions; as well as A processor, configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 5.

12. A storage medium, It is characterized in that The storage medium stores instructions, and when the instructions are executed by the processor, the method according to any one of claims 1 to 5 is executed.

Images