Image synthesis method and related device

Abstract

The invention discloses an image synthesis method and a related device, and relates to the technical field of image synthesis, and the method comprises the steps: carrying out the regional illumination statistical analysis processing of a to-be-synthesized background image, obtaining the respective illumination information representation values of a plurality of image regions corresponding to the background image, carrying out the illumination shadow removal processing of a to-be-synthesized target image, and obtaining the illumination information representation values of a plurality of image regions corresponding to the background image; and obtaining a shadow weakening image corresponding to the target image, performing illumination compensation processing on the shadow weakening image according to the respective illumination information representation values of the plurality of image areas to obtain a compensated image, and performing image synthesis processing based on the compensated image and the background image to obtain a target synthesis image composed of the target object and the background image. According to the method, the shadow caused by illumination in the target image can be firstly subjected to illumination shadow removal processing, and then illumination compensation processing is performed, so that the illumination conditions of the target object in the compensated image and the background image are more consistent, the naturalness and reality sense of image synthesis are improved, and the image synthesis effect is further improved.

Description

Technical Field

[0001] This application relates to the field of image synthesis technology, and particularly to an image synthesis method and related device. Background Art

[0002] With the development of image processing technology, users can synthesize the objects in the target image with the background image to obtain a synthesized image. However, the lighting conditions of the target image and the background image may be inconsistent, resulting in insufficient lighting realism and naturalness of the synthesized image, which greatly reduces the effect of image synthesis. Summary of the Invention

[0003] In view of the above problems, this application provides an image synthesis method and related device to achieve the purpose of synthesizing the target object in the target image and the background image into a target synthesized image with more realistic and natural lighting. The specific solutions are as follows:

[0004] The first aspect of this application provides an image synthesis method, including:

[0005] Perform regional lighting statistical analysis processing on the background image to be synthesized to obtain the lighting information characterization values of the respective multiple image regions corresponding to the background image, where the absolute value of the difference between the gray values of any two pixels in the same image region is less than or equal to a preset gray threshold, and the absolute value of the difference between the gray values of any two pixels in different image regions is greater than the gray threshold;

[0006] Perform de-lighting shadow processing on the target image to be synthesized to obtain a shadow-weakened image corresponding to the target image, where the target image is an image taken of a target object, and the de-lighting shadow processing is used to remove the shadow caused by the light source when taking the target image on the target object;

[0007] Perform lighting compensation processing on the shadow-weakened image according to the lighting information characterization values of the respective multiple image regions to obtain a compensated image;

[0008] Perform image synthesis processing based on the compensated image and the background image to obtain a target synthesized image composed of the target object and the background image.

[0009] In a possible implementation, the performing regional lighting statistical analysis processing on the background image to be synthesized to obtain the lighting information characterization values of the respective multiple image regions corresponding to the background image includes:

[0010] Perform image partitioning processing on the background image based on lighting consistency to obtain the multiple image regions;

[0011] Perform statistical analysis processing on the illumination brightness of each of the multiple image regions respectively to obtain the illumination intensity characterization values of the multiple image regions;

[0012] Perform statistical analysis processing on the illumination color of each of the multiple image regions respectively to obtain the illumination color compensation values of the multiple image regions;

[0013] Use the illumination intensity characterization values and illumination color compensation values of the multiple image regions as the illumination information characterization values of the multiple image regions;

[0014] The illumination compensation processing of the shadow-weakened image according to the illumination information characterization values of the multiple image regions to obtain a compensated image includes:

[0015] Perform illumination compensation processing on the shadow-weakened image according to the illumination intensity characterization values and illumination color compensation values of the multiple image regions to obtain the compensated image.

[0016] In a possible implementation, the image partitioning processing of the background image based on illumination consistency to obtain the multiple image regions includes:

[0017] Convert the background image into a grayscale image;

[0018] Perform Gaussian filtering on the grayscale image to obtain a filtered image;

[0019] Calculate the gradients of the filtered image in the horizontal and vertical directions to obtain the gradient image corresponding to the filtered image;

[0020] Divide the grayscale image into multiple initial regions based on the gradient image;

[0021] Perform merging processing on the multiple initial regions based on a preset gray value merging condition to obtain the multiple image regions.

[0022] In a possible implementation, the dividing the grayscale image into multiple initial regions based on the gradient image includes:

[0023] Perform non-maximum suppression processing on the gradient image to obtain a first gradient updated image;

[0024] Calculate the preset quantile of the first gradient updated image as the first threshold;

[0025] Determine a second threshold according to the first threshold, where the second threshold is less than the first threshold;

[0026] Perform extreme value removal processing on the first gradient updated image based on the first threshold and the second threshold to obtain a second gradient updated image;

[0027] Perform erosion and dilation processing on the second gradient updated image to obtain a third gradient updated image;

[0028] Perform region division processing on the grayscale image based on at least one of the first gradient updated image, the second gradient updated image, and the third gradient updated image to obtain the multiple initial regions.

[0029] In a possible implementation, the performing statistical analysis processing on the illumination brightness of each of the multiple image regions respectively to obtain the illumination intensity characterization values of the multiple image regions includes:

[0030] For each of the multiple image regions, calculate the brightness mean and contrast of the image region, and perform weighted fusion processing on the brightness mean and contrast of the image region based on the brightness mean weight to obtain the illumination intensity characterization value of the image region; so as to obtain the illumination intensity characterization values of the multiple image regions respectively.

[0031] In a possible implementation, the performing statistical analysis processing on the illumination color of each of the multiple image regions respectively to obtain the illumination color compensation values of the multiple image regions includes:

[0032] For each of the multiple image regions, calculate the R-channel mean, G-channel mean, and B-channel mean of the image region, perform color temperature estimation according to the R-channel mean, G-channel mean, and B-channel mean of the image region to obtain the color temperature value of the image region, and perform three different linear normalization processes on the color temperature value of the image region to obtain the R-channel compensation value, G-channel compensation value, and B-channel compensation value of the image region, as the illumination color compensation value of the image region; so as to obtain the illumination color compensation values of the multiple image regions respectively.

[0033] In a possible implementation, the performing color temperature estimation according to the R-channel mean, G-channel mean, and B-channel mean of the image region to obtain the color temperature value of the image region includes:

[0034] Taking the G-channel mean of the image region as a reference, perform proportional scaling on the R-channel mean and B-channel mean of the image region respectively to obtain a first color deviation of the R-channel compared to the G-channel and a second color deviation of the B-channel compared to the G-channel;

[0035] Determine a third color deviation of the B-channel compared to the R-channel under the influence of the G-channel according to the first color deviation and the second color deviation;

[0036] Perform linear adjustment on the third color deviation to obtain the color temperature value of the image region.

[0037] In a possible implementation, the process of performing de-lighting and shadow processing on the target image to be synthesized to obtain a shadow-weakened image corresponding to the target image includes:

[0038] Perform histogram equalization processing on the grayscale image of the target image to obtain a first intermediate image;

[0039] Convert the first intermediate image into a second intermediate image in the HSV color space;

[0040] Detect the shadow area mask of the target image according to the V value in the second intermediate image to obtain a shadow mask map;

[0041] Enhance the brightness of the target image according to the shadow mask map to obtain the shadow-weakened image.

[0042] In a possible implementation, the process of enhancing the brightness of the target image according to the shadow mask map to obtain the shadow-weakened image includes:

[0043] Perform morphological filtering on the shadow mask map to obtain a shadow mask filtered map;

[0044] Enhance the brightness of the target image according to the shadow mask filtered map to obtain the shadow-weakened image.

[0045] In a possible implementation, the process of performing light compensation processing on the shadow-weakened image according to the light intensity characterization values and light color compensation values of the multiple image regions to obtain the compensated image includes:

[0046] Perform light intensity compensation on the corresponding V value of the shadow-weakened image converted to the HSV color space according to the light intensity characterization values of the multiple image regions to obtain a third intermediate image in the HSV color space;

[0047] Perform light color compensation on the corresponding R value, G value, and B value of the third intermediate image converted to the RGB color space according to the light color compensation values of the multiple image regions to obtain an image in the RGB color space as the compensated image.

[0048] In a possible implementation, the process of performing image synthesis processing based on the compensated image and the background image to obtain a target composite image composed of the target object and the background image includes:

[0049] Determine the object area where the target object is located in the compensated image, and synthesize the object area into the background image to obtain an initial synthesized image;

[0050] Perform Gaussian filtering on the target area in the initial synthesized image to obtain the target synthesized image, where the target area includes the connection area between the object area and the background image and / or the connection area between different image areas within the object area.

[0051] The second aspect of the present application provides an image synthesis device, including: a lighting analysis module, a de-lighting shadow module, a lighting compensation module, and an image synthesis module;

[0052] The lighting analysis module is used to perform regional lighting statistical analysis on the background image to be synthesized, and obtain the lighting information characterization values of the respective multiple image areas corresponding to the background image, where the absolute value of the difference between the gray values of any two pixels in the same image area is less than or equal to a preset gray threshold, and the absolute value of the difference between the gray values of any two pixels in different image areas is greater than the gray threshold;

[0053] The de-lighting shadow module is used to perform de-lighting shadow processing on the target image to be synthesized to obtain a shadow-weakened image corresponding to the target image, where the target image refers to an image taken of a target object, and the de-lighting shadow processing is used to remove the shadow caused by the light source when taking the target image from the target object;

[0054] The lighting compensation module is used to perform lighting compensation processing on the shadow-weakened image according to the lighting information characterization values of the respective multiple image areas to obtain a compensated image;

[0055] The image synthesis module is used to perform image synthesis processing based on the compensated image and the background image to obtain a target synthesized image composed of the target object and the background image.

[0056] The third aspect of the present application provides a computer program product, including computer-readable instructions, which, when running on an electronic device, enable the electronic device to implement the image synthesis method of the first aspect or any implementation manner of the first aspect.

[0057] The fourth aspect of the present application provides an electronic device, including at least one processor and a memory connected to the processor, where:

[0058] The memory is used to store a computer program;

[0059] The processor is configured to execute the computer program, so that the electronic device can implement the image synthesis method according to the first aspect or any implementation manner of the first aspect described above.

[0060] A fifth aspect of the present application provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the image synthesis method according to the first aspect or any implementation manner of the first aspect described above.

[0061] By means of the above technical solution, for the image synthesis method provided by the present application, considering that if it is necessary to make the target object in the synthesized image and the background image more natural and realistic in terms of lighting conditions, the lighting information in the background image can be compensated to the target image where the target object is located to achieve lighting unity between the target object and the background image. Based on this, the present application first performs regional lighting statistical analysis processing on the background image to be synthesized to obtain the lighting information representation values of the respective image regions corresponding to the background image. Since the lighting conditions of the target image and the background image are inconsistent, if the lighting information representation values are directly compensated to the target image, although the lighting unity between the target object and the background image can be achieved to a certain extent, the shadow caused by the light source when the target image is taken on the target object still exists, resulting in low naturalness and realism of the image synthesis. Therefore, the present application first performs de-lighting shadow processing on the target image to be synthesized to obtain a shadow-weakened image corresponding to the target image, and then performs lighting compensation processing on the shadow-weakened image according to the lighting information representation values of the respective image regions to obtain a compensated image. Finally, image synthesis processing is performed based on the compensated image and the background image, and a target synthesized image with more realistic and natural lighting can be obtained, greatly improving the effect of image synthesis. At the same time, the entire image synthesis process only requires simple image processing and analysis, with low cost and high efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In combination with the drawings and with reference to the following specific embodiments, the above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic, and the original components and elements are not necessarily drawn to scale.

[0063] Figure 1 It is a schematic structural diagram of a system architecture provided by the present application;

[0064] Figure 2 It is a schematic flowchart of an image synthesis method provided by the present application;

[0065] Figure 3 It is a schematic diagram of a shadow mask provided by the present application;

[0066] Figure 4 Schematic structural diagram of an image synthesis device provided by this application;

[0067] Figure 5 Schematic structural diagram of an electronic device provided by this application. Specific embodiments

[0068] The embodiments of this application will be described below with reference to the accompanying drawings in the embodiments of this application. The terms used in the embodiments of this application are only for explaining the specific embodiments of this application, and are not intended to limit this application.

[0069] The embodiments of this application will be described below with reference to the accompanying drawings. Those of ordinary skill in the art will know that with the development of technology and the emergence of new scenarios, the technical solutions provided by the embodiments of this application are equally applicable to similar technical problems.

[0070] The terms "first", "second", etc. in the specification, claims and above-mentioned accompanying drawings of this application are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such terms can be interchanged under appropriate circumstances, which is only a way of distinction adopted when describing objects with the same attributes in the embodiments of this application. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion, so that a process, method, system, product or device including a series of units does not have to be limited to those units, but may include other units that are not clearly listed or are inherent to these processes, methods, products or devices.

[0071] This application can be applied to image synthesis scenarios where a target object needs to be synthesized into a background image. Several optional application scenarios are introduced below.

[0072] (1) In the production of TV dramas and movies, sometimes actors need to perform in front of a green or blue background, and later the actors are synthesized into the required background image (such as a sea level image).

[0073] (2) In advertising design, in order to highlight product features or create a specific atmosphere, it is often necessary to synthesize products or models into certain background images, such as synthesizing a car into a landscape image.

[0074] (3) In personal entertainment or social media, users often need to synthesize personal photos with various interesting backgrounds, such as synthesizing personal photos with popular scenic spot images.

[0075] It should be noted that in addition to the above scenarios, there are many scenarios where a target object needs to be synthesized into a background image, which will not be introduced one by one here.

[0076] Optionally, the image synthesis method provided in this application can be applied to, for example, Figure 1 the system architecture shown. The system may include a terminal 100 and a server 200. The server 200 may include one or more servers ( Figure 1 illustrated by taking one server as an example).

[0077] Of course, the terminal 100 or the server 200 can also be used alone to execute the image synthesis method provided in the embodiments of this application, and this application does not make any limitations.

[0078] Next, the product form of the terminal 100 will be described Figure 1 in

[0079] The terminal 100 in the embodiments of this application can be a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., and this application does not make any restrictions on this.

[0080] The terminal 100 may include a radio frequency unit, a memory, an input unit, a display unit, a camera (optional), an audio circuit (optional), a speaker (optional), a microphone (optional), a headphone jack (optional), a processor, an external interface, a power supply, and other components. Those skilled in the art can understand that the above components are only examples and do not constitute a limitation on the terminal or the multifunctional device. It may include more or fewer components, or combine some components, or different components.

[0081] The input unit can be used to receive input digital or character information, and generate key signal inputs related to the user settings and function controls of the portable multifunctional device. Specifically, the input unit may include a touch screen (optional) and / or other input devices. Specifically, other input devices may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

[0082] Among them, other input devices can receive input data, etc.

[0083] The display unit can be used to display information input by the user or information provided to the user, various menus of the terminal, an interactive interface, file display, and / or the playback of any multimedia file.

[0084] The memory can be used to store software codes related to the image synthesis method. The processor can execute the steps of the image synthesis method and can also schedule other units (such as the above input unit and display unit) to implement corresponding functions.

[0085] The radio frequency unit (optional) can be used to receive and send information or signals during a call.

[0086] Among them, in the embodiments of the present application, the radio frequency unit can send the background image and the target image to be synthesized to the server 200 and receive the processing result sent by the server 200, such as the target synthesized image.

[0087] It should be understood that the radio frequency unit is optional and can be replaced by other communication interfaces, such as a network port.

[0088] The terminal 100 further includes a power source (such as a battery) for supplying power to each component.

[0089] The terminal 100 further includes an external interface, which can be a standard Micro USB interface or a multi-pin connector. It can be used to connect the terminal 100 to other devices for communication and can also be used to connect a charger to charge the terminal 100.

[0090] Optionally, the server 200 includes a bus, a processor, a communication interface, and a memory. The processor, the memory, and the communication interface communicate with each other through the bus. Among them, the memory can be used to store software codes related to the image synthesis method. The processor can execute the steps of the image synthesis method of the chip and can also schedule other units to implement corresponding functions.

[0091] By studying the method of synthesizing the target object and the background image, the following solution is provided in the initial stage of the research: Adjust the illumination histogram of the target image where the target object is located to match the illumination characteristics of the background image, so as to achieve illumination consistency; at the same time, use color transfer technology to adjust the target image according to the color characteristics of the background image to make the two coordinate in hue and brightness.

[0092] However, this way of histogram matching and color transfer fails to fully consider the complex relationship between the illuminations of the target image and the background image, resulting in insufficient realism and naturalness of the synthesis result.

[0093] To improve the naturalness and realism of the illumination in the synthesis effect, a more in-depth study was conducted on the synthesis method of the target object and the background image. It was found that by leveraging the powerful learning ability of the deep learning model and the advantages of the physically guided architecture Cook-Torrance reflection model in terms of physical authenticity, material expressiveness, computational efficiency, and cross-domain applicability, the deep learning model (such as the Masked Autoencoder (MAE) for relighting tasks) can be combined with the physically guided architecture Cook-Torrance reflection model to simulate the interaction between light and the surface microstructure, and consider spatially varying roughness and reflectivity. After pre-training with a large amount of training data, the architecture combining MAE and the physically guided architecture Cook-Torrance reflection model can effectively enhance the realism and naturalness of the fused target image and background image.

[0094] However, due to the involvement of complex physical models and deep learning architectures, the computational requirements are relatively high, resulting in a slow processing speed and inability to handle real-time application scenarios. At the same time, although the self-supervised learning used in this method reduces the dependence on labeled data, a sufficient amount of high-quality data is still required to train an effective model, and the diversity and representativeness of the data have a significant impact on the model performance. Moreover, during the fine-tuning process, careful adjustment of hyperparameters and architecture design is needed to obtain the best results, which requires high professional knowledge and time investment, and the cost is relatively high.

[0095] To solve the above problems and achieve the goal of synthesizing a target composite image with more natural and realistic illumination at low cost and high efficiency, the embodiments of this application provide an image synthesis method. The image synthesis method of the embodiments of this application will be introduced in detail below with reference to the accompanying drawings.

[0096] Refer to Figure 2 , Figure 2 which is a schematic flowchart of an image synthesis method provided by the embodiments of this application. As Figure 2 shown, the image synthesis method may include:

[0097] Step S201: Perform region-based illumination statistical analysis on the background image to be synthesized to obtain the illumination information representation values of each of the multiple image regions corresponding to the background image.

[0098] Here, the background image refers to the image taken for the background to be synthesized, and the illumination information representation value represents the overall illumination condition of the image region.

[0099] Considering that different regions of the background image may be affected by different illuminations, if a single illumination information representation value is directly used to represent the overall illumination condition of the background image, the illumination information representation value may be inaccurate due to a large difference in the illumination effects on different regions.

[0100] Based on this, in order to more accurately describe the illumination effect on the background image, the background image can be processed by region-based illumination statistical analysis to obtain the illumination information characterization values of each of the multiple image regions corresponding to the background image.

[0101] Here, each image region represents a part of the background image with relatively consistent illumination conditions. Optionally, after partitioning, the multiple image regions satisfy the following conditions: the absolute value of the difference between the gray values of any two pixels in the same image region is less than or equal to a preset gray threshold, and the absolute value of the difference between the gray values of any two pixels in different image regions is greater than the gray threshold.

[0102] For example, assuming that the background image is divided into image region 1 and image region 2, then for any two pixels i and j in image region 1 (or image region 2), there must be , represents the gray value of pixel i, represents the gray value of pixel j, represents the gray threshold; for any pixel i in image region 1 and any pixel k in image region 2, there must be , where represents the gray value of pixel k.

[0103] Since the illumination conditions of the individual pixels within each image region are basically the same, using an illumination information characterization value for each image region can accurately represent the overall illumination conditions of each of the image regions.

[0104] Step S202: Perform de-illumination shadow processing on the target image to be synthesized to obtain a shadow-weakened image corresponding to the target image.

[0105] Among them, the target image refers to an image taken of a target object (such as a person), and the target image and the background image have the same size.

[0106] The above de-illumination shadow processing is used to remove the shadow caused by the light source when taking the target image on the target object.

[0107] It can be understood that the illumination conditions of the target image and the background image are usually inconsistent. If the illumination information characterization value is directly compensated into the target image, although the illumination of the target object and the background image can be unified to a certain extent, the shadow caused by the light source when taking the target image on the target object still exists, resulting in a low naturalness and realism of the image synthesis.

[0108] For example, assume that the background image is a naturally taken image when the background is illuminated by the sun, and the target image is an image taken when the target object is illuminated by a ceiling light above the head. Then, affected by the ceiling light, a shadow will be formed below the jawline of the target object's face. If this shadow is not removed, even after the target object is compensated for the illumination information representation value, the shadow will still exist in the synthesized image. Since after compensation, other regions in the synthesized image except for this shadow are only affected by the sun's illumination, while this shadow is affected not only by the sun's illumination but also by the ceiling light, resulting in low naturalness and realism.

[0109] Based on this, before performing illumination compensation on the target image, the target image can be processed to remove the illumination shadow first to obtain a shadow-weakened image corresponding to the target image.

[0110] Step S203: Perform illumination compensation processing on the shadow-weakened image according to the illumination information representation values of multiple image regions to obtain a compensated image.

[0111] Specifically, for each of the multiple image regions, the corresponding region in the shadow-weakened image can be processed for illumination compensation according to the illumination information representation value of this image region; after performing illumination compensation processing on all regions of the shadow-weakened image, the compensated image can be obtained.

[0112] After the illumination compensation processing, the illumination conditions received by the compensated image can be made basically the same as those received by the background image, thereby greatly improving the naturalness and realism of subsequent image synthesis.

[0113] Step S204: Perform image synthesis processing based on the compensated image and the background image to obtain a target synthesized image composed of the target object and the background image.

[0114] That is to say, in this embodiment, the target object in the compensated image can be synthesized with the background image to obtain the target synthesized image.

[0115] The image synthesis method provided by this application takes into account that if we want the target object in the synthesized image to be more natural and realistic in terms of lighting conditions compared to the background image, we can compensate the lighting information in the background image onto the target image where the target object is located to achieve lighting uniformity between the target object and the background image. Based on this, this application first performs regional lighting statistical analysis on the background image to be synthesized, obtaining the lighting information representation values for each of the multiple image regions corresponding to the background image. Since the lighting conditions of the target image and the background image are inconsistent, if the lighting information representation values are directly compensated onto the target image, although lighting uniformity between the target object and the background image can be achieved to a certain extent, the shadows caused by the light source when shooting the target image on the target object still exist, resulting in a low naturalness and realism in image synthesis. Therefore, this application first performs de-lighting shadow processing on the target image to be synthesized, obtaining a shadow-weakened image corresponding to the target image, and then performs lighting compensation processing on the shadow-weakened image according to the lighting information representation values for each of the multiple image regions, obtaining a compensated image. Finally, based on the compensated image and the background image, image synthesis processing is performed, and thus a target synthesized image with more realistic and natural lighting can be obtained, greatly improving the effect of image synthesis. At the same time, the entire image synthesis process only requires simple image processing and analysis, with low cost and high efficiency.

[0116] In some embodiments of this application, the process of "performing regional lighting statistical analysis on the background image to be synthesized, obtaining the lighting information representation values for each of the multiple image regions corresponding to the background image" in the previous step S201 is introduced.

[0117] In this embodiment, we can first perform image partitioning processing on the background image based on lighting consistency to obtain multiple image regions, and then perform statistical analysis processing on the lighting brightness for each of the multiple image regions respectively to obtain the lighting intensity representation values for each of the multiple image regions, and perform statistical analysis processing on the lighting color for each of the multiple image regions respectively to obtain the lighting color compensation values for each of the multiple image regions. The lighting intensity representation values and the lighting color compensation values for each of the multiple image regions are used as the lighting information representation values for each of the multiple image regions.

[0118] First, the process of "performing image partitioning processing on the background image based on lighting consistency to obtain multiple image regions" is introduced.

[0119] Considering that the image gradient can reflect the change of pixel values, and lighting changes will cause changes in the grayscale values of the image, we can detect the lighting change regions in the image by calculating the image gradient.

[0120] Based on this, optionally, the process of "performing illumination-uniform image partitioning on the background image to obtain multiple image regions" may include: converting the background image into a grayscale image; performing Gaussian filtering on the grayscale image to obtain a filtered image; calculating the gradients of the filtered image in the horizontal and vertical directions to obtain a gradient image corresponding to the filtered image; dividing the grayscale image into multiple initial regions based on the gradient image; using the multiple initial regions as the multiple image regions, or iteratively merging the multiple initial regions based on a preset gray value merging condition to obtain the multiple image regions. Here, Gaussian filtering (Gaussian filter) is used to reduce noise and details in the grayscale image.

[0121] The high gradient amplitude values in the above gradient image can represent different illuminations or regions where the illumination changes in the image, and the low gradient amplitude values can represent regions where the illumination is relatively uniform in the image. The gradient direction can be used to determine the direction of the edge and the direction of the illumination change.

[0122] Optionally, an edge detection operator (such as the Sobel operator) can be used to calculate the gradients of the filtered image in the horizontal and vertical directions to obtain a gradient image, and then the grayscale image can be divided into multiple initial regions based on the gradient image.

[0123] In an optional embodiment, considering that although the edge detection operator can obtain a gradient image, due to the influence of the specific algorithm and implementation method, there may be a problem of edges with a multi-pixel width in the gradient image, which affects the accuracy of subsequent partitioning. Therefore, the gradient image can be further optimized and updated, and then the grayscale image can be partitioned based on the updated gradient image to obtain multiple initial regions.

[0124] Based on this, optionally, the process of "dividing the grayscale image into multiple initial regions based on the gradient image" may include: performing non-maximum suppression processing on the gradient image to obtain a first gradient updated image; calculating a preset quantile of the first gradient updated image as a first threshold; determining a second threshold according to the first threshold, where the second threshold is less than the first threshold; performing extreme value removal processing on the first gradient updated image based on the first threshold and the second threshold to obtain a second gradient updated image; performing erosion and dilation processing on the second gradient updated image to obtain a third gradient updated image; performing region partitioning processing on the grayscale image based on at least one of the first gradient updated image, the second gradient updated image, and the third gradient updated image to obtain multiple initial regions. Here, non-maximum suppression is used to refine the edges to suppress pixels that are not local maxima, thereby retaining slender edges; erosion and dilation processing are used to remove small boundaries and noise.

[0125] The above preset quantiles can be set according to the actual scenario. For example, in some scenarios, the preset quantile can be 95%. That is, in this embodiment, after sorting the gradient amplitudes of all pixels of the first gradient update image in ascending order, the gradient amplitude at the 95% position is used as the first threshold.

[0126] Optionally, the process of "determining the second threshold according to the first threshold" may include: using a preset percentage (such as 20%) of the first threshold as the second threshold.

[0127] The strong edges and weak edges of the grayscale image can be effectively identified through the first threshold and the second threshold.

[0128] Optionally, at least one gradient update image among the first gradient update image, the second gradient update image, and the third gradient update image can be used to perform region division processing on the grayscale image based on the watershed algorithm to obtain multiple initial regions.

[0129] It should be noted that the process of obtaining three gradient update images through the above three-step gradient update is only an example. In addition, there can be other gradient update methods, such as performing only two-step gradient updates or one-step gradient updates among the above three-step gradient updates on the gradient image. The present application does not make specific limitations.

[0130] As introduced above, optionally, multiple initial regions can be directly determined as multiple image regions.

[0131] It can be understood that the gradient image is a pixel-level image. When performing region division on the grayscale image based on the pixel-level gradient image, the initial regions divided are usually relatively fine and numerous. Due to the large number of initial regions, performing statistical analysis processing on the illumination brightness and illumination color of each initial region will consume a lot of time. To improve efficiency, this embodiment can also perform merging processing on multiple initial regions based on preset gray value merging conditions to obtain multiple image regions.

[0132] Optionally, the gray value merging condition can be that the absolute value of the difference in average gray values is less than a preset gray mean threshold. That is, in this embodiment, multiple initial regions can be sorted in ascending order of average gray value first, and then the sorted multiple initial regions can be iteratively merged according to whether the absolute value of the difference in average gray values is less than the preset gray mean threshold to obtain multiple image regions.

[0133] For example, assume that multiple initial regions include initial regions 1 to 5, and the average gray values are 80, 89, 100, 103, and 108 respectively, and the gray mean threshold is 10.

[0134] In the first merge, since the absolute value of the difference between the average gray values of the initial region 1 and the initial region 2 is 9, which is less than the gray mean threshold of 10, it indicates that the lighting conditions of the initial region 1 and the initial region 2 are relatively consistent. Then, the initial region 1 and the initial region 2 are merged.

[0135] Suppose the average gray value of the region after merging the initial region 1 and the initial region 2 (hereinafter referred to as region 1-2) is 88. Then, in the second merge, since the absolute value of the difference between the average gray values of the initial region 3 and the initial region 4 is 3, which is less than the gray mean threshold of 10, it indicates that the lighting conditions of the initial region 3 and the initial region 4 are relatively consistent. Then, the initial region 3 and the initial region 4 are merged.

[0136] Suppose the average gray value of the region after merging the initial region 3 and the initial region 4 (hereinafter referred to as region 3-4) is 101. Then, in the third merge, since the absolute value of the difference between the average gray values of the region 3-4 and the initial region 5 is 7, which is less than the gray mean threshold of 10, it indicates that the lighting conditions of the region 3-4 and the initial region 5 are relatively consistent. Then, the region 3-4 and the initial region 5 are merged.

[0137] Suppose the average gray value of the region after merging the region 3-4 and the initial region 5 (hereinafter referred to as region 3-5) is 104. Then, after the above three iterative merges, the remaining regions include: region 1-2 and region 3-5. Since the absolute value of the difference between the average gray values of these two regions is greater than the gray mean threshold of 10, the merge ends. Then, the multiple image regions include: region 1-2 and region 3-5.

[0138] The following introduces the process of "performing statistical analysis processing on the lighting brightness of each image region in multiple image regions respectively to obtain the lighting intensity characterization values of the multiple image regions".

[0139] It can be understood that the lighting intensity directly determines the light flux reaching the object to be photographed. When the lighting intensity is high, the light reflected or transmitted by the object surface increases, and the overall gray value or color value of the pixels reflected in the image increases, resulting in an increase in the brightness mean. Conversely, when the lighting intensity is low, the brightness mean of the image will decrease. And the lighting intensity will also affect the contrast of the image region to a certain extent. For example, when the lighting intensity is moderate, the image contrast is high, and when the lighting intensity is too strong or too weak, the image contrast is low.

[0140] Based on this, optionally, in this embodiment, the illumination intensity characterization value of the image region can be obtained by statistically analyzing the brightness mean and contrast of the image region. Specifically, for each of the multiple image regions, calculate the brightness mean and contrast of the image region, and perform weighted fusion processing on the brightness mean and contrast of the image region based on the brightness mean weight to obtain the illumination intensity characterization value of the image region; thus obtaining the illumination intensity characterization values of the multiple image regions respectively.

[0141] In a specific implementation, in this embodiment, a region mask can be created for the multiple image regions first, and then the brightness mean and contrast of each image region are calculated after masking each image region in turn.

[0142] Optionally, for each image region, the calcHist method of OpenCV can be used to calculate the histogram of the grayscale image of the image region, so as to obtain the distribution of each brightness value in the image region, and then calculate the brightness mean of the image region.

[0143] Since the brightness standard deviation of the image is positively correlated with the contrast of the image, that is, the larger the brightness standard deviation of the image region, the higher the contrast of the image region; conversely, the smaller the brightness standard deviation of the image region, the lower the contrast of the image region. Based on this, optionally, the contrast of each image region can be characterized by the brightness standard deviation.

[0144] Here, the calculation formula of the brightness standard deviation is as follows:

[0145] Formula (1);

[0146] Where, represents the brightness standard deviation of the th image region, represents the total number of pixels in the th image region, represents the grayscale value of the th pixel in the th image region, represents the brightness mean of the th image region, , , and and are both positive integers, represents the total number of image regions.

[0147] Optionally, the process of "performing weighted fusion processing on the brightness mean and contrast of the image region based on the brightness mean weight to obtain the illumination intensity characterization value of the image region" can refer to the following formula (2).

[0148] Formula (2);

[0149] Wherein, represents the light intensity characterization value of the th image region, represents the brightness mean weight.

[0150] Finally, the process of "performing statistical analysis processing on the light color of each image region among multiple image regions to obtain the light color compensation value of each image region" is introduced.

[0151] In this embodiment, the light color can be quantified as a color temperature value to better describe the light color compensation value of the image region. Based on this, optionally, for each image region among multiple image regions, this embodiment can calculate the R-channel mean, G-channel mean, and B-channel mean of the image region, estimate the color temperature according to the R-channel mean, G-channel mean, and B-channel mean of the image region to obtain the color temperature value of the image region, and perform three different linear normalization processes on the color temperature value of the image region to obtain the R-channel compensation value, G-channel compensation value, and B-channel compensation value of the image region, as the light color compensation value of the image region; so as to obtain the light color compensation value of each image region.

[0152] More specifically, this embodiment can convert each image region to the RGB space, and then calculate the R-channel mean, G-channel mean, and B-channel mean of each image region in the RGB space. Here, the R-channel mean is obtained by taking the mean of the R values of all pixels in each image region, the G-channel mean is obtained by taking the mean of the G values of all pixels in each image region, and the B-channel mean is obtained by taking the mean of the B values of all pixels in each image region.

[0153] In an optional embodiment, the process of "estimating the color temperature according to the R-channel mean, G-channel mean, and B-channel mean of the image region to obtain the color temperature value of the image region" may include: taking the G-channel mean of the image region as a reference, performing proportional scaling on the R-channel mean and B-channel mean of the image region respectively to obtain the first color deviation of the R-channel compared to the G-channel and the second color deviation of the B-channel compared to the G-channel; determining the third color deviation of the B-channel compared to the R-channel under the influence of the G-channel according to the first color deviation and the second color deviation; performing linear adjustment on the third color deviation to obtain the color temperature value of the image region.

[0154] Optionally, the calculation process of the above color temperature value is as shown in the following formula (3).

[0155] Formula (3);

[0156] Wherein, represents the color temperature value of the th image region; , and respectively represent the average value of the R channel, the average value of the G channel, and the average value of the B channel of the th image region; and represent a preset adjustment coefficient for linearly adjusting the third color deviation, represents the first color deviation, represents the second color deviation, represents the third color deviation.

[0157] Optionally, the process of "performing three different linear normalization processes on the color temperature value of the image region to obtain the R-channel compensation value, the G-channel compensation value, and the B-channel compensation value of the image region" may include: preprocessing the color temperature value of the image region so that the color temperature value of the image region is limited within a preset color temperature range , ; performing three different linear normalization processes on the preprocessed color temperature value to obtain the R-channel compensation value, the G-channel compensation value, and the B-channel compensation value of the image region.

[0158] Among them, "preprocessing the color temperature value of the image region" includes: if the color temperature value of the image region is greater than , then set the color temperature value of the image region equal to , if the color temperature value of the image region is less than , then set the color temperature value of the image region equal to .

[0159] Optionally, the process of "performing three different linear normalization processes on the preprocessed color temperature value" may refer to the following formula (4).

[0160] Formula (4);

[0161] Among them, and represent the linear normalization coefficients of the preset channel; represents the th image region's channel compensation value; .

[0162] In this embodiment, the background image is divided into regions based on gradients, and pixels with relatively consistent lighting conditions in the background image can be divided into the same region, achieving accurate division of multiple image regions. Further, considering that the lighting information in the image is divided into two dimensions: lighting brightness and lighting color, this embodiment performs statistical analysis processing on the lighting brightness and lighting color of each image region, and can quickly calculate the lighting information characterization value of each image region through simple formulas in the statistical analysis processing, with higher efficiency and lower cost.

[0163] In some other embodiments of the present application, the process of the previous step S202, "Performing de-lighting shadow processing on the target image to be synthesized to obtain a shadow-weakened image corresponding to the target image", is explained.

[0164] In a possible implementation, the target image can be subjected to histogram equalization processing after grayscale conversion to obtain a first intermediate image, the first intermediate image is converted into a second intermediate image in the HSV space, the shadow area mask detection is performed on the target image according to the V value in the second intermediate image to obtain a shadow mask map, and the brightness of the target image is enhanced according to the shadow mask map to obtain a shadow-weakened image.

[0165] Optionally, the process of "performing histogram equalization processing on the target image after grayscale conversion to obtain a first intermediate image" may include: obtaining the grayscale image of the target image using the cvtColor method of OpenCV, then creating a CLAHE object with limited contrast using the createCLAHE method, and finally obtaining the first intermediate image according to the CLAHE object and the grayscale image.

[0166] When the light source when shooting the target image causes a shadow on the target object, the image brightness value (i.e., the V value in the HSV space) of the shadow area will show a large difference from the image brightness value of the non-shadow area. Therefore, the first intermediate image can be converted into a second intermediate image in the HSV space, and then the shadow area mask detection is performed on the target image according to the V value in the second intermediate image to obtain a shadow mask map.

[0167] Optionally, two shadow detection thresholds, lower_shadow and upper_shadow, can be preset, where lower_shadow < upper_shadow. Then, the shadow mask map is generated through the V value and the two shadow detection thresholds. Specifically, the pixels with the V value in the range of [lower_shadow, upper_shadow] can be marked as shadows, and the pixels in other ranges can be marked as non-shadows to obtain the shadow mask map. For example, see Figure 3 As shown, it is a schematic diagram of a shadow mask map of a portrait provided by the present application. As Figure 3, the shadows mainly appear on the head and the body edges of the portrait.

[0168] It should be understood that due to the influence of factors such as noise, the accuracy of the shadow mask image obtained using the V value is limited. To improve the accuracy of the shadow mask image, optionally, morphological filtering processing can be performed on the shadow mask image first to obtain a shadow mask filtered image, and then the target image can be brightness-enhanced according to the shadow mask filtered image to obtain a shadow-weakened image.

[0169] Here, morphological filtering processing can denoise and soften the edges of the shadow mask image, thereby improving the accuracy of the shadow mask image. Optionally, the morphological filtering processing can be dilation and erosion processing. Of course, the morphological filtering processing can also be other, which is not limited in this application.

[0170] It should be noted that in addition to performing morphological filtering processing on the shadow mask image, other processing can also be performed, such as using the region filling technique to fill the small holes in the shadow mask filtered image to further improve the accuracy of the shadow mask image.

[0171] This embodiment can remove or weaken the shadows caused by the light source when shooting the target image, avoiding the problem of low realism in image synthesis caused by shadows and improving the image synthesis effect.

[0172] As introduced above, the light information characterization value of the image region can include the light intensity characterization value and the light color compensation value. Based on this, the process of the above step S203 "performing light compensation processing on the shadow-weakened image according to the light information characterization values of the respective image regions to obtain a compensated image" can include: performing light compensation processing on the shadow-weakened image according to the light intensity characterization values and the light color compensation values of the respective image regions to obtain a compensated image.

[0173] Optionally, the process of "performing light compensation processing on the shadow-weakened image according to the light intensity characterization values and the light color compensation values of the respective image regions to obtain a compensated image" can include: performing light intensity compensation on the corresponding V value of the shadow-weakened image converted to the HSV space according to the light intensity characterization values of the respective image regions to obtain a third intermediate image in the HSV space; performing light color compensation on the corresponding R value, G value, and B value of the third intermediate image converted to the RGB space according to the light color compensation values of the respective image regions to obtain an image in the RGB space as the compensated image.

[0174] Optionally, the above light intensity compensation and light color compensation can be combined with the respective region masks obtained above to improve the accuracy.

[0175] Optionally, the process of "performing illumination intensity compensation on the corresponding V value of the shadow-weakened image converted to the HSV space according to the illumination intensity characterization values of the respective multiple image regions to obtain a third intermediate image in the HSV space" can use the following formula (5).

[0176] Formula (5);

[0177] Wherein, represents the brightness value of the pixels within the region corresponding to the th image region in the third intermediate image, that is, the brightness value after illumination intensity compensation; represents the brightness value of the pixels within the region corresponding to the th image region after the shadow-weakened image is converted to the HSV space (taking the th image region as an example, assuming that the corresponding region after the shadow-weakened image is converted to the HSV space is region , then represents the brightness value of the pixels in region ); and represent preset illumination intensity compensation coefficients.

[0178] Optionally, the process of "performing illumination color compensation on the corresponding R value, G value, and B value of the third intermediate image converted to the RGB space according to the respective illumination color compensation values of the multiple image regions" may include: for each image region, multiplying the R-channel compensation value of the image region by the R value of the corresponding region after the third intermediate image is converted to the RGB space, multiplying the G-channel compensation value of the image region by the G value of the corresponding region after the third intermediate image is converted to the RGB space, and multiplying the B-channel compensation value of the image region by the B value of the corresponding region after the third intermediate image is converted to the RGB space, to obtain an image in the RGB space, that is, the compensated image.

[0179] In this embodiment, by compensating the illumination color and illumination intensity of the target image, the illumination condition of the target object in the compensated image can be made substantially consistent with that of the background image, improving the naturalness and realism of the image synthesis, and thus improving the image synthesis effect.

[0180] Considering that when dividing the background image into regions, the illumination information characterization values of different image regions are different, then, according to different illumination information characterization values, the degrees of illumination compensation for different regions of the target image are different, which may cause the joints between different image regions to be unnatural. At the same time, when the target object in the compensated image is synthesized into the background image, the joints between the target object and the background image may also be unnatural, resulting in a low image synthesis effect.

[0181] Optionally, in order to improve the image synthesis effect, the process of step S204, "performing image synthesis processing on the compensated image and the background image to obtain a target synthesis image composed of the target object and the background image", may include: determining the object area where the target object is located in the compensated image, synthesizing the object area into the background image to obtain an initial synthesis image; performing Gaussian filtering processing on the target area in the initial synthesis image to obtain the target synthesis image, where the target area includes the connection area between the object area and the background image and / or the connection area between different image areas within the object area.

[0182] By performing Gaussian filtering processing on the connection area between the object area and the background image and the connection area between different image areas within the object area, the difference at the edges can be effectively reduced, the integrity of the target synthesis image can be improved, and thus the image synthesis effect can be enhanced.

[0183] The above introduced an image synthesis method provided by an embodiment of the present application. Next, an apparatus for executing the above image synthesis method will be introduced.

[0184] Please refer to Figure 4 , Figure 4 which is a schematic structural diagram of an image synthesis apparatus provided by an embodiment of the present application. As Figure 4 shown, the image synthesis apparatus may include:

[0185] A lighting analysis module 401, configured to perform sub-region lighting statistical analysis processing on the background image to be synthesized, and obtain the lighting information representation values of the respective multiple image regions corresponding to the background image, where the absolute value of the difference between the gray values of any two pixels in the same image region is less than or equal to a preset gray threshold, and the absolute value of the difference between the gray values of any two pixels in different image regions is greater than the gray threshold;

[0186] A de-lighting shadow module 402, configured to perform de-lighting shadow processing on the target image to be synthesized, and obtain a shadow-weakened image corresponding to the target image, where the target image refers to an image taken of the target object, and the de-lighting shadow processing is used to remove the shadow caused by the light source when taking the target image on the target object;

[0187] A lighting compensation module 403, configured to perform lighting compensation processing on the shadow-weakened image according to the lighting information representation values of the respective multiple image regions, and obtain a compensated image;

[0188] An image synthesis module 404, configured to perform image synthesis processing on the compensated image and the background image, and obtain a target synthesis image composed of the target object and the background image.

[0189] In a possible implementation, when the above-mentioned light analysis module performs regional light statistical analysis processing on the background image to be synthesized and obtains the light information characterization values of each of the multiple image regions corresponding to the background image, it can specifically be used for:

[0190] Perform image partitioning processing on the background image based on light consistency to obtain multiple image regions;

[0191] Perform statistical analysis processing on the light brightness of each of the multiple image regions respectively to obtain the light intensity characterization values of each of the multiple image regions;

[0192] Perform statistical analysis processing on the light color of each of the multiple image regions respectively to obtain the light color compensation values of each of the multiple image regions;

[0193] Use the light intensity characterization values and light color compensation values of each of the multiple image regions as the light information characterization values of each of the multiple image regions.

[0194] Correspondingly, when the above-mentioned light compensation module performs light compensation processing on the shadow-weakened image according to the light information characterization values of each of the multiple image regions to obtain the compensated image, it can specifically be used for:

[0195] Perform light compensation processing on the shadow-weakened image according to the light intensity characterization values and light color compensation values of each of the multiple image regions to obtain the compensated image.

[0196] In a possible implementation, when the above-mentioned light analysis module performs image partitioning processing on the background image based on light consistency to obtain multiple image regions, it can specifically be used for:

[0197] Convert the background image into a grayscale image;

[0198] Perform Gaussian filtering on the grayscale image to obtain a filtered image;

[0199] Calculate the gradients of the filtered image in the horizontal and vertical directions to obtain the gradient image corresponding to the filtered image;

[0200] Divide the grayscale image into multiple initial regions based on the gradient image;

[0201] Perform merging processing on the multiple initial regions based on the preset gray value merging conditions to obtain multiple image regions.

[0202] In a possible implementation, when the above-mentioned light analysis module divides the grayscale image into multiple initial regions based on the gradient image, it can specifically be used for:

[0203] Perform non-maximum suppression processing on the gradient image to obtain a first gradient updated image;

[0204] Calculate the preset quantile of the first gradient updated image as the first threshold value;

[0205] Determine a second threshold value according to the first threshold value, where the second threshold value is less than the first threshold value;

[0206] Perform extreme value removal processing on the first gradient updated image based on the first threshold value and the second threshold value to obtain a second gradient updated image;

[0207] Perform erosion and dilation processing on the second gradient updated image to obtain a third gradient updated image;

[0208] Perform region division processing on the grayscale image based on at least one of the first gradient updated image, the second gradient updated image, and the third gradient updated image to obtain a plurality of initial regions.

[0209] In a possible implementation, when the above-mentioned illumination analysis module performs statistical analysis processing on the illumination brightness of each of the multiple image regions to obtain the illumination intensity characterization values of the multiple image regions respectively, it can specifically be used for:

[0210] For each of the multiple image regions, calculate the brightness mean value and contrast of the image region, and perform weighted fusion processing on the brightness mean value and contrast of the image region based on the brightness mean value weight to obtain the illumination intensity characterization value of the image region; so as to obtain the illumination intensity characterization values of the multiple image regions respectively.

[0211] In a possible implementation, when the above-mentioned illumination analysis module performs statistical analysis processing on the illumination color of each of the multiple image regions to obtain the illumination color compensation values of the multiple image regions respectively, it can specifically be used for:

[0212] For each of the multiple image regions, calculate the R-channel mean value, G-channel mean value, and B-channel mean value of the image region, estimate the color temperature according to the R-channel mean value, G-channel mean value, and B-channel mean value of the image region to obtain the color temperature value of the image region, and perform three different linear normalization processes on the color temperature value of the image region to obtain the R-channel compensation value, G-channel compensation value, and B-channel compensation value of the image region as the illumination color compensation value of the image region; so as to obtain the illumination color compensation values of the multiple image regions respectively.

[0213] In a possible implementation, when the above-mentioned illumination analysis module estimates the color temperature according to the R-channel mean value, G-channel mean value, and B-channel mean value of the image region to obtain the color temperature value of the image region, it can specifically be used for:

[0214] Based on the average value of the G channel of the image region, the average values of the R channel and the B channel of the image region are respectively scaled proportionally to obtain a first color deviation of the R channel compared to the G channel and a second color deviation of the B channel compared to the G channel;

[0215] Based on the first color deviation and the second color deviation, determine a third color deviation of the B channel compared to the R channel under the influence of the G channel;

[0216] Linearly adjust the third color deviation to obtain the color temperature value of the image region.

[0217] In a possible implementation, when the above-mentioned de-lighting and shadow module performs de-lighting and shadow processing on the target image to be synthesized to obtain a shadow-weakened image corresponding to the target image, it can specifically be used for:

[0218] Perform histogram equalization processing on the grayscale image of the target image to obtain a first intermediate image;

[0219] Convert the first intermediate image into a second intermediate image in the HSV space;

[0220] Detect the shadow area mask of the target image according to the V value in the second intermediate image to obtain a shadow mask map;

[0221] Enhance the brightness of the target image according to the shadow mask map to obtain a shadow-weakened image.

[0222] In a possible implementation, when the above-mentioned de-lighting and shadow module enhances the brightness of the target image according to the shadow mask map to obtain a shadow-weakened image, it can specifically be used for:

[0223] Perform morphological filtering on the shadow mask map to obtain a shadow mask filtered map;

[0224] Enhance the brightness of the target image according to the shadow mask filtered map to obtain a shadow-weakened image.

[0225] In a possible implementation, when the above-mentioned light compensation module performs light compensation processing on the shadow-weakened image according to the light intensity characterization values and light color compensation values of multiple image regions to obtain a compensated image, it can specifically be used for:

[0226] According to the light intensity characterization values of multiple image regions, perform light intensity compensation on the corresponding V values obtained by converting the shadow-weakened image into the HSV space to obtain a third intermediate image in the HSV space;

[0227] According to the light color compensation values of multiple image regions, perform light color compensation on the corresponding R value, G value, and B value obtained by converting the third intermediate image into the RGB space to obtain an image in the RGB space as the compensated image.

[0228] In a possible implementation, when the above-mentioned image synthesis module performs image synthesis processing based on the compensated image and the background image to obtain a target synthesis image composed of the target object and the background image, it can specifically be used for:

[0229] Determine the object area where the target object is located in the compensated image, and synthesize the object area into the background image to obtain an initial synthesis image;

[0230] Perform Gaussian filtering on the target area in the initial synthesis image to obtain the target synthesis image, where the target area includes the connection area between the object area and the background image and / or the connection area between different image areas within the object area.

[0231] In an embodiment of the present application, an electronic device is further provided. The electronic device may include at least one processor and a memory connected to the processor, where:

[0232] The memory is used to store computer programs;

[0233] The processor is used to execute the computer program so that the electronic device can implement any one of the image synthesis methods provided in the embodiments of the present application.

[0234] Reference Figure 5 As shown, it shows a schematic structural diagram of an electronic device suitable for implementing the electronic device in the embodiments of the present application. The electronic device in the embodiments of the present application may include, but is not limited to, fixed terminals such as mobile phones, laptop computers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), desktop computers, and so on. Figure 5 The electronic device shown is only an example and should not bring any limitations to the functions and usage scopes of the embodiments of the present application.

[0235] As Figure 5 shown, the electronic device may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage device 508 into the random access memory (RAM) 503. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 503. The processing device 501, the ROM 502, and the RAM 503 are connected to each other through the bus 504. The input / output (I / O) interface 505 is also connected to the bus 504.

[0236] Typically, the following devices can be connected to the I / O interface 505: input devices 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 508 including, for example, a memory card, a hard disk, etc.; and a communication device 509. The communication device 509 can allow the electronic device to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 5 an electronic device with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices can be implemented or had.

[0237] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions run on an electronic device, the electronic device is enabled to implement any one of the image synthesis methods provided by the embodiments of the present application.

[0238] An embodiment of the present application also provides a computer-readable storage medium. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can be enabled to implement any one of the image synthesis methods provided by the embodiments of the present application.

[0239] In addition, it should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the attached drawings of the device embodiments provided by the present application, the connection relationship between modules indicates that they have a communication connection, which can be specifically implemented as one or more communication buses or signal lines.

[0240] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware. Of course, it can also be implemented by dedicated hardware including application-specific integrated circuits, dedicated CPUs, dedicated memories, dedicated components, etc. Generally, functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structures for implementing the same function can also be diverse, such as analog circuits, digital circuits, or dedicated circuits, etc. However, for the present application, in more cases, software program implementation is a better embodiment. Based on such an understanding, the technical solution of the present application, in essence, or the part that makes a contribution to the prior art, can be embodied in the form of a software product. The computer software product is stored in a readable storage medium, such as a floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disc of a computer, etc., and includes several instructions for causing a computer device (which can be a personal computer, training device, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0241] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product.

[0242] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a dedicated computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, training device, or data center to another website, computer, training device, or data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or a wireless manner (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can store, or a data storage device such as a training device or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

Claims

1. An image synthesis method, characterized in that, Including: Performing regional illumination statistical analysis processing on the background image to be synthesized, obtaining the illumination information characterization values of multiple image regions corresponding to the background image, wherein, for any two pixels in the same image region, the absolute value of the difference between their gray values is less than or equal to a preset gray threshold, and for any two pixels in different image regions, the absolute value of the difference between their gray values is greater than the gray threshold; Performing de-illumination shadow processing on the target image to be synthesized, obtaining the shadow-weakened image corresponding to the target image, wherein the target image refers to an image taken of a target object, and the de-illumination shadow processing is used to remove the shadow caused by the light source when taking the target image from the target object; Performing illumination compensation processing on the shadow-weakened image according to the illumination information characterization values of the multiple image regions, obtaining the compensated image; Performing image synthesis processing based on the compensated image and the background image, obtaining the target synthesized image composed of the target object and the background image.

2. The image synthesis method according to claim 1, characterized in that, The performing regional illumination statistical analysis processing on the background image to be synthesized, obtaining the illumination information characterization values of multiple image regions corresponding to the background image, includes: Performing image partitioning processing based on illumination consistency on the background image, obtaining the multiple image regions; Performing statistical analysis processing on the illumination brightness of each of the multiple image regions respectively, obtaining the illumination intensity characterization values of the multiple image regions; Performing statistical analysis processing on the illumination color of each of the multiple image regions respectively, obtaining the illumination color compensation values of the multiple image regions; Taking the illumination intensity characterization values and illumination color compensation values of the multiple image regions respectively as the illumination information characterization values of the multiple image regions; The performing illumination compensation processing on the shadow-weakened image according to the illumination information characterization values of the multiple image regions, obtaining the compensated image, includes: Performing illumination compensation processing on the shadow-weakened image according to the illumination intensity characterization values and illumination color compensation values of the multiple image regions, obtaining the compensated image.

3. The image synthesis method according to claim 2, characterized in that The performing image partitioning processing based on illumination consistency on the background image, obtaining the multiple image regions, includes: Converting the background image into a grayscale image; Performing Gaussian filtering on the grayscale image, obtaining the filtered image; Calculating the gradients of the filtered image in the horizontal and vertical directions, obtaining the gradient image corresponding to the filtered image; Dividing the grayscale image into multiple initial regions based on the gradient image; Performing merging processing on the multiple initial regions based on a preset gray value merging condition, obtaining the multiple image regions.

4. The image synthesis method according to claim 3, wherein The dividing the grayscale image into multiple initial regions based on the gradient image, includes: Performing non-maximum suppression processing on the gradient image, obtaining the first gradient updated image; Calculating the preset quantile of the first gradient updated image, taking it as the first threshold; Determining a second threshold according to the first threshold, the second threshold being less than the first threshold; Perform extreme value removal processing on the first gradient updated image based on the first threshold and the second threshold to obtain a second gradient updated image; Perform erosion and dilation processing on the second gradient updated image to obtain a third gradient updated image; Perform region division processing on the grayscale image based on at least one of the first gradient updated image, the second gradient updated image, and the third gradient updated image to obtain the multiple initial regions.

5. The image synthesis method according to claim 2, characterized in that The statistical analysis processing of the illumination brightness for each of the multiple image regions respectively to obtain the illumination intensity characterization values of the multiple image regions includes: For each of the multiple image regions, calculate the brightness mean and contrast of the image region, and perform weighted fusion processing based on the brightness mean weight on the brightness mean and contrast of the image region to obtain the illumination intensity characterization value of the image region; so as to obtain the illumination intensity characterization values of the multiple image regions respectively.

6. The image synthesis method according to claim 2, wherein The statistical analysis processing of the illumination color for each of the multiple image regions respectively to obtain the illumination color compensation values of the multiple image regions includes: For each of the multiple image regions, calculate the R-channel mean, G-channel mean, and B-channel mean of the image region, perform color temperature estimation based on the R-channel mean, G-channel mean, and B-channel mean of the image region to obtain the color temperature value of the image region, and perform three different linear normalization processes on the color temperature value of the image region to obtain the R-channel compensation value, G-channel compensation value, and B-channel compensation value of the image region, as the illumination color compensation value of the image region; so as to obtain the illumination color compensation values of the multiple image regions respectively.

7. The image synthesis method according to claim 6, characterized in that The performing color temperature estimation based on the R-channel mean, G-channel mean, and B-channel mean of the image region to obtain the color temperature value of the image region includes: Taking the G-channel mean of the image region as a reference, perform proportional scaling on the R-channel mean and B-channel mean of the image region respectively to obtain the first color deviation of the R-channel compared to the G-channel and the second color deviation of the B-channel compared to the G-channel; Determine the third color deviation of the B-channel compared to the R-channel under the influence of the G-channel according to the first color deviation and the second color deviation; Perform linear adjustment on the third color deviation to obtain the color temperature value of the image region.

8. The image synthesis method according to claim 1, wherein The performing de-illumination shadow processing on the target image to be synthesized to obtain the shadow-weakened image corresponding to the target image includes: Perform histogram equalization processing on the grayscale image of the target image to obtain a first intermediate image; Convert the first intermediate image to a second intermediate image in the HSV color space; Perform shadow region mask detection on the target image according to the V value in the second intermediate image to obtain a shadow mask image; Perform brightness enhancement on the target image according to the shadow mask image to obtain the shadow-weakened image.

9. The image synthesis method according to claim 8, wherein The performing brightness enhancement on the target image according to the shadow mask image to obtain the shadow-weakened image includes: Perform morphological filtering processing on the shadow mask image to obtain a shadow mask filtered image; Enhance the brightness of the target image according to the shadow mask filtering image to obtain the shadow-weakened image.

10. The image synthesis method according to claim 2, wherein The performing illumination compensation processing on the shadow-weakened image according to the illumination intensity characterization values and illumination color compensation values of the respective multiple image regions to obtain the compensated image includes: Performing illumination intensity compensation on the corresponding V value obtained by converting the shadow-weakened image into the HSV space according to the illumination intensity characterization values of the respective multiple image regions to obtain a third intermediate image in the HSV space; Performing illumination color compensation on the corresponding R value, G value, and B value obtained by converting the third intermediate image into the RGB space according to the illumination color compensation values of the respective multiple image regions to obtain an image in the RGB space as the compensated image.

11. The image synthesis method according to claim 1, wherein The performing image composition processing based on the compensated image and the background image to obtain a target composite image composed of the target object and the background image includes: Determine the object region where the target object is located in the compensated image, and composite the object region into the background image to obtain an initial composite image; Perform Gaussian filtering processing on the target region in the initial composite image to obtain the target composite image, where the target region includes the connection region between the object region and the background image and / or the connection region between different image regions within the object region.

12. An image synthesis device, characterized in that, including: An illumination analysis module, a de-illumination shadow module, an illumination compensation module, and an image composition module; The illumination analysis module is configured to perform sub-region illumination statistical analysis processing on the background image to be composed, and obtain the illumination information characterization values of the respective multiple image regions corresponding to the background image, where the absolute value of the difference between the gray values of any two pixels in the same image region is less than or equal to a preset gray threshold, and the absolute value of the difference between the gray values of any two pixels in different image regions is greater than the gray threshold; The de-illumination shadow module is configured to perform de-illumination shadow processing on the target image to be composed to obtain the shadow-weakened image corresponding to the target image, where the target image refers to an image taken of a target object, and the de-illumination shadow processing is used to remove the shadow caused by the light source when the target image is taken from the target object; The illumination compensation module is configured to perform illumination compensation processing on the shadow-weakened image according to the illumination information characterization values of the respective multiple image regions to obtain a compensated image; The image composition module is configured to perform image composition processing based on the compensated image and the background image to obtain a target composite image composed of the target object and the background image.

13. A computer program product, characterized in that, including computer-readable instructions, which, when running on an electronic device, cause the electronic device to implement the image composition method according to any one of claims 1 to 11.

14. An electronic device, characterized in that, including at least one processor and a memory connected to the processor, where: The memory is used to store a computer program; The processor is configured to execute the computer program so that the electronic device can implement the image composition method according to any one of claims 1 to 11.

15. A computer storage medium, characterized in that, The storage medium stores one or more computer programs, which, when executed by an electronic device, enable the electronic device to implement the image synthesis method according to any one of claims 1 to 11.

Images