Light and shadow optimization method and device in commodity graph synthesis, equipment and medium

Abstract

The invention provides a light and shadow optimization method and device in commodity graph synthesis, equipment and a medium, and the method comprises the steps: carrying out the enhancement of a back-removed graph, and obtaining an enhanced graph; darkening the background image to obtain a background darkened image; synthesizing the enhanced image and the background darkened image, and extracting to obtain first patent information; synthesizing the back-removed image and the background image, and extracting to obtain first setting information; sending the first plant information and the first setting information to a diffusion model, extracting to obtain second plant information, sending the second plant information and the first setting information to the diffusion model, and generating a second light and shadow guidance graph; the texture information of the first light and shadow guidance graph is migrated to the second light and shadow guidance graph to obtain the intermediate graph, and the color information of the background graph is migrated to the intermediate graph to obtain the final target graph, so that the commodity synthesis efficiency can be greatly improved, and the problem of insufficient light and shadow easily occurring in a traditional light and shadow synthesis algorithm can be effectively solved.

Description

Technical Field

[0001] The present invention relates to a method, apparatus, device and medium for optimizing light and shadow in product image synthesis. Background Art

[0002] The current process of synthesizing products and backgrounds mainly relies on manual operations. The specific steps include separately photographing products, graphic designers performing matte extraction, selecting backgrounds for synthesis, and using professional software such as Photoshop to optimize the light and shadow effects. However, this method has several deficiencies:

[0003] 1. High manual dependence: The entire process requires a large amount of manual intervention, especially in the matte extraction and light and shadow processing stages, which places high requirements on the technical proficiency and light and shadow balance sense of graphic designers.

[0004] 2. Limited batch processing ability: Due to mainly relying on manual operations, this method is difficult to handle the processing requirements of a large number of products, restricting production efficiency.

[0005] 3. Low efficiency: The efficiency of manual operations is relatively low, which not only prolongs the working cycle but also affects the overall work efficiency.

[0006] 4. Increased costs: Since more human resources need to be invested and high skills are required for graphic designers, this directly leads to an increase in the operating costs of enterprises.

[0007] In summary, the traditional method of synthesizing products and backgrounds has obvious limitations in terms of efficiency and cost control. There is an urgent need to achieve automated and intelligent upgrades through technological innovation to improve efficiency, reduce costs, and release human resources. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to provide a method, apparatus, device and medium for optimizing light and shadow in product image synthesis, which can greatly improve the efficiency of product synthesis and effectively solve the problem of insufficient light and shadow that easily occurs in traditional light and shadow synthesis algorithms.

[0009] In a first aspect, the present invention provides a method for optimizing light and shadow in product image synthesis, including the following steps:

[0010] Step 1: Obtain a background-removed image of only the product and a background image;

[0011] Step 2: Enhance the background-removed image by increasing the brightness of the product main body in the background-removed image to a first set value to obtain an enhanced image; map the background image to the HSV space, reduce the V space in the HSV space to a second set value, and then map it back to the RGB space to obtain a background-darkened image;

[0012] Step 3: Synthesize the enhanced image and the darkened background image to obtain the first synthesized image, and extract the first latent information from the first synthesized image through VAE decoding; Synthesize the background-removed image and the background image to obtain the second synthesized image, and extract the first set information from the second synthesized image using the ControlNet model;

[0013] Step 4: Send the first latent information and the first set information to the diffusion model to generate the first light and shadow guidance image; Extract the second latent information from the first light and shadow guidance image, and send the second latent information and the first set information to the diffusion model to generate the second light and shadow guidance image;

[0014] Step 5: Transfer the texture information of the first light and shadow guidance image to the second light and shadow guidance image through the high-contrast retention algorithm to obtain the intermediate image, and then use the welsh algorithm to transfer the color information of the background image to the intermediate image to obtain the final target RGB image.

[0015] In a second aspect, the present invention provides a device for optimizing light and shadow in product image synthesis, including:

[0016] An image acquisition module for acquiring a background-removed image of only the product and the background image;

[0017] An enhancement and darkening module for enhancing the background-removed image, increasing the brightness of the product main body in the background-removed image to a first set value to obtain an enhanced image; Mapping the background image to the HSV space, reducing the V space in the HSV space to a second set value, and then mapping it back to the RGB space to obtain a darkened background image;

[0018] An information extraction module for synthesizing the enhanced image and the darkened background image to obtain the first synthesized image, and extracting the first latent information from the first synthesized image through VAE decoding; Synthesizing the background-removed image and the background image to obtain the second synthesized image, and extracting the first set information from the second synthesized image using the ControlNet model;

[0019] A guidance generation module for sending the first latent information and the first set information to the diffusion model to generate the first light and shadow guidance image; Extracting the second latent information from the first light and shadow guidance image, and sending the second latent information and the first set information to the diffusion model to generate the second light and shadow guidance image;

[0020] A light and shadow optimization module for transferring the texture information of the first light and shadow guidance image to the second light and shadow guidance image through the high-contrast retention algorithm to obtain the intermediate image, and then using the welsh algorithm to transfer the color information of the background image to the intermediate image to obtain the final target RGB image.

[0021] In a third aspect, the present invention provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the method described in the first aspect is implemented.

[0022] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the method described in the first aspect is implemented.

[0023] One or more technical solutions provided by the present invention have at least the following technical effects or advantages:

[0024] The present invention can greatly improve the efficiency of commodity synthesis, can effectively solve the problem of insufficient light and shadow that easily occurs in traditional light and shadow synthesis algorithms, and can effectively improve the work efficiency of art designers.

[0025] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention are specifically given below. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The present invention will be further described below with reference to the accompanying drawings in conjunction with embodiments.

[0027] Figure 1 is a flowchart of the method in Embodiment 1 of the present invention;

[0028] Figure 2 is a structural schematic diagram of the device in Embodiment 2 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0029] Embodiment 1

[0030] As Figure 1 shown, this embodiment provides a method for optimizing light and shadow in commodity image synthesis, including the following steps:

[0031] Step 1: Obtain a background-removed image of only the commodity and a background image; the background-removed image is an image with only the commodity after removing the background;

[0032] Step 2: Enhance the background-removed image, increase the brightness of the commodity main body in the background-removed image to a first set value to obtain an enhanced image; map the background image to the HSV space, reduce the V space in the HSV space to a second set value, and then map it back to the RGB space to obtain a background-darkened image; make the brightness ratio of the enhanced image to the background-darkened image be 2:1 to 3:1;

[0033] Step 3: Synthesize the enhanced image and the background-darkened image to obtain the first synthesized image, and extract the first latent information from the first synthesized image through VAE decoding; Synthesize the background-removed image and the background image to obtain the second synthesized image, and use the ControlNet model to extract the first set information from the second synthesized image to control the generation of the image result; The synthesis process can be manually operated using software such as photoshop, and the first latent information is extracted and used as the starting point when the diffusion model generates.

[0034] Step 4: Send the first latent information and the first set information to the diffusion model to generate the first light and shadow guidance map; Extract the second latent information from the first light and shadow guidance map, and send the second latent information and the first set information to the diffusion model to generate the second light and shadow guidance map;

[0035] Step 5: Transfer the texture information of the first light and shadow guidance map to the second light and shadow guidance map through the high-contrast retention algorithm to obtain the intermediate map, and then use the welsh algorithm to transfer the color information of the background image to the intermediate map to obtain the final target RGB image. The high-contrast retention algorithm is used to retain the high and low frequency details of the picture, that is, the texture information.

[0036] In this embodiment, preferably, step 1 is specifically: Obtain the product image and the background image; Use the matte tool to matte the product image to obtain the background-removed image.

[0037] In this embodiment, preferably, step 2 is specifically: Enhance the background-removed image through Gamma transformation, increase the brightness of the product main body in the background-removed image by a first set value to obtain the enhanced image; Map the background image to the HSV space, reduce the V space in the HSV space by a second set value, and then map it back to the RGB space to obtain the background-darkened image.

[0038] In this embodiment, preferably, the diffusion model is Diffusion Models.

[0039] Based on the same inventive concept, the present application also provides an apparatus corresponding to the method in Embodiment 1, as detailed in Embodiment 2.

[0040] Embodiment 2

[0041] As Figure 2 shown, in this embodiment, an apparatus for optimizing light and shadow in product image synthesis is provided, including:

[0042] An image acquisition module, which acquires the background-removed image of only the product and the background image;

[0043] Enhanced darkening module: Enhance the background-removed image, increase the brightness of the commodity main body in the background-removed image to the first set value to obtain an enhanced image; map the background image to the HSV space, reduce the V space in the HSV space to the second set value, and then map it back to the RGB space to obtain a background-darkened image; make the brightness ratio of the enhanced image to the background-darkened image be 2:1 to 3:1;

[0044] Extraction and obtaining information module: Synthesize the enhanced image and the background-darkened image to obtain a first synthesized image, and extract the first latent information from the first synthesized image through VAE decoding; synthesize the background-removed image and the background image to obtain a second synthesized image, and use the ControlNet model to extract the first set information from the second synthesized image to control the generation of the image result; the synthesis process can be manually operated using software such as Photoshop, and the first latent information is extracted and used as the starting point when the diffusion model generates;

[0045] Generation guidance module: Send the first latent information and the first set information to the diffusion model to generate a first light and shadow guidance image; extract the second latent information from the first light and shadow guidance image, and send the second latent information and the first set information to the diffusion model to generate a second light and shadow guidance image;

[0046] Optimization of light and shadow module: Transfer the texture information of the first light and shadow guidance image to the second light and shadow guidance image through the high-contrast retention algorithm to obtain an intermediate image, and then use the welsh algorithm to transfer the color information of the background image to the intermediate image to obtain the final target RGB image. The high-contrast retention algorithm is used to retain the high and low frequency details of the picture, that is, the texture information.

[0047] In this embodiment, preferably, the image acquisition module is specifically: acquire a commodity image and a background image; perform matte extraction on the commodity image using a matte extraction tool to obtain a background-removed image.

[0048] In this embodiment, preferably, the enhanced darkening module is specifically: enhance the background-removed image through Gamma transformation, increase the brightness of the commodity main body in the background-removed image by the first set value to obtain an enhanced image; map the background image to the HSV space, reduce the V space in the HSV space by the second set value, and then map it back to the RGB space to obtain a background-darkened image.

[0049] In this embodiment, preferably, the diffusion model is Diffusion Models.

[0050] Since the device described in the second embodiment of the present invention is the device adopted for implementing the method of the first embodiment of the present invention, based on the method described in the first embodiment of the present invention, those skilled in the art can understand the specific structure and variations of the device, and thus will not be elaborated herein. Any device adopted for the method of the first embodiment of the present invention falls within the scope of protection of the present invention.

[0051] Based on the same inventive concept, this application provides an electronic device embodiment corresponding to the first embodiment. For details, see the third embodiment.

[0052] Embodiment Three

[0053] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, any implementation manner in the first embodiment can be realized.

[0054] Since the electronic device described in this embodiment is the device adopted for implementing the method in the first embodiment of this application, based on the method described in the first embodiment of this application, those skilled in the art can understand the specific implementation manners and various variations of the electronic device in this embodiment. Therefore, the implementation of how this electronic device realizes the method in the embodiments of this application will not be described in detail herein. Any device adopted by those skilled in the art for implementing the method in the embodiments of this application falls within the scope of protection of this application.

[0055] Based on the same inventive concept, this application provides a storage medium corresponding to the first embodiment. For details, see the fourth embodiment.

[0056] Embodiment Four

[0057] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, any implementation manner in the first embodiment can be realized.

[0058] The technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0059] This embodiment can greatly improve the efficiency of commodity synthesis, and can effectively solve the problem of insufficient light and shadow that is prone to occur in traditional light and shadow synthesis algorithms. It can perform the efficiency of pictures in large quantities, improving work efficiency. It reduces the human resources invested and reduces the skill requirements for graphic designers, which directly reduces the operating costs of enterprises.

[0060] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.

[0061] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for realizing the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0062] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing devices to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that realizes the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0063] These computer program instructions can also be loaded onto a computer or other programmable data processing devices, such that a series of operation steps are executed on the computer or other programmable devices to generate a computer-implemented process, so that the instructions executed on the computer or other programmable devices provide steps for realizing the functions specified in one Figure 1 flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0064] Although the specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments we described are illustrative rather than limiting the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered by the scope of the claims of the present invention.

Claims

1. A method for optimizing light and shadow in product image synthesis, characterized by: The steps include: Step 1: Get the background image and background image of the product only; Step 2: Enhance the background-removed image, increase the brightness of the main body of the product in the background-removed image to a first set value, and obtain an enhanced image; map the background image to the HSV space, reduce the V space in the HSV space to a second set value, and then map it back to the RGB space to obtain a background darkened image; Step 3, synthesize the enhanced image and the background darkened image to obtain a first synthesized image, and extract the first latent information from the first synthesized image through VAE decoding; The background-removed image and the background image are synthesized to obtain a second synthesized image, and the first setting information is extracted from the second synthesized image using a ControlNet model; Step 4: Send the first latent information and the first setting information to the diffusion model to generate a first light and shadow guidance map; extract the second latent information from the first light and shadow guidance map, and send the second latent information and the first setting information to the diffusion model to generate a second light and shadow guidance map; Step 5: Use the high contrast retention algorithm to transfer the texture information of the first light and shadow guidance map to the second light and shadow guidance map to obtain an intermediate map. Then use the Welsh algorithm to transfer the color information of the background map to the intermediate map to obtain the final target RGB image.

2. The method for optimizing light and shadow in product image synthesis according to claim 1, characterized in that: The step 1 specifically includes: obtaining a product image and a background image; and cutting out the product image using a cutting out tool to obtain a background-removed image.

3. The method for optimizing light and shadow in product image synthesis according to claim 1, characterized in that: The step 2 is specifically as follows: enhancing the background-removed image by Gamma transformation, increasing the brightness of the main body of the product in the background-removed image by a first set value, and obtaining an enhanced image; mapping the background image to the HSV space, reducing the V space in the HSV space by a second set value, and then mapping it back to the RGB space to obtain a background darkened image.

4. The method for optimizing light and shadow in product image synthesis according to claim 1, characterized in that: The diffusion models are Diffusion Models.

5. A device for optimizing light and shadow in product image synthesis, characterized in that: include: Obtain an image module to obtain a background image of only the product and a background image; The enhancement and darkening module enhances the background-removed image, increases the brightness of the main body of the product in the background-removed image to a first set value, and obtains an enhanced image; maps the background image to the HSV space, reduces the V space in the HSV space to a second set value, and then maps it back to the RGB space to obtain a background darkening image; The information module is extracted, the enhanced image and the background darkened image are synthesized to obtain a first synthesized image, and the first latent information is extracted from the first synthesized image through VAE decoding; The background-removed image and the background image are synthesized to obtain a second synthesized image, and the first setting information is extracted from the second synthesized image using a ControlNet model; Generate a guidance module, send the first latent information and the first setting information to the diffusion model, and generate a first light and shadow guidance map; extract the second latent information from the first light and shadow guidance map, and send the second latent information and the first setting information to the diffusion model to generate a second light and shadow guidance map; Optimize the light and shadow module, and use the high-contrast retention algorithm to migrate the texture information of the first light and shadow guidance map to the second light and shadow guidance map to obtain the intermediate map. Then use the Welsh algorithm to migrate the color information of the background map to the intermediate map to obtain the final target RGB image.

6. The device for optimizing light and shadow in product image synthesis according to claim 5, characterized in that: The image acquisition module specifically includes: acquiring a product image and a background image; and performing cutout on the product image through a cutout tool to obtain a background-removed image.

7. The device for optimizing light and shadow in product image synthesis according to claim 5, characterized in that: The enhancement darkening module specifically includes: enhancing the background-removed image through Gamma transformation, increasing the brightness of the main body of the product in the background-removed image by a first set value, and obtaining an enhanced image; mapping the background image to the HSV space, reducing the V space in the HSV space by a second set value, and then mapping it back to the RGB space to obtain a background darkening image.

8. The device for optimizing light and shadow in product image synthesis according to claim 5, characterized in that: The diffusion models are Diffusion Models.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 4 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images