Image generation processing method and electronic device

Abstract

Embodiments of the present application disclose an image generation processing method and an electronic device. The method comprises: determining an image to be processed, and separating a target main body image from the image to be processed; generating for the target main body image a design base image having a transparent background, and performing backup processing; using an artificial intelligence (AI) large language model to provide a background image complementing the transparent background part in the design base image, so as to obtain an AI-generated image, wherein in the AI-generated image, the area of the background image is greater than the area of the transparent background in the design base image; covering the AI-generated image with the backed-up design base image and performing image fusion processing to generate a target image, the target image comprising a complete target main body image, and shielding the edge of the boundary in the background image between the background image and the target main body image. According to the embodiments of the present application, high-quality images can be produced more efficiently and at lower costs.

Description

Image generation processing method and electronic device

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on September 27, 2023, with application number 202311272120.2 and application name “Image Generation and Processing Method and Electronic Device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] [Corrected 13.09.2024 according to Rule 91] The present application relates to the field of image processing technology, and in particular to an image generation and processing method and an electronic device.

[0003] [Corrected 13.09.2024 in accordance with Article 91][Deleted]

[0004] [Corrected 13.09.2024 in accordance with Article 91][Deleted] Background Art

[0005] In the product information service system, consumers' initial impression of a product is mainly formed through the product's main picture. The picture represents the product's external image. Therefore, the quality of the product's main picture will directly affect the number of clicks on the product, thereby increasing the store's traffic and browse-to-purchase conversion rate and other indicators.

[0006] To obtain high-quality product main images, some merchants may hire professional art teams to shoot real-life scenes of the products, but this approach can be very costly. Alternatively, some merchants or system technicians can use offline tools to perform image processing such as cutouts and merging. While this may reduce image production costs compared to hiring a professional team, the tools themselves have inherent technical barriers and complexity, resulting in low image production efficiency. Therefore, in actual applications, there are still a large number of low-quality product images that fail to increase click-through rates.

[0007] Therefore, how to produce high-quality product images at a lower cost and more efficiently has become a technical problem that needs to be solved by those skilled in the art.

[0008] Summary of the Invention

[0009] The present application provides an image generation and processing method and an electronic device, which can produce high-quality images at a lower cost and more efficiently.

[0010] This application provides the following solutions:

[0011] An image generation processing method, comprising:

[0012] Determine an image to be processed, and segment a target subject image from the image to be processed;

[0013] Generating a design base image with a transparent background for the target subject image and performing backup processing;

[0014] Using an artificial intelligence (AI) big language module to complete the background image for the transparent background portion of the design base image to obtain an AI-generated image, wherein the area of ​​the background image in the AI-generated image is larger than the area of ​​the transparent background in the design base image, and forms an occlusion of the edge portion of the target subject image;

[0015] The backup design base map is overlaid on the AI ​​generated image and image fusion processing is performed to generate a target image, wherein the target image includes a complete target subject image, and the target subject image blocks the edge of the background image at the junction with the target subject image.

[0016] The method of using the AI ​​big language module to complete the background image for the transparent background portion of the design base image includes:

[0017] Generate a mask image according to the position and size of the target subject image in the design base image, wherein the mask image includes a first area corresponding to the area where the target subject image is located in the design base image, and the portion outside the first area is a second area;

[0018] performing a first processing on the mask image so as to reduce the first area and expand the second area;

[0019] Using the design base image and the first processed mask image as input information, the AI ​​large language module is called to generate a background image, so that the generated background image is complemented to the enlarged second area while keeping the image content of the reduced first area in the design base image unchanged, to obtain the AI ​​generated image.

[0020] The performing a first processing on the mask image includes:

[0021] Gaussian blur processing is performed on the mask image so that the first area in the mask image is reduced and the second area is expanded.

[0022] Wherein, the image to be processed is an image related to a commodity;

[0023] The step of segmenting the target subject image from the image to be processed includes:

[0024] Identifying the category to which the commodity included in the image to be processed belongs;

[0025] Target detection is performed in the image to be processed according to the identified commodity category, and the target main body image is segmented from the image to be processed according to the detected target area.

[0026] If there are additional white edges around the segmented target subject image, the method further includes:

[0027] Starting from the outermost side of the white edge, pixels within the target width are selected inward as pixels to be processed;

[0028] The pixel value of the pixel to be processed on the transparency channel is raised to the Nth power to remove or weaken the white edge; wherein N is a natural number greater than 1.

[0029] If the segmented target subject image has jagged edges, the method further includes:

[0030] Gaussian blur processing is performed on the area where the jagged edge is located to remove or weaken the jagged edge.

[0031] The step of generating a design base image with a transparent background for the target subject image includes:

[0032] Providing a canvas interface and displaying the segmented target subject image on the canvas interface;

[0033] According to the movement and / or scaling operation performed by the user on the target subject image based on the canvas interface, the position and size of the target subject image in the canvas are determined, and the design base image with a transparent background is generated.

[0034] The AI ​​large language module is further associated with a control plug-in, and the control plug-in is used to control the background image generated by the AI ​​large language module to enhance the association between the generated background image and the target subject image;

[0035] The control plug-in reads the area of ​​the target subject image during the control process, and controls the area outside the target subject image using a lower control weight than the area where the target subject image is located;

[0036] The area of ​​the target subject image read by the control plug-in is larger than the actual area of ​​the target subject image.

[0037] Among them, also include:

[0038] Determine the target background style and / or scene description information so that when the AI ​​big language module is called, the target background style and / or scene description information is added to the input information, and the AI ​​big language module generates a background image consistent with the target background style and / or scene description information.

[0039] An image generation processing method, comprising:

[0040] Determine an image to be processed, and segment a target subject image from the image to be processed;

[0041] Generating a design base image with a transparent background for the target subject image;

[0042] In the process of the AI ​​large language module completing the background image for the transparent background area in the design base map, the generation process of the background image is controlled, and during the control process, the area of ​​the target subject image is read, and for the area outside the target subject image, a lower control weight relative to the area where the target subject image is located is used for control; wherein the area of ​​the target subject image read is larger than the actual area of ​​the target subject image.

[0043] The area for reading the target subject image includes:

[0044] Reading the area of ​​the target subject image according to the mask image corresponding to the design base image;

[0045] The mask image is generated by: generating an original mask image according to the position and size of the target subject image in the design base image, wherein the original mask image includes a first area corresponding to the area where the target subject image is located in the design base image, and the portion outside the first area is a second area;

[0046] By performing a second processing on the original mask image, the first area is enlarged and the second area is reduced, so that the corresponding area of ​​the target subject image is read according to the mask image obtained after the second processing.

[0047] An image segmentation processing method, comprising:

[0048] Determine the target subject image segmented from the original image;

[0049] If there are extra white edges around the segmented target subject image, starting from the outermost side of the white edges, pixels within the target width range are selected inward as pixels to be processed;

[0050] The pixel value of the pixel to be processed on the transparency channel is raised to the Nth power to remove or weaken the white edge; wherein N is a natural number greater than 1.

[0051] An image generation and processing device, comprising:

[0052] A subject segmentation unit, configured to determine an image to be processed and segment a target subject image from the image to be processed;

[0053] A design base image generating unit, configured to generate a design base image with a transparent background for the target subject image and perform backup processing;

[0054] An image generation unit, configured to use an artificial intelligence (AI) large language module to complete a background image for the transparent background portion of the design base image to obtain an AI-generated image, wherein the area of ​​the background image in the AI-generated image is larger than the area of ​​the transparent background in the design base image, and forms an occlusion of an edge portion of the target subject image;

[0055] A fusion processing unit is used to overlay the backup design base map onto the AI-generated image and perform image fusion processing to generate a target image, wherein the target image includes a complete target subject image, and the target subject image obscures the edge of the background image at the junction with the target subject image.

[0056] An image generation and processing device, comprising:

[0057] A subject segmentation unit, configured to determine an image to be processed and segment a target subject image from the image to be processed;

[0058] A design base image generating unit, configured to generate a design base image with a transparent background for the target subject image;

[0059] A generation control unit is used to control the generation process of the background image during the process of the AI ​​large language module completing the background image for the transparent background area in the design base map, and during the control process, read the area of ​​the target subject image, and use a lower control weight relative to the area where the target subject image is located for control of the area outside the target subject image; wherein the area of ​​the target subject image read is larger than the actual area of ​​the target subject image.

[0060] An image segmentation processing device, comprising:

[0061] a subject segmentation result determination unit, configured to determine a target subject image segmented from an original image;

[0062] a unit for determining pixels to be processed, configured to, if there are extra white edges around the segmented target subject image, select pixels within a target width range inward from the outermost side of the white edges as pixels to be processed;

[0063] A calculation processing unit is used to perform N-th power processing on the pixel value of the pixel to be processed on the transparency channel to remove or weaken the white edge; wherein N is a natural number greater than 1.

[0064] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of any of the aforementioned methods.

[0065] An electronic device, comprising:

[0066] one or more processors; and

[0067] A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of any of the aforementioned methods.

[0068] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0069] Through the embodiment of the present application, the main image can be segmented from the image to be processed specified by the user, and the AI ​​large model can be used to generate a background image for the main image while keeping the main image unchanged. In the process of generating the background image, in order to achieve a more natural fusion of the edge of the main image with the background image, the AI ​​large model can be used to generate a "larger circle" background image first. After the "larger circle" background image is added to the design base map to obtain the AI ​​generated image, the original design base map can be further covered on the AI ​​generated image through image fusion technology. In this way, the edge of the junction between the background image generated by the AI ​​large model and the main image will be blocked by the main image. Even if the above-mentioned edge of the background image contains pure colors or speculative content generated by the AI ​​large model, it will not be displayed. Therefore, a more natural fusion between the main image and the background image can be achieved, and a higher quality generated image can be obtained.

[0070] In a preferred implementation, after the main image is segmented from the image to be processed, "white edges" and "jaggies" can be removed to further improve the final image quality. When removing "white edges," the transparency channel pixel values ​​of the pixels within the "white edge" area are raised to the Nth power, allowing for automated "white edge" removal.

[0071] In addition, in order to make the background image generated by the AI ​​large model interact better with the main image in terms of contour, light and shadow relationship, etc., the plug-in program of the AI ​​large model can also be used to control the generation process of the background image. In addition, in order to avoid the edge diffusion of the main image or the single background image caused by setting the control weight too low or too high, the control logic of the plug-in program has also been improved. That is, the same control weight is no longer used for the entire image, but it can be distinguished. For areas outside the main image, a lower control weight can be used for control, so that this control acts more on the part of the area where the main image is located, avoiding the AI ​​large model from generating an overly simplistic background image and reducing the probability of partial edge diffusion of the main image. At the same time, in order to further avoid the edge diffusion of the main image, the main image area read by the control plug-in can be made slightly larger than the actual main image, so as to better solve the problem of edge diffusion of the main image.

[0072] Of course, any product implementing the present application does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0074] FIG1 is a schematic diagram of a system architecture provided in an embodiment of the present application;

[0075] FIG2 is a flow chart of a first method provided in an embodiment of the present application;

[0076] FIG3 is a schematic diagram of the “white edge” generated during the subject image segmentation process;

[0077] FIG4 is a schematic diagram of an image processing process provided by an embodiment of the present application;

[0078] FIG5 is a schematic diagram of processing a mask image according to an embodiment of the present application;

[0079] Figure 6 is a schematic diagram of the Control Net principle;

[0080] FIG7 is a schematic diagram of an AI-generated image and its fusion with a design base image provided by an embodiment of the present application;

[0081] FIG8 is a flow chart of a second method provided in an embodiment of the present application;

[0082] FIG9 is a flowchart of a third method provided in an embodiment of the present application;

[0083] FIG10 is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0084] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field are within the scope of protection of this application.

[0085] First, it's important to note that in product information service systems, the quality of product images, beyond the clarity of the product image itself and the appropriate shooting angle, also hinges on whether the image's main image and background convey a consistent sense of purpose. For example, if the background of a delicate product is cluttered or the tonal quality of the background fails to highlight the product's exquisiteness, the image quality will be significantly compromised. In practice, selecting a reasonable shooting angle and capturing clear product images is a relatively easy task. However, choosing an appropriate background image for the product is a more challenging task, requiring merchants to possess a keen sense of aesthetics. Without this, existing techniques often require the assistance of professional art teams. While image enhancement can be achieved through some graphics tools, most rely on manual labor, which still presents technical barriers and complexity.

[0086] In view of the above situation, in an embodiment of the present application, an image generation service can be provided for users such as merchants. In this service, users can be allowed to upload or specify specific images to be processed, and then a specific main image can be segmented from it. While keeping the content of this part of the image unchanged, the ability of the AI ​​(Artificial Intelligence) Large Language Model (LLM, referred to as the AI ​​large model) is used to generate a suitable background image, which is then combined into a complete image. Since the AI ​​large model has powerful drawing capabilities and the ability to understand multimodal information, etc., it can generate a more reasonable background image with a more unified mental expression for a specific main image, and in addition to the early training of the AI ​​large model, no additional manpower costs and technical thresholds are required during use. Therefore, image production can be carried out at a lower cost and more efficiently. Of course, in the specific implementation, the user can also select or the system can specify a specific background style and / or scene description information by default, so that the background image generated by the AI ​​large model meets the specific style or scene requirements.

[0087] In this way, when merchants need to generate images such as product main images for products, if they have already taken photos of the products in advance and obtained photos including the main image content of the products, and the quality of the main image content of the products in the photos is relatively high and does not need to be optimized, and only the background needs to be optimized, the image generation service provided by the embodiment of this application can be used. At this time, the photo can be uploaded to a specific service, and the service uses the AI ​​big model to generate a more reasonable background image for the photo that has a more unified mental expression with the main image of the product. There is no need to hire a professional art team to take the photo, nor is there any need to use existing mapping tools to manually replace the background and other beautification processes. Therefore, costs can be reduced and efficiency can be improved.

[0088] Among them, AI big models can refer to a type of foundation model (Foundation Model), specifically a model with a huge number of parameters trained using massive data and capable of adapting to a series of downstream tasks. For AI big models, not only is there a huge number of parameters in terms of parameter scale (with the continuous iteration of the model, the number of parameters will usually increase exponentially, from hundreds of millions to trillions, then to one trillion, or even more), but also, from the perspective of modal support, AI big models have also gradually evolved from supporting single tasks in a single modality such as pictures, images, text, voice, and video to supporting multiple tasks in multiple modalities. In other words, large models usually also have the ability to efficiently understand information from multiple modalities, cross-modal perception capabilities, and the ability to migrate and execute across differentiated tasks, and may even have multimodal information perception capabilities like the human brain.

[0089] From another perspective, AI big model is short for "artificial intelligence pre-trained large model", which includes the two meanings of "pre-training" and "big model". The combination of the two has produced a new artificial intelligence model, that is, after the model is pre-trained on a large-scale data set, it does not need fine-tuning, or only requires fine-tuning on a small amount of data to support various downstream applications. In other words, thanks to its "large-scale pre-training + fine-tuning" paradigm, the AI ​​big model can adapt well to different downstream tasks and demonstrate its strong versatility. This kind of universal AI big model, in the case of shared parameters, only needs to make corresponding fine-tuning in different downstream application scenarios to achieve superior performance, breaking through the limitations of traditional AI models that are difficult to generalize to other tasks.

[0090] From the perspective of processing results, the aforementioned AI models are also generative models. These models not only predict outcomes based on features but also "understand" how data is generated and "create" new data based on that understanding.

[0091] With the support of the above-mentioned capabilities of the AI ​​big model and existing knowledge, some pre-training of the basic AI big model can be carried out based on the scenario requirements in the embodiments of this application, so that the AI ​​big model has the ability to produce background images. For example, some high-definition, high-quality images can be collected in advance, and some images with unified mind between the main image and the background image can be included as positive samples, and those with inconsistent mind as negative samples, etc., and then input into the AI ​​big model, and the AI ​​big model produces specific background image content as required; for the content produced by the model, manual verification can also be carried out, and the accuracy information of the content can be fed back to the AI ​​big model, so that the AI ​​big model can continuously iterate and learn, and ultimately make the content produced have higher accuracy, etc.

[0092] In the process of producing background images through AI large models, in order to make the final generated image look more realistic (that is, more like an image obtained by photographing a subject such as a product in a scene with a specific background image, rather than attaching the product image to the background image), a variety of improvement methods are also provided in the embodiments of the present application.

[0093] First of all, since it is necessary to keep the main image part unchanged and regenerate the background image for this part of the image, the specific way to do it can be to first identify and segment the main image part from the original image to be processed (that is, the part of the image to be processed that belongs to the foreground or the main subject), and then use this part of the content as the content that needs to remain unchanged, and only regenerate the background part. In order to achieve this goal, after segmenting the main image part, a design base map with a transparent background can be generated for the main image, and then a mask map of the design base map is obtained. The area in the design base map that belongs to the main image can be covered by the mask map, thereby marking it as an area that does not require AI generation, and the other areas are areas where background images need to be generated by the AI ​​large model.

[0094] After the above processing, theoretically, the Mask corresponding to the design base map can be directly used as a mask, and the AI ​​large model can be used to generate the background image. However, this may cause the edge of the main image to blend unnaturally with the background. In response to this problem, improvements have been made in the embodiments of the present application. Specifically, the AI ​​large language module can be used to generate a "larger circle" background image first, and then the original design base map can be overlaid on the upper layer of the AI-generated image, so that the edge of the AI-generated background image (mainly the edge at the intersection with the main image) can be covered by the main image, so that the edge of the AI-generated background image will not be displayed, thereby improving the image generation effect.

[0095] In order to achieve this goal, after generating the Mask corresponding to the design base map, instead of using the Mask directly as a mask, the area in the Mask corresponding to the main image is first reduced inward, and accordingly, the surrounding area where the background image needs to be generated will become larger accordingly. In other words, the area of ​​the target main image read by the AI ​​large model is smaller than the actual area, so that the generated background image is slightly larger. In this way, when the background image is completed in the design base map, the edge part of the main image in the design base map can be blocked. Of course, in the image generated at this time, the main image lacks the content of the edge part. Therefore, the original design base map can be overlaid on the upper layer of the AI-generated image, so that in the final generated image, the content of the main image is complete and can be better integrated into the surrounding background image.

[0096] In order to further improve the effect of the final generated image, some improvements can also be made when segmenting the main image from the original image to be processed. For example, the main image segmented by ordinary segmentation algorithms may have a circle of "white edges" around it, or some edges may appear jagged, etc. Therefore, these defects can also be processed to further improve the authenticity of the final generated image.

[0097] Furthermore, in the process of generating background images using the AI ​​large model, although in theory the specific background image can be freely generated by the AI ​​large model, if the generation is completely free, there may be no interaction or association between the background image and the main image. In this case, even if the two are mentally unified, there may be defects in overall coordination and other aspects. Therefore, in a preferred manner, the control plug-in of the AI ​​large model can also be used to control the generation process of the background image. That is, the background image generated by the AI ​​large model can better interact with the content of the main image while maintaining a certain degree of freedom. However, since the control plug-in controls the generation of the entire image by default, this will create a contradiction: if the control weight is too low, the edges of the main image will spread; if the control weight is too high, the generated background image will be relatively simple and lack richness. To this end, the embodiments of the present application also provide corresponding improvement solutions. The above-mentioned improvements on white edge removal, anti-aliasing, and control plug-in will be described in detail later.

[0098] From the perspective of system architecture, referring to FIG1 , the embodiment of the present application can provide image generation services for merchants or technical personnel in the system, etc. The service can provide a server, and the specific AI large model can run on the server. For the client, it can exist in the form of an independent client program (App) or a web page, etc. The user can upload images through the client, or specify relevant images of a certain product, etc., and initiate an image generation request, which is then processed by the server. Among them, since it may take a certain amount of time for the server to call a specific AI large model for image generation, the server can also manage it in the form of a task queue. For example, after the user selects the required style description information and creates a task, he can enter the task queue of the server, and then the image generation service in the server can pull the task from the queue, and the generated image can be stored through the storage system for users to download, and so on.

[0099] The specific implementation scheme provided in the embodiments of this application is introduced in detail below.

[0100] Example 1

[0101] First, this embodiment 1 provides an image generation processing method from the perspective of the server. Referring to FIG2 , the method may include:

[0102] S201: Determine an image to be processed, and segment a target subject image from the image to be processed.

[0103] The image to be processed is the image for which the background image is generated. Specifically, the method for determining the image to be processed can support different usage scenarios. For example, it can be a picture manually uploaded by the user. Alternatively, since it can be associated with a specific product information service system, it can be input by specifying a product link. In this case, the image that meets the conditions can be parsed from the product link as the image to be processed, and so on.

[0104] In addition, in the case where users upload images to be processed, the uploaded images to be processed can also be pre-checked to filter out images containing people blocking products, multiple product bodies, etc. For example, the object detection service in the system can be called to detect all objects included in the image, and a certain threshold is used to determine whether a human body exists. Then, the element recognition service is called to determine whether the number of product elements in the image is greater than the threshold, and so on. For example, if the image uploaded by the user does not include a human body, and the number of product elements does not exceed the threshold (for example, three, this judgment is mainly used to ensure the quality of the final generated image. For example, if the same image contains too many product elements, each element may be relatively small and not suitable as the main image of the product. Therefore, it can be filtered out), then the detection passes, that is, the image uploaded by the user can be used as the image to be processed, and in subsequent steps, the main image can be segmented and the background image can be generated.

[0105] After determining the image to be processed, the target main image can be segmented out from the image to be processed, which is the so-called "cutout" processing. In specific implementation, the traditional "cutout" method can be used to achieve the segmentation of the main image, or, in another way, since the image to be processed in the embodiment of the present application may be an image related to the product, and the product itself has category information, this information can be used to assist in image segmentation and improve segmentation accuracy. Therefore, in this case, the category to which the product included in the image to be processed belongs can be first identified, and then, based on the identified product category, target detection can be performed in the image to be processed, and based on the detected target area, the target main image can be segmented out from the image to be processed. For example, the main image segmentation capability can be based on Segment Anything (a basic model for image segmentation) and Grounding DINO (for object detection, where the detected target is a range. If the range contains multiple objects that are difficult to distinguish, all of them can be covered). After predicting the product category with the help of the category prediction capability (the category name can also be machine-translated into English by a deployed Chinese-to-English model to adapt to the input language of Grounding DINO), the Grounding DINO object detection algorithm can identify the product's location in the main image based on the category. Segment Anything is then called to segment the product's mask image. Then, pixel mapping is performed between the mask image and the original product image to obtain the main image segmentation result.

[0106] It should be noted that the image to be processed may contain multiple product elements. In this case, when performing category recognition on the image to be processed, the probability values ​​corresponding to the multiple categories that may be contained in the image to be processed can be output. Among them, the probability value will also be affected by the position and size of the specific element in the image to be processed. Subsequently, the one with the higher probability value can be used as the product category to which the subject identified in the image to be processed belongs. In other words, this product category is used as the main product category expressed in the image to be processed, and the main image can be segmented based on the identified product category.

[0107] The segmentation result of the main image obtained by the above method may be a rough result. This roughness is mainly reflected in two aspects: 1) The edges of the product may have additional white edges (referred to as "white edges"). For example, as shown at 31 in Figure 3, the main image in the image is surrounded by a circle of "white edges". If subsequent image generation is directly based on this segmentation result, this "white edge" will continue to exist in the newly generated image; 2) Some jagged edges may also be present. Therefore, to ensure the quality of the final image generation, the image segmentation result can be further refined and optimized. This optimization can include removing "white edges" and removing jagged edges.

[0108] To remove "white edges," the inventors first analyzed the characteristics of "white edges" and discovered the following: The "white edges" formed around the main image by segmentation are actually a semi-transparent band of extra pixels. This band is composed of pixels within a target width range, typically about 5 pixels wide, extending from the outermost edge of the "white edge" to the innermost edge. These pixels exhibit a gradual transition from completely transparent to completely opaque in terms of transparency. Based on these characteristics, the present invention provides a specific method for removing "white edges." Specifically, each pixel in the extra pixel band is treated as a pixel to be processed. The pixel values ​​of these pixels in the transparency channel are then raised to the power of N (N can be a natural number greater than 1, such as 2, 3, etc.). This allows pixels with high transparency to become more transparent, while pixels with low transparency to become more opaque. The transparency curves of these pixels change from a linear curve to an exponential curve, thereby narrowing the "white edges" to a level that is imperceptible to the naked eye, thereby achieving the purpose of removing or reducing the white edges.

[0109] In addition, there are several methods for removing jagged edges in the prior art. In the embodiments of this application, these methods are preferably selected. It has been verified that Gaussian blur processing is more suitable for the specific scene characteristics of the embodiments of this application. Therefore, Gaussian blur processing can be performed on the area where the jagged edge is located to remove or weaken the jagged edge.

[0110] After removing the "white edges" and jagged edges, the segmented target subject image can have higher quality, and when it is subsequently fused with the background image generated by the AI ​​large model, a better fusion effect can be achieved.

[0111] S202: Generate a design base image with a transparent background for the target subject image and perform backup processing.

[0112] After the target subject image is segmented from the image to be processed, a design base map with a transparent background can be generated for the target subject image. Among them, the so-called design base map usually refers to the drawings necessary for designers to complete scheme design, floor plans, and analysis drawings, also known as the first drawing in the drawing stage. In an embodiment of the present application, the target subject image can be added to a canvas, which can have a transparent background. In this way, the canvas with a transparent background carrying the target subject image becomes a design base map. The design base map has the same size as the target image finally generated, and the subsequent AI large model is based on the design base map to complete the background image.

[0113] Among them, there are many ways to generate a specific design base map. For example, in one way, the system can set the canvas size, the position and size of the target subject image in the canvas by default, so as to generate a design base map with a transparent background. Or, in another way, the user can manually adjust the canvas size, the position and size of the target subject image in the canvas. For example, in a specific implementation, a canvas interface can be provided in the client interface. After completing the segmentation of the main image of the image to be processed, the canvas interface can be entered. The specific canvas can be displayed as a default size, and the target subject image can be displayed at a default position in the canvas with a default size. After that, the user can adjust the size of the canvas according to his or her needs or preferences, and can also perform operations such as moving and scaling the target subject image to change its position and size in the canvas, etc. After the user completes the operation, a specific design base map can be generated.

[0114] That is to say, the design base map includes the canvas and the target subject image, as well as the position, size and other information of the target subject image in the canvas, wherein the size of the canvas determines the size of the final generated image, and in the design base map, the background is transparent. For example, as shown in FIG4 , assuming that the original input image to be processed is “pic1” in FIG4 , after the subject image is segmented, the target subject image obtained can be as shown in “pic2” in FIG4 , and then the target subject image can be added to the canvas, and can also be dragged or scaled to generate the design base map shown in “pic3” in FIG4 . It should be noted here that in order to display the size of the canvas, and the position and size of the subject image in the canvas, the design base map shown in “pic3” in FIG4 displays a checkerboard-style background, which is actually the background taken from the client operation interface. Those skilled in the art can understand that the design base map is actually a transparent background.

[0115] Among them, after generating the design base map, you can also make a backup, that is, you can save another copy, one of which can be used to produce AI-generated images, and the other can be used to continue processing the AI-generated images. This will be introduced in detail later.

[0116] S203: Utilize an artificial intelligence (AI) big language module to complete the background image for the transparent background portion in the design base image to obtain an AI-generated image, wherein, in the AI-generated image, the area of ​​the background image is larger than the area of ​​the transparent background in the design base image, and forms an occlusion on the edge portion of the target subject image.

[0117] After generating a design base image with a transparent background, the artificial intelligence (AI) language module can be used to complete the background image for the transparent background portion of the design base image to obtain an AI-generated image. In this embodiment of the present application, the area of ​​the background image in the AI-generated image can be larger than the area of ​​the transparent background in the design base image. In other words, the AI ​​model will generate a "larger" background image. In this way, the background image will block the edge of the target main image.

[0118] In specific implementation, in order to achieve the above purpose, there can be a variety of ways. For example, in one way, a mask image can be first generated according to the position and size of the target subject image in the design base map, wherein the size of the mask image can also be consistent with the design base map, including a first area corresponding to the area where the target subject image is located in the design base map, and the part outside the first area is the second area. Among them, the mask image is usually binary expressed in black and white. In the embodiment of the present application, the first area can be black and the second area can be white, thereby representing that the first area is the area that needs to be shielded, that is, the area that does not need AI generation, or the area where the image content needs to remain unchanged. For example, the figure shown in "pic4" in Figure 4 is the mask image generated in the corresponding example (in this figure, only the black first area part is shown, and the white second area part is not shown).

[0119] After obtaining the mask image, as mentioned above, theoretically, the mask image can be directly used as a mask for the design base image. That is, the design base image and the mask image can be input into the AI ​​large model, and the AI ​​large model can then generate the background image. However, since the AI ​​large model does not include the area where the main image is located when generating the background image, it only generates the background image for the area outside the main image, and then uses the generated background image to complete the design base image. In other words, the image output by the AI ​​large model can be generated by splicing the background image generated by the AI ​​large model with the main image portion of the design base image. Although the AI ​​large model may refer to some information about the main image when generating the background image, this information is primarily pixel color. In other words, the specific content of the main image may be unknown to the AI ​​large model. Therefore, the AI ​​large model may generate the edges of the background image (primarily the edges where the background image meets the main image) as white or some other color, or it may generate some content based on speculation, etc. However, no matter which specific method is used, it may be difficult for the above-mentioned edge part of the background image to achieve a relatively natural transition with the main image content. That is, it still looks like the main image is pasted onto the background image, rather than an image obtained by actually shooting the main image in the scene corresponding to the background image.

[0120] Therefore, in an embodiment of the present application, the mask image can be processed first so that the first area in the mask image (that is, the area corresponding to the main image) is reduced, and correspondingly, the second area is expanded. Specifically, the first area can be reduced inward, and of course, the shape of the edge can be kept unchanged. For example, as shown in Figure 5, assuming that Figure 5 (A) is the original mask image, the black area in the middle represents the area where the target main image is located, that is, the above-mentioned first area, and the surrounding white area is the second area; if the mask image is directly used as the mask of the design base image, the AI ​​large model generates a background image for the second area, for example, as shown in the stripe-filled area in Figure 5 (B); however, in an embodiment of the present application, the first area in the mask image is first reduced, so that the second area is expanded, and then the background image is generated. For example, the processed mask image is shown in Figure 5 (C). It can be seen that the area of ​​the black area in the middle is smaller than the black area in Figure 5 (A). Of course, the edge shape of the black area can remain unchanged. Then, the processed mask image can be used as the mask of the design base image to generate the background image. The area where the AI ​​large model generates the background image is shown as the area filled with stripes in Figure 5(D). It can be seen that the area of ​​this area is larger than the area of ​​the background image generated in Figure 5(B).

[0121] In other words, if the role of the mask image is compared to "hollowing out" the part of the design base image that belongs to the main image area, so that the AI ​​large model can generate background images for other areas while keeping this part of the content unchanged, then after the first area in the mask image is reduced, it is equivalent to the "hollowed out" part of the design base image becoming smaller. Accordingly, the AI ​​large model can generate an extra circle of background image content.

[0122] In specific implementation, in order to reduce the above-mentioned first area of ​​the mask image and expand the second area, there can be multiple implementation methods. For example, in one method, Gaussian blur (specifically, large-kernel Gaussian blur, a larger kernel size will produce a stronger blurring effect) can be used to process the mask image, so that the white part in the mask image can be diffused and the black area can be reduced.

[0123] After obtaining the mask image after the first processing, the design base image and the processed mask image can be used as input information to call the AI ​​big model to generate a background image. At this time, the AI ​​big model can use the processed mask image as a mask for the design base image, and then generate a background image for the expanded second area. In specific implementation, Stable Diffusion and the like can be selected as the AI ​​big model in the embodiment of this application. Of course, before using the model, the model can also be fine-tuned according to the scenario in the embodiment of this application, for example, including collecting some high-quality product images, fine-tuning the model, and so on.

[0124] When generating a background image using the AI ​​large model, the AI ​​large model can be freely generated. However, if this is done completely freely, the background image and the main image may have no interaction or connection in terms of contour, light and shadow relationships, depth, and other aspects. In this case, even if the two are mentally unified, there may be defects in overall coordination and other aspects. Therefore, in a preferred embodiment, the background image generation process can also be controlled through the AI ​​large model's control plug-in. That is, while maintaining a certain degree of freedom, the background image generated by the AI ​​large model can better interact with the main image content in terms of contour, light and shadow relationships, depth, and other aspects, thereby improving the overall output quality. For example, for the Stable Diffusion model, the above control purpose can be achieved through the Control Net plug-in. Specifically, the design base map can be pre-processed to generate a control diagram for the Control Net plug-in control. For example, for the example shown in Figure 4, the control diagram can be shown as "pic5" in Figure 4, which is similar to a line drawing. That is, it can be generated by extracting lines included in the main image. These lines may include lines used to depict the edges, light and shadow, etc. of the main image. By inputting this control graph into the Control Net plug-in, the background image generation of the Stable Diffusion model can be controlled so that the generated background image can interact with the main image content.

[0125] However, the Control Net plug-in is a network that can exert control over each layer of the Stable Diffusion backbone network. By default, it controls the generation of the entire image. This creates a contradiction: if the control weight is too low, the edges of the main image will diffuse (which will cause the main image to produce a distorted visual effect); if the control weight is too high, the generated background image will be monotonous, and it will be difficult to find an appropriate weight to achieve a good balance. In this context, the embodiment of the present application also modifies the control logic of Control Net. Specifically, during the control process, the position of the main image in the design base map can be read. For the part outside the main image, the control weight can be reduced, thereby reducing the control over the background image.

[0126] To facilitate understanding, the following is a brief introduction to the principles of Control Net. ControlNet's function is to manipulate large AI models used for image generation, such as Stable Diffusion, through various conditions such as edge detection, sketch processing, and pose control. Specifically, it supports a variety of input types, including sketches, edge images, semantically segmented images, key features of human figures, Hough transform lines, depth maps, and human skeletons, to generate images that better meet user needs. Taking the Stable Diffusion model as an example, without ControlNet control, the original neural network in a Stable Diffusion model might look like the one shown in Figure 6(A). ControlNet locks the original Stable Diffusion model's neural network, setting it as a locked copy. It then creates a copy of the original network, called a trainable copy, upon which it applies control conditions. In other words, ControlNet first copies the Stable Diffusion model's weights to create a "trainable copy." In contrast, the original Stable Diffusion model is pre-trained on billions of images, so its parameters are "locked." This "trainable copy" only needs to be trained on a small dataset for a specific task in order to learn conditional control. As shown in Figure 6(B), the "locked model" and the "trainable copy" are connected through a 1×1 convolution layer, called the "zero convolution layer." The weights and biases of the zero convolution layer are initialized to 0, so that the training speed will be very fast, close to the speed of fine-tuning the Stable Diffusion model, and can even be trained on personal devices. Specifically, the control condition can be passed through zero convolution and then added to the original input. After the addition, it enters the imitation neural network block of ControlNet ("trainable copy"), and the network output is subjected to another zero convolution and added to the output of the original network to obtain the final output.

[0127] Among them, the process of ControlNet controlling the Stable Diffusion model involves control weights. In the existing technology, ControlNet uses the same control weight for the entire image. However, as mentioned above, in the scenario of the embodiment of the present application, setting the control weight too high or too low will cause corresponding problems, making it difficult to find the appropriate weight. Therefore, in the embodiment of the present application, a region-based control implementation scheme is adopted. Specifically, ControlNet can control on a pixel basis. When reading the pixel currently to be controlled, it can first determine whether the pixel is within the region where the main image is located. If so, it can be controlled using a first control weight. Otherwise, if the current pixel is outside the region where the main image is located, it can be controlled using a second control weight, where the first control weight is higher than the second control weight. In other words, ControlNet's control over the background image generated by the large AI model is reduced, allowing it to maintain a high degree of generation freedom to avoid the generated background image being too monotonous. At the same time, for pixels within the region where the main image is located, a higher weight can be used to control them to reduce the probability of problems such as deformation of the main image due to edge diffusion. It's important to note that the Stable Diffusion model's network structure may contain multiple layers, each corresponding to a different image sampling rate, such as 8×8, 16×16, 32×32, and 64×64. Accordingly, ControlNet can also include multiple layers to independently control each layer in the Stable Diffusion model. In this case, each layer can be controlled using the aforementioned region-by-region weight control scheme.

[0128] Furthermore, during the aforementioned area-based ControlNet control process, the position of the subject image read can be slightly larger than the actual position of the subject image; otherwise, edge diffusion may not be completely avoided. In other words, ControlNet is primarily used to create interaction between the background image and the subject image. However, this interaction cannot be completely based on the edges of the subject image. Instead, the interaction area needs to be slightly larger than the subject image area. This ensures that the generated image appears to be truly within the background image, rather than pasted onto it.

[0129] In a specific implementation, to achieve the above purpose, a second processing can be performed on the mask image corresponding to the design base image, so that the first area is expanded and the second area is reduced. The mask image after the second processing can be provided to the control plug-in, so that during the control process, the control plug-in uses a smaller control weight for the reduced second area than for the expanded first area.

[0130] There are multiple ways to perform the second processing on the mask image. For example, in one method, the mask image corresponding to the design base image can be first inverted and then Gaussian blurred to expand the first region and shrink the second region in the mask image. In other words, the mask image can be reverse-eroded to form a mask image with the first region expanded. The subject and non-subject locations can then be distinguished using the image pixel values ​​in the mask image.

[0131] After the AI ​​large model generates a background image for the second area, it can be completed into the design base image to obtain an AI-generated image. In this AI-generated image, the image content of the part covered by the first area after the reduction processing in the mask image remains unchanged, and the other areas are the AI-generated background images. For example, as shown in Figure 7, it is assumed that Figure 7 (A) is the design base image, the middle part is the main image, and the other areas are transparent backgrounds; when the mask image shown in Figure 5 (C) is used as a mask, the image generated by the AI ​​large model can be shown in Figure 7 (B). It can be seen that the background part of the design base image has been completed. However, it can be seen that since the first area of ​​the mask image has been reduced, the background image generated by the AI ​​large model will block the edge of the main image, that is, in this state, the main image is incomplete.

[0132] It should be noted here that the example shown in Figure 7 is only for illustrating the situation where the edge of the main image is blocked by the background image and cannot be fully displayed. In actual applications, the background image generated by the AI ​​large model is much better than that shown in Figure 7, and there will be no obvious black edges.

[0133] It should also be noted that when the AI ​​big model is specifically called for image generation, in addition to inputting the aforementioned design base map and the processed mask map, you can also input specific required background style and / or scene description information so that the AI ​​big model can generate a background image of the corresponding style or scene. Specifically, this background style and / or scene description information can be submitted after the user selects it. For example, as shown in Figure 4, you can select the "blue sea" style, and a corresponding legend can be provided for the specific style in the selection interface for the user's reference. Alternatively, if the user does not make a selection, the default style or scene can also be used, and so on. This style or scene description information can be used as a prompt word for the AI ​​big model, so that the AI ​​big model can generate a background image of the corresponding style or scene.

[0134] Among them, when providing multiple optional styles or scenes, when training the AI ​​big model, the AI ​​big model can also be trained in various different styles or scenes respectively. Different model parameters can be used correspondingly for different styles and scenes, so that the AI ​​big model can produce high-quality background images in a variety of different styles or scenes.

[0135] S204: Overlaying the backup design base map onto the AI ​​generated image and performing image fusion processing to generate a target image, wherein the target image includes a complete target subject image, and the target subject image obscures the edge of the background image at the junction with the target subject image.

[0136] Because the first area in the mask image is scaled down during the AI ​​large model's background image generation process, the edge areas of the subject image in the AI-generated image are obscured by the background image. Therefore, after obtaining the AI-generated image, the previously backed-up original design base image can be overlaid on the AI-generated image. At the same time, some image fusion processing can be performed to achieve better fusion of the specific images. This allows the subject image in the original design base image to in turn obscure the edges of the AI-generated background image (the edges at the junction with the subject image). This allows even solid-colored or otherwise speculative edges in the AI-generated background image to be obscured by the subject image, allowing the subject image edges to blend more naturally into the background image. For example, as shown in Figure 7(C), in the final generated target image, the subject image content remains intact, and the edges of the AI-generated background image that intersect with the subject content can be obscured. Furthermore, in a preferred embodiment, the segmented subject image can also be processed to remove "white edges" and "jaggies." This allows the final generated image to resemble the subject in a real scene, rather than an image artificially generated using a mapping tool.

[0137] After obtaining the target image, it can also be scored. If the scoring result meets the requirements, it can be saved to the server's storage system for users to download and use. For example, if the specific image to be processed is an image related to a product, after adding a background image using the method provided in the embodiment of the application, it can be used to publish it to the product information service system as the main product image, etc.

[0138] In order to facilitate a better understanding of the specific implementation scheme of the embodiment of the present application, the complete implementation process is introduced below in conjunction with the example shown in Figure 4 (of course, in actual application, some steps can be omitted).

[0139] 41: Before performing specific image generation, you can first select the network structure of the AI ​​large model. In addition, you can also use some high-quality high-definition images to train the model and determine the specific generation parameters;

[0140] 42: Specifically, when generating an image, the original image pic1 to be processed can be determined first;

[0141] 43: Perform subject image segmentation processing on the original image to segment the subject image pic2;

[0142] 44: Generate a transparent background design base map pic3 based on the subject segmentation result;

[0143] 45: Generate the mask image pic4 corresponding to the design base image;

[0144] 46: In an optional way, the diagram pic5 for Control Net control can be generated based on the design base diagram;

[0145] 47: In the optional mode, the user can select the desired scene;

[0146] 48: Generate prompts for communicating with the AI ​​model based on representative images corresponding to specific scenarios, which may include positive and negative prompts.

[0147] 49: Using the design base map pic3, the mask map pic4, and the map for Control Net control pic5 as input, the background map of the AI ​​large model is generated to obtain the AI ​​generated image pic6; among them, the part input to the AI ​​large model can reduce the main image area based on the original mask map, and accordingly, the non-main image area will become larger; for the part input to the Control Net, the main image area can be expanded based on the original mask map, and accordingly, the non-main image area will be reduced, so as to set different control weights for each area to avoid a single background image process and avoid edge diffusion in the main image;

[0148] 410: Fuse the AI-generated image pic6 with the backed-up design base image pic3 to obtain the final generated image pic7, which can be saved on the server.

[0149] 411: Users can obtain the specific download image pic8 according to the download address provided by the server.

[0150] In short, through the embodiments of the present application, the main image can be segmented from the image to be processed specified by the user, and the AI ​​large model can be used to generate a background image for the main image while keeping the main image unchanged. In the process of generating the background image, in order to achieve a more natural fusion of the edge of the main image with the background image, the AI ​​large model can be used to generate a "larger circle" background image. After the "larger circle" background image is added to the design base map to obtain the AI ​​generated image, the backup design base map can be further covered on the AI ​​generated image through image fusion technology. In this way, the edge of the junction between the background image generated by the AI ​​large model and the main image will be blocked by the main image. Even if the above-mentioned edge of the background image contains pure colors or speculative content generated by the AI ​​large model, it will not be displayed. Therefore, a more natural fusion between the main image and the background image can be achieved, and a higher quality generated image can be obtained.

[0151] In a preferred implementation, after the main image is segmented from the image to be processed, "white edges" and "jaggies" can be removed to further improve the final image quality. When removing "white edges," the transparency channel pixel values ​​of the pixels within the "white edge" area are raised to the Nth power, allowing for automated "white edge" removal.

[0152] In addition, in order to make the background image generated by the AI ​​large model interact better with the main image in terms of contour, light and shadow relationship, etc., the plug-in program of the AI ​​large model can also be used to control the generation process of the background image. In addition, in order to avoid the edge diffusion of the main image or the single background image caused by setting the control weight too low or too high, the control logic of the plug-in program has also been improved. That is, instead of using the same control right for the entire image, it can be differentiated. For areas outside the main image, a lower control weight can be used for control, so that this control acts more on the part of the area where the main image is located, avoiding the AI ​​large model from generating an overly simplistic background image and reducing the probability of partial edge diffusion of the main image. At the same time, in order to further avoid the edge diffusion of the main image, the main image area read by the control plug-in can be made slightly larger than the actual main image, so as to better solve the problem of edge diffusion of the main image.

[0153] Example 2

[0154] In the aforementioned embodiment 1, it is mentioned that in the process of background image generation by the AI ​​large model, the background image generation process can be controlled by the plug-in program, so that the generated background image can better interact with the main image in terms of contour, light and shadow relationship, depth, etc. At the same time, in the control process, a lower control weight can be used for areas outside the main image area to avoid the generated background image being too single; in addition, the read main image area can be slightly larger than the actual main image to avoid causing edge diffusion of the main image. This method can also exist independently, so it is separately protected in this embodiment 2. Specifically, this embodiment 2 provides an image generation processing method, see Figure 8, the method may include:

[0155] S801: Determine an image to be processed, and segment a target subject image from the image to be processed;

[0156] S802: Generate a design base image with a transparent background for the target subject image;

[0157] S803: In the process of the AI ​​large language module completing the background image for the transparent background area in the design base map, the generation process of the background image is controlled, and in the control process, the area of ​​the target main image is read, and for the area outside the target main image, a lower control weight relative to the area where the target main image is located is used for control; wherein, the area of ​​the target main image read is larger than the actual area of ​​the target main image.

[0158] In a specific implementation, the area of ​​the target subject image can be read according to the mask image corresponding to the design base image;

[0159] The mask image is generated by: generating an original mask image according to the position and size of the target subject image in the design base image, wherein the original mask image includes a first area corresponding to the area where the target subject image is located in the design base image, and the portion outside the first area is a second area;

[0160] By performing a second processing on the original mask image, the first area is enlarged and the second area is reduced, so that the corresponding area of ​​the target subject image is read according to the mask image obtained after the second processing.

[0161] Example 3

[0162] In the above-mentioned first embodiment, the "white edge" problem that may exist after the main image is segmented is also mentioned, and a corresponding solution is provided in the embodiment of the present application. This solution can also be applied to other scenarios outside the embodiment of the present application. Therefore, this solution is separately protected in this third embodiment. Specifically, this third embodiment also provides an image segmentation processing method, see Figure 9, which can specifically include:

[0163] S901: Determine a target subject image segmented from an original image;

[0164] S902: If there are additional white edges around the segmented target subject image, starting from the outermost side of the white edges, pixels within the target width range are selected inward as pixels to be processed;

[0165] S903: Raising the pixel value of the pixel to be processed on the transparency channel to the Nth power to remove or weaken the white edge; wherein N is a natural number greater than 1.

[0166] That is to say, no matter what application scenario is used or what algorithm is used to segment the main image, as long as there is a circle of "white edge" around the segmented main image, the method provided in the embodiment of the present application can be used to remove or weaken this "white edge".

[0167] For the parts not described in detail in Examples 2 and 3, please refer to the description in Example 1 and other parts of this specification, and no further details will be given here.

[0168] It should be noted that the embodiments of the present application may involve the use of user data. In actual applications, user-specific personal data can be used in the scheme described herein within the scope permitted by applicable laws and regulations, subject to the requirements of applicable laws and regulations of the country where the user is located (for example, with the user's explicit consent, effective notification to the user, etc.).

[0169] Corresponding to the first embodiment, the embodiment of the present application further provides an image generation and processing device, which may include:

[0170] A subject segmentation unit, configured to determine an image to be processed and segment a target subject image from the image to be processed;

[0171] A design base image generating unit, configured to generate a design base image with a transparent background for the target subject image and perform backup processing;

[0172] An image generation unit, configured to use an artificial intelligence (AI) large language module to complete a background image for the transparent background portion of the design base image to obtain an AI-generated image, wherein the area of ​​the background image in the AI-generated image is larger than the area of ​​the transparent background in the design base image, and forms an occlusion of an edge portion of the target subject image;

[0173] A fusion processing unit is used to overlay the backup design base map onto the AI-generated image and perform image fusion processing to generate a target image, wherein the target image includes a complete target subject image, and the target subject image obscures the edge of the background image at the junction with the target subject image.

[0174] Specifically, the image generation unit may include:

[0175] Generate a mask image according to the position and size of the target subject image in the design base image, wherein the mask image includes a first area corresponding to the area where the target subject image is located in the design base image, and the portion outside the first area is a second area;

[0176] performing a first processing on the mask image so as to reduce the first area and expand the second area;

[0177] Using the design base map and the first processed mask image as input information, the artificial intelligence AI big language module is called to generate a background image, so as to fill the generated background image into the enlarged second area while keeping the image content of the reduced first area in the design base map unchanged, thereby obtaining the AI-generated image.

[0178] Specifically, Gaussian blur processing may be performed on the mask image so as to reduce the first area in the mask image and expand the second area.

[0179] Wherein, the image to be processed is an image related to a commodity;

[0180] The main body segmentation unit can be specifically used for:

[0181] Identifying the category to which the commodity included in the image to be processed belongs;

[0182] Target detection is performed in the image to be processed according to the identified commodity category, and the target main body image is segmented from the image to be processed according to the detected target area.

[0183] If there are additional white edges around the segmented target subject image, the device further includes:

[0184] A white edge processing unit is used to select pixels within a target width range starting from the outermost side of the white edge and inward as pixels to be processed; the pixel values ​​of the pixels to be processed on the transparency channel are processed to the Nth power to remove or weaken the white edge; wherein N is a natural number greater than 1.

[0185] In addition, if the segmented target subject image has jagged edges, the apparatus may further include:

[0186] The serration processing unit is used to perform Gaussian blur processing on the area where the serration edge is located, so as to remove or weaken the serration edge.

[0187] Specifically, the design base map generating unit can be used to:

[0188] Providing a canvas interface and displaying the segmented target subject image on the canvas interface;

[0189] According to the movement and / or scaling operation performed by the user on the target subject image based on the canvas interface, the position and size of the target subject image in the canvas are determined, and the design base image with a transparent background is generated.

[0190] The AI ​​large language module is further associated with a control plug-in, and the control plug-in is used to control the background image generated by the AI ​​large language module to enhance the association between the generated background image and the target subject image;

[0191] The control plug-in reads the area of ​​the target subject image during the control process, and controls the area outside the target subject image using a lower control weight than the area where the target subject image is located;

[0192] The area of ​​the target subject image read by the control plug-in is larger than the actual area of ​​the target subject image.

[0193] In addition, the device may further include:

[0194] The style scene determination unit is used to determine the target background style and / or scene description information so that when the AI ​​big language module is called, the target background style and / or scene description information is added to the input information, and the AI ​​big language module generates a background image consistent with the target background style and / or scene description information.

[0195] Corresponding to the second embodiment, the embodiment of the present application further provides an image generation and processing device, which may include:

[0196] A subject segmentation unit, configured to determine an image to be processed and segment a target subject image from the image to be processed;

[0197] A design base image generating unit, configured to generate a design base image with a transparent background for the target subject image;

[0198] A generation control unit is used to control the generation process of the background image during the process of the AI ​​large language module completing the background image for the transparent background area in the design base map, and during the control process, read the area of ​​the target subject image, and use a lower control weight relative to the area where the target subject image is located for control of the area outside the target subject image; wherein the area of ​​the target subject image read is larger than the actual area of ​​the target subject image.

[0199] Specifically, the generation control unit may read the area of ​​the target subject image according to the mask image corresponding to the design base image;

[0200] The mask image is generated by: generating an original mask image according to the position and size of the target subject image in the design base image, wherein the original mask image includes a first area corresponding to the area where the target subject image is located in the design base image, and the portion outside the first area is a second area;

[0201] By performing a second processing on the original mask image, the first area is enlarged and the second area is reduced, so that the corresponding area of ​​the target subject image is read according to the mask image obtained after the second processing.

[0202] Corresponding to the third embodiment, the embodiment of the present application further provides an image segmentation processing device, which may include:

[0203] a subject segmentation result determination unit, configured to determine a target subject image segmented from an original image;

[0204] a unit for determining pixels to be processed, configured to, if there are extra white edges around the segmented target subject image, select pixels within a target width range inward from the outermost side of the white edges as pixels to be processed;

[0205] A calculation processing unit is used to perform N-th power processing on the pixel value of the pixel to be processed on the transparency channel to remove or weaken the white edge; wherein N is a natural number greater than 1.

[0206] In addition, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of any one of the methods in the aforementioned method embodiments are implemented.

[0207] And an electronic device comprising:

[0208] one or more processors; and

[0209] A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of the method described in any one of the aforementioned method embodiments.

[0210] 10 exemplarily shows the architecture of an electronic device, which may include a processor 1010, a video display adapter 1011, a disk drive 1012, an input / output interface 1013, a network interface 1014, and a memory 1020. The processor 1010, the video display adapter 1011, the disk drive 1012, the input / output interface 1013, the network interface 1014, and the memory 1020 may be communicatively connected via a communication bus 1030.

[0211] Among them, the processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit, processor), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., to execute relevant programs to implement the technical solutions provided in this application.

[0212] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store an operating system 1021 for controlling the operation of the electronic device 1000, and a basic input and output system (BIOS) for controlling the low-level operations of the electronic device 1000. In addition, a web browser 1023, a data storage management system 1024, and an image generation and processing system 1025, etc. can also be stored. The above-mentioned image generation and processing system 1025 can be an application program that specifically implements the operations of the aforementioned steps in the embodiment of the present application. In short, when the technical solution provided in this application is implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0213] The input / output interface 1013 is used to connect input / output modules to implement information input and output. The input / output modules can be configured as components within the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0214] The network interface 1014 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WIFI, Bluetooth, etc.).

[0215] The bus 1030 comprises a pathway for transmitting information between the various components of the device (eg, the processor 1010 , the video display adapter 1011 , the disk drive 1012 , the input / output interface 1013 , the network interface 1014 , and the memory 1020 ).

[0216] It should be noted that although the above device only shows the processor 1010, the video display adapter 1011, the disk drive 1012, the input / output interface 1013, the network interface 1014, the memory 1020, the bus 1030, etc., in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it will be understood by those skilled in the art that the above device may also include only the components necessary to implement the solution of the present application, and does not necessarily include all the components shown in the figure.

[0217] Through the description of the above embodiments, it can be seen that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present application or certain parts of the embodiments.

[0218] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0219] The above describes in detail the image generation and processing method and electronic device provided by this application. Specific examples are used herein to illustrate the principles and implementation methods of this application. The description of the above embodiments is intended only to help understand the method and core concept of this application. At the same time, those skilled in the art will appreciate that the specific implementation methods and scope of application may vary based on the concept of this application. In summary, the contents of this specification should not be construed as limiting this application.

Claims

1. An image generation processing method, characterized in that: include: Determine an image to be processed, and segment a target subject image from the image to be processed; Generating a design base map with a transparent background for the target subject image and performing backup processing; Using an artificial intelligence (AI) big language module to complete the background image for the transparent background part in the design base image, so as to obtain an AI-generated image, wherein in the AI-generated image, the area of ​​the background image is larger than the area of ​​the transparent background in the design base image, and forms an occlusion of the edge part of the target main image; The backup design base map is overlaid on the AI ​​generated image and image fusion processing is performed to generate a target image, wherein the target image includes a complete target subject image, and the target subject image obscures the edge of the background image at the junction with the target subject image.

2. The method according to claim 1, characterized in that The method of using the AI ​​big language module to complete the background image for the transparent background part in the design base map includes: Generate a mask image according to the position and size of the target subject image in the design base image, wherein the mask image includes a first area corresponding to the area where the target subject image in the design base image is located, and a portion outside the first area is a second area; Performing a first processing on the mask image so that the first area is reduced and the second area is expanded; Using the design base map and the first processed mask image as input information, the AI ​​large language module is called to generate a background image, so that the generated background image is complemented to the enlarged second area while keeping the image content of the reduced first area in the design base map unchanged, so as to obtain the AI ​​generated image.

3. The method according to claim 2, characterized in that The first processing of the mask image comprises: By performing Gaussian blur processing on the mask image, the first The first area shrinks and the second area expands.

4. The method according to claim 1, characterized in that: The image to be processed is an image related to the product; The step of segmenting a target subject image from the image to be processed comprises: Identifying the category to which the commodity included in the image to be processed belongs; Target detection is performed in the image to be processed according to the identified commodity category, and the target main body image is segmented from the image to be processed according to the detected target area.

5. The method according to claim 1, characterized in that If there are additional white edges around the segmented target subject image, the method further includes: Starting from the outermost side of the white edge, pixels within the target width range are selected inward as pixels to be processed; The pixel value of the pixel to be processed on the transparency channel is raised to the Nth power to remove or weaken the white edge; wherein N is a natural number greater than 1.

6. The method according to claim 1, characterized in that If the segmented target subject image has jagged edges, the method further includes: Gaussian blur processing is performed on the area where the jagged edge is located to remove or weaken the jagged edge.

7. The method according to claim 1, characterized in that The step of generating a design base image with a transparent background for the target subject image comprises: Providing a canvas interface, and displaying the segmented target subject image on the canvas interface; According to the moving and / or scaling operation performed by the user on the target subject image based on the canvas interface, the position and size of the target subject image in the canvas are determined, and the design base image with a transparent background is generated.

8. The method according to claim 1, characterized in that: The AI ​​language module is also associated with a control plug-in, which is used to control the AI The background image generated by the large language module is controlled to enhance the association between the generated background image and the target subject image; The control plug-in reads the area of ​​the target subject image during the control process, and controls the area outside the target subject image using a lower control weight than the area where the target subject image is located; The area of ​​the target subject image read by the control plug-in is larger than the actual area of ​​the target subject image.

9. The method according to any one of claims 1 to 8, characterized in that: Also includes: Determine the target background style and / or scene description information so that when the AI ​​big language module is called, the target background style and / or scene description information is added to the input information, and the AI ​​big language module generates a background image consistent with the target background style and / or scene description information.

10. An image generation processing method, characterized in that: include: Determine an image to be processed, and segment a target subject image from the image to be processed; Generating a design base map with a transparent background for the target subject image; In the process of the AI ​​large language module completing the background image for the transparent background area in the design base map, the generation process of the background image is controlled, and in the control process, the area of ​​the target main image is read, and for the area outside the target main image, a lower control weight relative to the area where the target main image is located is used for control; wherein the area of ​​the target main image read is larger than the actual area of ​​the target main image.

11. The method according to claim 10, characterized in that The area for reading the target subject image includes: Reading the area of ​​the target subject image according to the mask image corresponding to the design base image; The mask image is generated in the following manner: an original mask image is generated according to the position and size of the target subject image in the design base image, wherein the original mask image includes a first area corresponding to the area where the target subject image in the design base image is located, and the portion outside the first area is a second area; By performing a second processing on the original mask image, the first area is enlarged and the second area is The domain is reduced so that the corresponding area of ​​the target subject image can be read according to the mask image obtained after the second processing.

12. An image segmentation processing method, characterized in that: include: Determine a target subject image segmented from an original image; If there are additional white edges around the segmented target main image, starting from the outermost side of the white edges, pixels within the target width range are selected inward as pixels to be processed; The pixel value of the pixel to be processed on the transparency channel is raised to the Nth power to remove or weaken the white edge; wherein N is a natural number greater than 1.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 12 are implemented.

14. An electronic device, characterized in that: include: one or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of the method described in any one of claims 1 to 12.

Images