Image generation method, computer terminal, storage medium, and program product

Abstract

The invention discloses an image generation method, a computer terminal, a storage medium and a program product. The method comprises the steps that an input instruction acting on an operation interface is responded, an original image corresponding to the input instruction is determined, and the original image comprises a target object; matching the original image with a plurality of background images in a database to obtain a target background image matched with the target object; a target adapter is used for guiding an image generation model, a target background image is injected into a background area of the original image to generate a target image, and the target adapter is used for providing position information of a target object in the original image, edge information of the target object and content information of the background area; and displaying the target image on the operation interface. According to the method and the apparatus, the technical problems of relatively low efficiency of graph background replacement and relatively poor visual effect of the generated image in related technologies are solved.

Description

Technical Field

[0001] The present application relates to the fields of artificial intelligence and image generation, and specifically, to an image generation method, a computer terminal, a storage medium and a program product. Background Art

[0002] In the field of image generation, computer vision and image processing technology are used to replace the original background part of the original image with a new background, that is, to replace the background of the image, so that the background of the original image becomes more beautiful, meets the needs or is more attractive. It can be widely used in application scenarios such as e-commerce product image background replacement, portrait photography background replacement, video conferencing virtual background, photography post-processing, video production and special effects, providing users with a richer and more diverse visual experience.

[0003] At present, the background replacement of images in related technologies usually requires users or relevant technical personnel to manually determine the background image, and the determined background image has a low degree of compatibility with the original image, or directly generate the background image through a network model, and the generated background image is relatively monotonous. In addition, the process of replacing the background of the original image based on the determined background image requires a large amount of manual processing tasks, resulting in low efficiency in graphic background replacement and poor visual effect of the generated image.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide an image generation method, a computer terminal, a storage medium and a program product to at least solve the technical problems of low efficiency of graphic background replacement and poor visual effect of generated images in the related art.

[0006] According to one aspect of an embodiment of the present application, an image generation method is provided, the method comprising: responding to an input instruction acting on an operation interface, determining an original image corresponding to the input instruction, wherein the original image contains a target object; matching the original image with multiple background images in a database to obtain a target background image that matches the target object; using a target adapter to guide an image generation model, injecting the target background image into a background area of ​​the original image to generate a target image, wherein the background area is used to characterize other areas in the original image except for the area where the target object is located, and the target adapter is used to provide position information of the target object in the original image, edge information of the target object, and content information of the background area; and displaying the target image on the operation interface.

[0007] According to another aspect of an embodiment of the present application, an image generation method is also provided, the method comprising: responding to an input instruction acting on an operation interface, determining an original image corresponding to the input instruction, wherein the original image contains a first product; matching the original image with multiple background images in a database to obtain a target background image matching the first product, wherein the multiple background images are used to represent the image of a second product in an e-commerce platform; using a target adapter to guide an image generation model, injecting the target background image into a background area of ​​the original image to generate a target image, wherein the background area is used to represent other areas in the original image except the area where the first product is located, and the target adapter is used to provide position information of the first product in the original image, edge information of the first product, and content information of the background area; and displaying the target image on the operation interface.

[0008] According to another aspect of an embodiment of the present application, an image generation method is also provided, the method comprising: obtaining an original image by calling a first interface, wherein the first interface includes a first parameter, the parameter value of the first parameter includes the original image, and the original image includes a target object; matching the original image with multiple background images in a database to obtain a target background image that matches the target object; using a target adapter to guide an image generation model, injecting the target background image into a background area of ​​the original image to generate a target image, wherein the background area is used to characterize other areas in the original image except for the area where the target object is located, and the target adapter is used to provide position information of the target object in the original image, edge information of the target object, and content information of the background area; outputting the target image by calling a second interface, wherein the second interface includes a second parameter, the parameter value of the second parameter includes the target image.

[0009] According to another aspect of the embodiments of the present application, a computer terminal is further provided, including: a memory storing an executable program; and a processor for running the program, wherein the method in each embodiment of the present application is executed when the program is running.

[0010] According to another aspect of an embodiment of the present application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in each embodiment of the present application.

[0011] According to another aspect of the embodiments of the present application, a computer program product is also provided, including a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present application is implemented.

[0012] According to another aspect of an embodiment of the present application, a computer program product is also provided, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present application is implemented.

[0013] According to another aspect of the embodiments of the present application, a computer program is also provided, and when the computer program is executed by a processor, the methods in the various embodiments of the present application are implemented.

[0014] In an embodiment of the present application, an image generation method is provided, the method comprising: determining an original image corresponding to the input instruction according to an input instruction of an operation interface, the original image containing a target object; then, matching the original image with a plurality of background images in a database to find a target background image matching the target object; then, using a target adapter to guide an image generation model, injecting the target background image into a background area of ​​the original image to generate a target image, in the above process, the background area is used to characterize other areas in the original image except the area where the target object is located, and the target adapter provides position information of the target object in the original image, edge information of the target object, and content information of the background area; finally, displaying the generated target image on the operation interface. It is easy to notice that the present application matches the original image with multiple background images in the database, and determines the target background image by screening multiple background images, so as to quickly find the target background image with high adaptability to the target object and rich content, thereby reducing the time and difficulty of the user to manually select the background image; based on the position information of the target object in the original image provided by the target adapter, the generation model can accurately separate the target object from the original image, based on the edge information of the target object, it can help the generation model to better fuse the target object and the background, based on the content information of the background area, it can help the generation model to better generate a background that matches the target object, and using the target adapter to guide the image generation model can improve the accuracy and effect of image background replacement or background generation, and can effectively guide the image generation model to inject the target background image into the background area of ​​the original image, ensuring that the visual effect of the replaced image is more natural and realistic, and the generated target image is more natural and realistic, which can effectively improve the efficiency and quality of image processing, thereby solving the technical problems of low efficiency of graphic background replacement and poor visual effect of generated image in related technologies.

[0015] It is easy to notice that the above general description and the following detailed description are only for the purpose of exemplifying and explaining the present application, and do not constitute a limitation of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0017] Figure 1 is a schematic diagram of an application scenario of an image generation method according to an embodiment of the present application;

[0018] Figure 2 is a schematic diagram of a computer terminal as a computing node in a computing environment according to an embodiment of the present application;

[0019] Figure 3 is a flow chart of an image generating method according to an embodiment of the present application;

[0020] Figure 4 is a flowchart of an optional method for generating a commodity image according to an embodiment of the present application;

[0021] Figure 5 It is a schematic diagram of an optional method for generating a product image effect based on an image generation method according to an embodiment of the present application;

[0022] Figure 6 is a flowchart of an optional image generation method according to an embodiment of the present application;

[0023] Figure 7 is a flowchart of another optional image generation method according to an embodiment of the present application;

[0024] Figure 8 is a schematic diagram of an image generating device according to an embodiment of the present application;

[0025] Fig. 9 is a schematic diagram of an optional image generating device according to an embodiment of the present application;

[0026] Fig.10 is a schematic diagram of another optional image generating device according to an embodiment of the present application;

[0027] Fig.11 It is a structural block diagram of a computer terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following explanations:

[0031] The Stable Diffusion model (SD) can be an image generation model based on deep learning and is widely used in the field of AI painting.

[0032] The Intelligent Personalized Adapter Model (IP-Adapter) is an effective and lightweight adapter that can be used to implement the image prompting function of the pre-trained text-to-image diffusion model. In the SD model, the IP-Adapter controller can be used to implement style transfer and artistic processing, such as injecting the style of the reference image into the generated image, or injecting a feature of the reference image (such as a face) into the generated image.

[0033] The multimodal large-scale learning model (Contrastive Language–Image Pre-training, referred to as CLIP) can be a multimodal pre-trained neural network that can process text and images at the same time. The core idea of ​​this model is to use a large amount of paired data of images and texts for pre-training to learn the alignment relationship between images and texts. The model has the ability of multimodal learning and can simultaneously understand information of two different modalities, image and text, and establish connections between them. The CLIP model has shown strong application potential in many fields, including image classification, image retrieval, text generation, multimodal search, etc., and is particularly suitable for zero-sample learning tasks, that is, the model does not need to see new training examples of images or texts to make predictions.

[0034] According to an embodiment of the present application, an image generation method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0035] Considering that the machine learning model consumes a lot of computing resources of the mobile terminal, the above method provided in the embodiment of the present application can be applied to Figure 1 The application scenarios shown are not limited to these. Figure 1 In the application scenario shown, the machine learning model is deployed in the server 10, and the server 10 can be connected to one or more clients 20 through a local area network connection, a wide area network connection, an Internet connection, or other types of data networks. The client 20 here may include but is not limited to: smart phones, tablet computers, laptops, PDAs, personal computers, smart home devices, vehicle-mounted devices, etc. The client 20 can interact with the user through a graphical user interface to call the large model, thereby implementing the method provided in the embodiment of the present application.

[0036] It should be noted that, when the operating resources of the client device can meet the deployment and operating conditions of the large model, the embodiments of the present application can be carried out in the client device.

[0037] Figure 2 is a schematic diagram of a computer terminal as a computing node in a computing environment according to an embodiment of the present application, such as Figure 2As shown, computing environment 201 includes multiple computing nodes (such as servers) running on a distributed network (shown as 210-1, 210-2, ...) in the figure. Different computing nodes contain local processing and memory resources, and end users 202 can remotely run applications or store data in computing environment 201. Applications can be provided as multiple services 220-1, 220-2, 220-3 and 220-4 in computing environment 201, representing services "A", "D", "E" and "H" respectively.

[0038] The end user 202 can provide and access services through a web browser or other software application on the client, and in some embodiments, the end user 202's provision and / or request can be provided to the entry gateway 230. The entry gateway 230 can include a corresponding agent to handle the provision and / or request for the service (one or more services provided in the computing environment 201).

[0039] Services are provided or deployed based on various virtualization technologies supported by the computing environment 201. In some embodiments, services can be provided based on virtual machine (VM)-based virtualization, container-based virtualization, and / or similar methods. Virtual machine-based virtualization can be to simulate a real computer by initializing a virtual machine, and execute programs and applications without directly contacting any actual hardware resources. While the virtual machine virtualizes the machine, according to container-based virtualization, a container can be started to virtualize the entire operating system (OS) so that multiple workloads can run on a single operating system instance.

[0040] In an embodiment based on container virtualization, several containers of a service can be assembled into a Pod (e.g., a Kubernetes Pod). Figure 2 As shown, service 220-2 can be equipped with one or more Pods 240-1, 240-2, ..., 240-N (collectively referred to as Pods). Pods can include a proxy 245 and one or more containers 242-1, 242-2, ..., 242-M (collectively referred to as containers). One or more containers in a Pod process requests related to one or more corresponding functions of the service, and the proxy 245 generally controls network functions related to the service, such as routing, load balancing, etc. Other services can also be equipped with similar Pods.

[0041] During operation, executing a user request from end user 202 may require invoking one or more services in computing environment 201, and executing one or more functions of one service may require invoking one or more functions of another service. Figure 2As shown, service "A" 220-1 receives a user request from end user 202 from ingress gateway 230, service "A" 220-1 may call service "D" 220-2, and service "D" 220-2 may request service "E" 220-3 to perform one or more functions.

[0042] The computing environment described above can be a cloud computing environment, where the allocation of resources is managed by the cloud service provider, allowing the development of functions without considering the implementation, adjustment or expansion of servers. The computing environment allows developers to execute code in response to events without building or maintaining complex infrastructure. Services can be divided into a set of functions that can be automatically and independently scaled, rather than expanding a single hardware device to handle potential loads.

[0043] Under the above operating environment, this application provides Figure 3 The image generation method shown. Figure 3 FIG. 1 is a flow chart of an image generation method according to an embodiment of the present application. Figure 3 As shown, the following steps may be specifically included:

[0044] Step S302, responding to an input instruction applied to the operation interface, and determining an original image corresponding to the input instruction.

[0045] The original image contains the target object.

[0046] The above-mentioned operation interface may refer to the interface seen by the user when interacting with the computer system or application. Through the operation interface, the user can use input devices such as a mouse, keyboard or touch screen to input instructions, view information, perform operations, etc. The operation interface can be divided into command line interface and graphical user interface and other forms. The command line interface may be an interface for operation by inputting text commands, and the user can enter specific instructions to complete the operation, such as the command prompt of the operating system (Windows) and the terminal interface of the operating system (Linux). The graphical user interface may be an interface for operation through graphic elements such as buttons, menus, windows, etc., and the user can complete the operation by clicking and dragging the mouse. The specific form of the operation interface may be determined according to actual needs and is not limited here.

[0047] The above-mentioned input instructions may refer to instructions input by the user through the operation interface, and may include but are not limited to instructions input by the user using input devices such as a mouse, keyboard or touch screen. Specifically, the input designation in the present application may include but is not limited to the user selecting the original image through the operation interface, the address of the original image selected or input by the user, or the original image obtained directly by the user using the shooting device. It may also refer to the language or text information input by the user to describe the original image. The input instructions can be determined according to actual needs and are not limited here.

[0048] The above-mentioned original image may refer to an image selected or specified by the user through input instructions, which requires background replacement or background generation. For example, it may refer to a product picture or commodity picture that requires background replacement or background generation, and may not be limited to a front view, side view or top view, etc., or a real person image or cartoon character image that requires background replacement or background generation, etc., and may include a real image or an image drawn using drawing software, etc. The original image can be determined according to actual needs and is not limited here.

[0049] The above-mentioned target object may refer to the foreground object or image area when background replacement or background generation is performed in the original image. For example, it may be the actual corresponding product or commodity area in the product image or commodity image, or the character area in the real person image or cartoon character image that needs background replacement or background generation, etc. The original image may include one or more target objects, and the target object can be specifically determined according to actual needs and is not limited here.

[0050] In an optional embodiment, the system can receive input instructions generated by the user based on the operation interface. Based on the input instructions, the system can determine the original image that the user needs to replace the background or generate the background. The original image contains the target object. For example, the user can choose to import from the photo album or directly use the real-time image captured by the camera. The system can load the corresponding original image according to the user's choice. The system can also extract the target object from the original image, which may include background segmentation, edge detection, color segmentation and other technologies, so as to separate the target object from the original image, so as to facilitate subsequent replacement or generation of the background image selected by the user. For example, the user can select a product image as the original image based on the operation interface, and the system can determine the product in the original image as the target object, and then perform background replacement or background generation on the product to obtain an image containing the product and the new background; for another example, the user selects a photo containing a kitten as the original image in the operation interface, and the system can extract the kitten in the original image as the target object, and then place the kitten on the beach background and synthesize it to finally generate a new image in which the kitten looks like playing on the beach. In this way, the background replacement or background generation process of the original image is completed. The above-mentioned method of determining the original image and the target object can be determined according to actual needs and is not limited here.

[0051] Step S304: Match the original image with multiple background images in the database to obtain a target background image that matches the target object.

[0052] The above-mentioned database can be an image database pre-specified or self-developed by this application that contains relatively rich background images. In actual applications, the background images in the database can be updated to ensure the richness of the background images in the database. The database can be specifically determined according to actual needs and is not limited here.

[0053] In an optional embodiment, the original image can be matched with multiple background images in a database to obtain a target background image. Specifically, feature extraction can be performed on the original image and multiple background images in the database, such as color, texture, shape and other features, and a matching algorithm, such as a feature matching algorithm, a template matching algorithm, etc., can be used to compare the similarity between the original image and each background image to determine the target background image that matches the target object. For example, a user needs to replace the background of a product photo on an e-commerce platform to enhance the display of the product. First, the original image and multiple background images of the product can be collected, and then the original image and the background image can be matched using an image matching algorithm to find a suitable target background image, so that the successfully matched target background image can be subsequently synthesized with the original image to achieve a background replacement effect, which can improve the attractiveness and display effect of the product. The process of matching the original image with multiple background images in the database to obtain the target background image can be specifically determined according to actual needs and is not limited here.

[0054] Step S306 , using the target adapter to guide the image generation model, injecting the target background image into the background area of ​​the original image to generate the target image.

[0055] The background area is used to represent other areas in the original image except the area where the target object is located, and the target adapter is used to provide the position information of the target object in the original image, the edge information of the target object and the content information of the background area.

[0056] The above-mentioned position information may refer to the specific position coordinates of the target object in the original image. The position information may be obtained through target detection or semantic segmentation and other technologies, and is used to inform the generation model which area in the original image should be replaced with a new background.

[0057] The above-mentioned edge information may refer to the outline or boundary information of the target object. The edge information may be obtained through edge detection or boundary extraction techniques to help the generation model better integrate the target object and the new background, making the replaced image more natural and realistic.

[0058] The above-mentioned content information may refer to the content of other areas in the original image except the area where the target object is located, and may include the texture, color, lighting and other features of the background, which is used to generate a model that can well integrate the new background with other parts of the original image when replacing the background, making the entire image look more unified and harmonious.

[0059] The above-mentioned target adapter may refer to a model or algorithm for providing the position information of the target object in the original image, the edge information of the target object, and the content information of the background area. The target adapter may provide relevant information to the image generation model to guide the image generation model to generate the target image. Preferably, the target adapter may adopt an intelligent adapter model (Intelligent Personalized Adapter Model, referred to as IP-Adapter). The IP-Adapter model may be a model for describing human thinking and intelligence, comparing the working mode of human thinking and intelligence to the IP protocol and adapter in a computer network. In the IP-Adapter model, human thinking and intelligence are compared to the IP protocol for processing and transmitting information, and the adapter is similar to the human perception and cognitive system for receiving, processing and outputting information, emphasizing the information processing capability and adaptability of human thinking and intelligence. The specific type of the target adapter may be determined according to actual needs and is not limited here.

[0060] The above-mentioned image generation model may refer to a model or algorithm for generating a new image. The input of the image generation model may be the original image, the target background image and the information provided by the target adapter. The target object may be fused with the target background image to generate a new image so that the target object appears to be naturally integrated into the target background. Preferably, the image generation model may adopt a stable diffusion model (SD). The specific type of the image generation model may be determined according to actual needs and is not limited here.

[0061] In an optional embodiment, a target adapter can be used to guide an image generation model to perform background replacement. Specifically, target detection and segmentation can be performed on the original image to separate the target object from the background, which can be achieved by using a deep learning model. Then, the target adapter can be determined based on the position information, edge information and content information of the background area of ​​the target object in the original image. This can be achieved through a specific encoding method or a neural network model, and relevant information can be provided to the image generation model to guide the image generation model to generate a target image. The original image, the target background image and the information provided by the target adapter can be input into the image generation model, and the image generation model can output the target image. The image generation model can be a deep learning model such as a generative adversarial network or a variational autoencoder. The image generation model can generate a A new image is created in which the target object is placed in a new background while maintaining the style and content of the original image. The generated target image can also be post-processed, such as removing edge artifacts and smoothing transitions, to improve the quality and realism of the synthesized image. The target adapter provides the position information of the target object in the original image, so that the generation model can accurately separate the target object from the original image. The edge information of the target object provided by the target adapter can help the generation model better fuse the target object and the background. The content information of the background area can help the generation model better generate a background that matches the target object. Using the target adapter to guide the image generation model can improve the accuracy and effect of image background replacement or background generation. The generated target image is more natural and realistic, which can effectively improve the efficiency and quality of image processing.

[0062] Step S308: display the target image on the operation interface.

[0063] In an optional embodiment, the processed target image can be displayed on the operation interface, can be displayed in the form of an image, and can use an image display component, such as a picture frame or canvas, to achieve the display of the image; in the process of background replacement or generation, the user may make some adjustments and modifications, and the displayed target image can be updated in real time, which can be achieved by constantly refreshing the image display component so that the user can see the modified effect in time. By displaying the target image on the operation interface, the user can intuitively view the processed effect, make adjustments and optimizations in time, which helps to improve the user experience, increase the accuracy and efficiency of the operation, and can help the user better understand the effect of the image processing algorithm and improve the effect and quality of image processing.

[0064] In an embodiment of the present application, an image generation method is provided, the method comprising: determining an original image corresponding to the input instruction according to an input instruction of an operation interface, the original image containing a target object; then, matching the original image with a plurality of background images in a database to find a target background image matching the target object; then, using a target adapter to guide an image generation model, injecting the target background image into a background area of ​​the original image to generate a target image, in the above process, the background area is used to characterize other areas in the original image except the area where the target object is located, and the target adapter provides position information of the target object in the original image, edge information of the target object, and content information of the background area; finally, displaying the generated target image on the operation interface. It is easy to notice that the present application matches the original image with multiple background images in the database, and determines the target background image by screening multiple background images, so as to quickly find the target background image with high adaptability to the target object and rich content, thereby reducing the time and difficulty of the user to manually select the background image; based on the position information of the target object in the original image provided by the target adapter, the generation model can accurately separate the target object from the original image, based on the edge information of the target object, it can help the generation model to better fuse the target object and the background, based on the content information of the background area, it can help the generation model to better generate a background that matches the target object, and using the target adapter to guide the image generation model can improve the accuracy and effect of image background replacement or background generation, and can effectively guide the image generation model to inject the target background image into the background area of ​​the original image, ensuring that the visual effect of the replaced image is more natural and realistic, and the generated target image is more natural and realistic, which can effectively improve the efficiency and quality of image processing, thereby solving the technical problems of low efficiency of graphic background replacement and poor visual effect of generated image in related technologies.

[0065] In the above-mentioned embodiment of the present application, a target adapter is used to guide an image generation model, and a target background image is injected into the background area of ​​an original image to generate a target image, including: generating a mask image of the target object based on the position of the target object in the original image; performing edge detection on the target object in the original image to obtain an edge detection image of the target object; inputting the target background image, the mask image and the edge detection image of the target object into the target adapter, and obtaining control information output by the target adapter, wherein the control information includes position information, edge information and content information; inputting the original image into the image generation model, and using the control information to control the image generation model to generate the target image.

[0066] In an optional embodiment, a mask image of the target object can be generated based on the position of the target object in the original image. The position of the target object in the original image can be identified and extracted by an image segmentation algorithm or a deep learning model, and then a mask image of the target object is generated, that is, the pixel value of the target object can be set to 1 and the background pixel value can be set to 0. Edge detection is performed on the target object in the original image to obtain an edge detection image of the target object. The edge detection algorithm can be used to perform edge detection on the target object to obtain edge information of the target object and form an edge detection image. Then, the target background image, the mask image of the target object and the edge detection image can be input into the target adapter, and the control information output by the target adapter can be obtained. The target adapter can receive the target background image, the mask image of the target object and the edge detection image as input, and output control information, including position information, edge information and content information, for controlling the image generation model. Finally, the original image can be input into the image generation model, and the control information can be used to control the image generation model to generate the target image. The image generation model can generate the target image according to the original image and the control information to achieve the effect of background replacement or background generation.

[0067] In the above steps, the position information, edge information and content information of the target object in the original image can be extracted, so that the characteristics and shape of the target object can be more accurately retained during the background replacement or background generation process, making the generated target image more realistic and natural. The target object can be effectively separated from the background through the mask image and edge detection image of the target object, avoiding the unnatural mixing of the target object and the background during the background replacement or generation process. At the same time, the control information obtained by the target adapter can better guide the image generation model to adjust and optimize the target object when generating the target image, so that the generated target image is more consistent with the position, edge and content characteristics of the target object in the original image. The background replacement or background generation method based on the position and edge information of the target object can improve the accuracy and efficiency of image processing.

[0068] In the above-mentioned embodiment of the present application, the target adapter includes a feature extraction module, wherein the model size of the feature extraction module is larger than a preset size; the target background image, the mask image and the edge detection image of the target object are input into the target adapter, and the control information output by the target adapter is obtained, including: using the feature extraction module to perform feature extraction on the target background image to obtain background features; using the feature extraction module to perform feature extraction on the mask image of the target object to obtain mask features; using the feature extraction module to perform feature extraction on the edge detection image of the target object to obtain edge detection features; and generating control information based on the background features, the mask features and the edge detection features.

[0069] The above-mentioned feature extraction module may refer to a model for extracting abstract feature representations from input data. The feature representation may be a high-dimensional vector of data, which may capture key information and features of the data. Preferably, the feature extraction module may adopt a multimodal large-scale learning model (Contrastive Language–Image Pre-training, referred to as CLIP). The multimodal large-scale learning model may simultaneously process data of different modalities, such as images and text. The multimodal large-scale learning model learns how to map input data to a common feature space, so that data of different modalities in this feature space may be effectively represented and compared. This multimodal feature extraction module may be used for various tasks, such as image classification, image retrieval, image generation, etc. The feature extraction module may also be selected according to actual needs, which is not limited here.

[0070] The above-mentioned preset size may refer to the commonly used size of the feature extraction module in the related technology. For example, when the feature extraction module adopts a multimodal large-scale learning model (CLIP), the preset size is 224*224. At this time, the model size of the multimodal large-scale learning model can be adjusted to 336*336. The model size of the feature extraction module is set larger than the preset size, which can better capture the detailed information in the image. The model size and preset size of the feature extraction module can be determined according to actual needs and are not limited here.

[0071] In an optional embodiment, the target adapter can extract features of the target background image, the mask image of the target object, and the edge detection image, and generate control information to better adapt the target object and the background. The model size of the feature extraction module is set to be larger than the preset size, so that the detail information in the image can be better captured. The target background image, the mask image of the target object, and the edge detection image are input into the target adapter. The specific steps of the target adapter outputting the control information may include: using the feature extraction module to extract features of the target background image, and a suitable feature extraction model can be selected to extract key feature points, edge information, etc. in the image. Through the feature extraction model, a feature descriptor of the target background image can be obtained; then, the feature extraction module can be used to extract features of the mask image of the target object. The mask image can refer to a binary image that separates the target object from the original image, in which the pixel value of the target object is 1, and the background pixel value is 0. The same feature extraction algorithm is used to extract features from the mask image of the target object to obtain a feature descriptor of the target object. Then, the feature extraction module can be used to extract features from the edge detection image of the target object. The edge detection image refers to the edge information of the target object obtained by the edge detection algorithm. The feature extraction algorithm is used to extract features from the edge detection image of the target object to obtain an edge detection feature descriptor. Finally, control information can be generated based on background features, mask features and edge detection features. According to the background features, mask features and edge detection features, control information can be generated through an algorithm or model to achieve background replacement or background generation of the image. For example, a generative adversarial network can be used to generate a real background image, or the accurate segmentation of the target object can be achieved by fusing mask and edge information. Through these control information, the background replacement or background generation process of the image can be achieved.

[0072] In the above steps, the large-size model of the feature extraction module can be used to more accurately extract the features of the image, thereby improving the effect and quality of image processing. By extracting the features of the background, target object and edge, the content of the image can be more comprehensively understood, thereby better replacing or generating the background. The generated control information can more accurately guide the subsequent image processing process, making the final image effect more natural and realistic.

[0073] In the above embodiment of the present application, the original image is input into the image generation model, and the image generation model is controlled by control information to generate a target image, including: extracting features of the original image to obtain image features; extracting features of the prompt text corresponding to the confirmation instruction to obtain text features; and using the image generation model to fuse the image features, text features and control information to generate a target image.

[0074] In an optional embodiment, feature extraction can be performed on the original image, and a convolutional neural network can be used to extract the features of the image. At the same time, feature extraction can be performed on the prompt text corresponding to the confirmation instruction, and text embedding technology or a recurrent neural network can be used to extract the features of the text. Then, the image features, text features and control information can be fused. The fused features can be obtained by connecting the image features, text features and control information together and then inputting them into the generation model. Finally, the image generation model can be used to generate a target image, and a target image that meets the requirements can be generated based on the instructions and image features provided by the user to achieve personalized background replacement or background generation. By combining image features, text features and control information, the generated image content can be better controlled to improve the accuracy and authenticity of the generated image. At the same time, generation models such as generative adversarial networks can be used to generate high-quality images to meet user needs for background replacement or background generation.

[0075] In the above embodiment of the present application, the original image is matched with multiple background images in a database to obtain a target background image that matches the target object, including: performing a multi-dimensional analysis on the target object in the original image to obtain a multi-dimensional analysis result, wherein the multi-dimensional analysis result includes at least two types of information: category information, style information, and color information; based on the multi-dimensional analysis result, the original image is matched with the multiple background images to obtain the target background image.

[0076] In an optional embodiment, a multi-dimensional analysis can be performed on the target object in the original image. Specifically, the original image can be firstly subjected to target detection to determine the position and size of the target object. Then, a multi-dimensional analysis can be performed on the target object, including category information, style information, and color information. The category information can be determined by the shape and features of the target object, the style information can be determined by the texture and style of the target object, and the color information can be determined by the color distribution of the target object. Next, the original image can be matched with multiple background images in a database. Specifically, based on the results of the multi-dimensional analysis, the target object in the original image can be matched with multiple background images in the database. The similarity between the target object and each background image can be calculated, and then a background image matching the target object can be selected as the target background image. The matching process can be implemented using a machine learning algorithm or a deep learning model.

[0077] In the above steps, by performing a multi-dimensional analysis of the target object in the original image, the attributes of the target object can be determined more accurately, so that a background image matching the target object can be selected in the database to improve the quality of the replacement result; based on the category information, style information and color information of the target object, a personalized background image suitable for the target object can be selected to make the replaced image more attractive and unique, and background image matching can be performed using machine learning algorithms or deep learning models to automate background replacement tasks, reduce manual intervention and improve processing efficiency.

[0078] In the above-mentioned embodiment of the present application, based on the multi-dimensional analysis result, the original image is matched with multiple background images to obtain a target background image, including: obtaining a background image that matches the multi-dimensional analysis result from multiple background images to obtain at least one candidate background image; superimposing the image of the target object in the original image with at least one candidate background image to obtain at least one superimposed image; evaluating at least one superimposed image using an evaluation model to obtain an evaluation score of at least one superimposed image; and determining the target background image from at least one candidate background image based on the evaluation score of at least one superimposed image, wherein the evaluation score of the superimposed image corresponding to the target background image is greater than the evaluation scores of the superimposed images corresponding to other background images, and the other background images are used to represent any image other than the target background image in at least one candidate background image.

[0079] In an optional embodiment, the original image can be subjected to multi-dimensional analysis, including feature extraction of multiple dimensions such as color, texture, shape, etc., and a machine learning algorithm such as a convolutional neural network or a feature extraction algorithm can be used to obtain the multi-dimensional features of the original image; then, the obtained multi-dimensional features of the original image can be matched with the multi-dimensional features of multiple background images, and a feature matching algorithm such as nearest neighbor matching, support vector machine, etc. can be used to find a background image that matches the features of the original image, and then, according to the matching result, the background image that matches the features of the original image can be selected as a candidate background image, and the background image with a higher matching degree can be selected according to the matching degree. Then, the image of the target object in the original image can be superimposed with the candidate background image to generate a superimposed image, and an image synthesis algorithm such as image fusion can be used to superimpose the image of the target object with the background image; then, the generated superimposed image can be evaluated using an evaluation model to obtain an evaluation score of the superimposed image. The evaluation model can be an algorithm based on visual quality evaluation, and finally, the target background image can be determined from the candidate background images according to the evaluation score of the superimposed image. The background image corresponding to the superimposed image with the highest evaluation score is selected as the target background image.

[0080] In the above steps, through the multi-dimensional analysis results, the original image and the background image can be matched more carefully to ensure that a background image that is more suitable for the target object is selected, thereby improving the accuracy and precision of the match. After the image of the target object is superimposed with the candidate background image, the superimposed image is evaluated through the evaluation model, which can effectively evaluate the synthesis effect, ensure that the synthesized image quality is higher, and make the replaced image more natural and realistic.

[0081] In the above embodiment of the present application, when the multidimensional analysis result includes: category information, style information and color information, a background image matching the multidimensional analysis result is obtained from multiple background images to obtain at least one candidate background image, including: obtaining at least one first background image with the same category information from multiple background images; obtaining at least one second background image with the same style information from at least one first background image; and obtaining at least one candidate background image matching the color information from at least one second background image.

[0082] In an optional embodiment, when the multidimensional analysis result includes category information, style information and color information, a background image matching the multidimensional analysis result can be obtained from multiple background images to obtain at least one candidate background image. Specifically, first, based on the input category information, at least one first background image that is the same or similar to the category information can be screened out from multiple background images. This can be achieved through an image classification algorithm or a deep learning model, and the input category information can be compared with the category label of the background image to select a matching background image. Next, at least one second background image that is the same or similar to the style information can be obtained from at least one first background image. This can be achieved through an image feature extraction and comparison algorithm, and the style features of the first background image can be extracted and compared with the input style information to select a second background image that meets the requirements. Finally, at least one candidate background image that matches the color information can be obtained from at least one second background image. This can be achieved through a color feature extraction and matching algorithm, and the color features of the second background image can be extracted and compared with the input color information to select a candidate background image that meets the requirements.

[0083] In the above steps, by performing multi-dimensional analysis on category information, style information and color information, a background image that matches the input image features can be found more accurately, thereby improving the matching degree of background replacement; selecting a suitable background image for replacement based on the matching results of category information, style information and color information can make the replaced image more natural and realistic, and enhance the visual effect; by automatically obtaining matching background images from multiple background images, the user's operational difficulty in selecting a background image can be reduced, the user experience can be improved, the image background replacement can be made more accurate, convenient and practical, and the efficiency and quality of image processing can be improved.

[0084] In the above-mentioned embodiment of the present application, the method also includes: segmenting the target object in the original image to obtain a segmented image; matching the original image with multiple background images in a database to obtain a target background image that matches the target object, including: matching the segmented image with multiple background images to obtain a target background image; using a target adapter to guide the image generation model to inject the target background image into the background area of ​​the original image to generate a target image, including: using a target adapter to guide the image generation model to inject the target background image into the background area of ​​the original image to generate a target image.

[0085] In an optional embodiment, the target object in the original image can be segmented to obtain a segmented image, and the original image can be processed using an image segmentation algorithm to separate the target object from the background to generate a segmented image; then, the original image can be matched with multiple background images in a database to obtain a target background image that matches the target object, and the target object in the original image can be matched with multiple background images in the database through an image matching algorithm to find a target background image that matches the target object; finally, a target adapter can be used to guide an image generation model to inject the target background image into the background area of ​​the original image to generate a target image, and the target background image can be fused with a segmented image of the original image, and the target background image can be injected into the background area of ​​the original image through a target adapter-guided image generation model to generate a final target image.

[0086] In the above steps, by segmenting the target object in the original image, the outline and boundary of the target object can be accurately extracted, providing accurate target object information for subsequent background replacement; by matching the original image with multiple background images in the database, a target background image that matches the target object can be found, thereby achieving a more natural and realistic background replacement effect.

[0087] In the above-mentioned embodiment of the present application, the method also includes: obtaining multiple initial background images; screening multiple background images from the multiple initial background images according to the screening indicators of the multiple initial background images, wherein the screening indicators are used to characterize the probability that the multiple initial background images are selected within the historical time period and injected into the background of the historical image; and storing the multiple background images in a database.

[0088] In an optional embodiment, multiple initial background images can be obtained, and the initial background images can be natural background images or specific background images taken at different times and in different scenes; then, a screening index can be defined to evaluate the probability of multiple initial background images being selected and injected into the background of historical images within a historical time period, and the screening index can be determined based on factors such as selection conditions in historical data and user evaluations; then, qualified background images can be screened out from the multiple initial background images based on the screening index, and the score of each background image can be calculated, and then several images with higher scores can be selected as the final background images; finally, the multiple screened background images can be stored in a database, and the image files can be saved in the database, or the feature vectors of the images can be stored in the database for subsequent retrieval and use.

[0089] In the above steps, by obtaining multiple initial background images, the diversity of background images can be ensured, thereby providing more possibilities for selection, making the replaced image richer and more vivid; by evaluating and screening multiple initial background images through screening indicators, images that are more suitable for replacing the background can be selected, thereby improving the accuracy and naturalness of the replacement and reducing mismatches or over-processing; by storing multiple background images in a database, a background image library can be established, so that the system can automatically select suitable background images based on historical data and screening indicators, thereby improving the intelligence and automation processing capabilities of the system.

[0090] In the above embodiment of the present application, the method also includes: obtaining training data, wherein the training data includes: a training original image of a training object, a training background image, a mask image and an edge detection image of the training object, a training target image, and a training text for describing the training target image; inputting the training background image, the mask image and the edge detection image of the training object into an initial adapter, and obtaining training control information output by the initial adapter; inputting the training original image and the training text into an image generation model, and generating a training prediction image using the image generation model; adjusting the model parameters of the initial adapter based on the training prediction image and the training target image to obtain a target adapter.

[0091] In an optional embodiment, training data can be obtained. The original training image can be an image containing a training object, which can be obtained by shooting or from an image database; the training background image can be an image used to replace the background of the original image, which can be obtained by shooting or from an image database. The mask image and edge detection image of the training object can be obtained. The mask image is used to mark the position of the training object, and the edge detection image is used to detect the edge of the training object, which can be generated by an image segmentation algorithm; the training target image can be the final desired image, that is, a composite image containing the training object and a new background; the training text is used to describe the training target image, which can include image annotation information, description information, etc. During the training process, the training background image, the mask image and the edge detection image of the training object can be input into the initial adapter, and the training control information output by the initial adapter can be obtained. The original training image and the training text can be input into the image generation model, and the training prediction image can be generated using the image generation model. Based on the training prediction image and the training target image, the model parameters of the initial adapter can be adjusted to obtain the target adapter. This process can be to update the model parameters through the back propagation algorithm so that the generated image is closer to the target image.

[0092] In the above steps, by obtaining data such as the training original image, training background image, mask image and edge detection image of the training object, training target image and training text, sufficient information can be provided for training the model, so that the model can accurately identify and replace the object and background in the image. The training background image, mask image and edge detection image are input into the initial adapter, the training control information is obtained, and then the training original image and training text are input into the image generation model to generate the training prediction image. The model can be used to generate the prediction image while retaining the features of the original image. Based on the training prediction image and the training target image, the model parameters of the initial adapter are adjusted to obtain the target adapter. By adjusting the model parameters, the generated image can replace the background more accurately, and the quality and effect of image processing can be improved. Through the above steps, the technical effect of image background replacement can be achieved, and the accuracy and efficiency of image processing can be improved. By obtaining training data and adjusting model parameters, it can also adapt to different scenarios and needs and provide more personalized and professional image processing services.

[0093] The technical solution proposed in this application is explained below in combination with an optional application scenario. This application proposes a background injection method based on image library matching, which can at least solve the problem that it is difficult to find suitable background materials and the generated effect is not realistic and reasonable when making e-commerce product background images. This application uses a background image recommendation mechanism that combines multiple methods to provide a suitable and beautiful background reference for background change, and uses an innovative training method of a stable diffusion model plus an intelligent adapter model (SD+IP-Adapter) background injection model to train the model, thereby improving the realism and aesthetic performance of the generated effect.

[0094] For e-commerce merchants and designers, in order to achieve good results, they often need to frequently change the background of product promotional images. In order to solve the cost and time-consuming problems brought about by the traditional manual operation chain, and at the same time improve the authenticity and aesthetics of AI-generated images, this application proposes an innovative background injection method based on image library matching. This method can intelligently recommend background reference images that match the product style and function, and merge these reference backgrounds with product images to generate new product promotional images that are both realistic and beautiful, achieving the goal of reducing costs, improving efficiency, and quickly replicating high-quality promotional images.

[0095] The specific implementation steps may include: first, segmenting the product images uploaded by the user to separate the product bodies; then performing a multi-dimensional analysis of the product bodies, including the specific classification of the products, the color tone of the product bodies, style classification, shape feature analysis, etc., and then matching the analyzed features with an existing large-base background library, wherein the large-base background library may be a database of the present application after quality screening, high click-through rate / high conversion rate screening, style annotation, and feature analysis based on a large number of product promotional images. The background images in the database are of high quality, the pictures are relatively exquisite, and the click-through rate / conversion rate in historical performance are relatively good. In this way, new product images generated with reference to the background are likely to attract more users to click and purchase.

[0096] Among them, the matching method may include: product type matching, the user's product is consistent with the product type adapted by the background library. For example, if the product is in the category of skin care products, the candidate background library is also screened in the category of skin care products; style type matching, classifying the user's product by style, such as fresh / deep / serious / lively, etc., and the candidate background library also selects the corresponding style type library; color matching / aesthetic scoring, analyzing the color characteristics of the user's product and the color characteristics of each image in the candidate background library, recommending background images with high matching scores based on the hue matching algorithm, directly pasting the product on the background image in the background library, and sorting different combinations by using an aesthetic model, and comprehensively sorting the background recommendations based on the color matching and aesthetic scoring scores.

[0097] Optionally, product type / style type can be prioritized. For example, if the user's product image is skin care products + fresh style, the candidate library is the fresh label library in the skin care product background library. Then, each background image in the library is compared with the user's product to judge the color aesthetic score, and the background with a higher score can be selected.

[0098] According to the above matching, a background image with a higher recommendation value can be obtained, and the model (SD+IP-Adapter) trained in this application can be used for background injection, and the recommended background image can be injected into the background of the segmented product, so that the product and the background image are integrated to generate a natural and beautiful new product image. Among them, the model (SD+IP-Adapter) trained in this application is based on the open source model (IP-Adapter) training code. The following improvements are made: the CLIP model size is updated from 224 to 336, which can support larger feature inputs to obtain more original image details and show better texture; the training method can adopt the inpainting plus control network (inpainting+controlnet) mode training method, which is more in line with the logic when using reasoning, maintains the consistency of training and reasoning, and makes the reasoning effect closer to the expectation during training. At the same time, in order to adapt to this training method, the format of the training data is also different from the open source method. In addition to the final image and the description text of the image, this application also adds the mask image (mask) of the product segmentation and the edge detection image (canny) of the product for training.

[0099] In this solution, when the model (SD+IP-Adapter) is used for generation, only one background image may be referenced, or it may be determined according to actual needs, which is not limited here. The CLIP model can be used as a feature extractor of the model (SD+IP-Adapter), and can be a part of the model (SD+IP-Adapter). The CLIP model uses open source pre-trained parameters, and the parameters are fixed when the model (SD+IP-Adapter) is trained; the present application may include a process for generating a picture background, and a background similar to a recommended background image is generated for the segmented product through an inpainting operation.

[0100] The open source CLIP model can be used. CLIP is a multimodal model that can process text and image data simultaneously. This multimodal feature enables CLIP to make meaningful comparisons and predictions on text and images without special training. CLIP has zero-sample learning capabilities and can generalize on unseen labels without special training to classify them, making CLIP more flexible and adaptable when dealing with new tasks. The open source CLIP model will preprocess the image into two sizes: 224*224 and 336*336. The current model (SD+IP-Adapter) usually uses the CLIP model parameters corresponding to the 224*224 image, and the fine-tuned model (SD+IP-Adapter) is also mostly fine-tuned based on this. This application uses the CLIP model parameters corresponding to the 336*336 size to obtain more image feature information and retrain the model (SD+IP-Adapter).

[0101] The open source training model (SD+IP-Adapter) method in the related technology is usually based on the training of the Vincent graph model, while the present application is to use the model (SD+IP-Adapter) in the inpainting reasoning method. Although the model (SD+IP-Adapter) trained directly with the Vincent graph model can also achieve good results, the inpainting training method is consistent with the reasoning process of the application scenario of the present application. Using the inpainting method to train the model (SD+IP-Adapter) can better ensure the consistency and stability of the model application. The inpainting training requires the addition of a segmented mask image of the product. Similarly, during reasoning, the present application will also use a control network to control the product from extending to any unnecessary parts, so a control network is also added during training to maintain consistency, and the control network requires an edge detection image of the product. The addition of mask images and edge detection image data for training described here is to emphasize the addition of inpainting and control network training methods.

[0102] The experimental results also prove that this training method is indeed more effective for this generation process and can avoid some bad cases, such as halos around products, additional product extensions, and make the product placement more realistic, avoiding the "texture feeling". Both inpainting and inpainting are a mode of SD reasoning. When inpainting, you only need to enter a prompt word, a text description of the picture you want to generate, and other parameters to generate the picture. When inpainting, you need to enter a prompt word, an initial picture, and a mask image. This will generate an image that keeps the area under the mask image unchanged and generates other areas according to the prompt word description.

[0103] This application optimizes the training process of the background injection model (SD+IP-Adapter), increases the input size of the CLIP model during training, and combines the inpainting+controlnet mode to ensure high quality and consistency of the generated effect. This training method makes the performance in the reasoning process closer to the expectations during training, thereby ensuring that the final generated product image is more aesthetically pleasing and effectively reduces the problems of dangling and unrealistic images that appear in competing products; adopts a background image recommendation mechanism that combines multiple methods: through in-depth analysis of the specific characteristics of user products, this solution can intelligently recommend suitable options from a rich background library, significantly enhancing the realism and attractiveness of the generated image. Specific matching methods include: product type matching: ensuring that the user's products are consistent with the product types in the background library, such as skin care products will be matched with suitable skin care scenes; style type matching: classifying user products by style, such as fresh, deep, serious or lively, and recommending backgrounds of corresponding styles; color matching and aesthetic scoring: analyzing the color characteristics of user products and the color characteristics of candidate background images, recommending more suitable backgrounds through a tone matching algorithm, scoring different combinations in combination with aesthetic models, and comprehensively considering color matching and aesthetic scores to optimize the background recommendation sorting.

[0104] Innovative training method of the model (SD+IP-Adapter): The size of the CLIP model is updated from 224 to 336, which can support larger feature inputs to obtain more details of the original image and show better texture; the training method adopts the training method of the inpainting+controlnet mode, which is more in line with the logic when using reasoning, maintains the consistency of training and reasoning, and makes the reasoning effect closer to the expectation during training. At the same time, in order to adapt to this training method, the format of the training data is also different from the open source method. In addition to the final image and the description text of the image, this application also adds the mask image of the product segmentation and the edge detection image of the product for training. The model (SD+IP-Adapter) model trained in this way can improve the realism and aesthetic expression of the final generated background-changing product image, and is more helpful for merchants to promote their products.

[0105] Figure 4 is a flowchart of an optional method for generating a commodity image according to an embodiment of the present application, such as Figure 4 As shown, it includes: users uploading product pictures, performing product segmentation, performing product subject analysis, including product type, style type and color aesthetics, performing background library matching, using the target adapter to guide the image generation model to perform background injection, and generating new product promotional pictures.

[0106] Figure 5is a schematic diagram of an optional method for generating a product image effect based on an image generation method according to an embodiment of the present application, such as Figure 5 As shown, it shows a schematic diagram of the effect of users uploading products, segmenting the main image of the product, recommending background images and generating new product images.

[0107] According to an embodiment of the present application, an image generating method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0108] Figure 6 is a flow chart of an image generation method according to an embodiment of the present application, such as Figure 6 As shown, the following steps may be specifically included:

[0109] Step S602, in response to an input instruction applied to the operation interface, determining an original image corresponding to the input instruction, wherein the original image includes a first product.

[0110] In an optional embodiment, in response to an input instruction acting on an operation interface, determining that the original image corresponding to the input instruction is to implement background replacement of a product image, specifically, the user selects a product image whose background needs to be replaced on the operation interface, and based on the user's input instruction, the system can determine the original image corresponding to the input instruction, that is, an image containing the first product, which can include a process of retrieving a specified product image from a database or folder; the original image can be processed, including segmenting the product and the background, removing the background, adjusting the product position, and the like, to facilitate subsequent background replacement. Product image background replacement is widely used in e-commerce platforms, advertising, product display and other fields. By replacing the background, the product image can be made more beautiful and attractive, thereby improving the display effect and sales appeal of the product. At the same time, it can also enhance the visual effect of the product, making it easier for users to focus on the product itself.

[0111] Step S604, matching the original image with multiple background images in the database to obtain a target background image that matches the first product, wherein the multiple background images are used to represent the image of the second product in the e-commerce platform.

[0112] In an optional embodiment, the original image may be matched with multiple background images in a database. Specifically, feature extraction may be performed on the original image and the multiple background images in the database. For each background image, its similarity with the original image may be calculated. Based on the result of the similarity calculation, a background image similar to the original image may be selected as the target background image for matching. This ensures that the replaced product image is more consistent with the image of the second product in the e-commerce platform. By matching the background image, the product image may also be made more in line with the overall style and visual recognition of the e-commerce platform, thereby enhancing the brand image and user experience.

[0113] Step S606, using the target adapter to guide the image generation model, injecting the target background image into the background area of ​​the original image to generate the target image, wherein the background area is used to represent other areas in the original image except the area where the first product is located, and the target adapter is used to provide the position information of the first product in the original image, the edge information of the first product, and the content information of the background area.

[0114] In an optional embodiment, an image data set containing multiple products and corresponding backgrounds can be selected, and a target detection algorithm can be used to detect and locate the products in the original image to obtain the location information of the products. A semantic segmentation algorithm can be used to segment the products to obtain the edge information of the products. Then, the target detection and segmentation results can be used to build a target adapter model. Methods such as generative adversarial networks or conditional generative models can be used to train the image generation model using the information provided by the target adapter as a condition. New products and target backgrounds can be input into the model to generate new product images with the background replaced. The above method can be applied to the fields of product display and advertising production in e-commerce platforms. For example, in advertising production, product images of different styles can be generated according to different advertising themes and target audiences to improve advertising effectiveness and click-through rates.

[0115] Step S608: display the target image on the operation interface.

[0116] In an optional embodiment, the loaded product image can be displayed at a specified position on the operation interface, which can be achieved through the functions provided by the graphics library or the image processing library. While the target image is displayed, the user can perform interactive operations through the mouse or keyboard, such as selecting a suitable background replacement, adjusting the image position, etc. These interactive operations can be achieved by monitoring the user's input events; when the user performs an operation, the program can update the displayed target image in real time to feedback the user's operation effect. For example, when the user selects a new background replacement, the program needs to synthesize the new background image with the product image and display it. Through the above steps, the user can intuitively see the product image to be processed on the operation interface and perform related operations, thereby realizing functions such as product image background replacement. The interactive operation method allows the user to process the product image more conveniently, improving the operation efficiency and user experience.

[0117] Figure 7 is a flowchart of an optional image generation method according to an embodiment of the present application, such as Figure 7 As shown, the following steps may be specifically included:

[0118] Step S702: acquiring an original image by calling a first interface, wherein the first interface includes a first parameter, a parameter value of the first parameter includes an original image, and the original image includes a target object.

[0119] In an optional embodiment, the first interface includes a first parameter, the parameter value of the first parameter is an original image, and the original image contains a target object: through a programming language or a software development tool, a code can be written to call the first interface, the parameter value passed in the code is the original image, and then the interface is called to obtain the original image; after the interface call is successful, the original image data containing the target object can be obtained. Through the above steps, the original image can be obtained by calling the first interface, and then image processing and background replacement operations can be performed.

[0120] Step S704: Match the original image with multiple background images in the database to obtain a target background image that matches the target object.

[0121] Step S706, using the target adapter to guide the image generation model, injecting the target background image into the background area of ​​the original image to generate the target image, wherein the background area is used to characterize other areas in the original image except the area where the target object is located, and the target adapter is used to provide the position information of the target object in the original image, the edge information of the target object, and the content information of the background area.

[0122] Step S708: output the target image by calling the second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter includes the target image.

[0123] In an optional embodiment, different interfaces and algorithms can be called to implement the process of image background replacement. The second interface includes a second parameter. An interface or algorithm can be called to process the original image. The target image is output by calling the second interface, wherein the parameter value of the second parameter includes the image data of the target object. The user only needs to pass in the target image as a parameter, and the interface will automatically complete the image processing and background replacement operations, thereby simplifying the user's operation steps. The user can quickly achieve the effect of image background replacement, thereby improving the efficiency and accuracy of image processing.

[0124] According to an embodiment of the present application, an image generating device for implementing the above-mentioned image generating method is also provided. Figure 8 is a schematic diagram of an image generating device according to an embodiment of the present application, such as Figure 8 As shown, the device includes: a determination module 802 , a matching module 804 , an injection module 806 and a display module 808 .

[0125] Among them, a determination module is used to respond to an input instruction acting on an operation interface and determine an original image corresponding to the input instruction, wherein the original image contains a target object; a matching module is used to match the original image with multiple background images in a database to obtain a target background image that matches the target object; an injection module is used to use a target adapter to guide an image generation model and inject the target background image into the background area of ​​the original image to generate a target image, wherein the background area is used to characterize other areas in the original image except the area where the target object is located, and the target adapter is used to provide the position information of the target object in the original image, the edge information of the target object and the content information of the background area; a display module is used to display the target image on the operation interface.

[0126] Among them, the injection module is also used to generate a mask image of the target object based on the position of the target object in the original image; perform edge detection on the target object in the original image to obtain an edge detection image of the target object; input the target background image, the mask image of the target object and the edge detection image into the target adapter, and obtain control information output by the target adapter, wherein the control information includes position information, edge information and content information; input the original image into the image generation model, and use the control information to control the image generation model to generate the target image.

[0127] Among them, the target adapter includes a feature extraction module, wherein the model size of the feature extraction module is larger than the preset size; the injection module is also used to use the feature extraction module to perform feature extraction on the target background image to obtain background features; use the feature extraction module to perform feature extraction on the mask image of the target object to obtain mask features; use the feature extraction module to perform feature extraction on the edge detection image of the target object to obtain edge detection features; generate control information based on background features, mask features and edge detection features.

[0128] Among them, the injection module is also used to extract features from the original image to obtain image features; extract features from the prompt text corresponding to the confirmation instruction to obtain text features; and use the image generation model to fuse the image features, text features and control information to generate the target image.

[0129] Among them, the matching module is also used to perform multi-dimensional analysis on the target object in the original image to obtain a multi-dimensional analysis result, wherein the multi-dimensional analysis result includes at least two types of information: category information, style information and color information; based on the multi-dimensional analysis result, the original image is matched with multiple background images to obtain a target background image.

[0130] Among them, the matching module is also used to obtain a background image that matches the multidimensional analysis result from multiple background images to obtain at least one candidate background image; superimpose the image of the target object in the original image with at least one candidate background image to obtain at least one superimposed image; use the evaluation model to evaluate the at least one superimposed image to obtain an evaluation score of the at least one superimposed image; based on the evaluation score of the at least one superimposed image, determine the target background image from at least one candidate background image, wherein the evaluation score of the superimposed image corresponding to the target background image is greater than the evaluation scores of the superimposed images corresponding to other background images, and the other background images are used to represent any image other than the target background image in at least one candidate background image.

[0131] In which, when the multidimensional analysis result includes: category information, style information and color information, a background image matching the multidimensional analysis result is obtained from multiple background images to obtain at least one candidate background image, including: obtaining at least one first background image with the same category information from multiple background images; obtaining at least one second background image with the same style information from at least one first background image; obtaining at least one candidate background image matching the color information from at least one second background image.

[0132] Among them, the target object in the original image is segmented to obtain a segmented image; the original image is matched with multiple background images in a database to obtain a target background image that matches the target object, including: matching the segmented image with multiple background images to obtain a target background image; using a target adapter to guide the image generation model, injecting the target background image into the background area of ​​the original image to generate a target image, including: using a target adapter to guide the image generation model, injecting the target background image into the background area of ​​the original image to generate a target image.

[0133] Among them, multiple initial background images are obtained; multiple background images are screened out from the multiple initial background images according to the screening indicators of the multiple initial background images, wherein the screening indicators are used to characterize the probability that the multiple initial background images are selected within the historical time period and injected into the background of the historical image; and the multiple background images are stored in a database.

[0134] Among them, training data is obtained, wherein the training data includes: a training original image containing a training object, a training background image, a mask image and an edge detection image of the training object, a training target image, and a training text for describing the training target image; the training background image, the mask image and the edge detection image of the training object are input into an initial adapter, and training control information output by the initial adapter is obtained; the training original image and the training text are input into an image generation model, and a training prediction image is generated using the image generation model; based on the training prediction image and the training target image, the model parameters of the initial adapter are adjusted to obtain a target adapter.

[0135] According to an embodiment of the present application, an image generating method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0136] According to an embodiment of the present application, an image generating device for implementing the above-mentioned image generating method is also provided. Fig. 9 is a schematic diagram of an image generating device according to an embodiment of the present application, such as Fig. 9 As shown, the device includes: a determination module 902 , a matching module 904 , an injection module 906 and a display module 908 .

[0137] Among them, a determination module is used to respond to an input instruction acting on an operation interface and determine an original image corresponding to the input instruction, wherein the original image contains a first product; a matching module is used to match the original image with multiple background images in a database to obtain a target background image that matches the first product, wherein the multiple background images are used to represent the image of the second product in the e-commerce platform; an injection module is used to use a target adapter to guide an image generation model and inject the target background image into the background area of ​​the original image to generate a target image, wherein the background area is used to represent other areas in the original image except the area where the first product is located, and the target adapter is used to provide the position information of the first product in the original image, the edge information of the first product, and the content information of the background area; a display module is used to display the target image on the operation interface.

[0138] According to an embodiment of the present application, an image generating device for implementing the above-mentioned image generating method is also provided. Fig.10 is a schematic diagram of an image generating device according to an embodiment of the present application, such as Fig.10 As shown, the device includes: an acquisition module 1002 , a matching module 1004 , an injection module 1006 and an output module 1008 .

[0139] Among them, an acquisition module is used to acquire an original image by calling a first interface, wherein the first interface includes a first parameter, and the parameter value of the first parameter includes the original image, and the original image includes a target object; a matching module is used to match the original image with multiple background images in a database to obtain a target background image that matches the target object; an injection module is used to use a target adapter to guide an image generation model to inject the target background image into a background area of ​​the original image to generate a target image, wherein the background area is used to characterize other areas in the original image except the area where the target object is located, and the target adapter is used to provide position information of the target object in the original image, edge information of the target object, and content information of the background area; an output module is used to output the target image by calling a second interface, wherein the second interface includes a second parameter, and the parameter value of the second parameter includes the target image.

[0140] The embodiment of the present application may provide a computer terminal, which may be any computer terminal in the electronic device group. Optionally, in this embodiment, the computer terminal may also be replaced by a terminal device such as a mobile terminal.

[0141] Optionally, in this embodiment, the computer terminal may be located in at least one network device among a plurality of network devices of the computer network.

[0142] In this embodiment, the computer terminal can execute the program code in the following method.

[0143] In response to an input instruction acting on an operation interface, an original image corresponding to the input instruction is determined, wherein the original image contains a target object; the original image is matched with a plurality of background images in a database to obtain a target background image that matches the target object; a target adapter is used to guide an image generation model to inject the target background image into a background area of ​​the original image to generate a target image, wherein the background area is used to characterize other areas in the original image except for an area where the target object is located, and the target adapter is used to provide position information of the target object in the original image, edge information of the target object, and content information of the background area; and the target image is displayed on the operation interface.

[0144] Optionally, a target adapter is used to guide an image generation model, and a target background image is injected into a background area of ​​an original image to generate a target image, including: generating a mask image of the target object based on the position of the target object in the original image; performing edge detection on the target object in the original image to obtain an edge detection image of the target object; inputting the target background image, the mask image and the edge detection image of the target object into a target adapter, and obtaining control information output by the target adapter, wherein the control information includes position information, edge information and content information; inputting the original image into an image generation model, and using the control information to control the image generation model to generate the target image.

[0145] Optionally, the target adapter includes a feature extraction module, wherein a model size of the feature extraction module is larger than a preset size; a target background image, a mask image of a target object, and an edge detection image are input into the target adapter, and control information output by the target adapter is obtained, including: performing feature extraction on the target background image using the feature extraction module to obtain background features; performing feature extraction on the mask image of the target object using the feature extraction module to obtain mask features; performing feature extraction on the edge detection image of the target object using the feature extraction module to obtain edge detection features; and generating control information based on background features, mask features, and edge detection features.

[0146] Optionally, the original image is input into an image generation model, and control information is used to control the image generation model to generate a target image, including: extracting features from the original image to obtain image features; extracting features from prompt text corresponding to the confirmation instruction to obtain text features; and using the image generation model to fuse the image features, text features and control information to generate a target image.

[0147] Optionally, matching the original image with multiple background images in a database to obtain a target background image that matches the target object includes: performing a multi-dimensional analysis on the target object in the original image to obtain a multi-dimensional analysis result, wherein the multi-dimensional analysis result includes at least two of the following information: category information, style information, and color information; based on the multi-dimensional analysis result, matching the original image with the multiple background images to obtain the target background image.

[0148] Optionally, based on the multi-dimensional analysis result, the original image is matched with multiple background images to obtain a target background image, including: obtaining a background image that matches the multi-dimensional analysis result from the multiple background images to obtain at least one candidate background image; superimposing the image of the target object in the original image with at least one candidate background image to obtain at least one superimposed image; evaluating the at least one superimposed image using an evaluation model to obtain an evaluation score of the at least one superimposed image; and determining the target background image from the at least one candidate background image based on the evaluation score of the at least one superimposed image, wherein the evaluation score of the superimposed image corresponding to the target background image is greater than the evaluation scores of the superimposed images corresponding to other background images, and the other background images are used to represent any image other than the target background image in the at least one candidate background image.

[0149] Optionally, when the multidimensional analysis results include: category information, style information and color information, a background image matching the multidimensional analysis results is obtained from multiple background images to obtain at least one candidate background image, including: obtaining at least one first background image with the same category information from the multiple background images; obtaining at least one second background image with the same style information from at least one first background image; and obtaining at least one candidate background image matching the color information from at least one second background image.

[0150] Optionally, the method also includes: segmenting the target object in the original image to obtain a segmented image; matching the original image with multiple background images in a database to obtain a target background image that matches the target object, including: matching the segmented image with multiple background images to obtain a target background image; using a target adapter to guide the image generation model to inject the target background image into the background area of ​​the original image to generate a target image, including: using a target adapter to guide the image generation model to inject the target background image into the background area of ​​the original image to generate a target image.

[0151] Optionally, the method also includes: obtaining multiple initial background images; screening multiple background images from the multiple initial background images according to screening indicators of the multiple initial background images, wherein the screening indicators are used to characterize the probability that the multiple initial background images are selected within the historical time period and injected into the background of the historical image; and storing the multiple background images in a database.

[0152] Optionally, the method further includes: acquiring training data, wherein the training data includes: a training original image of a training object, a training background image, a mask image and an edge detection image of the training object, a training target image, and a training text for describing the training target image; inputting the training background image, the mask image and the edge detection image of the training object into an initial adapter, and acquiring training control information output by the initial adapter; inputting the training original image and the training text into an image generation model, and generating a training prediction image using the image generation model; and adjusting the model parameters of the initial adapter based on the training prediction image and the training target image to obtain a target adapter.

[0153] Optionally, Fig.11 is a structural block diagram of a computer terminal according to an embodiment of the present application, such as Fig.11 As shown, the computer terminal A may include: one or more (only one is shown in the figure) processors 1102, a memory 1104, a storage controller, and a peripheral interface, wherein the peripheral interface is connected to a radio frequency module, an audio module, and a display.

[0154] Among them, the memory can be used to store software programs and modules, such as program instructions / modules corresponding to the methods and devices in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implementing the methods in the above embodiments. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal A via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0155] The processor may call the information and application programs stored in the memory through the transmission device to execute the steps in each embodiment.

[0156] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0157] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0158] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods in each embodiment of the present application.

[0159] The embodiment of the present application further provides a computer-readable storage medium. Optionally, in this embodiment, the computer-readable storage medium can be used to store the program code executed by the method provided in the above embodiment.

[0160] Optionally, in this embodiment, the above storage medium may be located in any electronic device in a group of electronic devices in a computer network, or in any mobile terminal in a group of mobile terminals.

[0161] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following.

[0162] In response to an input instruction acting on an operation interface, an original image corresponding to the input instruction is determined, wherein the original image contains a target object; the original image is matched with a plurality of background images in a database to obtain a target background image that matches the target object; a target adapter is used to guide an image generation model to inject the target background image into a background area of ​​the original image to generate a target image, wherein the background area is used to characterize other areas in the original image except for an area where the target object is located, and the target adapter is used to provide position information of the target object in the original image, edge information of the target object, and content information of the background area; and the target image is displayed on the operation interface.

[0163] Optionally, a target adapter is used to guide an image generation model, and a target background image is injected into a background area of ​​an original image to generate a target image, including: generating a mask image of the target object based on the position of the target object in the original image; performing edge detection on the target object in the original image to obtain an edge detection image of the target object; inputting the target background image, the mask image and the edge detection image of the target object into a target adapter, and obtaining control information output by the target adapter, wherein the control information includes position information, edge information and content information; inputting the original image into an image generation model, and using the control information to control the image generation model to generate the target image.

[0164] Optionally, the target adapter includes a feature extraction module, wherein a model size of the feature extraction module is larger than a preset size; a target background image, a mask image of a target object, and an edge detection image are input into the target adapter, and control information output by the target adapter is obtained, including: performing feature extraction on the target background image using the feature extraction module to obtain background features; performing feature extraction on the mask image of the target object using the feature extraction module to obtain mask features; performing feature extraction on the edge detection image of the target object using the feature extraction module to obtain edge detection features; and generating control information based on background features, mask features, and edge detection features.

[0165] Optionally, the original image is input into an image generation model, and control information is used to control the image generation model to generate a target image, including: extracting features from the original image to obtain image features; extracting features from prompt text corresponding to the confirmation instruction to obtain text features; and using the image generation model to fuse the image features, text features and control information to generate a target image.

[0166] Optionally, matching the original image with multiple background images in a database to obtain a target background image that matches the target object includes: performing a multi-dimensional analysis on the target object in the original image to obtain a multi-dimensional analysis result, wherein the multi-dimensional analysis result includes at least two of the following information: category information, style information, and color information; based on the multi-dimensional analysis result, matching the original image with the multiple background images to obtain the target background image.

[0167] Optionally, based on the multi-dimensional analysis result, the original image is matched with multiple background images to obtain a target background image, including: obtaining a background image that matches the multi-dimensional analysis result from the multiple background images to obtain at least one candidate background image; superimposing the image of the target object in the original image with at least one candidate background image to obtain at least one superimposed image; evaluating the at least one superimposed image using an evaluation model to obtain an evaluation score of the at least one superimposed image; and determining the target background image from the at least one candidate background image based on the evaluation score of the at least one superimposed image, wherein the evaluation score of the superimposed image corresponding to the target background image is greater than the evaluation scores of the superimposed images corresponding to other background images, and the other background images are used to represent any image other than the target background image in the at least one candidate background image.

[0168] Optionally, when the multidimensional analysis results include: category information, style information and color information, a background image matching the multidimensional analysis results is obtained from multiple background images to obtain at least one candidate background image, including: obtaining at least one first background image with the same category information from the multiple background images; obtaining at least one second background image with the same style information from at least one first background image; and obtaining at least one candidate background image matching the color information from at least one second background image.

[0169] Optionally, the method also includes: segmenting the target object in the original image to obtain a segmented image; matching the original image with multiple background images in a database to obtain a target background image that matches the target object, including: matching the segmented image with multiple background images to obtain a target background image; using a target adapter to guide the image generation model to inject the target background image into the background area of ​​the original image to generate a target image, including: using a target adapter to guide the image generation model to inject the target background image into the background area of ​​the original image to generate a target image.

[0170] Optionally, the method also includes: obtaining multiple initial background images; screening multiple background images from the multiple initial background images according to screening indicators of the multiple initial background images, wherein the screening indicators are used to characterize the probability that the multiple initial background images are selected within the historical time period and injected into the background of the historical image; and storing the multiple background images in a database.

[0171] Optionally, the method further includes: acquiring training data, wherein the training data includes: a training original image of a training object, a training background image, a mask image and an edge detection image of the training object, a training target image, and a training text for describing the training target image; inputting the training background image, the mask image and the edge detection image of the training object into an initial adapter, and acquiring training control information output by the initial adapter; inputting the training original image and the training text into an image generation model, and generating a training prediction image using the image generation model; and adjusting the model parameters of the initial adapter based on the training prediction image and the training target image to obtain a target adapter.

[0172] The embodiment of the present application further provides a computer program product. Optionally, in this embodiment, the computer program product may include a computer program, and the computer program implements the method provided in the embodiment when executed by a processor.

[0173] The above-mentioned computer program product may refer to a software program that has been written, tested and released and can be run on a computer or other device. The computer program product may include an application, an operating system, tool software, etc., which is used to implement specific functions or solve specific problems.

[0174] The embodiments of the present application also provide a computer program product. Optionally, the computer program product may include a non-volatile computer-readable storage medium, which may be used to store a computer program, and when the computer program is executed by a processor, the method provided in the embodiments is implemented.

[0175] The above-mentioned non-volatile computer-readable storage medium may refer to a medium for storing data. The non-volatile computer-readable storage medium can keep the data from being lost when the power is off, and can be used to store long-term data, such as operating systems, applications, and user files. The non-volatile storage medium may include hard disk drives, solid-state drives, optical disks, and flash memory storage devices, etc.

[0176] The embodiment of the present application further provides a computer program. Optionally, in this embodiment, when the computer program is executed by a processor, the method provided in the above embodiment is implemented.

[0177] The above-mentioned computer program may refer to a collection of instructions used to tell a computer to perform a specific task or operation. A computer program may be written by a programmer using a specific programming language and may include algorithms, data structures, logic, and control flows. A computer program may be used for a variety of purposes, including application software, operating systems, and the like.

[0178] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0179] It should be noted that the preferred implementation scheme involved in the above embodiments of the present application is the same as the scheme provided in the above embodiments, as well as the application scenario and implementation process, but is not limited to the scheme provided in the above embodiments.

[0180] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0181] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0182] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0183] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0184] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

[0185] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An image generation method, characterized in that: include: In response to an input instruction acting on the operation interface, determining an original image corresponding to the input instruction, wherein the original image includes a target object; Matching the original image with a plurality of background images in a database to obtain a target background image that matches the target object; Using a target adapter to guide an image generation model, injecting the target background image into a background area of ​​the original image to generate a target image, wherein the background area is used to represent other areas of the original image except the area where the target object is located, and the target adapter is used to provide position information of the target object in the original image, edge information of the target object, and content information of the background area; The target image is displayed on the operation interface.

2. The method according to claim 1, characterized in that The method of using the target adapter to guide the image generation model to inject the target background image into the background area of ​​the original image to generate the target image includes: Based on the position of the target object in the original image, generating a mask image of the target object; Performing edge detection on the target object in the original image to obtain an edge detected image of the target object; Inputting the target background image, the mask image and the edge detection image of the target object into the target adapter, and acquiring control information output by the target adapter, wherein the control information includes the position information, the edge information and the content information; The original image is input into the image generation model, and the image generation model is controlled by using the control information to generate the target image.

3. The method according to claim 2, characterized in that The target adapter includes a feature extraction module, wherein a model size of the feature extraction module is larger than a preset size; The step of inputting the target background image, the mask image and the edge detection image of the target object into the target adapter, and obtaining control information output by the target adapter includes: Using the feature extraction module to extract features from the target background image to obtain background features; Using the feature extraction module to extract features from the mask image of the target object to obtain mask features; Using the feature extraction module to extract features from the edge detection image of the target object to obtain edge detection features; The control information is generated based on the background feature, the mask feature and the edge detection feature.

4. The method according to claim 2, characterized in that: The step of inputting the original image into the image generation model and using the control information to control the image generation model to generate the target image includes: Performing feature extraction on the original image to obtain image features; Extracting features of the prompt text corresponding to the confirmation instruction to obtain text features; The image generation model is used to fuse the image features, the text features and the control information to generate the target image.

5. The method according to claim 1, characterized in that The matching of the original image with a plurality of background images in a database to obtain a target background image matching the target object includes: Performing a multi-dimensional analysis on the target object in the original image to obtain a multi-dimensional analysis result, wherein the multi-dimensional analysis result includes at least two types of information: category information, style information, and color information; Based on the multi-dimensional analysis result, the original image is matched with the multiple background images to obtain the target background image.

6. The method according to claim 5, characterized in that The matching the original image with the multiple background images based on the multi-dimensional analysis result to obtain the target background image includes: Acquire a background image matching the multi-dimensional analysis result from the multiple background images to obtain at least one candidate background image; Superimposing the image of the target object in the original image with the at least one candidate background image to obtain at least one superimposed image; Using the evaluation model to evaluate the at least one superimposed image, to obtain an evaluation score of the at least one superimposed image; Based on the evaluation score of the at least one superimposed image, the target background image is determined from the at least one candidate background image, wherein the evaluation score of the superimposed image corresponding to the target background image is greater than the evaluation scores of the superimposed images corresponding to other background images, and the other background images are used to represent any one of the at least one candidate background images except the target background image.

7. The method according to claim 6, characterized in that In a case where the multidimensional analysis result includes: the category information, the style information, and the color information, obtaining a background image matching the multidimensional analysis result from the multiple background images to obtain at least one candidate background image includes: Acquire at least one first background image having the same category information as the plurality of background images; Acquire at least one second background image having the same style information as the at least one first background image; The at least one candidate background image matching the color information is obtained from the at least one second background image.

8. The method according to any one of claims 1 to 7, characterized in that The method further comprises: Segmenting the target object in the original image to obtain a segmented image; The matching of the original image with a plurality of background images in a database to obtain a target background image that matches the target object comprises: matching the segmented image with the plurality of background images to obtain the target background image; The method of using the target adapter to guide the image generation model to inject the target background image into the background area of ​​the original image to generate the target image includes: using the target adapter to guide the image generation model to inject the target background image into the background area of ​​the original image to generate the target image.

9. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Get multiple initial background images; Filtering the plurality of background images from the plurality of initial background images according to a filtering index of the plurality of initial background images, wherein the filtering index is used to represent the probability of the plurality of initial background images being selected and injected into the background of the historical image within a historical time period; The multiple background images are stored in the database.

10. The method according to any one of claims 1 to 7, characterized in that The method further comprises: Acquire training data, wherein the training data includes: a training original image containing a training object, a training background image, a mask image and an edge detection image of the training object, a training target image, and a training text for describing the training target image; Inputting the training background image, the mask image and the edge detection image of the training object into an initial adapter, and acquiring training control information output by the initial adapter; Inputting the training original image and the training text into the image generation model, and generating a training prediction image using the image generation model; Based on the training prediction image and the training target image, the model parameters of the initial adapter are adjusted to obtain the target adapter.

11. An image generation method, characterized in that: include: In response to an input instruction acting on the operation interface, determining an original image corresponding to the input instruction, wherein the original image includes a first product; Matching the original image with a plurality of background images in a database to obtain a target background image that matches the first product, wherein the plurality of background images are used to represent an image of a second product on the e-commerce platform; Using a target adapter to guide an image generation model, injecting the target background image into the background area of ​​the original image to generate a target image, wherein the background area is used to represent other areas of the original image except the area where the first product is located, and the target adapter is used to provide position information of the first product in the original image, edge information of the first product, and content information of the background area; The target image is displayed on the operation interface.

12. An image generation method, characterized in that: include: Acquire the original image by calling a first interface, wherein the first interface includes a first parameter, a parameter value of the first parameter includes the original image, and the original image includes the target object; Matching the original image with a plurality of background images in a database to obtain a target background image that matches the target object; Using a target adapter to guide an image generation model, injecting the target background image into a background area of ​​the original image to generate a target image, wherein the background area is used to represent other areas of the original image except the area where the target object is located, and the target adapter is used to provide position information of the target object in the original image, edge information of the target object, and content information of the background area; The target image is output by calling a second interface, wherein the second interface includes a second parameter, and a parameter value of the second parameter includes the target image.

13. A computer terminal, characterized in that: include: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 12 when running.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 12.

15. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 12.

Images