Picture generation method and device, computer equipment and storage medium

Abstract

The invention belongs to the technical field of artificial intelligence and big data, and relates to a picture generation method, which comprises the following steps: in response to clicking of any region in a first target region in an original picture, generating a clicking mark corresponding to clicking in the original picture; inputting the original picture comprising the click mark into a pre-trained SAM model to generate a first mask graph corresponding to a first target area; and based on the first mask graph, processing the first target area through a pre-trained table diffusion model to generate a new picture. The invention further provides a picture generation device, computer equipment and a storage medium. In addition, the invention also relates to a block chain technology, and the generated picture can be stored in a block chain. According to the invention, the picture with a good effect can be quickly generated.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence and big data technology, and in particular to image generation methods, devices, computer equipment and storage media. Background Art

[0002] For financial business (taking insurance as an example), insurance often requires marketing, and marketing cannot do without poster production. Therefore, poster production currently has a lot of demand in marketing scenarios. Previously, it was entirely dependent on designers, which was inefficient.

[0003] For this purpose, people have designed a method to automatically generate poster images based on a neural network model. However, the existing images generated by the neural network model are not perfect, or require complex processing to generate a more perfect new image. For example, when we need to redraw part of the image, we need to paint the redraw area black and then redraw it, which is usually very time-consuming. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a method, apparatus, computer device and storage medium for generating an image, the main purpose of which is to quickly generate an image with good effect.

[0005] In order to solve the above technical problems, the present application embodiment provides a method for generating an image, which adopts the following technical solution:

[0006] A method for generating an image comprises the following steps:

[0007] In response to clicking on any area in the first target area in the original image, generating a click mark corresponding to the click in the original image;

[0008] Inputting the original image including the point selection mark into a pre-trained SAM model to generate a first mask image corresponding to the first target area;

[0009] Based on the first mask image, the first target area is processed by a pre-trained stable diffusion model to generate a new image.

[0010] Furthermore, the step of processing the first target area by using a pre-trained stable diffusion model based on the first mask image to generate a new image specifically includes:

[0011] Extracting the first target area based on the first mask image;

[0012] Extracting the first target area from the original image based on the first mask image;

[0013] Extracting a third target area that matches the first target area;

[0014] Inputting the first mask image into the pre-trained stable diffusion model to generate a redrawn first target area;

[0015] The third target area is combined with the redrawn first target area to generate the new image.

[0016] Furthermore, the step of extracting a third target area adapted to the first target area specifically includes:

[0017] Extracting basic parameters of the first target area;

[0018] Based on the basic parameters, calling the third target area adapted to the basic parameters;

[0019] Obtain a pixel position of the third target area in the original image.

[0020] Furthermore, after extracting the first target area from the original image based on the first mask image and before combining the extracted first target area with the new background to generate the new image, the method further includes:

[0021] The first mask image is saved in a png format; wherein the first mask image in the png format represents the transparency of each pixel in the image by adding an additional alpha channel.

[0022] Furthermore, the step of performing image restoration on the first target area in the original image through a pre-trained stable diffusion model based on the first mask image to obtain the new image specifically includes:

[0023] The first mask image of the first target area is input into a pre-trained first stable diffusion model to generate a redrawn first target area to obtain the new image.

[0024] Furthermore, the step of performing image restoration on the first target area in the original image based on the first mask image to obtain the new image specifically includes:

[0025] Acquire a second mask image of a second target area;

[0026] Eliminate the area overlapping with the second mask image from the first mask image to obtain a modified mask image;

[0027] The modified mask image is input into a pre-trained second stable diffusion model to generate the redrawn first target area to obtain the new image.

[0028] In order to solve the above technical problems, the embodiment of the present application further provides a picture generation device, which adopts the following technical solution:

[0029] A mark generation module, configured to generate a point selection mark corresponding to the point selection in the original image in response to a point selection of any area in the first target area in the original image;

[0030] A mask generation module, used for inputting the original image including the point selection mark into a pre-trained SAM model to generate a first mask image corresponding to the first target area;

[0031] The image generation module is used to process the first target area through a pre-trained stable diffusion model based on the first mask image to generate a new image.

[0032] Furthermore, the picture generation module includes:

[0033] an object extraction submodule, configured to extract the first object area from the original image based on the first mask image;

[0034] A background combining submodule, used for extracting a third target area adapted to the first target area;

[0035] An image redrawing submodule, used for inputting the first mask image into the pre-trained stable diffusion model to generate a redrawn first target area;

[0036] The picture generation submodule is used to combine the third target area with the redrawn first target area to generate the new picture.

[0037] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0038] A computer device comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the image generation method as described in any one of the above items when executing the computer-readable instructions.

[0039] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0040] A computer-readable storage medium having computer-readable instructions stored thereon, wherein the computer-readable instructions, when executed by a processor, implement the steps of the image generation method as described in any one of the above items.

[0041] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0042] In response to clicking on any area in the first target area in the original image, the embodiment of the present application generates a click mark corresponding to the click in the original image, inputs the original image including the click mark into a pre-trained SAM model, generates a first mask image corresponding to the first target area, and then processes the first target area through the pre-trained stable diffusion model based on the first mask image to generate a new image. The user only needs to click on any area of ​​the first target area in the original image with a mouse, and then process it in sequence through the pre-trained SAM model and the pre-trained stable diffusion model, so as to simply and quickly generate a new image with better effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0045] Figure 2 A flowchart of an embodiment of a method for generating a picture according to the present application;

[0046] Figure 3 yes Figure 2 A flowchart of a specific implementation of step S203;

[0047] Figure 4 is a structural schematic diagram of an embodiment of a picture generating device according to the present application;

[0048] Figure 5 yes Figure 4 A structural schematic diagram of a specific implementation of the picture generation module shown;

[0049] Figure 6 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0051] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0052] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0053] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0054] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0055] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0056] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0057] It should be noted that the image generation method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the image generation device is generally arranged in the server / terminal device.

[0058] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0059] Continue to refer Figure 2 , shows a flow chart of an embodiment of a method for generating a picture according to the present application. The picture generating method comprises the following steps:

[0060] Step S201 , in response to a click on any area in the first target area in the original image, a click mark corresponding to the click is generated in the original image.

[0061] In this embodiment, the electronic device (eg, Figure 1 shown Server / Terminal Terminal equipment ) can generate a new image in response to receiving a click on any area in the first target area in the original image through a wired connection or a wireless connection. It should be noted that the above wireless connection method may include but is not limited to 3G / 4G / 5G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0062] The original images in the embodiments of the present application can be images from any field as needed. For example, taking the insurance business in the financial field, insurance often requires marketing, and marketing cannot be separated from poster production. Based on most of the image generation methods in the embodiments of the present application, poster images can be intelligently generated to reduce the threshold for manual design and improve efficiency.

[0063] The above-mentioned “first target area” may be, but is not limited to, an image of a specific target object in the original image, or a mark, logo, etc. in the original image.

[0064] Specifically, the user only needs to click any place in the first target area in the original image with the mouse, and the click mark is added to the original image in response to the user's click. For example, the original image includes a person carrying a schoolbag. If the schoolbag in the image (i.e., the first target area) is to be processed, the user can manually or automatically click any area in the schoolbag, thereby adding a dot-shaped mark to the schoolbag part in the original image.

[0065] Step S202: input the original image including the selected mark into the pre-trained SAM model to generate a mask image corresponding to the first target area.

[0066] Among them, the SAM model is a segmentation model. It only requires the user to click on any place in the target area of ​​the original image with a mouse, and the model can automatically obtain the first mask map of the area to complete the image segmentation. By generating the first mask map of the first target area, it is convenient to extract and modify the image (for example, coloring), so that new images can be generated more quickly and efficiently.

[0067] Continuing based on the previous example, the original picture including dot marks in any area corresponding to the school bag is input into the pre-trained SAM model, and the first mask map corresponding to the entire school bag can be extracted and generated.

[0068] Exemplarily, taking the generation of the first mask image of the first target area - school bag in the original image as an example, the first mask image is a black and white image, in which the school bag is a white area and other areas are black.

[0069] Step S203: Based on the first mask image, the first target area is processed by a pre-trained stable diffusion model to generate a new image.

[0070] Among them, the Stable diffusion model is an open source image generation model based on the diffusion model. Based on the first mask map of the local area extracted by the SAM model, the repainting function of the Stable diffusion model can easily achieve the repainting of the local area (ie, the first target area).

[0071] In response to clicking on any area in the first target area in the original image, the embodiment of the present application generates a click mark corresponding to the click in the original image, inputs the original image including the click mark into a pre-trained SAM model, generates a first mask image corresponding to the first target area, and then processes the first target area through the pre-trained stable diffusion model based on the first mask image to generate a new image. The user only needs to click on any area of ​​the first target area in the original image with a mouse, and then process it in sequence through the pre-trained SAM model and the pre-trained stable diffusion model, so as to simply and quickly generate a new image with better effect.

[0072] In some optional implementations of this embodiment, step S203 processes the first target area based on the first mask image by using a pre-trained stable diffusion model to generate a new image, which may specifically include the following method steps:

[0073] Extracting a first target area from the original image based on the first mask image;

[0074] extracting a third target area adapted to the first target area;

[0075] Input the first mask image into the pre-trained stable diffusion model to generate a redrawn first target area;

[0076] The third target area is superimposed on the redrawn first target area to generate a new image.

[0077] The embodiment of the present application can quickly extract the first target area from the original image based on the first mask image, and then superimpose the third target area with the first target area image redrawn based on the pre-trained stable diffusion model. Any background can be replaced as needed, so as to obtain a new image with richer elements and better effects.

[0078] In some optional implementations of this embodiment, after extracting the first target area based on the first mask image and before extracting the third target area adapted to the first target area, the following method steps may also be included:

[0079] Save the first mask image in png format.

[0080] Among them, the PNG format is a four-channel format, and the PNG format supports transparency effects, which is mainly achieved by adding an additional Alpha channel. The Alpha channel is an additional information layer used to indicate the transparency of each pixel in the image.

[0081] In the embodiment of the present application, by saving the first mask image in png format, only the area corresponding to the first target area to be displayed needs to be retained, which facilitates subsequent processing of the first target area.

[0082] In some optional implementations of this embodiment, the above “extracting a third target area adapted to the first target area” may specifically include the following method steps:

[0083] Extract basic parameters of the first target area.

[0084] Specifically, the basic parameters can be set arbitrarily as needed, for example, parameters related to shape description, such as basic shape judgment parameters such as circle, ellipse, rectangle, etc., or can also include related parameters such as the diameter of the circle, the length and height of the rectangle.

[0085] A third target area adapted to the basic parameters is retrieved.

[0086] Continuing with the previous example, assuming that the first target area is a circular logo, the diameter parameter of the circular logo is extracted; then the background image is traversed to find the third target area in the background image that matches the diameter parameter of the circle.

[0087] Get the pixel position of the second target area in the original image.

[0088] Subsequently, the redrawn first target area and the third target area may be superimposed based on the pixel positions to obtain a new image.

[0089] Exemplarily, taking the original picture including a cat as an example, assuming that you want to add an eye frame special effect mark (third target area) to the cat's eyes (i.e., the first target area), extract the shape and diameter parameters of the generated eye frame special effect mark, query a new background picture including the cat, extract the eyes that match the eye frame special effect mark, and then obtain the pixel position of the center of the eye, which is used as the pixel position of the center of the special effect mark. Subsequently, the first target area redrawn based on the pre-trained stable diffusion model can be superimposed on the third target area to obtain a new picture with richer elements and better effects.

[0090] It should be noted that each new background can be superimposed with one first target area or with multiple first target areas as needed, and each first target area can be superimposed with one background or with multiple backgrounds respectively, all of which fall within the scope of protection of this application.

[0091] The embodiment of the present application matches the second target area in the corresponding background based on the basic parameters of the image marker, so that a new image can be generated by a simple method of image superposition.

[0092] In some optional implementations of this embodiment, step S203 processes the first target area based on the first mask image by using a pre-trained stable diffusion model to generate a new image, which may specifically include the following method steps:

[0093] The first mask image of the first target area is input into a pre-trained first stable diffusion model to generate a redrawn first target area to obtain a new image.

[0094] Specifically, the first target area in the original image can be restored based on various pre-trained neural network models that are currently available or will be developed in the future. For example, based on an open source diffusion model-based image generation model, the black area in the first mask image is redrawn.

[0095] The embodiment of the present application combines the SAM model with the first image restoration model, that is, inputs the first mask map of the first target area generated based on the SAM model into the pre-trained first stable diffusion model to generate a redrawn first target area to obtain a new image, thereby quickly generating images and improving the robustness of image restoration.

[0096] like Figure 3 As shown, Figure 3 yes Figure 2 A flowchart of a specific implementation of step S203

[0097] In some optional implementations of this embodiment, step S203 processes the first target area based on the first mask image by using a pre-trained stable diffusion model to generate a new image, which may specifically include the following method steps:

[0098] Step S2031, obtaining a second mask image of a second target area;

[0099] Specifically, the second target area is usually at least partially the same as the first target area. For example, the first target area is a frog crawling on a flower in a certain posture, and the second target area only has the frog of the same size, and the frog maintains the same posture.

[0100] Step S2032, removing the area overlapping with the second mask image from the first mask image to obtain a modified mask image;

[0101] In one embodiment, the first mask image and the second mask image may be superimposed, and then the overlapping area after the superposition of the two images may be removed.

[0102] Continuing with the above example, after the first mask image and the second mask image are superimposed, the frog is removed, and then the modified mask image after the frog is removed only has part of the flower.

[0103] Step S2033, input the corrected mask image into the pre-trained second stable diffusion model to generate a redrawn first target area to obtain a new image.

[0104] Specifically, the corrected mask map can be redrawn based on various pre-trained neural network models that are currently available or will be developed in the future, such as: an image generation model based on an open source diffusion model.

[0105] Continuing with the example above, since the corrected mask image only has part of the flower, we can redraw the koala crawling on the flower to get a new picture.

[0106] In the embodiment of the present application, a corrected mask image is obtained by removing the area overlapping with the second mask image from the first mask image; the corrected mask image is input into a pre-trained second stable diffusion model to generate a repaired first target area to obtain a new image; and by combining multiple first mask images, the mask of the first target area can be modified and processed to obtain a new image with richer information transformation.

[0107] It should be emphasized that in order to further ensure the privacy and security of the above-mentioned original pictures and / or new pictures, the above-mentioned original pictures and / or new pictures can also be stored in a node of a blockchain.

[0108] The blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0109] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0110] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0111] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0112] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0113] Further references Figure 4 , as a response to the above Figure 2 The present application provides an embodiment of a picture generation device, and the embodiment of the device is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0114] like Figure 4 As shown, the image generation device 400 described in this embodiment includes: a mark generation module 401, a mask generation module 402, and an image generation module 403. Among them:

[0115] The mark generation module 401 is used for generating a point selection mark corresponding to the point selection in the original picture in response to a point selection of any area in the first target area in the original picture;

[0116] The mask generation module 402 is used to input the original image including the point selection mark into the pre-trained SAM model to generate a first mask image corresponding to the first target area;

[0117] The image generation module 403 is used to process the first target area through a pre-trained stable diffusion model based on the first mask image to generate a new image.

[0118] In response to clicking on any area in the first target area in the original image, the embodiment of the present application generates a click mark corresponding to the click in the original image, inputs the original image including the click mark into a pre-trained SAM model, generates a first mask image corresponding to the first target area, and then processes the first target area through the pre-trained stable diffusion model based on the first mask image to generate a new image. The user only needs to click on any area of ​​the first target area in the original image with a mouse, and then process it in sequence through the pre-trained SAM model and the pre-trained stable diffusion model, so as to simply and quickly generate a new image with better effect.

[0119] In some optional implementations of this embodiment, the picture generation module 403 specifically includes:

[0120] The target extraction submodule is used to extract a first target area from the original image based on the first mask image;

[0121] A background combining submodule, used for extracting a third target area adapted to the first target area;

[0122] An image redrawing submodule, used for inputting the first mask image into a pre-trained stable diffusion model to generate a redrawn first target area;

[0123] The picture generation submodule is used to combine the third target area with the redrawn first target area to generate a new picture.

[0124] The embodiment of the present application can quickly extract the first target area from the original image based on the first mask image, and then superimpose the third target area with the first target area redrawn based on the pre-trained stable diffusion model, so as to obtain a new image with richer elements and better effects.

[0125] In some optional implementations of this embodiment, the image generating device may further include:

[0126] The image storage module is used to save the first mask image in png format; wherein the first target area in png format represents the transparency of each pixel in the image by adding an additional alpha channel.

[0127] In the embodiment of the present application, by saving the first mask image in png format, only the area corresponding to the first target area to be displayed needs to be retained, which facilitates subsequent processing of the first target area.

[0128] In some optional implementations of this embodiment, the background combination submodule specifically includes:

[0129] A parameter extraction unit, used to extract basic parameters of the first target area;

[0130] A background retrieving unit, used to retrieve a third target area adapted to the basic parameters;

[0131] The position acquisition unit is used to acquire the pixel position of the third target area in the new background image.

[0132] The embodiment of the present application matches the second target area in the corresponding background based on the basic parameters of the image marker, so that a new image can be generated by a simple method of image superposition.

[0133] In some optional implementations of this embodiment, the picture generation module 403 specifically includes:

[0134] The first redrawing unit is used to input the first mask image of the first target area into the pre-trained first stable diffusion model to generate a redrawn first target area to obtain a new image.

[0135] In the embodiment of the present application, the first mask image of the first target area is input into a pre-trained first stable diffusion model to generate a redrawn first target area to obtain a new image, thereby improving the robustness of image restoration.

[0136] like Figure 5 As shown, in some optional implementations of this embodiment, the picture generation module 403 specifically includes:

[0137] A second acquisition unit 4031 is used to acquire a second mask image of a second target area;

[0138] The mask correction unit 4032 is used to remove the area overlapping with the second mask image from the first mask image to obtain a corrected mask image;

[0139] The second redrawing unit 4033 is used to input the corrected mask image into the pre-trained second stable diffusion model to generate a redrawn first target area to obtain a new image.

[0140] In the embodiment of the present application, a corrected mask image is obtained by removing the area overlapping with the second mask image from the first mask image; the corrected mask image is input into a pre-trained second stable diffusion model to generate a repaired first target area to obtain a new image; and by combining multiple first mask images, the mask of the first target area can be modified and processed to obtain a new image with richer information transformation.

[0141] To solve the above technical problems, the present application also provides a computer device. Figure 6 , Figure 6 This is a basic structural block diagram of the computer device in this embodiment.

[0142] The computer device 6 includes a memory 61, a processor 62, and a network interface 63 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 6 having a memory 61, a processor 62, and a network interface 63, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital processor (DSP), an embedded device, etc.

[0143] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0144] The memory 61 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 61 can be an internal storage unit of the computer device 6, such as a hard disk or memory of the computer device 6. In other embodiments, the memory 61 can also be an external storage device of the computer device 6, such as a plug-in hard disk equipped on the computer device 6, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc. Of course, the memory 61 can also include both the internal storage unit of the computer device 6 and its external storage device. In this embodiment, the memory 61 is generally used to store the operating system and various application software installed on the computer device 6, such as computer-readable instructions of the image generation method, etc. In addition, the memory 61 can also be used to temporarily store various types of data that have been output or are to be output.

[0145] The processor 62 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip in some embodiments. The processor 62 is generally used to control the overall operation of the computer device 6. In this embodiment, the processor 62 is used to run computer-readable instructions stored in the memory 61 or process data, such as computer-readable instructions for running the image generation method.

[0146] The network interface 63 may include a wireless network interface or a wired network interface. The network interface 63 is generally used to establish a communication connection between the computer device 6 and other electronic devices.

[0147] In response to clicking on any area in the first target area in the original image, the embodiment of the present application generates a click mark corresponding to the click in the original image, inputs the original image including the click mark into a pre-trained SAM model, generates a first mask image corresponding to the first target area, and then processes the first target area through the pre-trained stable diffusion model based on the first mask image to generate a new image. The user only needs to click on any area of ​​the first target area in the original image with a mouse, and then process it in sequence through the pre-trained SAM model and the pre-trained stable diffusion model, so as to simply and quickly generate a new image with better effect.

[0148] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the image generation method as described above.

[0149] In response to clicking on any area in the first target area in the original image, the embodiment of the present application generates a click mark corresponding to the click in the original image, inputs the original image including the click mark into a pre-trained SAM model, generates a first mask image corresponding to the first target area, and then processes the first target area through the pre-trained stable diffusion model based on the first mask image to generate a new image. The user only needs to click on any area of ​​the first target area in the original image with a mouse, and then process it in sequence through the pre-trained SAM model and the pre-trained stable diffusion model, so as to simply and quickly generate a new image with better effect.

[0150] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0151] The non-Company software tools or components appearing in the embodiments of this application are merely examples and do not represent actual use.

[0152] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A method for generating an image, characterized in that: The steps include: In response to clicking on any area in the first target area in the original image, generating a click mark corresponding to the click in the original image; Inputting the original image including the point selection mark into a pre-trained SAM model to generate a first mask image corresponding to the first target area; Based on the first mask image, the first target area is processed by a pre-trained stable diffusion model to generate a new image.

2. The image generation method according to claim 1, characterized in that: The step of processing the first target area based on the first mask image by using a pre-trained stable diffusion model to generate a new image specifically includes: Extracting the first target area from the original image based on the first mask image; Extracting a third target area that matches the first target area; Inputting the first mask image into the pre-trained stable diffusion model to generate a redrawn first target area; The third target area is combined with the redrawn first target area to generate the new image.

3. The image generation method according to claim 2, characterized in that: The step of extracting a third target area adapted to the first target area specifically includes: Extracting basic parameters of the first target area; Based on the basic parameters, calling the third target area adapted to the basic parameters; Obtain a pixel position of the third target area in the original image.

4. The image generation method according to claim 2 or 3, characterized in that: After extracting the first target from the original picture based on the first mask image and before combining the extracted first target with the new background, the method further includes: The first mask image is saved in a png format; wherein the first mask image in the png format represents the transparency of each pixel in the image by adding an additional alpha channel.

5. The image generation method according to any one of claims 1 to 3, characterized in that: The step of processing the first target area based on the first mask image by using a pre-trained stable diffusion model to generate a new image specifically includes: Get the pre-trained first stablediffusion model; The first mask image of the first target area is input into the pre-trained first stable diffusion model to generate a redrawn first target area to obtain the new image.

6. The image generation method according to any one of claims 1 to 3, characterized in that: The step of processing the first target area based on the first mask image by using a pre-trained stable diffusion model to generate a new image specifically includes: Acquire a second mask image of a second target area; Eliminating an area overlapping with the second mask image from the first mask image to obtain a modified mask image; The modified mask image is input into a pre-trained second stable diffusion model to generate the redrawn first target area to obtain the new image.

7. A picture generating device, characterized in that: include: A mark generation module, configured to generate a point selection mark corresponding to the point selection in the original image in response to a point selection of any area in the first target area in the original image; A mask generation module, used for inputting the original image including the point selection mark into a pre-trained SAM model to generate a first mask image corresponding to the first target area; The image generation module is used to process the first target area through a pre-trained stable diffusion model based on the first mask image to generate a new image.

8. The image generating device according to claim 7, characterized in that: The picture generation module includes: an object extraction submodule, configured to extract the first object area from the original image based on the first mask image; A background combining submodule, used for extracting a third target area adapted to the first target area; An image redrawing submodule, used for inputting the first mask image into the pre-trained stable diffusion model to generate a redrawn first target area; The picture generation submodule is used to combine the third target area with the redrawn first target area to generate the new picture.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the picture generation method according to any one of claims 1 to 6 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the picture generation method according to any one of claims 1 to 6 are implemented.

Images