446 results about "Image editing" patented technology

Systems and methods for multi-modal image editing are provided. In one aspect, a system and method for multi-modal image editing includes identifying an image, a prompt identifying an element to be added to the image, and a mask indicating a first region of the image for depicting the element. The system then generates a partially noisy image map that includes noise in the first region and image features from the image in a second region outside the first region. A diffusion model generates a composite image map based on the partially noisy image map and the prompt. In some cases, the composite image map includes the target element in the first region that corresponds to the mask.

The invention discloses an image editing method and system based on diffusion model inversion and attention optimization, and relates to the technical field of image editing, and the method comprises the steps: mapping a source image to a potential space through a pre-trained automatic encoder, and obtaining an initial noise feature; an EF noise space inversion algorithm is adopted to process the initial noise features, a high-variance noise graph is generated, and an intermediate result under each time step is obtained through a decoder; constructing an improved U-Net network based on edit-friendly feature reweighting and an edit-friendly attention mechanism, fusing the target prompt information into a denoising process of the improved U-Net network through a cross attention mechanism, and performing feature optimization based on the improved U-Net network; and reconstructing the potential features through a U-Net decoder, and generating an edited image conforming to the target prompt information. On the low-cost premise that complex model fine tuning and retraining are not needed, the method is superior in text-guided controllable editing tasks, and the image generation quality is improved.

Embodiments of the present disclosure provide an image editing method and apparatus, a device, and a storage medium. The image editing method comprises: acquiring an area to be edited in an original image and a target prompt word corresponding to said area, wherein the target prompt word is text information used for describing an expected effect of image editing; determining a target mask image on the basis of said area, and adding preset noise to said area on the basis of the target mask image by means of an image editing model to obtain a local noise image; and on the basis of the target prompt word and by means of the image editing model, performing noise prediction processing and image generation processing on an area to be edited in the local noise image, and outputting a target image on the basis of a noise prediction result and an image generation result. The embodiments of the present disclosure solve the problem that image editing methods in the related art cannot accurately modify and edit local areas, thereby improving image-text consistency and image generation quality.

Some implementations are directed to editing a source image based on a user request to edit the source image. The source image and the user request to edit the source image can be processed, using an image-editing system, to generate one or more image editing instructions. The one or more image editing instructions can indicate an image mask that edit (or preserves) one or more portions of the source image and / or can indicate a target object to be present in the edited image to replace a source object in the source image. Based on the one or more image editing instructions and source image, an edited image that shares the one or more portions with the source image and that differs from the source image by replacing the source object in the source image with the target object can be generated.

Systems and methods for performing image modification. In particular, the system can, using a rectified flow neural network, perform an image inversion and image editing process to generate a modified image that has been modified according to a conditioning input received by the system.

A method, apparatus, non-transitory computer readable medium, and system for image processing include obtaining an input image and an input mask, wherein the input mask indicates a region of the input image to be modified and generating, using a first image generation model, an intermediate result based on the input image and the input mask, wherein the intermediate result modifies the region of the input image indicated by the input mask. A second image generation model generates a synthetic image based on the input image and the intermediate result, wherein the synthetic image depicts the input image with content from the modified region at a higher level of detail than the intermediate result.

A computer system and a computer-implement method include obtaining a source image and a modification input that indicates a target edit to the source image and generating a modification encoding representing the target edit. An image generation model generates an output image that depicts the source image with the target edit based on the source image and the modification encoding. The image generation model is trained to perform a pose modification task and a part replacement task.

The invention discloses a face image reconstruction method based on semantic identity feature decoupling and consistency reservation of a diffusion model, and the method comprises the steps: 1, obtaining and preprocessing a face image set of identity labeling, and generating a face feature point distribution diagram, a semantic mask diagram and a description text; 2, multi-modal features are extracted and fused through a semantic identity extraction network; 3, carrying out noise adding and de-noising processing by utilizing a diffusion model, and combining a reconstructed network and semantic identity loss optimization; and 4, face image reconstruction is completed. According to the method, in the face image reconstruction process, the driving requirements of semantic information such as texts for image editing can be accurately captured, fine-grained semantic features and identity features are decoupled, the core identity features of the face can be effectively reserved, and loss of identity consistency caused by semantic editing is avoided; therefore, technical support is provided for application scenes with high requirements on face identity accuracy in the field of computer vision, and the reliability and practicability of face image reconstruction are improved.

The invention discloses a self-adaptive diffusion image editing method and system based on concept attention, and the method comprises the following steps: constructing a paired data set; analyzing the editing instruction, and extracting a key concept; a pre-trained T5 language model is utilized to convert the key concept into text embedding, and the text embedding is mapped to an image feature space; modifying a diffusion model based on a Transform architecture, embedding a concept attention module in an attention layer of a multi-modal diffusion converter, calculating an attention score between image features and concept embedding, and generating a concept saliency map; in the denoising process, the weight of the target area is adjusted by using the concept saliency map so as to realize accurate editing. According to the method, under the condition that the global image quality is not affected, the editing precision can be improved, interference to a non-target area is reduced, and meanwhile, reinforcement learning and real-time feedback are combined, so that the model can be adaptively optimized, and an editing result better meeting the user requirement is generated.

Described embodiments generally relate to a computer-implemented method for editing an image. The method includes accessing an image; identifying at least a first area of the image and a second area of the image; configuring a model to generate an edited image based on the first area of the image and the second area of the image, wherein the edited image comprises a first area of the edited image and a second area of the edited image; wherein the model is configured to generate the edited image such that the first area of the edited image differs from the first area of the image less than the second area of the edited image differs from the second area of the image.

The invention discloses an operation interaction method and system applied to camera image editing, and relates to the technical field of image processing, and the method comprises the steps: obtaining a voice semantic heat map, a pointing intensity map, a touch confidence map and a gazing confidence map based on a multi-modal interaction data packet, and calculating an image feature matrix at the same time; fusing into a multi-modal evidence graph through a normalized scale; performing semantic segmentation according to the image feature matrix to obtain a semantic segmentation first draft and a pixel-by-pixel category confidence coefficient, and performing position correlation weighting on the pixel-by-pixel category confidence coefficient by taking the multi-modal evidence graph as a confidence coefficient modulation factor to generate a candidate object mask sequence; and performing highlight display on the candidate object mask sequence, and performing conflict resolution and priority rearrangement in combination with the multi-mode evidence graph to generate a target object mask. According to the method, deep fusion of the interaction intention and image segmentation is realized, the precision and consistency of candidate region detection are improved, and the stability of real-time rendering and the reliability of an editing result are improved.

The present disclosure relates to systems, methods, and non-transitory computer-readable media that perform text-to-image editing using executable code generated from natural language text input. For instance, in one or more embodiments, the disclosed systems receive, from a client device, a digital image and natural language text input providing instructions for modifying the digital image. The disclosed systems also generate, using a large language model, executable action code for modifying the digital image in accordance with the instructions of the natural language text input, the executable action code being compatible with an editing application. The disclosed systems further modify the digital image by executing the executable action code via the editing application and provide the modified digital image for display via a graphical user interface of the client device.

The present disclosure describes techniques for implementing drag-based image editing. Feature maps are generated based on latent representations of an image by a first sub-model of a machine learning model. The first sub-model is configured to preserve an identity of the image. Embeddings corresponding to at least one pair of points are generated by a second sub-model of the machine learning model. Each pair of points comprises a handle point and a target point. The handle point identifies an area of the image. The target point indicates a target location to which the area is to be relocated. The feature maps and the embeddings are injected into a third sub-model of the machine learning model to guide a process of generating a target image by the third sub-model. The target image depicts the area of the image relocated at the target location.

The embodiment of the invention relates to agent-guided image editing. A method, apparatus, non-transitory computer readable medium and system for image processing includes obtaining an input image and an input mask, wherein the input mask indicates a region in the input image to be modified; and generating an intermediate result based on the input image and the input mask using the first image generation model, where the intermediate result modifies a region indicated by the input mask in the input image. The second image generation model generates a composite image based on the input image and the intermediate result, where the composite image depicts the input image at a higher level of detail with content from the modified region than the intermediate result.

The invention provides a three-dimensional scene video editing method based on point cloud guidance, and the method comprises the steps: obtaining an original video of a three-dimensional scene, and estimating the three-dimensional point cloud of the scene in a specified frame of the video and camera parameters of each video frame; determining an editing reference image of the specified frame according to the image of the specified frame, the pixel-level mask and the editing area description text; estimating the edited depth of the specified frame according to the edited reference image to obtain an edited three-dimensional point cloud corresponding to the specified frame; according to the mask of the specified frame and the pre-edit depth map and the post-edit depth map corresponding to the image of the specified frame, constructing a three-dimensional grid model used for surrounding an edit area, and transmitting the mask of the specified frame to the view angle of other frames by using the three-dimensional grid model to obtain masks of other frames; and obtaining a point cloud rendering image of each frame rendered according to the edited three-dimensional point cloud and the camera parameters of each frame, generating an image editing result of each frame according to the point cloud rendering image of each frame, the image, the mask and the editing reference image, and splicing the image editing result into an edited video.

A method for generating an image editing dataset is provided. The method may include obtaining a candidate image for inputting an AI model from among at least one candidate image. The method may include determining an editing operation to be performed by the AI model on the candidate image from among at least one editing operation. The method may include determining a base prompt and a control prompt based on inputting of the candidate image and the editing operation into the AI model, wherein the base prompt comprises base instructions for detection of at least one object within the candidate image, and the control prompt comprises control instructions relevant to the editing operation to be performed by the AI model on the candidate image. The method may include generating the image editing dataset based on the base prompt and the control prompt.

A computer-implemented method includes receiving a request for a type of output image and a prompt from a user that describes an output image. The method further includes selecting, based on the type of output image and the prompt, a machine-learning model from a set of machine-learning models. The method further includes providing the request and the prompt as input to the selected machine-learning model. The method further includes generating, by the selected machine-learning model, the output image that satisfies the request and the prompt.

The invention provides a background-reserved image editing method and device, electronic equipment and a program product. The method comprises the following steps: dividing an input image into a foreground image and a background image through a mask; wherein the foreground image comprises a foreground mark, and the background image comprises a background mark; executing an inversion step of inverting the input image into a noise space and executing a de-noising step; wherein in the inversion step, a K value and a V value of a background mark are stored in each time step and an attention layer, in the denoising step, only a foreground mark is processed, and the K value and the V value of the foreground mark are connected with the cached K value and the cached V value of the background mark; and connecting the foreground image output by the de-noising step with the background image before the execution of the inversion step to generate an edited image with a reserved background. According to the background-reserved image editing method and device, the electronic equipment and the program product provided by the invention, background-consistent image editing is realized through a non-training method, the image quality is improved, and the image editing cost is reduced.

Some implementations are directed to editing a source image based on a user request to edit the source image. The source image and the user request to edit the source image can be processed, using an image-editing system, to generate one or more image editing instructions. The one or more image editing instructions can indicate an image mask that edit (or preserves) one or more portions of the source image and / or can indicate a target object to be present in the edited image to replace a source object in the source image. Based on the one or more image editing instructions and source image, an edited image that shares the one or more portions with the source image and that differs from the source image by replacing the source object in the source image with the target object can be generated.

Some implementations are directed to editing a source image, where the source image is one generated based on processing a source natural language (NL) prompt using a Large-scale language-image (LLI) model. Those implementations edit the source image based on user interface input that indicates an edit to the source NL prompt, and optionally independent of any user interface input that specifies a mask in the source image and / or independent of any other user interface input. Some implementations of the present disclosure are additionally or alternatively directed to applying prompt-to-prompt editing techniques to editing a source image that is one generated based on a real image, and that approximates the real image.

The invention relates to the technical field of image enhancement, in particular to an image editing method, device and equipment and a storage medium, and the method comprises the steps: obtaining original input data containing an original image and corresponding scene structure information, carrying out the editing region screening processing in the original image based on the scene structure information, and obtaining a target editing region, obtaining a pre-training diffusion model, performing obstacle editing processing on the original image in combination with a preset image editing strategy and a target editing area to generate an initial enhanced image, performing consistency optimization on the initial enhanced image according to the scene structure information to obtain a target enhanced image, and generating annotation information based on the target enhanced image, and finally, carrying out quality screening processing on the target enhancement graph and the annotation information to obtain target screening data, and carrying out barrel division evaluation processing on the target screening data to obtain target enhancement data, thereby enriching training data and solving the problem of insufficient coverage of special-shaped obstacles in traditional data enhancement.

Provided is a method, performed by an electronic device, of editing an image, including obtaining an image, obtaining an edit prompt for the image, generating an edited image by using a diffusion model that uses the image and the edit prompt as input data, and outputting the edited image. The generating of the edited image comprises applying different image generation strengths to a plurality of regions in the image, based on a segmentation map representing the plurality of regions.

The invention discloses an image editing method and device and a storage medium. The method comprises the steps that diffusion processing is conducted on an original image through a diffusion model; determining the total number of noise reduction iterations by using a first model trained in advance; and in each noise reduction processing process, sampling a network module of the noise prediction network in the diffusion model by using a pre-trained second model, and carrying out noise reduction processing by using the network module selected by sampling to obtain an edited target image. By applying the scheme of the embodiment of the invention, the first model determines the total number of iterations of noise reduction, and the second model samples the network module of the noise prediction network, so that the proper number of iterations and the proper network module can be adaptively selected for different images, and the noise reduction efficiency is improved on the basis of ensuring the image quality. The complexity of a long iteration process and a network structure is greatly avoided, and the generation efficiency of image editing is effectively improved.

The invention discloses a complex instruction image editing method based on a multi-modal large language model, and the method comprises the steps: firstly constructing a data set expansion strategy based on a data set of multi-round image editing, carrying out the two-stage preprocessing, and constructing a complex instruction image editing data set; secondly, constructing a context prompt template, decoupling an editing instruction in a complex instruction image editing data set by using a multi-modal large language model to obtain a sub-editing instruction and an editing area, injecting a diffusion model with space-time perception background enhancement, including a space-time perception cross attention module and a background enhancement module, and carrying out weighted fusion on the obtained features to obtain an editing result; and an edited image is decoupled, output and edited through the variational auto-encoder. And finally, carrying out fine tuning by adopting a two-stage training strategy, and continuously training until the whole model is converged. According to the method, the instruction and background consistency of complex instruction image editing is remarkably improved, and the problems that complex instructions are neglected, background information is lost, and non-intention editing exists in an existing method are solved.

In some implementations, the techniques described herein relate to a method including: (i) identifying, by a processor, an image. (ii) receiving, by the processor, natural language instructions for editing the image, the natural language instructions including a location within the image and an editing instruction, (iii) editing, by a machine learning model executed by the processor, the location within the image based on the natural language instructions by (a) identifying a region within the image that corresponds to the location in the natural language instructions and (b) editing the identified region by applying the editing instruction to the identified region to generate an edited image, and (iv) causing, by the processor, display of the edited image.

The embodiment of the invention provides an image processing method and device, electronic equipment and a storage medium. The method comprises the steps that an image editing page is displayed, the image editing page comprises a parameter setting area and a result display area, and the parameter setting area comprises commodity images and candidate category options; in response to a selection event for the candidate category options, obtaining an object category value corresponding to a target category option, and if the object category value meets a preset category condition, displaying candidate style options in the parameter setting area; and in response to a selection event for the candidate style option, obtaining a style parameter corresponding to a target style option, generating a target image of the target object according to the style parameter, and displaying the target image in the result display area. According to the embodiment of the invention, the target image can be automatically generated based on the style parameters, the image editing difficulty is reduced, and the image generation efficiency is improved.

The present disclosure relates to systems, non-transitory computer-readable media, and methods for utilizing machine learning models to generate modified digital images. In particular, in some embodiments, the disclosed systems generate image editing directions between textual identifiers of two visual features utilizing a language prediction machine learning model and a text encoder. In some embodiments, the disclosed systems generated an inversion of a digital image utilizing a regularized inversion model to guide forward diffusion of the digital image. In some embodiments, the disclosed systems utilize cross-attention guidance to preserve structural details of a source digital image when generating a modified digital image with a diffusion neural network.

The invention discloses a spatial self-adaptive plug-and-play watermarking method and device for image editing traceability, and the method comprises the steps: obtaining the hidden space feature representation of an edited image through an image editing model according to an input original image and an editing condition; constructing a structured watermark grayscale image used for bearing image editing traceability core information, and encoding the structured watermark grayscale image into watermark potential features through a watermark encoder; based on the watermark potential features, watermark embedding of content and structure perception is realized in the submerged space, and the submerged space feature representation of the edited image with the watermark is acquired; and inputting the hidden space feature representation of the edited image into an image decoder, decoding the edited image into an edited image with a watermark, inputting the edited image with the watermark into a watermark decoder, extracting an embedded watermark grayscale image, and restoring complete traceability metadata information based on the watermark grayscale image. The device comprises a processor and a memory. According to the invention, high visual quality of the image is maintained while reliable traceability is realized.

Provided is a method and apparatus for automating personalized artificial intelligence (AI) image editing, which can generate an output into which a personal style of a user is incorporated when an image is edited by incorporating editing requirements of the user. The method includes an input step of receiving an input image and an user edit instruction from a user terminal, a personalized text encoding step of converting the user edit instruction into a personalized image editing command into which personal characteristics and preference have been incorporated, a personalized denoising step of generating an output image by editing the input image by incorporating the personalized image editing command, and an output step of outputting an edited output image.

The embodiment of the invention relates to the technical field of artificial intelligence, and discloses an advertisement material generation method and device based on a multi-modal large model, equipment and a storage medium, and the method comprises the steps: receiving an advertisement material generation command, inputting title data into a large language model according to the command, carrying out the core semantic reservation and compression processing, and outputting a prompt title; if the commodity main image has no background, inputting the commodity main image and the commodity category into a large-scale visual language model to generate a scenarized background prompt word; inputting the commodity main image and the background cue word into an image editing model for semantic fusion and synthesis to obtain a background synthesis main image; and loading the image-text advertisement template and the template information, and generating batch advertisement materials according to the image-text advertisement template, the template information, the background synthesis main image, the prompt title and the price information. Through the mode, the method is adaptive to diversified commodity characteristics, and collaborative understanding and fusion are carried out on image and text multi-modal information, so that the advertisement quality and the batch generation efficiency are improved.