723 results about "Image editing" patented technology

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for enabling artificial intelligence to generate new images based on contextual data and to generate digital components based on the images. In one aspect, a method includes receiving one or more queries from a client device of a user. A digital component is selected based on the one or more queries. A customized digital component is generated by obtaining an image of an object corresponding to the selected digital component and generating, using a language model, an image editing prompt for editing the image based on digital component data related to the digital component and query data including the one or more queries and contextual data. The image and the image editing prompt are provided to an image editing model. An edited image is received and used to generate the customized digital component.

The embodiment of the invention discloses an image editing method and device, a storage medium and electronic equipment, and the method comprises the steps: obtaining a to-be-edited image and an editing instruction; generating lexical elements of the target image based on the to-be-edited image and the editing instruction by using a pre-trained multi-modal large model; intercepting visual lexical elements of the target image from the lexical elements of the target image; the hidden space representation of the target image is generated by using a pre-trained diffusion model based on the hidden space representation of the to-be-edited image and the text lexical elements and the visual lexical elements corresponding to the editing instruction, and the pre-trained diffusion model has a multi-condition guide architecture; and decoding the hidden space representation of the target image to obtain the target image. According to the method, the visual lexical elements generated by the multi-modal large model are added to serve as constraint conditions, so that the generated target image is closer to an editing instruction of a user, and the image editing effect is improved.

The invention provides a picture editing method and system based on multi-modal condition adaptation, and the method comprises the steps: obtaining a first text vector: obtaining a picture description corresponding to a picture through processing, processing the picture description through an editor, and obtaining a first text vector; obtaining a first text vector based on the picture information; a second text vector acquisition step: processing the used editing instruction to obtain a second text vector based on the editing instruction; a fusion feature acquisition step: fusing the first text vector and the second text vector through weight to obtain a fusion feature; an image potential feature code acquisition step; in the image editing step, the injected condition information is received, meanwhile, the received potential features and potential noise of the image are denoised, and the image desired by the user is gradually generated in the iterative denoising process under the guidance of the received condition information; and an image restoration step. According to the invention, the stability, controllability and accuracy of image editing can be improved.

The invention relates to the technical field of computers, in particular to a multi-mode-based three-dimensional model editing method and device, equipment and a medium. The method comprises the following steps: acquiring a multi-modal fusion image personalized generation method and a source three-dimensional model; obtaining a personalized image generation network corresponding to the source three-dimensional model, and training the personalized image generation network; based on the reference image list and the corresponding text description, multi-modal embedded information is obtained after fusion; taking the trained personalized image generation network as a skeleton, and adding a multi-personalized learning module to obtain a multi-personalized image editing network; and editing the source three-dimensional scene image according to the multi-modal embedded information by using a multi-personalized image editing network to obtain a single-view-angle edited image. The multi-mode embedded information generated by the method allows a user to input prompt information of multiple modes such as texts, images, sketches and the like, complementarity of prompt of each mode is mined, so that controllability of editing is improved, and high-quality three-dimensional model editing is realized.

Systems and methods for multi-modal image editing are provided. In one aspect, a system and method for multi-modal image editing includes identifying an image, a prompt identifying an element to be added to the image, and a mask indicating a first region of the image for depicting the element. The system then generates a partially noisy image map that includes noise in the first region and image features from the image in a second region outside the first region. A diffusion model generates a composite image map based on the partially noisy image map and the prompt. In some cases, the composite image map includes the target element in the first region that corresponds to the mask.

Certain aspects of the present disclosure provide techniques and apparatus for improved machine learning. In an example method, a first latent tensor generated during a first iteration of processing data using a denoising backbone of a diffusion machine learning model is accessed. A guidance scale is generated based on processing the first latent tensor using a guidance machine learning model. A second latent tensor is generated during a second iteration of processing data using the denoising backbone based on the first latent tensor and the first guidance scale, and an output from the diffusion machine learning model is generated based at least in part on the second latent tensor.

The invention discloses an instruction-driven personalized fashion image editing method. The method comprises the following steps: 1, defining categories of editing tasks, and constructing an original image-reference image-target image-text editing instruction quaternary data set for different editing tasks; 2, constructing a target semantic network, generating target image semantic information following an editing instruction and an original image, and taking the target image semantic information as human body semantic information of an editing model; 3, constructing a unified editing network, including constructing a semantic control network, adding a visual joint module and applying a low-rank fine tuning module, so that a plurality of editing tasks use the same framework to obtain corresponding editing capabilities; and 4, constructing a multi-task low-rank adjustment module, and enabling the framework to have the capability of aligning different editing instructions to different editing tasks through joint training. Finally, an independent and unified framework among different tasks is realized. According to the method, experiments are carried out on a constructed specific data set, and good results are obtained in both quantitative and qualitative aspects.

The invention discloses a modal language model image editing technology fusing non-perpetual culture elements. The modal language model image editing technology comprises the steps that a model is selected and trained, an LLaMA model is selected as a basis, LoRA is introduced for adaptive fine adjustment, and in this way, the model is subjected to adaptive adjustment under the condition that original parameter freezing is kept; according to the method, the understanding and reasoning capabilities in instruction editing are enhanced by combining MLLM (such as LLaVA), the MLLM can perform collaborative learning across text and image modalities and extract deep semantic information, so that the model can not only process basic instructions, but also understand complex non-perpetual culture elements, and the method has great significance in improving the understanding of the model on the non-perpetual culture elements. An enhanced bidirectional interaction mechanism is designed, and the mechanism realizes deep interaction between image and text features through a cross attention mechanism, so that the image features can be used as query and key value pairs to perform bidirectional communication with the text features, and the expression of a model in a complex non-perpetual scene is improved.

The invention discloses an image editing method and system based on diffusion model inversion and attention optimization, and relates to the technical field of image editing, and the method comprises the steps: mapping a source image to a potential space through a pre-trained automatic encoder, and obtaining an initial noise feature; an EF noise space inversion algorithm is adopted to process the initial noise features, a high-variance noise graph is generated, and an intermediate result under each time step is obtained through a decoder; constructing an improved U-Net network based on edit-friendly feature reweighting and an edit-friendly attention mechanism, fusing the target prompt information into a denoising process of the improved U-Net network through a cross attention mechanism, and performing feature optimization based on the improved U-Net network; and reconstructing the potential features through a U-Net decoder, and generating an edited image conforming to the target prompt information. On the low-cost premise that complex model fine tuning and retraining are not needed, the method is superior in text-guided controllable editing tasks, and the image generation quality is improved.

The embodiment of the invention provides an image editing and model training method and device, a storage medium and a program product. In the embodiment of the invention, an image-text editing model is provided, image editing is carried out on a target text area of an original image from a font dimension and a style dimension, guarantee is provided for the accuracy of image editing through text coding features and character morphological features in the font dimension, and the consistency with a background is kept; in the style dimension, by injecting text style features, the original image style and additional styles are kept, and the availability rate of image editing is improved. Image editing is carried out from the two dimensions of font and style, and the image quality of the obtained image can be improved. Furthermore, in the layout dimension and model training process, local and transverse and longitudinal modification of various alignment modes is supported through a layout loss function and adaptive learning layout information, and the image quality is further improved.

Electronic devices, methods, and storage mediums for editing images are provided. An image editing method comprises receiving a first user input for a first image. The method comprises determining whether the first user input indicates an instruction to add a first object. The method comprises, when the first user input indicates an instruction to add the first object, generating a first preliminary object image for the first object. The method comprises displaying a second image including the first object, the second image being generated based on the first preliminary object image and being associated with the first image. The method comprises, when a second user input indicates an instruction related to the first object is received for the second image, displaying a third image in which at least one of a size or a location of the first object is changed according to the second user input.

The embodiment of the invention provides a picture editing method, related equipment and a picture editing system. Editing types (such as deletion, dragging, replacement, addition, color adjustment and the like) and editing parameters (such as replaceable contents, dragging target positions, newly added contents, color adjustment numerical values and the like) are recommended to a user according to various features such as mask features, depth features, contour features, color features and the like of an editing area selected by the user, and an editing thought is provided for the user; moreover, whether the editing instruction is reasonably matched with the editing area selected by the user can be judged, so that the user is prompted that the editing instruction cannot be executed, the user can be recommended to select a new editing area, or the user is helped to modify the editing instruction, and the user experience is improved. The problem that the editing effect does not conform to the common sense logic due to the fact that the user randomly inputs the editing instruction is solved, so that the trial and error times when the user edits the picture can be reduced, and the picture output efficiency is remarkably improved.

Embodiments of the present disclosure provide an image editing method and apparatus, a device, and a storage medium. The image editing method comprises: acquiring an area to be edited in an original image and a target prompt word corresponding to said area, wherein the target prompt word is text information used for describing an expected effect of image editing; determining a target mask image on the basis of said area, and adding preset noise to said area on the basis of the target mask image by means of an image editing model to obtain a local noise image; and on the basis of the target prompt word and by means of the image editing model, performing noise prediction processing and image generation processing on an area to be edited in the local noise image, and outputting a target image on the basis of a noise prediction result and an image generation result. The embodiments of the present disclosure solve the problem that image editing methods in the related art cannot accurately modify and edit local areas, thereby improving image-text consistency and image generation quality.

A method, apparatus, non-transitory computer readable medium, apparatus, and system for image processing include obtaining a text prompt and an input image depicting a person, generating a latent code based on the text prompt and the input image, wherein the latent code is optimized by an identity preserving loss, and generating, using an image generator of a machine learning model, a synthetic image based on the latent code, wherein the synthetic image includes an element of the text prompt and preserves an identity of the person in the input image.

Some implementations are directed to editing a source image based on a user request to edit the source image. The source image and the user request to edit the source image can be processed, using an image-editing system, to generate one or more image editing instructions. The one or more image editing instructions can indicate an image mask that edit (or preserves) one or more portions of the source image and / or can indicate a target object to be present in the edited image to replace a source object in the source image. Based on the one or more image editing instructions and source image, an edited image that shares the one or more portions with the source image and that differs from the source image by replacing the source object in the source image with the target object can be generated.

Embodiments of the disclosure relate to interactive point-based image editing. According to example embodiments of the present disclosure, a user edit input for a source image is obtained to indicate at least one handle point and at least one target point in the source image. A feature map is extracted from the source image using a diffusion model at an iteration step of an inverse denoising diffusion process performed on the source image. The feature map is then updated based on the user edit input. Then a target image is generated based on the updated feature map using the diffusion model through a denoising diffusion process performed on the updated feature map.

Systems and methods for performing image modification. In particular, the system can, using a rectified flow neural network, perform an image inversion and image editing process to generate a modified image that has been modified according to a conditioning input received by the system.

A method, apparatus, non-transitory computer readable medium, and system for image processing include obtaining an input image and an input mask, wherein the input mask indicates a region of the input image to be modified and generating, using a first image generation model, an intermediate result based on the input image and the input mask, wherein the intermediate result modifies the region of the input image indicated by the input mask. A second image generation model generates a synthetic image based on the input image and the intermediate result, wherein the synthetic image depicts the input image with content from the modified region at a higher level of detail than the intermediate result.

A computer system and a computer-implement method include obtaining a source image and a modification input that indicates a target edit to the source image and generating a modification encoding representing the target edit. An image generation model generates an output image that depicts the source image with the target edit based on the source image and the modification encoding. The image generation model is trained to perform a pose modification task and a part replacement task.

The invention discloses a network topology image editing method, device and equipment and a readable storage medium, and is applied to the technical field of network topology images, and the method comprises the steps: recognizing a to-be-edited network topology image and network topology editing demand information, and obtaining a target editing intention and editing parameter information; editing to-be-edited network topology information corresponding to the to-be-edited network topology image by using an editing agent based on the target editing intention and the editing parameter information to obtain edited network topology information; and generating a target topological image based on the edited network topological information, and generating a target network topological image corresponding to the target topological image. Compared with the current mode that the network topological image can only be edited manually, the method has the advantages that the target editing intention and the editing parameter information are intelligently recognized, so that the editing agent is utilized to perform intelligent editing according to the target editing intention and the editing parameter information, and the topological image editing efficiency and accuracy are greatly improved.

The invention provides an image editing method and device based on a neural radiation field, equipment and a storage medium, and the method comprises the steps: obtaining a plurality of to-be-processed images, and carrying out the sampling processing of the to-be-processed images, so as to obtain the pixel information of a plurality of sampling points; pixel information of the sampling points is input into a preset neural radiation field optimization model, feature information corresponding to the sampling points is output through the preset neural radiation field optimization model, and radiation rotation invariant constraints are added into the preset neural radiation field optimization model; generating a target neural renderer according to the feature information corresponding to the sampling point; generating a three-dimensional neural point cloud segmentation mask according to the target neural renderer; in response to an editing request of a user, generating an edited image according to the three-dimensional neural point cloud segmentation mask; the technical effect of improving the image editing efficiency and editing controllability based on the neural radiation field is achieved.

A method is provided that includes: obtaining an editing instruction input by a user in a current round of a dialogue and history dialogue information in at least one history round of the dialogue, wherein the history dialogue information comprises a history dialogue text and at least one history image; determining a source image to be edited from the at least one history image based on the editing instruction and the history dialogue information; and editing the source image to generate a target image based on the editing instruction.

The invention discloses a face image reconstruction method based on semantic identity feature decoupling and consistency reservation of a diffusion model, and the method comprises the steps: 1, obtaining and preprocessing a face image set of identity labeling, and generating a face feature point distribution diagram, a semantic mask diagram and a description text; 2, multi-modal features are extracted and fused through a semantic identity extraction network; 3, carrying out noise adding and de-noising processing by utilizing a diffusion model, and combining a reconstructed network and semantic identity loss optimization; and 4, face image reconstruction is completed. According to the method, in the face image reconstruction process, the driving requirements of semantic information such as texts for image editing can be accurately captured, fine-grained semantic features and identity features are decoupled, the core identity features of the face can be effectively reserved, and loss of identity consistency caused by semantic editing is avoided; therefore, technical support is provided for application scenes with high requirements on face identity accuracy in the field of computer vision, and the reliability and practicability of face image reconstruction are improved.

The invention discloses a self-adaptive diffusion image editing method and system based on concept attention, and the method comprises the following steps: constructing a paired data set; analyzing the editing instruction, and extracting a key concept; a pre-trained T5 language model is utilized to convert the key concept into text embedding, and the text embedding is mapped to an image feature space; modifying a diffusion model based on a Transform architecture, embedding a concept attention module in an attention layer of a multi-modal diffusion converter, calculating an attention score between image features and concept embedding, and generating a concept saliency map; in the denoising process, the weight of the target area is adjusted by using the concept saliency map so as to realize accurate editing. According to the method, under the condition that the global image quality is not affected, the editing precision can be improved, interference to a non-target area is reduced, and meanwhile, reinforcement learning and real-time feedback are combined, so that the model can be adaptively optimized, and an editing result better meeting the user requirement is generated.

Described embodiments generally relate to a computer-implemented method for editing an image. The method includes accessing an image; identifying at least a first area of the image and a second area of the image; configuring a model to generate an edited image based on the first area of the image and the second area of the image, wherein the edited image comprises a first area of the edited image and a second area of the edited image; wherein the model is configured to generate the edited image such that the first area of the edited image differs from the first area of the image less than the second area of the edited image differs from the second area of the image.

Disclosed in the embodiments of the present disclosure are a video generation method, an apparatus, an electronic device and a storage medium. The method comprises: during an image editing process of subjecting an initial image to a series of image editing operations to obtain a target image, acquiring initial key frames, the initial key frames being intermediate images between the initial image and the target image during the image editing process, the different initial key frames being intermediate images generated when different image editing operations of the series of image editing operations are triggered during the image editing process, and the intermediate images showing editing effects corresponding to the image editing operations; and generating a target video according to the initial key frames so as to record the image editing process of the target image. The present disclosure can be used for recording an image editing process.

The invention provides an editable and self-closed-loop dynamic OLED automobile tail light control system and method, and belongs to the technical field of automobile tail light intelligent control. The system comprises a vehicle speed detection module used for acquiring speed information of a vehicle in real time; the photoelectric sensing module is used for monitoring the illumination intensity of ambient light around the vehicle in real time; the image editing equipment is used for acquiring user-defined image and / or character information and analyzing the image and / or character information into array data; the control module is used for generating a brightness control instruction corresponding to the illumination intensity through the illumination intensity; generating a pattern display instruction corresponding to the speed information through the speed information of the vehicle; analyzing and recombining the array data and then generating an image driving control instruction corresponding to the array data; and the OLED automobile tail lamp driving module is used for adjusting the display brightness of the OLED automobile tail lamp based on the brightness control instruction and controlling the OLED automobile tail lamp to display patterns and / or characters based on the pattern display instruction and / or the image driving control instruction. And the real-time updating of the display effect of the OLED tail lamp is realized.

The invention discloses an operation interaction method and system applied to camera image editing, and relates to the technical field of image processing, and the method comprises the steps: obtaining a voice semantic heat map, a pointing intensity map, a touch confidence map and a gazing confidence map based on a multi-modal interaction data packet, and calculating an image feature matrix at the same time; fusing into a multi-modal evidence graph through a normalized scale; performing semantic segmentation according to the image feature matrix to obtain a semantic segmentation first draft and a pixel-by-pixel category confidence coefficient, and performing position correlation weighting on the pixel-by-pixel category confidence coefficient by taking the multi-modal evidence graph as a confidence coefficient modulation factor to generate a candidate object mask sequence; and performing highlight display on the candidate object mask sequence, and performing conflict resolution and priority rearrangement in combination with the multi-mode evidence graph to generate a target object mask. According to the method, deep fusion of the interaction intention and image segmentation is realized, the precision and consistency of candidate region detection are improved, and the stability of real-time rendering and the reliability of an editing result are improved.

In an image processing apparatus 100, an image acquiring portion 11 acquires image information. A position acquiring portion 12 acquires an image capturing position and an image capturing direction of the image information. An image-capturing propriety information acquiring portion 13 acquires image-capturing propriety information indicating that a position of an information terminal 200 corresponds to image-capturing propriety setting information. An image editing portion 14 performs a masking process to a range based on the information terminal 200 in the image information acquired by the image acquiring portion 11, in accordance with image-capturing propriety setting information of the image-capturing propriety information indicating that the position of the information terminal 200 in the image-capturing propriety information corresponds to the image capturing range based on the image capturing position and the image capturing direction acquired by the position acquiring portion 12.

The present disclosure relates to systems, methods, and non-transitory computer-readable media that perform text-to-image editing using executable code generated from natural language text input. For instance, in one or more embodiments, the disclosed systems receive, from a client device, a digital image and natural language text input providing instructions for modifying the digital image. The disclosed systems also generate, using a large language model, executable action code for modifying the digital image in accordance with the instructions of the natural language text input, the executable action code being compatible with an editing application. The disclosed systems further modify the digital image by executing the executable action code via the editing application and provide the modified digital image for display via a graphical user interface of the client device.