Multimodal image color segmenter and editor

Abstract

Embodiments of this disclosure relate to multimodal image color segmenters and editors. Systems and methods for color replacement are described. Embodiments of this disclosure include a color replacement system that adjusts an image based on a source color and a target color input by a user. For example, a source color may be replaced by a target color throughout the image. In some embodiments, the user provides voice or text input identifying the source color to be replaced. The user can then provide voice or text input identifying the target color to replace the source color. The color replacement system creates a source color embedding, segments the image based on the source color embedding, and then replaces the color of the segmented portions of the image with the target color.

Description

Technical Field

[0001] The following content generally deals with image editing, and more specifically with color replacement. Background Technology

[0002] Image editing refers to the process of adjusting an image digitally or otherwise to modify its appearance. For example, computer-based image editing software provides the ability to modify images quickly and efficiently. In some cases, non-destructive editing processes can be used to edit digital images.

[0003] Color replacement is the process of changing one color in an image to another. Typically, color replacement involves manually selecting pixels with a given color or selecting the RGB representation of the color and identifying pixels in the image that have the same or similar RGB values.

[0004] However, manually selecting pixels to be replaced is both time-consuming and inaccurate. Because the distances between colors in the RGB space do not necessarily correspond to human color perception, selecting colors based on RGB values ​​can also lead to inaccurate choices. Furthermore, many users find it difficult to select a desired set of colors by specifying RGB values. Therefore, there is a need in the art for improved systems and methods for color replacement that can efficiently select a desired color in an image and replace it with another color. Summary of the Invention

[0005] This disclosure describes systems and methods for color replacement. Embodiments of this disclosure include a color replacement system that adjusts an image based on a source color and a target color input by a user. For example, a source color may be replaced with a target color throughout the image. In some embodiments, the user provides voice or text input identifying the source color to be replaced. The user can then provide voice or text input identifying the target color used to replace the source color. The color replacement system creates a source color embedding, segments the image based on the source color embedding, and then replaces the color of the segmented portions of the image with the target color.

[0006] A method, apparatus, non-transient computer-readable medium, and system for color replacement are described. One or more embodiments of the method, apparatus, non-transient computer-readable medium, and system include: using a color encoder to generate color embeddings for a plurality of pixels of an image; identifying a source color embedding corresponding to a source color within the image; segmenting the image to produce color segmentation by comparing the source color embeddings with pixel color embeddings, wherein the color segmentation indicates portions of the image corresponding to the source color; receiving a target color input corresponding to a target color; generating a target color embedding by applying a color text embedding network to the target color input; identifying the target color based on the target color embedding; and replacing the source color in the image with the target color based on the color segmentation and the target color embedding.

[0007] A method, apparatus, non-transient computer-readable medium, and system for color replacement are described. One or more embodiments of the method, apparatus, non-transient computer-readable medium, and system include: receiving an image; a source color input identifying a source color and a target color input identifying a target color; generating a source color embedding for the source color based on the source color input; generating color pixel embeddings for a plurality of pixels in the image; segmenting the image to produce color segmentation by comparing the source color embeddings with the pixel color embeddings; generating a target color embedding based on the target color input; identifying a target color representation of the target color; and replacing the source color in the image with the target color based on the color segmentation and the target color representation.

[0008] An apparatus, system, and method for color replacement are described. One or more embodiments of the apparatus, system, and method include: a color text embedding network configured to generate a source color embedding based on a source color input and a target color embedding based on a target color input; a color encoder configured to generate pixel color embeddings for a plurality of pixels in an image; an image segmentation component configured to segment the image to produce color segmentation by comparing the source color embedding with the pixel color embedding; and a color replacement component configured to replace the source color in the image with the target color based on the color segmentation and the target color embedding. Attached Figure Description

[0009] Figure 1 An example of a color replacement diagram according to an aspect of this disclosure is shown.

[0010] Figure 2 An example of a color replacement process according to aspects of this disclosure is shown.

[0011] Figure 3 An example of an image with colors replaced according to aspects of this disclosure is shown.

[0012] Figure 4An example of a color replacement device according to aspects of this disclosure is shown.

[0013] Figure 5 An example of a process for color embedding according to aspects of this disclosure is shown.

[0014] Figures 6 to 7 An example of a process for color replacement according to aspects of this disclosure is shown.

[0015] Figure 8 An example of a process for color segmentation according to aspects of this disclosure is shown.

[0016] Figure 9 An example of a process for color replacement according to aspects of this disclosure is shown. Detailed Implementation

[0017] This disclosure describes systems and methods for color replacement. Embodiments of this disclosure include a color replacement system that adjusts the color of an image based on a source color and a target color input by a user. For example, the source color may be replaced by the target color throughout the image. In some embodiments, the user provides voice or text input identifying the source color to be replaced and the target color to replace the source color. The color replacement system creates a source color embedding, segments the image based on the source color embedding, and then replaces the color of the segmented portions of the image with the target color. In some examples, the source color is replaced by the target color throughout the image, thereby providing the user with the ability to quickly and efficiently adjust the colors of an image.

[0018] Images can contain hundreds or thousands of different colors. These colors can be located in multiple places within the image itself. For example, an image of a tree can have thousands of leaves. If a designer only wants to change the color of the leaves, they might need to edit each leaf individually. This process can be very time-consuming and could lead to errors in the final product.

[0019] Conventional image editing software performs color replacement by allowing users to manually select pixels with a given color or by selecting the RGB representation of the color and identifying pixels in the image with the same or similar RGB values. However, manually selecting pixels to replace is both time-consuming and inaccurate. Because the distances between colors in the RGB space do not necessarily correspond to human color perception, selecting colors based on RGB values ​​can also lead to inaccurate choices. Furthermore, many users find it difficult to select a desired set of colors by specifying RGB values.

[0020] Embodiments of this disclosure provide a system for replacing a source color with a target color by receiving natural language input that identifies a source color, a target color, or both. In some embodiments, the color may be input to a speech-to-text program. A color text embedding network embeds the text input to create a color embedding for the source color, while the colors of individual pixels are also embedded in the same color embedding space using a color encoder. Pixels having the same or similar color as the source color are identified based on the color embedding, and these pixels are replaced with the target color.

[0021] By employing unconventional steps to perform color replacement based on natural language color input, embodiments of this disclosure enable image editing software to perform fast and accurate color replacement without relying on manual pixel selection or RGB color selection. Furthermore, embodiments of this disclosure can replace colors in an image while preserving variations in shade (e.g., due to differences in saturation or brightness).

[0022] Embodiments of this disclosure can be used in the context of image editing software applications. For example, a color replacement apparatus based on this disclosure can receive natural language speech or text as input and efficiently segment and replace colors in an image based on the input speech or text. Reference Figures 1 to 3 Examples of applying the concepts of this invention in the context of image editing are provided. References Figure 4 and Figure 5 Details about the architecture of the example color replacement device are provided. (Reference) Figures 6 to 9 An example of a process for color replacement is provided.

[0023] Color replacement system

[0024] Figure 1 An example of a color replacement diagram according to an aspect of this disclosure is shown. The example shown includes user 100, user device 105, cloud 110, color replacement device 115, and database 120.

[0025] This disclosure describes systems and methods for changing the background of an image using colors presented by a user (i.e., in the form of text or voice). For example, a user can quickly replace colors in an image editing application or visualize e-commerce products with different colors while preserving color chromaticity variations.

[0026] Manually identifying image regions with similar hue is both complex and time-consuming. However, embodiments of this disclosure allow users to speak or type color text and then segment the image based on that color text. Color text can be in various languages ​​and may include spelling errors or refer to complex colors with specific hues (e.g., blue and red). Embodiments of this disclosure do not rely on object masks. This allows multiple objects with the same color to be selected simultaneously. Embodiments of this disclosure improve user interaction by utilizing voice or text to provide colors and instructions to the tool.

[0027] exist Figure 1 In the example, the image may contain an unwanted background color. In this case, the image was taken on a rainy day, and the sky is gray. For a more aesthetically pleasing image, a blue sky would be more desirable. The user can input an image and say a phrase such as "convert gray to blue." The system will identify the gray pixels in the image and convert the identified pixels to blue.

[0028] User 100 communicates with color replacement device 115 via user equipment 105 and cloud 110. For example, user 100 can provide the image to be replaced, the source color, and the target color for replacement. In some examples, the image can be retrieved from database 120. Figure 1 The diagram shows how to identify the source and target colors from a single input phrase. Figure 1 In the example illustrated, the image includes buildings in the rain. User equipment 105 transmits source color text and target color text to color replacement device 115. In some examples, user equipment 105 communicates with color replacement device 115 via cloud 110.

[0029] According to some embodiments, user device 105 presents candidate image colors to user 100, allowing user 100 to select a source color from a list of colors appearing in the image. In some examples, user device 105 displays color segmentation to user 100. In some examples, user device 105 receives feedback from user 100 regarding the color segmentation. In some examples, user device 105 displays a color palette to user 100 based on either the source or target color (i.e., giving the user an understanding of the range of colors to be replaced). In some examples, user device 105 receives luminance and saturation values, allowing the user to fine-tune the chromaticity of one or more colors used to replace the source color.

[0030] User equipment 105 may be a personal computer, laptop computer, mainframe computer, handheld computer, personal assistant, mobile device, or any other suitable processing device. User equipment 105 is for reference only. Figure 4 Examples of corresponding components described or including references Figure 4 The aspects of the corresponding components described.

[0031] A cloud 110 is a computer network configured to provide on-demand availability of computer system resources, such as data storage and computing power. In some examples, a cloud 110 provides resources without active management by user 100. The term "cloud 110" is sometimes used to describe a data center available to many users 100 via the Internet. Some large cloud 110 networks have functionality distributed across multiple locations from a central server. If a server has a direct or close connection to user 100, then that server is designated as an edge server. In some cases, a cloud 110 is limited to a single organization. In other examples, a cloud 110 is available to many organizations. In one example, a cloud 110 includes a multi-tiered communication network comprising multiple edge routers and a core router. In another example, a cloud 110 is based on a local collection of switches in a single physical location.

[0032] Color replacement device 115 performs color segmentation and color replacement on an image. In some cases, color replacement device 115 can receive natural language speech or text as input, and segment the colors of the image based on the input speech or text, then replace the colors of the image. An encoder can be used to convert the color text into a corresponding color embedding, which is in the same space as the pixel color embedding. Color replacement device 115 is a reference... Figure 4 Examples of corresponding components described or including references Figure 4 The aspects of the corresponding components described.

[0033] Database 120 is an organized collection of data. For example, database 120 stores data in a specified format called a schema. Database 120 can be configured as a single database 120, a distributed database 120, multiple distributed databases 120, or an emergency backup database 120. In some cases, a database 120 controller can manage the storage and processing of data within database 120. In some cases, user 100 interacts with the database 120 controller. In other cases, the database 120 controller can operate automatically without user 100 interaction.

[0034] Figure 2 Examples of color replacement processes according to aspects of this disclosure are shown. In some examples, these operations are performed by a system including a processor that executes a set of code to control functional elements of the device. Additionally or alternatively, some processes are performed using dedicated hardware. Typically, these operations are performed according to the methods and processes described according to aspects of this disclosure. In some cases, the operations described herein consist of various sub-steps or are performed in combination with other operations.

[0035] Some embodiments of this disclosure provide users with the ability to segment regions from an image based on color text and perform replacements with another color text (i.e., the chroma and brightness of the segmented regions remain unchanged). In some embodiments, the color embedding used is a histogram-based vector. Therefore, the elements in the embedding represent color chroma. A slider is provided that can determine the chroma range of a color (and thus adjust the dominance of the color), while segmenting regions based on the color embedding similarity score between the region pixels and the color embedding of the text color. As the hue portion of the color is replaced, the user adjusts the saturation and brightness of the replaced color region. Voice can be used to increase the saturation and brightness of the replaced color, the size of the color region to be segmented, and to provide semantic segmentation regions. Some embodiments of this disclosure provide a theme editor tool that uses the dominant color in the image and performs replacements using a user-provided color theme to obtain different images using the same color theme more quickly.

[0036] In operation 200, the user provides an image to the system. The image can be in any file format, such as JPEG, RAW, HEIC, etc. Alternatively, the image can reside in a database and can be provided to the system by the user. In some cases, this step involves operations such as those described in the reference. Figure 1 The user described or may be referenced as follows Figure 1 The user action described.

[0037] In operation 205, the user provides voice or text input with a source color. The voice input is provided to a multilingual text encoder to convert the text into a color embedding. The system disclosed herein can utilize any natural language color input. For example, the user can input red, crimson, sand red, or iron oxide. Text input can also be provided to the system in the form of natural language text from a keyboard, mouse, touchpad, etc. The source color can be a user-defined color to be replaced.

[0038] In operation 210, the system segments colors in the image. Color segmentation is performed by extracting color embeddings for unique pixels in the image using a color pixel encoder. The user can use an automatic color tagger to search for colors. The automatic tagger recommends colors in textual form based on the colors present in the image. The user can consider any color to be segmented in the natural language spectrum. In some cases, this step involves operations such as reference... Figure 1 and Figure 4 The described color replacement device or may be derived from, as in the reference Figure 1 and Figure 4 The described color replacement device is executed.

[0039] In operation 215, the user provides voice or text input with a target color. The voice input is provided to a multilingual text encoder to convert the text into a color embedding. Text input can also be provided to the system in natural language text form from a keyboard, mouse, touchpad, etc. The target color can be a user-defined color that replaces the source color.

[0040] In operation 220, the system replaces the source color with the target color to create an adjusted image. While the hue portion of a pixel's hue, saturation, and lightness (HSL) values ​​is replaced, different lighting and shadows in the image are preserved. Some embodiments of this disclosure are used for style editing of real-world images containing different colors. A user can speak of colors to segment parts, and then replace the segmented areas with color text (i.e., basic, complex, or specific colors). Some embodiments of this disclosure are used to perform palette mapping (i.e., mapping multiple painting colors to different sets of colors and transforming the original image according to color text provided by the user). As the hue portion of a color is replaced, the user can adjust the saturation and lightness of the replaced color area. In some cases, this step involves operations such as those described in the references... Figure 1 and Figure 4 The described color replacement device or may be derived from, as in the reference Figure 1 and Figure 4 The described color replacement device is executed.

[0041] In some embodiments, when replacing a color, the hue dimension can be replaced while preserving variations in chroma and brightness within the masked portion of the image. For example, controls can be provided to the user to adjust portions of the image based on color dominance and to control the saturation (chroma) and brightness of the replaced color. Some embodiments of this disclosure use an automatic tagger that suggests color tags for a given image for the user to perform color segmentation with improved accuracy. The input to the developed model is text. Therefore, the user uses a speech-to-text tool to give instructions (via voice) regarding the colors to be segmented and replaced. The user can use voice to increase the saturation and brightness of the replaced color and provide semantic segmentation regions.

[0042] In operation 225, the adjusted image is sent back to the user. The user can save the adjusted image after being satisfied with the changes in the color segmentation. This process can also be repeated for different colors or different images.

[0043] Figure 3 An example of an image 310 with its colors replaced according to aspects of this disclosure is shown. The example shown includes the original image 300, the segmented image 305, and the image 310 with its colors replaced.

[0044] The original image 300 is the original image input by the user. The bidirectional background shadow represents the single color to be replaced based on the source color input from the user. In an example scene, the bidirectional shadow represents a gray sky, such as... Figure 1 and Figure 2 The text is quoted in the original.

[0045] The segmented image 305 is an intermediate image generated by the color replacement system of this disclosure. Figure 3 In the example scenario, the segmented image 305 is divided into two regions: a bright region and a dark region. The bright region has been determined not to be the target color. The dark region has been determined to be the target color. Therefore, the dark region will be replaced with the source color. In some examples, an image segmentation mask can be presented to the user to make it clearer which parts of the image will be replaced with the other color.

[0046] Image 310, with its colors replaced, is the final image produced by the color replacement system of this disclosure. The segmented background of the image is replaced with the target color, represented by diagonal shading.

[0047] Network architecture

[0048] exist Figure 4 and Figure 5 This document describes apparatus, systems, and methods for color replacement. One or more embodiments of the apparatus, systems, and methods include: an image segmentation component configured to segment an image to produce color segmentation by comparing a source color with pixel color embeddings for a plurality of pixels in the image; a color text embedding network configured to generate a target color embedding corresponding to a target color based on target color input text; and a color replacement component configured to replace a source color in an image with the target color based on the color segmentation and querying the color embedding.

[0049] Examples of the aforementioned apparatus, systems, and methods also include a color encoder configured to generate pixel color embeddings in the same embedding space as the target color embedding. Examples of the aforementioned apparatus, systems, and methods also include a user device configured to receive source color text input for a source color and target color text input for a target color, and to display an image in which the source color is replaced by the target color.

[0050] Figure 4 An example of a color replacement apparatus 400 according to an aspect of this disclosure is shown. The illustrated example includes a color replacement apparatus 400 having a memory unit 405, a processor unit 410, a user device 415, an image segmentation component 420, a color text embedding network 425, a color replacement component 430, and a color encoder 435. The color replacement apparatus 400 is a reference. Figure 1Examples of corresponding components described or including references Figure 1 The aspects of the corresponding components described.

[0051] Examples of memory cells 405 include random access memory (RAM), read-only memory (ROM), or a hard disk. Examples of memory devices include solid-state memory and hard disk drives. In some examples, the memory is used to store computer-readable, computer-executable software including instructions that, when executed, cause a processor to perform the various functions described herein. In some cases, the memory contains a basic input / output system (BIOS), which controls basic hardware or software operations, such as interaction with peripheral components or devices. In some cases, a memory controller operates the memory cells. For example, a memory controller may include a row decoder, a column decoder, or both. In some cases, the memory cells within the memory store information in the form of logical states.

[0052] Processor unit 410 is an intelligent hardware device (e.g., a general-purpose processing component, a digital signal processor (DSP), a central processing unit (CPU), a graphics processing unit (GPU), a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a programmable logic device, a discrete gate or transistor logic component, a discrete hardware component, or any combination thereof). In some cases, the processor is configured to use a memory controller to operate a memory array. In other cases, the memory controller is integrated into the processor. In some cases, the processor is configured to execute computer-readable instructions stored in memory to perform various functions. In some embodiments, the processor includes dedicated components for modem processing, baseband processing, digital signal processing, or transmission processing.

[0053] User equipment 415 may be a personal computer, laptop computer, mainframe computer, handheld computer, personal assistant, mobile device, or any other suitable processing device. User equipment 415 is a reference. Figure 1 Examples of corresponding components described or including references Figure 1 The aspects of the corresponding components described.

[0054] According to some embodiments, the image segmentation component 420 segments the image to produce color segmentation by comparing a source color with a pixel color embedding for a set of pixels in the image. In some examples, the image segmentation component 420 identifies a set of image colors in the image. In some examples, the image segmentation component 420 receives an instruction from a user to identify a source color from the colors in the image. In some examples, the image segmentation component 420 identifies a set of pixel clusters in the image and selects pixels from each pixel cluster, where a set of pixels corresponds to the selected pixels. In some examples, pixel clusters are identified based on having similar pixel colors. In some examples, the image segmentation component 420 updates the color segmentation based on feedback regarding image segmentation, where the source color is replaced based on the updated color segmentation.

[0055] According to some embodiments, the color text embedding network 425 generates source color embeddings and target color embeddings based on source color text inputs and target color text inputs, respectively. In some examples, color segmentation is based on source color embeddings. In some examples, the source color or target color is extracted from an audio signal. In some examples, the color text embedding network 425 determines that the target color text input corresponds to a primary color and identifies a set of related colors by adding or modifying text to the target color text input. For example, the color text embedding network 425 can generate related color embeddings for related colors, where the target color embedding is based on the related color embedding. The color text embedding network 425 is a reference... Figure 5 Examples of corresponding components described or including references Figure 5 The aspects of the corresponding components described.

[0056] According to some embodiments, the color replacement component 430 replaces the source color in an image with a target color based on color segmentation and target color embedding. In some examples, the color replacement component 430 replaces the hue and then adjusts the image based on brightness and saturation values.

[0057] According to some embodiments, the color replacement component 430 replaces a source color in an image with a target color based on color segmentation and target color embedding. In some examples, the color replacement component 430 identifies the hue, saturation, and lightness (HSL) color representation of the target color based on color embedding, and then identifies the hue of the target color based on the HSL color representation. In some examples, the color replacement component 430 also identifies lightness and saturation values ​​based on user input. In some examples, the color replacement component 430 identifies the replacement color based on the hue, lightness, and saturation values ​​of the target color. In some examples, the color replacement component 430 receives a lightness adjustment value, a saturation adjustment value, or both from the user, wherein the lightness value or saturation value is based on the lightness adjustment value or the saturation adjustment value, respectively.

[0058] According to some embodiments, color encoder 435 generates a color embedding for a pixel, and generates the pixel color embedding in the same embedding space as the target color embedding. Color encoder 435 is a reference... Figure 5 Examples of corresponding components described or including references Figure 5 The aspects of the corresponding components described.

[0059] In some examples, the color replacement device 400 calculates a similarity score for each pixel in the pixel set and also identifies a similarity threshold. The color replacement device 400 then determines whether the similarity score for each pixel in the pixel set is less than the similarity threshold, where color segmentation is based on this determination. In some examples, the color replacement device 400 calculates the cosine similarity between the source color embedding and each pixel color embedding in the pixel color embedding, where the similarity score is based on cosine similarity. In some examples, the color replacement device 400 displays a threshold control element to the user. In some examples, the color replacement device 400 receives a threshold control value from the threshold control element, where the similarity threshold is based on the threshold control value.

[0060] Figure 5 An example of a process for color embedding according to aspects of this disclosure is shown. The illustrated example includes a color item 500, an encoder 505, a color embedding network 510, and an embedded color representation 530. According to some embodiments, the encoder 505 embeds the color item 500 into a text embedding space to produce an embedded color item 500. According to some embodiments, the encoder 505 can be trained to embed the color item 500 into a text embedding space to generate an embedded color item 500. In one embodiment, the color embedding network 510 includes a fully connected layer 515, a rectified linear unit 520, and a least squares function 525.

[0061] Some embodiments of this disclosure use a multilingual text encoder to convert text into color embeddings. A color pixel encoder converts RGB values ​​into color embeddings for segmenting image regions using a similarity score metric. The color pixel encoder computes the color embeddings of pixels by converting the RGB space to the LAB space. This conversion is performed because two color vectors that are close to each other in the RGB space (i.e., low Euclidean distance, L2) may not be perceptually close with respect to human color vision. The LAB space is designed to be perceptually consistent with human color vision (i.e., numerical changes in LAB values ​​correspond to an equal amount of perceptual visual change). A 3D histogram used in the LAB space is computed to find good intervals by identifying combinations of intervals suitable for color similarity searching.

[0062] For example, intervals of histograms of size [9,7,8] and [10,10,10] can be used. Two histograms are computed using the intervals [9,7,8] and [10,10,10], and these two histograms are concatenated to obtain a feature vector. The square root of the numbers in the feature vector is calculated to obtain the final color embedding. Finding the square root can disadvantage the dominant color and give more weight to other colors in the image. For example, RGB values ​​can be converted to their corresponding 1504-dimensional color embedding by taking the RGB values ​​individually to obtain two non-zero values ​​in the feature vector (i.e., one non-zero value in a color histogram of size 504 and one of size 1000).

[0063] A method for text-based image search is described. Embodiments of the method are configured to receive text input, wherein the text input includes a color item 500. For example, the color item 500 may be “yellow,” “magenta,” “blue-green,” etc., but this disclosure is not limited to these colors and various color items 500 can be recognized. Additionally, the color item 500 is not limited to English and may come from any natural language, such as Spanish, French, Italian, etc.

[0064] Additionally, embodiments of the method are configured to use encoder 505 and color embedding network 510 to generate an embedded color representation 530 for color item 500. Embodiments of the method are also configured to select a palette for color item 500 based on the embedded color item (e.g., color item 500 embedded in a color space via encoder 505), perform an image search based on the palette, and return search results based on the palette. Search results may include images determined to include the color item.

[0065] According to some embodiments, encoder 505 embeds color item 500 into a text embedding space to produce embedded color items. First, encoder 505 is used to convert color item 500 into a cross-lingual sentence embedding. For example, encoder 505 could be a cross-lingual sentence encoder. If a cross-lingual sentence encoder is not used, another sentence encoder can be used and trained using colors from different languages. According to some embodiments, encoder 505 can be trained to embed color item 500 into a text embedding space to generate embedded color items.

[0066] Cross-linguistic sentence embeddings are fed into a color embedding network 510, which may include blocks of fully connected (FC), ReLU, and least squares layers. The least squares layers (i.e., L2 Norm) constrain the values ​​in a manner that makes the values ​​range from 0 to 1 and are used in the final block when the color embedding values ​​are in the range of 0 to 1. In some examples, the fully connected layer 515 (FC), the rectified linear unit 520 (ReLU), and the least squares function 525 (L2 Norm) may be referred to as neural network layers. Typically, the color embedding network 510 may include any number of layers (e.g., any number of groups of fully connected layers 515, rectified linear units 520, and least squares functions 525).

[0067] The multilingual text encoder transforms colored text into corresponding color embeddings in the same space as the pixel color embeddings. The dataset used consists of colored text and corresponding RGB values, which are then converted into color embeddings using a color pixel encoder. A cross-lingual sentence model (e.g., Multilingual Universal Sentence Encoder, USE) is used to transform the colored text into cross-lingual sentence embeddings. These cross-lingual sentence embeddings are then passed to fully connected blocks of piecewise linear and weighted regularization functions (e.g., rectified linear activation units, ReLU, and L2 normalization layers).

[0068] The weight regularization (e.g., L2 normalization layer) limits the range of values ​​(i.e., 0 to 1). A negative mining strategy is used to collect negative samples in mini-batches, which involves obtaining color embeddings (i.e., text with different colors) that are closest to the color embeddings of the samples for which negative samples are to be found. Hard negatives are obtained using the negative mining method. Therefore, a loss function in metric learning (e.g., metric learning loss or triplet loss) is used to make the generated color embeddings close to the corresponding positive color embeddings (i.e., far from the negative color embeddings). Some embodiments of this disclosure use cross-lingual multimodal text to perform colorization on embedding models with multiple embedding styles.

[0069] In the example scenario, embodiments of this disclosure convert RGB values ​​into corresponding 1504-dimensional color embeddings, and two non-zero values ​​are determined in the feature vector because one of the two color histograms of sizes 504 and 1000 has a non-zero value. The embedded color representation 530 can reside in the LAB space. The LAB space is a color representation that includes lightness, red, green, blue, and yellow. The LAB space can be used to detect subtle changes or differences in color.

[0070] Color replacement

[0071] Methods, apparatuses, non-transient computer-readable media, and systems for color replacement are described. One or more embodiments of the method, apparatus, non-transient computer-readable media, and system include: segmenting an image to produce color segmentation by comparing a source color with pixel color embeddings for a plurality of pixels in the image; generating a target color embedding corresponding to a target color by applying a color text embedding network to a target color text input; and replacing the source color in the image with the target color based on the color segmentation and the target color embedding.

[0072] Examples of the methods, apparatus, non-transient computer-readable media, and systems described above also include: receiving source color text input. Some examples also include: applying a color text embedding network to the source color text input to produce a source color embedding, wherein color segmentation is based on the source color embedding.

[0073] Examples of the methods, apparatus, non-transient computer-readable media, and systems described above also include: identifying multiple image colors in an image. Some examples also include: presenting image colors to a user. Some examples also include: receiving an instruction from the user that identifies a source color from among the colors in the image. Some examples of the methods, apparatus, non-transient computer-readable media, and systems described above also include: using a color encoder to generate a color embedding for pixels.

[0074] Examples of the methods, apparatus, non-transient computer-readable media, and systems described above also include: determining that a target color text input corresponds to a primary color. Some examples also include: identifying multiple related colors by adding or modifying text to the target color text input. Some examples also include: generating a related color embedding for the related colors, wherein the target color embedding is based on the related color embedding.

[0075] Examples of the methods, apparatus, non-transient computer-readable media, and systems described above also include: identifying multiple pixel clusters in an image. Some examples further include: selecting a pixel from each pixel cluster, wherein multiple pixels correspond to the selected pixel. In some examples, pixel clusters are identified based on pixels having similar colors.

[0076] Examples of the methods, apparatuses, non-transient computer-readable media, and systems described above further include: generating a source color embedding for a source color. Examples also include: calculating a similarity score for each pixel in the pixel set. Examples further include: identifying a similarity threshold. Examples further include: determining whether the similarity score for each pixel in the pixel set is less than the similarity threshold, wherein color segmentation is based on this determination. Examples of the methods, apparatuses, non-transient computer-readable media, and systems described above further include: calculating a cosine similarity between the source color embedding and each pixel color embedding in the pixel color embedding, wherein the similarity score is based on cosine similarity.

[0077] Examples of the methods, apparatus, non-transient computer-readable media, and systems described above also include: displaying a threshold control element to a user. Some examples also include: receiving a threshold control value from the threshold control element, wherein the similarity threshold is based on the threshold control value.

[0078] Examples of the methods, apparatus, non-transient computer-readable media, and systems described above also include: displaying a color segment to a user. Some examples also include: receiving feedback from the user regarding the color segment. Some examples also include: updating the color segment based on the feedback, wherein the source color is replaced based on the updated color segment. Examples of the methods, apparatus, non-transient computer-readable media, and systems described above also include: displaying a color palette to the user based on a source color or a target color.

[0079] Examples of the methods, apparatus, non-transient computer-readable media, and systems described above also include: receiving luminance and saturation values. Some examples also include: adjusting an image based on luminance and saturation values. Some examples of the methods, apparatus, non-transient computer-readable media, and systems described above also include: receiving an audio signal. Some examples also include: extracting a source color or a target color from the audio signal.

[0080] According to another embodiment, a method, apparatus, non-transient computer-readable medium, and system for color replacement are also described. One or more embodiments of the method, apparatus, non-transient computer-readable medium, and system include: receiving an image, source color text input identifying a source color, and target color text input identifying a target color; generating a source color embedding for the source color based on the source color text input; generating color pixel embeddings for a plurality of pixels in the image; segmenting the image to produce color segmentation by comparing the source color embeddings with the pixel color embeddings; generating a target color embedding based on the target color text input; and replacing the source color in the image with the target color based on the color segmentation and the target color embedding.

[0081] Examples of the methods, apparatus, non-transient computer-readable media, and systems described above include: identifying an HSL color representation for a target color. Some examples also include: identifying the hue of the target color based on the HSL color representation. Some examples also include: identifying brightness and saturation values. Some examples also include: identifying a replacement color based on the hue, brightness, and saturation values ​​of the target color.

[0082] Examples of the methods, apparatus, non-transient computer-readable media and systems described above also include receiving a brightness adjustment value, a saturation adjustment value, or both from a user, wherein the brightness value or saturation value is based on the brightness adjustment value or the saturation adjustment value, respectively.

[0083] Figure 6 Examples of processes for color replacement according to aspects of this disclosure are shown. In some examples, these operations are performed by a system including a processor that executes a set of code to control functional elements of a device. Additionally or alternatively, some processes are performed using dedicated hardware. Typically, these operations are performed according to the methods and processes described according to aspects of this disclosure. In some cases, the operations described herein consist of various sub-steps or are performed in combination with other operations.

[0084] In operation 600, the system segments the image to produce color segmentation by comparing the source color with pixel color embeddings for a set of pixels in the image. For example, the source color embedding may be generated based on the source color input text, while the pixel color embedding is generated based on the pixel color. Each pixel (or pixel sample) in the image can be compared to the source color based on the embedding. If the color of a pixel is close to the source color, those pixels can be included in the selected region. In some cases, this step involves operations such as referencing... Figure 4 The image segmentation component described may be derived from, as in the reference Figure 4 The image segmentation component described is executed.

[0085] In operation 605, the system generates a target color embedding corresponding to the target color by applying a color text embedding network to the target color text input. In some cases, this step involves operations such as those described in the reference. Figure 4 and Figure 5 The described color text embedding network or can be derived from, as in the reference Figure 4 and Figure 5 The described color text is embedded in the network execution.

[0086] In operation 610, the system replaces the source color in the image with the target color based on color segmentation and target color embedding. For example, the target color embedding can be converted to HSL format. Hue can be used to replace the hue of pixels in the selected segment. In some cases, the user can also adjust the saturation or brightness of the replaced pixels (e.g., using sliders provided in the user interface). In some cases, this step involves operations such as those described in the reference. Figure 4 The described color replacement component or can be found in references. Figure 4 The described color replacement component is executed.

[0087] Figure 7 Examples of processes for color replacement according to aspects of this disclosure are shown. In some examples, these operations are performed by a system including a processor that executes a set of code to control functional elements of a device. Additionally or alternatively, some processes are performed using dedicated hardware. Typically, these operations are performed according to the methods and processes described according to aspects of this disclosure. In some cases, the operations described herein consist of various sub-steps or are performed in combination with other operations.

[0088] In operation 700, the system receives an image, source color text input identifying the source color, and target color text input identifying the target color. The image can be input by the user. Alternatively, the image can be stored in a database and retrieved from the database. Both the source and target colors can be input via voice and converted to text, or input in text form. In some cases, this step involves operations such as those described in the reference... Figure 1 and Figure 4 The user equipment described or may be provided by reference Figure 1 and Figure 4 The user equipment described is executed.

[0089] In operation 705, the system generates a source color embedding for the source color based on the source color text input. The color text input can be speech-to-text input. In some cases, this step involves operations such as those described in the reference. Figure 4 and Figure 5 The described color text embedding network or can be derived from, as in the reference Figure 4 and Figure 5 The described color text is embedded in the network execution.

[0090] In operation 710, the system generates a color pixel embedding for a set of pixels in the image. In some cases, this step involves operations such as those described in the reference. Figure 4 and Figure 5 The described color text embedding network or can be derived from, as in the reference Figure 4 and Figure 5 The described color text is embedded in the network execution.

[0091] In operation 715, the system segments the image to produce color segmentation by comparing the source color embedding with the pixel color embedding. The image can be segmented into two or more segments. In some cases, this step involves operations such as those described in the reference. Figure 4 The image segmentation component described may be derived from, as in the reference Figure 4 The image segmentation component described is executed.

[0092] In operation 720, the system generates a target color embedding based on the target color text input. The color text input can be speech-to-text input. In some cases, this step involves procedures such as those described in the reference. Figure 4 and Figure 5 The described color text embedding network or can be derived from, as in the reference Figure 4 and Figure 5 The described color text is embedded in the network execution.

[0093] In operation 725, the system replaces the source color in the image with the target color based on color segmentation and target color embedding. In some cases, this step involves operations such as those described in the reference. Figure 4 The described color replacement component or can be found in references. Figure 4 The described color replacement component is executed.

[0094] Figure 8 Examples of processes for color segmentation according to aspects of this disclosure are shown. In some examples, these operations are performed by a system including a processor that executes a set of code to control functional elements of the device. Additionally or alternatively, some processes are performed using dedicated hardware. Typically, these operations are performed according to the methods and processes described according to aspects of this disclosure. In some cases, the operations described herein consist of various sub-steps or are performed in combination with other operations.

[0095] Color segmentation is performed by extracting color embeddings for unique pixels in an image using a color pixel encoder. Users can search for colors using an automatic color tagger, which can recommend colors present in an image in text form. Users can consider any color to be segmented. The automatic color tagger is created using color text and a predefined list of corresponding color embeddings, and a multilingual text encoder is used to generate the corresponding color embeddings. For each pixel color embedding, the closest color text is found from the similarity score using dot product or squared distance (i.e., selecting the closest). A histogram of the closest colors is created, and suitable colors can be provided to the user as tags or a word cloud. A speech-to-text tool is used to convert the user-provided input (i.e., colors) in text or speech form into text, and a multilingual text encoder is used to find the color embeddings.

[0096] In operation 800, the system segments the image to produce color segmentation by comparing the source color with pixel color embeddings for a set of pixels in the image. In some cases, this step involves operations such as those described in the reference. Figure 4 The image segmentation component described may be derived from, as in the reference Figure 4 The image segmentation component described is executed.

[0097] In operation 805, the system generates a target color embedding corresponding to the target color by applying a color text embedding network to the target color text input. In some cases, this step involves operations such as those described in the reference. Figure 4 and Figure 5 The described color text embedding network or can be derived from, as in the reference Figure 4 and Figure 5 The described color text is embedded in the network execution.

[0098] In operation 810, the system replaces the source color in the image with the target color based on color segmentation and target color embedding. In some cases, this step involves operations such as those described in the reference. Figure 4 The described color replacement component or can be found in references. Figure 4 The described color replacement component is executed.

[0099] In operation 815, the system generates a source color embedding for the source color. Color text input can be speech-to-text input. In some cases, this step involves operations such as those described in the reference. Figure 4 and Figure 5 The described color text embedding network or can be derived from, as in the reference Figure 4 and Figure 5 The described color text is embedded in the network execution.

[0100] In operation 820, the system calculates a similarity score for each pixel in the image. A color embedding with pixel color embedding is used to obtain the similarity score. Pixel indices are sorted in descending order of similarity score. A threshold (determined by moving a slider in the user device) is used to select similar pixel indices to represent the segmented portion (in its original color), and the remaining pixel indices are displayed in grayscale. The threshold determines the variation in color text that is segmented or captured in the image. In some cases, this step involves operations such as reference... Figure 4 and Figure 5 The described color text embedding network or can be derived from, as in the reference Figure 4 and Figure 5 The described color text is embedded in the network execution.

[0101] In operation 825, the system identifies a similarity threshold. The color pixel encoder converts RGB values ​​into color embeddings for segmenting image regions using a similarity score metric. The color pixel encoder computes the color embeddings of pixels by converting the RGB space to the LAB space. In some cases, this step involves operations such as reference... Figure 4 and Figure 5 The described color text embedding network or can be derived from, as in the reference Figure 4 and Figure 5 The described color text is embedded in the network execution.

[0102] In operation 830, the system determines whether the similarity score for each pixel in a pixel is less than a similarity threshold, upon which color segmentation is based. For pixel color embedding, the closest color text is found from the similarity scores using dot product or squared distance (i.e., selecting the nearest). A histogram of the closest colors is created, and appropriate colors can be provided to the user as labels or word clouds. In some cases, this step involves operations such as references... Figure 4 and Figure 5 The described color text embedding network or can be derived from, as in the reference Figure 4 and Figure 5 The described color text is embedded in the network execution.

[0103] Figure 9 Examples of processes for color replacement according to aspects of this disclosure are shown. In some examples, these operations are performed by a system including a processor that executes a set of code to control functional elements of a device. Additionally or alternatively, some processes are performed using dedicated hardware. Typically, these operations are performed according to the methods and processes described according to aspects of this disclosure. In some cases, the operations described herein consist of various sub-steps or are performed in combination with other operations.

[0104] Color replacement involves a target color provided by the user to replace a segment (i.e., the source color). When the target color is provided by the user, a multilingual text encoder is used to find the color embedding. The target color embedding is mapped to the nearest RGB value from a predefined list of color texts used to create an automatic color tagger. Similarity scores are given between target color texts. The color text is mapped to the RGB values ​​of the closest color texts in the list.

[0105] In operation 900, the system receives an image, source color text input identifying the source color, and target color text input identifying the target color. The image can be input by the user. Alternatively, the image can be stored in a database and retrieved from the database. Both the source and target colors can be input via voice and converted to text, or input in text form. In some cases, this step involves operations such as those described in the reference. Figure 1 and Figure 4 The user equipment described or may be provided by reference Figure 1 and Figure 4 The user equipment described is executed.

[0106] In operation 905, the system generates a source color embedding for the source color based on the source color text input. The color text input can be speech-to-text input. In some cases, this step involves operations such as those described in the reference. Figure 4 and Figure 5 The described color text embedding network or can be derived from, as in the reference Figure 4 and Figure 5 The described color text is embedded in the network execution.

[0107] In operation 910, the system generates a color pixel embedding for a set of pixels in the image. In some cases, this step involves operations such as those described in the reference. Figure 4 and Figure 5 The described color text embedding network or can be derived from, as in the reference Figure 4 and Figure 5 The described color text is embedded in the network execution.

[0108] In operation 915, the system segments the image to produce color segmentation by comparing the source color embedding with the pixel color embedding. The image can be segmented into two or more segments. In some cases, this step involves operations such as those described in the reference. Figure 4 The image segmentation component described may be derived from, as in the reference Figure 4 The image segmentation component described is executed.

[0109] In operation 920, the system generates a target color embedding based on the target color text input. The color text input can be speech-to-text input. In some cases, this step involves procedures such as those described in the reference. Figure 4 and Figure 5 The described color text embedding network or can be derived from, as in the reference Figure 4 and Figure 5 The described color text is embedded in the network execution.

[0110] In operation 925, the system identifies the HSL color representation for the target color. In some cases, this step involves procedures such as those described in the reference. Figure 4 The described color replacement component or can be found in references. Figure 4 The described color replacement component is executed.

[0111] In operation 930, the system identifies the hue of the target color based on the HSL color representation. In some cases, this step involves procedures such as those described in the reference. Figure 4 The described color replacement component or can be found in references. Figure 4The described color replacement component is executed.

[0112] In operation 935, the system identifies the brightness and saturation values. In some cases, this step involves procedures such as those described in the reference. Figure 4 The described color replacement component or may be derived from, as in the reference Figure 4 The described color replacement component is executed.

[0113] The target color and the RGB values ​​of the pixels in the segmented area are converted to the corresponding HSL (Hue, Saturation, and Luminance) space. (Without changing the luminance and saturation) the hue values ​​of the HSL values ​​of the segmented pixels are replaced with the hue values ​​of the HSL values ​​of the user-provided color text, so that the chromaticity and color variations in the segmented area remain intact.

[0114] In operation 940, the system identifies the replacement color based on the hue, brightness, and saturation values ​​of the target color. In some cases, this step involves procedures such as those described in the reference. Figure 4 The described color replacement component or can be found in references. Figure 4 The described color replacement component is executed.

[0115] In operation 945, the system replaces the source color in the image with the target color based on color segmentation and target color embedding. In some cases, this step involves operations such as those described in the reference. Figure 4 The described color replacement component or can be found in references. Figure 4 The described color replacement component is executed.

[0116] Users can use sliders to change the brightness and saturation values. For slider values ​​below 0.5, the change relative to 0.5 is subtracted from the brightness or saturation value of the pixels in the segmented region, while for slider values ​​above 0.5, the change relative to 0.5 is added. After calculating the HSL values ​​of the segmented pixels, the HSL space is converted back to the RGB space, and this portion is overlaid on the original image. For example, if the user provides dark blue as the target color, when the hue is replaced, if the original segmented portion is a darker shade, the replaced color will be a darker version of the color mentioned by the user. Therefore, users can use sliders to adjust the brightness and saturation values.

[0117] Increasing saturation makes the object's color closer to the user-provided color (e.g., dark blue). Increasing the brightness of the target area will increase the brightness or saturation value of the segmented pixels equally, while the object's chromaticity remains intact. The user can save the image after being satisfied with the changes to the color segmentation. This process can be repeated for different colors.

[0118] Tools can be used more efficiently and easily with the functionality of user devices to translate user (voice-based) commands into instructions that the UI understands. For example, if a user wants to change blue to red, the tool can use a predefined list of colors (for recognizing colors in sentences) or a Named Entity Recognition (NER) model to identify blue and red.

[0119] Basic colors (i.e., blue, green) can be used for color segmentation, which involves dividing the color chromaticity using a tool by referring to the chromaticity. Therefore, for basic colors, the average of the multilingual text color embeddings generated for the chromaticity is used. For example, for blue, the average of the color embeddings for blue, dark blue, and light blue is used, and the new color embedding represents blue. This process can be performed offline for basic colors.

[0120] UI functionality that provides users with the ability to perform color segmentation by creating bounding boxes around regions can keep certain areas intact. Models (such as semantic-based or edge-based segmentation models) can be used to obtain pre-segmented regions, within which users obtain color-based segmented portions. This tool is used when a color is prominent among multiple objects, but the user focuses on a particular object or region and wants to segment that portion using that color.

[0121] In some embodiments, theme generation can be added as a feature in the tool to modify images based on color themes. An automatic color tagger can be used to determine the dominant color names in an image uploaded by a user. Broader colors (e.g., primary colors or chromaticities of primary colors) are used to segment larger portions of the image. For example, three dominant color names selected as input (of different primary color categories) are used to segment and replace color portions to obtain a theme-based result of the image (e.g., a vector image without a complex color distribution).

[0122] The description and accompanying drawings described herein represent exemplary configurations and do not represent all implementations within the scope of the claims. For example, operations and steps may be rearranged, combined, or otherwise modified. Furthermore, structures and devices may be represented in block diagram form to illustrate relationships between components and to avoid confusion of described concepts. Similar components or features may have the same name but may have different reference numerals corresponding to different drawings.

[0123] Some modifications to this disclosure may be apparent to those skilled in the art, and the principles defined herein can be applied to other variations without departing from the scope of this disclosure. Therefore, this disclosure is not limited to the examples and designs described herein, but is accorded the broadest scope based on the principles and novel features disclosed herein.

[0124] The described systems and methods can be implemented or performed by devices including general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic, discrete hardware components, or any combination thereof. A general-purpose processor can be a microprocessor, a conventional processor, a controller, a microcontroller, or a state machine. A processor can also be implemented as a combination of computing devices (e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors incorporating a DSP core, or any other such configuration). Therefore, the functions described herein can be implemented in hardware or software and can be performed by a processor, firmware, or any combination thereof. If these functions are implemented in software executed by a processor, then these functions can be stored on a computer-readable medium in the form of instructions or code.

[0125] Computer-readable media include non-transient computer storage media and communication media, with communication media including any medium that facilitates the transmission of code or data. Non-transient storage media can be any available medium that can be accessed by a computer. For example, non-transient computer-readable media can include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), optical disc (CD) or other optical disc storage devices, magnetic disk storage devices, or any other non-transient medium used to carry or store data or code.

[0126] Furthermore, connection components may be appropriately referred to as computer-readable media. For example, if code or data is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technology (such as infrared, radio, or microwave signals), then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technology are all included in the definition of media. Combinations of media are also included within the scope of computer-readable media.

[0127] In this disclosure and the following claims, the word "or" indicates an inclusive list, such that a list of, for example, X, Y, or Z refers to X or Y or Z or XY or XZ or YZ or XYZ. The phrase "based on" is also not used to indicate a closed set of conditions. For example, a step described as "based on condition A" could be based on both condition A and condition B. In other words, the phrase "based on" should be interpreted as meaning "at least partially based on". Furthermore, the words "a" or "an" indicate "at least one".

Claims

1. An image editing method comprising: generating, using a color encoder, a plurality of color embeddings for a plurality of pixels of an image, respectively; identifying a source color embedding corresponding to a source color within the image; segmenting the image by comparing the source color embedding to pixel color embeddings to produce a color segmentation, wherein the color segmentation indicates a portion of the image corresponding to the source color; receiving a target color input corresponding to a target color; generating a target color embedding by applying a color text embedding network to the target color input; identifying the target color based on the target color embedding; and replacing the source color with the target color in the image based on the color segmentation and the target color embedding.

2. The method of claim 1, further comprising: receiving source color text; and generating the source color embedding based on the source color text using the color text embedding network.

3. The method of claim 1, further comprising: identifying a plurality of image colors in the image; presenting the image colors to a user; and receiving an indication from the user identifying the source color from among the image colors in the image.

4. The method of claim 1, further comprising: identifying a color palette based on the source color embedding, wherein the color palette comprises a plurality of colors related to the source color; and displaying the color palette to a user.

5. The method of claim 1, further comprising: determining that the target color input corresponds to a primary color; identifying a plurality of related colors by adding modification text to the target color input; and generating related color embeddings for the related colors using the color text embedding network, wherein the target color embedding is based on the related color embeddings.

6. The method of claim 1, further comprising: identifying a plurality of pixel clusters in the image; and selecting a pixel from each of the pixel clusters, wherein the plurality of pixels correspond to the selected pixels.

7. The method of claim 6, wherein: the pixel clusters are identified based on having similar pixel colors.

8. The method of claim 1, further comprising: computing a similarity score for each of the pixels by comparing the source color embedding to the pixel color embeddings; identifying a similarity threshold; and determining whether the similarity score for each of the pixels is less than the similarity threshold, wherein the color segmentation is based on the determination.

9. The method of claim 8, further comprising: computing a cosine similarity between the source color embedding and each of the pixel color embeddings, wherein the similarity score is based on the cosine similarity.

10. The method of claim 8, further comprising: displaying a threshold control element to a user; and receiving a threshold control value from the threshold control element, wherein the similarity threshold is based on the threshold control value.

11. The method of claim 1, further comprising: ​ ​ ​ ​ ​ ​ ​ displaying the color segmentation to a user; receiving feedback from the user for the color segmentation; and updating the color segmentation based on the feedback.

12. The method of claim 11, further comprising: receiving a lightness value and a saturation value; and adjusting the image based on the lightness value and the saturation value.

13. The method of claim 1, further comprising: receiving an audio signal; and extracting the source color or the target color from the audio signal.

14. An image editing method, comprising: receiving an image, a source color input identifying a source color, and a target color input identifying a target color; generating a source color embedding for the source color based on the source color input; generating a plurality of color pixel embeddings for a plurality of pixels in the image, respectively; segmenting the image by comparing the source color embedding to the pixel color embeddings to produce a color segmentation; generating a target color embedding based on the target color input; identifying a target color representation for the target color; and replacing the source color with the target color in the image based on the color segmentation and the target color representation.

15. The method of claim 14, further comprising: identifying a hue of the target color based on the target color representation, wherein the target color representation comprises an HSL representation; identifying a lightness value and a saturation value; and identifying a replacement color based on the hue of the target color, the lightness value, and the saturation value.

16. The method of claim 15, further comprising: receiving a lightness adjustment value, a saturation adjustment value, or both from a user, wherein the lightness value or the saturation value is based on the lightness adjustment value or the saturation adjustment value, respectively.

17. The method of claim 14, further comprising: computing a LAB space color representation for each of the plurality of pixels, wherein the color pixel embeddings are based on LAB space representations.

18. An image editing apparatus, comprising: a color text embedding network configured to generate a source color embedding based on a source color input, and to generate a target color embedding based on a target color input; a color encoder configured to generate a plurality of pixel color embeddings for a plurality of pixels in an image, respectively; an image segmentation component configured to segment the image by comparing the source color embedding to the pixel color embeddings to produce a color segmentation; and a color replacement component configured to replace a source color with a target color in the image based on the color segmentation and the target color embedding.

19. The apparatus of claim 18, further comprising: an audio converter configured to convert a speech input to the source color input or the target color input.

20. The apparatus of claim 18, further comprising: a user interface configured to receive the source color input for the source color and the target color input for the target color, and to display the image with the source color replaced by the target color.

Images