Method and apparatus for generating a composite image based on colors extracted from an input image

Abstract

In some implementations, a method includes obtaining a first image; determining a characteristic attribute of each of a plurality of pixels within the first image; determining a dominant hue based on one or more characteristic attributes of each of the plurality of pixels within the first image; determining a plurality of hues that meet a predetermined perceptual threshold relative to the dominant hue based on the characteristic attributes of each of the plurality of pixels within the first image, wherein the plurality of hues are different from the dominant hue; and generating a second image based at least in part on the dominant hue and the plurality of hues. In some implementations, the second image corresponds to a color gradient generated based on the dominant hue within the first image. In some implementations, the second image is set as wallpaper for a user interface.

Description

Technical Field

[0001] The present disclosure relates generally to image processing, and in particular, to systems, methods, and apparatus for generating composite images (eg, color gradients) based on colors extracted from an input image. Background Art

[0002] Images are often used as wallpaper for the Home screen user interface. However, it can be difficult to view an image used as wallpaper because multiple application icons are located within the Home screen user interface and overlay the image. Therefore, it is advantageous to automatically generate wallpaper for the Home screen user interface, such as a color gradient based on the image set as the wallpaper for the Wake screen user interface. BRIEF DESCRIPTION OF THE DRAWINGS

[0003] So that the present disclosure may be understood by those skilled in the art, a more detailed description may be had with reference to aspects of certain exemplary implementations, some of which are illustrated in the accompanying drawings.

[0004] Figure 1A Exemplary color spaces according to some implementations are shown.

[0005] Figure 1B Shown according to some specific implementations Figure 1A An exemplary three-dimensional (3D) representation of the color space in .

[0006] Figure 2 An exemplary data processing architecture according to some implementations is shown.

[0007] Figure 3 An exemplary data structure of a pixel feature vector according to some implementations is shown.

[0008] Figures 4A to 4F An exemplary pixel clustering scenario is shown according to some implementations.

[0009] Figure 5 Representations of composite images are shown according to some implementations.

[0010] Figure 6A An exemplary input image is shown that is set as the background of a wake screen user interface according to some implementations.

[0011] Figure 6B According to some specific implementations, Figure 6A Example synthetic images generated from the input images in .

[0012] Figure 7 is a flowchart representation of a method for generating a composite image based on a portion of pixels within an input image, according to some specific implementations.

[0013] Figure 8 is a block diagram of an exemplary electronic device according to some implementations.

[0014] As is common practice, the various features shown in the accompanying drawings may not be drawn to scale. Therefore, the dimensions of various features may be arbitrarily expanded or reduced for clarity. Furthermore, some of the accompanying drawings may not depict all components of a given system, method, or apparatus. Finally, similar reference numerals may be used to denote similar features throughout the specification and accompanying drawings. Summary of the Invention

[0015] Various embodiments disclosed herein include devices, systems, and methods for generating a composite image (e.g., a second image, such as a color gradient) based on colors extracted from an input image (e.g., a first image). According to some embodiments, the method is performed at a device including a non-volatile memory and one or more processors coupled to the non-volatile memory. The method includes obtaining a first image; determining one or more characteristic attributes of each of a plurality of pixels within the first image; determining a dominant hue based on the one or more characteristic attributes of each of the plurality of pixels within the first image; determining a plurality of hues that meet a predetermined perceptual threshold relative to the dominant hue based on the one or more characteristic attributes of each of the plurality of pixels within the first image, wherein the plurality of hues are different from the dominant hue; and generating a second image based at least in part on the dominant hue and the plurality of hues.

[0016] According to some specific implementations, a device includes one or more processors, non-volatile memory, and one or more programs; the one or more programs are stored in the non-volatile memory and are configured to be executed by the one or more processors, and the one or more programs include instructions for performing or causing the performance of any of the methods described herein. According to some specific implementations, a non-volatile computer-readable storage medium has instructions stored therein that, when executed by one or more processors of the device, cause the device to perform or cause the performance of any of the methods described herein. According to some specific implementations, a device includes one or more processors, non-volatile memory, and means for performing or causing the performance of any of the methods described herein. DETAILED DESCRIPTION

[0017] Numerous details are described to provide a thorough understanding of the example implementations shown in the accompanying drawings. However, the accompanying drawings illustrate only some example aspects of the present disclosure and, therefore, should not be considered limiting. One of ordinary skill in the art will appreciate that other effective aspects and / or variations do not include all of the specific details described herein. In addition, well-known systems, methods, components, devices, and circuits are not described in detail in order to avoid obscuring more relevant aspects of the example implementations described herein.

[0018] Specific implementations described herein include methods and devices for generating a composite image (e.g., a color gradient) based on colors extracted from an input image. In some implementations, the composite image corresponds to a color gradient based on a portion of pixels within the input image. Images are often used as wallpapers for home screen user interfaces. However, it can be difficult to view images used as wallpapers because multiple application icons are located within the home screen user interface and overlay the image. Therefore, it would be advantageous to automatically generate a wallpaper for the home screen user interface that is a color gradient or a solid color based on the image set as the wallpaper for the wake screen user interface. Thus, when transitioning from the wake screen user interface to the home screen user interface, visual continuity is maintained, and a person may feel as if he / she is diving deeper into the wallpaper associated with the wake screen user interface. Providing visual continuity between user interfaces reduces visual stress and also reduces visual clutter. Therefore, the method reduces the cognitive burden on the user when interacting with the user interface, thereby creating a more efficient human-computer interface.

[0019] Figure 1A Examples of perceptual color spaces 100 according to some implementations are shown. While relevant features are shown, those skilled in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and so as not to obscure more relevant aspects of the exemplary implementations disclosed herein.

[0020] To this end, as a non-limiting example, color space 100 corresponds to the Munsell color system. According to some implementations, color space 100 specifies colors based on three visual attributes: (A) hue 102; (B) chroma 104 associated with the amount of the color judged in proportion to a neutral (gray) color of the same value; and (C) a value 106 associated with the brightness, lightness, or luminosity of the color. Figure 1A As shown, color space 100 includes three independent color attributes that are cylindrically represented in three dimensions as irregular pure colors: (A) hue 102, measured in degrees around a horizontal circle; (B) chroma 104, measured radially outward from a neutral (gray) vertical axis; and (C) value 106, measured vertically on the core cylinder from 0 (black) to 10 (white). Thus, in some implementations, the cylindrical representation of color space 100 can be divided into multiple pie slices stacked along a vertical axis associated with value 106 (brightness).

[0021] Each circular slice of color space 100 is divided into five primary hues: red, yellow, green, blue, and violet, as well as five intermediate hues between the adjacent primary hues (e.g., yellow-red). Value, or lightness, varies vertically along the color solid, from black (value 0) at the bottom to white (value 10) at the top. Neutral gray lies on the vertical axis between black and white. Chroma, measured radially from the center of each slice, indicates the relative amount of color; lower chroma indicates less purity (more faded, like pastels). Therefore, a color is fully specified by listing three numbers: hue, value, and chroma.

[0022] Figure 1B Shown according to some specific implementations Figure 1A 1. An exemplary three-dimensional (3D) representation 150 of the color space 100 in FIG. The 3D representation 150 is not a regular geometric cylinder, sphere, pyramid, etc., but is irregular. For example, Munsell determined the spacing of colors along these dimensions by measuring human visual responses. In each dimension, the Munsell colors are as close as possible to perceptual uniformity, which makes the resulting shape very irregular. For example, we see perceptually that the highest chroma of a yellow hue has a much higher value than the highest chroma of blue. Therefore, the color space 100 and its 3D representation are related to human perception of color. One of ordinary skill in the art will appreciate that Figure 1A Color space 100 and its Figure 1B The 3D representation 150 in FIG. 1 (associated with the Munsell color system) may be replaced with various other color systems that are also associated with human perception and human visual response, such as IPT, CIECAM02, CIELAB, iCAM06, etc. A person of ordinary skill in the art will appreciate that Figure 1A Color space 100 and its Figure 1B The 3D representation 150 in (associated with the Munsell color system) may be replaced with various other color systems not associated with human perception, such as RGB, RGBA, CMYK, HSL, HSV, YIQ, YCbCr, etc.

[0023] Figure 2 An exemplary data processing architecture 200 is shown according to some implementations. Although relevant features are shown, one of ordinary skill in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and to not obscure more relevant aspects of the exemplary implementations disclosed herein. In some implementations, the data processing architecture 200 (or at least a portion thereof) includes Figure 8 in or integrated with the electronic device 800 shown.

[0024] like Figure 2As shown, the data processing architecture 200 obtains input data associated with multiple modalities, including image data 202A, depth data 202B, and other sensor data 202C. In some implementations, the image data 202A corresponds to one or more images obtained by the electronic device 800. For example, the electronic device 800 receives the image data 202A from a local source (e.g., a solid-state drive (SSD), a hard disk drive (HDD), etc.) or a remote source (e.g., another user's device, a cloud / file server, etc.). In another example, the electronic device 800 retrieves the image data 202A from a local source (e.g., an SSD, an HDD, etc.) or a remote source (e.g., another user's device, a cloud / file server, etc.). In yet another example, the electronic device 800 captures the image data 202A using an outward-facing or inward-facing image sensor (e.g., one or more cameras). As an example, the image data 202A includes an image currently set as the background or wallpaper of the wake-up screen user interface of the electronic device 800.

[0025] In some implementations, depth data 202B represents the scene or physical environment associated with image data 202A. As an example, assuming that image data 202A includes an image of a face, depth data 202B may include a 3D mesh associated with the face. As another example, assuming that image data 202A includes a living room scene, depth data 202B may include a 3D point cloud associated with the living room scene. In some implementations, other sensor data 202C represents the scene or physical environment associated with image data 202A or includes other metadata associated with image data 202A. As an example, other sensor data 202C may include eye tracking information associated with a user, ambient audio data associated with the scene or physical environment, ambient lighting measurements associated with the scene or physical environment, environmental measurements associated with the scene or physical environment (e.g., temperature, humidity, pressure, etc.), physiological measurements associated with objects within the scene or physical environment (e.g., pupil dilation, gaze direction, heart rate, etc.), and the like.

[0026] According to some implementations, the image data 202A corresponds to an ongoing or continuous time series of images or values. Then, the time series converter 220 is configured to generate one or more time frames of image data from the continuous image data stream. Each time frame of image data includes a time portion of the image data 202A. In some implementations, the time series converter 220 includes a windowing module 222 that is configured to generate a time series of images or values ​​for each time T1, T2, ..., T N One or more time frames or portions of the image data 202A are marked and separated. In some implementations, each time frame of the image data 202A is conditioned or otherwise pre-processed by a pre-filter (not shown).

[0027] According to some implementations, the depth data 202B corresponds to an ongoing or continuous time series of values. Then, the time series converter 220 is configured to generate one or more time frames of depth data from the continuous audio data stream. Each time frame of the depth data includes a time portion of the depth data 202B. In some implementations, the time series converter 220 includes a windowing module 222 configured to generate a time series of values ​​for the time periods T1, T2, ..., T N One or more time frames or portions of the depth data 202B are labeled and separated. In some implementations, each time frame of the depth data 202B is conditioned or otherwise pre-processed by a pre-filter (not shown).

[0028] According to some implementations, the other sensor data 202C corresponds to an ongoing or continuous time series value. Then, the time series converter 220 is configured to generate one or more time frames of the other sensor data from the continuous stream of the other sensor data. Each time frame of the other sensor data includes a time portion of the other sensor data 202C. In some implementations, the time series converter 220 includes a windowing module 222 configured to generate a time series value for each of the time frames T1, T2, ..., T N One or more time frames or portions of the other sensor data 202C are labeled and separated. In some implementations, each time frame of the other sensor data 202C is conditioned or otherwise pre-processed by a pre-filter (not shown).

[0029] In various implementations, the data processing architecture 200 includes a privacy subsystem 230 that includes one or more privacy filters associated with user information and / or identification information (e.g., at least some portions of the image data 202A, depth data 202B, and other sensor data 202C). In some implementations, the privacy subsystem 230 selectively prevents and / or restricts the data processing architecture 200, or portions thereof, from acquiring and / or transmitting user information. To this end, the privacy subsystem 230 receives user preferences and / or selections from a user in response to a prompt for the user preferences and / or selections. In some implementations, the privacy subsystem 230 prevents the data processing architecture 200 from acquiring and / or transmitting user information unless and until the privacy subsystem 230 obtains informed consent from the user. In some implementations, the privacy subsystem 230 anonymizes (e.g., scrambles or obfuscates) certain types of user information. For example, the privacy subsystem 230 receives user input specifying which types of user information the privacy subsystem 230 anonymizes. As another example, the privacy subsystem 230 may, independently of user specification (eg, automatically) anonymize certain types of user information that may include sensitive and / or identifying information.

[0030] In some implementations, the task engine 242 is configured to perform one or more tasks on the image data 202A, the depth data 202B, and / or the other sensor data 202C to detect objects, features, activities, etc. associated with the input data. In some implementations, the task engine 242 is configured to detect one or more features (e.g., lines, edges, corners, ridges, shapes, etc.) within the image data 202A based on various computer vision techniques, and to label pixels within the image based on these features. In some implementations, the task engine 242 is configured to identify one or more objects within the image data 202A based on various computer vision techniques (such as semantic segmentation, etc.), and to label pixels within the image based on these techniques. In some implementations, the task engine 242 is configured to determine the visual saliency of one or more objects identified within the image data 202A based on eye tracking information and various computer vision techniques to determine the user's gaze point and / or the importance of each of the one or more objects.

[0031] In some implementations, the foreground / background discriminator 244 is configured to label pixels within the image as foreground and / or background pixels based on the depth data 202B. In some implementations, the color analyzer 246 is configured to determine color attributes of pixels within the image, such as hue values, chroma values, and brightness values.

[0032] In some implementations, the pixel characterization engine 250 is configured to generate pixel feature vectors 251 for at least some pixels of the image within the image data 202A based on the output of the task engine 242, the foreground / background discriminator 244, and / or the color analyzer 246. According to some implementations, the pixel feature vector 251 includes a plurality of feature attributes for each pixel of the image. In some implementations, each pixel is associated with a pixel feature vector that includes feature attributes (e.g., hue value, chroma value, brightness value, coordinates, various labels, etc.). Figure 3 An exemplary pixel feature vector 310 is described in more detail.

[0033] like Figure 3 As shown, the pixel feature vector 310 of the corresponding pixel in the image includes: a hue value 312 of the corresponding pixel, a chroma value 314 of the corresponding pixel, a brightness value 316 of the corresponding pixel, coordinates 318 of the corresponding pixel (e.g., the 2D image plane of the corresponding pixel and optional 3D absolute coordinates of the object associated with the corresponding pixel), a foreground or background label 320 of the corresponding pixel, one or more feature / object labels 322 of the corresponding pixel, and one or more other labels 324 of the corresponding pixel.

[0034] In some implementations, the optional filter subsystem 252 is configured to apply one or more filters or constraints to the pixel feature vectors 251 to generate a set of filtered pixel feature vectors 253. For example, the filter subsystem 252 removes pixel feature vectors that include a foreground label. In another example, the filter subsystem 252 removes pixel feature vectors that include a background label. In yet another example, the filter subsystem 252 removes pixel feature vectors that are not associated with a feature / object label. In yet another example, the filter subsystem 252 removes pixel feature vectors that are not associated with a predefined set of feature labels (e.g., circle, line, square, block, cylinder, etc.). In yet another example, the filter subsystem 252 removes pixel feature vectors that are not associated with a predefined set of object labels (e.g., plant, leaf, animal, person, human face, human skin, sky, etc.). In yet another example, the filter subsystem 252 removes pixel feature vectors that are associated with tonal values ​​within one or more predefined ranges of restricted tones. In yet another example, the filter subsystem 252 removes pixel feature vectors associated with chroma values ​​outside a predefined chroma range. In yet another example, the filter subsystem 252 removes pixel feature vectors associated with luminance values ​​outside a predefined luminance range. For example, the filter subsystem 252 removes pixel feature vectors associated with low visual saliency. In yet another example, the filter subsystem 252 removes pixel feature vectors associated with pixel coordinates near an edge or perimeter of the image data 202A.

[0035] In some implementations, the optional filter subsystem 252 is configured to apply a weight to the pixel feature vector 251 based on the above-mentioned filter or constraint, rather than removing the pixel feature vector. Thus, in some implementations, the image generator 266 can generate one or more composite images 275 based on pixels associated with pixel feature vectors having weights greater than a predefined value. According to some implementations, the image generator 266 can sort portions within the composite image based on the weights of the corresponding pixel feature vectors.

[0036] In some implementations, the clustering engine 262 is configured to perform a clustering algorithm (e.g., k-means clustering, a k-means clustering variant, or another clustering algorithm) on the pixels associated with the set of filtered pixel feature vectors 253 based on at least some of their feature attributes (e.g., hue value, chroma value, brightness value, or a suitable combination thereof). Figures 4A to 4F The performance of the clustering algorithm is described in more detail. In some implementations, after clustering the pixels associated with the set of filtered pixel feature vectors 253, the optional filter subsystem 264 is configured to apply one or more filters or constraints to the pixels associated with the set of filtered pixel feature vectors 253. Figures 4A to 4FThe application of some post-clustering filters or constraints is described in more detail.

[0037] Figures 4A to 4F An exemplary pixel clustering scene 400 according to some implementations is shown. Although relevant features are shown, one of ordinary skill in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and to not obscure more relevant aspects of the exemplary implementations disclosed herein. In some implementations, the pixel clustering scene 400 is composed of Figure 8 The electronic device 800 shown in FIG. 8 or a component thereof, such as the clustering engine 262 or the filter subsystem 264, is executed.

[0038] like Figure 4A As shown, the electronic device 800 or a component thereof (e.g., the clustering engine 262) plots pixels associated with a set of filtered pixel feature vectors 253 based on their hue and chroma values ​​for a 2D color space 402. Those skilled in the art will appreciate that pixels can be plotted for a 3D color space based on their hue, chroma, and luminance values, but for simplicity, a 2D color space 402 is used in this example.

[0039] like Figures 4A to 4F As shown, the 2D color space 402 is divided into five primary colors: red, yellow, green, blue and purple, and five intermediate colors (e.g., yellow-red) between the adjacent primary colors. Those skilled in the art will appreciate that in various other specific implementations, Figures 4A to 4F The number of hue regions or regions within the 2D color space 402 can be any number. Hue values ​​are measured by degrees around the 2D color space 402. Chroma values ​​are measured radially outward from the center (neutral gray) of the 2D color space 402. Thus, undersaturated colors are located closer to the center of the 2D color space 402, while oversaturated colors are located closer to the outer edges of the 2D color space 402. According to some implementations, if two or more pixels have the same hue value and chroma value, the clustering engine 262 can associate a counter that indicates the number of pixels associated with the sample when the pixel is mapped against the 2D color space 402.

[0040] like Figure 4B As shown, the electronic device 800 or a component thereof (e.g., the clustering engine 262) divides the pixels into k clusters based on the k-means clustering algorithm. Here, assuming k=3, the clustering engine 262 divides the pixels into three (3) clusters, including clusters 410, 420, and 430. Figure 4B As shown, two pixels in the red and yellow-red regions outside of clusters 410, 420, and 430 are displayed in a cross-hatched pattern to indicate that those pixels have been removed.

[0041] like Figure 4C As shown, the electronic device 800 or a component thereof (e.g., the filter subsystem 264) removes undersaturated pixels within the first chroma exclusion zone 442. Figure 4C , pixels within the first chroma exclusion zone 442 are shown in a cross-hatched pattern to indicate that those pixels have been removed. Figure 4C As shown, the filter subsystem 264 also removes oversaturated pixels within the second chroma exclusion zone 444. Figure 4C , pixels within the second chroma exclusion zone 444 are displayed in a cross-hatched pattern to indicate that those pixels have been removed. According to some implementations, removing undersaturated and oversaturated pixels is optional. According to some implementations, the widths of the first chroma exclusion zone 442 and the second chroma exclusion zone 444 can be predefined or deterministic.

[0042] like Figure 4D As shown, the electronic device 800 or a component thereof (e.g., the filter subsystem 264) determines the center of mass or gravity of each of the clusters 410, 420, and 430. Figure 4D , centroid 452 indicates the center of mass or center of gravity of cluster 410. Figure 4D , centroid 454 indicates the center of mass or center of gravity of cluster 420. Figure 4D , centroid 456 indicates the center of mass or center of gravity of cluster 430 .

[0043] like Figure 4E As shown, the electronic device 800 or a component thereof (e.g., the filter subsystem 264) determines a first hue inclusion region 462 corresponding to X° (e.g., 10°) on either side of the centroid 452, and removes pixels associated with the cluster 410 that are outside the first hue inclusion region 462. Figure 4E , pixels within cluster 410 and within the purple-blue region are shown with a cross-hatched pattern to indicate that those pixels have been removed. According to some implementations, pixels that were not previously within cluster 410 or removed based on a previous filter can be re-included or re-associated with cluster 410 if the pixel is within first hue inclusion region 462. According to some implementations, the degree associated with first hue inclusion region 462 can be predefined or deterministic.

[0044] like Figure 4E As shown, the electronic device 800 or a component thereof (e.g., the filter subsystem 264) determines a second hue inclusion region 464 corresponding to X° on either side of the centroid 454 and removes pixels associated with the cluster 420 that are outside the second hue inclusion region 464. Figure 4EAs shown, the electronic device 800 or a component thereof (e.g., the filter subsystem 264) determines a third hue inclusion region 466 corresponding to X° on either side of the centroid 456 and removes pixels associated with the cluster 430 outside the third hue inclusion region 466.

[0045] like Figure 4F As shown, the electronic device 800 or a component thereof (e.g., the clustering engine 262) identifies candidate pixels for the image generator 266. Figure 4F In , candidate pixels are shown with solid black fill.

[0046] refer to Figure 2 In some implementations, the image generator 266 is configured to generate one or more composite images 275. As an example, the image generator 266 generates one or more composite images 275 based on the candidate pixels. Figure 4F As another example, the image generator 266 generates a composite image based on a portion of the candidate pixels that are within the cluster with the most candidate pixels, e.g. Figure 4F In another example, the image generator 266 is based on the clustering 410 in Figure 4F 4. The image generator 266 generates M composite images using a portion of the candidate pixels within the M clusters with the most candidate pixels. In this example, assuming M=2, the image generator 266 generates a first composite image using the candidate pixels within cluster 410 and a second composite image using the candidate pixels within cluster 420. In this example, M≤k, which is associated with the k-means clustering algorithm.

[0047] Figure 5 Representations of composite images 500 and 550 according to some implementations are shown. Although relevant features are shown, one of ordinary skill in the art will recognize from this disclosure that various other features are not shown for the sake of brevity and so as not to obscure more relevant aspects of the exemplary implementations disclosed herein. For example, Figure 5 shows a set of candidate pixels (e.g., Figure 4F 4 (the remaining pixels within cluster 410 are shown with black fill in the image). For example, first composite image 500 includes a plurality of portions 502A-502K (sometimes referred to herein as portions 502). According to some implementations, each portion 502 is associated with a corresponding pixel within the set of candidate pixels. In some implementations, portions 502 correspond to horizontal regions, vertical regions, diagonal regions, etc. One of ordinary skill in the art will appreciate that portions 502 can be associated with numerous shapes, geometric shapes, partitions, etc.

[0048] In some implementations, the image generator 266 generates each of the portions 502 based on the hue value and the chroma value of the corresponding pixel within the set of candidate pixels. Thus, for example, portion 502B is generated based on the hue value and the chroma value of the corresponding pixel within the set of candidate pixels. In some implementations, the image generator 266 performs fading, smoothing, blending, interpolation, and / or similar operations between each portion 502. Thus, the first composite image 500 can resemble a color gradient.

[0049] like Figure 5 As shown, the first composite image 500 includes four (4) portions 502D because the set of candidate pixels includes four (4) pixels having the same hue value and chroma value. Similarly, the first composite image 500 includes two (2) portions 502A because the set of candidate pixels includes two (2) pixels having the same hue value and chroma value.

[0050] like Figure 5 As shown, portions 502 within the first composite image 500 are arranged from brightest at the bottom to least bright at the top based on the luminance values ​​of the associated pixels. Thus, with reference to the first composite image 500, a first portion 502B of the first composite image 500 is generated based on the hue value and chroma value of the corresponding first candidate pixel associated with the highest luminance value from the set of candidate pixels. Furthermore, with reference to the first composite image 500, an Nth portion 502F of the first composite image 500 is generated based on the hue value and chroma value of the corresponding Nth candidate pixel associated with the lowest luminance value from the set of candidate pixels.

[0051] For example, Figure 5 A representation of a second composite image 550 associated with the set of candidate pixels is shown. The second composite image 550 is similar to and modified from the first composite image 500. Therefore, like reference numbers are used and only the differences are discussed for brevity. Figure 5 As shown, the portions 502 within the second composite image 550 are arranged from least at the bottom to most at the top based on pixel instance counts. Thus, with reference to the second composite image 550, a portion 502K of the second composite image 550 is generated based on hue values ​​and chroma values ​​of corresponding candidate pixels from the set of candidate pixels, where the count within the set of candidate pixels is one (1) pixel instance. Additionally, with reference to the second composite image 550, four (4) portions 502D of the second composite image 550 are generated based on hue values ​​and chroma values ​​of corresponding candidate pixels from the set of candidate pixels associated with a count of four (4) pixel instances within the set of candidate pixels. One of ordinary skill in the art will appreciate that the portions 502 can be arranged or ordered in a variety of ways other than based on luminance or pixel instance counts (e.g., hue values, chroma values, weights set by the filter subsystem 252, etc.).

[0052] Figure 6A An exemplary input image 610 is shown that is set as wallpaper for the wake screen user interface 600 according to some implementations. For example, the input image 610 (e.g., image data 202A) is fed into the data processing system 200 along with associated metadata. Figure 6A As shown, input image 610 includes pixels 662, 664, 666, and 668 associated with related colors (shown as different cross-hatched patterns). As an example, pixels 662, 664, 666, and 668 are associated with hue and chroma values ​​corresponding to different shades of red within input image 610, where, for example, pixel 662 is associated with the primary hue / shade.

[0053] Figure 6B An exemplary composite image 660 (eg, one of the one or more composite images 275) generated by the data processing system 200 based on the input image 610 is shown according to some implementations. Figure 2 , the color analyzer 246 determines the hue value, chroma value, and luminance value of pixels within the input image 610 (e.g., including pixels 662, 664, 666, and 668), and the clustering engine 262 clusters pixels based on their hue value and chroma value for a color space (e.g., Figures 4A to 4F 2D color space 402 in ) draws the pixels within the input image 610. In this example, continue to refer to Figure 2 , clustering engine 262 identifies a set of candidate pixels (e.g., pixels 662, 664, 666, and 668), as described above with respect to Figures 4A to 4F 4 and 666. The pixel clustering scenario 400 in FIG. 4 is described, and the image generator 266 generates a composite image 660 based on the set of candidate pixels (e.g., pixels 662, 664, 666, and 668) or at least a portion thereof, as described above with respect to FIG. Figure 5 Thus, composite image 660 corresponds to a color gradient generated based on hue and chroma values ​​associated with pixels 662, 664, 666, and 668, which are closely related in color (e.g., shades of red).

[0054] like Figure 6BAs shown, the composite image includes a plurality of portions 672, 674, 676, and 678. The color of portion 672 is based on the hue value and chroma value of pixel 662. Furthermore, in some implementations, a size 663 (e.g., length) of portion 672 is based on a pixel instance count corresponding to a number of pixels in input image 610 that have the same hue value and chroma value as pixel 662 (or within a predefined variance thereof). Similarly, the color of portion 674 is based on the hue value and chroma value of pixel 664. Furthermore, in some implementations, a size 665 (e.g., length) of portion 674 is based on a pixel instance count corresponding to a number of pixels in input image 610 that have the same hue value and chroma value as pixel 66 (or within a predefined variance thereof).

[0055] The color of portion 676 is based on the hue value and chroma value of pixel 666. Furthermore, in some implementations, a size 667 (e.g., length) of portion 676 is based on a pixel instance count corresponding to a number of pixels in input image 610 that have the same hue value and chroma value as pixel 666 (or within a predefined variance thereof). Similarly, the color of portion 678 is based on the hue value and chroma value of pixel 668. Furthermore, in some implementations, a size 669 (e.g., length) of portion 678 is based on a pixel instance count corresponding to a number of pixels in input image 610 that have the same hue value and chroma value as pixel 668 (or within a predefined variance thereof).

[0056] For example, in response to detecting a user input to set the input image 610 as the wallpaper of the wake screen user interface 600, the electronic device 800 or a component thereof (e.g., Figure 2 62 / 855,729, filed May 31, 2019, and incorporated herein by reference in its entirety for further description of setting wallpapers for wake-up and home screen user interfaces.

[0057] Figure 7is a flowchart representation of a method 700 for generating a composite image based on a portion of pixels within an input image, according to some implementations. In various implementations, the method 700 is performed by a processor having one or more processors and a non-transitory memory (e.g., Figure 8 The method 700 is performed by a device (e.g., an electronic device 800) or a component thereof. In some implementations, the method 700 is performed by a processing logic component (including hardware, firmware, software, or a combination thereof). In some implementations, the method 700 is performed by a processor that executes code stored in a non-transitory computer-readable medium (e.g., a memory). In some implementations, the device corresponds to one of a wearable computing device, a mobile phone, a tablet computer, a laptop computer, a desktop computer, an information kiosk, etc. Some operations in the method 700 are optionally combined, and / or the order of some operations is optionally changed.

[0058] As described below, method 700 generates a composite image to be set as wallpaper for another user interface (e.g., a home screen) based on an input image of wallpaper set as a user interface (e.g., a wake-up screen) to maintain visual continuity between user interfaces. The method provides visual continuity between user interfaces, thereby reducing the amount of user interaction with the device. Reducing the amount of user interaction with the device reduces wear and tear on the device and, for battery-powered devices, extends the battery life of the device. The method also reduces the cognitive burden on the user when interacting with the user interface, thereby creating a more efficient human-computer interface.

[0059] As shown in block 702, method 700 includes obtaining a first image. In some implementations, the first image corresponds to an input or reference image of method 700. For example, the first image corresponds to a wallpaper of a wake-up screen user interface. For example, a reference image Figure 2 , a data processing system 200 included in or integrated with electronic device 800 obtains image data 202A including a first image. For example, data processing system 200 receives image data 202A from a local or remote source. In another example, data processing system 200 retrieves image data 202A from a local or remote source. In yet another example, data processing system 200 captures image data 202A using an outward-facing or inward-facing image sensor. As one example, image data 202A includes an image set as wallpaper for a wake-up screen user interface of electronic device 800.

[0060] In some implementations, method 700 is triggered in response to detecting a user input corresponding to setting a first image as the wallpaper of a wake-up screen user interface. Thus, the second image is generated based on the first image and is set as the wallpaper of the home screen user interface. See U.S. patent application Ser. No. 62 / 855,729, filed May 31, 2019, attorney docket number P42113USP1 / 27753-50286PR, the entirety of which is incorporated herein by reference for further description of setting wallpapers for wake-up and home screen UIs. In some implementations, method 700 is triggered in response to detecting a user input corresponding to setting a first image as the wallpaper of a home screen user interface.

[0061] In some implementations, method 700 is triggered in response to detecting user input corresponding to selecting a "smart gradient" or "color gradient" home screen processing option within a wallpaper setting user interface. Thus, a second image is generated based on the first image and is set as the wallpaper of the home screen user interface within a preview pairing that shows the first image as the wallpaper of the wake screen user interface and the second image as the wallpaper of the home screen user interface. See U.S. patent application Ser. No. 62 / 855,729, attorney docket No. P42113USP1 / 27753-50286PR, filed May 31, 2019, which is incorporated herein by reference in its entirety for further description of the wallpaper setting user interface and home screen processing options.

[0062] As shown in block 704, method 700 includes determining one or more characteristic attributes for each of a plurality of pixels within the first image. In some implementations, a portion of the one or more characteristic attributes corresponds to color parameters, including hue values, saturation values, brightness / brightness values, and the like. Thus, in some implementations, the one or more characteristic attributes include hue values, saturation values, and brightness values ​​for the corresponding pixels. In some implementations, a portion of the one or more characteristic attributes corresponds to derived or relative parameters, such as object labels, feature labels, foreground / background labels, and the like.

[0063] In some implementations, the device generates a pixel feature vector for at least some pixels in the first image, the pixel feature vector including at least one or more feature attributes. For example, the pixel feature vector for a corresponding pixel in the first image includes color parameters (e.g., hue value, hue / saturation value, brightness / luminance value, etc.), depth parameters (e.g., foreground / background label), and other parameters (e.g., image plane coordinates, features / objects, etc.).

[0064] For example, reference Figure 2, the data processing system 200 or one or more components thereof (e.g., task engine 242, foreground / background discriminator 244, and color analyzer 246) analyzes the first image to generate one or more feature attributes (e.g., color parameters, depth parameters, and / or other parameters) for each of at least some pixels within the first image. Continuing with this example, the data processing system 200 or a component thereof (e.g., pixel characterization engine 250) generates a pixel feature vector for at least some pixels within the first image, the pixel feature vector including the one or more feature attributes (e.g., color parameters, depth parameters, and / or other parameters). Figure 3 As shown, the pixel feature vector 310 of the corresponding pixel includes: a hue value 312 of the corresponding pixel, a chromaticity value 314 of the corresponding pixel, a brightness value 316 of the corresponding pixel, coordinates 318 of the corresponding pixel (e.g., the 2D image plane of the corresponding pixel and optionally the 3D absolute coordinates of the object associated with the corresponding pixel), a foreground or background label 320 of the corresponding pixel, one or more feature / object labels 322 of the corresponding pixel, and one or more other labels 324 of the corresponding pixel.

[0065] As shown in block 706, method 700 includes determining a dominant hue based on one or more characteristic attributes of each of a plurality of pixels within the first image. In some implementations, the dominant hue corresponds to a hue value associated with a significant number of pixels within the first image. In some implementations, the dominant hue corresponds to a median, mean, or centroid hue value within a predefined hue range or radius associated with the significant number of pixels within the first image.

[0066] In some implementations, the device maps pixels within the first image to a color space (e.g., 2D or 3D) based on their color parameters and performs a clustering algorithm, such as k-means, on them. For example, the color space is selected to correspond to human perception (e.g., Munsell color system, IPT, CIECAM02, CIELAB, iCAM06, etc.). For example, referring to Figure 2 and Figure 4A , the data processing system 200 or a component thereof (e.g., the clustering engine 262) plots the pixels associated with the first image based on their hue values ​​and chroma values ​​with respect to the 2D color space 402. For example, referring to Figure 2 and Figure 4B , the data processing system 200 or a component thereof (e.g., the clustering engine 262) divides the pixels associated with the first image into k clusters based on the k-means clustering algorithm. Here, assuming k=3, the clustering engine 262 divides the pixels into three clusters, including clusters 410, 420, and 430.

[0067] In some implementations, when using the k-means algorithm, k is set to a predefined or determined value to generate fine-grained clusters with low dispersion. For example, k ≥ 3. In some implementations, the dominant hue corresponds to the cluster associated with a large number of pixels in the first image. In some implementations, the dominant hue corresponds to the median, mean, or centroid hue value within the cluster associated with a large number of pixels in the first image. For example, referring to Figure 2 and Figure 4F , the data processing system 200 or its components determine the dominant hue by determining the median, hue, or centroid of the cluster 410 (eg, the cluster with the most candidate pixels). Figure 2 and Figure 4F , data processing system 200 or a component thereof determines the dominant hue by identifying pixels within cluster 410 having the highest pixel instance count (eg, the cluster with the most candidate pixels).

[0068] In some implementations, if two identical dominant hues exist in the first image, the device randomly or pseudo-randomly selects one of the two identical dominant hues. In some implementations, the device determines whether the first image is a black and white image before continuing the method. If the first image is a black and white image, the device may change the clustering algorithm, plot the pixels for a different color space, and / or perform a different filtering operation (e.g., analyzing only the luminance values ​​of the pixels in the black and white image to generate a dark to light or light to dark gradient). As an example, if the first image is a black and white image, the device abandons removing undersaturation and oversaturation, such as Figure 4C shown.

[0069] In some implementations, determining the dominant hue within the first image includes discarding one or more pixels within the first image that are associated with luminance values ​​outside a range of luminance values. Thus, in some implementations, the device discards outlier pixels that are too bright or too dark. For example, referring to Figure 2 , the data processing system 200 or a component thereof (e.g., the filter subsystem 254 or 264) discards pixels within the first image that are associated with luminance values ​​outside of a luminance range. In some implementations, the luminance range corresponds to a predefined range of luminance values. In some implementations, the luminance range corresponds to a determined range of luminance values ​​based on hue values, chroma values, and / or luma values ​​associated with pixels within the first image.

[0070] In some implementations, determining the dominant hue within the first image includes discarding one or more pixels within the first image that are associated with saturation values ​​outside a range of saturation values. Thus, in some implementations, the device discards outlier pixels within the first image that are oversaturated or undersaturated. For example, referring to Figure 2 and Figure 4C, the data processing system 200 or a component thereof (e.g., the filter subsystem 264) discards undersaturated pixels within the first chroma exclusion zone 442 and also discards oversaturated pixels within the second chroma exclusion zone 444. In some implementations, the first chroma exclusion zone 442 and the second chroma exclusion zone 444 are associated with a predefined range of chroma values. In some implementations, the first chroma exclusion zone 442 and the second chroma exclusion zone 444 are associated with a determined range of chroma values ​​based on hue values, chroma values, and / or luminance values ​​associated with pixels within the first image.

[0071] In some implementations, determining the dominant hue within the first image includes discarding one or more pixels within the first image that are associated with the foreground of the first image based on depth information associated with the first image. In some implementations, the second image is based on pixels associated with the background of the first image and the hues therein. Alternatively, in some implementations, the device discards one or more pixels within the first image that are associated with the background of the first image. For example, referring to Figure 2 , the data processing system 200 or a component thereof (e.g., the filter subsystem 254 or 264) discards pixels in the first image that are associated with the foreground label. Figure 2 , data processing system 200 or a component thereof (eg, filter subsystem 254 or 264 ) discards pixels within the first image that are associated with the background label.

[0072] As shown in box 708, method 700 includes determining a plurality of hues that meet a predetermined perceptual threshold relative to a dominant hue based on one or more characteristic attributes of each of a plurality of pixels in the first image, wherein the plurality of hues are different from the dominant hue. In some embodiments, the plurality of hues include one or more hues that are similar to but different from the dominant hue. In some embodiments, the plurality of hues are associated with pixels in the image that are associated with hue values ​​within a predefined hue angle relative to the dominant hue. In some embodiments, the hue angle is a predefined number of degrees (e.g., + / - 10°) on either side of the dominant hue. In some embodiments, the hue angle is biased in one direction based on the dominant hue. In some embodiments, the plurality of hues are associated with pixels in the image that are associated with hue values ​​associated with the dominant hue within a cluster. In some embodiments, the plurality of hues are associated with pixels in the image that are associated with hue values ​​within a dispersion threshold that is associated with a Y standard deviation relative to the dominant hue. For example, with reference to Figure 2 and Figure 4F After determining that the primary hue corresponds to a pixel within cluster 410 , data processing system 200 or components thereof determine a plurality of hues by identifying other pixels within cluster 410 .

[0073] In some implementations, the predetermined perception threshold corresponds to a hue angle. For example, the hue angle limits the various hues to natural lighting environments and / or human visual perception. For example, referring to Figure 2 and Figure 4E , the data processing system 200 or a component thereof (e.g., the filter subsystem 264) discards pixels within the cluster 410 that are outside the first hue inclusion region 462 and correspond to pixels on either side of the centroid 452 at X° (e.g., 10). As another example, the hue angle is replaced by a dispersion threshold value associated with the Y standard deviation relative to the dominant hue.

[0074] As shown in block 710, method 700 includes generating a second image based at least in part on the primary hue and the plurality of hues. In some implementations, the second image corresponds to a color gradient generated based on the primary hue and the plurality of hues within the first image. In some implementations, the second image corresponds to a grid / checkerboard pattern or another template pattern filled with the primary hue and the plurality of hues within the first image. In some implementations, the second image includes a template character whose clothing is colored using the primary hue and the plurality of hues within the first image. In some implementations, the second image corresponds to a template image (e.g., a group of balloons) colored using the primary hue and the plurality of hues within the first image.

[0075] In some implementations, method 700 includes setting the second image as a wallpaper for a user interface, such as a home screen user interface or a wake screen user interface. In some implementations, method 700 includes setting a background for a movie summary field, etc., based on a dominant hue and multiple hues within the first image associated with a movie poster. In some implementations, method 700 includes setting a background for an album summary field, a scrolling lyrics field, associated buttons, etc., based on a dominant hue and multiple hues within the first image associated with album cover art.

[0076] In some implementations, when the dominant hue is the only hue in the first image, the second image may be a solid color corresponding to the dominant hue. In some implementations, when the dominant hue is the only hue in the first image, the second image may be a combination of the dominant hue and one or more other hues randomly selected within a predefined hue angle from the dominant hue.

[0077] In some implementations, the second image includes multiple parts, at least one part for the primary hue and at least one part for each of the multiple hues. In some implementations, the size of the part corresponds to the number of pixels associated with the corresponding hue within the first image. In some implementations, two or more composite images are generated based on the first image (e.g., smart gradient or color gradient options). Thus, for example, two primary hues are identified by setting k=4 and taking the M clusters with the highest number of samples (where M≤k). In this example, the device generates a first color gradient (e.g., a first composite image) based on the first primary hue and multiple hues associated with pixels close to it, and generates a second color gradient (e.g., a second composite image) based on the second primary hue and multiple hues associated with pixels close to it.

[0078] For example, reference Figure 2 , the data processing system 200 or a component thereof (eg, the image generator 266) generates one or more composite images 275 based on the candidate pixels, the candidate pixels being Figure 4F As another example, the image generator 266 generates a composite image based on a portion of the candidate pixels that are within the cluster with the most candidate pixels, such as Figure 4F In another example, the image generator 266 is based on the clustering 410 in Figure 4F 4. The image generator 266 generates M composite images using a portion of the candidate pixels within the M clusters with the most candidate pixels. In this example, assuming M=2, the image generator 266 generates a first composite image using the candidate pixels within cluster 410 and a second composite image using the candidate pixels within cluster 420. In this example, M≤k, which is associated with the k-means clustering algorithm.

[0079] In some implementations, the second image includes one or more portions associated with different hues within the first image, including a first portion of the second image associated with a primary hue and a second portion of the second image associated with a corresponding hue from the plurality of hues. For example, each of the one or more portions corresponds to a horizontal or vertical band. In some implementations, the shape or size of the portions is based on objects, features, etc. identified within the first image. For example, if the first image includes many circular or curved features, the portions may correspond to curves or ellipsoids.

[0080] For example, Figure 5 shows a set of candidate pixels (e.g., Figure 4F4 (the remaining pixels within the clusters 410 in the set of candidate pixels). For example, the first composite image 500 includes a plurality of portions 502A-502K (sometimes referred to herein as portions 502). According to some implementations, each portion in the portions 502 is associated with a corresponding pixel within the set of candidate pixels. In some implementations, the portions 502 correspond to horizontal regions, vertical regions, diagonal regions, etc. In some implementations, the image generator 266 generates each of the portions 502 based on the hue value and chroma value of the corresponding pixel within the set of candidate pixels. Thus, for example, portion 502B is generated based on the hue value and chroma value of the corresponding pixel within the set of candidate pixels. In some implementations, the image generator 266 performs fading, smoothing, blending, interpolation, and / or similar operations between each portion 502. Thus, the first composite image 500 can resemble a color gradient.

[0081] In some implementations, a first size of a first portion of the second image associated with a dominant hue is based at least in part on a first number of pixels associated with the dominant hue within the first image, and wherein a second size of a second portion of the second image associated with a respective hue from the plurality of hues is based at least in part on a second number of pixels associated with the respective hue within the first image. For example, the first size and the second size correspond to one or more dimension values, such as length and / or width values. Figure 5 As shown, the first composite image 500 includes four (4) portions 502D because the set of candidate pixels includes four (4) pixels having the same hue value and chroma value. Similarly, the first composite image 500 includes two (2) portions 502A because the set of candidate pixels includes two (2) pixels having the same hue value and chroma value.

[0082] In some implementations, a first size of a first portion of the second image and a second size of a second portion of the second image are constrained by predefined dimensional criteria. As an example, the first portion and the second portion are constrained to be A% of the second image. Thus, even if multiple pixels in the first image have the same hue and chroma values, the portion sizes associated with those pixels are constrained to be A% of the second image. For another example, the first portion and the second portion are constrained to a predefined length or width.

[0083] In some implementations, a predefined transformation operation is performed between the one or more portions. In some implementations, the predefined transformation operation corresponds to a smoothing, blending, or interpolation operation. For example, the device avoids stripes, lines, blocks, etc. in the second image and instead creates a color gradient for the second image.

[0084] In some implementations, one or more portions are arranged according to one or more ranking criteria. In some implementations, the one or more ranking criteria correspond to brightness. Thus, for example, the device orders the portions based on perceived brightness from brightest to darkest (or darkest to brightest), or vice versa. In some implementations, the one or more ranking criteria correspond to pixel instance counts. Thus, for example, the device orders the portions in ascending or descending order based on the number of pixels associated with them in the first image.

[0085] like Figure 5 As shown, portions 502 within first composite image 500 are arranged from brightest at the bottom to least bright at the top based on the luminance values ​​of the associated pixels. Thus, with reference to first composite image 500, first portion 502B of first composite image 500 is generated based on the hue and chroma values ​​of the corresponding first candidate pixel associated with the highest luminance value. Furthermore, with reference to first composite image 500, Nth portion 502F of first composite image 500 is generated based on the hue and chroma values ​​of the corresponding Nth candidate pixel associated with the lowest luminance value.

[0086] like Figure 5 As shown, the portions 502 within the second composite image 550 are arranged from least at the bottom to most at the top based on the pixel instance counts. Thus, with reference to the second composite image 550, a portion 502K of the second composite image 550 is generated based on the hue values ​​and chroma values ​​of corresponding candidate pixels having a count of one (1) pixel instance within the set of candidate pixels. Additionally, with reference to the second composite image 550, four (4) portions 502D of the second composite image 550 are generated based on the hue values ​​and chroma values ​​of corresponding candidate pixels associated with a count of four (4) pixel instances within the set of candidate pixels.

[0087] In some implementations, method 700 includes: after generating the second image, detecting an input corresponding to modifying the first image to generate a modified first image; and in response to detecting the input, updating the second image based on one or more visual attributes of each of a plurality of pixels in the modified first image. For example, the input may correspond to a cropping input relative to the first image, application of an image filter to the first image, a labeling or annotation input relative to the first image, etc. Thus, the device updates the second image based on the modification or change to the first image.

[0088] It should be understood that Figure 7 The specific order in which the operations are described is merely exemplary and is not intended to indicate that the order is the only order in which the operations can be performed. A person of ordinary skill in the art will recognize many ways to reorder the operations described herein.

[0089] Figure 88 is a block diagram of an example of an electronic device 800 (e.g., a wearable computing device, a mobile phone, a tablet computer, a laptop computer, a desktop computer, a kiosk, etc.) according to some implementations. Although some specific features are shown, those skilled in the art will appreciate from this disclosure that various other features are not shown for the sake of brevity and so as not to obscure more relevant aspects of the implementations disclosed herein. For this purpose, as a non-limiting example, in some implementations, the electronic device 800 includes one or more processing units 802 (e.g., a microprocessor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a graphics processing unit (GPU), a central processing unit (CPU), a processing core, etc.), one or more input / output (I / O) devices and sensors 806, one or more communication interfaces 808 (e.g., a Universal Serial Bus (USB), Institute of Electrical and Electronics Engineers (IEEE) 802.3x, IEEE 802.11x, IEEE 802.16x, Global System for Mobile Communications (GSM), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Global Positioning System (GPS), infrared (IR), Bluetooth, ZIGBEE, and / or similar types of interfaces), one or more programming (e.g., I / O) interfaces 810, one or more displays 812, one or more optional internal-facing and / or external-facing image sensors 814, one or more optional depth sensors 816, memory 820, and one or more communication buses 804 for interconnecting these and various other components.

[0090] In some implementations, the one or more communication buses 804 include circuits that interconnect and control communications between system components. In some implementations, the one or more I / O devices and sensors 806 include at least one of an inertial measurement unit (IMU), an accelerometer, a gyroscope, a thermometer, one or more physiological sensors (e.g., a blood pressure monitor, a heart rate monitor, a blood oxygen sensor, a blood glucose sensor, etc.), one or more microphones, one or more speakers, a haptic engine, a heating and / or cooling unit, a skin shear engine, etc.

[0091] In some implementations, the one or more displays 812 are configured to present a user interface or other content to a user. In some implementations, the one or more displays 812 correspond to holographic, digital light processing (DLP), liquid crystal display (LCD), liquid crystal on silicon (LCoS), organic light emitting field effect transistor (OLET), organic light emitting diode (OLED), surface conduction electron emitter display (SED), field emission display (FED), quantum dot light emitting diode (QD-LED), microelectromechanical system (MEMS), and / or similar display types. In some implementations, the one or more displays 812 correspond to waveguide displays such as diffractive, reflective, polarization, holographic, etc.

[0092] In some implementations, the one or more optional interior- and / or exterior-facing image sensors 814 correspond to one or more RGB cameras (e.g., with complementary metal oxide semiconductor (CMOS) image sensors or charge coupled device (CCD) image sensors), IR image sensors, event-based cameras, etc. In some implementations, the one or more optional depth sensors 816 correspond to sensors that measure depth based on structured light, time of flight, etc.

[0093] Memory 820 includes high-speed random access memory, such as dynamic random access memory (DRAM), static random access memory (SRAM), double data rate random access memory (DDR RAM) or other random access solid-state memory devices. In some specific implementations, memory 820 includes non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices or other non-volatile solid-state storage devices. Memory 820 optionally includes one or more storage devices remotely located from one or more processing units 802. Memory 820 includes non-transitory computer-readable storage media. In some specific implementations, memory 820 or the non-transitory computer-readable storage medium of memory 820 stores the following programs, modules and data structures or their subsets, including optional operating system 830, data acquirer 842, data sender 844, presentation engine 846 and data processing system 200.

[0094] The operating system 830 includes processes for handling various basic system services and for performing hardware-related tasks.

[0095] In some implementations, the data acquirer 842 is configured to obtain data (e.g., presentation data, user interaction data, sensor data, location data, etc.) from at least one of the I / O devices and the sensor 806 of the electronic device 800 and another local and / or remote source. To this end, in various implementations, the data acquirer 842 includes instructions and / or logic for these instructions, as well as heuristics and metadata for the heuristics.

[0096] In some implementations, the data transmitter 844 is configured to transmit data (e.g., presentation data, location data, user interaction data, etc.) to another device. To this end, in various implementations, the data transmitter 844 includes instructions and / or logic for instructions, as well as heuristics and metadata for the heuristics.

[0097] In some implementations, the presentation engine 846 is configured to present a user interface and other content to a user via the one or more displays 812. To this end, in various implementations, the presentation engine 846 includes instructions and / or logic for such instructions, as well as heuristics and metadata for such heuristics.

[0098] In some implementations, the data processing system 200 is configured to generate one or more composite images based on the image data 202A, the depth data 202B, and / or the other sensor data 202C, as described above with reference to FIG. Figure 2 To this end, in various implementations, the data processing system 200 includes a task engine 242, a foreground / background discriminator 244, a color analyzer 246, a pixel characterization engine 250, filter subsystems 252 / 264, a clustering engine 262, and an image generator 266. Figure 2 The aforementioned components of data processing system 200 are described and will not be described again for the sake of brevity.

[0099] Although the data acquirer 842, data transmitter 844, rendering engine 846, and data processing system 200 are illustrated as residing on a single device (e.g., electronic device 800), it should be understood that in other implementations, any combination of the data acquirer 842, data transmitter 844, rendering engine 846, and data processing system 200 may be located in separate computing devices.

[0100] also, Figure 8 It serves more as a functional description of various features present in a particular implementation than as a structural representation of the implementations described herein. As one of ordinary skill in the art will recognize, items shown separately may be combined, and some items may be separated. For example, Figure 8Some functional modules shown separately in the figure may be implemented in a single module, and the various functions of a single functional block may be implemented by one or more functional blocks in various specific implementations. The actual number of modules and the division of specific functions and how the features are distributed among them will vary depending on the specific implementation and, in some specific implementations, will depend in part on the specific combination of hardware, software, and / or firmware selected for a particular embodiment.

[0101] Although various aspects of specific implementations within the scope of the appended claims have been described above, it should be apparent that the various features of the above-described specific implementations can be embodied in a variety of forms, and any specific structures and / or functions described above are merely illustrative. Based on this disclosure, it should be understood by those skilled in the art that the aspects described herein can be implemented independently of any other aspects, and two or more of these aspects can be combined in various ways. For example, any number of aspects set forth herein can be used to implement an apparatus and / or a method can be practiced. In addition, in addition to or different from one or more aspects set forth herein, other structures and / or functions can be used to implement such an apparatus and / or such a method can be practiced.

[0102] It will also be understood that, although the terms "first," "second," etc., may be used herein to describe various elements, these elements should not be limited by these terms. These terms are simply used to distinguish one element from another. For example, a first node may be referred to as a second node, and similarly, a second node may be referred to as a first node, which changes the meaning of the description as long as all occurrences of "first node" are consistently renamed and all occurrences of "second node" are consistently renamed. A first node and a second node are both nodes, but they are not the same node.

[0103] The terms used herein are merely for describing specific implementations and are not intended to limit the claims. As used in the description of this specific implementation and the appended claims, the singular forms "a" and "the" are intended to also cover the plural forms, unless the context clearly indicates otherwise. It will also be understood that the terms "and / or" used herein refer to and cover any and all possible combinations of one or more of the associated listed items. It will also be understood that the term "comprising" when used in this specification specifies the presence of stated features, integers, steps, operations, elements and / or parts, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts, and / or their groupings.

[0104] As used herein, the term “if” may be interpreted to mean “when the precondition is true” or “when the precondition is true” or “in response to determining” or “upon determining” or “in response to detecting” that the precondition is true, depending on the context. Similarly, the phrase “if it is determined that [the precondition is true]” or “if [the precondition is true]” or “when [the precondition is true]” is to be interpreted to mean “upon determining that the precondition is true” or “in response to determining” or “upon determining” that the precondition is true or “when detecting that the precondition is true” or “in response to detecting” that the precondition is true, depending on the context.

Claims

1. A method comprising: At a device including a non-transitory memory and one or more processors coupled to the non-transitory memory: obtaining a first image; determining one or more characteristic attributes of each pixel in a plurality of pixels within the first image; determining a dominant hue of the first image based on the one or more characteristic attributes of each pixel in the plurality of pixels within the first image; determining a plurality of hues that meet a predetermined perceptual threshold relative to the dominant hue of the first image based on the one or more characteristic attributes of each of the plurality of pixels within the first image, wherein the plurality of hues are different from the dominant hue, and wherein determining the plurality of hues comprises comparing the dominant hue of the first image to each of the one or more characteristic attributes of each of the plurality of pixels within the first image; as well as A second image is generated based at least in part on the primary hue and the plurality of hues.

2. The method of claim 1 , wherein the second image comprises one or more portions associated with different hues within the first image, including a first portion of the second image associated with the primary hue, and a second portion of the second image associated with a corresponding hue from the plurality of hues. The method according to claim 2 , wherein a predefined conversion operation is performed between the one or more parts. The method of claim 2 , wherein the one or more portions are arranged according to one or more arrangement criteria.

5. The method of claim 1 , wherein a first size of a first portion of the second image associated with the primary hue is based at least in part on a first number of pixels within the first image associated with the primary hue, and wherein a second size of a second portion of the second image associated with a respective hue from the plurality of hues is based at least in part on a second number of pixels within the first image associated with the respective hue. The method of claim 5 , wherein the first size of the first portion of the second image and the second size of the second portion of the second image are constrained by a predefined dimensional standard. The method of claim 1 , wherein the predetermined perceptual threshold corresponds to a hue angle.

8. The method of claim 1 , wherein determining the dominant hue within the first image comprises at least one of: discarding one or more pixels within the first image that are associated with luminance values ​​outside a range of luminance values; discarding one or more pixels within the first image that are associated with saturation values ​​outside a range of saturation values; or One or more pixels within the first image associated with a foreground of the first image are discarded based on depth information associated with the first image.

9. The method according to claim 1, further comprising: After generating the second image, detecting an input corresponding to modifying the first image to generate a modified first image; as well as In response to detecting the input, the second image is updated based on one or more visual attributes of each of a plurality of pixels in the modified first image.

10. The method of claim 1, wherein the one or more characteristic attributes correspond to a hue value, a saturation value, and a brightness value.

11. A device comprising: one or more processors; non-transitory memory; and one or more programs stored in the non-transitory memory, which, when executed by the one or more processors, cause the apparatus to: obtaining a first image; determining one or more characteristic attributes of each pixel in a plurality of pixels within the first image; determining a dominant hue of the first image based on the one or more characteristic attributes of each pixel in the plurality of pixels within the first image; determining a plurality of hues that meet a predetermined perceptual threshold relative to the dominant hue of the first image based on the one or more characteristic attributes of each of the plurality of pixels within the first image, wherein the plurality of hues are different from the dominant hue, and wherein determining the plurality of hues comprises comparing the dominant hue of the first image to each of the one or more characteristic attributes of each of the plurality of pixels within the first image; as well as A second image is generated based at least in part on the primary hue and the plurality of hues.

12. The apparatus of claim 11 , wherein the second image comprises one or more portions associated with different hues within the first image, including a first portion of the second image associated with the primary hue, and a second portion of the second image associated with a corresponding hue from the plurality of hues.

13. The apparatus of claim 11 , wherein a first size of a first portion of the second image associated with the primary hue is based at least in part on a first number of pixels within the first image associated with the primary hue, and wherein a second size of a second portion of the second image associated with a corresponding hue from the plurality of hues is based at least in part on a second number of pixels within the first image associated with the corresponding hue.

14. The apparatus of claim 11 , wherein determining the dominant hue within the first image comprises at least one of: discarding one or more pixels within the first image that are associated with luminance values ​​outside a range of luminance values; discarding one or more pixels within the first image that are associated with saturation values ​​outside a range of saturation values; or One or more pixels within the first image associated with a foreground of the first image are discarded based on depth information associated with the first image.

15. The apparatus of claim 11, wherein the one or more characteristic attributes correspond to a hue value, a saturation value, and a brightness value.

16. A non-transitory memory storing one or more programs that, when executed by one or more processors of a device, cause the device to: obtaining a first image; determining one or more characteristic attributes of each pixel in a plurality of pixels within the first image; determining a dominant hue of the first image based on the one or more characteristic attributes of each pixel in the plurality of pixels within the first image; determining a plurality of hues that meet a predetermined perceptual threshold relative to the dominant hue of the first image based on the one or more characteristic attributes of each of the plurality of pixels within the first image, wherein the plurality of hues are different from the dominant hue, and wherein determining the plurality of hues comprises comparing the dominant hue of the first image to each of the one or more characteristic attributes of each of the plurality of pixels within the first image; as well as A second image is generated based at least in part on the primary hue and the plurality of hues.

17. A non-volatile memory according to claim 16, wherein the second image includes one or more portions associated with different tones within the first image, including a first portion of the second image associated with the primary tones, and a second portion of the second image associated with corresponding tones among the multiple tones.

18. A non-volatile memory according to claim 16, wherein a first size of a first portion of the second image associated with the primary hue is based at least in part on a first number of pixels associated with the primary hue within the first image, and wherein a second size of a second portion of the second image associated with a corresponding hue from the plurality of hues is based at least in part on a second number of pixels associated with the corresponding hue within the first image.

19. The non-transitory memory of claim 16, wherein determining the dominant hue within the first image comprises at least one of: discarding one or more pixels within the first image that are associated with luminance values ​​outside a range of luminance values; discarding one or more pixels within the first image that are associated with saturation values ​​outside a range of saturation values; or One or more pixels within the first image associated with a foreground of the first image are discarded based on depth information associated with the first image.

20. The non-transitory memory of claim 16, wherein the one or more characteristic attributes correspond to a hue value, a saturation value, and a brightness value.

Images