Image filtering interface

Abstract

In one embodiment, a method includes obtaining an initial image and applying each image filter of a set of one or more N image filters to the initial image to create N filtered images. The method further includes generating a similarity metric for each pair of images in the image set including the initial image and the N filtered images, and determining a location of each image in the image set within the image filter space based on the similarity metrics. The method further includes generating a selectable image filter interface based on the image filter space; and providing a selectable image filter interface for display, where each selectable region of the selectable image filter interface is associated with a particular filtering of the initial image.

Description

Technical Field

[0001] The present application generally relates to an electronic interface for filtering electronic images. Background Art

[0002] An electronic image can be modified by filtering the image. Filtering changes the value of one or more pixels in the image, most commonly changing the visual appearance of the image. For example, an image filter can be used to brighten part or all of an image, or to adjust the visual appearance of one or more colors in part or all of the image. Image filtering is typically performed by software running on a computing device that has access to the image, such as a computing device that stores the image in memory. Summary of the Invention

[0003] One embodiment provides a method that can include obtaining an initial image. The method can include applying each image filter in a set of one or more N image filters to the initial image to create N filtered images. The method can include generating a similarity metric for each pair of images in an image set that includes the initial image and the N filtered images. The method can include determining the position of each image in the image set within an image filter space based on the similarity metric. The method can include generating an optional image filter interface based on the image filter space. The method can include providing the optional image filter interface for display, wherein each optional region of the optional image filter interface is associated with a particular filtering of the initial image.

[0004] One embodiment provides a system that can include one or more computer-readable storage media containing instructions. The system can include one or more processors coupled to the computer-readable storage media, and the one or more processors can be operated to run the instructions. The instructions can be run to obtain an initial image. The instructions can be run to apply each image filter in a set of one or more N image filters to the initial image to create N filtered images. The instructions can be run to generate a similarity metric for each pair of images in an image set that includes the initial image and the N filtered images. The instructions can be run to determine the position of each image in the image set within an image filter space based on the similarity metric. The instructions can be run to generate an optional image filter interface based on the image filter space. The instructions can be run to provide the optional image filter interface for display, wherein each optional region of the optional image filter interface is associated with a particular filtering of the initial image. Brief Description of the Drawings

[0005] Figure 1 Illustrates an exemplary method for filtering an image.

[0006] Figure 2Shows an example method for creating an improved interface for filtering images.

[0007] Figure 3 Shows an example spring force model.

[0008] Figure 4 Shows the average RGB value of an example filtered image.

[0009] Figure 5 Shows aspects of an example process for creating an improved interface.

[0010] Figure 6 Shows an example UI with an example interface.

[0011] Figure 7 Shows an example of pre - determining a filter layout in an image filter space for an image set.

[0012] Figure 8 Shows an example method for determining a filtered image based on a specific selection within an optional image filter interface.

[0013] Figure 9 Shows the Figure 2 portion of an example process corresponding to the example method.

[0014] Figure 10 Shows an example computing system. Detailed Description

[0015] Figure 1 Shows a typical method 100 for filtering an image. For example, an image 105 can be presented to a user, and a predefined set of filters can be applied to the image, resulting in a predefined set of filtered images 110 being displayed to the user. The filters can include image filters. The filters can include image filters applied to an image. Figure 1 The method in involves scrolling through tiles of filtered thumbnail images 110 to select an image filter. This process is cumbersome and it is difficult to compare different image filters, especially when the filters are not visible on the same screen, and decision fatigue can prevent the user from exploring the entire space of possible image styles. Additionally, these image filters may not be sorted in an intuitive way, and the transition from one filter to the next is not seamless. Also, in the typical method, it is difficult to tell how the filters are related to each other. Further, the filter space in the interface presented to the user is limited to the predefined filters available for selection, and the user has to leave the interface to access a larger or different filter space, provided such access is even available.

[0016] Figure 2An example method for creating an improved interface for filtering images (which, in some embodiments, may include one or more image frames forming a video or animation) is shown. As more fully explained herein, the improved interface provides an intuitive and seamless way for a user to explore the image filter space. In addition, the filter space allows the user to not only immediately select from predefined filters, but also mix the filters together to create customized filtering. Further, the interface creates groups of similar filters that are close to each other, thus allowing the user to quickly and efficiently navigate to similar or different filtered visual effects as needed.

[0017] Figure 2 Step 205 of the example of includes obtaining an initial image. For example, the operation of obtaining the initial image may include accessing the initial image. The initial image may be, for example, an image created by a computing device. The initial image may be, for example, an image captured by a computing device that is performing Figure 2 some or all of the steps of the example method. For example, a smart phone may include one or more cameras for capturing images, and the initial image may be an image captured by the smart phone. In an embodiment, step 205 includes accessing an image from the memory of a client computing device (such as a smart phone, laptop computer, TV, etc.). In an embodiment, step 205 may include receiving an image from another computing device, such as, for example, receiving an image from a server computing device or another client device that stores the image. The present disclosure contemplates that, in an embodiment, a single computing device may perform Figure 2 all of the steps of the example method. In an embodiment, multiple computers may perform the steps; for example, a first computing device may access the image, while one or more other computing devices may perform other steps, such as steps including filtering the accessed initial image.

[0018] Figure 2 Step 210 of the example method of includes applying each of a set of one or more N image filters to the initial image to create N filtered images. In an embodiment, the set of image filters may be a predefined set of image filters that exists on the computing device. In an embodiment, the set of image filters may be different for different classes of images. For example, portrait images may be associated with a first set of filters, images of natural landscapes may be associated with a second set of filters, images of food may be associated with a third set of images, etc. In an embodiment, some of the filters in the set of image filters for one class of images may overlap with the set of image filters for another class of images.

[0019] The present disclosure contemplates that, in step 210, any suitable method can be used to apply N image filters and generate N filtered images from an initial image. For example, a neural network can be used to generate the filtered images. As an example, a look-up table can be used to generate N filtered images, such as by mapping input pixel values to output pixel values according to a predefined table. In an embodiment, different look-up tables can be used for each channel of the initial image (e.g., each channel of an RGB image, although any suitable color space format is also contemplated herein). In this example, each filtered image is created by applying the look-up table corresponding to a particular filter. In Figure 2 the example, N filtered images are generated by applying N image filters to the initial image. Thus, there are N + 1 images: N filtered images plus the initial image.

[0020] In an embodiment, one or more preprocessing steps can be performed on the set of N + 1 images. Although preprocessing is optional in an embodiment, preprocessing can be performed to improve runtime or accuracy. As an example, each image can first be converted from the RGB color space to a different color space (e.g., sRGB, HSV, LAB, etc.), such as a color space that is closer to the way humans perceive color visually. For example, the LAB color space separates a high-fidelity luminance component from a more compressible chrominance component. After converting the initial RGB image to another color space, an embodiment can then downsample the initial image and compute a quantized image from the downsampled image. For example, in response to converting the initial RGB image to another color space, the initial image is downsampled and a quantized image is computed from the downsampled image. The downsampled image represents the image with fewer and distinct colors. As an example, an embodiment can quantize the image to have only 128 colors, where each color is an RGB / YCbCr / LAB, etc. triple value. Quantization can be based on, for example, the redundancy of images of natural scenes. Thus, subsequent operations are performed in a smaller dimensional space, providing faster algorithm performance and / or faster processing time. As explained above, some or all of the preprocessing steps can be optional, e.g., depending on the computational requirements.

[0021] Figure 2 Step 215 of the example method includes generating a similarity metric for each pair of images, where each image in the pair of images is taken from a set of images including the initial image and the N filtered images. As described above, the set of images includes N + 1 images. An embodiment can use the preprocessed images (such as the downsampled image or the quantized image described above) to compute the similarity metric between different filtered images.

[0022] A similarity metric identifies the similarity between filtered image pairs. For example, embodiments may use a distance metric as the similarity metric to determine the similarity between filtered image pairs. The present disclosure contemplates that any suitable distance metric may be used, such as, for example, Euclidean, PSNR, or neural network-based methods. In an embodiment, using the distance metric results in a total of n C2 (choosing two from n) unique scalar distances, where n here is the number of items in a set (e.g., N + 1 images (N filtered images plus the initial image)). In an embodiment, distance calculations may be performed on a 2D image array, for example, using each pixel value and its location as input to the distance calculation. One embodiment may reduce the 2D image representation to a 1D array by creating a histogram of pixel values for each channel.

[0023] For example, an embodiment may calculate the similarity metric by using the Wasserstein distance d:

[0024]

[0025] where u represents the histogram of a given channel in a predefined filtered image, and v represents the histogram of the same channel in another filtered image in the image pair. In an embodiment, the distance metric (in this example, the Wasserstein distance) may be calculated for each channel and summed over n image channels (e.g., three image channels) to obtain a single scalar distance value for each image pair:

[0026]

[0027] One embodiment may create a distance matrix that contains the distance scalar values, for example:

[0028]

[0029] Figure 2Step 220 of the example method includes determining the position of each image in the image set within the image filter space based on a similarity metric. For example, the above example outputs an array of pairwise distances between each combination of images in a set of N+1 images. Then, one embodiment may assign coordinates on a 2D layout to each image in the image set based on the pairwise distances. One embodiment may group perceptually similar filters and determine the coordinate positions of the filters. The present disclosure contemplates that placing the filtered images within the image filter space may be achieved by any suitable method such as machine learning, clustering, heuristics, simulation, etc. Further, while the following examples discuss placing the filters in a 2D space (e.g., because the resulting filter interface may be presented on a 2D display), the present disclosure contemplates that such image placement may be performed in other dimensions, such as in a 3D space when presenting 3D content to a viewer.

[0030] As an example of generating image coordinates in a 2D image filter space, an embodiment may construct a 2D layout that is restricted to a predetermined width and height. As an example, an embodiment may use a spring force model, where there are N+1 nodes, each node representing an image in the set, and each node having N links representing the distances between image pairs. In this example, the initial distance values between the images are obtained from the pairwise distances discussed above. The spring force model is an example of a physical simulation that seeks to minimize the energy between the nodes to produce a layout constrained by characteristics including but not limited to width, height, link distance, gravity, charge, etc. In an embodiment, a similarity metric such as the distance metric discussed above may be used as the initial condition for the link distance in the spring force model. The simulation is run, and the equilibrium of the model corresponds to the final coordinates of the image nodes. In an embodiment, the spring force model simulation outputs coordinates bounded by a predefined region such as [-1,1]. An embodiment may perform a coordinate transformation to scale these coordinates to the predefined layout width and height. Figure 3 An example spring force model 310 is shown, which, after coordinate transformation and simulation, associates each image in the image set with a specific coordinate in a 2D space. Figure 3 The layout 320 in shows an example image set overlaid on the corresponding coordinate positions after coordinate transformation.

[0031] Figure 2 Step 225 of the example method includes generating an optional image filter interface based on the image filter space. Figure 5 and Figure 6 An example image filter interface is shown, as described more fully below.

[0032] In an embodiment, step 225 may include determining, for each point in the image filter space layout (e.g., Figure 3the distances and weights to each point in layout 320 such that each point is associated with a particular filter, since as shown in e.g. Figure 3 image 320 of Figure 3 , using a set of N + 1 images results in a sparse image filter space. For example, in the above example, step 220 results in N + 1 coordinates being scaled to the HxW layout. For any point (x, y) in this 2D space where 0 ≤ x ≤ W and 0 ≤ y ≤ H, one embodiment calculates the distance between this point and each of the N + 1 image coordinates. The embodiment may use a distance metric such as the Euclidean distance in order to calculate the N + 1 distances between any coordinate (x, y) and the N + 1 coordinates of the N filtered images and 1 initial image.

[0033] For example, the embodiment may use an inverse - distance - based weighting function based on obtaining an array of N + 1 distances for each point, such that the weight of an image coordinate that is farther from the target (x, y) coordinate is smaller than the weight given to an image coordinate that is closer to the target (x, y) coordinate.

[0034] After obtaining an array of N + 1 distances for each point, the embodiment may use an inverse - distance - based weighting function to make the weight of an image coordinate that is farther from the target (x, y) coordinate smaller than the weight given to an image coordinate that is closer to the target (x, y) coordinate. In a subsequent step, this inverse - distance weighting function defines how much each existing image coordinate contributes to each target location in the interface. As an example of determining the distances and weights for each point in the image filter space layout, in a 2D layout, the Euclidean distance between point p and point q is:

[0035]

[0036] The weights in this example can be calculated by using the inverse - square distance:

[0037]

[0038] This process is repeated for each (x, y) coordinate in the image filter layout that is not an existing image coordinate (e.g., the coordinates corresponding to the N + 1 images), and the image filter layout has a total of H × W coordinates. The embodiment stores the weight file in the memory of the device for use in, for example, real - time image rendering when the user selects a point in the optional image filter interface, as described more fully below.

[0039] Step 225 may include generating an optional image filter interface using the output coordinates and weights. As an example, an embodiment may first define a single scalar value (e.g., an (R, G, B) value) to represent color information in the N + 1 images. For each of the N + 1 images, the embodiment may compute a scalar value to represent the image information, meaning that each coordinate will be represented by a single color on the display. A sparse 2D grid is obtained, where only N + 1 of the HxW values are defined. To determine these missing values, the embodiment uses weighted interpolation of the N + 1 RGB values to fill the remaining display color values of the interface.

[0040] As an example of using a single scalar value to represent color information. For example, the average value may include the mean. For example, the average color may include the mean color. The embodiment may use the mean of the image. For example, the embodiment may compute the average (R, G, B) value for each of the N + 1 filtered images, e.g., by dividing the RGB input image into its three channels and, for each channel, computing the average pixel intensity value of the total 3 values for each image. The embodiment may store each of these average RGB values at the corresponding filter coordinates in the image filter space. For example, Figure 4 The average RGB values 410 of the example filtered images 420 are shown, where each of the example filtered images 420 is the result of applying an example predefined filter to a specific initial image. For example, each average RGB value 410 of the example filtered images 420 may be different.

[0041] After obtaining the scalar values, N + 1 values in the 2D grid (in this example, N + 1 (R, G, B) values) are obtained, each scalar value corresponding to one of the N + 1 filtered images. However, in the HxW grid, we have H*W - (N + 1) missing values. To fill these missing values, the embodiment may use a 2D grid interpolation method. Any suitable method (such as nearest neighbor, linear, or cubic interpolation) may be used. As an example, the embodiment may use thin plate spline interpolation. To perform non-linear interpolation, such an embodiment must first initialize the boundary conditions on the 2D grid. For each of the 4 boundary (corner) coordinates, the embodiment determines k nearest neighbor filter coordinates and, for example, interpolates the output scalar (in this example, (R, G, B)) value from the corresponding weights w using the inverse square distance as above. i The output scalar (in this example, (R, G, B)) value is interpolated as described above. The following equation may be used to determine the output channel value c out .

[0042]

[0043] Then, the embodiment may apply thin plate spline interpolation to fill the remaining display color values of the 2D interface. Then the filter interface is obtained after obtaining all the scalar values.Figure 5 An example process is shown that starts with the image coordinates of N+1 images, calculates the average color for each of these coordinates, and then uses 2D grid interpolation to fill in the scalar values (e.g., average (R,G,B) color) for each point in the 2D grid that does not correspond to an existing image coordinate. Eventually, a filter interface (or a touchpad in this example) is obtained, where each point in the interface represents the average (R,G,B) color value that would result from selecting that point in the interface given the initial image.

[0044] Figure 2 Step 230 of the example includes providing an optional image filter interface for display, where each optional region of the optional image filter interface is associated with a specific filtering of the initial image. Figure 6 An example of an interface that can be provided by step 230 is shown. For example, Figure 6 Interface 610 is shown. The user interface can include interface 610 and image 620, which shows how the image would appear if the filtering corresponding to a specific point (e.g., point 630) were applied to the image. In an embodiment, the UI can include a preset list of filtered images (such as that shown in list 640). In an embodiment, toggle keys 645 and 650 can select the optional filter interface and the preset list of filtered images for displaying image 620.

[0045] For example, as Figure 6 shown, an embodiment can output a filtered image given any arbitrary (x,y) coordinates on a given filter interface. For example, an image filtered based on the given (x,y) coordinates can be presented. For example, a predefined filter of the k nearest neighbors to the given (x,y) coordinates and the corresponding weights of those filters determined for the (x,y) coordinates can be used to generate the filter. Figure 6 An example of determining the image filtering associated with point 630 using the weights of those filters determined for point 630 by a filter of k = 3 nearest neighbors is shown. The filtered image can be created by weighted interpolation of all RGB values in the image using the filter of k nearest neighbors to produce the finally output filtered image. For example, an embodiment can use linear interpolation to interpolate the RGB pixel values. The present disclosure contemplates that the values to be interpolated are not limited to the RGB color space and any suitable interpolation method can be used.

[0046] In an embodiment, whenever the user selects an initial image to edit, the filter arrangement process can be performed in real time (e.g., as Figure 2(as explained in example steps 215-225 of an example method). In an embodiment, the filter arrangement can be created in advance all at once, for example, by analyzing an image set and determining an appropriate static filter layout based on the set, thus saving computational resources. For example, a later embodiment can combine the image arrays between all the images in an image category and calculate the total distance summed over these images for each pair of predefined filters. Thus, each image category (e.g., the type of image) can have a predefined static filter arrangement in the image filter space.

[0047] Figure 7 Shows an example of pre-determining a filter layout in an image filter space for an image set. Figure 7 Shows four different images, but each image shares a predefined filter arrangement in the image filter space. Thus, even when different initial images are input, the profile of the optional image filter interface tends to remain the same or similar. For example, region 710 exists in each of the interfaces of the four different images. Figure 7 Also shows that even for similar filter arrangements, different interfaces for different images will likely still look different, for example, due to differences in image colors, which in Figure 2 the example results in different average colors being displayed in the interface.

[0048] Figure 8 Shows an example method for determining a filtered image based on a specific selection within an optional image filter interface. As Figure 7 shown, for example, the user has selected a point x,y (shown by the gray dot). This point does not correspond to the position of a predefined filter, so as described above, the k nearest neighbor filters are selected and weighted (in this example, k = 3) in order to obtain a filtered image corresponding to the point x,y. Figure 8 Shows the filtering associated with each of the k = 3 filters, which will be interpolated to obtain the filtering associated with the point x,y.

[0049] An embodiment obtains the k nearest neighbor filters and the corresponding weights, and then performs weighted interpolation of the RGB values of these k filters. For example, an embodiment can split the k output images into their component channels and interpolate the final pixel values according to the following formula to obtain I out (the output filtered image):

[0050]

[0051] where k is the number of neighboring filters, I i is the i-th filtered image, and w iis the weight associated with the i-th filter. As shown in this example, each image channel of the k filtered images can be weighted and summed according to the example equation above to obtain each channel of the resulting filtered image corresponding to the point x, y.

[0052] In an embodiment, when the user selects a point x, y on the interface, interpolation of nearby filters is performed in real time. The embodiment can store low-resolution images for each of some or all of the points x, y on the interface on the user's device or in the cloud (or both), such that when the user selects a point x, y, no interpolation is required. In a later embodiment, the low-resolution images will be enlarged for display, sending, and / or saving the image to the photo library.

[0053] As explained with reference to Figure 2 the example method of, the optional image filtering interface displays the average color (e.g., average (R, G, B) triple value) for a particular point on the interface. The actual filtering of the image occurs, for example, on a per-pixel basis, so similar average colors displayed on the interface do not necessarily represent similar filtering, but only the average color after filtering. Thus, as shown in the example 2D interface illustrated in the figure, different regions of the interface can have similar colors, but as described above, their relative distances in the interface tend to generally represent the similarity between the overall filtering applied to the image. In an example, the distance between regions in the interface can also represent constraints imposed during filter placement and interface generation, e.g., restricting a spring model simulation to a 2D space.

[0054] Figure 9 shows a portion of an example process corresponding to the Figure 2 example method of. As Figure 8 shown, the user can select an image for filtering (e.g., an initial image). The initial image can be selected from, for example, a gallery or a camera application. After selecting the image, a predefined set of filters is applied to the image. In response to selecting the initial image, the predefined set of filters can be applied to the initial image. The filtered images are arranged in an image filter space (such as the 2D image filter space shown in Figure 8 ). An optional image filter interface is generated and then presented to the user on the UI, where each point x, y in the interface corresponds to a particular filtering of the selected image.

[0055] In appropriate cases, the embodiment can repeat Figure 2 one or more steps of the method of. Although this disclosure describes and illustrates specific steps of the Figure 2 method occurring in a particular order, this disclosure contemplates any suitable steps of the Figure 2 method occurring in any suitable order. Additionally, although this disclosure describes and illustrates performing Figure 2a particular component, apparatus, or system of a particular step of a method (such as Figure 10 a computer system), but the present disclosure contemplates any suitable combination of any suitable components, apparatuses, or systems that perform Figure 2 any suitable step of the method. Additionally, the present disclosure contemplates that some or all of the computational operations described herein (including Figure 10 the steps of the example methods shown in Figure 2 ) may be performed by circuitry of a computing device (such as

[0056] a computing device), by a processor incorporated into a non-transitory computer-readable storage medium, or by any suitable combination thereof.

[0057] Figure 10 Example computer system 1000 is shown. In certain embodiments, one or more computer systems 1000 perform one or more steps of one or more methods described or shown herein. In certain embodiments, one or more computer systems 1000 provide the functionality described or shown herein. In certain embodiments, software running on one or more computer systems 1000 performs one or more steps of one or more methods described or shown herein, or provides the functionality described or shown herein. Certain embodiments include one or more portions of one or more computer systems 1000. In this document, references to computer systems may, where appropriate, include computing devices, and vice versa. Additionally, references to computer systems may, where appropriate, include one or more computer systems.

[0058] The present disclosure contemplates any suitable number of computer systems 1000. The present disclosure contemplates computer systems 1000 in any suitable physical form. By way of example and not limitation, computer system 1000 can be an embedded computer system, a system on a chip (SOC), a single board computer system (SBC) (such as, for example, a computer-on-module (COM) or a system-on-module (SOM)), a desktop computer system, a laptop or notebook computer system, an interactive kiosk, a mainframe, a computer system network, a mobile phone, a personal digital assistant (PDA), a server, a tablet computer system, or a combination of two or more of these. In appropriate instances, computer system 1000 can include one or more computer systems 1000; be single or distributed; span multiple locations; span multiple machines; span multiple data centers; or reside in a cloud, which can include one or more cloud components in one or more networks. In appropriate instances, one or more computer systems 1000 can perform one or more steps of one or more of the methods described or shown herein without substantial spatial or temporal limitation. By way of example and not limitation, one or more computer systems 1000 can perform one or more steps of one or more of the methods described or shown herein in real time or in batch mode. In appropriate instances, one or more computer systems 1000 can perform one or more steps of one or more of the methods described or shown herein at different times or at different locations.

[0059] In a particular embodiment, computer system 1000 includes a processor 1002, a memory 1004, a storage 1006, an input / output (I / O) interface 1008, a communication interface 1010, and a bus 1012. Although the present disclosure describes and shows a particular computer system having a particular number of particular components in a particular arrangement, the present disclosure contemplates any suitable computer system having any suitable number of any suitable components in any suitable arrangement.

[0060] In certain embodiments, the processor 1002 includes hardware for running instructions, such as instructions for writing a computer program. By way of example and not limitation, to run instructions, the processor 1002 may retrieve (or fetch) the instructions from an internal register, an internal cache, the memory 1004, or the storage 1006; decode and run them; and then write one or more results to an internal register, an internal cache, the memory 1004, or the storage 1006. In certain embodiments, the processor 1002 may include one or more internal caches for data, instructions, or addresses. In appropriate instances, the present disclosure contemplates a processor 1002 that includes any suitable number of any suitable internal caches. By way of example and not limitation, the processor 1002 may include one or more instruction caches, one or more data caches, and one or more translation lookaside buffers (TLBs). The instructions in the instruction cache may be copies of the instructions in the memory 1004 or the storage 1006, and the instruction cache may accelerate the retrieval of these instructions by the processor 1002. The data in the data cache may be copies of the data in the memory 1004 or the storage 1006 for operations by the instructions running at the processor 1002; results of previous instructions running at the processor 1002 for access by subsequent instructions running at the processor 1002 or for writing to the memory 1004 or the storage 1006; or other suitable data. The data cache may accelerate the read or write operations of the processor 1002. The TLB may accelerate the virtual address translation of the processor 1002. In certain embodiments, the processor 1002 may include one or more internal registers for data, instructions, or addresses. In appropriate instances, the present disclosure contemplates a processor 1002 that includes any suitable number of any suitable internal registers. In appropriate instances, the processor 1002 may include one or more arithmetic logic units (ALUs); be a multi-core processor; or include one or more processors 1002. Although the present disclosure describes and illustrates particular processors, the present disclosure contemplates any suitable processor.

[0061] In a particular embodiment, the memory 1004 includes a main memory for storing instructions for the processor 1002 to execute or data for the processor 1002 to operate on. By way of example and not limitation, the computer system 1000 may load instructions from the storage 1006 or another source (such as, for example, another computer system 1000) into the memory 1004. Then, the processor 1002 may load the instructions from the memory 1004 into internal registers or an internal cache. To execute the instructions, the processor 1002 may retrieve the instructions from the internal registers or the internal cache and decode them. During or after executing the instructions, the processor 1002 may write one or more results (which may be intermediate results or final results) to the internal registers or the internal cache. Then, the processor 1002 may write one or more of these results to the memory 1004. In a particular embodiment, the processor 1002 only executes instructions in one or more of the internal registers or the internal cache or the memory 1004 (and not in the storage 1006 or elsewhere), and only operates on data in one or more of the internal registers or the internal cache or the memory 1004 (and not in the storage 1006 or elsewhere). One or more memory buses (which may each include an address bus and a data bus) may couple the processor 1002 to the memory 1004. As described below, the bus 1012 may include one or more memory buses. In a particular embodiment, one or more memory management units (MMUs) reside between the processor 1002 and the memory 1004 and facilitate access to the memory 1004 requested by the processor 1002. In a particular embodiment, the memory 1004 includes random access memory (RAM). Where appropriate, the RAM may be volatile memory. Where appropriate, the RAM may be dynamic RAM (DRAM) or static RAM (SRAM). Additionally, where appropriate, the RAM may be single-port RAM or multi-port RAM. The present disclosure contemplates any suitable RAM. Where appropriate, the memory 1004 may include one or more memories 1004. Although the present disclosure describes and illustrates particular memories, the present disclosure contemplates any suitable memory.

[0062] In certain embodiments, the memory 1006 includes a mass storage for data or instructions. By way of example and not limitation, the memory 1006 can include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive or a combination of two or more of these. In suitable cases, the memory 1006 can include removable media or non-removable (or fixed) media. In suitable cases, the memory 1006 can be internal or external to the computer system 1000. In certain embodiments, the memory 1006 is non-volatile solid-state memory. In certain embodiments, the memory 1006 includes read-only memory (ROM). In suitable cases, the ROM can be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory or a combination of two or more of these. The present disclosure contemplates mass storage 1006 in any suitable physical form. In suitable cases, the memory 1006 can include one or more memory control units that facilitate communication between the processor 1002 and the memory 1006. In suitable cases, the memory 1006 can include one or more memories 1006. Although the present disclosure describes and illustrates particular memories, the present disclosure contemplates any suitable memory.

[0063] In certain embodiments, the I / O interface 1008 includes hardware, software, or both, providing one or more interfaces for communication between the computer system 1000 and one or more I / O devices. In suitable cases, the computer system 1000 can include one or more of these I / O devices. One or more of these I / O devices can implement communication between a person and the computer system 1000. By way of example and not limitation, such I / O can include a keyboard, a keypad, a microphone, a monitor, a mouse, a printer, a scanner, a speaker, a still camera, a stylus, a tablet computer, a touch screen, a trackball, a video camera, another suitable I / O, or a combination of two or more of these. The I / O devices can include one or more sensors. The present disclosure contemplates any suitable I / O devices and any suitable I / O interface 1008 for them. In suitable cases, the I / O interface 1008 can include one or more devices or software drivers that enable the processor 1002 to drive one or more of these I / O devices. In suitable cases, the I / O interface 1008 can include one or more I / O interfaces 1008. Although the present disclosure describes and illustrates particular I / O interfaces, the present disclosure contemplates any suitable I / O interface.

[0064] In certain embodiments, communication interface 1010 includes hardware, software, or both that provide one or more interfaces for communication (such as, for example, packet-based communication) between computer system 1000 and one or more other computer systems 1000 or one or more networks. By way of example and not limitation, communication interface 1010 may include a network interface controller (NIC) or network adapter for communicating with an Ethernet or other wired-based network, or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network (such as a WI-FI network). The present disclosure contemplates any suitable network and any suitable communication interface 1010 therefor. By way of example and not limitation, computer system 1000 may communicate with an ad hoc network, a personal area network (PAN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), or one or more portions of the Internet or a combination of two or more of these. One or more portions of one or more of these networks may be wired or wireless. As an example, computer system 1000 may communicate with a wireless PAN (WPAN) (such as, for example, a Bluetooth WPAN), a WI-FI network, a WI-MAX network, a cellular telephone network (such as, for example, a Global System for Mobile Communications (GSM) network), or other suitable wireless networks or a combination of two or more of these networks. In appropriate instances, computer system 1000 may include any suitable communication interface 1010 for any one of these networks. In appropriate instances, communication interface 1010 may include one or more communication interfaces 1010. Although the present disclosure describes and shows particular communication interfaces, the present disclosure contemplates any suitable communication interface.

[0065] In certain embodiments, bus 1012 includes hardware, software, or both that couple the components of computer system 1000 to each other. By way of example and not limitation, bus 1012 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an INFINIBAND interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCIe) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or another suitable bus or a combination of two or more of these buses. In appropriate instances, bus 1012 may include one or more buses 1012. Although the present disclosure describes and shows particular buses, the present disclosure contemplates any suitable bus or interconnect.

[0066] In this document, where appropriate, a computer-readable non-transitory storage medium may include one or more semiconductor-based integrated circuits (ICs) or other integrated circuits (e.g., field-programmable gate arrays (FPGAs) or application-specific ICs (ASICs)), hard disk drives (HDDs), hybrid hard disk drives (HHDs), optical discs, optical disc drives (ODDs), magneto-optical discs, magneto-optical drives, floppy disks, floppy disk drives (FDDs), magnetic tapes, solid-state drives (SSDs), RAM drives, secure digital cards or drives, any other suitable computer-readable non-transitory storage medium, or any suitable combination of two or more of these. Where appropriate, the computer-readable non-transitory storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile.

[0067] In this document, unless otherwise expressly stated or indicated by context, "or" is inclusive rather than exclusive. Thus, in this document, unless otherwise expressly stated or indicated by context, "A or B" means "A, B, or both". Additionally, unless otherwise expressly stated or indicated by context, "and" is both conjunctive and disjunctive. Thus, in this document, unless otherwise expressly stated or indicated by context, "A and B" means "A and B, jointly or severally".

[0068] The scope of the present disclosure includes all changes, substitutions, variations, alterations, and modifications to the example embodiments described or illustrated herein that would be understood by a person of ordinary skill in the art. The scope of the present disclosure is not limited to the example embodiments described or illustrated herein. Moreover, although the present disclosure describes and illustrates the corresponding embodiments herein as including specific components, elements, features, functions, operations, or steps, any of these embodiments may include any combination or arrangement of any components, elements, features, functions, operations, or steps described or illustrated anywhere herein that would be understood by a person of ordinary skill in the art.

[0069] According to an embodiment of the present disclosure, a method may include: accessing an initial image.

[0070] According to an embodiment of the present disclosure, a method may include: obtaining an initial image.

[0071] According to an embodiment of the present disclosure, a method may include: applying each image filter in a set of one or more N image filters to the initial image to create N filtered images.

[0072] According to an embodiment of the present disclosure, a method may include: generating a similarity metric for each image pair in an image set including the initial image and the N filtered images.

[0073] According to an embodiment of the present disclosure, a method may include: determining a position of each image in an image set within an image filter space based on a similarity metric.

[0074] According to an embodiment of the present disclosure, a method may include: generating an optional image filter interface based on the image filter space.

[0075] According to an embodiment of the present disclosure, a method may include: providing an optional image filter interface for display, wherein each optional region of the optional image filter interface is associated with a specific filtering of an initial image.

[0076] According to an embodiment of the present disclosure, N image filters in a set of one or more N image filters may be predetermined based on a category associated with the initial image.

[0077] According to an embodiment of the present disclosure, the similarity metric may include a scalar distance metric based on a difference between image channel histogram values of an image pair.

[0078] According to an embodiment of the present disclosure, a difference between image channel histogram values of an image pair may be determined for each of a plurality of image channels.

[0079] According to an embodiment of the present disclosure, the image filter space may include a two-dimensional space having a predetermined width and height.

[0080] According to an embodiment of the present disclosure, the operation of determining a position of each image in the image set within the image filter space may include: using a spring force model to determine the position.

[0081] According to an embodiment of the present disclosure, the images in the image set may include nodes in the spring force model.

[0082] According to an embodiment of the present disclosure, each similarity metric may be associated with a link distance in the spring force model.

[0083] According to an embodiment of the present disclosure, each optional region of the optional image filter interface may be represented by a specific color, and the specific color represents a specific filtering associated with the optional region.

[0084] According to an embodiment of the present disclosure, the operation of generating the optional image filter interface may include: determining an average color representing a filter for each of the one or more N image filters in the set.

[0085] According to an embodiment of the present disclosure, for each of a plurality of points in an optional image filter interface that are not associated with an image filter in a set of one or more N image filters, the method may include: determining an image filter for the point by weighted interpolation of the image filters in the set of image filters based on the distance of each image filter in the set of one or more N image filters to the point.

[0086] According to an embodiment of the present disclosure, the method may include: displaying an optional image filter interface on a display of a client computing device.

[0087] According to an embodiment of the present disclosure, the method may include: receiving user input selecting a particular region of the optional image filter interface.

[0088] According to an embodiment of the present disclosure, the method may include: determining whether the selected region is associated with a particular image filter in the set of image filters.

[0089] According to an embodiment of the present disclosure, when the selected region is associated with a particular image filter in the set of one or more N image filters, the method may include: applying the particular image filter.

[0090] According to an embodiment of the present disclosure, when the selected region is not associated with a particular image filter in the set of image filters, the method may include: applying an image filter based on the proximity of the selected region to one or more of the N image filters.

[0091] According to an embodiment of the present disclosure, the method may include: selecting k nearest neighbor image filters.

[0092] According to an embodiment of the present disclosure, the method may include: interpolating an image filter based on the distance between the selected region and each of the k image filters.

[0093] According to an embodiment of the present disclosure, the method may include: receiving user input selecting a particular region of the optional image filter interface.

[0094] According to an embodiment of the present disclosure, the method may include: displaying an initial image filtered based on the selection of the particular region.

[0095] According to an embodiment of the present disclosure, a computer-readable storage medium stores at least one instruction that, when executed, causes at least one processor to perform a corresponding method.

[0096] According to an embodiment of the present disclosure, at least one processor is configured to access an initial image.

[0097] According to an embodiment of the present disclosure, at least one processor is configured to apply each image filter in a set of one or more N image filters to an initial image to create N filtered images.

[0098] According to an embodiment of the present disclosure, at least one processor is configured to generate a similarity metric for each pair of images in an image set including the initial image and the N filtered images.

[0099] According to an embodiment of the present disclosure, at least one processor is configured to determine the position of each image in the image set within an image filter space based on the similarity metric.

[0100] According to an embodiment of the present disclosure, at least one processor is configured to generate an optional image filter interface based on the image filter space.

[0101] According to an embodiment of the present disclosure, at least one processor is configured to provide an optional image filter interface for display, wherein each optional region of the optional image filter interface is associated with a specific filtering of the initial image.

[0102] According to an embodiment of the present disclosure, at least one processor is configured to display the optional image filter interface on a display of a client computing device.

[0103] According to an embodiment of the present disclosure, at least one processor is configured to receive user input selecting a specific region of the optional image filter interface.

[0104] According to an embodiment of the present disclosure, at least one processor is configured to determine whether the selected region is associated with a specific image filter in the set of image filters.

[0105] According to an embodiment of the present disclosure, at least one processor is configured to apply a specific image filter.

[0106] According to an embodiment of the present disclosure, at least one processor is configured to apply an image filter based on the proximity of the selected region to one or more of the N image filters.

[0107] According to an embodiment of the present disclosure, at least one processor is configured to receive user input selecting a specific region of the optional image filter interface.

[0108] According to an embodiment of the present disclosure, at least one processor is configured to display the initial image filtered based on the selection of the specific region.

[0109] The first image filter may include each image filter in a set of one or more N image filters.

[0110] The second image filter may include an image filter from one or more sets of N image filters.

[0111] The third image filter may include an image filter from one or more sets of N image filters, where weighting is performed according to the distance from each image filter in the one or more sets of N image filters.

[0112] The fourth image filter may include an image filter based on the proximity of a selected region to one or more of the N image filters.

[0113] The fifth image filter may include an image filter from among the k nearest neighbor image filters.

Claims

1. A method, comprising: Obtain an initial image; Apply each image filter in a set of one or more N image filters to the initial image to create N filtered images; Generate a similarity metric for each pair of images in an image set including the initial image and the N filtered images; Determine the position of each image in the image set within an image filter space based on the similarity metric; Generate an optional image filter interface based on the image filter space; And Provide the optional image filter interface for display, where each optional region of the optional image filter interface is associated with a specific filtering of the initial image.

2. The method according to claim 1, wherein The N image filters in the set of one or more N image filters are pre-determined based on a category associated with the initial image.

3. The method according to any one of claims 1 and 2, wherein The similarity metric includes a scalar distance metric based on the difference between the image channel histogram values of the image pair.

4. The method according to claim 3, wherein Determine the difference between the image channel histogram values of the image pair for each of a plurality of image channels.

5. The method according to any one of claims 1 to 4, wherein The image filter space includes a two-dimensional space of a predetermined width and height.

6. The method according to any one of claims 1 to 5, wherein The operation of determining the position of each image in the image set within the image filter space includes: using a spring force model to determine the position, where each image in the image set includes a node in the spring force model, and each similarity metric is associated with a link distance in the spring force model.

7. The method according to any one of claims 1 to 6, wherein Each optional region of the optional image filter interface is represented by a specific color, where the specific color represents a specific filtering associated with the optional region.

8. The method according to any one of claims 1 to 7, wherein The operation of generating the optional image filter interface includes: determining an average color representing the filter for each image filter in the set of one or more N image filters.

9. The method according to any one of claims 1 to 8, wherein The operation of generating the optional image filter interface includes: For each point among a plurality of points in the optional image filter interface that is not associated with an image filter in the set of one or more N image filters, determine an image filter for the point by weighted interpolation of the image filters in the set of one or more N image filters according to the distance from each image filter in the set of one or more N image filters to the point.

10. The method according to any one of claims 1 to 9, further comprising: Display the optional image filter interface on a display of a client computing device.

11. The method according to any one of claims 1 to 10, further comprising: Receive user input selecting a specific region of the optional image filter interface; Determine whether the selected region is associated with a specific image filter in the set of one or more N image filters; When the selected region is associated with a specific image filter in the set of one or more N image filters, apply the specific image filter; And When the selected region is not associated with a specific image filter in the set of one or more N image filters, apply an image filter based on the proximity of the selected region to one or more image filters in the N image filters.

12. The method according to claim 11, wherein The operation of applying an image filter based on the proximity of a selected region to one or more of the N image filters includes: selecting k nearest neighbor image filters and interpolating the image filters based on the distances between the selected region and each of the k image filters.

13. The method according to any one of claims 1 to 10, further comprising: Receiving user input selecting a specific region of the optional image filter interface; And Displaying the initial image filtered based on the selection of the specific region.

14. A computer-readable storage medium for storing instructions, wherein, When the instructions are run by at least one processor, causing the at least one processor to perform the method according to any one of claims 1 to 13.

15. A system, comprising: One or more computer-readable storage media containing instructions; And One or more processors coupled to the computer-readable storage media, the one or more processors being operative to run the instructions to perform the following operations: Obtaining an initial image; Applying each of a set of one or more N image filters to the initial image to create N filtered images; Generating a similarity metric for each pair of images in an image set including the initial image and the N filtered images; Determining the position of each image in the image set within an image filter space based on the similarity metric; Generating an optional image filter interface based on the image filter space; And Providing the optional image filter interface for display, wherein each optional region of the optional image filter interface is associated with a specific filtering of the initial image.

Images