Evaluation of Similar Content-Based Images

Abstract

Implementations generally relate to evaluating similar content-based images. In some implementations, a method includes receiving a first image including at least one first object. The method further includes receiving a second image including at least one second object. The method further includes calculating a structural similarity index measure (SSIM) value based on the at least one first object and the at least one second object. The method further includes calculating a scale invariant feature transform (SIFT) value based on the at least one first object and the at least one second object. The method further includes calculating a histogram value based on the at least one first object and the at least one second object. The method further includes calculating a similarity score based on the SSIM value, the SIFT value, and the histogram value.

Description

Technical Field

[0001] [Cross - Reference to Related Applications] This application claims priority based on U.S. Provisional Patent Application No. 63 / 184,274, entitled "OBJECTIVE METHOD TO EVALUATE SIMILARITY FOR CONTENTS - BASED IMAGES", filed on May 5, 2021 (Client Reference No.: SYP340001US01), and this document is incorporated herein by reference as if it were entirely set forth herein for all purposes.

Background Art

[0002] To find similar images on the Internet, image search algorithms are used. The search method can involve visual similarity of the pixels of the image (e.g., similar colors). The search can involve the mean squared error (MSE) method, the peak signal - to - noise ratio (PSNR) method, and other subjective evaluations for measuring image similarity. Since the MSE method and the PSNR method estimate the absolute error based on per - pixel analysis, as a result of a search based on a query image of a red jalapeno, an image of a red tomato may be retrieved due to the amount of similar red pixels. These techniques are generally used to compare the quality of the original image with that of the restored image, or can be used for matching the original image with a distorted image (e.g., by blur, rotation, scale, illumination, etc.).

Summary of the Invention

Means for Solving the Problems

[0003] Implementations generally relate to the evaluation of similar content-based images. In some implementations, a system includes one or more processors and logic encoded on one or more non-transitory computer-readable storage media for execution by the one or more processors. The logic, when executed, is operable to cause the one or more processors to perform operations including receiving a first image including at least one first object, receiving a second image including at least one second object, calculating a structural similarity index measure (SSIM) value based on the at least one first object and the at least one second object, calculating a scale-invariant feature transform (SIFT) value based on the at least one first object and the at least one second object, calculating a histogram value based on the at least one first object and the at least one second object, and calculating a similarity score based on the SSIM value, the SIFT value, and the histogram value.

[0004] Regarding the system further, in some implementations, the SSIM value is based on one or more of the luminance, contrast, and structure of the first and second images. In some implementations, the SIFT value is based on one or more predetermined features of the first and second images. In some implementations, the histogram value is based on the frequency of predetermined histogram values of the first and second images. In some implementations, the logic is further operable to cause the one or more processors to perform an operation including calculating an adjusted SIFT value based on the SSIM value when executed. In some implementations, the logic is further operable to cause the one or more processors to perform an operation including calculating an adjusted histogram value based on the SSIM value and the SIFT value when executed. In some implementations, the logic is further operable to cause the one or more processors to perform an operation including calculating the sum of the SSIM value, the adjusted SIFT value, and the adjusted histogram value when executed.

[0005] In some implementations, a non-transitory computer-readable storage medium is provided that includes program instructions. When the instructions are executed by one or more processors, the instructions cause the one or more processors to perform operations including receiving a first image including at least one first object, receiving a second image including at least one second object, calculating a structural similarity index measure (SSIM) value based on at least one first object and at least one second object, calculating a scale-invariant feature transform (SIFT) value based on at least one first object and at least one second object, calculating a histogram value based on at least one first object and at least one second object, and calculating a similarity score based on the SSIM value, the SIFT value, and the histogram value.

[0006] Regarding the computer-readable storage medium further, in some implementations, the SSIM value is based on one or more of the luminance, contrast, and structure of the first image and the second image. In some implementations, the SIFT value is based on one or more predetermined features of the first image and the second image. In some implementations, the histogram value is based on the frequency of predetermined histogram values of the first image and the second image. In some implementations, the instructions are further operable to cause the one or more processors to perform an operation including calculating an adjusted SIFT value based on the SSIM value when executed. In some implementations, the instructions are further operable to cause the one or more processors to perform an operation including calculating an adjusted histogram value based on the SSIM value and the SIFT value when executed. In some implementations, the instructions are further operable to cause the one or more processors to perform an operation including calculating the sum of the SSIM value, the adjusted SIFT value, and the adjusted histogram value when executed.

[0007] In some implementations, the method includes receiving a first image including at least one first object, receiving a second image including at least one second object, calculating a Structural Similarity Index Measure (SSIM) value based on the at least one first object and the at least one second object, calculating a Scale-Invariant Feature Transform (SIFT) value based on the at least one first object and the at least one second object, calculating a histogram value based on the at least one first object and the at least one second object, and calculating a similarity score based on the SSIM value, the SIFT value, and the histogram value.

[0008] Regarding the method further, in some implementations, the SSIM value is based on one or more of the luminance, contrast, and structure of the first image and the second image. In some implementations, the SIFT value is based on one or more predetermined features of the first image and the second image. In some implementations, the histogram value is based on the frequency of predetermined histogram values of the first image and the second image. In some implementations, the method further includes calculating an adjusted SIFT value based on the SSIM value. In some implementations, the method further includes calculating an adjusted histogram value based on the SSIM value and the SIFT value.

[0009] By referring to the remainder of the specification and the attached drawings, the characteristics and advantages of the specific implementations disclosed herein can be further understood.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

DETAILED DESCRIPTION OF THE INVENTION

[0011] The implementations described in this specification generally relate to the evaluation of similar content-based images. As will be described in more detail in this specification, the system searches for and ranks images similar to a query image. The system arranges the subjectively determined search results (e.g., using the human visual system, etc.) in order of similarity. Instead, when comparing the found images with the query image, the results are objectively calculated based on content and / or objects. For example, the system analyzes the features and structures of images to provide quantitative search results.

[0012] As will be described in more detail herein, in various implementations, the system receives a first image including at least one first object and a second image including at least one second object. Next, the system calculates a structural similarity index measure (SSIM) value based on at least one first object and at least one second object. The system also calculates a scale invariant feature transform (SIFT) value based on at least one first object and at least one second object. The system also calculates a histogram value based on at least one first object and at least one second object. Next, the system calculates a similarity score based on the SSIM value, the SIFT value, and the histogram value. As will be described in more detail herein, the system adjusts the SIFT value and the histogram value to calculate a more accurate similarity score. Although the implementations disclosed herein are described in the context of still images, the implementations can also be applied to video image frames.

[0013] FIG. 1 is a block diagram of an example network environment 100 for evaluating similar content-based images that can be used in some implementations described herein. In some implementations, network environment 100 includes a system 102 that includes a server device 104 and a database 106. Network environment 100 also includes a client device 110 that communicates with system 102 via network 150 of network environment 100. Network 150 can be any suitable communication network or combination of networks, including network types such as a Bluetooth network, a Wi-Fi network, the Internet, and the like.

[0014] As will be described in more detail herein, system 102 receives a first image that includes at least one first object. The first image is also referred to as a query image, and this image is used in image search. System 102 performs a search to find other images having similar content. In various implementations, system 102 also ranks the images based on similarity. During the search, system 102 receives at least a second image that includes at least one second object. For ease of explanation, the second image will be described. In various implementations, system 102 processes many other images (e.g., hundreds of images, thousands of images, etc.) in the same manner as the second image.

[0015] As will be described in more detail herein, in various implementations, system 102 calculates a structural similarity index measure (SSIM) value based on the first object in the first image and the second object in the second image. System 102 also calculates a scale-invariant feature transform (SIFT) value based on the first object and at least one second object. System 102 calculates a histogram value based on the first object and the second object. Next, system 102 calculates a similarity score based on the SSIM value, the SIFT value, and the histogram value. System 102 performs these and other steps on the plurality of images found by system 102 in the search. Further implementations for the generation and implementation of the investigation will be described in more detail herein.

[0016] For ease of explanation, FIG. 1 shows one block for each of system 102, server device 104, database 106, and client device 110. Blocks 102, 104, 106, and 110 can also represent multiple systems, server devices, databases, and clients. In other implementations, environment 100 may not have all of the components shown in the figure and / or may have other types of elements instead of or in addition to the elements shown herein.

[0017] The implementations described in this specification are executed by system 102, although in other implementations, the execution of the implementations described in this specification can be facilitated by any suitable component or combination of components associated with system 102, or by any suitable one or more processors associated with system 102.

[0018] FIG. 2 is an example flowchart of evaluating similar content-based images according to some implementations. Referring to both FIGS. 1 and 2, the method starts at block 202, and a system such as system 102 receives a first image including a first object. The first image can be referred to as a query image, and system 102 performs a search for images similar to the query image. As will be described in more detail herein, system 102 also ranks the images found in the search based on their similarity to the query image.

[0019] FIG. 3 is an example query image 300 of red jalapeno peppers according to some implementations. The figure shows three jalapeno peppers that are red in this example. For ease of explanation, the first image can be described herein in the context of one object. In various implementations, as shown in query image 300, there can be one or more objects within the first image. The number of objects included in a given image can vary depending on the particular implementation. For clarity, the terms first image and query image can be used synonymously.

[0020] At block 204, system 102 receives a second image including a second object. The second image is one of many images that the system finds in an image search for images similar to the query image.

[0021] The system searches the Internet and / or other suitable databases for search result images. The terms "second image" and "search result image" can be used synonymously. The system then performs the other steps described herein to compare the second / search result image with the first / query image.

[0022] Figure 4 is an example 400 of an image of green jalapeno peppers found in an image search result according to some implementations. The figure shows three jalapeno peppers that are green in this example. For ease of explanation, the second image can be described herein in the context of one object. In various implementations, as shown in image 400, one or more objects can be present within the first image. The number of objects included in a given image can vary depending on the particular implementation. For clarity, the terms "second image" and "search result image" can be used synonymously.

[0023] The implementations described herein result in search image results having objects of the same content as the objects in the query image even if the colors of the objects are different. The reason for this is that such searches are based on the content within the comparison images rather than simply the pixels within the comparison images. Such a mode of search will be described in more detail herein.

[0024] In block 206, system 102 calculates a Structural Similarity Index Measure (SSIM) value based on the first one or more objects within the query image and the second one or more objects within the search result image. In various implementations, the system calculates the SSIM value based on a predetermined comparison measurement between two given samples such as the first and second images.

[0025] A predetermined comparative measurement for the SSIM value can include color, luminance, contrast, and structure. The structure can include shape. Thus, in various implementations, the SSIM value is based on one or more of the color, luminance, contrast, and structure of the first image and the second image. In various implementations, the system analyzes objects in the image for these measurements. The predetermined comparative measurement can vary depending on the particular implementation.

[0026] The system calculates an SSIM value based on the first image and the second image. In some implementations, the SSIM value is a value between 0 and 1. A value of +1 indicates that the first image and the second image are very similar or the same. A value of 0 indicates that the first image and the second image are very different.

[0027] Images such as image 300 (red jalapeno pepper) in FIG. 3 and image 400 (green jalapeno pepper) in FIG. 4 are visually similar because they are structurally similar. They have similar shapes (e.g., the shape of a jalapeno pepper) and similar features (e.g., the stem of a jalapeno). In this example, the system calculates an SSIM value close to 1. Thus, the system includes image 400 in the set of images similar to image 300 even if image 400 has objects of a different color than those of image 300. This is because the system analyzes the content of the images (not just the color). The search result image 400 is similar to the query image 300 based at least on the structural similarity. In other words, images 300 and 400 are content-wise similar. Thus, the implementations described herein provide a blind evaluation of images that are not only visually similar but also content-wise or feature-wise similar.

[0028] In block 208, system 102 calculates Scale-Invariant Feature Transform (SIFT) values based on a first object and a second object. In various implementations, the SIFT values are based on one or more predetermined features of the first image and the second image. In various implementations, SIFT is a feature detection algorithm in computer vision. The SIFT algorithm finds corresponding relationships within the first image and the second image. The corresponding relationships of the local features within each of these images can be called the keypoints of the image. These keypoints are scale-invariant and rotation-invariant and can be used to evaluate the similarity between the first image and the second image. In addition to being based on the keypoints, the SIFT values are also based on a comparison between the background of the search result image and the background of the query image. In various implementations, the system analyzes the keypoints and backgrounds of the objects within the image for these measurements.

[0029] The example described above relates to the green jalapeno pepper found in the search result image that is similar to the red jalapeno pepper in the query image. In this example, there is no background to consider within the image. In various implementations, especially when the content of the image is more complex with accompaniments such as the background, the system includes additional SIFT values to provide additional accuracy in determining the similarity between a given search result image and the query image.

[0030] The following query image example of FIG. 5, and the related search result images of FIGS. 6 and 7 are contextually more complex. As will be described in more detail herein in connection with the remaining steps of the process of FIG. 2, and the images of FIGS. 5, 6, and 7, the system employs additional analysis to calculate additional values. These additional analysis and values facilitate the system to more accurately find similar search result images and more accurately rank the search result images based on similarity. Below, after describing the images of FIGS. 5, 6, and 7, the additional analysis and calculations that the system performs when finding and ranking such images will be described.

[0031] Figure 5 is an example query image 500 of a superhero according to some implementations. The content of a given query image can vary depending on the particular implementation. For example, in various implementations, the superhero in image 500 can wear a suit in any number of color combinations of a particular color combination (e.g., black and yellow, black and red, etc.). The suit can also include a face mask of a particular color combination. Both the suit and the associated face mask can have other identifying marks or patterns. As shown, most of the image area is covered by the superhero. Also, most of the background of the image is a dark color, with some dull light - colored portions.

[0032] Figure 6 is an example image 600 of a superhero found in an image search result according to some implementations. In this example image, the content of the image includes the same superhero as shown in query image 500. In this example implementation, the superhero in image 600 is wearing the same suit that includes the same face mask as the superhero in image 500 of Figure 5. Also, the superhero covers less than half of the image area, and most of the background within the image is a light color, with something that appears like the aerial perspective of city buildings.

[0033] Figure 7 is an example image 700 of a person found in an image search result according to some implementations. In this example image, the content of the image includes a person. This person is wearing clothing of colors that are different but somewhat similar to the suits of the superheroes in images 500 and 600 of Figures 5 and 6 respectively. This person is also wearing a face mask that is different from the masks of the superheroes in images 500 and 600 of Figures 5 and 6 respectively.

[0034] In various implementations, the system calculates adjusted SIFT values based on SSIM values. For example, the system can calculate adjusted SIFT values by multiplying the SIFT values (associated with a given search result image and query image) by the SSIM coefficients (associated with the given search result image and query image). With respect to images 500, 600, and 700, the system can include images 600 and 700 in a set of search result images similar to query image 500.

[0035] In block 210 of FIG. 2, system 102 calculates histogram values based on a first object and a second object. A histogram indicates how high or low the occurrence frequency of specific values in the first image and the second image is. In various implementations, the histogram is a distribution of its discrete intensity levels, and the range can extend from 0 to L - 1. In various implementations, the histogram values are also based on various factors such as mood and brightness. For example, a given image can have a certain level of brightness in a specific image area. Such a brightness level can indicate time (e.g., morning, noon, evening, etc.). Such a brightness level can also indicate whether an object in a given image exists indoors or outdoors, etc.

[0036] In various implementations, the histogram values are based on the frequencies of predetermined histogram values of the first image and the second image. In various implementations, the histogram is a normalized histogram. Also, the normalization of the histogram is a technique for converting the discrete distribution of intensity into a discrete distribution of probability. The system divides each value of the histogram by the number of pixels to perform this conversion.

[0037] In various implementations, the system calculates adjusted histogram values based on SSIM values and SIFT values. For example, the system can calculate adjusted histogram values by multiplying the histogram values by the SSIM coefficients and the SIFT coefficients.

[0038] In block 212, system 102 calculates a similarity score based on SSIM values, SIFT values, and histogram values. In various implementations, the system applies the following equations to the first and second images. SSIM+(1 - SSIM) * SIFT+(1 - SSIM) * (1 - SIFT) * Histogram. The system calculates a similarity score in the range of 0 to 1. Also, as shown herein, the system calculates a similarity score for each search result image found during the search.

[0039] In various implementations, the system calculates the sum of the SSIM value, the adjusted SIFT value, and the adjusted histogram value to calculate the similarity score. The similarity score approaches 0 as the similarity between at least one first object and at least one second object decreases, and approaches 1 as the similarity between at least one first object and at least one second object increases.

[0040] SSIM+(1 - SSIM) * SIFT+(1 - SSIM) * (1 - SIFT) * In an example of histogram calculation, the system can calculate the SSIM value as 0.8, which indicates a relatively high similarity (e.g., similar objects). The system can calculate the SIFT value as 0.1, which indicates a relatively low similarity (e.g., dissimilar background). The system can calculate the adjusted SIFT value as (1 - 0.8) * 0.1, i.e., (0.2) * 0.1, i.e., 0.02. The system can calculate the histogram as 0.1, which indicates a relatively low similarity (e.g., dissimilar mood, lighting, etc.). The system can calculate the adjusted histogram value as (1 - 0.8) * (1 - 0.1) * 0.1, i.e., (0.2) * (0.9) *(0.1), that is, calculate it as 0.018. As a result, the similarity score becomes 0.8218.

[0041] In various implementations, the system ranks the search result images 600 and 700 (among other search result images) based on their similarity to the query image. Referring to the query image 500 in FIG. 5 and the respective search result images 600 and 700 in FIGS. 6 and 7, the system ranks image 600 higher than image 700 from the perspective of similarity and the resulting similarity score. For example, image 600 includes an object similar to the same superhero as that of the query image 500 based on the SSIM value. Image 600 includes a different background than that of the query image 500 based on the SIFT value, but the SSIM value has a higher weight than the SIFT value. Therefore, the system ranks image 600 higher than image 700 based only on the SSIM value and the SIFT value.

[0042] Also, image 600 includes a different mood than that of the query image 500 based on the histogram value, but the SSIM value has a higher weight than the histogram value. Therefore, the system ranks image 600 higher than image 700 based only on the SSIM value and the histogram value.

[0043] Conventional sorting techniques may rank image 700 higher than image 600 by focusing only on the pixels. Since system 102 analyzes and compares images based on the SSIM value, the SIFT value, and the histogram value, it ranks the search result images more accurately.

[0044] Steps, operations, or calculations may be shown in a particular order, but in certain implementations, the order can be changed. Other step orders are possible depending on the particular implementation. In some particular implementations, multiple steps shown sequentially herein can be executed simultaneously. Also, some implementations can have not all of the steps and / or can have other steps instead of or in addition to the steps shown herein.

[0045] The implementations described in this specification bring various advantages. For example, the implementations described in this specification can also be applied to reverse image search, label matching, image tracking, and image recognition. The implementations can also be used for keyframe search, similar image search, and / or reverse search of videos.

[0046] FIG. 8 is a block diagram of a network environment example 800 that can be used in some implementations described in this specification. In some implementations, the network environment 800 includes a system 802 that includes a server device 804 and a database 806. For example, the system 802 can be used to implement the system 102 of FIG. 1 and to execute the implementations described in this specification. The network environment 800 includes client devices 810, 820, 830, and 840, which can communicate with the system 802 and / or with each other directly or via the system 802. The network environment 800 also includes a network 850 that enables the system 802 and the client devices 810, 820, 830, and 840 to communicate. The network 850 can be any suitable communication network such as a Wi-Fi network, a Bluetooth network, the Internet, etc.

[0047] For ease of explanation, FIG. 8 shows one block for each of the system 802, the server device 804, and the network database 806, and four blocks for the client devices 810, 820, 830, and 840. The blocks 802, 804, and 806 can also represent multiple systems, server devices, and network databases. Also, any number of client devices can exist. In other implementations, the environment 800 can also not have all of the components shown in the figure and / or can have other elements including other types of elements instead of or in addition to the elements shown in this specification.

[0048] The implementations described in this specification are executed by the server device 804 of the system 802. However, in other implementations, the execution of the implementations described in this specification can be facilitated by any suitable component or combination of components related to the system 802, or any suitable one or more processors related to the system 802.

[0049] In various implementations described in this specification, the processors of the system 802 and / or the processors of any of the client devices 810, 820, 830, and 840 cause the elements (e.g., information, etc.) described in this specification to be displayed within a user interface on one or more display screens.

[0050] FIG. 9 is a block diagram of an example computer system 900 that can be used in some implementations described in this specification. For example, the computer system 900 can be used to implement the server device 804 of FIG. 8 and / or the system 102 of FIG. 1, and to execute the implementations described in this specification. In some implementations, the computer system 900 can include a processor 902, an operating system 904, a memory 906, and an input / output (I / O) interface 908. In various implementations, the processor 902 can be used to implement the various functions and features described in this specification and to execute the implementation of the methods described in this specification. Although the processor 902 is described as executing the implementations described in this specification, the steps described can also be executed by any suitable component or combination of components of the computer system 900, or any suitable one or more processors related to the computer system 900 or any suitable system. The implementations described in this specification can be executed on a user device, on a server, or in a combination thereof.

[0051] The computer system 900 includes a software application 910 that can be stored on the memory 906, or any other suitable storage location, or a computer-readable medium. The software application 910 provides instructions that enable the processor 902 to execute the implementations described herein and other functions. The software application can also include an engine, such as a network engine, that executes various functions related to one or more networks and network communications. The components of the computer system 900 can be implemented by any combination of one or more processors, or any combination of hardware devices, as well as any combination of hardware, software, firmware, etc.

[0052] For ease of explanation, FIG. 9 shows one block for each of the processor 902, operating system 904, memory 906, I / O interface 908, and software application 910. These blocks 902, 904, 906, 908, and 910 can also represent multiple processors, operating systems, memories, I / O interfaces, and software applications. In various implementations, the computer system 900 may not have all of the components shown in the figure and / or may have other elements including other types of elements instead of or in addition to the elements shown herein.

[0053] Although specific implementations have been described, these specific implementations are merely illustrative and not limiting. The concepts shown in these examples can also be applied to other examples and implementations.

[0054] In various implementations, software for execution by one or more processors is encoded on one or more non-transitory computer-readable media. This software, when executed by one or more processors, performs the implementations and other functions described herein.

[0055] For the implementation of routines of a particular implementation, any suitable programming language can be used, including C, C++, C#, Java, JavaScript, assembly language, etc. Different programming techniques such as procedural or object-oriented can be used. These routines can be executed on a single processing device or multiple processors. Although steps, operations or calculations may be shown in a specific order, this order can be changed in different specific embodiments. In some specific implementations, a plurality of steps shown sequentially herein can also be executed simultaneously.

[0056] A particular implementation can be implemented in a non-transitory computer-readable storage medium (also referred to as a machine-readable storage medium) used by or connected to an instruction execution system, apparatus or device. A particular implementation can also be implemented in the form of control logic in software or hardware or a combination thereof. The control logic can execute the implementations and other functions described herein when executed by one or more processors. For example, a tangible medium such as a hardware storage device can be used for storing control logic that can include executable instructions.

[0057] A particular implementation can be implemented by using a programmable general-purpose digital computer and / or by using application-specific integrated circuits, programmable logic devices, field-programmable gate arrays, optical, chemical, biological, quantum or nano-engineering systems, components and mechanisms. Generally, the functions of a particular implementation can be realized by any means well known in the art. Distributed networked systems, components and / or circuits can also be used. The communication or transfer of data can be by wired, wireless or any other means.

[0058] A "processor" can include any suitable hardware and / or software system, mechanism, or component that processes data, signals, or other information. The processor can include a general-purpose central processing unit, multiple processing units, a dedicated circuit for realizing functions, or other systems having such. The processing need not be restricted by geographical location or have time limitations. For example, the processor can execute its functions in "real time", "offline", "batch mode", etc. Some of the processing can also be executed by different (or the same) processing systems at different times and in different locations. A computer can be any processor that communicates with a memory. The memory can be any suitable data storage, memory, and / or non-transitory computer-readable storage medium, including an electronic storage device such as random access memory (RAM), read-only memory (ROM), magnetic storage devices (such as hard disk drives), flash memory, optical storage devices (such as CDs or DVDs), magnetic or optical disks, or other tangible media suitable for storing instructions (such as program or software instructions) executed by the processor. For example, a tangible medium such as a hardware storage device can be used to store control logic including executable instructions. The instructions can also be provided as electrical signals, for example, included in the form of service-type software (SaaS) distributed from a server (such as a distributed system and / or a cloud computing system).

[0059] Also, it will be understood that when useful according to a particular application, one or more of the elements shown in the drawings / figures can be implemented in a more separated or integrated form, or in some cases removed or made inoperable. Implementing a program or code storable in a machine-readable medium that enables a computer to execute any of the above-described methods is also included within the spirit and scope of the present invention.

[0060] As used throughout this specification and the following claims, the articles "a" and "the" include plural referents unless the context clearly dictates otherwise. Also, as used throughout this specification and the following claims, the meaning of "in" includes the meanings of "in" and "on" unless the context clearly dictates otherwise.

[0061] While specific implementations have been described herein, the above disclosure contemplates modifications, various changes, and substitutions, and in some instances, it should be understood that some features of a specific implementation may be used without the use of corresponding other features without departing from the described scope and spirit. Accordingly, many modifications can be made to adapt a particular situation or material to the basic scope and spirit.

Description of Reference Numerals

[0062] 100 Network environment 102 System 104 Server device 106 Database 110 Client 150 Network

Claims

1. A system comprising: one or more processors; and logic encoded on one or more non-transitory computer-readable storage media for execution by the one or more processors, wherein the logic, when executed, receives a first image including at least one first object; receives a second image including at least one second object; calculates a Structural Similarity Index Measure (SSIM) value based on the at least one first object and the at least one second object; calculates a Scale-Invariant Feature Transform (SIFT) value based on the at least one first object and the at least one second object; calculates an adjusted SIFT value by multiplying the SIFT value by an SSIM coefficient, where the larger the SSIM value, the smaller the coefficient; calculates a histogram value based on the at least one first object and the at least one second object; calculates an adjusted histogram value by multiplying the histogram value by the SSIM coefficient and a SIFT coefficient, where the larger the SIFT value, the smaller the coefficient; calculates the sum of the SSIM value, the adjusted SIFT value, and the adjusted histogram value; calculates a similarity score based on the sum of the SSIM value, the adjusted SIFT value, and the adjusted histogram value; and is operable to cause the one or more processors to perform operations including these. A system characterized by this.

2. The SSIM value is based on one or more of the luminance, contrast, and structure of the first image and the second image. The system according to claim 1.

3. The SIFT value is based on one or more predetermined features of the first image and the second image. The system according to claim 1.

4. The histogram value is based on the frequency of predetermined histogram values of the first image and the second image. The system according to claim 1.

5. A non-transitory computer-readable storage medium storing program instructions, where the program instructions, when executed by one or more processors, receive a first image including at least one first object; Receiving a second image including at least one second object; Calculating a Structural Similarity Index Measure (SSIM) value based on the at least one first object and the at least one second object; Calculating a Scale-Invariant Feature Transform (SIFT) value based on the at least one first object and the at least one second object; Calculating an adjusted SIFT value by multiplying the SIFT value by an SSIM coefficient, where the larger the SSIM value, the smaller the coefficient; Calculating a histogram value based on the at least one first object and the at least one second object; Calculating an adjusted histogram value by multiplying the histogram value by the SSIM coefficient and a SIFT coefficient, where the larger the SIFT value, the smaller the coefficient; Calculating the sum of the SSIM value, the adjusted SIFT value, and the adjusted histogram value; Calculating a similarity score based on the sum of the SSIM value, the adjusted SIFT value, and the adjusted histogram value; A non-transitory computer-readable storage medium operable to cause the one or more processors to perform operations including the above.

6. The SSIM value is based on one or more of the luminance, contrast, and structure of the first image and the second image. The computer-readable storage medium according to claim 5.

7. The SIFT value is based on one or more predetermined features of the first image and the second image. The computer-readable storage medium according to claim 5.

8. The histogram value is based on the frequency of predetermined histogram values of the first image and the second image. The computer-readable storage medium according to claim 5.

9. A computer-implemented method for evaluating similar content-based images, comprising: Receiving a first image including at least one first object; Receiving a second image including at least one second object; Calculating a Structural Similarity Index Measure (SSIM) value based on the at least one first object and the at least one second object; Calculating Scale-Invariant Feature Transform (SIFT) values based on the at least one first object and the at least one second object; Calculating an adjusted SIFT value by multiplying the SIFT value by an SSIM coefficient, where the larger the SSIM value, the smaller the coefficient; Calculating histogram values based on the at least one first object and the at least one second object; Calculating an adjusted histogram value by multiplying the histogram value by the SSIM coefficient and a SIFT coefficient, where the larger the SIFT value, the smaller the coefficient; Calculating the sum of the SSIM value, the adjusted SIFT value, and the adjusted histogram value; Calculating a similarity score based on the sum of the SSIM value, the adjusted SIFT value, and the adjusted histogram value; A method characterized by including the above.

10. The SSIM value is based on one or more of the luminance, contrast, and structure of the first image and the second image. The method according to claim 9.

11. The SIFT value is based on one or more predetermined features of the first image and the second image. The method according to claim 9.

12. The histogram value is based on the frequency of predetermined histogram values of the first image and the second image. The method according to claim 9.

13. Further including calculating an adjusted SIFT value based on the SSIM value. The method according to claim 9.

14. Further including calculating an adjusted histogram value based on the SSIM value and the SIFT value. The method according to claim 9.

Images