Image theme color retrieval method and device based on ES

Abstract

The present invention relates to the field of image retrieval technology, and discloses an ES-based image theme color retrieval method and device, which aims to improve the image retrieval performance. The scheme mainly includes: obtaining multiple original image data and preprocessing the original image data; respectively determining the theme color data corresponding to each original image data, and saving each original image data and its corresponding theme color data to ES, wherein the theme color data at least includes the first proportion of each theme color in the corresponding original image data and the first HSV data corresponding to each theme color; obtaining the theme color to be retrieved, converting the theme color to be retrieved into second HSV data, and determining the first similarity score of each original image data in ES according to the second HSV data; and determining the retrieval result according to the first similarity score. The present invention enables users to quickly and accurately retrieve the required images, and is particularly suitable for the media industry.

Description

Technical Field

[0001] The present invention relates to the technical field of picture retrieval, and in particular to an ES-based picture theme color retrieval method and device. Background Art

[0002] With the rapid development of the Internet, a large amount of image data has appeared on the Internet. How to quickly retrieve these images has become a very important issue. Especially in the media industry, there is a great demand for image retrieval for images and graphic news, especially image retrieval based on image theme colors.

[0003] Most of the existing image retrieval methods based on theme color are based on color histogram intersection, Euclidean distance, Manhattan distance, etc., but these retrieval methods have problems such as complex algorithms, low retrieval accuracy, and poor retrieval performance. Summary of the invention

[0004] The present invention aims to provide an ES-based picture theme color retrieval method and device to improve picture retrieval performance so that users can quickly and accurately retrieve required pictures.

[0005] The technical solution adopted by the present invention to solve the above technical problems is:

[0006] On the one hand, a method for retrieving image theme colors based on ES is provided, comprising the following steps:

[0007] Acquire multiple original image data, and preprocess the original image data;

[0008] Determine the theme color data corresponding to each original picture data respectively, and save each original picture data and its corresponding theme color data to ES, wherein the theme color data at least includes a first proportion of each theme color in the corresponding original picture data and a first HSV data corresponding to each theme color;

[0009] Acquire a theme color to be retrieved, convert the theme color to be retrieved into second HSV data, and determine a first similarity score of each original image data in ES according to the second HSV data;

[0010] A search result is determined according to the first similarity score.

[0011] Furthermore, the original image data is preprocessed, specifically including:

[0012] Scale the image sizes corresponding to all original image data to the preset size.

[0013] Furthermore, the determining of the theme color data corresponding to the original image data specifically includes:

[0014] Get all sample pixels representing colors in the original image data;

[0015] Extracting theme colors corresponding to the original picture data according to the sample pixel points and based on a clustering algorithm, and determining a first proportion of each theme color in the corresponding original picture data according to the extracted theme colors;

[0016] Convert each theme color into the first HSV data in the HSV color space.

[0017] Further, determining a first similarity score of each original image data in the ES according to the second HSV data specifically includes:

[0018] Filter the original image data in ES;

[0019] For each filtered original image data, a first preset number of theme colors with a first proportion at the top are selected, the remaining theme colors are discarded, and a second proportion of each selected theme color is recalculated according to the first proportion;

[0020] Calculate the second similarity scores of the first HSV data and the second HSV data corresponding to each selected theme color respectively;

[0021] The first similarity score of each original image data is determined according to the second proportion of each selected theme color and the corresponding second similarity score.

[0022] Furthermore, filtering the original image data in the ES specifically includes:

[0023] The original image data in ES that does not meet the first preset condition is filtered according to the theme color to be retrieved, wherein the first preset condition is: the original image data contains at least one theme color, the H value, S value and V value in the first HSV data corresponding to the theme color are within a first preset range, and the first proportion of the theme color in the corresponding original image data is greater than a preset value, and the first preset range is determined according to the H value, S value and V value in the second HSV data.

[0024] Further, the calculating a second similarity score between the first HSV data corresponding to the theme color and the second HSV data specifically includes:

[0025] In the filtered original image data, original image data that does not meet the second preset condition is determined according to the theme color to be retrieved, and its second similarity score is set to 0, wherein the second preset condition is that the original image data contains at least one theme color, and the H value, S value, and V value of the theme color corresponding to the first HSV data are within a second preset range, and the second preset range is determined according to the H value, S value, and V value in the second HSV data;

[0026] For the original image data that meets the second preset condition, the similarity scores of the first HSV data corresponding to the theme color in the original image data and the second HSV data in the H dimension, S dimension and V dimension are determined based on the attenuation function, and the similarity scores in the H dimension, S dimension and V dimension are added to obtain the corresponding second similarity score.

[0027] Furthermore, the method further comprises:

[0028] When the theme color data is saved in the ES, only the theme color data corresponding to the second preset number of theme colors with a first leading proportion is saved in the ES.

[0029] Further, determining the first similarity score of each original image data according to the second proportion of each selected theme color and the corresponding second similarity score specifically includes:

[0030] For each filtered original picture data, a weighted average calculation is performed on the second similarity score of each theme color according to the second proportion to obtain a first similarity score of each original picture data.

[0031] Further, determining the search result according to the first similarity score specifically includes:

[0032] The original picture data are sorted from high to low according to the first similarity, and the sorted original picture data are returned.

[0033] On the other hand, a device for searching image theme colors based on ES is provided, comprising:

[0034] A preprocessing unit, used to obtain a plurality of original picture data and preprocess the original picture data;

[0035] A first determining unit, used to respectively determine the theme color data corresponding to each original picture data, and save each original picture data and its corresponding theme color data to ES, wherein the theme color data at least includes a first proportion of each theme color in the corresponding original picture data and a first HSV data corresponding to each theme color;

[0036] The second determination unit is used to obtain the theme color to be retrieved, convert the theme color to be retrieved into second HSV data, determine the first similarity score of each original picture data in ES according to the second HSV data; and determine the search result according to the first similarity score.

[0037] The beneficial effects of the present invention are as follows: the ES-based picture theme color retrieval method and device described in the present invention respectively performs similarity scores on the original picture data in ES based on the proportion of the picture theme color and the corresponding HSV data, and then determines the retrieval results based on the similarity score results. The present invention improves the performance of picture retrieval based on theme colors, and improves the retrieval efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a flow chart of the ES-based image theme color retrieval method according to an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram of the process of extracting and storing theme colors according to an embodiment of the present invention;

[0040] Figure 3 A schematic diagram of the process of searching for the theme color of an image according to an embodiment of the present invention;

[0041] Figure 4 It is a structural schematic diagram of the ES-based image theme color retrieval device described in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0043] The present invention aims to provide an ES-based image theme color retrieval method and device, and its main technical scheme includes: obtaining multiple original image data and preprocessing the original image data; respectively determining the theme color data corresponding to each original image data, and saving each original image data and its corresponding theme color data to ES, wherein the theme color data at least includes a first proportion of each theme color in the corresponding original image data and a first HSV data corresponding to each theme color; obtaining a theme color to be retrieved, converting the theme color to be retrieved into a second HSV data, and determining a first similarity score of each original image data in ES according to the second HSV data; and determining a retrieval result according to the first similarity score.

[0044] Elasticsearch (ES) is a search server based on Lucene. It provides a distributed multi-user full-text search engine based on a RESTful web interface. Elasticsearch is developed in Java and released as open source under the terms of the Apache license. It is a popular enterprise search engine. Elasticsearch is used in cloud computing and can achieve real-time search. It is stable, reliable, fast, and easy to install and use.

[0045] Based on the above search engine, the ES-based image theme color retrieval method and device provided by the present invention is mainly divided into two processes: one is the theme color extraction and storage process, and the other is the image theme color retrieval process, wherein the theme color extraction and storage process mainly includes: firstly, a large amount of original image data is obtained, and after preprocessing each original image data, the theme color contained in each original image data is determined respectively, and the theme color data corresponding to each original image data is determined, and the theme color data includes the proportion of each theme color and the corresponding first HSV data, and then each original image data and theme color data are correspondingly saved in ES, and the theme color extraction and storage process can be completed. The image theme color retrieval process mainly includes: firstly, the theme color to be retrieved input by the user is obtained, and the theme color to be retrieved is converted into the second HSV data in the HSV color space, and then the image is retrieved in ES according to the second HSV data. Specifically, the first similarity score of each original image data in ES relative to the theme color to be retrieved is determined mainly according to the theme color proportion and the similarity between HSV data, and finally the retrieval result is determined according to the first similarity score to complete the image retrieval.

[0046] Example

[0047] The ES-based image theme color retrieval method described in the embodiment of the present invention is as follows: Figure 1 As shown, the following steps are included:

[0048] Step 1: obtaining a plurality of original picture data and preprocessing the original picture data;

[0049] Specifically, this embodiment can obtain a large amount of original image data based on an existing image database or directly from the Internet, and then uniformly scale the image size corresponding to the obtained original image data to a preset size, for example, uniformly scaled to 256*256, to improve the accuracy of image retrieval.

[0050] Step 2: Determine the theme color data corresponding to each original picture data, and save each original picture data and its corresponding theme color data to ES, wherein the theme color data at least includes the first proportion of each theme color in the corresponding original picture data and the first HSV data corresponding to each theme color;

[0051] After preprocessing the original image data, the theme color data corresponding to each original image data is determined respectively, mainly including:

[0052] Step 21, obtaining all sample pixel points representing colors in the original image data;

[0053] Step 22: extracting the theme colors corresponding to the original picture data based on the sample pixel points and a clustering algorithm, and determining a first proportion of each theme color in the corresponding original picture data based on the extracted theme colors;

[0054] In this embodiment, the clustering algorithm can be a K-means algorithm, a GMM algorithm, a Mean shift algorithm, a Quickshift algorithm, etc. Among them, the K-means algorithm is a relatively simple and commonly used clustering algorithm, and it is relatively easy to implement. The following is an illustration of the theme color extraction process using the K-means algorithm distance. The K-means algorithm includes four steps:

[0055] (1) Randomly select the initial cluster center;

[0056] (2) Assign each target to the nearest center (the nearest metric refers to the Euclidean distance from the target to the center);

[0057] (3) Based on the clusters in the previous step, recalculate the cluster center (the average value of all points to the center in the previous step);

[0058] (4) Repeat (2) until the cluster center no longer changes.

[0059] It should be noted that an original picture data usually has multiple theme colors. After obtaining the multiple theme colors corresponding to each original picture data, this embodiment needs to determine the first proportion of each theme color in the corresponding original picture data. In order to reduce the storage pressure of the database, for each original picture data, this embodiment only saves the second preset number of theme color data with the top first proportion to ES, where the second preset number can be set according to actual conditions, for example, 10.

[0060] Step 23: Convert each theme color into the first HSV data in the HSV color space.

[0061] As described above, the theme color data includes the first proportion of each theme color in the corresponding original image data and the first HSV data corresponding to each theme color. In this embodiment, for each original image data, a second preset number of theme colors with the highest first proportion are selected, and they are converted into the first HSV data in the HSV color space and saved to ES. At this time, the first HSV data is the center point (H1, S1, V1), where the H1 value ranges from 0 to 360, the S1 value ranges from 0 to 100, and the V1 value ranges from 0 to 100.

[0062] Step 3: Obtain the theme color to be retrieved, convert the theme color to be retrieved into second HSV data, and determine the first similarity score of each original image data in ES according to the second HSV data;

[0063] like Figure 2 As shown, steps 1 and 2 are used to complete the process of extracting and storing theme colors. Figure 3As shown, steps 3 and 4 are used to complete the image theme color retrieval process, which specifically includes:

[0064] When the user needs to search for images based on the theme color, the theme color to be searched input by the user can be obtained through the front-end color block, and the theme color to be searched can be converted into the second HSV data in the HSV color space. At this time, the second HSV data is the center point (H2, S2, V2), and then the first similarity score of each original image data is determined in ES based on the second HSV data.

[0065] In order to further improve the retrieval efficiency, this embodiment first filters the original image data in ES before performing similarity scoring, specifically including:

[0066] The original image data in ES that does not meet the first preset condition is filtered according to the theme color to be retrieved, wherein the first preset condition is: the original image data contains at least one theme color, the H value, S value and V value in the first HSV data corresponding to the theme color are within a first preset range, and the first proportion of the theme color in the corresponding original image data is greater than a preset value, and the first preset range is determined according to the H value, S value and V value in the second HSV data.

[0067] For example, for each original image data, determine whether it contains at least one theme color that meets the following conditions: the first proportion of the theme color in the corresponding original image data is greater than 10%, and the first HSV data corresponding to the theme color also satisfies: H2-30≤H1≤H2+30; S2-15≤S1≤S2+15; V2-15≤V1≤V2+15; if so, enter the subsequent judgment process, otherwise, discard the corresponding original image data.

[0068] In order to reduce the calculation pressure, after filtering the original image data in the ES, this embodiment selects a first preset number of theme colors with a first proportion at the top for each filtered original image data, discards the remaining theme colors, and recalculates the second proportion of each selected theme color according to the first proportion;

[0069] It can be understood that for each original image data, after discarding the theme color, the second proportions of the selected theme colors need to be recalculated so that the sum of the second proportions of the selected theme colors is 100%, wherein the first preset number can be set according to actual conditions, for example 5.

[0070] After recalculating the proportions, this embodiment calculates the second similarity scores of the first HSV data and the second HSV data corresponding to each selected theme color. In order to further reduce the calculation pressure, the calculation process of the second similarity score in this embodiment is as follows:

[0071] In the filtered original image data, original image data that does not meet the second preset condition is determined according to the theme color to be retrieved, and its second similarity score is set to 0, wherein the second preset condition is that the original image data contains at least one theme color, and the H value, S value, and V value of the theme color corresponding to the first HSV data are within a second preset range, and the second preset range is determined according to the H value, S value, and V value in the second HSV data;

[0072] For example, in the filtered original image data, determine whether it contains at least one theme color that meets the following conditions: the first HSV data corresponding to the theme color simultaneously satisfies: H2-15≤H1≤H2+15; S2-10≤S1≤S2+10; V2-10≤V1≤V2+10; if so, enter the subsequent judgment process, otherwise, set the corresponding second similarity score to 0.

[0073] For the original image data that meets the second preset condition, the similarity scores of the first HSV data corresponding to the theme color in the original image data and the second HSV data in the H dimension, S dimension and V dimension are determined based on the attenuation function, and the similarity scores in the H dimension, S dimension and V dimension are added to obtain the corresponding second similarity score.

[0074] It can be understood that the attenuation function assumes that the similarity gradually decreases with the degree of deviation from the center point. Based on this, this embodiment, for each original image data, respectively, performs similarity scoring on each selected theme color in the H dimension, S dimension, and V dimension. Specifically, first, the H dimension of the second HSV data corresponding to the theme color to be retrieved is taken as the center point, and the attenuation function is used to score. The farther from the center point, the lower the score, and the closer to the center point, the higher the score. Similarly, the S dimension and V dimension of the second HSV data corresponding to the theme color to be retrieved are taken as the center point in turn, and the attenuation function is used to score. Finally, the similarity scores on the H dimension, S dimension, and V dimension are added to obtain the corresponding second similarity score.

[0075] After obtaining the second similarity score of the HSV data, the first similarity score of each original image data is determined according to the second proportion of each selected theme color and the corresponding second similarity score, specifically including: for each filtered original image data, the second similarity score of each theme color is weighted averaged according to the second proportion to obtain the first similarity score of each original image data.

[0076] It can be understood that the larger the second proportion of the theme color is, the larger the corresponding weight is. By performing a weighted average calculation on the second similarity scores according to the second proportion, the first similarity scores of the original picture data can be obtained.

[0077] Step 4: Determine the search result according to the first similarity score.

[0078] It can be understood that the higher the first similarity score is, the more similar the images are considered to be. After obtaining the first similarity score, this embodiment sorts the original image data from high to low according to the first similarity, and returns the sorted original image data to complete the image retrieval.

[0079] Based on the above technical solution, this embodiment also proposes an ES-based image theme color retrieval device, such as Figure 4 As shown, including:

[0080] A preprocessing unit, used to obtain a plurality of original picture data and preprocess the original picture data;

[0081] A first determining unit, used to respectively determine the theme color data corresponding to each original picture data, and save each original picture data and its corresponding theme color data to ES, wherein the theme color data at least includes a first proportion of each theme color in the corresponding original picture data and a first HSV data corresponding to each theme color;

[0082] The second determination unit is used to obtain the theme color to be retrieved, convert the theme color to be retrieved into second HSV data, determine the first similarity score of each original picture data in ES according to the second HSV data; and determine the search result according to the first similarity score.

[0083] It can be understood that since the ES-based image theme color retrieval device described in the embodiment of the present invention is a device for implementing the ES-based image theme color retrieval method described in the embodiment, for the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method.

Claims

1. The ES-based image theme color retrieval method is characterized by: The following steps are involved: Acquire multiple original image data, and preprocess the original image data; Determine the theme color data corresponding to each original picture data respectively, and save each original picture data and its corresponding theme color data to ES, wherein the theme color data at least includes a first proportion of each theme color in the corresponding original picture data and a first HSV data corresponding to each theme color; Acquire a theme color to be retrieved, convert the theme color to be retrieved into second HSV data, and determine a first similarity score of each original image data in ES according to the second HSV data; Determine a search result according to the first similarity score; Determining a first similarity score of each original picture data in the ES according to the second HSV data specifically includes: Filter the original image data in ES; For each filtered original image data, a first preset number of theme colors with a first proportion at the top are selected, the remaining theme colors are discarded, and a second proportion of each selected theme color is recalculated according to the first proportion; Calculate the second similarity scores of the first HSV data and the second HSV data corresponding to each selected theme color respectively; Determine the first similarity score of each original image data according to the second proportion of each selected theme color and the corresponding second similarity score; The filtering of the original image data in the ES specifically includes: Filter the original image data in the ES that do not meet the first preset condition according to the theme color to be retrieved, wherein the first preset condition is: the original image data contains at least one theme color, the H value, S value and V value of the theme color in the first HSV data are within a first preset range, and the first proportion of the theme color in the corresponding original image data is greater than a preset value, and the first preset range is determined according to the H value, S value and V value in the second HSV data; Calculating the second similarity score between the first HSV data corresponding to each selected theme color and the second HSV data specifically includes: In the filtered original image data, original image data that does not meet the second preset condition is determined according to the theme color to be retrieved, and its second similarity score is set to 0, wherein the second preset condition is that the original image data contains at least one theme color, and the H value, S value, and V value of the theme color corresponding to the first HSV data are within a second preset range, and the second preset range is determined according to the H value, S value, and V value in the second HSV data; For the original image data that meets the second preset condition, the similarity scores of the first HSV data corresponding to the theme color in the original image data and the second HSV data in the H dimension, S dimension and V dimension are determined based on the attenuation function, and the similarity scores in the H dimension, S dimension and V dimension are added to obtain the corresponding second similarity score.

2. The ES-based image theme color retrieval method according to claim 1, characterized in that: Preprocessing the original image data specifically includes: Scale the image sizes corresponding to all original image data to the preset size.

3. The ES-based image theme color retrieval method according to claim 1, characterized in that: Determine the theme color data corresponding to each original image data, specifically including: Get all sample pixels representing colors in the original image data; Extracting theme colors corresponding to the original picture data according to the sample pixel points and based on a clustering algorithm, and determining a first proportion of each theme color in the corresponding original picture data according to the extracted theme colors; Convert each theme color into the first HSV data in the HSV color space.

4. The ES-based image theme color retrieval method according to claim 1, characterized in that: The method further includes: When the theme color data is saved in the ES, only the theme color data corresponding to the second preset number of theme colors with a first leading proportion is saved in the ES.

5. The ES-based image theme color retrieval method according to claim 1, characterized in that: Determining the first similarity score of each original image data according to the second proportion of each selected theme color and the corresponding second similarity score specifically includes: For each filtered original picture data, a weighted average calculation is performed on the second similarity score of each theme color according to the second proportion to obtain a first similarity score of each original picture data.

6. The ES-based image theme color retrieval method according to claim 1, characterized in that: Determining the search result according to the first similarity score specifically includes: The original picture data are sorted from high to low according to the first similarity, and the sorted original picture data are returned.

7. The image theme color retrieval device based on ES is characterized by: include: A preprocessing unit, used to obtain a plurality of original picture data and preprocess the original picture data; A first determining unit, used to respectively determine the theme color data corresponding to each original picture data, and save each original picture data and its corresponding theme color data to ES, wherein the theme color data at least includes a first proportion of each theme color in the corresponding original picture data and a first HSV data corresponding to each theme color; A second determination unit is used to obtain a theme color to be retrieved, convert the theme color to be retrieved into second HSV data, and determine a first similarity score of each original picture data in ES according to the second HSV data; and determining a search result according to the first similarity score; Determining a first similarity score of each original picture data in the ES according to the second HSV data specifically includes: Filter the original image data in ES; For each filtered original image data, a first preset number of theme colors with a first proportion at the top are selected, the remaining theme colors are discarded, and a second proportion of each selected theme color is recalculated according to the first proportion; Calculate the second similarity scores of the first HSV data and the second HSV data corresponding to each selected theme color respectively; Determine the first similarity score of each original image data according to the second proportion of each selected theme color and the corresponding second similarity score; The filtering of the original image data in the ES specifically includes: Filter the original image data in the ES that do not meet the first preset condition according to the theme color to be retrieved, wherein the first preset condition is: the original image data contains at least one theme color, the H value, S value and V value of the theme color in the first HSV data are within a first preset range, and the first proportion of the theme color in the corresponding original image data is greater than a preset value, and the first preset range is determined according to the H value, S value and V value in the second HSV data; Calculating the second similarity score between the first HSV data corresponding to each selected theme color and the second HSV data specifically includes: In the filtered original image data, original image data that does not meet the second preset condition is determined according to the theme color to be retrieved, and its second similarity score is set to 0, wherein the second preset condition is that the original image data contains at least one theme color, and the H value, S value, and V value of the theme color corresponding to the first HSV data are within a second preset range, and the second preset range is determined according to the H value, S value, and V value in the second HSV data; For the original image data that meets the second preset condition, the similarity scores of the first HSV data corresponding to the theme color in the original image data and the second HSV data in the H dimension, S dimension and V dimension are determined based on the attenuation function, and the similarity scores in the H dimension, S dimension and V dimension are added to obtain the corresponding second similarity score.

Images