Similar image retrieving device

Abstract

To properly retrieve a similar image even for a photograph of a wall covering material having a simple pattern of a light color.SOLUTION: A similar image retrieval device 1 according to the present disclosure includes a storage unit 11 that stores a candidate image, a plurality of feature amounts extracted from the candidate image, and attribute information in association with each other, an image acquisition unit 12 that acquires an input image input by a user, a feature amount extraction unit 13 that extracts a plurality of feature amounts from the input image, an attribute information acquisition unit 14 that acquires attribute information input by the user, a retrieval unit 15 that retrieves a similar image of the input image from the candidate images on the basis of similarity between the plurality of feature amounts extracted from the input image and the plurality of feature amounts extracted from the candidate images, and an output unit 16 that selects and outputs a predetermined number of candidate images from the candidate images retrieved by the retrieval unit 15 on the basis of the attribute information acquired by the attribute information acquisition unit 14 and the attribute information stored in the storage unit 11, and the attribute information includes information indicating a color and a pattern.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a similar image search device. [Background technology]

[0002] BACKGROUND ART Conventionally, techniques for searching for images similar to an input image have been known (see, for example, Patent Documents 1 to 3).

[0003] Specifically, this technique searches for an image similar to the input image from among the candidate images based on the degree of similarity between the feature amounts of the input image and the feature amounts of the candidate images. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-251850 [Patent Document 2] Japanese Patent Application Laid-Open No. 2011-203776 [Patent Document 3] Japanese Patent Application Laid-Open No. 2012-58940 Summary of the Invention [Problem to be solved by the invention]

[0005] For many years, the interior design industry has faced the challenge of not knowing the product numbers of old or damaged wall coverings or flooring materials when replacing them.

[0006] Specifically, the person carrying out the reupholstering work would contact the manufacturer, and the manufacturer's representative would search through the many samples in the sample book to find similar products to the current wall coverings, flooring, etc., but this process required a considerable amount of man-hours.

[0007] Furthermore, there was inconsistency among the staff members regarding which wall coverings and flooring materials were deemed similar, and it was necessary to efficiently identify candidates for similar wall coverings and flooring materials.

[0008] Furthermore, since it is difficult for manufacturers to secure the space to store all of their past sample books, they are being called upon to solve this problem through digital measures.

[0009] In response to the above-mentioned problems, it has been found that even when attempting to search for similar images using the techniques of Patent Documents 1 to 3 as input images of current wall coverings, flooring materials, etc., it is often not possible to search for appropriate similar images for the following reasons.

[0010] The first reason is that for wall coverings and flooring materials, which often have simple patterns in light colors, the techniques in Patent Documents 1 to 3 mentioned above are unable to properly extract the features of the input image, and there is a possibility that images that are completely unsimilar to the input image may be searched for as similar images.

[0011] The second reason is that, due to lighting, photographs of current wall coverings, flooring materials, etc. may be taken in colors different from their actual colors, which could result in images that are completely unsimilar to the input image being searched for as similar images.

[0012] Therefore, the present disclosure has been made in consideration of the above-mentioned problems, and aims to provide a similar image search device that can appropriately search for similar images even for photographs of wall coverings, flooring materials, etc., which often have simple patterns in light colors. [Means for solving the problem]

[0013] A first feature of the present disclosure is a similar image search device, comprising: a storage unit that stores a candidate image, a plurality of feature amounts extracted from the candidate image, and attribute information in association with each other; The apparatus comprises an image acquisition unit that acquires an input image input by a user, a feature extraction unit that extracts the plurality of feature amounts from the input image, an attribute information acquisition unit that acquires the attribute information input by the user, a search unit that searches the candidate images for images similar to the input image based on the similarity between the plurality of feature amounts extracted from the input image and the plurality of feature amounts extracted from the candidate images, and an output unit that selects a predetermined number of candidate images from the candidate images searched by the search unit based on the attribute information acquired by the attribute information acquisition unit and the attribute information stored in the memory unit, and outputs the selected candidate images, wherein the attribute information includes information indicating color and pattern. [Effects of the Invention]

[0014] According to the present disclosure, it is possible to provide a similar image search device that can appropriately search for similar images even for photographs of wall coverings, flooring materials, etc., which often have simple patterns in light colors. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram showing an example of functional blocks of a similar image search device 1 according to an embodiment. [Figure 2] FIG. 2 is a diagram showing an example of a similar image search screen displayed by the output unit 16 of the similar image search device 1 according to an embodiment. [Figure 3] FIG. 3 is a diagram showing an example of a similar image search screen displayed by the output unit 16 of the similar image search device 1 according to an embodiment. [Figure 4] FIG. 4 is a diagram showing an example of a similar image search screen displayed by the output unit 16 of the similar image search device 1 according to an embodiment. [Figure 5] FIG. 5 is a diagram showing an example of a screen displaying search results of similar images displayed by the output unit 16 of the similar image search device 1 according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In the following description of the drawings, identical or similar parts are designated by identical or similar reference numerals. However, it should be noted that the drawings are schematic, and the dimensional ratios and the like may differ from those of the actual parts. Therefore, specific dimensions and the like should be determined with reference to the following description. Furthermore, parts in the drawings may have different dimensional relationships and ratios. In this specification and the drawings, elements having substantially the same functions and configurations are designated by the same reference numerals to avoid redundant description, and elements not directly related to the present invention are not shown.

[0017] (First embodiment) Hereinafter, a similar image search device 1 according to a first embodiment of the present invention will be described with reference to Fig. 1 to Fig. 5. Fig. 1 is a diagram showing an example of functional blocks of the similar image search device 1 according to this embodiment, Figs. 2 to 4 are diagrams showing an example of a search screen for similar images displayed by the output unit 16 of the similar image search device 1 according to this embodiment, and Fig. 5 is a diagram showing an example of a screen displaying search results for similar images displayed by the output unit 16 of the similar image search device 1 according to this embodiment.

[0018] The similar image search device 1 according to this embodiment can search for images similar to any input image.

[0019] For example, the similar image search device 1 can use a photograph of an interior item such as a wall covering, flooring material, or curtain as an input image and search for images similar to the input image. Here, wall coverings include decorative sheets, decorative films, wallpaper, ceiling materials, etc.

[0020] In the following, in this embodiment, we will use as an example a case in which a photograph of a wall covering is used as an input image to search for similar images of wallpaper, but the present invention is not limited to such a case and can be applied to cases in which similar images of any input image are searched for.

[0021] As shown in FIG. 1, the similar image search device 1 according to this embodiment includes a memory unit 11, an image acquisition unit 12, a feature extraction unit 13, an attribute information acquisition unit 14, a search unit 15, and an output unit 16.

[0022] The storage unit 11 stores candidate images, a plurality of feature amounts extracted from the candidate images, and attribute information in association with each other. Here, the attribute information may include information indicating the type, information indicating the color, information indicating the pattern (texture), sample book identification information (sample book name and sample book publication number), information indicating the start date of publication in the sample book, information indicating the end date of publication in the sample book, information indicating the thickness of the interior item, information indicating the price of the interior item, etc. Note that the attribute information may include any information recorded in the product database of interior items.

[0023] Here, the plurality of feature quantities may be a predetermined feature vector defined by any method, for example, such a predetermined feature vector may include feature quantities such as color, pattern, material, etc.

[0024] The image acquisition unit 12 acquires an input image input by a user. For example, the image acquisition unit 12 can be operated by an input device such as a keyboard or a mouse, and may acquire the input image input by the user via such an input device. Alternatively, the image acquisition unit 12 may acquire the input image input by the user from a user terminal 2 (for example, a smartphone or a PC) via a network.

[0025] The feature extraction unit 13 extracts the above-mentioned multiple feature amounts (for example, predetermined feature vectors) from the input image acquired by the image acquisition unit 12.

[0026] The attribute information acquisition unit 14 acquires attribute information input by a user. For example, the attribute information acquisition unit 14 can be operated by an input device such as a keyboard or a mouse, and may acquire the attribute information input by the user via such an input device. Alternatively, the attribute information acquisition unit 14 may acquire the attribute information input by the user from the user terminal 2 via a network.

[0027] The search unit 15 searches the candidate images for an image similar to the input image based on the similarity between a plurality of feature amounts extracted from the input image and a plurality of feature amounts extracted from the candidate images.

[0028] For example, the search unit 15 may calculate the absolute value (distance) of the difference between a predetermined feature vector extracted from the input image and a predetermined feature vector extracted from a candidate image as the above-mentioned similarity, and search for candidate images whose absolute value of such difference is less than a predetermined value.

[0029] The search unit 15 may search for a similar image to a partial image region A (see FIG. 4 described later) selected by the user from the input image acquired by the image acquisition unit 12.

[0030] The output unit 16 selects a predetermined number of candidate images from the candidate images searched by the search unit 15 based on the attribute information acquired by the attribute information acquisition unit 14 and the attribute information stored in the memory unit 11, and outputs the selected candidate images.

[0031] For example, the output unit 16 may output (display) the selected candidate image on an output device such as a display (see FIG. 5 described later), or may output (transmit) the selected candidate image to the user terminal 2 via a network.

[0032] Hereinafter, with reference to FIGS. 2 to 4, an example of screen transitions when searching for an image similar to an input image using the similar image search device 1 will be described.

[0033] First, on the screen shown in Figure 2, the user clicks the "Select File" button, selects the desired input image (RE53096_C01.jpg in the example of Figure 2), and clicks the "Next" button, which transitions to the screen shown in Figure 3.

[0034] Here, the user may input attribute information in the pull-down menus for "type," "color," and "texture" on the screen shown in FIG.

[0035] For example, "Type" is a pull-down menu for inputting attribute information indicating the type of input image (e.g., wallpaper, etc.), "Color" is a pull-down menu for inputting attribute information indicating the color of the input image, and "Texture" is a pull-down menu for inputting attribute information indicating the pattern of the input image.

[0036] Second, the screen shown in Fig. 3 displays the input image selected by the user on the screen shown in Fig. 2, and if the input image is the desired input image, the user clicks the "Search" button. On the other hand, if the input image is not the desired input image, the user clicks the "Search for a different image" button to reselect the input image on the screen shown in Fig. 2.

[0037] Here, as shown in Figure 4, the user selects an image area A, which is a part of the input image displayed on the screen shown in Figure 3, and clicks the "Search" button to search for images similar to that image area A.

[0038] With this configuration, for example, when performing a similarity search using an image of wallpaper at home taken with a mobile device as an input image, even if there is dirt or scratches on part of the wallpaper, it is possible to search for suitable similar images to the input image by searching for similar images in the image area excluding the dirt, scratches, etc.

[0039] Third, on the screen shown in FIG. 5, the output unit 16 displays the image selected as the similar image to the input image described above.

[0040] As described above, according to this embodiment, by including in the attribute information information indicating the color of the input image and information indicating the pattern of the input image, it is possible to avoid the situation where an image that is completely unsimilar to the input image is searched for, even when the input image is a photograph of a wall covering with a simple pattern in a light color.

[0041] Furthermore, according to this embodiment, even if a photo of current wall coverings, flooring materials, etc. is taken in a color different from the actual color due to the influence of lighting, the user can input information indicating the color of the input image as attribute information, thereby avoiding a situation where an image that is completely dissimilar to the input image is searched for as a similar image.

[0042] Furthermore, according to this embodiment, by including in the attribute information information indicating at least one of the publication start date and the publication end date of the input image in the sample book, when a photograph of an interior item is used as the input image, it is possible to avoid outputting an image of an interior item that is published in a sample book that will be discontinued in the near future (or an interior product that was published in a sample book that is already discontinued). Note that the same interior item may be published in multiple different sample books.

[0043] Here, one sample book may be discontinued, but the other may continue to feature interior goods, and it is possible to avoid outputting images of the interior goods featured in the former sample book and output images of the interior goods featured in the latter sample book.

[0044] Furthermore, it is preferable that wall coverings have a certain thickness so that the unevenness of the old base is not noticeable when the wall covering is replaced. In this regard, according to the present embodiment, by including information indicating the thickness of the interior item in the attribute information, it is possible to output, as similar images, only photographs of wall coverings that can make the unevenness of the old base less noticeable.

[0045] Furthermore, according to this embodiment, by including information indicating the price of the input image in the attribute information, it is possible to output only similar images of interior goods in a desired price range.

[0046] In addition, since old wall coverings, flooring materials, etc. are often scratched or stained, there are cases where a photograph of the wall coverings, flooring materials, etc. is taken in a state that differs from the actual color or pattern. In this regard, according to the present embodiment, even in such cases, it is possible to avoid a situation where an image that is completely dissimilar to the input image is searched for as a similar image.

[0047] Although the present invention has been described in detail using the above-described embodiments, it is clear to those skilled in the art that the present invention is not limited to the embodiments described herein. The present invention can be implemented in modified and altered forms without departing from the spirit and scope of the present invention as defined by the claims. Therefore, the description in this specification is intended to be illustrative and explanatory and does not have any limiting meaning on the present invention. [Explanation of symbols]

[0048] 1...Similar image search device 11...Storage section 12...Image acquisition unit 13...Feature extraction unit 14…Attribute information acquisition 15...Search section 16...Output section 2...User terminal

Claims

1. a storage unit that stores a candidate image, a plurality of feature amounts extracted from the candidate image, and attribute information in association with each other; an image acquisition unit that acquires an input image input by a user; a feature extraction unit that extracts the plurality of feature amounts from the input image; an attribute information acquisition unit that acquires the attribute information input by the user; a search unit that searches the candidate images for an image similar to the input image based on the similarity between the plurality of feature amounts extracted from the input image and the plurality of feature amounts extracted from the candidate images; an output unit that selects a predetermined number of candidate images from the candidate images searched by the search unit based on the attribute information acquired by the attribute information acquisition unit and the attribute information stored in the storage unit, and outputs the selected candidate images; The attribute information includes information indicating color and pattern.

2. The similar image search device according to claim 1 , wherein the attribute information includes information indicating at least one of a publication start date in a sample book and a publication end date in the sample book.

3. The similar image search device according to claim 1 , wherein the attribute information includes information indicating thickness.

4. The similar image search device according to claim 1 , wherein the attribute information includes information indicating a price.

5. The similar image search device according to claim 1 , wherein the search unit searches for a similar image in a partial image area selected by the user in the input image acquired by the image acquisition unit.

Images