Similar image search device

Abstract

To appropriately search for similar images even for photographs of wall covering materials having simple patterns in light colors. [Solution] The similar image search device 1 of the present disclosure includes a memory unit 11 that stores candidate images, multiple features extracted from the candidate images, and attribute information in association with each other, an image acquisition unit 12 that acquires an input image input by a user, a feature extraction unit 13 that extracts multiple features from the input image, an attribute information acquisition unit 14 that acquires the attribute information input by the user, a search unit 15 that searches for images similar to the input image from among the candidate images based on the similarity between the multiple features extracted from the input image and the multiple features extracted from the candidate images, and an output unit 16 that selects and outputs a predetermined number of candidate images from the candidate images searched by the search unit 15 based on the attribute information acquired by the attribute information acquisition unit 14 and the attribute information stored in the memory unit 11, and the attribute information includes information indicating color and pattern.

Description

[Technical field]

[0001] The present disclosure relates to a similar image search device. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there are known techniques for searching for an image similar to an input image (see, for example, Patent Documents 1 to 3).

[0003] Specifically, this technique searches for an image similar to the input image from among the candidate images, based on the degree of similarity between the feature amounts of the input image and the feature amounts of the candidate images. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2009-251850 A [Patent Document 2] JP 2011-203776 A [Patent Document 3] JP 2012-58940 A Summary of the Invention [Problem to be solved by the invention]

[0005] For many years, the interior design industry has faced the problem of not knowing the product numbers of old or damaged wall coverings or flooring materials when replacing them.

[0006] Specifically, the person carrying out the reupholstering work would contact the manufacturer, and the manufacturer's staff would then search through the many samples in their sample books to find similar products to the current wall coverings, flooring, etc., but this process required a considerable amount of man-hours.

[0007] In addition, there was variation among the staff in which wall coverings, flooring materials, etc. were deemed similar, and there was a need to efficiently identify candidates for similar wall coverings, flooring materials, etc.

[0008] Furthermore, as it is difficult for manufacturers to secure the space to store all of their past sample books, there is a need to solve this problem through digital measures.

[0009] In response to the above-mentioned problems, it has been found that even if one attempts to search for similar images by using the techniques of Patent Documents 1 to 3 above and inputting photographs of current wall coverings, flooring materials, etc., it is often not possible to find appropriate similar images for the following reasons.

[0010] The first reason is that, for wall coverings and flooring materials, which often have simple patterns in light colors, the techniques described in Patent Documents 1 to 3 above are unable to properly extract the features of the input image, and there is a risk that images that are completely unsimilar to the input image will be searched for as similar images.

[0011] The second reason is that, due to the influence of lighting, photographs of current wall coverings, flooring materials, etc. may be taken in colors different from their actual colors, which could result in the search for similar images that are completely unsimilar to the input image.

[0012] Therefore, the present disclosure has been made in consideration of the above-mentioned problems, and aims to provide a similar image search device that can appropriately search for similar images even in photographs of wall coverings, flooring materials, etc., which often have simple patterns in light colors. [Means for solving the problem]

[0013] A first feature of the present disclosure is a similar image search device, comprising: a storage unit that stores a candidate image, a plurality of feature amounts extracted from the candidate image, and attribute information in association with each other; The present invention relates to a computer-readable storage device that includes an image acquisition unit that acquires an input image input by a user, a feature extraction unit that extracts the multiple feature amounts from the input image, an attribute information acquisition unit that acquires the attribute information input by the user, a search unit that searches for images similar to the input image from among the candidate images based on a similarity between the multiple feature amounts extracted from the input image and the multiple feature amounts extracted from the candidate images, and an output unit that selects a predetermined number of candidate images from the candidate images searched by the search unit based on the attribute information acquired by the attribute information acquisition unit and the attribute information stored in the memory unit, and outputs the selected candidate images, wherein the attribute information includes information indicating a color and a pattern. Effect of the Invention

[0014] According to the present disclosure, it is possible to provide a similar image search device that can appropriately search for similar images even for photos of wall coverings, flooring materials, and the like, which often have simple patterns in light colors. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram showing an example of functional blocks of a similar image search device 1 according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of a similar image search screen displayed by the output unit 16 of the similar image search device 1 according to an embodiment. [Diagram 3] FIG. 3 is a diagram showing an example of a similar image search screen displayed by the output unit 16 of the similar image search device 1 according to an embodiment. [Figure 4] FIG. 4 is a diagram showing an example of a similar image search screen displayed by the output unit 16 of the similar image search device 1 according to an embodiment. [Diagram 5] FIG. 5 is a diagram showing an example of a screen displaying search results of similar images displayed by the output unit 16 of the similar image search device 1 according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] A preferred embodiment of the present invention will be described in detail below with reference to the accompanying drawings. In the following description of the drawings, the same or similar parts are given the same or similar reference numerals. However, it should be noted that the drawings are schematic and the ratios of the dimensions are different from the actual ones. Therefore, the specific dimensions should be determined with reference to the following description. In addition, the drawings may include parts with different dimensional relationships and ratios. In this specification and the drawings, elements having substantially the same functions and configurations are given the same reference numerals to avoid repeated explanations, and elements not directly related to the present invention are not illustrated.

[0017] (First embodiment) Hereinafter, the similar image search device 1 according to the first embodiment of the present invention will be described with reference to Fig. 1 to Fig. 5. Fig. 1 is a diagram showing an example of functional blocks of the similar image search device 1 according to this embodiment, Fig. 2 to Fig. 4 are diagrams showing an example of a search screen for similar images displayed by the output unit 16 of the similar image search device 1 according to this embodiment, and Fig. 5 is a diagram showing an example of a screen displaying search results for similar images displayed by the output unit 16 of the similar image search device 1 according to this embodiment.

[0018] The similar image search device 1 according to this embodiment can search for images similar to an arbitrary input image.

[0019] For example, the similar image search device 1 can search for images similar to an input image, which is a photograph of an interior product such as a wall covering, a flooring material, or a curtain. Here, the wall covering includes decorative sheets, decorative films, wallpaper, ceiling materials, and the like.

[0020] In the following embodiment, an example will be described in which a photograph of a wall covering is used as an input image to search for similar images of wallpaper; however, the present invention is not limited to this case and can be applied to cases in which similar images of any input image are searched for.

[0021] As shown in FIG. 1, the similar image search device 1 according to this embodiment includes a memory unit 11, an image acquisition unit 12, a feature extraction unit 13, an attribute information acquisition unit 14, a search unit 15, and an output unit 16.

[0022] The storage unit 11 stores the candidate images, a plurality of feature amounts extracted from the candidate images, and attribute information in association with each other. Here, the attribute information may include information indicating the type, information indicating the color, information indicating the pattern (texture), sample book identification information (sample book name and sample book publication number), information indicating the start date of publication in the sample book, information indicating the end date of publication in the sample book, information indicating the thickness of the interior product, information indicating the price of the interior product, etc. Note that the attribute information may include any information recorded in the product database of interior products.

[0023] Here, the plurality of feature amounts may be a predetermined feature vector defined by any method, for example, the predetermined feature vector may include feature amounts of color, pattern, material, etc.

[0024] The image acquiring unit 12 acquires an input image input by a user. For example, the image acquiring unit 12 can be operated by an input device such as a keyboard or a mouse, and may acquire an input image input by a user via such an input device. Alternatively, the image acquiring unit 12 may acquire an input image input by a user from a user terminal 2 (for example, a smartphone or a PC) via a network.

[0025] The feature extraction unit 13 extracts the above-mentioned multiple feature amounts (for example, a predetermined feature vector) from the input image acquired by the image acquisition unit 12.

[0026] The attribute information acquisition unit 14 acquires attribute information input by a user. For example, the attribute information acquisition unit 14 may be operated by an input device such as a keyboard or a mouse, and may acquire attribute information input by a user via such an input device. Alternatively, the attribute information acquisition unit 14 may acquire the attribute information input by a user from the user terminal 2 via a network.

[0027] The search unit 15 searches the candidate images for an image similar to the input image, based on the similarity between a plurality of feature amounts extracted from the input image and a plurality of feature amounts extracted from the candidate images.

[0028] For example, the search unit 15 may calculate the absolute value (distance) of the difference between a predetermined feature vector extracted from the input image and a predetermined feature vector extracted from a candidate image as the above-mentioned similarity, and search for candidate images whose absolute value of such difference is less than or equal to a predetermined value.

[0029] The search unit 15 may search for a similar image to a partial image region A (see FIG. 4 described later) selected by a user from the input image acquired by the image acquisition unit 12.

[0030] The output unit 16 selects a predetermined number of candidate images from the candidate images searched by the search unit 15 based on the attribute information acquired by the attribute information acquisition unit 14 and the attribute information stored in the memory unit 11, and outputs the selected candidate images.

[0031] For example, the output unit 16 may output (display) the selected candidate image to an output device such as a display (see FIG. 5 described later). Alternatively, the output unit 16 may output (transmit) the selected candidate image to the user terminal 2 via a network.

[0032] Hereinafter, with reference to Figs. 2 to 4, an example of screen transition when searching for an image similar to an input image using the similar image search device 1 will be described.

[0033] First, on the screen shown in FIG. 2, the user clicks the “Select File” button, selects the desired input image (RE53096_C01.jpg in the example of FIG. 2), and clicks the “Next” button, which transitions to the screen shown in FIG. 3.

[0034] Here, the user may input attribute information in the pull-down menus of "Type," "Color," and "Texture" on the screen shown in FIG.

[0035] For example, "Type" is a pull-down menu for inputting attribute information indicating the type of input image (e.g., wallpaper, etc.), "Color" is a pull-down menu for inputting attribute information indicating the color of the input image, and "Texture" is a pull-down menu for inputting attribute information indicating the pattern of the input image.

[0036] Secondly, the screen shown in Fig. 3 displays the input image selected by the user on the screen shown in Fig. 2, and if the input image is the desired input image, the user clicks the "Search" button. On the other hand, if the input image is not the desired input image, the user clicks the "Reselect image / Search" button to reselect the input image on the screen shown in Fig. 2.

[0037] Here, as shown in Figure 4, the user selects image area A, which is a part of the input image displayed on the screen shown in Figure 3, and clicks the ``Search'' button to search for images similar to that image area A.

[0038] With this configuration, for example, when performing a similar search using an image of wallpaper at home photographed with a mobile device as an input image, even if part of the wallpaper is dirty or scratched, it is possible to search for suitable similar images to the input image by searching for similar images in the image area excluding the dirty, scratched, etc.

[0039] Thirdly, on the screen shown in FIG. 5, the output unit 16 displays the image selected as the similar image to the above-mentioned input image.

[0040] As described above, according to this embodiment, by including in the attribute information information indicating the color of the input image and information indicating the pattern of the input image, even when the input image is a photograph of a wall covering having a simple pattern in a light color, it is possible to avoid a situation in which an image that is completely unsimilar to the input image is searched for.

[0041] Furthermore, according to this embodiment, even if a photograph of current wall coverings, flooring materials, etc. is taken in a color different from the actual color due to the influence of lighting, the user can input information indicating the color of the input image as attribute information, thereby avoiding a situation in which an image that is completely dissimilar to the input image is searched for as a similar image.

[0042] Furthermore, according to this embodiment, by including in the attribute information information indicating at least one of the publication start date and the publication end date of the input image in the sample book, when a photo of an interior item is used as the input image, it is possible to avoid outputting an image of an interior item published in a sample book that will be discontinued in the near future (or an interior product that was published in a sample book that is already discontinued). Note that the same interior item may be published in multiple different sample books.

[0043] Here, one sample book may be discontinued but the other may continue to feature interior goods, and it is possible to avoid outputting images of interior goods featured in the former sample book and output images of interior goods featured in the latter sample book.

[0044] In addition, it is preferable that the wall covering has a certain thickness so that the unevenness of the old base is not noticeable when the wall covering is replaced. In this regard, according to the present embodiment, by including information indicating the thickness of the interior goods in the attribute information, it is possible to output only photos of wall coverings that can make the unevenness of the old base less noticeable as similar images.

[0045] Furthermore, according to this embodiment, by including information indicating the price of the input image in the attribute information, it is possible to output only similar images of interior goods in a desired price range.

[0046] In addition, since old wall coverings, flooring materials, etc. are often scratched or dirty, there are cases where a photograph of the wall coverings, flooring materials, etc. is taken in a state different from the actual color or pattern. In this regard, according to the present embodiment, even in such a case, it is possible to avoid a situation where an image that is completely dissimilar to the input image is searched for as a similar image.

[0047] Although the present invention has been described in detail using the above-mentioned embodiment, it is clear to those skilled in the art that the present invention is not limited to the embodiment described in the present specification. The present invention can be implemented in modified and altered forms without departing from the spirit and scope of the present invention defined by the description of the claims. Therefore, the description in this specification is intended to be illustrative and does not have any limiting meaning on the present invention. [Explanation of symbols]

[0048] 1. Similar image search device 11...Storage section 12...Image acquisition section 13...Feature extraction unit 14…Attribute information acquisition 15…Search section 16...Output section 2. User terminal

Claims

1. A storage unit that stores candidate images of wall coverings or flooring materials, a plurality of feature amounts extracted from the candidate images, and attribute information including information indicating colors and patterns, in association with each other; an image acquisition unit that acquires an input image of a wall covering or flooring material input by a user; a feature extraction unit that extracts the plurality of feature amounts from the input image; an attribute information acquisition unit that acquires the attribute information input by the user; a search unit that searches the candidate images for an image similar to the input image based on a similarity between the plurality of feature amounts extracted from the input image and the plurality of feature amounts extracted from the candidate images; an output unit that selects a predetermined number of the candidate images from the candidate images searched by the search unit based on the attribute information acquired by the attribute information acquisition unit and the attribute information stored in the storage unit, and outputs the selected candidate images; The search unit searches for a similar image to a partial image area selected by the user from the input image acquired by the image acquisition unit.

2. The similar image search device according to claim 1 , wherein the attribute information includes information indicating at least one of a publication start date in a sample book and a publication end date in the sample book.

3. The similar image search device according to claim 1 , wherein the attribute information includes information indicating a thickness.

4. The similar image search device according to claim 1 , wherein the attribute information includes information indicating a price.

Images