Search system and search method

Abstract

To provide a retrieval system capable of retrieving an object which a user wants to retrieve according to image data of the object even when the user does not know the name of the object.SOLUTION: A retrieval system comprises: an accepting section that accepts information related to a category of a commodity item from a terminal; an extracting section that extracts one or more commodity items belonging to a category indicated by the accepted information related to the category; and a display control section that allows a display section of the terminal to display image data of an individual commodity item included in the extracted one or more commodity items. The display control section extracts image data of an individual combination commodity item included in one or more combination commodity items to be combined with the extracted one or more commodity items or a commodity item belonging to the same category as the accepted category, based on respective feature data of one or more image data included in a predetermined image medium, and allows the display section to display the extracted image data.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a search system that supports a user in selecting and determining an object to be searched for when searching for an object via the Internet. and search method This application claims priority based on Japanese Patent Application No. 2008-205730 filed in Japan on August 8, 2008, and Japanese Patent Application No. 2008-262035 filed in Japan on October 8, 2008, the contents of which are incorporated herein by reference.

Background Art

[0002] Conventionally, when a user searches for an object (e.g., a product, etc.) and then purchases it through a network such as the Internet, a system that supports the user in selecting and determining the object to be searched for has been used (see, for example, Patent Document 1). For example, a user uses various search engines on the Internet to input the name of an object (e.g., a product name, etc.) or the characteristics of the object, and searches for the corresponding object. Further, if necessary, the user searches for stores that sell the above-mentioned corresponding object. Then, a server device having the above-described search engine presents an image of an object corresponding to the user's preference or its price to the user's terminal via the Internet in the searched store, or supports the user's search for an object by providing related objects, etc.

[0003] In recent years, when buying products, it has become more common to purchase products from virtual stores in virtual malls on the Internet without going to retail stores that conduct face-to-face sales. As a result, even in areas where there are no large retail stores in the vicinity, it has become possible to purchase a favorite product from a plurality of products of many types while staying at home. (See, for example, Patent Document 2).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

[0005] A search support system according to an aspect of the present invention is a search system connected to a user's terminal via a network, including an object information including image data regarding an object, a combination information indicating a combination of the objects, and a user information including identification information for each user stored in a database, and a search unit that extracts other objects to be combined with the specified object from the combination information stored in the database based on the image data of the object specified by the user, wherein information including the image data regarding the other extracted objects is transmitted to the terminal via the network, the user information includes history information regarding at least one of search, recommendation, and purchase of the object, the history information includes together with the information indicating the purchased object information indicating that is the search or the recommendation has been performed the object that has not been purchase and and further includes a control unit that acquires information to be provided to the user based on the history information and transmits the information to be provided to the terminal via the network. Further, a search method according to an aspect of the present invention is a search method performed by a search system connected to a user's terminal via a network, the search system having a database storing object information including image data regarding an object, combination information indicating a combination of the objects, and user information including identification information for each user, the user information including history information regarding at least one of search, recommendation, and purchase of the object, The history information includes the information indicating the purchased object and the information indicating the object for which the search or recommendation was made but not purchased. The search method includes: the search system extracting, from the combination information stored in the database, other objects to be combined with the specified object based on the image data of the object specified by the user; the search system transmitting, via the network, information including the image data related to the extracted other objects to the terminal; and the search system obtaining information to be provided to the user based on the history information and transmitting the information to be provided to the terminal via the network.

Brief Description of the Drawings

[0006] [Figure 1] It is a block diagram showing a configuration example of a search support system according to an embodiment of the present invention. [Figure 2] It is a conceptual diagram showing a configuration example of a product table stored in the database 16 of FIG. 1. [Figure 3] It is a conceptual diagram showing a configuration example of a user registration table stored in the database 16 of FIG. 1. [Figure 4] It is a flowchart showing an operation example of product search in a search support system according to an embodiment. [Figure 5] It is a flowchart showing an operation example of collecting preference information in a search support system according to an embodiment. [Figure 6] It is a block diagram showing a modification example of a configuration example of a search support system in the present embodiment. [Figure 7] It is a block diagram showing a configuration example of a product search support system according to an embodiment of the present invention. [Figure 8] It is a table recording the attribute information of the combined product items in category 1 (shirt) of the product item database 26 in FIG. 7. [Figure 9] It is a table recording the attribute information of the combined product items in category 2 (trousers) of the product item database 26 in FIG. 7. [Figure 10] It is a table recording the attribute information of the combined product items in category 3 (jacket) of the product item database 26 in FIG. 7. [Figure 11] It is a table showing the correspondence between the sold product items in category 1 and the combined product items in category 1 collected from similar image media. [Figure 12] It is a table showing the correspondence between the sold product items in category 2 and the combined product items in category 2 collected from similar image media. [Figure 13] It is a table showing the correspondence between the sold product items in category 3 and the combined product items in category 3 collected from similar image media. [Figure 14] It is a table showing the combination between each category of the combined product items collected from the image media in the combination information database 28 in FIG. 7. [Figure 15] It is a table recording the attribute information of the combined product items in category 1 (shirt) of the combined product image information database 29 in FIG. 7. [Figure 16] It is a table recording the attribute information of the combined product items in category 2 (trousers) of the combined product image information database 29 in FIG. 7. [Figure 17] It is a table recording the attribute information of the combined product items in category 3 (jacket) of the combined product image information database 29 in FIG. 7. [Figure 18] It is a flowchart showing an operation example of the product search support system in FIG. 7. [Figure 19] It is a conceptual diagram showing the configuration of the user table in the history database 31 in FIG. 7. [Figure 20] It is a conceptual diagram showing the configuration of the purchase history table in the history database 31 in FIG. 7. [Figure 21] It is a conceptual diagram explaining the segmentation of image data by the Graph-Cut method.

Embodiments for Carrying Out the Invention

[0007] In the conventional examples described in Patent Document 1 and Patent Document 2 mentioned above, when a user searches for an object, the user inputs text data indicating the name of the object or the like, or selects from images of the object prepared in advance for searching. For example, when the user only has information about the image of the object and does not know the name of the object, it has been found that there is a problem that the user cannot search for the object that the user actually wants to search for or an object similar to the object to be searched. Therefore, an aspect according to the present invention provides a search support system, a search support method, and a search support program that can search for an object that a user wants to search for based on image data of the object even if the user does not know the name of the object. <First Embodiment> Hereinafter, a search support system according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration example of the search support system according to the embodiment. In this figure, the search support system has a search server 1 and a user terminal 2 connected to the search server 1 via an information communication network I such as the Internet. Here, the user terminal 2 is a terminal possessed by the user and is identified by user identification information unique to each user. Hereinafter, in this embodiment, as an example of an object that the user wants to search for, the search and purchase of fashion products, clothes, shoes, necklaces, hats, and other products will be described. Note that in this embodiment, it is not limited to products such as fashion products, clothes, shoes, necklaces, and hats, and it is also applicable to objects that can be searched via the Internet or a network, such as electrical appliances, furniture, and paintings. In addition, the object in this embodiment includes, as an example, products, articles, electrical appliances, furniture, paintings, buildings including stores and companies, plants, animals, real estate (including the exterior, interior, floor plan, etc. of a condominium), scenery, and the like.

[0008] The search server 1 is a server that assists the user in searching for products in stores on the Internet, etc., and has an extraction unit 11, a type determination unit 12, a search unit 13, a combination extraction unit 14, a preference extraction unit 15, a database 16, and a transmission / reception unit 17. As shown in FIG. 2, in the database 16, corresponding to the product identification information that identifies each product, there are the name of the product, the product image data (object image data) that is the image data of the product, the feature information extracted from the product image data, the store information of the store selling the product, the price of the product, the combination information of other products combined with the product, and the type identification information indicating the type of the product (product genres such as clothes, shoes, hats, necklaces, etc.) are stored in association with each other in a product table. In this database 16, for example, information on products from stores that have registered as members in the search support system is sequentially accumulated. The above-mentioned feature information is obtained by converting elements such as color, shape, pattern, and texture into numerical data (for example, a vector in a dimension corresponding to the number of elements). Also, when directly comparing the images of the product that the user wants to search for and the product images (object images) stored in the database 16, the type determination unit 12 may be omitted.

[0009] The above-mentioned extraction unit 11 inputs the input image data (image data provided by the user) transmitted from the user terminal 2 via the transmission / reception unit 17, and extracts the search part in the input image data. More specifically, the extraction unit 11 extracts the contour (search part) of the product image area in the input image data and generates contour image data (image data of the search part). Here, in the extraction process of the contour image data, the contour is a part where the density value of the image changes rapidly, and a differential operation is performed to extract it. However, in a digital image, since the data is arranged intermittently at regular intervals, the differential is approximated by an operation (difference) that takes the difference between adjacent pixels, and the part where the pixel density changes rapidly is extracted as the contour.

[0010] The type determination unit 12 searches for template image data corresponding to the contour image data from the template image data in the contour data table of products stored in the database 16 in advance from the above-mentioned contour image data, and reads the type identification information set in the database 16 corresponding to the template image data. It is desirable that this template image data is stored in advance in the above-mentioned contour data table of the database 16 as a template for photographing a large number of products of various types from multiple angles and comparing them with the contour.

[0011] The search unit 13 extracts the features of the search image data obtained by performing contour extraction on the input image data from the storage area of the type corresponding to the above-mentioned type identification information, and searches for the obtained feature information and the feature information with high coincidence or similarity from the image data having the type identification information corresponding to the determined type in the above-mentioned product table in the database 16, and extracts a preset number of images of products with high similarity in descending order of similarity. Here, when the type determination unit 12 is not provided, the search unit 13 compares the image of the product obtained by performing contour extraction on the input image data with the product image data in the entire product table, and extracts the product image data with high similarity to the image of the product on which the above-mentioned contour extraction is performed. In addition, the search unit 13 transmits the product information (object information) corresponding to the searched product image data to the user terminal 2 of the user. The above-mentioned product information is information including at least one of the name (product name), the sales store selling the product corresponding to the product image data, the URL of the store, the telephone number and address of the store, and the price of the product.

[0012] The preference extraction unit 15 randomly extracts a preset number of image data of other products (products of the same type as the product searched by the user) included in the type of the product searched by the user from the above-mentioned product table and transmits them to the user terminal 2. The user terminal 2 displays the image data transmitted from the search server 1 on a display screen (not shown). In addition, the preference extraction unit 15 inputs the determination results of like or dislike for each product (for example, product image data stored in the database 16) transmitted from the user terminal 2, and for each user, preference information including at least one of shape, color, pattern, and texture is written and recorded in the user registration table shown in FIG. 3 of the database 16.

[0013] Here, the user registration table stores purchase information (including purchased product identification information, purchase date and time, purchased store, purchased price, etc.) indicating what products have been purchased in the past corresponding to the user identification information, preference information extracted by the preference extraction unit 15, and point information indicating the number of points the user has. These points can be used in the same way as cash when paying for a purchased product. That is, when the discount coupon information is added to the above-mentioned store information in the search server 1 and the user purchases the product searched using the discount coupon information, the purchase information regarding the product is stored in the above-mentioned user registration table of the database 16. Then, the search server 1 adds the points corresponding to the price of the purchased product as the number of points in the point information to the target user in the above-mentioned user registration table. Note that the preference extraction unit 15 may extract preference information for each type from the above-mentioned purchase information. The combination extraction unit 14 searches for other types of products that are likely to be purchased in combination with the extracted product, for products corresponding to the combination information stored in advance corresponding to the preference information, and transmits them to the user's terminal.

[0014] Next, with reference to FIGS. 1 and 4, the operation of the search support system in this embodiment will be described. FIG. 4 is a flowchart for explaining an operation example of the search support system in this embodiment. In the following description, the case where the type determination unit 12 is provided will be described. If the type determination unit 12 is not provided, the processing of the type determination unit 12 will be omitted from the flowchart. The user reads an image of a model wearing a favorite piece of clothing from a magazine or the like using a color scanner or the like connected to the user terminal 2, or takes a picture with a digital camera, or downloads an image from the Internet (step S1). Then, the user inputs the read image, the captured image, or the downloaded image as input image data into the user terminal 2, and determines whether to select the area of the image of the product to be searched based on the presence or absence of multiple types of products in this input image data (step S2). Note that the above input image data may be stored in the user terminal 2 in advance, or may be stored in an external terminal.

[0015] At this time, for example, when trying to search for the product name and the store selling a skirt, if the input image data shows only the skirt, the process proceeds to step S4. On the other hand, when not only the image of the skirt part but also the whole model is photographed, the process proceeds to step S3. Then, the user marks the image of the skirt area part with a line or the like using an image processing tool (any tool that can draw a line on the input image data) installed in the user terminal 2 in the input image data to select the image (step S3), and then proceeds to step 4.

[0016] The user terminal 2 transmits the input image data (including the data for which the image has been selected) input by the user to the search server 1 via the information communication network I as a search request signal together with its own user identification information (step S4).

[0017] Note that in step S2 described above, the search server 1 may determine the presence or absence of multiple types of products for the input image data received from the user terminal 2, and transmit the determined types to the user terminal 2. In this case, in step S3, the type to be searched is selected from the types for the input image data received from the search server 1, and the process proceeds to step S4.

[0018] Next, when the search server 1 receives the above search request signal from the user terminal 2, it outputs the above input image data to the extraction unit 11. The extraction unit 11 performs contour extraction (extraction of the search part) of the product image in the input input image data. At this time, if no area is marked in the input image data, the extraction unit 11 extracts the contour image data of the entire input image data. On the other hand, if there is a marked area part, it extracts the contour image data of the image of that area part. Then, the type determination unit 12 searches the contour data table corresponding to the types of products stored in the database 16 in advance from the above contour image data for the template image data corresponding to the contour image data, and reads the type identification information corresponding to the template image data from the contour data table. (Step S5).

[0019] The search unit 13 performs feature extraction of the image data (search image data) to be searched in the input image data corresponding to the inside of the contour in the above contour image data, and searches the product table in the database 16 for product image data similar to the feature information (for example, each information based on color and shape) of the search image data. (Step S6). At this time, the search unit 13 calculates the distance of the vector in each element of the feature information (for example, if the feature information is information based on color, blue, red, yellow, etc.), and calculates the similarity of whether this distance matches, whether this distance is close or far. Note that those with a close distance are considered to have a high similarity. In addition, the search unit 13 extracts a preset number of products from the above product table in descending order of similarity.

[0020] Next, the search unit 13 transmits product information (information including at least one of the product name, the price of the product, the store selling the product image data, the URL, the telephone number of the store, and the address) corresponding to the searched product image data to the user's user terminal 2 via the transmission / reception unit 17 (step S7). Here, as an example, when the search request signal is transmitted from the user terminal 2, the transmission / reception unit 17 reads the address on the network of the user terminal 2 added at that time, and transmits the above product information to this address. Then, the user terminal 2 displays the product name, image data, store, and price information of the search result transmitted from the product sales search server 1 on the display screen. In step S5 described above, the search server 1 may extract the product image data to be searched from the database 16 based on the input image data included in the search request signal only when it is detected that the user identification information included in the search request signal is registered in the user registration table in the database 16. Further, when the search server 1 detects that the user identification information included in the search request signal is not registered in the user registration table, for example, it may transmit to the user terminal 2 that user registration based on the user identification information is required.

[0021] Thereby, the user can obtain information such as the product name of the product he / she likes or a product similar to that product, the store information indicating the store selling the product, and the price of the product by transmitting the image data.

[0022] Here, discount coupon information is added to the store information. When purchasing the searched product using this discount coupon information, the product sales search server 1 charges the target store an amount corresponding to the price of the product.

[0023] Next, with reference to FIGS. 1 and 5, a process will be described in which the search server 1 according to the present embodiment extracts user preference information and presents other products that can be combined with the above-described searched products based on the preference information. FIG. 5 is a flowchart for explaining an operation example of collecting user preference information and introducing other products based on this preference information. The preference extraction unit 15 extracts a preset number of shoes from the product table of the same type as the product searched by the user. For example, if the product searched by the user is shoes (step S11), and transmits the product image data and product identification information thereof (step S12). At this time, the products of each type to be transmitted to the user are characteristic products with different colors, shapes, patterns, etc., and are set as products that enable a fashion coordinator to extract the user's preference information.

[0024] Then, when the user terminal 2 receives the product image data of the product for preference extraction from the search server 1, it sequentially displays the product image data (display image) of the product on the display screen. At this time, if the user likes the product of the product image data displayed on the above display screen, the user clicks the "Good button" with an input means such as a mouse. On the other hand, if the user does not like the product of the product image data displayed, the user clicks the "No button" with a mouse or the like to make a selection (step S13). When the "Good button" or the "No button" is selected, the user terminal 2 displays the product image data of the next product on the display screen, and these processes are continued until the user's approval or disapproval judgment selection is completed for all or a predetermined number of the received product image data.

[0025] When the approval or disapproval judgment on the above-described product image data is completed, the user terminal 2 transmits the judgment result information in which the approval or disapproval judgment data of the product is associated with each product identification information to the search server 1, adding its own user identification information (step S14). When the determination result information is input, the transmission / reception unit 17 determines whether the added user identification information is registered in the user registration table. If it is registered, the determination result information is output to the preference extraction unit 15.

[0026] Next, when the determination result information is input, the preference extraction unit 15 selects preference information corresponding to the pass / fail determination pattern from the preference information table in the database 16. Here, the pass / fail determination pattern is a data pattern of a pass determination and a fail determination corresponding to product identification information arranged in a predetermined order. Then, the preference extraction unit 15 writes and stores the obtained preference information in the user registration table in the database 16 corresponding to the user identification information (step S15).

[0027] Next, the combination extraction unit 14 selects combination information of products corresponding to the above-mentioned preference information from the combination information table in the database 16. In this combination information table, for example, when preference information is extracted for shoes, combination information corresponding to clothes, bags, hats, etc. corresponding to the preference is selected. Then, the combination extraction unit 14 extracts product image data such as clothes, bags, hats, etc. that match or are similar to the above combination information (step S16), and for the model, in accordance with the shoes, the selected clothes, bags, hats, etc. are worn. The wearing image data is, when there is only one combination, singular, or when there are multiple combinations, multiple types. For each combination, combination identification information is added and transmitted to the user terminal 2 (step S17).

[0028] When receiving the wearing image data, the user terminal 2 displays this wearing image data on the display screen. Thereby, the user can obtain information on the fashion combination corresponding to his / her preference for the products already searched or purchased. Here, when the user makes a pass / fail judgment as described above, the user terminal 2 associates the pass / fail judgment data for each combination identification information, adds its own user identification information as judgment result information, and transmits it to the search server 1.

[0029] Then, when the judgment result information is input, the transmission / reception unit 17 determines whether the added user identification information is registered in the user registration table. If it is registered, the judgment result information is output to the preference extraction unit 15. The preference extraction unit 15 generates combination preference information according to a pass / fail judgment pattern corresponding to the input combination identification information. This combination preference information corresponds to the shape, color, pattern, texture, etc. of shoes, and consists of the combination of the shape, color, pattern, texture of other clothes, bags, hats, etc. that have received a positive judgment. Each time the user purchases a product, it will be learned, and the accuracy of the combination preference information will be improved. That is, next, if the user purchases a bag based on this combined wearing image data, the preference information for the bag is retrieved, and the combination preference information for shoes, clothes, hats, etc. corresponding to this preference information can be obtained. Therefore, the preferences of each user will be gradually narrowed down.

[0030] Also, as a challenge combination (a combination that is not usually selected by the user or a combination different from the above-mentioned preference information), several products corresponding to the combination information that is opposite to the combination information selected from the preference information by the fashion coordinator are randomly selected from the product table, and the selected products may be inserted into some of the combinations when generating the wearing image data. As a result, the user's preferences become polarized, and there is a possibility that the desire to purchase will increase more.

[0031] Also, as shown in FIG. 6, the search support system in this embodiment may be configured to include a user terminal 2a having an extraction unit 11a that has the same extraction function as the above-described extraction unit 11, and a search server 1a excluding the extraction unit 11. In this case, based on the input image data acquired by the user terminal 2a or the input image data stored in the user terminal 2a, the extraction unit 11a extracts the search part (for example, the product part) of the input image data, and only transmits the image data corresponding to the search part to the search server 1a as the input image data. Also, since the parts other than the extraction unit 11a are the same as those in the above-described embodiment, the description thereof is omitted. In this way, the user only needs to transmit to the search server 1a only the image data corresponding to the search part among the input image data as the image data to be transmitted. For example, there is no need to transmit the image data corresponding to the user's face part or the face part of another person to the search server 1a.

[0032] In addition, in this embodiment, the search support system of the present invention can be applied not only to products such as clothes and shoes but also to the above-described objects. Also, for example, the search support system of the present invention can be applied to services provided in accommodation facilities (such as hotels and inns) and restaurants. For example, in the above-described accommodation facility, the search support system of the present invention can be applied when selecting a combination of room layouts, a combination of window views, a combination of a room or its layout and a window view, and the like. Also, for example, in the above-described restaurant or the like, the search support system of the present invention can be applied when selecting a combination of restaurant interiors, a combination of window views, a combination of music, a combination of an interior and a window view, a combination of an interior and music, a combination of a window view and music, and the like.

[0033] <Second Embodiment> Hereinafter, a product search support system according to an embodiment of the present invention will be described with reference to the drawings. Conventionally, when purchasing a combination product (such as clothing, combination furniture, combination household appliances, etc.) that combines multiple different types (categories) of products, if you do not actually go to a retail store, you cannot directly ask a store clerk which combination is appropriate. Also, when selecting products that match well when combined with the products the user has purchased or will purchase from the vast array of products in the virtual store, it is impossible to confirm one by one with the coordinators of the combination products, and since the user cannot make the selection themselves, they will not be able to make a purchase.

[0034] For example, in the case of clothing, even if a user wants to select pants or a shirt that matches the jacket they are purchasing, they cannot confirm with a top-notch fashion coordinator, worry about selecting an impossible combination, and end up giving up on purchasing matching pants or shirts. Here, clothing refers to all the clothes, accessories (such as ornaments, bags, shoes, hats, etc.) that a person wears on their natural body.

[0035] This embodiment also solves such problems. In the purchase of combination products, for the products that the user is purchasing or has purchased, products that match well when combined are extracted from the product group, and the extracted products are recommended to the user to assist the user in purchasing combination products.

[0036] FIG. 7 is a block diagram showing a configuration example of a product search support system according to the present embodiment. The present invention provides, when purchasing a sales product item in any category in a product item combination consisting of sales product items in a plurality of categories, sales product items in the remaining categories in the product item combination considered to be a good combination with the purchased sales product item as recommended products, so as to assist the user in easily selecting, from a large number of sales product items, sales product items considered to be a good combination with the sales product item to be purchased. In the following description, all products in clothing (products in categories such as blazers, shirts, and trousers) are regarded as product items, the product items actually sold by the virtual store are regarded as sales product items, and each product item combined in the image medium described later is described as a combined product item.

[0037] In this figure, the product search support system is composed of a product search support device 100 and one or more user terminals 200. The product search support device 100 includes a control unit 21, a similar item search unit 22, a combination search unit 23, a product search unit 24, a transmission / reception unit 25, a product item database 26, a product item image database 27, a combination information database 28, a combined product image information database 29, and a combined product image database 30. The user terminal 200 is, for example, a personal computer equipped with an Internet browser provided in each home of each user. The product search support device 100 and the user terminal 200 are connected via an information communication network I including the Internet.

[0038] The product item database 26 has a storage area with a plurality of table configurations for each type of product item. That is, as shown in FIG. 8, the product item database 26 has a storage area with a table configuration in which the attribute information of the sales product items being sold is stored for each of the identification information A1 to An of the product items of shirts as category 1. Similarly, as shown in FIG. 9, the merchandise item database 26 has a storage area with a table structure in which the attribute information of the corresponding sold merchandise item is stored for each identification information B1 to Bn of the pants merchandise item as category 2. Similarly, as shown in FIG. 10, the merchandise item database 26 has a storage area with a table structure in which the attribute information of the corresponding sold merchandise item is stored for each identification information C1 to Cn of the jacket merchandise item as category 3. Furthermore, although not shown, the merchandise item database 26 has a storage area that stores a plurality of tables such as shoes, blouses, and bags as other categories of clothing in addition to the above-described categories. In each table of the merchandise item database 26, a column for similar merchandise items is provided as one of the attribute information, and the merchandise items in the combined merchandise image information database 29 that are the same as or similar to the sold merchandise item are associated and described as groups of similar merchandise items for each sold merchandise item as shown in FIGS. 11 (category 1 - shirt), 12 (category 2 - pants), and 13 (category 3 - jacket), respectively.

[0039] Here, the attribute information includes the identification information of the combined merchandise items (merchandise items in the combined merchandise image information database 29 described later) associated as the same or similar, the feature data of the image extracted from the image data of the sold merchandise item, the sales information such as the price and brand of the sold merchandise item, and the like. The types and acquisition methods of the feature data of the image will be described later. The merchandise item image database 27 stores the image data for each sold merchandise item in each category stored in the merchandise item database 26 corresponding to the identification information of each of the above-mentioned sold merchandise items.

[0040] The combination information database 28 uses, as combined product items, the clothing products worn in combination by models, etc. in fashion magazines, fashion catalogs, and image media on the Internet, i.e., each of the product items worn in combination, and stores combinations of the combined product items and the combined product items combined with them, along with their respective corresponding relationships, by associating them with respective identification numbers. For example, in a fashion magazine, if a model is wearing product items of a shirt with identification information a1-1, pants with identification information b1-2, and a jacket with identification information c1-7, as shown in the first row of the combination information database 28 in FIG. 14, the combined product item of the combination of identification information a1-1, identification information b1-2, and identification information c1-7 will be stored in association as a set. This combination information represents combinations of product items created by fashion designers as combinations, or combinations of product items created by fashion designers combined by fashion coordinators, as combinations of combined product items. Therefore, each combined product item has a refined combination, and with this combination, when worn, there is a high possibility that people who see the fashion will accept it naturally without feeling any discomfort.

[0041] The combined product image information database 29 stores, by category, combined product items in clothing obtained from image media such as fashion magazines, fashion catalogs, and designers' collection information on the Internet (materials or devices presenting images such as magazine photos and illustrations). Here, the combined product image information database 29 is configured, for example, with the table structures shown in FIGS. 15, 16, and 17. Figure 15 shows the accumulation of shirts as combined product items in Category 1, Figure 16 shows the accumulation of pants as combined product items in Category 2, and Figure 17 shows the accumulation of jackets as combined product items in Category 3. In the combined product image information database 29, identification information is assigned to the combined product items, and for each identification information, the attribute information of the corresponding combined product item is stored. This attribute information includes, for example, the identification information of similar sold product items, the feature data of the image extracted from the image data of the combined product item, the sales information such as the price and brand of the combined product item, and so on. The types and acquisition methods of the feature data will be described later. Also, in each table of the combined product image information database 29, a column for similar product items is provided as one of the attribute information, and the product items in the product item database 26 that are the same as or similar to the combined product item are described in association as shown in Figure 15 (Category 1 - shirt), Figure 16 (Category 2 - pants), and Figure 17 (Category 3 - jacket).

[0042] In the combined product image information database 29, in the table of Figure 15, the identification information a1-1 to a1-k is regarded as the combined product item that is the same as or similar to the sold product item A1, the identification information a2-1 to a2-k is regarded as the combined product item that is the same as or similar to the sold product item A2, and in order, the identification information an-1 to an-m is stored as the combined product item that is the same as or similar to the sold product item An. Similarly, in the combined product image information database 29, in the table of Figure 16, the identification information b1-1 to b1-m is regarded as the combined product item that is the same as or similar to the sold product item B1, the identification information b2-1 to b2-r is regarded as the combined product item that is the same as or similar to the sold product item B2, and in order, the identification information bn-1 to bn-m is stored as the combined product item that is the same as or similar to the sold product item Bn. Similarly, in the combined product image information database 29, in the table of FIG. 17, the identification information c1-1 to c1-q is used as a combined product item identical or similar to the sold product item C1, the identification information c2-1 to c2-k is used as a combined product item identical or similar to the sold product item B2, and in order, the identification information cn-1 to cn-m is stored as a combined product item identical or similar to the sold product item Cn. The combined product image database 30 stores the image data for each combined product item in each category stored in the combined product image information database 29, corresponding to the identification information of each of the above combined product items.

[0043] In addition, the above-described product item database 26, product item image database 27, combination information database 28, combined product image information database 29, and combined product image database 30 may be classified by gender, age group, fashion trend (for example, casual, conservative, older brother style, adult style, street style, Ura-Hara (Harajuku) style, mode style for men, while for women, gal style, older sister style, conservative style, teen style, celeb style, etc.). When classified in this way, before performing the coordination process, it is necessary to obtain from the user the gender, age (such as teens, twenties, thirties, etc.), fashion trend, etc. as classification information.

[0044] The control unit 21 is activated when the virtual store it manages is accessed from the user terminal 200, obtains classification information such as gender, age, and fashion trend, and transmits to the user terminal 200 a processing program (operated by a browser of the user terminal, etc.) for performing display and selection operations on the combined items described later. Here, the control unit 21 and the user terminal 200 perform transmission and reception of each data via the information communication network I and the transmission and reception unit 25. The user terminal 200 activates the above processing program with an internal browser and performs data transmission and reception with the product search support device 100 in the process of selecting an image display or a sold product item or a combined product item. In addition, when the control unit 21 receives response data for the above screen data from the user terminal 200, it selects a table of any one product item from a plurality of product item databases 26 corresponding to the gender, age, and fashion trends, and transmits category information indicating the category of the combined product item, for example, character information such as shirts, trousers, jackets, shoes, or image information, to the user terminal 200 via the transmission / reception unit 25 and the information communication network I.

[0045] In addition, when the control unit 21 receives category information indicating the type of category selected by the user, for example, category information indicating a shirt, transmitted from the user terminal 200, it selects the table shown in FIG. 8 with the shirt (category 1) as the product item database 26 as the category, reads the identification information A1 to An of the product items in this table, reads the image data of the product items up to the identification numbers A1 to An from the product item image database 27, converts it into thumbnail image data (reducing the number of bits of the image data), adds the identification number of the corresponding sales product item to each, and transmits it to the user terminal 200. In addition, when the identification number of the sales product item selected for purchase by the user from the thumbnail image is input, the control unit 21 performs an order receiving process for the order and transmits the input identification number to the similar item search unit 22. Here, the order receiving process includes charging the user terminal 200 the price corresponding to the price described in the attribute information, checking the inventory of the ordered sales product item, and procedures for delivering the sales product item to the address input by the user.

[0046] The similar item search unit 22 searches for a plurality of combined product items in which the identification number matching the identification number of the input sales product item is described in the column of similar product items from the table (category 1) of the combined product image information database 29, extracts the combined product items identical or similar to the above sales product item as similar product items, and outputs them to the combination search unit 23. The combination search unit 23 searches the combination information database 28 for product items in different categories, for example, pants (category 2), corresponding to each of the similar product items with identification numbers input from the similar item search unit 22, extracts the corresponding combined product items, and outputs the identification information of the combination as the second combined product item to the control unit 21. In addition, the control unit 21 reads out, from the combined product image database 30 corresponding to the identification information of a plurality of combinations of the identification information of the similar product items input from the combination search unit 23 and the identification information of the second combined product item, the image data corresponding to each identification information, transmits it to the user terminal 200, and when the user inputs the identification information of the combined product items in the selected combination, outputs the received identification information to the product search unit 24.

[0047] The product search unit 24 searches the product item database 26 for the actually sold sales product items corresponding to the identification information of the second combined product items input, and uses them as recommended sales product items in different categories to be combined with the sales product items purchased by the user. It reads out the image data of this recommended sales product item from the product item image database 27 based on the identification information and transmits it to the user terminal 200. In addition, the product search unit 24 compares the amount of the recommended sales product item with the amount of the sales product item purchased by the user, determines whether it is within the set range (for example, within 0.5 to 2 times the amount of the purchased combined product), and if it is less than or equal to the purchase amount, transmits it to the user terminal 200, and if it is outside the range, it may not transmit the recommended product.

[0048] Next, the operation of the product search support system according to this embodiment will be described with reference to FIGS. 7 and 18. FIG. 18 is a flowchart showing an operation example of the product search support system in FIG. 7. Hereinafter, the product input by the user will be described as a product item, the product actually sold in the virtual store as a sales product item (product item database 26), and the product extracted from the image medium used when selecting a combination of products as a combined product item (combined product image information database 29). When the user terminal 200 accesses a virtual store managed by the product search support device 100 via the information communication network I by the user's operation, the control unit 21 transmits to the user terminal 200 information for acquiring gender, age, and fashion trends, and a processing program for displaying a combination item described later and performing a selection operation on the displayed image. In the user terminal 200, an image display processing unit and a selection processing unit are provided in an Internet browser by the above processing program, and the image data transmitted from the product search support device 100 is displayed, and editing processing and selection processing of the image data are performed.

[0049] Then, when the control unit 21 receives answer data (for example, data selected from a plurality of respective options displayed on the screen) for the gender, age, and fashion trends input to the above screen data from the user terminal 200, the control unit 21 selects a corresponding table from a plurality of tables in the product item database 26 corresponding to the gender, age, and fashion trends, and transmits category information indicating the category of the sold product items, for example, character information such as shirts, trousers, jackets, shoes, or image information, to the user terminal 200 via the transmission / reception unit 25 and the information communication network I. When character information or image information is input, the user terminal 200 displays a category (for example, shirts, trousers, jackets, etc.) based on the character information or image information on the display unit according to the above processing program, and performs a display (for example, "please select") prompting the user to select which one to purchase. When the user selects any category, the user terminal 200 transmits category information indicating the category selected by the user, for example, category information indicating a shirt, to the product search support device 100 (step F1).

[0050] That is, when receiving the category information of the product item that the user wants to purchase, the control unit 21 selects the table shown in FIG. 8 with shirts as the category (category 1) from the tables of the product item database 26 shown in FIGS. 8 to 10, reads all of the identification information A1 to An of the sold product items in this table, reads the image data of the sold product items up to the identification numbers A1 to An from the product item image database 27, adds the identification numbers of the sold product items corresponding to each thumbnail image as the image data of the thumbnail image, and transmits it to the user terminal 200. When the image data of the thumbnail image is input, the user terminal 200 displays the thumbnail images of the sold product items with the identification numbers A1 to An on the display unit (step F2). Then, when the user selects any of the displayed thumbnail images, the user terminal 200 transmits the identification information of the selected thumbnail image to the product search support device 100.

[0051] When receiving the identification information of the selected thumbnail image, the control unit 21 performs an order receiving process for the sold product item with this identification number, that is, a shirt, as the sold product item selected by the user as the purchase target from the thumbnail image, and transmits the input identification number to the similar item search unit 22. Then, for example, when the identification number A1 of the sold product item purchased by the user is input, the similar item search unit 22 extracts from the tables of the combined product image information database 29 shown in FIGS. 15 to 17 as a candidate group consisting of one or a plurality of combined product items that match or are similar to this identification number A1 (step F3). Here, for example, when the identification number A1 of the sold product item is input from the combined product image information database 29, the similar item search unit 22 extracts the combined product items with the identification numbers a1-1 to a1-q in which the identification number A1 is described in the column of the similar product item in the attribute data as a candidate group of the similar combined product items, and outputs it to the combination search unit 23.

[0052] The combination search unit 23 searches, from the combination information database 28 shown in FIG. 14, for different categories corresponding to each of the combination product items of the identification numbers in the candidate group input from the similar item search unit 22, for example, combination product items of pants, and extracts the corresponding combination product items of pants (for example, b1-2, b2-3, b1-4,... corresponding to a1-1, a1-2, a1-3,... shown in FIG. 14), and outputs, as the second combination product items, the respective identification information in the combination to the control unit 21 (step F4). Here, as the category recommended for the sales product item to be combined with the sales product item purchased by the user, it may be preset corresponding to the purchased product, or the user may be first allowed to select from a plurality of categories. Then, the control unit 21 reads, from the combination product image database 30 corresponding to each identification information, the image data corresponding to each of the plurality (the number of combination product items corresponding to the candidate group) of combinations of the identification information of the combination product items input from the combination search unit 23 and the identification information of the second combination product items, adds the respective identification information to the image data of each combination product item, and transmits it to the user terminal 200.

[0053] When the identification information of the combination product items (corresponding to shirts), the identification information of the second combination product items (corresponding to pants), and the image data corresponding thereto are input, the user terminal 200 displays the image data of each combination on the display unit (step F5). Also, a plurality of three-dimensional human images corresponding to each body type created by CG (Computer Graphics) are displayed at the end of the display screen. By selecting any one of them, the user terminal 200 overlays and displays the shirt and pants of the combination product items on the human image. Further, the user terminal 200 may perform image processing to capture the face of the user itself with the web camera mounted by the user and overlay the selected face area on the face portion of the CG human image for display.

[0054] Next, when the user selects any one from the plurality of combinations displayed on the display unit (step F6), for example, when selecting the combination of identification information a1-1 and identification information b1-2, the user terminal 200 transmits the identification information b1-2 of the second combined product item (corresponding to the pants) in the combination selected by the user to the product search support device 100. When the identification information b1-2 of the second combined product item is input via the control unit 21, the product search unit 24 searches the table shown in FIG. 9 corresponding to the category of pants in the product item database 26 (step F7), and extracts the sales product item which is the identification information B1 corresponding to this identification information b1-2 (step F8). Then, the control unit 21 searches for and reads out the image data corresponding to the identification information B1 of the combined sales product item extracted by the product search unit 24 from the product item image database 27. In addition, the control unit 21 searches for and reads out the image data corresponding to the identification information A1 purchased by the user from the product item image database 27, and transmits it together with the image data of the sales product item of the identification information B1 to the user terminal 200.

[0055] When the image data of the sales product item is input, the user terminal 200 displays the combined image data on the display unit (step F9). At this time, similar to step F5 described above, a plurality of three-dimensional human images corresponding to each body type created by CG are displayed at the end of the display screen, and by selecting any one of them, the user terminal 200 superimposes and displays the shirt (identification information A1) and pants (identification information B1) of the sales product item on the human image. Also, the user terminal 200 may capture the user's face with the mounted web camera and superimpose and display the selected face area on the face part of the human image. Then, when the user terminal 200 selects whether to purchase or not purchase the recommended product of the sales product item with the identification information B1 from the options on the display screen (step F10), it transmits the identification information A1 and the identification information B1 to the product search support device 100 together with the information indicating whether to purchase or not purchase. When the information indicating whether to purchase or not purchase, the identification information A1, and the identification information B1 are input, the control unit 21 performs an order receiving process in the case of purchasing, in the same manner as in the case of the sales product item with the identification information A1.

[0056] Also, a history database 31 shown by a broken line in FIG. 7 may be provided in the product search support device 100. The history database 31 is composed of a user table having the table configuration shown in FIG. 19 and a purchase history table having the table configuration shown in FIG. 20 for each user. The user table in FIG. 19 is configured by associating user identification information for identifying each user, which is assigned to each user who has registered as a member or purchased a sales product item, with at least the name of the user and the user's email address corresponding to this user identification number. In addition, the purchase history table in FIG. 20 is provided for each user, is identified by the above user identification information, corresponds to the access date and time when the virtual store was accessed, and stores as a history the identification information of the sales product item purchased at the access date and time, the identification information of the sales product item that was searched but not purchased, and the identification information of the combined product item that was recommended but not purchased for the purchased sales product item. If there is no purchased sales product item, "-" is stored.

[0057] For example, when the user accesses the virtual store from the user terminal 200 and registers as a member by inputting the name and email address on the member registration page, the control unit 21 assigns identification information and adds it to the user table for registration. Then, the control unit 21 searches for sales product items in the category selected by the user. When the user purchases the sales product item, the purchased sales product item is stored in the area of the identification information of the purchased product item. On the other hand, when the search is made but the item is not purchased, the identification information of the searched sales product item is stored in the area of the identification information of the product item that was searched for but not purchased. Also, when the control unit 21 purchases a sales product item as a recommended product to be combined with the above-mentioned purchased sales product item for the user, in the above purchase history table, the purchased sales product item is stored in the area of the identification information of the purchased product item. On the other hand, when the sales product item as the above-mentioned recommended product is not purchased, it is stored in the area of the identification information of the product item that was recommended but not purchased.

[0058] Then, when the user accesses the virtual store again and searches for a sales product item, the control unit 21 selects the user's purchase history table from the purchase history database 31 based on the user identification information input by the user, and searches the selected purchase history table based on the identification information of this sales product item. At this time, when the identification information of the sales product item searched by the user is detected in the area of the identification information of the purchased product item, the date and time of purchase of the same sales product item is added and transmitted to the user terminal 200, and is displayed on the display screen to notify the user of the combined merchandise item that has already been purchased.

[0059] Also, when the identification information of the sales product item being searched is detected in the area of the identification information of the product item that was searched for but not purchased, the date and time of the search for the same sales product item is added and transmitted to the user terminal 200, and is displayed on the display screen to notify the user of information that stimulates the desire to purchase, such as "This is the product item that was also searched for before. Isn't it your favorite type?" In addition, when the identification information of the sales product item being searched is detected in the area of the identification information of the product item that was recommended but not purchased, the control unit 21 adds the date and time when the same sales product item was recommended and the image data of the sales product item purchased at that time, and transmits it to the user terminal 200, and displays it on the display screen to notify the user of information that stimulates the purchase desire, such as "This is a product item that goes well when combined with the product item you purchased before."

[0060] Next, the feature data will be described. As described in FIGS. 8 to 10 and FIGS. 15 to 17, the feature data is, for example, the result of performing a two-dimensional Fourier transform on the pattern of the fabric of clothing for each color space of R (red), G (green), and B (blue). The control unit 21 generates the element data RD, GD, and BD of this feature data by two-dimensional Fourier transform. At this time, if the sales product item or the combined product item is a shirt or a jacket, the width of the clothes or the shoulder width is used as the reference value for the length when performing the two-dimensional Fourier transform, so as to ensure data consistency in the processing between each sales product item and the combined product item. Also, if it is pants, the width of the waist part is used as the reference value for the length when performing the two-dimensional Fourier transform, so as to ensure data consistency in the processing between each sales and combined product item. That is, in order to confirm the similarity by the feature data, and to determine the personality such as the size of the pattern, it is necessary to standardize the dimensions of each part of the whole with the dimension of any position of the ornament as the reference value, so that the two-dimensional Fourier transform results of the combined products within the same category can be made consistent. When imaging the image data for performing the two-dimensional Fourier transform, shirts, jackets, pants, etc. are placed on a flat floor, flattened, and then imaged with a digital camera or the like. Also, the shape as the element data of the feature data is, for example, the ratio of the sleeve length to the shoulder width if it is a shirt or a jacket, and the ratio of the crotch width to the hem width if it is pants. Next, for the texture, expand the fabric part with the largest area and perform a two-dimensional Fourier transform on the shape of the fabric. At this time, keep the magnification constant and ensure the consistency of the two-dimensional Fourier transform data obtained from the image data between different combined items.

[0061] As described above, the clerk in the virtual store collects feature data from the image data of the sold merchandise items by the control unit 21, and also collects attribute data from the image data of the combined merchandise items collected from image media such as fashion catalogs or the Internet. Then, for the combined merchandise items with similarity, perform clustering on the combined merchandise items collected from the image media, using the feature data of the sold merchandise items as the centroid data of the block. Here, for example, the control unit 21 obtains the distance between the comparison feature vector composed of the feature data of the image data of each combined merchandise item collected from the image media input by the clerk and the reference feature vector composed of the feature data of the image data of each sold merchandise item actually sold, and performs processing to regard the combined merchandise item having a feature vector closer in distance to the reference feature vector of each sold merchandise item and each other combined merchandise item as a block similar to the sold merchandise item considered to be close, and generates the correspondence relationship between the combined merchandise items in the tables of FIGS. 15, 16, and 17 and the sold merchandise items in the columns of the similar merchandise items. Similarly, generate the correspondence relationship between the combined merchandise items in the columns of the sold merchandise items and the similar merchandise items in the tables of FIGS. 8, 9, and 10. That is, the identification information described in the column of the similar merchandise items in the tables of FIGS. 8, 9, and 10 is the identification information of the combined merchandise items that are closer in distance, that is, have similarity, compared to other sold merchandise items for the sold merchandise item corresponding to the identification information. Here, the control unit 21 may, for example, store the similar product items in the columns of each of FIGS. 8, 9, and 10 in ascending order of distance, that is, in descending order of similarity. Thus, when selecting the combined product item most similar to the sold product item, the similar item search unit 22 can more easily extract the most similar combined product item or the top h items (h is preset) from the combined product items collected from the image medium, starting from FIGS. 15 to 17.

[0062] Also, as described above, instead of performing clustering in advance by comparing the reference feature vectors of each sold product item with the feature data of the combined product items collected from the image medium having comparison feature vectors at a short distance and providing a column for describing the identification information of the similar product items, the similarity may be calculated each time when searching for similar combined product items. For example, in the search for the combined product items obtained from the image medium similar to the sold product item in step F3, the similar item search unit 22 may calculate the similarity (the shorter the distance, the higher the similarity) based on the distance between the above-mentioned reference feature vector and the comparison feature vector, and extract the combined product items collected from the image medium having similarity with the sold product item. At this time, similarly, in the search for the sold product items similar to the combined product items obtained from the image medium in step F7, the product search unit 24 may calculate the similarity based on the distance between the above-mentioned reference feature vector and the comparison feature vector, and extract the sold product items having similarity with the combined product items collected from the image medium. Furthermore, the similar item search unit 22 and the product search unit 24 may calculate the distance between the sold product items and the combined product items collected from the image medium, and extract the one with the highest similarity or the top h items with the highest similarity. As described above, according to the present embodiment, in the purchase of a combination product, for the product that the user purchases or has purchased, the product that matches when combined is extracted from the group of products being sold, and the extracted product is recommended to the user. Therefore, when selecting a matching product from many products in a virtual store on the Internet, there is no need for the user to worry about whether it matches or not by themselves, and the combination product can be easily selected.

[0063] <Third Embodiment> In the second embodiment, the product search support device 100 transmits the sales product items to the user terminal 200 by means of thumbnail images and allows the user to select them on the screen of the user terminal 200. In the third embodiment, after the user accesses the virtual store, the image data of the clothing that the user wants to purchase, which is read from an image medium by a scanner or the like or downloaded from the Internet, is input to the user terminal 200 as the image data of the product item. The user terminal 200 transmits this image data to the product search support device 100, and the product search support device 100 may extract the sales product items similar to the combined product items of this image data. The operation after extracting the sales product items is the same as the operation after the user selects the sales product items from the thumbnail images in the second embodiment.

[0064] The configuration of the product search support device 100 according to the third embodiment is the same as that of the second embodiment. Hereinafter, only the operations different from those of the second embodiment will be described. When the user accesses the virtual store using the user terminal 200, the product search support device 100 is activated, and the control unit 21 sends the image information of an input screen (search by clicking on the options with a mouse or the like) that asks whether to select the sales product items from the thumbnail images on the user terminal 200 or to extract the similar sales product items from the product item database 26 based on the image data of the clothing input by the user to the user terminal 200. Then, when the user decides to select a product to purchase from the thumbnail images and a response signal indicating the selection by the thumbnail images is input from the user terminal 200, the control unit 21, as in the second embodiment, adds identification information of the sales product items to the respective image data and transmits thumbnail images of a plurality of sales product items to the user terminal 200. After the user selects one of the thumbnail images thereafter, the same processing as in the second embodiment is performed.

[0065] On the other hand, when the user selects to extract similar sales product items from the product item database 26 based on the image data input by the user, the user terminal 200 transmits a response signal indicating selection by the image data input by the user to the product search support device 100. Thereby, the control unit 21 transmits information on the input screen for inputting the image data to the user terminal 200. The user causes the user terminal 200 to read image data of clothing to be purchased (image media on paper such as fashion magazines or fashion catalogs) by a scanner, or images the clothing with a digital camera and reads the captured image data, or reads image data obtained via the Internet into the user terminal 200. When the image data is read, this image data is displayed in the image data display area of the input screen on the user terminal 200 as shown in FIG. 21. The user selects the clothing part of the image data that the user wants to purchase, for example, the shirt part to be extracted, with a dashed line H1, selects the part outside the shirt part to be extracted (the part near the outer periphery of the shirt) with a dashed line 2, selects a category name with a combo box, and when the user clicks the send button on the display screen, the user terminal 200 detects that a process of transmitting the image data is requested, and transmits the image data of the clothing with the dashed line H1 and the dashed line H2 drawn, the image data of the clothing without the lines drawn, and category information indicating the category name of the selected clothing to the product search support device 100.

[0066] When receiving the image data of the clothing with the dashed lines H1 and H2 drawn thereon and the image data of the clothing without the dashed lines drawn thereon, the control unit 21 performs a segmentation process on the above-mentioned image data of the clothing by the Graph-Cut method to segment the shirt part and the other parts, and extracts the shirt part. That is, the control unit 21 obtains the boundary between the region having the same pixel value as the pixels on the dashed line H1 drawn on the shirt part and the region having the same pixel value as the pixels on the dashed line H2 drawn outside the shirt part as the position where the error of the error function based on the gradation degree is minimized, thereby performing segmentation. Here, when the control unit 21 extracts the shirt part, as described above, it performs two-dimensional Fourier transform and extracts the shape data as an element in the feature data, and outputs it as the detection target feature data.

[0067] Then, the control unit 21 selects a table corresponding to the category of the extracted clothing. For example, in this embodiment, it selects the shirt table from the product item database 26. After selecting the shirt table, the control unit 21 calculates the distance between the reference feature vector composed of the feature data of each sold product item in the table and the target feature vector composed of the above-mentioned detection target feature data, and extracts the sold product items having feature data similar to the detection target feature data from the above-mentioned shirt table up to the top h items similar, for example, up to the top 5 items. Here, since it is difficult for the control unit 21 to extract the texture from the image data transmitted from the user in the feature data, when generating the feature vector from the feature data, the texture is excluded from the elements of the vector, and the distance between the two feature vectors to be compared is calculated. Next, the control unit 21 reads out the image data of each sold product item from the product item image database 27 according to the identification information of these top 5 items, and transmits them to the user terminal 200 as thumbnail images corresponding to the identification information respectively. The subsequent processing is the same as the processing after transmitting the thumbnail image in step F2 in the flowchart of FIG. 18 in the second embodiment.

[0068] <Fourth Embodiment> <In the third embodiment, a sales product item that is the same as or similar to the image data of the product item input by the user is searched for from the product item database 26 that stores the sales product items being sold, the searched sales product item is presented to the user, and a process of recommending a sales product item of another category to be combined with the searched sales product item is performed.> <In the fourth embodiment, an operation of searching for a sales product item to be combined with the clothing already owned by the user is performed.>

[0069] <The user takes a picture of his or her own clothing, for example, a shirt, with a digital camera, and causes the user terminal 200 to read the image data of the shirt from the digital camera.> <Then, when the user accesses the virtual store using the user terminal 200, the product search support device 100 is activated, and the control unit 21 selects a sales product item from the thumbnail images on the user terminal 200, or extracts similar sales product items from the product item database 26 based on the image data of the clothing input by the user, or recommends a sales product item to be combined with the clothing of the image data input by the user. The control unit 21 transmits the image information of the input screen (search by clicking on the options with a mouse or the like) to the user terminal 200.>

[0070] <Then, when a response signal indicating the selection by the thumbnail image is input from the user terminal 200 by the user's decision to select the product to be purchased from the thumbnail images, the control unit 21, as in the second embodiment, adds the identification information of the sales product item to each image data and transmits the thumbnail images of a plurality of sales product items to the user terminal 200.> <After the user selects any one from the thumbnail images thereafter, the same processing as in the second embodiment is performed.> On the one hand, when the user selects to extract the sales product items from the product item database 26 based on the image data input by the user himself / herself, the user terminal 200 transmits a response signal indicating selection based on the image data input by the user to the product search support device 100. The subsequent processing is the same as that in the third embodiment.

[0071] Also, when the user selects to recommend sales product items in other categories that are combined with the clothing of the image data input by the user himself / herself, the user terminal 200 transmits a response signal indicating that it has selected to recommend sales product items combined with the clothing of the image data input by the user to the product search support device 100. Hereinafter, the process of recommending sales product items combined with the clothing of the image data input by the user according to the fourth embodiment will be described. The configuration of the product search support device 100 according to the fourth embodiment is the same as that in the second embodiment. Only the operations different from those in the first and third embodiments will be described below.

[0072] As described above, when the user selects to recommend sales product items combined with the clothing of the image data input by the user himself / herself, the user terminal 200 transmits a response signal indicating that it recommends sales product items combined with the clothing of the image data input by the user to the product search support device 100. Thereby, the control unit 21 transmits the information of the input screen for inputting the image data to the user terminal 200. The user captures the image data of the clothing (for example, a shirt) he / she owns for which he / she wants to recommend combined sales product items with an imaging device such as a digital camera, and causes the user terminal 200 to read the image data of the captured clothing from this imaging device.

[0073] When the image data read by the user terminal 200 is displayed, the user selects and inputs the category of the clothing in the image data displayed in the image data display area from the combo box in the category input field near the image data display area. When the user selects the send button on the input screen using a pointing device such as a mouse, the user terminal 200 detects that a process of transmitting image data is requested, and transmits the image data displayed in the image data display area to the product search support device 100 together with category information indicating the above category. When image data is input, the control unit 21 performs a two-dimensional Fourier transform on the image data of clothing, for example, the image data of a shirt, and extracts shape data as an element in the feature data, and outputs it to the similar item search unit 22 together with category information as detection target feature data.

[0074] Then, the similar item search unit 22 selects a table corresponding to the category of the extracted clothing, for example, a shirt table, from the combined product image information database 29. After selecting the shirt table, the similar item search unit 22 calculates the distance between the reference feature vector composed of the feature data of each combined product item in the table and the target feature vector composed of the detection target feature data, and selects a combined product item having feature data similar to the detection target feature data from the shirt table. Extract the top h items, for example, the top 5 items. Here, since it is difficult for the control unit 21 to extract the texture from the image data transmitted from the user in the feature data, when generating the feature vector from the feature data, the texture is removed from the elements of the vector to generate the reference feature vector and the detection target feature vector.

[0075] Then, the similar item search unit 22 outputs the identification information of the top 5 combined product items with the highest detected similarity, for example, 5 shirts, to the combination search unit 23. When the identification information of the combined product item is input, the combination search unit 23 reads out the identification information of other categories, for example, the combined item of pants, stored corresponding to each of the five identification information of the input combined product item in the combination information database 28. Next, the combination search unit 23 transmits to the control unit 21 the combinations of the identification information for each of the five combinations of a shirt and trousers. The subsequent processing is the same as the processing after step F5 in the flowchart of FIG. 18.

[0076] <Fifth Embodiment> In the combined product image information database 29, it may be possible to store, in association with the identification information, the attribute data of the image data of old clothing collected from old fashion magazines or fashion catalogs in the past, such as 10 years ago or 20 years ago. Also, in the combined product image database 30, the image data of old clothing is stored corresponding to the above identification signal. Then, corresponding to the combinations of combined product items in past fashion magazines or fashion catalogs, etc., a table of combinations of combined product items between different categories shown in FIG. 14 is created in the combination information database 28. As described above, by configuring the combined product image information database, the combined product image database 30, and the combination information database 28, when new sales product items similar to past designs are sold, it becomes possible to easily extract the sales product items to be combined into new combined product items similar to past designs from the sales product items currently being sold, using the combinations in the past designs. In this embodiment, clothing has been described as an example of the product item, but it can be easily applied to all general products that are combined products combining a plurality of different types (categories) of products, such as combined furniture and combined household appliances.

[0077] In each of the above-described embodiments, a program for realizing the functions of each part of the search servers 1 and 1a and the product search support device 100 may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed to perform product search support processing. Here, the "computer system" is assumed to include hardware such as an OS and peripheral devices. Also, the "computer system" is assumed to include a WWW system equipped with a homepage providing environment (or display environment). Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built into a computer system. Furthermore, the "computer-readable recording medium" also includes a volatile memory (RAM) inside a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and that holds the program for a certain period of time.

[0078] Also, the above program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the "transmission medium" for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication wire) such as a telephone line. Also, the above program may be for realizing a part of the above-described functions. Furthermore, it may be a so-called difference file (difference program) that can realize the above-described functions in combination with a program already recorded in a computer system. As described above, according to the aspect of the present invention, it is possible to easily search for an object that a user wants to search for or an object similar to the object.

Industrial Applicability

[0079] The present invention is suitable for use in a search support system that assists a user in selecting and determining an object to be searched for when searching for an object via the Internet, and in similar technologies, and can perform a search for an object that the user wants to search for based on image data of the object.

Explanation of Signs

[0080] 1…Search server 2…User terminal 11…Extraction unit 12…Type determination unit 13…Search unit 14…Combination extraction unit 15…Preference extraction unit 16…Database 17…Transmission / reception unit 18…Transmission / reception unit I…Information communication network 100…Product search support device 200…User terminal 21…Control unit 22…Similar item search unit 23…Combination search unit 24…Product search unit 25…Transmission / reception unit 26…Product item database 27…Product item image database 28…Combination information database 29…Combination product image information database 30…Combination product image database 31…History database

Claims

1. A search system connected to a user's terminal via a network, a database in which object information including image data relating to objects, combination information indicating combinations of the objects, and user information including identification information for each user are stored; a search unit that extracts, based on the image data of the object designated by the user, another object to be combined with the designated object from the combination information stored in the database; Equipped with The extracted information including the image data regarding the other objects is transmitted to the terminal via the network; The user information includes history information regarding at least one of searching, recommending, and purchasing the object; the history information includes information indicating the object that has been purchased, and information indicating the object that has been searched for or recommended but not purchased; the search system further includes a control unit that acquires information to be provided to the user based on the history information and transmits the information to be provided to the terminal via the network; Search system.

2. 2. The search system according to claim 1, If the specified object has been purchased in the past, the provided information includes the image data of the purchased object.

3. 2. The search system according to claim 1, If the specified object has been recommended in the past but not purchased, the provided information includes a notification that the specified object is a good match with the object purchased in the past.

4. 2. The search system according to claim 1, The information on the other object includes the image data of the specified object and the image data obtained by combining the image data of the other object.

5. 2. The search system according to claim 1, The object information includes at least a name in addition to the image data.

6. 2. The search system according to claim 1, The other objects include objects that are in a different category from the designated object.

7. The search system according to any one of claims 1 to 6, The object includes at least one of a commodity, an article, an electrical appliance, furniture, a painting, a building, a plant, an animal, real estate, and a landscape; The merchandise includes at least one of fashion merchandise, clothes, shoes, necklaces, and hats; The buildings include stores and companies.

8. The search system according to claim 7, The search unit extracts information about the specified object from the object information, The extracted information about the object is transmitted to the terminal via the network.

9. 9. The search system according to claim 8, The image data of the specified object is at least one of an image read by a scanner, an image captured by a camera, and an image downloaded from the Internet.

10. A search method performed by a search system connected to a user's terminal via a network, comprising: The search system has a database in which object information including image data relating to objects, combination information indicating combinations of the objects, and user information including identification information for each user are stored; The user information includes history information regarding at least one of searching, recommending, and purchasing the object; the history information includes information indicating the object that has been purchased, and information indicating the object that has been searched for or recommended but not purchased; The search method includes: The search system extracts, based on the image data of the object designated by the user, other objects to be combined with the designated object from the combination information stored in the database; The search system transmits information including the image data related to the extracted other objects to the terminal via the network; the search system acquires information to be provided to the user based on the history information, and transmits the information to be provided to the user via the network to the terminal; Search methods including.

Images