Retrieval system

Abstract

To provide a retrieval system capable of retrieving an object a user wishes to retrieve, through image data of the object even if the user does not know a name of the object.SOLUTION: A retrieval system includes: a database storing object information including image data regarding an object, combination information showing a combination of objects with each other, and user information; and a retrieval part configured to extract, from the combination information, the other object to be combined to a designated object according to the image data of the designated object. Information on image data regarding the extracted other object is transmitted to a terminal. The user information includes history information regarding at least one of retrieval, recommendation, and purchase of an object. The history information includes information showing an object having been purchased and information showing an object having been retrieved or recommended but not having been purchased. The retrieval system acquires provision information to a user according to the history information while transmitting the provision information to the terminal.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a search system that assists a user in selecting and determining an object to be searched for when searching for an object via the Internet. This application claims priority based on Japanese Patent Application No. 2008-205730 filed in Japan on August 8, 2008, and Japanese Patent Application No. 2008-262035 filed in Japan on October 8, 2008, the contents of which are incorporated herein by reference.

Background Art

[0002] Conventionally, when a user searches for an object (e.g., a product, etc.) and then purchases it through a network such as the Internet, a system that assists the user in selecting and determining the object to be searched for has been used (see, for example, Patent Document 1). For example, the user uses various search engines on the Internet to input the name of the object (e.g., product name, etc.) or the characteristics of the object, and searches for the corresponding object. Further, if necessary, the user searches for the stores that sell the above-mentioned corresponding objects. Then, a server device having the above search engine presents an image of the object corresponding to the user's preference or its price to the user's terminal via the Internet in the searched store, or provides assistance to the user's search for the object by providing related objects, etc.

[0003] In recent years, when buying products, it has become more common to purchase products from virtual stores in virtual malls on the Internet without going to retail stores that conduct face-to-face sales. As a result, even in areas where there are no large retail stores in the neighborhood, it has become possible to purchase products that the user likes from a plurality of products of many types while staying at home. (See, for example, Patent Document 2).

Prior Art Documents

Patent Documents

[0004] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 2004-246585 [Patent Document 2] Japanese Unexamined Patent Application Publication No. 2002-150138 [Summary of the Invention]

[0005] A search system according to an aspect of the present invention is a search system connected to a user's terminal via a network, comprising a database storing object information including image data regarding an object, combination information indicating combinations of the objects, and user information including identification information for each user; and a search unit configured to extract, from the combination information stored in the database, other objects to be combined with the specified object based on the image data of the object specified by the user, wherein information including the image data regarding the other extracted objects is transmitted to the terminal via the network, the user information includes history information regarding at least one of search, recommendation, and purchase of the object, the history information includes information indicating the purchased object and information indicating the object that has been searched or recommended but not purchased, and the search system further comprises a control unit configured to acquire information to be provided to the user based on the history information and transmit the information to be provided to the terminal via the network. [Brief Description of the Drawings]

[0006]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Embodiments for Carrying Out the Invention

[0007] In the conventional examples described in the above Patent Document 1 and Patent Document 2, when a user searches for an object, the user inputs text data indicating the name of the object or selects from the images of the objects prepared in advance for searching. For example, when the user only has information about the image of the object and does not know the name of the object, it has been found that there is a problem that the user cannot search for the object that the user actually wants to search for or an object similar to the object to be searched. Therefore, an aspect according to the present invention provides a search support system, a search support method, and a search support program that can search for an object that a user wants to search for based on the image data of the object even if the user does not know the name of the object. <First Embodiment> Hereinafter, a search support system according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration example of the search support system according to the embodiment. In this figure, the search support system has a search server 1 and a user terminal 2 connected to the search server 1 via an information communication network I such as the Internet. Here, the user terminal 2 is a terminal possessed by the user and is identified by user identification information unique to each user. Hereinafter, in this embodiment, taking the search and purchase of fashion products, clothes, shoes, necklaces, hats, etc. as examples of the objects that the user wants to search for. Note that in this embodiment, it is not limited to fashion products, clothes, shoes, necklaces, hats, etc., and it can also be applied to objects that can be searched via the Internet or a network, such as electrical appliances, furniture, paintings, etc. Also, the objects in this embodiment include, as an example, products, goods, electrical appliances, furniture, paintings, buildings including stores and companies, plants, animals, real estate (including the exterior, interior, floor plan, etc. of a condominium), scenery, and the like.

[0008] The search server 1 is a server that assists the user in searching for products in online stores and the like on the Internet, and has an extraction unit 11, a type determination unit 12, a search unit 13, a combination extraction unit 14, a preference extraction unit 15, a database 16, and a transmission / reception unit 17. As shown in FIG. 2, in the database 16, corresponding to the product identification information for identifying each product, there are the name of the product, the product image data (object image data) which is the image data of the product, the feature information extracted from the product image data, the store information selling this product, the price of this product, the combination information with other products combined with this product, and the type identification information indicating the type of the product (product genres such as clothes, shoes, hats, necklaces, etc.) are associated and stored in a product table. In this database 16, for example, information on products from stores that have registered as members in the search support system is sequentially accumulated. The above-mentioned feature information is obtained by converting elements such as color, shape, pattern, texture, etc. into numerical data (for example, a vector in a dimension corresponding to the number of elements). Also, when directly comparing the images of the product that the user wants to search for and the product images (object images) stored in the database 16, the type determination unit 12 may be omitted.

[0009] The extraction unit 11 inputs the input image data (image data provided by the user) transmitted from the user terminal 2 via the transmission / reception unit 17, and extracts the search part within the input image data. More specifically, the extraction unit 11 extracts the contour (search part) of the image area of the product within the input image data, and generates contour image data (image data of the search part). Here, in the extraction process of the contour image data, the contour is a part where the density value of the image changes abruptly, and a differential operation is performed to extract this. However, in a digital image, since the data is arranged in a discontinuous manner at regular intervals, the differential is approximated by an operation (difference) that takes the difference between adjacent pixels, and the part where the density of the pixels changes abruptly is extracted as the contour.

[0010] The type determination unit 12 searches the template image data in the contour data table of the products stored in the database 16 in advance from the above-mentioned contour image data, and reads the type identification information set in the database 16 corresponding to the template image data. It is desirable that this template image data is stored in the above-mentioned contour data table of the database 16 in advance as a template for photographing a large number of products of various types from multiple angles and comparing them with the contour.

[0011] The search unit 13 performs feature extraction of the search image data for which contour extraction has been performed in the input image data from the storage area of the type corresponding to the above-mentioned type identification information, and searches for the obtained feature information and image data having feature information with a high degree of coincidence or similarity from the image data having the type identification information corresponding to the determined type in the product table in the database 16, and extracts a preset number of the image data of the products with a high degree of similarity in descending order of similarity. Here, when the type determination unit 12 is not provided, the search unit 13 compares the image of the product for which contour extraction has been performed in the input image data with the product image data in the entire product table, and extracts the product image data with a high degree of similarity to the image of the product for which the above-mentioned contour extraction has been performed. In addition, the search unit 13 transmits product information (object information) corresponding to the searched product image data to the user's user terminal 2. Note that the above-mentioned product information includes at least one of a name (product name), a store that sells the product corresponding to the product image data, the URL of the store, the phone number and address of the store, and the price of the product.

[0012] The preference extraction unit 15 randomly extracts from the product table a preset number of image data of other products (products of the same type as the product searched by the user) included in the type of product searched by the user, and transmits the data to the user terminal 2. The user terminal 2 displays the image data transmitted from the search server 1 on a display screen (not shown). In addition, the preference extraction unit 15 inputs the determination result of like or dislike for each product (for example, product image data stored in the database 16) transmitted from the user terminal 2, and writes and records for each user preference information including at least one of shape, color, pattern, and texture in the user registration table shown in FIG. 3 of the database 16.

[0013] Here, the user registration table stores purchase information (including purchased product identification information, purchase date and time, purchased store, purchased price, etc.) indicating what products have been purchased in the past corresponding to the user identification information, the preference information extracted by the preference extraction unit 15, and point information indicating the number of points the user has. These points can be used in the same way as cash when paying for the purchase of a product. That is, when discount coupon information is added to the above-mentioned store information in the search server 1 and the user purchases the product searched using the discount coupon information, the purchase information regarding the product is stored in the above-mentioned user registration table of the database 16. Then, the search server 1 adds the points corresponding to the price of the purchased product as the number of points in the point information to the target user in the user registration table. Note that the preference extraction unit 15 may extract preference information for each type from the above-mentioned purchase information. The combination extraction unit 14 searches for other types of products that are likely to be purchased in combination with the extracted product, among the products corresponding to the combination information stored in advance corresponding to the preference information, and transmits the result to the user's terminal.

[0014] Next, with reference to FIGS. 1 and 4, the operation of the search support system in the present embodiment will be described. FIG. 4 is a flowchart for explaining an operation example of the search support system in the present embodiment. In the following description, the case where the type determination unit 12 is provided will be described. If the type determination unit 12 is not provided, the processing of the type determination unit 12 will be omitted from the flowchart. The user reads an image of a model wearing a favorite piece of clothing from a magazine or the like using a color scanner or the like connected to the user terminal 2, or takes a picture with a digital camera, or downloads an image from the Internet (step S1). Then, the user inputs the read image, the taken image, or the downloaded image as input image data to the user terminal 2, and determines whether to select the area of the image of the product to be searched based on the presence or absence of multiple types of products in the input image data (step S2). Note that the above input image data may be stored in the user terminal 2 in advance, or may be stored in an external terminal.

[0015] At this time, for example, when trying to search for the product name and the store selling a skirt, if the input image data shows only the skirt, the process proceeds to step S4. On the other hand, when not only the image of the skirt part but also the whole model is photographed, the process proceeds to step S3. Then, the user marks the image of the skirt area part with a line or the like using an image processing tool installed in the user terminal 2 (any tool that can draw a line on the input image data) in the input image data to select the image (step S3), and then proceeds to step 4.

[0016] The user terminal 2 transmits the input image data (including the data for which image selection has been performed) input by the user, together with its own user identification information, to the search server 1 via the information communication network I as a search request signal (step S4).

[0017] In step S2 described above, the search server 1 may determine the presence or absence of a plurality of types of products for the input image data received from the user terminal 2, and transmit the determined types to the user terminal 2. In this case, in step S3, the types to be searched for are selected from the types for the input image data received from the search server 1, and the process proceeds to step S4.

[0018] Next, when the search server 1 receives the above search request signal from the user terminal 2, it outputs the above input image data to the extraction unit 11. The extraction unit 11 performs contour extraction (extraction of the search part) of the product image in the input input image data. At this time, when no area is marked in the input image data, the extraction unit 11 extracts the contour image data of the entire input image data. On the other hand, if there is a marked area part, it extracts the contour image data of the image of that area part. Then, the type determination unit 12 searches the contour data table corresponding to the types of products stored in the database 16 in advance for the template image data corresponding to the contour image data from the above contour image data, and reads the type identification information corresponding to the template image data from the contour data table. (Step S5).

[0019] The search unit 13 performs feature extraction of the image data (search image data) to be searched in the input image data corresponding to the inside of the contour in the above contour image data, and searches the product table in the database 16 for product image data similar to the feature information of the search image data (for example, each information based on color and shape). (Step S6). At this time, the search unit 13 calculates the vector distance for each element of the feature information (for example, if the feature information is information based on color, such as blue, red, or yellow), and calculates the similarity as to whether this distance matches, or whether this distance is close or far. Note that those with a closer distance are considered to have a higher similarity. Also, the search unit 13 extracts a preset number of products from the above product table in descending order of similarity.

[0020] Next, the search unit 13 transmits, via the transmission / reception unit 17, product information (information including at least one of the product name, product price, product image data sales store, URL, store phone number, and address) corresponding to the searched product image data to the user's user terminal 2 (step S7). Here, as an example, when the search request signal is transmitted from the user terminal 2, the transmission / reception unit 17 reads out the network address of the user terminal 2 added at that time, and transmits the above product information to this address. Then, the user terminal 2 displays on the display screen the product name, image data, store, and price information of the search results transmitted from the product sales search server 1. In step S5 described above, the search server 1 may extract the product image data to be searched from the database 16 based on the input image data included in the search request signal only when it is detected that the user identification information included in the search request signal is registered in the user registration table in the database 16. Also, when the search server 1 detects that the user identification information included in the search request signal is not registered in the user registration table, for example, it may transmit to the user terminal 2 a message indicating that user registration based on the user identification information is required.

[0021] Thereby, the user can obtain information such as the product name of a product they like or a product similar to that product, store information indicating the store selling the product, and the price of that product by transmitting the image data.

[0022] Here, discount coupon information is added to the store information. When a user purchases a searched product using this discount coupon information, the product sales search server 1 charges the target store an amount corresponding to the price of the product.

[0023] Next, with reference to FIGS. 1 and 5, a process will be described in which the search server 1 according to the present embodiment extracts user preference information and presents other products that can be combined with the above-mentioned searched products based on the preference information. FIG. 5 is a flowchart for explaining an operation example of collecting user preference information and introducing other products based on this preference information. The preference extraction unit 15 extracts a preset number of products of the same type as the product searched by the user. For example, when the product searched by the user is shoes, the preference extraction unit 15 extracts shoes from the product table (step S11) and transmits the product image data and product identification information thereof (step S12). At this time, the products of each type to be transmitted to the user are characteristic products with different colors, shapes, patterns, etc., and are set as products from which a fashion coordinator can extract the user's preference information.

[0024] Then, when the user terminal 2 receives the product image data of the product for preference extraction from the search server 1, the user terminal 2 sequentially displays the product image data (display image) of the product on the display screen. At this time, if the user likes the product of the product image data displayed on the above display screen, the user clicks the "Good button" by an input means such as a mouse. On the other hand, if the user does not like the product of the product image data displayed, the user clicks the "No button" by a mouse or the like to make a selection (step S13). When the "Good button" or the "No button" is selected, the user terminal 2 displays the product image data of the next product on the display screen, and these processes are continued until the user's yes / no judgment selection is completed for all or a predetermined number of the received product image data.

[0025] When the determination of the quality of the above-mentioned product image data is completed, the user terminal 2 transmits, for each product identification information, determination result information corresponding to the quality determination data of the product to the search server 1, adding its own user identification information (step S14). When the determination result information is input, the transmission / reception unit 17 determines whether the added user identification information is registered in the user registration table. If it is registered, the transmission / reception unit 17 outputs the above determination result information to the preference extraction unit 15.

[0026] Next, when the determination result information is input, the preference extraction unit 15 selects preference information corresponding to the pattern of the quality determination from the preference information table in the database 16. Here, the pattern of the quality determination is a data pattern of good determination and no determination corresponding to product identification information arranged in a predetermined order. Then, the preference extraction unit 15 writes and stores the obtained preference information in the user registration table in the database 16 corresponding to the user identification information (step S15).

[0027] Next, the combination extraction unit 14 selects combination information of products corresponding to the above-mentioned preference information from the combination information table in the database 16. In this combination information table, for example, when preference information is extracted for shoes, combination information corresponding to clothes, bags, hats, etc. corresponding to the preference is selected. Then, the combination extraction unit 14 extracts product image data such as clothes, bags, hats, etc. that match or are similar to the above combination information (step S16), and for the model, fits the selected clothes, bags, hats, etc. according to the shoes. When there is only one combination, the wearing image data is sent to the user terminal 2 as a single piece, or when there are multiple combinations, multiple types are sent to the user terminal 2, adding combination identification information to each combination (step S17).

[0028] When the user terminal 2 receives the wearing image data, it displays this wearing image data on the display screen. As a result, the user can obtain information on combinations of fashions corresponding to his or her preferences for the products that he or she has already searched for or purchased. Here, when the user makes a pass / fail determination as described above, the user terminal 2 associates pass / fail determination data with each combination identification information, adds its own user identification information as determination result information, and transmits it to the search server 1.

[0029] Then, when the determination result information is input, the transmission / reception unit 17 determines whether the added user identification information is registered in the user registration table. If it is registered, it outputs the above determination result information to the preference extraction unit 15. The preference extraction unit 15 generates combination preference information based on a pass / fail determination pattern corresponding to the input combination identification information. This combination preference information corresponds to the shape, color, pattern, texture, etc. of shoes, and consists of combinations of the shape, color, pattern, texture, etc. of other clothes, bags, hats, etc. that have received a passing determination. Each time the user purchases a product, it will be learned, and the accuracy of the combination preference information will be improved. That is, next, if the user purchases a bag based on this combined wearing image data, the preference information for the bag is searched, and combination preference information for shoes, clothes, hats, etc. corresponding to this preference information can be obtained. Therefore, the preferences of each user will be gradually narrowed down.

[0030] Also, as a challenge combination (a combination that is not normally selected by the user or a combination different from the above-mentioned preference information), for the combination information selected from the preference information, several products corresponding to the combination information that is the opposite of that of the fashion coordinator are randomly selected from the product table, and the selected products may be inserted into some of the combinations when generating the wearing image data. As a result, the user's preferences may become polarized, increasing the possibility of increased purchase desire.

[0031] Also, as shown in FIG. 6, the search support system in this embodiment may be configured to include a user terminal 2a having an extraction unit 11a with the same extraction function as the above-described extraction unit 11, and a search server 1a excluding the extraction unit 11. In this case, based on the input image data acquired by the user terminal 2a or the input image data stored in the user terminal 2a, the extraction unit 11a extracts the search part (for example, the product part) of the input image data, and only transmits the image data corresponding to the search part as the input image data to the search server 1a. Also, since the parts other than the extraction unit 11a are the same as those in the above-described embodiment, the description thereof is omitted. In this way, the user only needs to transmit to the search server 1a only the image data corresponding to the search part among the input image data as the image data to be transmitted. For example, there is no need to transmit to the search server 1a the image data corresponding to the user's face part or the face parts of other people.

[0032] Note that in this embodiment, the search support system of the present invention can be applied not only to products such as clothes and shoes but also to the above-described objects. Also, for example, the search support system of the present invention can be applied to services provided in accommodation facilities (such as hotels and inns) and restaurants. For example, in the above-described accommodation facilities, the search support system of the present invention can be applied when selecting combinations of room layouts, combinations of window views, combinations of rooms or their layouts and window views, and the like. Also, for example, in the above-described restaurants and the like, the search support system of the present invention can be applied when selecting combinations of restaurant interiors, combinations of window views, combinations of music, combinations of interiors and window views, combinations of interiors and music, combinations of window views and music, and the like.

[0033] <Second Embodiment> Hereinafter, a product search support system according to an embodiment of the present invention will be described with reference to the drawings. Conventionally, when purchasing a combination product (such as clothing, combination furniture, combination household appliances, etc.) that combines multiple different types (categories) of products, if you do not actually go to a retail store, you cannot directly ask a store clerk which combination is appropriate. Also, when selecting a product that matches well when combined with the product the user has purchased or will purchase from the vast product range of the virtual store, it is impossible to confirm with each coordinator of the combination products one by one, and since the user cannot make the selection themselves, they will not be able to make a purchase.

[0034] For example, in the case of clothing, even if the user wants to select pants or a shirt that matches the jacket they are purchasing, they cannot confirm with a top-notch fashion coordinator, worry about selecting an impossible combination, and will give up purchasing matching pants or shirts. Here, clothing refers to all the clothes, accessories (ornaments, bags, shoes, hats, etc.) that a person wears on their natural body.

[0035] This embodiment also solves such problems. In the purchase of combination products, for the products that the user purchases or has purchased, products that match when combined are extracted from the product range, and the extracted products are recommended to the user to assist the user in purchasing combination products.

[0036] FIG. 7 is a block diagram showing a configuration example of a product search support system according to the embodiment. The present invention, when purchasing a sales product item of any category in a product item combination consisting of sales product items of a plurality of categories, presents, as recommended products, the sales product items of the remaining categories in the product item combination that is considered to be a good combination with the purchased sales product item, so as to assist the user in easily selecting, from a large number of sales product items, the sales product items that are considered to be a good combination with the sales product item to be purchased. In the following description, all products in the clothing category (products in categories such as blazers, shirts, and pants) are regarded as product items, the product items actually sold by the virtual store are regarded as sales product items, and each product item combined in the image medium described later is described as a combined product item.

[0037] In this figure, the product search support system is composed of a product search support device 100 and one or more user terminals 200. The product search support device 100 includes a control unit 21, a similar item search unit 22, a combination search unit 23, a product search unit 24, a transmission / reception unit 25, a product item database 26, a product item image database 27, a combination information database 28, a combined product image information database 29, and a combined product image database 30. The user terminal 200 is, for example, a personal computer equipped with an Internet browser and provided in each home of each user. The product search support device 100 and the user terminal 200 are connected via an information communication network I including the Internet.

[0038] The product item database 26 has a storage area with a plurality of table configurations for each type of product item. That is, as shown in FIG. 8, the product item database 26 has a storage area with a table configuration in which the attribute information of the sales product items being sold is stored for each of the identification information A1 to An of the shirt product items as category 1. Similarly, as shown in FIG. 9, the merchandise item database 26 has a storage area with a table structure in which the attribute information of the corresponding merchandise items for sale is stored for each identification information B1 to Bn of the pants merchandise items as category 2. Similarly, as shown in FIG. 10, the merchandise item database 26 has a storage area with a table structure in which the attribute information of the corresponding merchandise items for sale is stored for each identification information C1 to Cn of the jacket merchandise items as category 3. Furthermore, although not shown, the merchandise item database 26 has a storage area that stores a plurality of tables such as shoes, blouses, and bags as other categories of clothing in addition to the above-described categories. In each table of the merchandise item database 26, a column for similar merchandise items is provided as one of the attribute information, and the merchandise items in the combined merchandise image information database 29 that are the same as or similar to the merchandise items for sale are grouped as similar merchandise items for each of the merchandise items for sale, as shown in FIGS. 11 (category 1 - shirt), 12 (category 2 - pants), and 13 (category 3 - jacket), respectively.

[0039] Here, the attribute information includes the identification information of the combined merchandise items (merchandise items in the combined merchandise image information database 29 described later) associated as the same or similar, the feature data of the image extracted from the image data of the merchandise items for sale, the sales information such as the price and brand of the merchandise items for sale, and so on. The types and acquisition methods of the feature data of the image will be described later. In the merchandise item image database 27, the image data for each merchandise item for sale in each category stored in the merchandise item database 26 is stored corresponding to the identification information of each of the above-described merchandise items for sale.

[0040] The combination information database 28 regards, in fashion magazines, fashion catalogs, and image media on the Internet, the clothing products worn in combination by models, etc., that is, each of the product items worn in combination, as combination product items, and stores the combinations of the combination product items combined with the combination product items and their respective corresponding relationships, associating each with an identification number. For example, in a fashion magazine, when a model is wearing each product item of a shirt with identification information a1-1, pants with identification information b1-2, and a jacket with identification information c1-7, as shown in the first row of the combination information database 28 in FIG. 14, the combination product item of the combination of identification information a1-1, identification information b1-2, and identification information c1-7 is stored in association as a set. This combination information is the combination of product items created by fashion designers as combinations, or the combination of product items created by fashion designers combined by fashion coordinators, regarded as the combination of combination product items. Therefore, each combination product item is a sophisticated combination, and with this combination, when worn, there is a high possibility that people who see the fashion will accept it naturally without feeling a sense of discomfort.

[0041] The combination product image information database 29 accumulates combination product items in clothing obtained from image media such as fashion magazines, fashion catalogs, and designers' collection information on the Internet (materials or devices presenting images such as magazine photos and illustrations) by category. Here, the combination product image information database 29 is configured, for example, with the table structures shown in FIGS. 15, 16, and 17. FIG. 15 shows an item of combined product accumulated with a shirt as Category 1, FIG. 16 shows an item of combined product accumulated with trousers as Category 2, and FIG. 17 shows an item of combined product accumulated with a jacket as Category 3. In the combined product image information database 29, identification information is assigned to the combined product items, and for each identification information, the attribute information of the corresponding combined product item is stored. This attribute information is, for example, the identification information of similar sold product items, the feature data of the image extracted from the image data of the combined product item, the sales information such as the price and brand of the combined product item, and so on. The types and acquisition methods of the feature data will be described later. In addition, each table in the combined product image information database 29 is provided with a column for similar product items as one of the attribute information, and the product items in the product item database 26 that are the same as or similar to the combined product item are described in association with each other as shown in FIG. 15 (Category 1 - shirt), FIG. 16 (Category 2 - trousers), and FIG. 17 (Category 3 - jacket).

[0042] In the combined product image information database 29, in the table of FIG. 15, the identification information a1-1 to a1-k is stored as a combined product item that is the same as or similar to the sold product item A1, the identification information a2-1 to a2-k is stored as a combined product item that is the same as or similar to the sold product item A2, and in order, the identification information an-1 to an-m is stored as a combined product item that is the same as or similar to the sold product item An. Similarly, in the combined product image information database 29, in the table of FIG. 16, the identification information b1-1 to b1-m is stored as a combined product item that is the same as or similar to the sold product item B1, the identification information b2-1 to b2-r is stored as a combined product item that is the same as or similar to the sold product item B2, and in order, the identification information bn-1 to bn-m is stored as a combined product item that is the same as or similar to the sold product item Bn. Similarly, in the combined product image information database 29, in the table of FIG. 17, the identification information c1-1 to c1-q is used as a combined product item that is the same as or similar to the sold product item C1, the identification information c2-1 to c2-k is used as a combined product item that is the same as or similar to the sold product item B2, and in order, the identification information cn-1 to cn-m is stored as a combined product item that is the same as or similar to the sold product item Cn. In the combined product image database 30, the image data for each combined product item in each category stored in the combined product image information database 29 is stored corresponding to the identification information of each of the above combined product items.

[0043] Also, the above-described product item database 26, product item image database 27, combination information database 28, combined product image information database 29, and combined product image database 30 may be classified by gender, age, fashion trends (for example, casual, conservative, older brother style, adult style, street style, Ura-Hara (Harajuku) style, mode style for men, and gal style, older sister style, conservative style, teen style, celeb style, etc. for women). When classified in this way, before performing the coordination process, it is necessary to obtain from the user the classification information such as gender, age (teenagers, twenties, thirties, etc.), and fashion trends.

[0044] The control unit 21 is activated when the virtual store it manages is accessed from the user terminal 200, obtains classification information such as gender, age, and fashion trends, and transmits to the user terminal 200 a processing program (operating by a browser of the user terminal, etc.) for performing display and selection operations on the combined items described later. Here, the control unit 21 and the user terminal 200 perform transmission and reception of each data via the information communication network I and the transmission / reception unit 25. The user terminal 200 activates the above processing program with an internal browser and performs transmission and reception of data with the product search support device 100 in the process of selecting an image display or a sold product item or a combined product item. In addition, when the control unit 21 receives response data for the above screen data from the user terminal 200, it selects a table of any one product item from a plurality of product item databases 26 corresponding to the gender, age, and fashion trends, and transmits category information indicating the category of the combined product item, for example, character information such as shirts, trousers, jackets, shoes, or image information, to the user terminal 200 via the transmission / reception unit 25 and the information communication network I.

[0045] In addition, when the control unit 21 receives category information indicating the type of category selected by the user, for example, category information indicating a shirt, transmitted from the user terminal 200, it selects the table shown in FIG. 8 with the shirt (category 1) as the category of the product item database 26, reads the identification information A1 to An of the product items in this table, reads the image data of the product items up to the identification numbers A1 to An from the product item image database 27, converts it into image data of thumbnail images (reducing the number of bits of the image data), adds the identification number of the corresponding sales product item to each, and transmits it to the user terminal 200. In addition, when the identification number of the sales product item selected for purchase by the user from the thumbnail image is input, the control unit 21 performs an order receiving process for the order and transmits the input identification number to the similar item search unit 22. Here, the order receiving process includes billing the user terminal 200 for the price described in the attribute information, checking the inventory of the ordered sales product item, and procedures for delivering the sales product item to the address input by the user.

[0046] The similar item search unit 22 searches a table (category 1) of the combined product image information database 29 for a plurality of combined product items in which the identification number matching the identification number of the input sales product item is described in the column of similar product items, extracts the combined product items identical or similar to the above sales product item as similar product items, and outputs them to the combination search unit 23. The combination search unit 23 searches the combination information database 28 for product items in different categories, for example, pants (category 2), corresponding to each of the similar product items with identification numbers input from the similar item search unit 22, extracts the corresponding combination product items, and outputs the identification information of the combination to the control unit 21 as the second combination product items. Also, the control unit 21 reads out, from the combined product image database 30, the image data corresponding to the identification information of a plurality of combinations of the identification information of the similar product items input from the combination search unit 23 and the identification information of the second combination product items, corresponding to each identification information, and transmits it to the user terminal 200. When the user inputs the identification information of the combination product items in the selected combination, the received identification information is output to the product search unit 24.

[0047] The product search unit 24 searches the product item database 26 for the actually sold sales product items corresponding to the identification information of the input second combination product items, and uses them as recommended sales product items in different categories to be combined with the sales product items purchased by the user. The image data of this recommended sales product item is read out from the product item image database 27 based on the identification information and transmitted to the user terminal 200. Also, the product search unit 24 compares the amount of the recommended sales product item with the amount of the sales product item purchased by the user, determines whether it is within the set range (for example, within 0.5 to 2 times the amount of the purchased combination product), and if it is less than or equal to the purchase amount, it may be transmitted to the user terminal 200, and if it is outside the range, the recommended product may not be transmitted.

[0048] Next, the operation of the product search support system according to this embodiment will be described with reference to FIGS. 7 and 18. FIG. 18 is a flowchart showing an operation example of the product search support system in FIG. 7. Hereinafter, the product input by the user will be described as a product item, the product actually sold in the virtual store as a sales product item (product item database 26), and the product extracted from the image medium used when selecting a combination of products as a combination product item (combination product image information database 29). When the user terminal 200 accesses a virtual store managed by the product search support device 100 via the information communication network I by the user's operation, the control unit 21 transmits to the user terminal 200 information for acquiring gender, age, and fashion trends, and a processing program for displaying a combination item described later and performing a selection operation on the displayed image. Inside the user terminal 200, the processing program in the Internet browser includes an image display processing unit and a selection processing unit, and is configured to display image data transmitted from the product search support device 100, perform editing processing on the image data, and perform selection processing on the image data.

[0049] Then, when the control unit 21 receives response data (for example, data selected from a plurality of options respectively displayed on the screen) regarding the gender, age, and fashion trends input to the above screen data from the user terminal 200, the control unit 21 selects a corresponding table from a plurality of tables in the product item database 26 corresponding to the gender, age, and fashion trends, and transmits category information indicating the category of the sold product items, for example, character information such as shirts, pants, jackets, shoes, or image information, to the user terminal 200 via the transmission / reception unit 25 and the information communication network I. When character information or image information is input, the user terminal 200 displays a category (for example, shirts, pants, jackets, etc.) based on the character information or image information on the display unit according to the above processing program, and performs a display (for example, "please select") prompting the user to select which one to purchase. When the user selects any category, the user terminal 200 transmits category information indicating the category selected by the user, for example, category information indicating shirts, to the product search support device 100 (step F1).

[0050] That is, when receiving the category information of the product item that the user wants to purchase, the control unit 21 selects the table shown in FIG. 8 with shirts as the category (category 1) from the tables of the product item database 26 shown in FIGS. 8 to 10, reads all of the identification information A1 to An of the sold product items in this table, reads the image data of the sold product items up to the identification numbers A1 to An from the product item image database 27, adds the identification numbers of the sold product items corresponding to each thumbnail image as the image data of the thumbnail image, and transmits it to the user terminal 200. When the image data of the thumbnail image is input, the user terminal 200 displays the thumbnail images of the sold product items with the identification numbers A1 to An on the display unit (step F2). Then, when the user selects any of the displayed thumbnail images, the user terminal 200 transmits the identification information of the selected thumbnail image to the product search support device 100.

[0051] When receiving the identification information of the selected thumbnail image, the control unit 21 performs an order receiving process for the sold product item with this identification number, that is, a shirt, as the sold product item selected by the user as the purchase target from the thumbnail image, and transmits the input identification number to the similar item search unit 22. Then, for example, when the identification number A1 of the sold product item purchased by the user is input, the similar item search unit 22 extracts from the tables of the combined product image information database 29 shown in FIGS. 15 to 17 as a candidate group consisting of one or more combined product items that match or are similar to this identification number A1 (step F3). Here, for example, when the identification number A1 of the sold product item is input from the combined product image information database 29, the similar item search unit 22 extracts the combined product items with the identification numbers a1-1 to a1-q in which the identification number A1 is described in the column of the similar product item in the attribute data as a candidate group of the similar combined product items, and outputs it to the combined search unit 23.

[0052] The combination search unit 23 searches the combination information database 28 shown in FIG. 14 for different categories corresponding to each of the combination product items of the identification numbers in the candidate group input from the similar item search unit 22, for example, combination product items of trousers, and extracts the corresponding combination product items of trousers (for example, b1-2, b2-3, b1-4,... corresponding to a1-1, a1-2, a1-3,... shown in FIG. 14), and outputs the respective identification information in the combination as the second combination product item to the control unit 21 (step F4). Here, the category recommended as the sales product item to be combined with the sales product item purchased by the user may be set in advance corresponding to the purchased product, or the user may be allowed to select from a plurality of categories first. Then, the control unit 21 reads out the image data corresponding to each of the identification information of a plurality (the number of combination product items corresponding to the candidate group) of combinations of the identification information of the combination product item input from the combination search unit 23 and the identification information of the second combination product item from the combination product image database 30 corresponding to the respective identification information, adds the respective identification information to the image data of each combination product item, and transmits it to the user terminal 200.

[0053] When the identification information of the combination product item (corresponding to a shirt) and the identification information of a plurality of combinations of the identification information of the second combination product item (corresponding to trousers) and the corresponding image data are input, the user terminal 200 displays the image data of each combination on the display unit (step F5). Also, a plurality of three-dimensional human images corresponding to each body type created by CG (Computer Graphics) are displayed at the end of the display screen. By selecting any one of them, the user terminal 200 overlays and displays the shirt and trousers of the combination product item on the human image. Further, the user terminal 200 may perform image processing to capture the user's own face with the web camera mounted by the user and overlay the selected face area on the face part of the CG human image for display.

[0054] Next, when the user selects any one from the plurality of combinations displayed on the display unit (step F6), for example, when selecting the combination of identification information a1-1 and identification information b1-2, the user terminal 200 transmits the identification information b1-2 of the second combined product item (corresponding to the pants) in the combination selected by the user to the product search support device 100. When the identification information b1-2 of the second combined product item is input via the control unit 21, the product search unit 24 searches the table shown in FIG. 9 corresponding to the category of pants in the product item database 26 (step F7), and extracts the sales product item that is the identification information B1 corresponding to this identification information b1-2 (step F8). Then, the control unit 21 searches for and reads out the image data corresponding to the identification information B1 of the combined sales product item extracted by the product search unit 24 from the product item image database 27. In addition, the control unit 21 searches for and reads out the image data corresponding to the identification information A1 purchased by the user from the product item image database 27, and transmits it together with the image data of the sales product item of the identification information B1 to the user terminal 200.

[0055] When the image data of the sales product item is input, the user terminal 200 displays the combined image data on the display unit on the display unit (step F9). At this time, similar to step F5 described above, a plurality of three-dimensional human images corresponding to each body type created by CG are displayed at the end of the display screen, and by selecting any one of them, the user terminal 200 overlays and displays the shirt (identification information A1) and pants (identification information B1) of the sales product item on the human image. Further, the user terminal 200 may capture the user's face with the mounted web camera and display the selected face area overlaid on the face portion of the human image. Then, when the user terminal 200 selects whether to purchase or not purchase the recommended product of the sales product item with the identification information B1 from the options on the display screen (step F10), it transmits the identification information A1 and the identification information B1 to the product search support device 100 together with the information indicating whether to purchase or not purchase. When the identification information A1 and the identification information B1 are input together with the information indicating whether to purchase or not purchase, the control unit 21 performs an order receiving process in the case of purchasing, in the same manner as in the case of the sales product item with the identification information A1.

[0056] Also, a history database 31 indicated by a broken line in FIG. 7 may be provided in the product search support device 100. The history database 31 is composed of a user table having the table configuration shown in FIG. 19 and a purchase history table having the table configuration shown in FIG. 20 for each user. The user table in FIG. 19 associates user identification information for identifying each user assigned to each user who has registered as a member or purchased a sales product item, and at least the user's name and the user's user email address corresponding to this user identification number. In addition, the purchase history table in FIG. 20 is provided for each user, is identified by the above user identification information, corresponds to the access date and time when the virtual store was accessed, and includes the identification information of the sales product item purchased at the access date and time, the identification information of the sales product item that was searched but not purchased, and the identification information of the combined product item that was recommended but not purchased for the purchased sales product item as a history. If there is no purchased sales product item, "-" is stored.

[0057] For example, when the user accesses the virtual store from the user terminal 200 and registers as a member by entering the name and email address on the member registration page, the control unit 21 assigns identification information and adds it to the user table for additional registration. Then, the control unit 21 searches for sales product items in the category selected by the user. When the user purchases a sales product item, the purchased sales product item is stored in the area of the identification information of the purchased product item. On the other hand, when the search is made but not purchased, the identification information of the searched sales product item is stored in the area of the identification information of the product item that was searched for but not purchased. Also, when the control unit 21 purchases a sales product item as a recommended product to be combined with the above-mentioned purchased sales product item, in the above purchase history table, the purchased sales product item is stored in the area of the identification information of the purchased product item. On the other hand, when the sales product item as the above-mentioned recommended product is not purchased, it is stored in the area of the identification information of the product item that was recommended but not purchased.

[0058] Then, when the user accesses the virtual store again and searches for a sales product item, the control unit 21 selects the purchase history table of this user from the purchase history database 31 based on the user identification information input by the user, and searches the selected purchase history table based on the identification information of this sales product item. At this time, when the identification information of the sales product item being searched by the user is detected in the area of the identification information of the purchased product item, the date and time of purchase of the same sales product item are added and sent to the user terminal 200, and displayed on the display screen to notify the user of the combined commercialized item that has already been purchased.

[0059] Also, when the identification information of the sales product item being searched is detected in the area of the identification information of the product item that was searched for but not purchased, the date and time of the search for the same sales product item are added and sent to the user terminal 200, and displayed on the display screen to notify the user of information that stimulates the desire to purchase, such as "This is the product item that was also searched before. Isn't it your favorite type?" In addition, when the identification information of the sales product item being searched is detected in the area of the identification information of the product item that was recommended but not purchased, the control unit 21 adds the date and time when the same sales product item was recommended and the image data of the sales product item purchased at that time, and transmits it to the user terminal 200, and displays it on the display screen to notify the user of information that stimulates the desire to purchase, such as "This is a product item that goes well when combined with the product item you purchased before."

[0060] Next, the feature data will be described. As described in FIGS. 8 to 10 and FIGS. 15 to 17, the feature data is, for example, the result of performing a two-dimensional Fourier transform on the pattern of the fabric of clothing for each color space of R (red), G (green), and B (blue). The control unit 21 generates the element data RD, GD, and BD of this feature data by two-dimensional Fourier transform. At this time, if the sales product item or the combined product item is a shirt or a jacket, the width of the clothes or the shoulder width is used as the reference value for the length when performing the two-dimensional Fourier transform, so as to ensure data consistency in the processing between each sales product item and the combined product item. Also, if it is pants, the width of the waist part is used as the reference value for the length when performing the two-dimensional Fourier transform, so as to ensure data consistency in the processing between each sales and combined product item. That is, in order to confirm the similarity by the feature data and determine the character such as the size of the pattern, it is necessary to standardize the dimensions of each part of the whole with the dimension of any position of the ornament as the reference value, so that the two-dimensional Fourier transform results of the combined products within the same category can be made consistent. When imaging the image data for performing the two-dimensional Fourier transform, shirts, jackets, pants, etc. are placed on a flat floor, flattened, and then imaged with a digital camera or the like. Also, the shape as the element data of the feature data is, for example, the ratio of the sleeve length to the shoulder width for a shirt or a jacket, and the ratio of the crotch width to the hem width for pants. Next, for the texture, the fabric part with the largest area is enlarged, and a two-dimensional Fourier transform of the fabric shape is performed. At this time, with a constant magnification rate, the consistency of the two-dimensional Fourier transform data obtained from the image data between different combination items is ensured.

[0061] As described above, the clerk in the virtual store collects feature data from the image data of the sold merchandise items by the control unit 21, and also collects attribute data from the image data of the combination merchandise items collected from image media such as fashion catalogs or the Internet. Then, for the combination merchandise items having similarity, clustering of the combination merchandise items collected from the image media is performed with the feature data of the sold merchandise items as the centroid data of the block. Here, for example, the control unit 21 obtains the distance between the comparison feature vector composed of the feature data of the image data of each combination merchandise item collected from the image media input by the clerk and the reference feature vector composed of the feature data of the image data of each sold merchandise item, and determines the combination merchandise item having a feature vector closer in distance to the reference feature vector of each sold merchandise item and each other combination merchandise item as a block similar to the sold merchandise item considered to be close, and generates the correspondence relationship between the combination merchandise items and the sold merchandise items in the columns of combination merchandise items and similar merchandise items in the tables of FIGS. 15, 16, and 17. Similarly, the correspondence relationship between the combination merchandise items and the sold merchandise items in the columns of sold merchandise items and similar merchandise items in the tables of FIGS. 8, 9, and 10 is generated. That is, the identification information described in the column of similar merchandise items in the tables in FIGS. 8, 9, and 10 is the identification information of the combination merchandise items that are closer in distance, that is, have similarity, compared to other sold merchandise items for the sold merchandise item with the corresponding identification information. Here, for example, the control unit 21 may store the similar product items in the columns of the similar product items in FIGS. 8, 9, and 10 in ascending order of distance, that is, in descending order of similarity. As a result, when selecting the combined product item that is most similar to the sold product item, the similar item search unit 22 can more easily extract the most similar combined product item or the top h items (h is preset) from the combined product items collected from the image medium, starting from FIGS. 15 to 17.

[0062] Also, as described above, instead of performing clustering in advance by comparing the reference feature vector of each sold product item with the feature data of the combined product items collected from the image medium having a comparison feature vector at a short distance and providing a column for describing the identification information of the similar product items, the similarity may be calculated each time when searching for similar combined product items. For example, in the search for the combined product items obtained from the image medium similar to the sold product item in step F3, the similar item search unit 22 may calculate the similarity (the shorter the distance, the higher the similarity) based on the distance between the above-mentioned reference feature vector and the comparison feature vector, and extract the combined product items collected from the image medium having similarity with the sold product item. At this time, similarly, in the search for the sold product items similar to the combined product items obtained from the image medium in step F7, the product search unit 24 may calculate the similarity based on the distance between the above-mentioned reference feature vector and the comparison feature vector, and extract the sold product items having similarity with the combined product items collected from the image medium. Furthermore, the similar item search unit 22 and the product search unit 24 may calculate the distance between the sold product items and the combined product items collected from the image medium, and extract the one with the highest similarity or the top h items with the highest similarity. As described above, according to the present embodiment, in the purchase of a combination product, for the product that the user purchases or has purchased, the product that matches when combined is extracted from the group of products being sold, and the extracted product is recommended to the user. Therefore, when selecting a matching product from many products in a virtual store on the Internet, there is no need for the user to worry about whether it matches or not by himself / herself, and the combination product can be easily selected.

[0063] <Third Embodiment> In the second embodiment, the product search support device 100 transmits the sales product items to the user terminal 200 by means of thumbnail images and allows the user to select them on the screen of the user terminal 200. In the third embodiment, after the user accesses the virtual store, the image data of the clothing that the user wants to purchase, which is read from an image medium by a scanner or the like or downloaded from the Internet, is input to the user terminal 200 as the image data of the product item. The user terminal 200 transmits this image data to the product search support device 100, and the product search support device 100 may extract the sales product items similar to the combined product items of this image data. The operation after extracting the sales product items is the same as the operation after the user selects the sales product items from the thumbnail images in the second embodiment.

[0064] The configuration of the product search support device 100 according to the third embodiment is the same as that of the second embodiment. Hereinafter, only the operations different from those of the second embodiment will be described. When the user accesses the virtual store using the user terminal 200, the product search support device 100 is activated, and the control unit 21 sends the image information of an input screen (search by clicking on the options with a mouse or the like) that asks whether to select the sales product items from the thumbnail images on the user terminal 200 or to extract the similar sales product items from the product item database 26 based on the image data of the clothing input by the user to the user terminal 200. Then, when the user decides to select a product to purchase from the thumbnail images and a response signal indicating the selection is input from the user terminal 200 based on the thumbnail images, the control unit 21, as in the second embodiment, adds identification information of the sales product items to the respective image data and transmits thumbnail images of a plurality of sales product items to the user terminal 200. After the user selects one of the thumbnail images thereafter, the process is the same as that of the second embodiment.

[0065] On the other hand, when the user selects to extract similar sales product items from the product item database 26 based on the image data input by the user, the user terminal 200 transmits a response signal indicating the selection based on the image data input by the user to the product search support device 100. Thereby, the control unit 21 transmits information on the input screen for inputting the image data to the user terminal 200. The user causes the user terminal 200 to read, by a scanner, image data of clothing to be purchased (image media on paper such as fashion magazines or fashion catalogs), or images the clothing with a digital camera and reads the captured image data, or reads image data obtained via the Internet into the user terminal 200. When the image data is read, this image data is displayed in the image data display area of the input screen on the user terminal 200 as shown in FIG. 21. The user selects the clothing portion of the image data that the user wants to purchase, for example, the shirt portion to be extracted, with a dashed line H1, selects the portion outside the shirt portion to be extracted (the portion near the outer periphery of the shirt) with a dashed line 2, selects a category name with a combo box, and clicks the transmit button on the display screen. Thereby, the user terminal 200 detects that a process of transmitting the image data is requested, and transmits the image data of the clothing with the dashed line H1 and the dashed line H2 drawn, the image data of the clothing without the lines drawn, and category information indicating the category name of the selected clothing to the product search support device 100.

[0066] When receiving the image data of the clothing with the dashed lines H1 and H2 drawn thereon and the image data of the clothing without the dashed lines drawn thereon, the control unit 21 performs a segmentation process on the image data of the clothing by the Graph-Cut method to segment the shirt part and the other parts, and extracts the shirt part. That is, the control unit 21 determines the boundary between the region having the same pixel value as the pixels on the dashed line H1 drawn on the shirt part and the region having the same pixel value as the pixels on the dashed line H2 drawn outside the shirt part as the position where the error of the error function based on the gradation is minimized, thereby performing segmentation. Here, when the control unit 21 extracts the shirt part, as described above, it performs two-dimensional Fourier transform and extracts the shape data as an element in the feature data, and outputs it as the detection target feature data.

[0067] Then, the control unit 21 selects a table corresponding to the category of the extracted clothing. For example, in this embodiment, it selects the shirt table from the product item database 26. After selecting the shirt table, the control unit 21 calculates the distance between the reference feature vector composed of the feature data of each sold product item in the table and the target feature vector composed of the detection target feature data, and extracts the sold product items having feature data similar to the detection target feature data from the shirt table up to the top h items, for example, up to the top 5 items. Here, since it is difficult for the control unit 21 to extract the texture from the image data transmitted from the user in the feature data, when generating the feature vector from the feature data, the texture is excluded from the elements of the vector, and the distance between the two feature vectors to be compared is calculated. Next, the control unit 21 reads out the image data of each sold product item from the product item image database 27 according to the identification information of the top 5 items, and transmits it to the user terminal 200 as a thumbnail image corresponding to the identification information respectively. The subsequent processing is the same as the processing after transmitting the thumbnail image in step F2 in the flowchart of FIG. 18 in the second embodiment.

[0068] <Fourth Embodiment> In the third embodiment, a sales product item identical or similar to the image data of the product item input by the user is searched from the product item database 26 that stores the sales product items being sold, the searched sales product item is presented to the user, and processing is performed to recommend sales product items of other categories that are combined with the searched sales product item. In the fourth embodiment, an operation is performed to search for sales product items that are combined with the clothing already owned by the user.

[0069] The user takes a picture of their own clothing, for example, a shirt, with a digital camera, and causes the user terminal 200 to read the image data of the shirt from the digital camera. Then, when the user accesses the virtual store through the user terminal 200, the product search support device 100 is activated, and the control unit 21 sends to the user terminal 200 the image information of an input screen (search by clicking on options with a mouse or the like) that asks whether to select a sales product item from the thumbnail images, or extract similar sales product items from the product item database 26 based on the image data of the clothing input by the user, or recommend sales product items that are combined with the clothing of the image data input by the user.

[0070] Then, when a response signal indicating the selection by the thumbnail image is input from the user terminal 200 by the user's decision to select the product to purchase from the thumbnail images, the control unit 21, similar to the second embodiment, adds the identification information of the sales product item to each image data and sends the thumbnail images of the plurality of sales product items to the user terminal 200. After the user selects any one from the thumbnail images thereafter, the processing is the same as that of the second embodiment. On the one hand, when the user selects to extract the sales product item from the product item database 26 based on the image data input by the user himself / herself, the user terminal 200 transmits a response signal indicating that it is selected based on the image data input by the user to the product search support device 100. The subsequent processing is the same as that in the third embodiment.

[0071] Also, when the user selects to recommend sales product items of other categories to be combined with the clothing of the image data input by the user himself / herself, the user terminal 200 transmits a response signal indicating that it has selected to recommend sales product items to be combined with the clothing of the image data input by the user to the product search support device 100. Hereinafter, the process of recommending sales product items to be combined with the clothing of the image data input by the user according to the fourth embodiment will be described. The configuration of the product search support device 100 according to the fourth embodiment is the same as that in the second embodiment. Hereinafter, only the operations different from those in the first and third embodiments will be described.

[0072] As described above, when the user selects to recommend sales product items to be combined with the clothing of the image data input by the user himself / herself, the user terminal 200 transmits a response signal indicating that it recommends sales product items to be combined with the clothing of the image data input by the user to the product search support device 100. Thereby, the control unit 21 transmits the information of the input screen for inputting the image data to the user terminal 200. The user captures the image data of the clothing (for example, a shirt) he / she has for which he / she wants to recommend the combined sales product items with an imaging device such as a digital camera, and causes the user terminal 200 to read the image data of the clothing captured by this imaging device.

[0073] When the image data read by the user terminal 200 is displayed, the user selects and inputs the category of the clothing in the image data displayed in the image data display area from the combo box in the category input field near the image data display area. When the user selects the send button on the input screen using a pointing device such as a mouse, the user terminal 200 detects that a process of transmitting image data is requested, and transmits the image data displayed in the image data display area to the product search support device 100 together with the category information indicating the category. When image data is input, the control unit 21 performs a two-dimensional Fourier transform on the image data of clothing, for example, the image data of a shirt, and extracts shape data as an element in the feature data, and outputs it to the similar item search unit 22 together with the category information as the detection target feature data.

[0074] Then, the similar item search unit 22 selects a table corresponding to the category of the extracted clothing, for example, a shirt table, from the combined product image information database 29. After selecting the shirt table, the similar item search unit 22 calculates the distance between the reference feature vector composed of the feature data of each combined product item in the table and the target feature vector composed of the detection target feature data, and selects a combined product item having feature data similar to the detection target feature data from the shirt table. Extract the top h items, for example, the top 5 items. Here, since it is difficult for the control unit 21 to extract the texture from the image data transmitted from the user in the feature data, when generating the feature vector from the feature data, the texture is removed from the elements of the vector to generate the reference feature vector and the detection target feature vector.

[0075] Then, the similar item search unit 22 outputs the combined product items up to the fifth highest detected similarity, for example, the identification information of five shirts, to the combined search unit 23. When the identification information of the combined product item is input, the combined search unit 23 reads out the identification information of other categories, for example, the combined item of pants, stored corresponding to each of the five identification information of the input combined product item in the combined information database 28. Next, the combination search unit 23 transmits to the control unit 21 the combinations of the identification information for each of the five combinations of a shirt and pants. The subsequent processing is the same as the processing after step F5 in the flowchart of FIG. 18.

[0076] <Fifth Embodiment> In the combined product image information database 29, it may be possible to store the attribute data of the image data of old clothing collected from old fashion magazines or fashion catalogs such as those from 10 years ago or 20 years ago in the past, corresponding to the identification information. Also, in the combined product image database 30, the image data of old clothing is stored corresponding to the above identification signal. Then, corresponding to the combinations of combined product items in past fashion magazines or fashion catalogs, etc., a table of combinations of combined product items between different categories shown in FIG. 14 is created in the combination information database 28. As described above, by configuring the combined product image information database, the combined product image database 30, and the combination information database 28, when new sales product items similar to past designs are sold, it becomes possible to easily extract the sales product items to be combined into new combined product items similar to past designs from the sales product items currently on sale, using the combinations in past designs. In this embodiment, clothing has been described as an example of a product item, but it can be easily applied to all products in general that are combined products of multiple different types (categories) of products, such as combined furniture, combined household appliances, etc.

[0077] In each of the above-described embodiments, a program for realizing the functions of each part of the search servers 1, 1a, and the product search support device 100 may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed to perform product search support processing. Here, the "computer system" shall include hardware such as an OS and peripheral devices. Also, the "computer system" shall include a WWW system equipped with a homepage providing environment (or display environment). Further, the "computer-readable recording medium" refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built into a computer system. Furthermore, the "computer-readable recording medium" shall include a volatile memory (RAM) inside a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, and that holds the program for a certain period of time.

[0078] Also, the above program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the "transmission medium" for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication wire) such as a telephone line. Also, the above program may be for realizing a part of the functions described above. Furthermore, it may be a so-called difference file (difference program) that can realize the functions described above in combination with a program already recorded in a computer system. As described above, according to the aspect of the present invention, an object desired to be searched by a user or an object similar to the object can be easily searched.

Industrial Applicability

[0079] The present invention is suitable for use in a search support system that assists a user in selecting and determining an object to be searched for when searching for an object via the Internet, and in similar technologies, and can perform a search for an object that the user wants to search for based on image data of the object.

Explanation of Signs

[0080] 1…Search server 2…User terminal 11…Extraction unit 12…Type determination unit 13…Search unit 14…Combination extraction unit 15…Preference extraction unit 16…Database 17…Transmission / reception unit 18…Transmission / reception unit I…Information communication network 100…Product search support device 200…User terminal 21…Control unit 22…Similar item search unit 23…Combination search unit 24…Product search unit 25…Transmission / reception unit 26…Product item database 27…Product item image database 28…Combination information database 29…Combination product image information database 30…Combination product image database 31…History database

Claims

【Claim 1】 A search system connected to a user's terminal via a network, comprising: a database storing object information including image data regarding an object, combination information indicating combinations of the objects, and user information including identification information for each user; a search unit that extracts, from the combination information stored in the database, other objects to be combined with the specified object based on the image data of the object specified by the user; comprising: information including the image data regarding the other extracted objects is transmitted to the terminal via the network; the user information includes history information regarding at least one of searching, recommending, and purchasing of the object; the history information includes information indicating the purchased object and information indicating the object that was searched or recommended but not purchased; the search system further comprises a control unit that acquires provision information for the user based on the history information and transmits the provision information to the terminal via the network; search system.

Images