Recommendation system, server program and client program

Abstract

A server device 20 comprises: an interest information formation means 21 for forming interest information, which is a vector obtained by digitizing a plurality of adjectives expressing a user, item information, and store information; a distance calculation means 22 for calculating a first distance between interest information pieces for the user and the item information, and a second distance between interest information pieces for the user and the store information; a recommendation information determination means 23 for determining recommendation information in which the item information corresponding to the first distance and the store information corresponding to the second distance are recommended to the user if the first distance and the second distance meet a prescribed condition; and a recommendation information provision means 24 for providing the recommendation information to a client device.

Description

[Technical Field]

[0001] The present invention relates to a recommendation system, a server program, and a client program that provide recommendation information for recommending products and services to users. [Background technology]

[0002] Nowadays, various information such as website browsing history, keywords searched on search engines, and purchasing history at online stores is analyzed for each user, and information (recommendations) about products and services that suit each user's preferences is provided.

[0003] For example, Patent Document 1 discloses a technology for identifying products and personal information including age and gender from images or videos, and providing information (recommendations) about recommended products according to age and gender.

[0004] However, the technology disclosed in Patent Document 1 provides recommendations based only on images, and it is difficult to say that the user's interests and concerns are appropriately reflected. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-161116 Summary of the Invention [Problem to be solved by the invention]

[0006] The present invention has been made in consideration of the above circumstances, and aims to provide a recommendation system, server program, and client program that can make recommendations that reflect a user's preferences, interests, and concerns. [Means for solving the problem]

[0007] An aspect of the present invention that solves the above problem is a recommendation system comprising a client device owned by a user and a server device that provides the client device with recommendation information, which is item information and store information recommended to the user, wherein the server device comprises: an interest information formation means that forms interest information, which is a vector that quantifies a plurality of adjectives that describe the user, the item information, and the store information; a distance calculation means that calculates a first distance between the interest information about the user and the interest information about the item information, and a second distance between the interest information about the user and the interest information about the store information; a recommendation information determination means that, if the first distance meets a predetermined condition, determines the item information corresponding to the first distance to be recommendation information recommended to the user, and, if the second distance meets a predetermined condition, determines the store information corresponding to the second distance to be recommendation information recommended to the user; and a recommendation information provision means that provides the recommendation information to the client device.

[0008] Another aspect of the present invention that solves the above problem is a server program that functions as an interest information forming means for forming interest information, which is a vector quantified for each of a plurality of adjectives that express the user, the item information, and the store information, on a client device owned by the user, the server program including: an interest information forming means for forming interest information, which is a vector quantified for each of a plurality of adjectives that express the user, the item information, and the store information; a distance calculation means for calculating a first distance between the interest information about the user and the interest information about the item information, and a second distance between the interest information about the user and the interest information about the store information; a recommendation information determination means for determining, if the first distance meets a predetermined condition, that the item information corresponding to the first distance is the recommendation information recommended to the user, and for determining, if the second distance meets a predetermined condition, that the store information corresponding to the second distance is the recommendation information recommended to the user; and a recommendation information providing means for providing the recommendation information to the client device.

[0009] Another aspect of the present invention that solves the above problem is a client program that functions as an interest collection means for collecting clip information selected by the user from the information obtained by the client device, behavioral history which is information representing the user's behavior that can be detected by the client device, and purchase history which is information about products and services purchased by the user via the client device, and transmitting the collected information to the server device, and as a recommendation information display means for receiving and displaying the recommendation information transmitted by the server device based on the clip information, behavioral history, and purchase history. [Effects of the Invention]

[0010] According to the present invention, a recommendation system, a server program, and a client program are provided that can make recommendations that reflect a user's preferences, interests, and concerns. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating the functions of a recommendation system. [Figure 2] FIG. 10 is a diagram illustrating an example of a screen displayed on a smartphone. [Figure 3] FIG. 10 is a diagram showing an example of interest information formed from item information and store information. [Figure 4] FIG. 10 illustrates an example of forming interest information about a user. [Figure 5] FIG. 10 is a diagram showing the positional relationship of interest information relating to users, item information, and store information. [Figure 6] FIG. 10 is a diagram showing the positional relationship between a smartphone and a store when recommendation information is provided to the smartphone. [Figure 7] FIG. 10 is a diagram for explaining correction of the distance between pieces of interest information. [Figure 8] FIG. 10 is a diagram illustrating narrowing down of item information. [Figure 9] FIG. 1 is a diagram illustrating a processing flow in a recommendation system. [Figure 10] FIG. 1 is a diagram illustrating a method for predicting user behavior. [Figure 11] FIG. 1 is a diagram illustrating a method for predicting user behavior. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of the present invention will be described. Note that the description of the embodiments is merely an example, and the present invention is not limited to the following description.

[0013] First Embodiment As shown in FIG. 1, the recommendation system 1 includes a smartphone 10 and a server device 20 that can communicate with each other via the Internet. The recommendation system 1 determines what to recommend to a user from among item information, which is information about products, and store information, which is information about stores, and provides the information to the smartphone 10 owned by the user. Note that HTTP can be used as an example of communication via the Internet, but this is not limiting, and a unique protocol implemented on TCP / IP may also be used. Furthermore, communication standards other than the Internet may also be used.

[0014] The smartphone 10 is an example of a client device as claimed, and although not shown, is a general mobile terminal device carried by a user, equipped with a CPU, RAM, storage devices such as a flash disk, input / output devices, communication means, a camera, GPS, etc. The server device 20 is a general computer, although not shown, equipped with a CPU, RAM, storage devices such as a hard disk, input / output devices, communication means, etc.

[0015] A client program is installed in the storage device of the smartphone 10, and the client program is loaded into RAM and executed by the CPU. A server program is installed in the storage device of the server device 20, and the server program is loaded into RAM and executed by the CPU.

[0016] Information about users who use the recommendation system 1 (hereinafter referred to as user information) is registered in the server device 20. The user information includes an ID to identify the user, as well as a password, name, address, etc. By sending and receiving the ID and password, the server program and client program can authenticate that the user is registered in the recommendation system 1. By authenticating the user in this way, it becomes possible to perform the various processes described below for each user.

[0017] Item information and store information are stored in the storage device of the server device 20. As will be described in detail later, item information and store information that matches the user's interests and concerns is provided to the user as recommendation information.

[0018] Item information is information about products, services, and events that are the subject of commercial transactions. For example, it may include the name of the product, service, or event, detailed specifications and descriptions, price, classification (category), location information (information indicating the location of the store selling the product, the store or location providing the service, or the location where the event is being held), image data, etc.

[0019] Store information is information about stores and event venues that sell products, provide services, or host events, such as the name of the store or event venue, location information for the store or event venue, detailed descriptions, prices, categories, and image data.

[0020] The item information and store information are stored in the storage device of the server device 20 and can be read by the server program. The item information and store information are configured so that, by specifying a product, service, or event, it is possible to search for store information that sells the product, provides the service, or is related to the event venue. Such item information and store information may be created based on existing information available via the Internet, or may be information created by a person observing actual products, services, events, stores, etc.

[0021] The client program is a program (smartphone application) that causes the smartphone 10 to function as interest gathering means 11, AR information presenting means 12, and recommendation information presenting means 13.

[0022] The interest collection means 11 collects information representing the user's interests, concerns, and preferences (hereinafter referred to as interests, etc.) via the user's smartphone 10, and transmits the information to the server device 20. Examples of such information representing the user's interests, etc. include clip information, behavioral history, and purchase history, which will be described below.

[0023] The clip information refers to information that is selected by the user as being of interest, etc., from among the information obtained by the smartphone 10. For example, when the user captures an image of an object of interest using the camera function of the smartphone 10, the image data obtained by capturing the image becomes the clip information.

[0024] For example, although not shown, the interest collection means 11 displays a button for calling up a camera function on the screen of the smartphone 10. When the user taps the button, the interest collection means 11 calls up the camera function of the smartphone 10 as shown in FIG. 2(a) and makes it ready to take a photo. The user places an object of interest (a shirt in the example of the figure) in a photo frame 101 displayed on the screen 100 of the smartphone 10 and taps the photo button 102. When the photo button 102 is tapped, the interest collection means 11 saves image data formed by the camera function of the smartphone 10 as clip information.

[0025] In this way, image data captured by the camera function may be used as clip information, or the user may be prompted to select image data that has already been captured, and the selected image data may be used as clip information.

[0026] Generally, applications executed on the smartphone 10 have a function of sharing information handled by each application with other applications. By using such a sharing function, information handled by an application other than the client program executed on the smartphone 10 may be used as clip information.

[0027] An example of another application is a browser. A user shares a website being viewed in the browser with a client program (interest collecting means 11) using a sharing function. The interest collecting means 11 saves website information obtained from the browser (for example, a URL and information such as character data (HTML format, etc.) and moving images pointed to by the URL) as clip information.

[0028] Of course, the client program (interest collecting means 11) may obtain information handled by an application other than a browser using a sharing function and use that information as clip information. For example, in the case of a social networking service (SNS) application, a user may use the sharing function to have the client program (interest collecting means 11) share articles and comments viewed using the application, information about other users connected on the SNS, and the like, as subjects of interest. The client program (interest collecting means 11) then saves those articles, comments, and the like as clip information.

[0029] The behavioral history is information that indicates the user's behavior in the real world and can be detected by the smartphone 10. A specific example of the behavioral history is information that indicates that the user has visited a real store. If the current location obtained by the GPS of the smartphone 10 is within a certain range from the location of the store, the interest gathering means 11 considers that the user has visited that store. Then, the interest gathering means 11 saves information that indicates the stores that the user has visited, the date and time of the visit, etc. as the behavioral history.

[0030] A purchase history is information about products and services that a user has actually purchased. For example, as shown in FIG. 2(b), the interest gathering means 11 displays a browser 103 on the screen 100 of the smartphone 10. When a user operates the browser 103 to browse an online shopping site, if the user taps a purchase button 104 and actually makes a purchase, the interest gathering means 11 saves, as a purchase history, information such as the "product targeted by the purchase button 104" and the "date and time of purchase." Of course, the purchase history does not necessarily have to be obtained through the browser 103, and may be obtained, for example, through a dedicated application for accessing the online shopping site.

[0031] The interest collecting means 11 associates the collected clip information, behavioral history, and purchase history with the user and transmits them to the server device 20. The transmitted clip information, behavioral history, and purchase information are recorded in the storage device of the server device 20 in association with the user who owns the smartphone 10 that sent the information. The interest collecting means 11 can also delete already collected clip information, etc., in response to a user operation. Specifically, the interest collecting means 11 displays already registered clip information via a user interface displayed on the smartphone 10, and allows the user to select clip information, etc. to be deleted in response to a user operation. The interest collecting means 11 then notifies the server device 20 that the selected clip information, etc., is to be deleted. The server device 20 then deletes the clip information, etc.

[0032] The AR information presenting means 12 transmits the clip information stored by the interest collecting means 11 based on the user's operation to the server device 20, and presents the AR information received from the server device 20.

[0033] AR information is information about an object captured as image data stored as clip information. As shown in FIG. 2(c), the image data of the shirt captured in FIG. 2(a) is presented on screen 100 together with the image data 105 and AR information 106 including the shirt's manufacturer, brand name, main sales locations, price, etc. In this way, by simply capturing an image of an object that interests the user, the user can obtain AR information about that object. A method for obtaining AR information corresponding to clip information in server device 20 will be described later.

[0034] The recommendation information presentation means 13 transmits the current location acquired by the GPS of the smartphone 10 to the server device 20, and item information and store information of locations close to the current location are sent from the server device 20 as recommendation information, which is then displayed on the screen 100. The recommendation information is item information and store information that are presumed to be of interest to the user.

[0035] As shown in Figure 2(d), suppose that a user is carrying a smartphone 10 and comes near a store or location that offers a product, service, or event that the user is likely to be interested in. At this time, the recommendation information presentation means 13 displays on the screen 100, as recommendation information transmitted from the server device 20, item information 107 (information about a shirt) that is likely to interest the user and store information 108 that actually sells the product that is the subject of the item information.

[0036] The figure shows item information 107 relating to shirts that are likely to interest the user (or the shirts themselves that have been identified as targets of interest by the interest gathering means 11). Also shown is store information 108 relating to actual stores that sell the products shown in the item information 107.

[0037] The recommendation information presentation means 13 presents the recommendation information on the screen 100 when a store or the like that sells products or the like that are likely to interest the user is within a predetermined range from the location of the smartphone 10. The recommendation information may be presented once or multiple times. Details of the transmission and reception of the recommendation information to and from the server device 20 will be described later.

[0038] Using the smartphone 10 described above, item information regarding products etc. that the user is likely to be interested in, and store information that sells those products etc., can be provided to the user near the stores that sell those products etc.

[0039] Generally speaking, a user operates the smartphone 10 to come across an object of interest among various pieces of information. However, it is rare for the user to take immediate action to actually interact with the object of interest. In some cases, the user may forget that they were interested. In other words, there is a time gap between when the user becomes interested in a product and when they actually interact with that product. For this reason, it may happen that the user does not actually interact with the product despite having an interest in it.

[0040] However, with the smartphone 10, the above-described recommendation information is provided when the user is near a store or the like that sells products or the like that the user is interested in. This can evoke memories of products or the like that the user was interested in in the past, and can strongly motivate the user to actually touch the products or the like.

[0041] From the user's perspective, there is an advantage in that the user can easily come into contact with products etc. that the user has been interested in in the past while carrying the smartphone 10. Also, from the viewpoint of regarding the recommendation information as an advertisement for products etc., it is possible to increase the possibility that the user will become interested and actually visit a store etc. and purchase the product etc. Therefore, compared to advertisements provided by conventional browsers etc., it has an excellent advertising effect that is more likely to lead to the user's purchasing behavior.

[0042] The server program is a program that causes the server device 20 to function as interest information generation means 21, distance calculation means 22, recommendation information determination means 23, recommendation information provision means 24, distance correction means 25, and AR information search means 26.

[0043] The interest information generation means 21 generates interest information for each of the user information, item information, and store information. The interest information is an n-dimensional vector in which each of a plurality of adjectives is quantified. The adjectives are selected to characterize the user information, item information, and store information. Quantifying an adjective means assigning a discrete or continuous numerical value according to the degree of the adjective.

[0044] A method for generating interest information will be described with reference to Fig. 3. When interest information is generated from item information and store information, it is performed, for example, as follows.

[0045] For example, an information processing device (e.g., a personal computer) separate from the server device 20 displays a form on a screen for inputting interest information. A person then looks at the name, description, price, and image that make up the item information, determines the adjectives and their numerical values ​​that should be applied to the product or other item that the item information targets, and inputs them into the form. Through this manual process, interest information consisting of vectors corresponding to the number of adjectives for each item information is formed in the information processing device. Interest information is formed in a similar manner for store information.

[0046] In the example shown in FIG. 3, item information I1 is one of the multiple pieces of item information stored in server device 20, related to a shirt, and is composed of the brand name, manufacturer name, price, retailer, product overview, image data of the shirt, etc. Based on this item information I1, adjectives that characterize the item information I1 are appropriately selected and quantified. In this example, the adjectives are "natural material," "simple," "shape," "feminine," "comfortable," and "shapeless." These adjectives are quantified to represent their respective degrees. For example, if the quantification is on a five-point scale from 1 to 5, each adjective is quantified as "3, 4, 5, 3, 4, 4." The higher the number, the stronger the degree of the adjective. For example, for "natural material," a higher number indicates that the shirt is made of natural materials. For "shape," a higher number indicates that the shape is better.

[0047] Store information S1 is one of the multiple store information stored in server device 20, and is related to a specific store, and is composed of store name, business type, price range, location, store overview, image data of the store exterior, etc. Based on this store information S1, interest information Is1 is created. In this example, the adjectives are "natural material," "natural," "simple," "feminine," and "soft." Each adjective is then quantified as "3, 4, 4, 3, 4."

[0048] In the above example, different adjectives are used for the item information and the store information, but they may be the same. Also, common adjectives may be used for multiple pieces of item information, or different adjectives may be used for each piece of item information. The same applies to store information. Also, there is no particular limit to the number of adjectives that make up the interest information. Different numbers of adjectives may be used for the item information and the store information, or the same number of adjectives may be used. Furthermore, the same number of adjectives may be used for multiple pieces of item information, or different numbers of adjectives may be used for each piece of item information. The same applies to store information.

[0049] The information processing device transmits the interest information thus generated to the server device 20 via a communication means such as the Internet or a removable storage medium, and the server device 20 stores the received interest information in a storage device. Of course, the interest information may be generated by the server device 20 or by any information processing device other than a personal computer, such as a tablet.

[0050] Furthermore, interest information does not necessarily have to be generated manually, but may be generated by machine learning. For example, machine learning is performed using the manually generated interest information as training data and the item information and store information that are the source of the training data as input values ​​to generate a learning model. By providing item information and store information for which interest information is unknown to the learning model generated in this way, interest information can be obtained.

[0051] Next, a method for forming interest information about a user will be described with reference to Fig. 4. As described above, the storage device of the server device 20 records the clip information, behavior history, and purchase history transmitted from the interest collection means 11 of the smartphone 10. The interest information formation means 21 forms interest information about the user from the clip information, behavior history, and purchase information about the user.

[0052] As described above, clip information mainly contains image data relating to products, stores, events, etc. that the user is interested in. The interest information generating means 21 matches the image data contained in such clip information with the image data contained in the item information and store information, and extracts the matching item information and store information. Since a known method can be used to match image data, details will be omitted. However, this can be done by extracting feature amounts from the image data, comparing the feature amounts, and determining whether the difference is equal to or less than a predetermined value.

[0053] In addition to extracting item information and the like from such image data, for example, the interest information forming means 21 may extract item information and store information by searching for information consisting of characters such as product names contained in the item information and store information that matches information consisting of characters such as product names contained in the clip information.

[0054] The interest information generation means 21 extracts item information and store information from the behavior history and purchase history in the same way as it extracts item information and store information from such clip information. In the example shown in the figure, item information I2 is extracted from the clip information, store information S1 from the behavior history, and item information I3 from the purchase history.

[0055] The interest information generating means 21 generates interest information for a user by performing a predetermined calculation on the interest information generated for each extracted item information and store information as described above. For example, the predetermined calculation may involve averaging the obtained interest information for each element, and using the average value as an element to generate a vector for the user's interest information.

[0056] In the example shown in the figure, a predetermined calculation is performed on interest information Ii2, interest information Ii3, and interest information Is1. If each element (a quantified adjective) of interest information Ix is a(x)i (i = 1 to n; the maximum number of elements among all interest information), it can be expressed as follows: Ii2={a(i2)1,a(i2)2,…,a(i2)n} Ii3={a(i3)1,a(i3)2,…,a(i3)n} Is1={a(s1)1,a(s1)2,…,a(s1)n} The interest information Iu for a user can be calculated as follows by taking the average of each element of the interest information as a predetermined calculation. Iu={average of a(i2)1·a(i3)1·a(s1)1, average of a(i2)2·a(i3)2·a(s1)2, …, the average of a(i2)n·a(i3)n·a(s1)n}

[0057] Note that interest information related to item information and store information may use different adjectives or may contain different numbers. In such cases, a predetermined calculation is performed between elements with the same adjective. For example, if "a(i2)1" is an element that quantifies "natural materials," then "a(i3)1" and "a(s1)1" are also elements that quantify "natural materials." Furthermore, if interest information Ii3 does not contain an element that quantifies "natural materials," the value is appropriately supplemented, such as by setting "a(i3)1" to zero.

[0058] The interest information forming means 21 is a means for enabling the server device 20 to realize various functions, such as the function of inputting interest information via a form on an information processing device as described above, the function of transmitting that interest information to the server device 20, the function of forming the learning model as described above, the function of obtaining interest information using the learning model, and the function of forming interest information about the user from clip information, etc.

[0059] The distance calculation means 22 calculates a first distance between the interest information about the user and the interest information about the item information, and a second distance between the interest information about the user and the interest information about the store information. The distance between the interest information is a measure representing the proximity between two pieces of interest information, and can be, for example, Euclidean distance or cosine similarity.

[0060] For example, as shown in Figure 5, if interest information is treated as a two-dimensional vector, interest information Iu, Ii1-Ii4, and Is1-Is3 for user, item, and store information can be plotted on a two-dimensional plane. First distances d11-d14 (only d11 is shown) are the distances from interest information Iu for user information to interest information Ii1-Ii4 for item information. Second distances d21-d23 (only d21 is shown) are the distances from interest information Iu for user information to interest information Is1-Is3 for store information.

[0061] User interest information Iu is formed based on clip information and other items that interest the user. Therefore, the shorter the first distance between the item information targeted by each of interest information Ii1 to Ii4, the more likely the user will be interested in that item information. Similarly, the shorter the second distance between the store information targeted by each of interest information Is1 to Ii3, the more likely the user will be interested in that store information.

[0062] If the first distance satisfies a predetermined condition, the recommendation information determination means 23 determines that the recommendation information recommends item information corresponding to the first distance to the user. Also, if the second distance satisfies a predetermined condition, the recommendation information determination means 23 determines that the recommendation information recommends store information corresponding to the second distance to the user.

[0063] The predetermined condition may be that the first distance is smaller than a threshold value determined for the first distance. Alternatively, the predetermined condition may be that the first distance falls within a predetermined rank from the top when the first distances are sorted in ascending order. A similar predetermined condition may be set for the second distance. Note that the predetermined condition applied to the first distance and the predetermined condition applied to the second distance may be the same or different.

[0064] For example, the predetermined condition is that the first distance and the second distance are equal to or less than a threshold value Th (within a circle with a radius of Th). In this case, the first distance d11 and the second distance d21 satisfy the predetermined condition.

[0065] Furthermore, if the predetermined condition is that the first distance must be within the top two in ascending order of the first distance, then the first distances d11, d12, d13, and d14 are arranged in ascending order. Of these, the top two, first distances d11 and d12, satisfy the predetermined condition. If the same predetermined condition is set for the second distance, second distances d21 and d22 satisfy the predetermined condition.

[0066] Then, taking the first distance d11 as an example of what meets the predetermined condition, the recommendation information determination means 23 determines the item information I1 that is the source of the interest information Ii1 corresponding to the first distance d11 as the recommendation information. Similarly, taking the second distance d21 as an example of what meets the predetermined condition, the recommendation information determination means 23 determines the store information S1 that is the source of the interest information Is1 corresponding to the second distance d21 as the recommendation information.

[0067] The recommendation information providing means 24 provides the recommendation information to the smartphone 10 carried by the user. There are no particular limitations on the timing for providing the recommendation information, but it is preferable that the recommendation information be provided when the location information of the item information or store information included in the recommendation information is within a predetermined range from the current location of the smartphone 10.

[0068] As shown in FIG. 6, the recommendation information providing means 24 receives the position P of the smartphone 10 sent from the recommendation information presenting means 13 of the smartphone 10 .

[0069] Meanwhile, the recommendation information presentation means 13 searches for store information obtained as recommendation information by the recommendation information determination means 23 that exists within a predetermined range C centered on the position P whose store location information was received from the smartphone 10. As a result, it is assumed that store information S1 is found. The recommendation information provision means 24 transmits the search result, store information S1, to the smartphone 10 as recommendation information.

[0070] In this way, even if there are multiple stores (store information S1, store information S2) within a predetermined range C from the smartphone 10, only the store information S1 that is likely to interest the user is provided to the user's smartphone 10.

[0071] The distance correction means 25 corrects the first distance and the second distance based on the user's interests, etc. The corrected first distance and second distance are referred to as the first corrected distance and the second corrected distance. The first corrected distance is obtained by correcting the first distance so that it becomes smaller the higher the user's interest in the item information that was the basis for calculating the first distance. The second corrected distance is obtained by correcting the second distance so that it becomes smaller the higher the user's interest in the store information that was the basis for calculating the second distance.

[0072] A specific example of such correction will be described with reference to Fig. 7. Distance correction means 25 corrects first distances d11 to d13 based on image data of item information stored as clip information and purchase history. Correction of first distance d11 will be described below, but the same applies to first distances d12 to d13.

[0073] The distance correction means 25 calculates the similarity between the image data a of the item information that is the basis for calculating the first distance d11 and the image data b and c of the item information I2 and I3 stored as clip information. The similarity of the image data can be obtained by a known method, so a detailed explanation will be omitted. In this case, the similarity between the image data a and the image data b and the similarity between the image data a and the image data c are obtained, and for example, the highest similarity is selected.

[0074] Furthermore, the distance correction means 25 searches the storage device for the number of purchases of the item corresponding to the item information I1 based on the user's purchase history.

[0075] The distance correction means 25 defines a function f that uses the similarity of these images and the number of purchases as parameters and outputs a coefficient α that reduces the first distance d11 as the similarity increases and the number of purchases increases. The distance correction means 25 then calculates the first corrected distance D11 by multiplying the first distance d11 by the coefficient α obtained by applying the image similarity and the number of purchases to the function f. The first corrected distances D12 to D13 can be obtained in the same manner as for the first distances d12 to d13.

[0076] For example, suppose that the similarity between image data b or image data c and image data a is high. Since image data b or image data c was stored as clip information that the user was interested in, it is considered that the item of image data a is likely to interest the user. When the similarity of the images is high in this way, a coefficient α that reduces the first distance d11 is calculated using the function f. Furthermore, when the number of purchases is high, it is considered that the item information of the purchase target is likely to interest the user. When the number of purchases is high in this way, a coefficient α that reduces the first distance d11 is calculated using the function f.

[0077] Next, we will explain the correction of the second distance by the distance correction means 25. The distance correction means 25 corrects the second distances d21 to d24 based on the image data of the store information stored as clip information and the behavior history. Below, we will explain the correction of the second distance d21, but the same applies to the second distances d22 to d24.

[0078] The distance correction means 25 calculates the similarity between the image data d of the store information that is the basis for calculating the second distance d21 and the image data e of the store information S1 stored as clip information. The similarity of the image data can be obtained by a known method, so a detailed explanation will be omitted. If there is multiple store information as clip information, the highest similarity is selected in the same manner as in the first distance correction.

[0079] Furthermore, the distance correction means 25 searches the storage device for the number of visits to the stores corresponding to the store information S1 to S3 based on the user's behavior history.

[0080] The distance correction means 25 defines a function g that uses the similarity of these images and the number of visits as parameters and outputs a coefficient β that reduces the second distance d21 as the similarity and the number of visits increase. The distance correction means 25 then calculates the second corrected distance D21 by multiplying the second distance d21 by the coefficient β obtained by applying the similarity of the images and the number of visits to the function g. The second corrected distances D22 to D24 can be obtained in the same manner as above.

[0081] For example, suppose that the similarity between image data d and image data e is high. Since image data e was stored as clip information that the user was interested in, it is considered that the store information of image data d is likely to interest the user. When the similarity of the images is high in this way, a coefficient β that reduces the second distance d21 is calculated using function g. Furthermore, when the number of visits is high, it is considered that the store information about the visited store is likely to interest the user. When the number of visits is high in this way, a coefficient β that reduces the second distance d21 is calculated using function g.

[0082] The function f is not limited to the above. For example, only the image similarity may be used as a parameter, or only the number of purchases may be used as a parameter. The function g is not limited to the above. For example, only the image similarity may be used as a parameter, or only the number of visits may be used as a parameter. Alternatively, image data of item information or store information that was added as clip information relatively recently may be used. Furthermore, the function may be fixed, or may be obtained by machine learning.

[0083] By performing the correction as described above, the interest information related to the item information and the store information becomes closer in first distance or second distance to the user's interest information Iu as the user is more likely to be interested in the interest information. The recommendation information determination means 23 determines the recommendation information based on whether the first corrected distance and the second corrected distance, which are the results of the correction by the distance correction means 25, satisfy predetermined conditions.

[0084] For example, as shown in Fig. 5, the first corrected distance D11 (corrected first distance d11) for the interest information Ii1 is outside the threshold Th, and the information is not determined to be recommended information. Also, the first corrected distance D12 (corrected first distance d12) for the interest information Ii2 is within the threshold Th, and the information is determined to be recommended information. By correcting the first distance and the second distance in this way, information that is likely to interest the user is more likely to be determined as recommended information by the recommendation information determination means 23, and conversely, information that is unlikely to interest the user is not determined as recommended information by the recommendation information determination means 23.

[0085] The distance correction means 25 performs correction after determining the first distance and the second distance. It also performs correction when clip information, behavioral history, and behavioral history about the user are added, changed, or deleted. Addition, change, or deletion of clip information, etc. about the user means that the user's interests, etc. change. In other words, the first distance and the second distance are corrected according to changes in the user's interests, etc. Therefore, the distance correction means 25 can provide recommendation information that follows changes in the user's interests, etc.

[0086] The AR information search means 26 receives image data that is one of the clip information sent from the smartphone 10, identifies the object captured in the image data, and further sends information about the object to the smartphone 10 (AR information presentation means 12) as AR information.

[0087] The process of obtaining AR information from image data will be described with reference to Figure 8. Assume that N items of item information are stored in the storage device of server device 20. Objects are extracted from the image data included in each item information by image processing, and their features are extracted. Note that interest information is generated for each of the N items of item information by interest information generation means 21.

[0088] Next, the AR information search means 26 narrows down the item information using the user's interest information Iu. Specifically, the AR information search means 26 calculates the distance between the user's interest information Iu and the interest information Ii for each item information. This distance can be obtained in the same manner as described for the distance calculation means 22. In other words, the AR information search means 26 targets the item information and store information determined to be recommended information as shown in FIG. 5 for subsequent processing.

[0089] Next, the AR information search means 26 generates interest information Iimg for the image data a using the interest information generation means 21. Then, the AR information search means 26 calculates the distance between each piece of interest information Ii of the previously narrowed-down item information and the interest information Iimg. This distance can be obtained in the same manner as described for the distance calculation means 22. The AR information search means 26 targets item information whose distance is equal to or less than a predetermined value for subsequent processing. As a result, the item information is narrowed down to M (N>M).

[0090] Next, the AR information search means 26 receives the image data a (clip information) from the smartphone 10, performs image analysis to extract an object captured in the image data a, and extracts a feature amount A for the object. These image analyses and extraction of the feature amount of the object are well known, so detailed explanations will be omitted.

[0091] The AR information search means 26 performs pattern matching between each of the M feature amounts of the image data included in the item information and the feature amount A of the image data a. Pattern matching between these feature amounts of image data is a known method, and therefore a detailed description thereof will be omitted.

[0092] As a result of the pattern matching, the AR information search means 26 obtains item information Ib including image data of feature amount B that is pattern-matched with feature amount A. The AR information search means 26 transmits the item information Ib to the smartphone 10 as AR information.

[0093] In this way, the AR information search means 26 can search for item information Ib containing image data that matches the image data a based on the feature amount of the image data a. This allows the user to know item information such as the product name and manufacturer related to an object of interest simply by capturing an image of the object.

[0094] Furthermore, before performing pattern matching of the image data features, the AR information search means 26 narrows down the number of pieces of item information to be pattern-matched using the interest information Iu and interest information Iimg. This makes it possible to significantly reduce the amount of calculation required for pattern matching. Furthermore, because the interest information Iu and interest information Iimg are closely related to the user's interests, the item information narrowed down by these information is highly likely to interest the user, making it possible to provide the user with useful item information (AR information).

[0095] Although the case where one item information Ib is obtained as a result of pattern matching has been described, the present invention is not limited to this case. Even if multiple item information is obtained as a result of pattern matching, all or part of the multiple item information may be used as AR information. Furthermore, although the AR information search means 26 targets item information, it can also process store information in the same manner. Furthermore, although narrowing down is performed using interest information Iu and interest information Iimg, such narrowing down is not necessary.

[0096] The flow of processing in the recommendation system 1 configured from the smartphone 10 and server device 20 described above will be described with reference to FIG.

[0097] The interest collecting means 11 forms clip information etc. based on the user's operation of the smartphone 10 and transmits it to the server device 20 (step S1). In the server device 20, the AR information searching means 26 searches for AR information based on the clip information etc. received from the smartphone 10 and transmits it to the smartphone 10 (step S2). In the smartphone 10, the AR information presenting means 12 displays the AR information on the screen of the smartphone 10.

[0098] When the smartphone 10 transmits information to the server device 20 to delete the clip information, etc., the server device 20 deletes the clip information, etc., of the user.

[0099] The server device 20 generates interest information for the user of the smartphone 10 that sent the clip information (step S3). Note that the interest information generation means 21 also generates interest information for item information and store information at any timing.

[0100] Next, in the server device 20, the distance calculation means 22 calculates the first distance and the second distance between the item information and the interest information of the store information (step S4), and the distance correction means 25 corrects the first distance and the second distance (step S5). Then, the recommendation information determination means 23 determines the recommendation information based on the first distance and the second distance (step S6).

[0101] The process leading to the determination of such recommendation information is performed every time clip information or the like is added from the smartphone 10 to the server device 20 or every time clip information or the like is deleted.

[0102] In this way, when the user operates the smartphone 10 and an object that the user is interested in is formed as clip information or the like (step S1), AR information about the object itself is provided to the user (step S2). On the other hand, although it is determined from the clip information or the like what recommendation information is likely to interest the user (step S6), it is not immediately transmitted to the user's smartphone 10.

[0103] Meanwhile, while the user is moving around with the smartphone 10, the recommendation information presenting means 13 periodically transmits location information of the smartphone 10 to the server device 20 (step S7). The server device 20 transmits store information and item information within a predetermined range from the smartphone 10 to the smartphone 10 as recommendation information by the recommendation information providing means 24 (step S8).

[0104] The recommendation system 1, server device 20, and server program executed on the server device 20 (hereinafter referred to as recommendation system 1, etc.) of this embodiment described above form interest information for users, item information, and store information, and determine recommendation information based on whether the first distance and second distance calculated from the information meet specified conditions, and provide the recommendation information to the user.

[0105] Because user interest information is a numerical representation of the user's interests, etc., it can be said that the user is likely to be interested in item information or store information for which the first distance and second distance meet predetermined conditions. Therefore, compared to conventional technologies that recommend items to users based only on image data, it is possible to provide the user with item information and store information that more appropriately reflects the user's interests, etc., as recommended information.

[0106] Furthermore, the recommendation system 1 etc. corrects the first distance and the second distance so that the first distance and the second distance become smaller as the degree of interest etc. of the user increases. This makes it possible to provide recommendation information that follows changes in the user's interest etc.

[0107] Furthermore, the recommendation system 1 etc. provides the recommendation information to the client device when the location information of the recommended item information or store information is within a predetermined range from the current location of the smartphone 10. This allows the user to recall from memory products etc. that the user has been interested in in the past while carrying the smartphone 10, and strongly motivates the user to visit a store to actually touch the products etc. while they are out and about. Furthermore, from the perspective of regarding the recommendation information as an advertisement for products etc., it has a superior advertising effect in that it is more likely to lead to user purchasing behavior than advertisements provided by conventional browsers etc.

[0108] Furthermore, the recommendation system 1 etc. forms interest information based on the clip information, behavioral history, and purchase history transmitted from the smartphone 10. This makes it possible to provide recommendation information that is more likely to interest the user.

[0109] Furthermore, the recommendation system 1 etc. provides the smartphone 10 with AR information corresponding to the image data captured as clip information. When searching for this AR information, the item information and store information that are the results narrowed down based on the interest distance (i.e., the results determined as recommendation information) are targeted. This narrows down the large amount of item information and store information, making it possible to significantly reduce the amount of calculation required when pattern-matching the image data captured as clip information with the image data contained therein.

[0110] The client program executed on the smartphone 10 of the present embodiment described above collects clip information, behavioral history, and purchase history, and transmits them to the server device 20. Then, recommendation information that is likely to interest the user based on the clip information, etc., is received from the server device 20 and displayed on the screen.

[0111] This allows item information and store information that more appropriately reflects the user's interests, etc. to be provided to the user as recommendation information.

[0112] Second Embodiment In the first embodiment, when an item or store is located within a predetermined range from the current location of the smartphone 10, recommendation information is transmitted from the server device 20 to the smartphone 10, and the recommendation information is displayed on the smartphone 10. However, the recommendation information may be transmitted to the smartphone 10 in advance. The recommendation system according to this embodiment will be described below with reference to FIG. 10. Note that a description of the same configuration and processing as in the embodiment will be omitted.

[0113] The server device 20 stores map information representing the location information of various buildings, roads, etc. in a storage device. This map information and areas are shown in FIG. 10. Specifically, store S1 and store S2 are shown from the map information. An area refers to a certain range on a map. In the example shown in the same figure, three circular areas X, Y, and Z are shown. The coordinates of each area are the center position of each area.

[0114] The server device 20 also includes a behavior prediction means for predicting areas where the user will arrive in the future. The behavior prediction means will be described in detail later.

[0115] Assume that the behavior prediction means predicts that the user will arrive at area Y. In this case, the recommendation information providing means 24 transmits, to the smartphone 10, before the user arrives at area Y, item information and store information that contain location information that is determined to be within area Y and that is recommended information for the user. In the example shown in the figure, the recommendation information providing means 24 transmits recommendation information about store S1 to the smartphone 10.

[0116] After the recommendation information is acquired in advance in this manner, when the recommendation information display means of the smartphone 10 detects that the smartphone 10 has actually entered the area X, it displays the previously received recommendation information on the screen of the smartphone 10.

[0117] A method for estimating user behavior using the behavior prediction means will be described below with reference to Fig. 11. The behavior prediction means obtains a user movement vector a, a user past vector b, and a movement sample vector c for each road.

[0118] The user's movement vector a represents the direction in which the user is currently moving. Specifically, the behavior prediction means samples the location information transmitted from the smartphone 10 at appropriate time intervals. FIG. 11(a) shows a situation in which the behavior prediction means has obtained seven pieces of location information for the smartphone 10 as a result of the user's behavior. The behavior prediction means calculates the difference between two consecutive pieces of location information to obtain movement vectors V1 to V6. The behavior prediction means selects movement vectors by going back from the most recent movement vector until the total length of the movement vectors reaches, for example, 1 km. As a result, it is assumed that movement vectors V3 to V6 are selected. The behavior prediction means then combines the selected movement vectors to obtain the user's movement vector a.

[0119] The user's past vector b represents the likely direction of travel based on the user's past behavior. Specifically, it is calculated from the difference between the area the user frequently visits and the user's current location. As shown in Figure 11(b), the behavior prediction means identifies the areas the user frequently visits. For example, for each of multiple areas on the map, the number of times the user actually visited and the length of time they stayed there are obtained. This can be obtained from the location information and map information obtained from the user's smartphone 10. As a result, it is assumed that the heat area HA that the user most frequently visited is obtained from the user's past behavior. The behavior prediction means calculates the past vector b from the difference between the user's current location information and the location information of the center of the heat area HA.

[0120] The movement sample vector c represents a direction obtained from the relationship between the flow of people on a road and the direction the user is heading. Specifically, the flow of people is stored as a vector for major roads from map information. For example, as shown in Figure 11(c), on road R, there is a vector p that represents the flow of people from one point P2 to another point P1. Since vectors representing the flow of people on such roads vary depending on the time of day, it is preferable to obtain them for each time period. Then, if the vector from point P2 to the location of the smartphone is defined as vector ap, the behavior prediction means calculates the difference between vector p and vector ap to obtain movement sample vector c.

[0121] The behavior prediction means then calculates a predicted vector n indicating the user's destination by multiplying the movement vector a, the past vector b, and the movement sample vector c, each of which is weighted, by the movement vector a, as shown in the following formula. n=|a|×next next=Weight1×a + Weight2×b + Weight3×c Weight1 to Weight3 are weights that may be set arbitrarily or obtained by the least squares method, etc. Also, |a| is the magnitude of the movement vector a.

[0122] In other words, vector n is a combination of movement vector a, which is the user's current direction of movement, past vector b, which is likely to be a movement direction based on past behavior, and movement sample vector c, which takes into account the flow of people on road R. Rather than simply considering the user's movement vector a, it is possible to predict the direction the user will head with greater accuracy because it takes into account past behavior and the flow of people on surrounding roads.

[0123] After calculating vector n in this way, the behavior prediction means identifies the area that is located ahead of vector n from area X where the user is located in the map information. In the example of Fig. 10, the area that is located ahead of vector n is area Y, so area Y is estimated to be the area that the user is likely to reach.

[0124] Here, if recommendation information is transmitted in advance without predicting user behavior, all areas adjacent to area X where the user is present may be considered as targets for transmitting recommendation information in advance since the user may potentially arrive there. In the example shown in FIG. 10, area Z adjacent to area X is also a target for transmitting recommendation information in advance. In such a case, recommendation information for store S2 included in area Z, which the user will not actually arrive at, will also be transmitted to smartphone 10, which may result in wasted communication traffic.

[0125] However, according to the recommendation system of this embodiment, an area Y that the user is likely to arrive at is estimated, and recommendation information for a store S1 in the area Y is transmitted to the smartphone 10 before the user arrives at the area Y. This makes it possible to minimize the amount of recommendation information transmitted to the user's smartphone 10, thereby reducing the amount of communication traffic. [Explanation of symbols]

[0126] 1... Recommendation system, 10... Smartphone, 11... Interest collection means, 12... Information presentation means, 13... Recommendation information presentation means, 20... Server device, 21... Interest information formation means, 22... Distance calculation means, 23... Recommendation information determination means, 24... Recommendation information provision means, 25... Distance correction means, 26... Information search means

Claims

1. A client device owned by a user; a server device that provides the client device with recommendation information that is recommended to the user from among item information and store information, The server device an interest information generating means for generating interest information that is a vector obtained by quantifying a plurality of adjectives that represent the user, the item information, and the store information; distance calculation means for calculating a first distance between the interest information about the user and the interest information about the item information, and a second distance between the interest information about the user and the interest information about the store information; a recommendation information determination means for determining, if the first distance satisfies a predetermined condition, that the item information corresponding to the first distance is recommendation information to be recommended to the user, and for determining, if the second distance satisfies a predetermined condition, that the store information corresponding to the second distance is recommendation information to be recommended to the user; a recommendation information providing means for providing the recommendation information to the client device; A recommendation system characterized by:

2. The recommendation system according to claim 1, The server device includes a distance correction means that corrects the first distance so that it becomes smaller as the user's interest in the item information increases, and corrects the second distance so that it becomes smaller as the user's interest in the store information increases. A recommendation system characterized by:

3. The recommendation system according to claim 1 or 2, The recommendation information providing means provides the recommendation information to the client device when the location information of the item information or the store information, which is the recommendation information, is within a predetermined range from the current location of the client device. A recommendation system characterized by:

4. The recommendation system according to any one of claims 1 to 3, the client device comprises an interest collection means for collecting clip information selected by the user from among information obtained by the client device, a behavior history which is information representing the user's behavior that can be detected by the client device, and a purchase history which is information about products and services purchased by the user via the client device, and transmitting the collected information to the server device; The interest information generating means generates the interest information about the user based on the clip information, the behavior history, and the purchase history. A recommendation system characterized by:

5. The recommendation system according to claim 4, the interest collecting means transmits image data captured by the client device as the clip information to the server device; The server device extracting the item information and the store information determined as the recommendation information by the recommendation information determination means; an AR information search means for selecting, from among feature amounts of image data included in the item information and the store information, feature amounts that pattern-match with feature amounts of an object captured in the image data received from the interest collection means, and transmitting the information to the client device as AR information; The client device includes an AR information presenting means for displaying the AR information received from the server device as the clip information related to the captured image data. A recommendation system characterized by:

6. The recommendation system according to any one of claims 1 to 5, the server device includes a behavior prediction means for predicting an area that the user will reach in the future; The recommendation information providing means provides the item information and the store information relating to items and stores present in the area as the recommendation information to the client device before the user arrives in the area. A recommendation system characterized by:

7. A server device that transmits recommendation information, which is item information and store information recommended to a client device owned by a user, to the client device, an interest information generating means for generating interest information that is a vector quantified for each of a plurality of adjectives that express the user, the item information, and the store information; distance calculation means for calculating a first distance between the interest information about the user and the interest information about the item information, and a second distance between the interest information about the user and the interest information about the store information; a recommendation information determination means for determining, if the first distance satisfies a predetermined condition, that the item information corresponding to the first distance is the recommendation information recommending the user, and for determining, if the second distance satisfies a predetermined condition, that the store information corresponding to the second distance is the recommendation information recommending the user; and causing the device to function as a recommendation information providing means for providing the recommendation information to the client device. A server program characterized by:

8. A client device that receives recommendation information, which is item information and store information recommended to a user, from a server device; an interest collection means for collecting clip information selected by the user from among the information obtained by the client device, a behavior history which is information representing the user's behavior that can be detected by the client device, and a purchase history which is information about products and services purchased by the user via the client device, and transmitting the collected information to the server device; The server device functions as a recommendation information display unit that receives and displays the recommendation information transmitted by the server device based on the clip information, the behavior history, and the purchase history. A client program characterized by:

Images