Method, computer device, and computer program for providing recommendation information based on a local knowledge graph

Abstract

To provide a method, computer device and computer program for providing recommendation information based on a local knowledge graph.SOLUTION: A recommendation method includes the steps of: generating a local knowledge graph of a heterogeneous graph structure based on a plurality of items constituting a region; performing node type-dependent embedding in the local knowledge graph using node features that differ from each other; and providing a recommendation item corresponding to each of the node types based on the node type-dependent embedding.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The following description relates to techniques for providing location-based recommendation information. [Background technology]

[0002] With the spread of mobile and ubiquitous infrastructure, various services are evolving into customized services for individuals.

[0003] One of the custom services is to provide information about surrounding places based on the user's location information, preference information, and the like.

[0004] As an example of a place recommendation technology, Patent Document 1 (May 26, 2014) discloses a technology that classifies and recommends places such as restaurants and stores by theme. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Korean Patent Publication No. 10-2014-0062684 Summary of the Invention [Problem to be solved by the invention]

[0006] By using a single local knowledge graph, multiple items (POIs, users, documents) related to the local area can be recommended.

[0007] By utilizing structural information resulting from the relationships between items, it is possible to provide an advanced recommendation system. [Means for solving the problem]

[0008] A recommendation method executed on a computer device, the computer device including at least one processor configured to execute computer-readable instructions contained in a memory, the recommendation method including: generating, by the at least one processor, a local knowledge graph with a heterogeneous graph structure based on a plurality of items that constitute a region; performing, by the at least one processor, embedding by node type using node features that differ from each other in the local knowledge graph; and providing, by the at least one processor, recommended items corresponding to each of the node types based on the embedding by node type.

[0009] According to one aspect, the performing step may include extracting one node type as a target node and at least one other node type as a context node from the regional knowledge graph, and modeling a relationship graph between the target node and the context node.

[0010] According to another aspect, the modeling step may use multimodal data including image, text, and categorical data to model the relationship between the target node and the context node.

[0011] According to another aspect, the modeling may include extracting positive samples and negative samples by random walk-based sampling.

[0012] According to another aspect, the modeling may apply edges with different weights to a relationship graph between the target node and the context node using a low-rank positive loss.

[0013] According to another aspect, the generating step may generate the regional knowledge graph by configuring POIs, users who visit the POIs, and documents produced for the POIs as nodes of different types, and configuring relationships between the nodes as edges.

[0014] According to another aspect, the executing step may include extracting follow-based user embeddings using a user-user graph in which each user is set as a node, information related to the user is set as a user node feature, and relationships between users are set as edges, and extracting visit-based user embeddings using a user-POI graph in which users and POIs visited by the users are set as nodes, information related to the user is set as a user node feature, information related to the POIs is set as a POI node feature, and user preference information for POIs is set as edges.

[0015] According to another aspect, the providing step may include a step of searching for an embedding similar to the source embedding based on the follow-based user embedding to select a follow-based recommended user, a step of searching for an embedding similar to the source embedding based on the visit-based user embedding to select a visit-based recommended user, and a step of arranging and providing the follow-based recommended users and the visit-based recommended users according to a certain criterion.

[0016] According to another aspect, the providing step may include a step of searching for an embedding similar to the source embedding based on the follow-based user embedding and the visit-based user embedding to select a recommended user, and a step of providing the recommended user together with a recommendation reason that shows similar characteristics to the source embedding.

[0017] According to another aspect, the executing step may include extracting POI embeddings using a POI-user graph in which POIs and users who have visited the POIs are set as nodes, information related to the POIs is set as POI node features, information related to the users is set as user node features, and user preference information for the POIs is set as edges.

[0018] According to another aspect, the providing step may include a step of searching for an embedding similar to the source embedding based on the POI embedding to select a recommended POI, and a step of providing the recommended POI together with a recommendation reason that shows similar characteristics to the source embedding.

[0019] According to another aspect, the executing step may include extracting document embeddings using a document-POI graph in which the document and the POI to which the document is registered are set as nodes, information related to the document are set as document node features, information related to the POI are set as POI node features, and document interest information for the POI is set as edges.

[0020] According to yet another aspect, the extracting step may include extracting text embeddings by learning text included in the document using a word embedding model, extracting image embeddings by learning images included in the document using a deep learning model, and learning document nodes reflecting the text embeddings, the image embeddings, and meta information included in the document to obtain a document embedding matrix for the document-POI graph.

[0021] A computer program for causing a computer to execute the recommendation method is provided.

[0022] A computer device is provided, comprising at least one processor configured to execute computer-readable instructions stored in a memory, wherein the at least one processor generates a regional knowledge graph with a heterogeneous graph structure based on a plurality of items that constitute a region, performs embedding for each node type using different node features in the regional knowledge graph, and provides recommended items corresponding to each of the node types based on the embedding for each node type. [Effects of the Invention]

[0023] According to an embodiment of the present invention, it is possible to recommend items related to a region, such as POIs, users, and documents, based on a region knowledge graph. By generating a region knowledge graph with a heterogeneous graph structure consisting of POI nodes, user nodes, and document nodes, it is possible to provide more sophisticated recommendations by taking into account not only the unique features possessed by each item but also the relationships with other nodes connected on the graph.

[0024] According to an embodiment of the present invention, by embedding each node using different types of nodes and structural information according to a heterogeneous graph structure, it is possible to minimize the data sparcity problem, and by using various multimodal data such as images and text, it is possible to maximize the amount of information obtained from each modality. [Brief explanation of the drawings]

[0025] [Figure 1] FIG. 1 illustrates an example of a network environment in accordance with an embodiment of the present invention. [Figure 2] FIG. 1 is a block diagram illustrating an example computer system according to an embodiment of the present invention. [Figure 3]FIG. 2 illustrates example components that may be included in a processor of a computer system in accordance with one embodiment of the present invention. [Figure 4] 1 is a flowchart illustrating an example of a method that can be performed by a computer system in accordance with an embodiment of the present invention. [Figure 5] 1 is an exemplary diagram illustrating a user recommendation modeling process according to an embodiment of the present invention; [Figure 6] 1 is an exemplary diagram illustrating a user recommendation modeling process according to an embodiment of the present invention; [Figure 7] 1 is an exemplary diagram illustrating a user recommendation modeling process according to an embodiment of the present invention; [Figure 8] 1 is an exemplary diagram illustrating a user recommendation modeling process according to an embodiment of the present invention; [Figure 9] 4 is an exemplary diagram illustrating a POI recommendation modeling process according to an embodiment of the present invention; [Figure 10] 4 is an exemplary diagram illustrating a POI recommendation modeling process according to an embodiment of the present invention; [Figure 11] 1 is an exemplary diagram illustrating a document (review) recommendation modeling process according to an embodiment of the present invention; [Figure 12] 1 is an exemplary diagram illustrating a document (review) recommendation modeling process according to an embodiment of the present invention; [Figure 13] 1 is an exemplary diagram illustrating a document (review) recommendation modeling process according to an embodiment of the present invention; [Figure 14] 1 is an exemplary diagram illustrating a document (review) recommendation modeling process according to an embodiment of the present invention; [Figure 15] 1 is an exemplary diagram illustrating a document (review) recommendation modeling process according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0026] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0027] SUMMARY OF THE INVENTION Embodiments of the present invention relate to techniques for providing location-based recommendation information.

[0028] Embodiments including those specifically disclosed herein can recommend multiple items related to a region by utilizing a single local knowledge graph.

[0029] In this specification, an item may refer to a search or recommendation target. The item corresponds to an element that constitutes a region, and in this embodiment, may include points of interest (POIs) located within the region, users who are targets of visiting, using, or purchasing from the POIs, and user-generated content (UGC) documents (e.g., reviews) created for the POIs.

[0030] An item recommendation system according to an embodiment of the present invention may be realized by at least one computer system, and an item recommendation method according to an embodiment of the present invention may be executed by at least one computer system included in the item recommendation system. In this case, a computer program according to an embodiment of the present invention may be installed and executed in the computer system, and the computer system may execute the item recommendation method according to an embodiment of the present invention under the control of the executed computer program. The computer program may be recorded on a computer-readable recording medium in combination with the computer system to cause the computer to execute the item recommendation method.

[0031] FIG. 1 is a diagram showing an example of a network environment in one embodiment of the present invention. The network environment in FIG. 1 shows an example including multiple electronic devices 110, 120, 130, and 140, multiple servers 150 and 160, and a network 170. FIG. 1 is merely an example for explaining the invention, and the number of electronic devices and the number of servers are not limited to those shown in FIG. 1. Furthermore, the network environment in FIG. 1 is merely an example of an environment applicable to this embodiment, and environments applicable to this embodiment are not limited to the network environment in FIG. 1.

[0032] The electronic devices 110, 120, 130, and 140 may be fixed or mobile terminals implemented by computers. Examples of the electronic devices 110, 120, 130, and 140 include smartphones, mobile phones, navigation systems, personal computers (PCs), laptops, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), tablets, game consoles, wearable devices, Internet of Things (IoT) devices, virtual reality (VR) devices, and augmented reality (AR) devices. While FIG. 1 illustrates a smartphone as an example of the electronic device 110, in embodiments of the present invention, the electronic device 110 may represent one of a variety of physical computer systems capable of communicating with other electronic devices 120, 130, and 140 and / or servers 150 and 160 via a network 170 using a substantially wireless or wired communication method.

[0033] The communication method is not limited, and may include not only communication methods using communication networks (for example, a mobile communication network, a wired Internet, a wireless Internet, and a broadcast network) that can be included in network 170, but also short-range wireless communication between devices. For example, network 170 may include any one or more of networks such as a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), and the Internet. Furthermore, network 170 may include any one or more of network topologies including, but not limited to, a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or hierarchical network, etc.

[0034] Each of the servers 150 and 160 may be realized by one or more computer devices that communicate with the plurality of electronic devices 110, 120, 130, and 140 via the network 170 and provide instructions, code, files, content, services, etc. For example, the server 150 may be a system that provides a first service to the plurality of electronic devices 110, 120, 130, and 140 connected via the network 170, and the server 160 may be a system that provides a second service to the plurality of electronic devices 110, 120, 130, and 140 connected via the network 170. As a more specific example, the server 150 may provide a service (such as a recommendation service, for example) targeted by an application, which is a computer program installed and executed in the plurality of electronic devices 110, 120, 130, and 140, as a first service to the plurality of electronic devices 110, 120, 130, and 140. As another example, the server 160 may provide, as a second service, a service of distributing files for installing and executing the above-mentioned application to the multiple electronic devices 110, 120, 130, and 140.

[0035] 2 is a block diagram showing an example of a computer device according to an embodiment of the present invention. Each of the electronic devices 110, 120, 130, and 140 and each of the servers 150 and 160 described above may be realized by a computer device 200 shown in FIG.

[0036] Such a computing device 200 may include a memory 210, a processor 220, a communication interface 230, and an input / output interface 240, as shown in FIG.

[0037] Memory 210 is a computer-readable recording medium and may include random access memory (RAM), read-only memory (ROM), and a persistent mass storage device such as a disk drive. Here, a persistent mass storage device such as a ROM or a disk drive may be included in computer device 200 as a separate persistent storage device distinct from memory 210. Memory 210 may also store an operating system and at least one program code. Such software components may be loaded into memory 210 from a computer-readable recording medium separate from memory 210. Such separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, or a memory card. In other embodiments, software components may be loaded into memory 210 through communication interface 230, which is not a computer-readable recording medium. For example, software components may be loaded into memory 210 of computer device 200 based on a computer program installed by a file received via network 170.

[0038] Processor 220 may be configured to process computer program instructions by performing basic arithmetic, logic, and input / output operations. The instructions may be provided to processor 220 by memory 210 or by communication interface 230. For example, processor 220 may be configured to execute instructions received according to program code stored in a storage device such as memory 210.

[0039] The communication interface 230 may provide a function for the computer device 200 to communicate with other devices (e.g., the above-mentioned storage device) via the network 170. For example, requests, instructions, data, files, etc. generated by the processor 220 of the computer device 200 in accordance with program code stored in a storage device such as the memory 210 may be transmitted to other devices via the network 170 under the control of the communication interface 230. Conversely, signals, instructions, data, files, etc. from other devices may be received by the computer device 200 via the communication interface 230 of the computer device 200 via the network 170. The signals, instructions, data, etc. received via the communication interface 230 may be transmitted to the processor 220 or the memory 210, and files, etc. may be recorded on a storage medium (e.g., the above-mentioned permanent storage device) that the computer device 200 may further include.

[0040] The input / output interface 240 may be a means for interfacing with the input / output device 250. For example, the input device may include a device such as a microphone, keyboard, or mouse, and the output device may include a device such as a display or speaker. As another example, the input / output interface 240 may be a means for interfacing with a device that integrates input and output functions into one, such as a touch screen. The input / output device 250 may be configured as a single device together with the computer device 200.

[0041] Also, in other embodiments, computing device 200 may include fewer or more components than those shown in Figure 2. However, most prior art components need not be explicitly shown in the figures. For example, computing device 200 may be implemented to include at least some of the input / output devices 250 described above, and may further include other components such as a transceiver, a database, etc.

[0042] Specific embodiments of a method and system for providing recommendation information based on a local knowledge graph are described below.

[0043] FIG. 3 is a block diagram illustrating an example of components that may be included in a processor of a computer system in one embodiment of the present invention, and FIG. 4 is a flowchart illustrating an example of an item recommendation method that may be executed by a computer system in one embodiment of the present invention.

[0044] The computer device 200 according to this embodiment may provide a recommendation service to a client through a dedicated application installed on the client or through connection to a web / mobile site associated with the computer device 200 .

[0045] 3, the processor 220 of the computer device 200 may include a graph generator 310, a node embedding unit 320, and an item recommender 330 as components for executing the item recommendation method described below. Depending on the embodiment, the components of the processor 220 may be selectively included or excluded from the processor 220. Also, depending on the embodiment, the components of the processor 220 may be separated or combined to express the functions of the processor 220.

[0046] Such processor 220 and components of processor 220 may control computer device 200 to perform steps included in the item recommendation method described below. For example, processor 220 and components of processor 220 may be implemented to execute instructions from operating system code and at least one program code contained in memory 210.

[0047] Here, the components of the processor 220 may be representations of different functions executed by the processor 220 according to instructions provided by the program code recorded in the computer device 200. For example, the graph generator 310 may be used as a functional representation of the processor 220 that controls the computer device 200 according to the above-mentioned instructions so that the computer device 200 generates a local knowledge graph.

[0048] The processor 220 may read the necessary instructions from the memory 210 loaded with instructions related to the control of the computing device 200. In this case, the read instructions may include instructions for controlling the processor 220 to perform the item recommendation method described below.

[0049] The steps included in the item recommendation method described below may be performed in a different order than that shown in the figures, and some of the steps may be omitted or additional processes may be included.

[0050] Referring to FIG. 4, in operation 410, the graph generator 310 may generate a local knowledge graph with a heterogeneous graph structure based on a plurality of items related to a local area. As shown in FIG. 3, the graph generator 310 may generate a single integrated local knowledge graph by utilizing various cross-domain data, such as POI data 301, user data 302, and document data 303. The POI data 301 is POI metadata that can be extracted based on a big data-based analysis model (context analysis model) or information entered by a POI administrator, and may include the atmosphere, menu, search purpose, industry, price, etc. of the POI. The user data 302 may include user relationship information (e.g., follow data, follower data, etc.) and interest data (e.g., likes, keeps, receipt reviews, etc.). The document data 303 may include images, text, menu information, keyword reviews, etc. included in documents. The local knowledge graph may be broadly composed of three node types, in other words, POI nodes, user nodes, and document nodes. Information such as the POI's industry, price range, price range within the industry, representative menu, atmosphere, and search purpose may be used as POI node features, the user's gender, age, and area of ​​activity may be used as user node features, and the image, text, order price, order menu, visit frequency, and visit purpose included in the document may be used as document node features.

[0051] In step 420, the node embedding unit 320 may embed each node in the local knowledge graph using different node features. The node embedding unit 320 may embed the corresponding node by utilizing different node type features and structural information for each node type. The node embedding unit 320 may extract one node type from the local knowledge graph as a target node (i.e., a pin node), extract at least one node type different from the target node as a context node (i.e., a board node), and embed the target node by modeling the relationship between the target node and the context node.

[0052] The node embedding unit 320 may generate a relationship graph between pin nodes and board nodes by interacting with each other using a graph convolution network (GCN) model (e.g., a PinSAGE-based model) based on a bipartite graph or heterogeneous graph. In this case, the desired graph may be generated using virtual vertices or virtual edges. For example, industry or region vertices may be added to newly define relationships between POIs, and edges may be newly defined based on user data to reduce data sparseness and add trends through collaborative filtering. The node embedding unit 320 may also use various node features in the region domain to model the relationship between the target node and the context node. For example, multimodal data such as images, text, and categorical data may be used.

[0053] The node embedding unit 320 may add sampling logic to improve randomness in order to provide recommendations that match the target domain knowledge when extracting positive samples and negative samples for model training. For example, the node embedding unit 320 may use nodes with dissimilar features as negative samples during a random walk-based sampling process. For example, if POIs are in the same industry but have different metadata, the POIs may be selected as negative samples, and learning may be performed so that POIs with different features in the same industry are spaced apart from each other.

[0054] The node embedding unit 320 may use low-rank positive loss to improve graph modeling. Edges in the local knowledge graph may be weighted differently because they are generated by explicit feedback. By using low-rank positive loss considering the strength of different edges, it is possible to reflect the differences even among positive samples, thereby enabling learning to make highly related samples closer to each other.

[0055] In other words, the node embedding unit 320 can extract necessary nodes and features for each node type from the local knowledge graph, design a graph suitable for the corresponding node type, and then extract embeddings for each node type through graph modeling. That is, the node embedding unit 320 can build a user recommendation model, a POI recommendation model, and a document recommendation model using one local knowledge graph.

[0056] In step 430, the item recommendation unit 330 may provide recommended items corresponding to each node type based on the embedding of each node type. One of the recommended items is for recommending a user, and the user recommendation range may be expanded by using a homogeneous graph-based model as well as a heterogeneous graph-based model as a user recommendation model. To provide meaningful recommended users, high-quality users may be pre-selected and recommended based on reviews, activities, etc. Another of the recommended items is for recommending POIs, and a similarity measure (e.g., Jaccard Distance) may be used to grasp the similarity and similar characteristics between POIs, and POIs may be recommended along with the reason for recommendation. To determine the recommendation quality of the POIs, POIs that meet certain conditions, such as franchise POIs or POIs with no image, may be excluded from the recommendation targets. To provide diverse recommendation results, recommendation targets may be managed to prevent POIs from being repeatedly recommended in the same industry. Yet another recommendation item is a document recommendation, where information extracted from the document (e.g., menu, price, etc.) can be used to recommend documents that are similar not only in terms of embedding but also in terms of actual metadata.

[0057] FIG. 5 is a flowchart illustrating a user recommendation modeling process in one embodiment of the present invention.

[0058] The processor 220 may extract nodes representing users and features of the nodes from the local knowledge graph and use them for user recommendation modeling.

[0059] In step 501, the processor 220 may generate a user-user homogeneous graph, which is a graph showing relationships between follow-based users, and a user-POI heterogeneous graph, which is a graph showing relationships between users and POIs.

[0060] In step 502, the processor 220 may model user embeddings by GCN model-based learning for each of the user-user homogeneous graph and the user-POI heterogeneous graph.

[0061] There are two main types of user recommendation models.

[0062] First, when a user follows another user, under the assumption that the user has similar tastes (interests), the user in the following relationship can be embedded nearby based on a follow-based graph deep learning model that reflects the characteristics of the users in the following relationship in node features.

[0063] 6, the processor 220 may generate the user-user graph 600 using a homogeneous graph represented by nodes and edges of the same type, which can represent follow relationships with users. In other words, to represent relationships between users, the user-user graph 600 may be constructed by setting each user as a node, setting information such as the user's gender, age, area of ​​interest (area of ​​activity), and interest category as node features, and setting follow relationships as edges. The processor 220 may extract user embedding values ​​by training a GCN model on the user-user graph 600. For example, the processor 220 may use one of the GCN models to sample neighbors by random walk, and use the GraphSAGE model, which uses a subgraph (aggregator, mini-batch) as input that is not affected by the graph structure (i.e., topology).

[0064] The processor 220 may provide recommended users corresponding to the area of ​​interest of the source user being targeted by modeling the embedding of the user-user graph 600. Referring to FIG. 7, when a source user embedding 701 is given, the processor 220 may search for user embeddings similar to the source user embedding 701 using a vector search engine 700 and provide the user embeddings to recommended users 702. For example, the processor 220 may select other users who have many documents such as reviews based on the area of ​​interest of the source user as recommendation candidates, and then divide the recommendation candidates into a group of users who are in a follow relationship with the source user and a group of users who have the same interest category as the source user, and recommend users according to the reason for recommendation.

[0065] Second, under the assumption that users who visit the same POI have similar preferences, users with the same POI visit history can be embedded close to each other based on a visit-based graph deep learning model that reflects the characteristics of users and POIs in node features.

[0066] Referring to FIG. 8 , the processor 220 may generate the user-POI graph 800 using a knowledge graph that can reflect specific knowledge, i.e., the context of a regional domain, among heterogeneous graphs that can simultaneously include information about users and POIs. Unlike homogeneous graphs that handle a single node type and edge type, a heterogeneous graph represents a graph using multiple node types and edge types. Among these, a knowledge graph can record knowledge about a given domain in a graph format. A bipartite graph may be used to represent the relationship between users and POIs, and the relationship and weight value for each edge may be expressed using a rating. The user-POI graph 800 may be constructed by setting the user and the POIs visited by the user as nodes, setting information such as the user's gender, age, area of ​​interest, and category of interest as user node features, setting information such as the POI's industry, price range, atmosphere, search purpose, representative menu, popular topics, and receipt keywords as POI node features, and setting the user's preferences for the POI (e.g., visited, liked, kept, etc.) as edges. The processor 220 may extract the embedding values ​​of users by training the user-POI graph 800 on a GCN model. For example, by using the pinSAGE model, which is one of the GCN models, users may be configured as pin nodes, which are target nodes, and POIs may be configured as board nodes, which are context nodes. In this case, the pinSAGE model may set neighborhoods using dictionary knowledge called a metapath, sample neighborhoods using a random walk, and define and utilize importance scores for neighboring nodes based on the number of visits to the POIs using the random walk. Therefore, by using the pinSAGE model, differences in user preferences for POIs can be reflected on the user-POI graph 800.

[0067] The processor 220 may provide recommended users corresponding to the source user's area of ​​interest through embedding by modeling the user-POI graph 800. As described with reference to FIG. 7, when the source user embedding 701 is given, the processor 220 may search for user embeddings similar to the source user embedding 701 using the vector search engine 700 and provide the user embeddings to the recommended users 702. In this case, the processor 220 may select other users who have many documents such as reviews as recommendation candidates based on the source user's area of ​​interest, and then divide the recommendation candidates into a group of users who have the same POI visit history as the source user and a group of users who have the same interest category as the source user, and recommend users according to the reason for recommendation.

[0068] In user recommendations, the processor 220 may provide recommended users based on following and recommended users based on visiting, arranged according to a certain standard. For example, recommended users based on following may be arranged first, followed by recommended users based on visiting.

[0069] In some embodiments, instead of dividing user embeddings into follow-based and visit-based embeddings, the average of the two embeddings can be used as a user feature that reflects user information.

[0070] FIG. 9 is a flowchart illustrating a POI recommendation modeling process in an embodiment of the present invention.

[0071] The processor 220 may extract nodes indicating POIs and features of the nodes from the local knowledge graph and use them for POI recommendation modeling.

[0072] In step 901, the processor 220 may generate a POI-user heterogeneous graph, which is a graph showing the relationships between POIs and users.

[0073] At step 902, the processor 220 may model POI embeddings for the POI-user heterogeneous graph by GCN model-based learning.

[0074] Unlike collaborative filtering, POI recommendation modeling can embed POIs with similar characteristics close to each other based on a graph deep learning model that reflects the characteristics of POIs and users in node features. POI embedding that reflects the user's preferences can generate recommended POIs that have similar properties to the preference POIs that the user has visited in the past.

[0075] Similar to the user-POI graph 800 described with reference to Fig. 8, the POI-user heterogeneous graph may be generated based on the metadata of POIs and the metadata and interest information of users. In order to properly record the relationship structure between POIs and users, the POI-user heterogeneous graph may be configured in the form of a bipartite graph.

[0076] The POI-user graph may be constructed by setting POIs and users who have visited the POIs as nodes, setting information such as the POI's industry, price range, atmosphere, search purpose, representative menu, popular topics, and receipt keywords as POI node features, setting information such as the user's gender, age, area of ​​interest, and category of interest as user node features, and setting the user's preferences for the POI (e.g., visited, liked, kept, etc.) as edges. The processor 220 may extract user embedding values ​​by training a GCN model on the POI-user graph. For example, by using the pinSAGE model, which is one of the GCN models, for POI recommendation modeling, POIs may be configured as pin nodes and users may be configured as board nodes, as opposed to user recommendation modeling.

[0077] To embed similar POIs close to each other by users who connect to the same POI, a POI-user-POI meta path may be defined and a random walk may be performed. A low-rank positive graph may be created for POIs that reflect a relatively low weight among the POIs visited by the user, so that they have less impact on the POI embedding, and a low-rank positive term may be reflected in the loss function.

[0078] The processor 220 may recommend POIs similar to a preference POI, which is a POI visited by a user, through embedding based on POI-user graph modeling. Re-ranking logic may be added to rank POIs with similar real-world metadata among similar POIs. Referring to FIG. 10 , when a source POI embedding 1001 corresponding to a preference POI is given, the vector search engine 700 may be used to search for POI embeddings similar to the source POI embedding 1001 and provide them as recommended POIs 1002. The processor 220 may select at least one recommended POI 1002 in order of similarity to the preference POI. In this case, the processor 220 may calculate the similarity between the preference POI and each recommended POI 1002 and the industry, price range, atmosphere, search purpose, main menu, popular topics, receipt keywords, etc., used as POI node features. Among the features of the recommended POI 1002, the feature with the greatest similarity to the hobby POI may be selected as the reason for recommending the POI. In other words, after selecting recommended POIs 1002 similar to the hobby POI, the processor 220 may provide, for each recommended POI 1002, a reason for recommendation indicating what characteristics are similar to the hobby POI. In some embodiments, POIs that have many documents such as reviews or have a large number of user visit histories may be selected as recommendation candidates, and then the recommended candidate POIs may be divided into groups such as POIs with similar business types to the hobby POI, POIs with similar representative menus, and POIs with similar visit purposes, and the POIs may be recommended according to the recommendation reason.

[0079] FIG. 11 is a flowchart illustrating a document recommendation modeling process in one embodiment of the present invention.

[0080] A specific embodiment will be described using a review of a POI as a typical example of a document.

[0081] The processor 220 may extract nodes representing reviews and features of the nodes from the local knowledge graph and use them for review recommendation modeling.

[0082] In step 1101, the processor 220 may generate a review-POI heterogeneous graph, which is a graph showing the relationships between reviews and POIs.

[0083] At step 1102, the processor 220 may model review embeddings for the review-POI heterogeneous graph by GCN model-based learning.

[0084] It is possible to recommend similar reviews by understanding reviews posted by users after visiting a POI directly. In this case, rather than simply recommending reviews with similar text or the type of business visited, it is possible to provide recommendations that take into account the intrinsic features of the review by embedding a multi-modality property that comprehensively considers the text, images, and meta information extracted from the review.

[0085] A review-POI graph may be constructed by setting reviews and POIs to which the reviews are registered as nodes, setting information such as review images, review text, receipt images, receipt keywords, consumption prices, and consumption menu items as review node features, setting information such as the POI's industry, price range, atmosphere, search purpose, representative menu items, popular topics, and receipt keywords as POI node features, and setting preferences expressed in the reviews (e.g., receipt reviews, reservation reviews, and preference-based virtual edges) as edges. The processor 220 may extract embedding values ​​of reviews by training a GCN model on the review-POI graph. For example, using the pinSAGE model, which is one of the GCN models, reviews may be configured as pin nodes and POIs may be configured as board nodes. In other words, pinSAGE may be used to obtain multi-modality embeddings that comprehensively consider image, text, and receipt-based information included in reviews, and then similar reviews may be recommended based on the multi-modality embeddings.

[0086] FIG. 12 is an exemplary diagram illustrating a review text embedding process.

[0087] The text included in the review may include text written by the user as a review, text extracted from a receipt image uploaded by the user (price, menu name, industry, etc.), etc. Referring to Figure 12, raw text 1201 extracted from the review may be trained by a word embedding model 1200 (e.g., Word2Vec) to obtain an embedding vector for each word, and then text features 1202 reflecting text information may be extracted by calculating the average of the embedding values ​​obtained from the word embedding model 1200.

[0088] FIG. 13 is an exemplary diagram illustrating a review image embedding process. 13, raw images 1301 extracted from a review are input into a deep learning model 1300 (e.g., VGG16) pre-trained with an image dataset to extract a feature map of the final layer, thereby obtaining a primary embedding vector for each image. Since the primary image features 1302 are extracted using a large network and therefore have a scarcity problem, a dimensionality reduction algorithm (e.g., PCA) may be used to reduce the dimensions to a desired level to obtain final image features 1303. If a review contains multiple images, the average of the embedding values ​​obtained from each image may be calculated to extract image features 1303 that reflect the image information.

[0089] FIG. 14 is an exemplary diagram illustrating a pinSAGE-based review embedding process.

[0090] A multi-modality embedding that comprehensively considers various information included in a review may be obtained. Referring to FIG. 14 , a review-POI heterogeneous graph 1400 indicating the relationship between reviews and POIs may be generated. Here, the graph may be configured in the form of a multi-modal bipartite graph to generate a network that reflects the structural characteristics of reviews and POIs with various attributes. Pre-trained text features 1202 and image features 1303 may be used to reflect the modality of a review. Additionally, review nodes may be defined to reflect various meta-information included in a review (e.g., average menu price, number of likes for a review, number of detailed reviews for a review, etc.), and learning may be performed based on the pre-trained review nodes to obtain a review embedding matrix 1410 for the review-POI heterogeneous graph 1400.

[0091] 15, the processor 220 may use the vector search engine 700 to find the similarity between a reference review 1501 given in the review embedding matrix 1410 and other reviews, and may select a certain number (K) of reviews that are ranked in order of the closest similarity distance. At this time, the selected reviews may be defined as reviews 1502 similar to the reference review 1501 and recommended. For example, after first searching a list of reviews located in a region similar to the reference review 1501, reviews whose content is similar to the reference review 1501 may be selected and recommended.

[0092] Therefore, the processor 220 uses a user-POI-document heterogeneous graph composed of one huge regional knowledge graph, and embeds user nodes through modeling based on the user-user graph and the user-POI graph, embeds POI nodes through modeling based on the POI-user graph, and embeds document nodes through modeling based on the document-POI graph, thereby providing user recommendations, POI recommendations, and document recommendations based on the embedding of each node type.

[0093] As described above, according to an embodiment of the present invention, it is possible to recommend items related to a region, such as points of interest (POIs), users, and documents, based on a region knowledge graph. By generating a region knowledge graph with a heterogeneous graph structure consisting of points of interest (POIs), users, and documents, it is possible to provide more advanced recommendations by considering not only the unique features possessed by each item but also its relationships with other nodes connected to it on the graph. Furthermore, according to an embodiment of the present invention, the heterogeneous graph structure allows each node to be embedded using different types of nodes and structural information, thereby minimizing the data scarcity problem, and by using various multimodal data such as images and text, it is possible to maximize the amount of information obtained from each modality.

[0094] The above-described devices may be implemented using hardware components, software components, and / or a combination of hardware and software components. For example, the devices and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or various devices capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the OS. The processing device may also access, record, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, a single processing device may be described. However, those skilled in the art will understand that a processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing device may include multiple processors or one processor and one controller. Other processing configurations, such as parallel processors, are also possible.

[0095] Software may include computer programs, codes, instructions, or a combination of one or more of these, which may configure a processing device to operate as desired or may independently or collectively instruct the processing device. The software and / or data may be embodied in any type of machine, component, physical device, computer storage medium, or device to be interpreted by the processing device or to provide instructions or data to the processing device. The software may be distributed and stored and executed in a distributed manner on computer systems connected by a network. The software and data may be stored on one or more computer-readable storage media.

[0096] Methods according to embodiments may be implemented in the form of program instructions executable by various computer means and recorded on a computer-readable medium. Here, the medium may continuously record a computer-executable program or temporarily record it for execution or download. Furthermore, the medium may be various recording or storage means in the form of a single piece of hardware or multiple pieces of hardware combined together. It is not limited to media directly connected to a computer system, but may also be distributed over a network. Examples of media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; ROMs, RAMs, flash memories, and the like, configured to record program instructions. Other examples of media include recording media or storage media managed by application stores that distribute applications, or by sites, servers, and the like that provide or distribute various software.

[0097] Although the embodiments have been described above based on limited examples and drawings, those skilled in the art will appreciate that various modifications and variations may be made from the above description. For example, the described techniques may be performed in an order different from that described, and / or the described system, structure, device, circuit, or other element may be coupled or combined in a manner different from that described, or may be substituted or replaced by other elements or equivalents, and still achieve suitable results.

[0098] Therefore, different embodiments that are equivalent to the claims are within the scope of the appended claims. [Explanation of symbols]

[0099] 310: Graph generation unit 320: Node embedding unit 330: Item Recommendation Department

Claims

1. 1. A recommendation method implemented on a computer device, comprising: the computing device includes at least one processor configured to execute computer-readable instructions contained in a memory; The recommendation method includes: generating, by the at least one processor, a local knowledge graph having a heterogeneous graph structure based on a plurality of items constituting a local area; performing, by the at least one processor, embedding by node type using different node features in the regional knowledge graph; and providing, by the at least one processor, recommended items corresponding to each of the node types based on the embeddings for each of the node types. Including, The generating step includes: generating the local knowledge graph by configuring the POIs, the users who are to visit the POIs, and the documents produced for the POIs as different types of nodes and configuring the relationships between the nodes as edges; Including, The performing step includes: extracting a node representing a user and a feature of the node representing the user from the local knowledge graph, setting each user as a node, setting information related to the user as a user node feature, and extracting a follow-based user embedding using a user-user graph that sets a follow relationship with the user as an edge; Recommendation methods, including:

2. The performing step includes: extracting one node type as a target node and at least one other node type as a context node from the local knowledge graph, and modeling a relationship graph between the target node and the context node; The recommendation method of claim 1 , comprising:

3. The modeling step includes: Modeling relationships between the target node and the context node using multimodal data including images, text, and categorical data. The recommendation method according to claim 2, characterized in that:

4. The modeling step includes: The recommendation method according to claim 2, further comprising the step of extracting positive samples and negative samples by random walk-based sampling.

5. The modeling step includes: Applying different weighted edges to the relationship graph between the target node and the context node using low-rank positive loss. The recommendation method according to claim 2, characterized in that:

6. The performing step includes: extracting a node indicating a POI and a feature of the node indicating the POI from the local knowledge graph, setting a user and a POI visited by the user as a node, setting information related to the user as a user node feature, setting information related to the POI as a POI node feature, and setting user preference information for the POI as an edge, and extracting a visit-based user embedding using the user-POI graph; The recommendation method of claim 1 , comprising:

7. The providing step comprises: searching for an embedding similar to the source embedding based on the follow-based user embedding to select a follow-based recommended user; searching for an embedding similar to the source embedding based on the visit-based user embedding to select a visit-based recommended user; and and providing the following-based recommended users and the visiting-based recommended users in accordance with a predetermined standard. The recommendation method of claim 6, comprising:

8. The providing step comprises: searching for a source embedding similar to the source embedding based on the follow-based user embedding and the visit-based user embedding to select a recommended user; and Providing the recommended user together with a recommendation reason showing similar characteristics to the source embedding. The recommendation method of claim 6, comprising:

9. The performing step includes: extracting a node indicating a POI and a feature of the node indicating the POI from the local knowledge graph, setting the POI and the user who visited the POI as a node, setting information related to the POI as a POI node feature, setting information related to the user as a user node feature, and setting user preference information for the POI as an edge, and extracting a POI embedding using the POI-user graph; The recommendation method of claim 1 , comprising:

10. The providing step comprises: searching for an embedding similar to the source embedding based on the POI embedding to select a recommended POI; and providing the recommended POI along with a recommendation reason showing similar characteristics to the source embedding; The recommendation method of claim 9, comprising:

11. The performing step includes: extracting a node indicating a document and a feature of the node indicating the document from the local knowledge graph, setting the document and the POI where the document is registered as a node, setting information related to the document as a document node feature, setting information related to the POI as a POI node feature, and setting document interest information for the POI as an edge, and extracting document embeddings using the document-POI graph; The recommendation method of claim 1 , comprising:

12. The extracting step comprises: extracting text embeddings by training the text contained in the document with a word embedding model; extracting image embeddings by learning images contained in documents using a deep learning model; and acquiring a document embedding matrix for the document-POI graph by learning document nodes that reflect the text embedding, the image embedding, and meta-information included in the document; The recommendation method of claim 11 , comprising:

13. A computer program for causing a computer to execute the recommendation method according to any one of claims 1 to 12.

14. 1. A computer device comprising: at least one processor configured to execute computer-readable instructions contained in the memory; Including, The at least one processor A regional knowledge graph with a heterogeneous graph structure is generated based on the multiple items that make up the region. performing embedding for each node type using different node features in the regional knowledge graph; providing recommended items corresponding to each of the node types based on the embedding of each of the node types; The local knowledge graph is generated by configuring POIs, users who visit the POIs, and documents produced for the POIs as nodes of different types, and configuring the relationships between the nodes as edges; The embedding may be Extracting nodes representing users and features of the nodes representing the users from the local knowledge graph, setting each user as a node, setting information related to the user as a user node feature, and setting follow relationships with the user as edges, and extracting follow-based user embeddings using a user-user graph. To be carried out by A computer device comprising:

15. The at least one processor extracting one node type as a target node and at least one other node type as a context node from the regional knowledge graph, and modeling a relationship graph between the target node and the context node; 15. The computer device of claim 14.

16. The at least one processor Extracting nodes indicating POIs and features of the nodes indicating the POIs from the local knowledge graph, setting users and POIs visited by the users as nodes, setting information related to the users as user node features, setting information related to the POIs as POI node features, and setting user preference information for POIs as edges, and extracting visit-based user embeddings using a user-POI graph; Searching for a source embedding and an embedding similar to the follow-based user embedding and the visit-based user embedding to provide recommended users 15. The computer device of claim 14.

17. The at least one processor Extracting nodes indicating POIs and features of the nodes indicating the POIs from the local knowledge graph, setting POIs and users who visited the POIs as nodes, setting information related to the POIs as POI node features, setting information related to the users as user node features, and setting user preference information for the POIs as edges, and extracting POI embeddings using a POI-user graph; Searching for an embedding similar to the source embedding based on the POI embedding to provide a recommended POI.

15. The computer device of claim 14.

18. The at least one processor Extracting a node indicating a document and a feature of the node indicating the document from the local knowledge graph, setting the document and the POI to which the document is registered as a node, setting information related to the document as a document node feature, setting information related to the POI as a POI node feature, and setting document interest information for the POI as an edge, and extracting document embeddings using the document-POI graph; By using the document embedding, a source embedding and similar embeddings are searched for to provide recommended documents.

15. The computer device of claim 14.

Images