Method and apparatus for providing content based on a knowledge graph

Abstract

Content recommendations are provided based on a knowledge graph using, for example, a processor and a memory of a device. In some embodiments, sensors are used. The content recommendations efficiently utilize the processor, the memory, and / or the sensors. A knowledge graph is maintained at the device and another knowledge graph is maintained at the server. The knowledge graph at the device is generated by, for example, obtaining log history information, generating a device knowledge graph, generating a pattern knowledge graph associated with a user's behavior pattern, and / or updating the device knowledge graph by adding the pattern knowledge graph to the device knowledge graph. In some examples, a knowledge graph is generated at the server based on the pattern knowledge graph.

Description

Technical Field

[0001] The present disclosure relates to a method and an apparatus for providing content based on a knowledge graph. More specifically, the present disclosure relates to a method and an apparatus for providing content by generating a knowledge graph based on user analysis. Background Art

[0002] Unlike existing rule-based intelligent systems, an artificial intelligence (AI) system is a computer system that itself performs machine learning and problem-solving and becomes more intelligent. As the use of AI systems has increased, their recognition rates have improved and they can more accurately understand user preferences. Existing rule-based intelligent systems are gradually being replaced by AI systems based on deep learning.

[0003] AI technology includes machine learning (deep learning) and elemental technologies using machine learning. Machine learning is an algorithm technology that itself classifies and learns the characteristics of input data. Elemental technologies are technologies that use machine learning algorithms such as deep learning. Elemental technologies include technical fields such as language understanding, visual understanding, inference or prediction, knowledge representation, and motion control.

[0004] The various fields to which AI technology is applied are as follows. Language understanding is a technology for recognizing, applying, and processing human languages and characters, and includes natural language processing, machine translation, dialogue systems, question and answer, speech recognition and synthesis, etc. Visual understanding is a technology for recognizing and processing objects like human vision, and includes object recognition, object tracking, image retrieval, person recognition, scene understanding, spatial understanding, image enhancement, etc. Inference or prediction is a technology for determining, logically inferring, and predicting information, and includes knowledge / probability-based inference, optimization prediction, preference-based planning, recommendation, etc. Knowledge representation is a technology for automatically processing human experience information into knowledge data, and includes knowledge construction (data generation and classification), knowledge management (data utilization), etc. Motion control is a technology for controlling the autonomous driving of vehicles and the movement of robots, and includes movement control (navigation, collision, driving, etc.), operation control (behavior control), etc.

[0005] In addition, there is also a need for an AI technology that can effectively expand the knowledge graph associated with a user through user analysis. Summary of the Invention

[0006] Solution to the Problem

[0007] There is provided a method and an apparatus for providing content based on a knowledge graph in a running device.

[0008] There is also provided a method and an apparatus for continuously updating and generating a knowledge graph in a running device.

[0009] There is also provided a method and apparatus for separately managing a knowledge graph generated by a server and a knowledge graph generated by a device so as to continuously update the knowledge graph in a running device.

[0010] There is provided a method and apparatus for determining a pattern associated with a user through user analysis and combining the determined pattern with a knowledge graph generated by a server.

[0011] There is provided a method and apparatus for determining a pattern associated with a user through user analysis and determining functions to be provided to the user according to the determined pattern.

[0012] Additional aspects will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the presented embodiments of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other aspects, features and advantages of certain embodiments of the present disclosure will become more apparent from the following description when taken in conjunction with the accompanying drawings, in which:

[0014] Figure 1 is a schematic diagram of a system for providing a knowledge graph according to an embodiment of the present disclosure;

[0015] Figure 2 is a flowchart of a method for generating a knowledge graph executed by a device according to an embodiment of the present disclosure;

[0016] Figure 3 is a flowchart of a method for generating a device knowledge graph executed by a device according to an embodiment of the present disclosure;

[0017] Figure 4 is a flowchart of a method for generating a pattern knowledge graph executed by a device according to an embodiment of the present disclosure;

[0018] Figure 5A is a flowchart of a method for generating a pattern knowledge graph based on the correspondence relationship between entities executed by a device according to an embodiment of the present disclosure;

[0019] Figure 5B illustrates an example of generating a device knowledge graph according to an embodiment of the present disclosure;

[0020] Figure 5C illustrates an example of generating a pattern according to an embodiment of the present disclosure;

[0021] Figure 5D illustrates an example of recommending content based on a pattern according to an embodiment of the present disclosure;

[0022] Figure 6AA flowchart of a method for updating a device knowledge graph executed by a device according to an embodiment of the present disclosure;

[0023] Figure 6B An example of combining entities based on name similarity according to an embodiment of the present disclosure is shown;

[0024] Figure 6C An example of combining entities based on instance similarity according to an embodiment of the present disclosure is shown;

[0025] Figure 6D An example of combining entities based on structural similarity according to an embodiment of the present disclosure is shown;

[0026] Figure 7 A flowchart of a method for providing a function by using a pattern knowledge graph executed by a device according to an embodiment of the present disclosure;

[0027] Figure 8 A flowchart of a method for determining recommended content executed by a device according to an embodiment of the present disclosure;

[0028] Figure 9 Examples of content recommendation based on a server knowledge graph and content recommendation based on a device knowledge graph according to an embodiment of the present disclosure are shown;

[0029] Figure 10A An example of a method for generating a device knowledge graph according to an embodiment of the present disclosure is shown;

[0030] Figure 10B An example of a device knowledge graph according to an embodiment of the present disclosure is shown;

[0031] Figure 10C An example of a specific process for generating a device knowledge graph according to an embodiment of the present disclosure is shown;

[0032] Figure 10D An example of a specific process for generating a device knowledge graph according to an embodiment of the present disclosure is shown;

[0033] Figure 11 Examples of a device knowledge graph and a pattern knowledge graph according to an embodiment of the present disclosure are shown;

[0034] Figure 12 A block diagram of a device according to an embodiment of the present disclosure;

[0035] Figure 13 A detailed block diagram of a device according to an embodiment of the present disclosure;

[0036] Figure 14 A block diagram of a processor of a device according to an embodiment of the present disclosure;

[0037] Figure 15 A block diagram of a device knowledge data generator of a device according to an embodiment of the present disclosure;

[0038] Figure 16 A block diagram of a knowledge data manager of a device according to an embodiment of the present disclosure; and

[0039] Figure 17 A block diagram of a server according to an embodiment of the present disclosure. Detailed implementation manners

[0040] According to an embodiment of the present disclosure, a method of operating a device for generating a knowledge graph includes: obtaining log history information related to the operation of the device; generating a device knowledge graph based on the log history information; generating a pattern knowledge graph associated with the behavior pattern of the user of the device based on the entities and the relationships between the entities in the device knowledge graph; updating the device knowledge graph by adding the generated pattern knowledge graph to the device knowledge graph; and updating the server knowledge graph generated by the server based on the generated pattern knowledge graph.

[0041] According to another embodiment of the present disclosure, a device for providing content based on a knowledge graph includes: a communication interface; a memory storing one or more instructions; and a processor configured to execute the one or more instructions to: obtain log history information related to the operation of the device; generate a device knowledge graph based on the log history information; generate a pattern knowledge graph associated with the behavior pattern of the user of the device based on the entities and the relationships between the entities in the device knowledge graph; update the device knowledge graph by adding the generated pattern knowledge graph to the device knowledge graph; and update the server knowledge graph generated by the server based on the generated pattern knowledge graph.

[0042] A method of operating a device for generating a knowledge graph is provided herein, the method including: obtaining log history information related to the operation of the device; generating a device knowledge graph based on the log history information; generating a pattern knowledge graph associated with the behavior pattern of the user of the device based on the first entity and the relationship between the first entities in the device knowledge graph; updating the device knowledge graph by adding the pattern knowledge graph to the device knowledge graph; updating the server knowledge graph generated by the server based on the pattern knowledge graph; and providing content recommendations to a user based on the updated device knowledge graph.

[0043] The present disclosure provides a device for providing content based on a knowledge graph. The device includes: a communication interface; a memory storing one or more instructions; and a processor configured to execute the one or more instructions to: obtain log history information related to the operation of the device, generate a device knowledge graph based on the log history information, generate a pattern knowledge graph associated with the behavior pattern of a user of the device based on the relationship between a first entity and the first entity in the device knowledge graph, update the device knowledge graph by adding the pattern knowledge graph to the device knowledge graph, update a server knowledge graph generated by a server based on the pattern knowledge graph, and provide content recommendations to the user based on the updated device knowledge graph.

[0044] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0045] When describing the embodiments of the present disclosure, technical content that is well known in the art to which the present disclosure pertains and is not directly related to the present disclosure will not be described. By omitting unnecessary descriptions, the present disclosure will be more clearly conveyed without confusing the subject matter of the present disclosure.

[0046] For the same reason, some elements in the drawings are enlarged, omitted, or schematically shown. Additionally, the size of each element does not reflect the actual size. In each drawing, the same reference numerals are assigned to the same or corresponding elements.

[0047] From the following embodiments of the present disclosure described in detail in conjunction with the accompanying drawings, the advantages and features of the present disclosure and the methods for achieving them will become more apparent. However, it will be understood that the present disclosure is not limited to the following embodiments of the present disclosure, and various modifications can be made without departing from the scope of the present disclosure. The embodiments of the present disclosure set forth herein are provided so that the present disclosure will be thorough and complete, and will fully convey the concept of the present disclosure to those of ordinary skill in the art. The present disclosure should be defined by the appended claims. Throughout the specification, the same reference numerals indicate the same elements.

[0048] Throughout the present disclosure, the expression “at least one of a, b, or c” indicates: only a; only b; only c; both a and b; both a and c; both b and c; all of a, b, and c, or variations thereof.

[0049] It will be understood that the respective blocks of the flowcharts and combinations of flowcharts can be executed by computer program instructions. Since these computer program instructions can be embedded in the processors of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, the instructions executed by the processors of the computer or other programmable data processing apparatus generate modules for performing the functions described in the blocks of the flowcharts. Since these computer program instructions can also be stored in a computer-usable or computer-readable memory that can direct a computer or other programmable data processing apparatus to act in a particular manner, the instructions stored in the computer-usable or computer-readable memory can also produce an article of manufacture that includes instruction modules for performing the functions described in the blocks of the flowcharts. Since the computer program instructions can also be installed on a computer or other programmable data processing apparatus, the instructions for performing operations of a computer or other programmable data apparatus by generating a computer-implemented process via a series of operations performed on the computer or other programmable data processing apparatus can provide operations for performing the functions described in the blocks of the flowcharts.

[0050] In addition, each block may represent a module, a section, or a portion of code that includes one or more executable instructions for performing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur out of order. For example, depending on the functions involved, two consecutive blocks shown may actually be executed substantially simultaneously, or the blocks may sometimes be executed in the reverse order.

[0051] As used herein, the terms "module" or " -ware" refer to a software element or a hardware element, such as a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC), and the terms "module" or " -ware" perform certain functions. However, the terms "module" or " -ware" are not limited to software or hardware. The terms "module" or " -ware" can be configured in an addressable storage medium or can be configured to reproduce one or more processors. Thus, for example, the terms "module" or " -ware" include elements such as software elements, object-oriented software elements, class elements, and task elements, processes, functions, attributes, procedures, subroutines, program code segments, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided in the elements and "modules" or " -ware" can be combined with fewer elements and "modules" or " -ware" or can be separated from additional elements and "modules" or " -ware". In addition, the elements and "modules" or " -ware" can be implemented to reproduce one or more central processing units (CPUs) or secure multimedia cards in a device. In addition, in embodiments of the present disclosure, "modules" or " -ware" can include one or more processors.

[0052] Generally, most recommendation systems are server-based recommendation systems. However, in server-based recommendation systems, due to user privacy issues, only limited information can be transmitted to the server, and thus it may be difficult to make user-customized recommendations. Therefore, a recommendation system based on a running device knowledge graph for recommending content based on the device knowledge graph has been developed. However, in a recommendation system based on a running device knowledge graph, when determining the recommended content by only using the server knowledge graph received from the server, the actual behavior and needs of the device user may not be reflected.

[0053] Technical problem. There are the following technical problems: If the device also provides suggestions (e.g., content recommendations) based on the device's computational workload using the device's memory and the suggestions are inappropriate, then the user will ignore or reject the content recommendations, and the device's computational workload and memory usage are not efficient.

[0054] Therefore, it is necessary to analyze the user's context, behavior patterns, etc. to recommend user-customized content and provide services. In the following, a method for providing an appropriate function reflecting user characteristics to a user by analyzing behavior patterns associated with the user's current context and providing functions according to the analyzed behavior patterns is described.

[0055] Technical solution. A technical solution reflecting user characteristics including behavior patterns is provided, and improvements are provided in the provided suggestions or content recommendations. The provided suggestions or content recommendations are based on the device's computational workload and also use the device's memory (for an exemplary embodiment, see the processor 1240 and memory 1350 below Figure 13 and in some cases use device sensors ( Figure 13 item 1310). The acceptance or adoption by the user is increased, and the device's computational workload, memory usage, and possibly sensors are made more efficient.

[0056] In the present disclosure, as used herein, the term "knowledge graph" may refer to data generated when constructing certain information in the form of an ontology according to a certain standard. Generally, the components of an ontology are classes, relationships, and attributes. For example, certain information can be expressed in the form of a Resource Description Framework (RDF). RDF is a standard model for data exchange on the Internet. Information expressed in the form of RDF can be expressed in the form of node-edge-node. In this case, node-edge-node can refer to subject-predicate-object.

[0057] In addition, in embodiments of the present disclosure, a knowledge graph may refer to data generated in the form of a graph because certain information such as real-world entities and relationships between entities is constructed in an ontology form according to certain criteria. In embodiments of the present disclosure, a knowledge graph may include configurations such as entities, edges, and attributes.

[0058] In embodiments of the present disclosure, an entity may refer to a named object in the real world. For example, an entity may include names of people such as "Tom" or "James Cameron", names of sports such as "baseball", names of applications such as "YouTube", or similar names. In embodiments of the present disclosure, an entity may refer to a node in the above-mentioned knowledge graph.

[0059] In embodiments of the present disclosure, an edge is a configuration that connects entities and may indicate the connection relationship between entities. For example, the "baseball" entity and the "interest" entity may be connected by an edge. In the present disclosure, the relationship between entities in the knowledge graph may mean the connection relationship between nodes in the knowledge graph.

[0060] In embodiments of the present disclosure, an attribute may refer to specific information about an entity. That is to say, an attribute may refer to a way of expressing a specific entity. For example, when the "baseball" entity and the "interest" entity are connected by an edge indicating type, the type of the "baseball" entity may be classified as "interest". At this time, the type may refer to an attribute of the "baseball" entity.

[0061] In the present disclosure, log history information may mean information about all contexts that occur inside or around a device recorded over time. For example, log history information may refer to data in which information sensed in real time from temperature and humidity sensors, global positioning system (GPS) data, information about the time when an event occurs (such as the network access history of a user using the device and the history of applications frequently used by the user in the device), and device status (for example, power, central processing unit (CPU) utilization, memory utilization, etc.) are recorded together. In embodiments of the present disclosure, log history information may be referred to as raw data.

[0062] In embodiments of the present disclosure, the log history information may include data indicating operations of a device. In such a case, the data indicating operations of the device may be collected in the device when a user uses the device. In embodiments of the present disclosure, the data indicating operations of the device may refer to log history information associated with operations of the device. For example, the data indicating operations of the device may include, among multiple pieces of log history information, the network access history of the user using the device and the history of applications frequently used by the user in the device. Additionally, the data indicating operations of the device may refer to data indicating events that occur during the execution of software in the device.

[0063] In the present disclosure, the server knowledge graph may be a knowledge graph generated by a server. The server knowledge graph may be generated by the server based on data collected from multiple devices. In embodiments of the present disclosure, the server knowledge graph may be updated regularly or irregularly. In embodiments of the present disclosure, the server knowledge graph may be generated to support a search engine by a server managing the search engine, may be generated to support a social network service by a server managing the social network service, or may be generated to support a product sales service by a server managing the product sales service.

[0064] In the present disclosure, the device knowledge graph may be a knowledge graph generated by a user's device. The device knowledge graph may be generated based on log history information collected by the user's device. Since the device knowledge graph is generated based on the log history information of the user's device, the device knowledge graph may refer to a graph that enhances knowledge dedicated to the user compared to the above-mentioned server knowledge graph.

[0065] In embodiments of the present disclosure, the server knowledge graph generated by the server and the device knowledge graph generated by the device may be managed separately. For example, the server may generate a server knowledge graph by using data generated by the server, and the device may generate a device knowledge graph based on log history information collected by the device. The server may update the server knowledge graph regularly or irregularly and may transmit the updated server knowledge graph to the device. The device may update the device knowledge graph based on the server knowledge graph received from the server and the log history information about the device.

[0066] In the present disclosure, a pattern knowledge graph may refer to a knowledge graph indicating the behavior patterns of users of a device. The pattern knowledge graph may be generated based on patterns indicating the association relationships between specific entities. For example, when a user of a device performs a specific behavior (e.g., watching YouTube associated with baseball) in a specific context (e.g., making a reservation for a baseball game), the entity indicating the specific context may be connected to the entity indicating the specific behavior. When the structure (e.g., RDF structure) in which the entity indicating the specific context is connected to the entity indicating the specific behavior repeats in a time series, this repeated structure may be generated as a pattern. The pattern knowledge graph may mean that the generated graph is expressed in the form of a knowledge graph having a node-edge-node configuration.

[0067] In the present disclosure, knowledge data indicating the operations of a device may refer to data obtained by processing data indicating the operations of the device into data that can be used to generate a knowledge graph. In an embodiment of the present disclosure, the data indicating the operations of the device may be included in log history information. For example, in the knowledge data, the log history information is processed such that the subject-predicate-object displayed in the form of a knowledge graph can be expressed in text form.

[0068] Hereinafter, the present disclosure will be described in detail with reference to the accompanying drawings.

[0069] Figure 1 is a schematic diagram of a system for providing a knowledge graph according to an embodiment of the present disclosure.

[0070] Reference Figure 1 , the system for providing a knowledge graph may include a server 110, a network, and a device 130. However, the present disclosure is not limited thereto.

[0071] The server 110 may generate a server knowledge graph 120 based on data received from multiple devices (e.g., a first device 132, a second device 134, a third device 136, an Nth device 138, etc.). In an embodiment of the present disclosure, the data for generating the server knowledge graph 120 is big data generated by web crawling. The server knowledge graph 120 may be referred to as a well-known knowledge graph.

[0072] The server knowledge graph 120 may include one or more server knowledge graphs. For example, the server knowledge graph 120 may include a first server knowledge graph 122 and a second server knowledge graph 124. In an embodiment of the present disclosure, the first server knowledge graph 122 may refer to a server knowledge graph generated earlier in time than the second server knowledge graph 124. Additionally, the second server knowledge graph 124 may refer to a server knowledge graph in the first server knowledge graph 122 that is updated based on data received from devices or big data collected by web crawling.

[0073] Device 130 may obtain log history information related to the operation of device 130, receive server knowledge graph 120 generated by server 110 from server 110, and generate device knowledge graph 140 by using server knowledge graph 120 and log history information. Specific examples related to the generation of device knowledge graph 140 will be described below with reference to FIG. 10. Additionally, device 130 may generate a pattern knowledge graph associated with the behavior pattern of the user of device 130 based on the entities in device knowledge graph 140 and the relationships between the entities. Additionally, device 130 may update device knowledge graph 140 by adding the pattern knowledge graph to device knowledge graph 140.

[0074] In an embodiment of the present disclosure, device 130 may collect data from structured sources or unstructured sources in order to obtain knowledge data associated with the generation of a knowledge graph. In an embodiment of the present disclosure, structured sources may include relational databases, subscriptions, catalogs, directories, etc. Additionally, in an embodiment of the present disclosure, unstructured sources may include web pages, text, voice, images, videos, etc.

[0075] In an embodiment of the present disclosure, device 130 may collect data from configurations included in web pages by using rule-based, tree-based, or machine learning-based methods.

[0076] In an embodiment of the present disclosure, device 130 may collect data from text. For example, device 130 may collect personal and professional information related to people included in the text in emails and calendars, text information included in the text in social media, etc.

[0077] As described above, device 130 may collect data from structured sources or unstructured sources. Additionally, device 130 may match data collected from different sources. For example, device 130 may detect matched entities from the collected data based on probabilistic matching models, distance-based models, and declarative matching rules and constraints, etc. Device 130 may perform knowledge refinement operations on the detected entities. For example, device 130 may refine the detected entities by using techniques such as knowledge fusion, error detection, and fact inference. Through the above process, device 130 may obtain knowledge data associated with the generation of a knowledge graph.

[0078] Device 130 may refer to one or more devices. For example, device 130 may include a first device 132, a second device 134, a third device 136, ……, and / or an Nth device 138.

[0079] The device knowledge graph 140 may refer to one or more knowledge graphs. For example, the device knowledge graph 140 may refer to a first device knowledge graph 142, a second device knowledge graph 144, a third device knowledge graph 146, …, and / or an Nth device knowledge graph 148. In an embodiment of the present disclosure, the device knowledge graph 140 may be updated by using a pattern knowledge graph. In this case, the updated device knowledge graph may also be included in the device knowledge graph 140. Although devices each having one device knowledge graph are shown in Figure 1 , the device 130 may have one or more device knowledge graphs.

[0080] The device 130 may be a smart phone, a tablet personal computer (PC), a PC, a smart television (TV), a mobile phone, a personal digital assistant (PDA), a laptop computer, a media player, a micro server, a global positioning system (GPS) device, an e-book terminal, a digital broadcast terminal, a navigation device, a self-service terminal, an MP3 player, a digital camera, a household appliance, and other mobile or non-mobile computing devices, but the present disclosure is not limited thereto. Additionally, the device 130 may be a wearable device having communication functions and data processing functions, such as a watch, glasses, a headband, or a ring. However, the present disclosure is not limited thereto, and the device 130 may include any type of device capable of sending and receiving data through the server 110 and the network.

[0081] The network may include a local area network (LAN), a wide area network (WAN), a value-added network (VAN), a mobile radio communication network, a satellite communication network, or any combination thereof. The network is an integrated data communication network that enables the Figure 1 network configuration entities shown to communicate with each other efficiently, and may include a wired Internet, a wireless Internet, and a mobile wireless communication network.

[0082] The wireless communication may be, for example, wireless LAN (Wi-Fi), Bluetooth, Bluetooth Low Energy, Zigbee, Wi-Fi Direct (WFD), ultra-wideband (UWB), infrared communication (Infrared Data Association (IrDA)), near-field communication (NFC), etc., but the present disclosure is not limited thereto.

[0083] In an embodiment of the present disclosure, the server 110 may generate a first server knowledge graph 122 based on data collected from devices or big data collected by web crawling. The server 110 may transmit the generated first server knowledge graph 122 to the first device 132. The first device 132 may generate a first device knowledge graph 142 by using the received first server knowledge graph 122 and log history information related to the operation of the first device 132. Additionally, the first device 132 may generate a pattern knowledge graph associated with the behavior pattern of the user of the first device 132, and update the first device knowledge graph 142 by using the generated pattern knowledge graph. In the same manner as described above, the second device 134, the third device 136, ……, and the Nth device 138 may also generate a second device knowledge graph 144, a third device knowledge graph 146, and an Nth device knowledge graph, respectively. Additionally, the second device 134, the third device 136, ……, and the Nth device 138 may update the respective generated device knowledge graphs by using pattern knowledge graphs associated with the behavior patterns of the users of the devices.

[0084] In an embodiment of the present disclosure, after the server 110 generates the first server knowledge graph 122, a second server knowledge graph 124 may be generated by using new or added data. To use the knowledge graph in the device 130, the server 110 may transmit the second server knowledge graph 124 to the first device 132. However, since the updated first device knowledge graph exists in the first device 132, a conflict may occur between the second server knowledge graph 124 and the updated first device knowledge graph when using the knowledge graph.

[0085] In an embodiment of the present disclosure, since the updated first device knowledge graph includes information related to the behavior pattern of the user of the first device 132, the first device 132 may use the updated first device knowledge graph instead of the second server knowledge graph 124. Alternatively, the first device 132 may generate a new device knowledge graph based on the second server knowledge graph 124 and the updated first device knowledge graph. Hereinafter, a method for generating the device knowledge graph 140 and updating the device knowledge graph 140 based on the pattern knowledge graph according to specific embodiments of the present disclosure will be described.

[0086] Figure 2 is a flowchart of a method for generating a knowledge graph executed by the device 130 according to an embodiment of the present disclosure.

[0087] Reference Figure 2, in operation 201, device 130 may obtain log history information related to the operation of device 130. That is, device 130 may collect log data generated when the user of device 130 uses device 130. For example, when the user of device 130 inputs and searches for baseball in an Internet search window, device 130 may obtain log history information regarding the search record.

[0088] In operation 203, device 130 may generate device knowledge graph 140 based on the log history information. For example, device 130 may convert the log history information into knowledge data. Device 130 may generate device knowledge graph 140 by connecting at least one of the entities in server knowledge graph 120 to the knowledge data based on the relationship between the knowledge data and the entities in server knowledge graph 120 stored in device 130.

[0089] In operation 205, device 130 may generate a pattern knowledge graph based on the entities and the relationships between the entities in device knowledge graph 140. For example, device 130 may identify entities associated with the behavior of the user of device 130 and entities associated with the context from the device knowledge graph 140 generated in operation 205. Device 130 may generate a pattern knowledge graph based on the corresponding relationships between the identified entities.

[0090] In operation 207, device 130 may update device knowledge graph 140. For example, device 130 may update device knowledge graph 140 by connecting the pattern knowledge graph generated in operation 205 to the device knowledge graph 140 generated in operation 203.

[0091] In operation 209, device 130 may update server knowledge graph 120 based on the pattern knowledge graph. For example, device 130 may update server knowledge graph 120 by connecting the entities in the pattern knowledge graph to the entities in server knowledge graph 120.

[0092] As referenced Figure 2 above, device knowledge graph 140 may be generated based on log history information, etc., a pattern knowledge graph may be generated from the generated device knowledge graph 140, and device knowledge graph 140 and server knowledge graph 120 may be updated by using the generated pattern knowledge graph. That is, based on the generated pattern knowledge graph, device knowledge graph 140 generated by device 130 and server knowledge graph 120 generated by server 110 may be managed separately. Although Figure 2Although not shown in the figure, device 130 may transmit the updated server knowledge graph 120 to server 110, and server 110 may use the received updated server knowledge graph 120 to expand the server knowledge graph 120 and transmit the expanded server knowledge graph back to device 130.

[0093] As described below, Figure 3 A method for generating the device knowledge graph 140 is shown.

[0094] Figure 3 FIG. 9 is a flowchart of a method for generating the device knowledge graph 140 executed by device 130 according to an embodiment of the present disclosure. In an embodiment of the present disclosure, Figure 3 The operation of... may be included in Figure 2 Operation 205 of....

[0095] Referring to Figure 3 , in operation S301, device 130 may generate knowledge data from the log history information. For example, device 130 may generate knowledge data by inputting the log history information into a knowledge graph generation model. In an embodiment of the present disclosure, the knowledge graph generation model may refer to such a data generation model that processes the input data when inputting data such as log history information and outputs knowledge data for generating a knowledge graph. Using the knowledge graph generation model, device 130 may obtain knowledge data expressed in text form from the log history information.

[0096] In operation S303, device 130 may generate the device knowledge graph 140 by connecting at least one of the entities in the server knowledge graph 120 to the knowledge data based on the relationship between the knowledge data and the entities in the server knowledge graph 120. For example, device 130 may receive the server knowledge graph 120 generated by server 110 from server 110 and store the received server knowledge graph 120 in the memory of device 130. Device 130 may select one or more of the entities in the stored server knowledge graph 120 for connecting the knowledge data. That is, device 130 may select an entity associated with the knowledge data from among the entities in the server knowledge graph 120. Device 130 may expand the server knowledge graph 120 by connecting the knowledge data to the selected entity in the server knowledge graph 120. The device knowledge graph 140 may be generated by the expansion of the server knowledge graph 120 as described above.

[0097] In an embodiment of the present disclosure, related to Figure 3Specific associated examples are as follows. Device 130 may obtain historical information about a schedule (e.g., a baseball game) described in a calendar application, such as {calendar, "baseball game", 20190705:17:00, 20190705:21:00}. Device 130 may generate knowledge data from the obtained log historical information, such as {CalendarActivity, hasTitle, "baseball game"}. In this case, the knowledge data may indicate that the name of the schedule described in the calendar application is baseball game.

[0098] Device 130 may connect the knowledge data to the server knowledge graph 120. For example, the knowledge data and the server knowledge graph 120 may be connected or associated as follows {CalendarActivity, hasMeaning, Baseball_Event}. In this case, {CalenderActivity} may refer to the knowledge data, and {Baseball_Event} may refer to an entity included in the server knowledge graph 120. {CalendarActivity, hasMeaning, Baseball_Event} may indicate that the schedule described in the calendar application is a baseball event. As described above, the device knowledge graph 140 may be generated by connecting the knowledge data to the server knowledge graph 120.

[0099] Figure 4 is a flowchart of a method for generating a pattern knowledge graph executed by device 130 according to an embodiment of the present disclosure. In an embodiment of the present disclosure, Figure 4 The operation may be included in Figure 2 Operation 207 of

[0100] Referring to Figure 4 , in operation 401, device 130 may identify an entity associated with the user's behavior and an entity associated with the user's context from the device knowledge graph 140. In an embodiment of the present disclosure, the device knowledge graph 140 may include entities associated with the behavior of the user of device 130. For example, the user's behavior may include the user's application usage, uniform resource locator (URL) access, and video playback on device 130. The device knowledge graph 140 may include entities learned from the user's application usage history, URL access history, and video playback history. For example, based on the user's video playback application usage history on device 130, the device knowledge graph 140 may include an entity indicating the video playback application.

[0101] In addition, the device knowledge graph 140 may include entities associated with the context of the user of the device 130. For example, the context of the user may refer to the context in which the user of the device 130 is located in the past, present, or future, such as the schedule described in the calendar application of the device 130. For example, when a baseball game is described in the calendar application of the device 130, the device knowledge graph 140 may include an entity indicating baseball. In this case, the entity indicating baseball may refer to the context associated with baseball.

[0102] In operation 403, the device 130 may generate a pattern knowledge graph based on entities associated with an action, entities associated with a context, and the relationships between entities associated with an action and entities associated with a context. For example, when a user performs a specific action in a specific context while using the device 130, the device 130 may determine the connection relationship between the specific context and the specific action. For example, when the user enters a search term related to baseball on the device 130, the device 130 may identify an entity indicating baseball. Additionally, when the user enters a search term related to baseball and then executes a video playback application, an entity indicating the video playback application may be identified.

[0103] The device 130 may generate a pattern knowledge graph indicating the behavior pattern of the user by connecting the entity indicating baseball to the entity indicating the video playback application. That is, the device 130 may determine the pattern in which the user executes the video playback application in the context associated with baseball.

[0104] Figure 5A is a flowchart of a method for generating a pattern knowledge graph according to the association relationship between entities, performed by the device 130 according to an embodiment of the present disclosure. In an embodiment of the present disclosure, Figure 5A The operations of Figure 4 may be included in

[0105] Reference Figure 5A , in operation 501, the device 130 may determine the association relationship between an entity associated with a context and an entity associated with an action. In an embodiment of the present disclosure, the association relationship between entities may refer to the relationship in which entities are connected to each other in the knowledge graph. For example, the device 130 may identify an entity indicating baseball as an entity associated with the context of the user. The device 130 may identify an entity associated with the video playback application as an entity associated with the user's action. When the user enters a search term related to baseball and then executes the video playback application, the device 130 may determine such an association relationship, that is, the entity indicating baseball and the entity indicating the video playback application are associated with each other.

[0106] In operation 503, device 130 may generate a pattern knowledge graph based on the determined association relationship. In an embodiment of the present disclosure, device 130 may generate a pattern for generating a pattern knowledge graph based on the association relationship between an entity associated with a context and an entity associated with an action. For example, an entity associated with a context and an entity associated with an action may be connected to form an RDF structure. In this case, when the same RDF structure repeats in a time series, the repeated RDF structure may be generated as a pattern. In some embodiments, the number of occurrences of the same RDF structure is counted. When the counted number meets or exceeds a specific threshold, the repeated RDF structure is considered a pattern.

[0107] For example, an entity indicating baseball and an entity indicating a video playback application may be connected to form an RDF structure. When the RDF structure in which the entity indicating baseball and the entity indicating the video playback application are connected repeats in a time series, the repeated RDF structure may be generated as a pattern. That is, the structure in which the entity indicating baseball and the entity indicating the video playback application are connected may be generated as a pattern. Device 130 may generate a pattern knowledge graph indicating the behavior pattern of the user based on the generated pattern. Device 130 may store the generated pattern knowledge graph in the memory of device 130 and may use the stored pattern knowledge graph to update device knowledge graph 140.

[0108] According to an embodiment of the present disclosure, a pattern may mean that a user frequently (e.g., a specific threshold or more, e.g., as described above) performs a specific action under various conditions. Device 130 may analyze the actions in device 130 used by the user to identify which action the user frequently performs under various conditions and accordingly pattern the user's behavior. When the user corresponds to a specific condition, device 130 may provide a recommendation associated with the specific action to the user based on the user's pattern. For example, when the user watches a video playback application (e.g., YouTube) during a month and then repeats the action of exercising 100 times, device 130 may determine the action of exercising as a pattern that occurs after watching the video playback application. Thereafter, when the user executes the video playback application through device 130, device 130 may provide a recommendation (e.g., notification display, vibration, etc.) associated with exercising to the user. Figures 5B to 5D Show a specific example of pattern determination.

[0109] In Figures 5B to 5DIn [the above], the atlas database 510 may be included in the device 130. The atlas database 510 may include a server knowledge graph 512 and a device knowledge graph 514. The server knowledge graph 512 is a knowledge graph constructed in the server 110 and stored in the device 130, and may refer to a knowledge graph indicating common knowledge or knowledge associated with recommendations. The server knowledge graph 512 may correspond to Figure 1 the server knowledge graph 120. Additionally, the device knowledge graph 514 is a knowledge graph constructed in the device 130, and may correspond to Figure 1 the device knowledge graph 140.

[0110] According to an embodiment of the present disclosure, a user may repeatedly perform a specific behavior under specific conditions. When the number of times of repeatedly performing the specific behavior under the specific conditions is greater than or equal to a threshold, the device 130 may recognize the user's repeated behavior greater than or equal to the threshold as a pattern. In addition to the above threshold, the pattern recognition of the device 130 may also be determined based on various measured values.

[0111] For example, when the user has used the device 130 for one month, a device knowledge graph 514 as shown in Figure 5B may be generated. The user may repeatedly perform the activity of exercising after the activity of watching YouTube. As shown in Figure 5B , when the user of the device 130 exercises, the activity entity in the device knowledge graph 514 may be connected to the entity associated with exercise in the server knowledge graph 512. When the user of the device 130 watches YouTube, the activity entity in the device knowledge graph 514 may be connected to the entity associated with YouTube in the server knowledge graph 512.

[0112] In an embodiment of the present disclosure, when the behavior repeated three or more times per month is set as a pattern, the exercise after watching YouTube may be determined as the pattern of the user of the device 130. When the pattern is determined as described above, the pattern of the user may be learned as shown in Figure 5C . That is, the user pattern of exercising after watching YouTube may be expressed as the "Pattern1" entity of the device knowledge graph 514. However, the ontology scheme of the pattern may be expressed differently and is not limited to the example shown in Figure 5C . Figure 5D An example showing the device 130 making content recommendations to the user based on the pattern is shown.

[0113] Referring to Figure 5D , the device 130 may provide content recommendations to the user of the device 130 by utilizing the knowledge of the above pattern and predefined inference rules. For example, in Figure 5CIn [the above example], when a user of device 130 performs an activity associated with YouTube (such as Activity 7), device 130 can use the knowledge of Mode 1 to predict that the user will perform an activity associated with exercise. Therefore, device 130 can recommend content associated with exercise to the user. For example, device 130 can display a notification associated with exercise on the display of device 130, or can output vibration or sound. That is, device 130 can predict which behavior the user of device 130 will perform based on inference and a mode (the mode is based on queries or rules), and make a recommendation associated therewith.

[0114] Figure 6A is a flowchart of a method for updating a device knowledge graph executed by device 130 according to an embodiment of the present disclosure. In an embodiment of the present disclosure, Figure 6A The operation of [this part] can be included in Figure 2 operation 209 of [this part].

[0115] Referring to Figure 6A , in operation 601, device 130 can select an entity in the device knowledge graph 140 to which the mode knowledge graph is to be connected from among the entities. For example, device 130 can identify one or more entities for connecting the mode knowledge graph from among the entities stored in the device knowledge graph 140. For example, when an entity indicating the user is included in the device knowledge graph 140, device 130 can select the entity indicating the user to connect the mode knowledge graph. A specific method for device 130 to select an entity to connect the mode knowledge graph will be described below.

[0116] In operation 603, device 130 can connect the mode knowledge graph to the selected entity. For example, device 130 can select an entity indicating the user from among the entities included in the device knowledge graph 140, and connect the mode knowledge graph to the entity indicating the user. Connecting the mode knowledge graph indicating the mode to the entity indicating the user can mean that the behavior pattern of the user of device 130 is the behavior pattern indicated by the mode knowledge graph. For example, a mode knowledge graph including an entity indicating baseball and an entity indicating a video playback application can be connected to the entity indicating the user in the device knowledge graph 140. In this way, when the user of device 130 is in a context associated with baseball, a mode of executing a video playback application can be expressed. As described above, the device knowledge graph 140 can be updated by connecting the mode knowledge graph to some entities selected from the device knowledge graph 140. That is, the updated device knowledge graph 140 can include information related to the behavior pattern of the user.

[0117] In an embodiment of the present disclosure, a specific method for device 130 to select an entity to connect the mode knowledge graph is as follows.

[0118] In an embodiment of the present disclosure, the device 130 may select an entity to which the pattern knowledge graph is to be connected based on name similarity, instance similarity, or structural similarity.

[0119] In an embodiment of the present disclosure, name similarity may refer to label similarity of sub-properties of each category. The device 130 may confirm the similarity of categories by comparing the names of the sub-properties of the categories (or entities) included in each of the multiple graphs. The categories may be combined based on the confirmed similarity. For example, referring to Figure 6B , there may be a knowledge graph 1 having categories such as Item, DVD, Book, and CD, and a knowledge graph 2 having categories such as Volume, Essay, and Literature.

[0120] In an embodiment of the present disclosure, the name of the title attribute, which is one of the sub-properties of the Item class of the knowledge graph 1, may be the same as the name of the title attribute, which is one of the sub-properties of the Volume class of the knowledge graph 2. Since there are sub-properties with the same name, the device 130 may determine that the Item class of the knowledge graph 1 and the Volume class of the knowledge graph 2 are similar to each other. The device 130 may select the Volume class of the knowledge graph 2 as the entity to be connected to the Item class of the knowledge graph 1, and may combine the Item class of the knowledge graph 1 with the Volume class of the knowledge graph 2.

[0121] In an embodiment of the present disclosure, instance similarity may refer to the similarity of instances belonging to each category. The device 130 may combine the categories based on the similarity of the instances belonging to the categories included in each of the multiple graphs. For example, referring to Figure 6C , there may be a knowledge graph 1 having categories such as Item, DVD, Book, and CD, and a knowledge graph 2 having categories such as Volume, Essay, and Literature.

[0122] In an embodiment of the present disclosure, the book class of the knowledge graph 1 may include instances such as Bertrand Russell: My Life, Albert Camus: La Chute, etc. Additionally, the volume class of the knowledge graph 2 may include instances such as Bertrand Russell: My Life, Albert Camus: La Chute, etc. Since the instances of the book class of the knowledge graph 1 and the instances of the volume class of the knowledge graph 2 are the same as each other, the device 130 may determine that the book class of the knowledge graph 1 and the volume class of the knowledge graph 2 are similar to each other. The device 130 may select the volume class of the knowledge graph 2 as the entity to be connected to the book class of the knowledge graph 1, and may combine the book class of the knowledge graph 1 and the volume class of the knowledge graph 2.

[0123] In an embodiment of the present disclosure, structural similarity may refer to the similarity of the data types and configurations of the sub-properties of each class. The device 130 may confirm the similarity of classes by comparing the similarity of the data types and configurations of the sub-properties of the classes included in each of the multiple graphs. Classes may be combined based on the confirmed similarity. For example, referring to Figure 6D , there may be a knowledge graph 1 having classes such as item, DVD, book, and CD, and a knowledge graph 2 having classes such as volume, essay, and document.

[0124] In an embodiment of the present disclosure, the book class of the knowledge graph 1 may include sub-properties such as price, title, doi, pp, and author. The volume class of the knowledge graph 2 may include sub-properties such as number of pages, isbn, author, and title. In this case, the book class of the knowledge graph 1 and the volume class of the knowledge graph 2 may have the same sub-properties such as title and author. Additionally, since the price, pp in the sub-properties of the book class of the knowledge graph 1 and the number of pages in the sub-properties of the volume class of the knowledge graph 2 are all expressed as integers, the price, pp, and the number of pages may be determined as similar data. Pp is a term referring to the page range of a book, and pp. 3-6 may mean pages 3 to 6 of the book. Additionally, since the title in the sub-properties of the book class of the knowledge graph 1 and the title in the sub-properties of the volume class of the knowledge graph 2 are all expressed as strings, the title may be determined as similar data. Additionally, the doi (Digital Object Identifier) in the sub-properties of the book class of the knowledge graph 1 and the isbn (International Standard Book Number) in the sub-properties of the volume class of the knowledge graph 2 are expressed as uniform resource locators (uri), and the doi and isbn may be determined as similar data.

[0125] As described above, since the device 130 determines that the configuration and data of the sub-properties of the Book class in the knowledge graph 1 are similar to the configuration and data of the sub-properties of the Book class in the knowledge graph 2, the device 130 can select the Book volume class in the knowledge graph 2 as the entity to be connected to the Book class in the knowledge graph 1, and combine the Book class in the knowledge graph 1 with the Book volume class in the knowledge graph 2.

[0126] According to an embodiment of the present disclosure, the method for updating the device knowledge graph with reference to the Figure 6A pattern knowledge graph described above can be equally applied to the method for updating the server knowledge graph with reference to the pattern knowledge graph. For example, the device 130 can select at least one entity in the server knowledge graph 120 to which the pattern knowledge graph is to be connected. The device 130 can connect the pattern knowledge graph to the selected at least one entity. Since the method for selecting the entity to which the pattern knowledge graph is to be connected has been described in the method for updating the device knowledge graph above, its description is omitted for convenience.

[0127] Figure 7 is a flowchart of a method for providing a function by the device 130 according to an embodiment of the present disclosure by using a pattern knowledge graph.

[0128] Referring to Figure 7 , in operation 701, the device 130 can obtain context data indicating the current context of the device 130. The current context of the device 130 may refer to the specific environment in which the current device 130 is located. For example, the current context of the device 130 can be determined by the current location of the device 130, whether the user uses the device 130, etc.

[0129] In an embodiment of the present disclosure, the context data may refer to information related to the current context of the device 130. For example, when the user of the device 130 goes to watch a baseball game, the device 130 can confirm that the current location of the device 130 is at the baseball stadium, and can determine the current context of the device 130 as the context associated with the baseball game. In addition, the device 130 can obtain information related to the baseball game as context data.

[0130] In addition, when the user of the device 130 is sleeping, the device 130 can confirm that the user does not use the device 130 within a specific time and can determine the current context of the device 130 as the context that the user is sleeping. In addition, the device 130 can obtain information related to the user's sleep as context data.

[0131] For example, when the user of device 130 is located at the user's home, device 130 can confirm the current location of device 130 and can determine the current context of device 130 as the context where the user is at the user's home. Additionally, device 130 can obtain information related to the context where the user is at the user's home as context data.

[0132] In operation 703, device 130 can determine the function to be provided to the user by using the context data and the pattern knowledge graph. That is, device 130 can use the context data to determine what context the user of device 130 is currently in. In the pattern knowledge graph, entities associated with the determined current context can be identified. For example, when the context data includes information related to baseball, device 130 can determine that the user of device 130 is in a context associated with baseball. Device 130 can determine what actions the user mainly performs in the context indicated by the context data from the pattern knowledge graph. For example, when the context indicated by the context data is baseball, device 130 can identify entities associated with baseball and entities associated with a video playback application from the pattern knowledge graph. Device 130 can determine the pattern in which the user mainly executes the video playback application in the context associated with baseball based on the connection relationships between the identified entities. Therefore, device 130 can determine to execute the video playback application as the function to be provided to the user. In an embodiment of the present disclosure, the process of determining the function to be provided to the user may include determining the recommended content to be provided to the user.

[0133] In operation 705, device 130 can execute the determined function. In an embodiment of the present disclosure, device 130 can execute the function determined in operation 703 without receiving a separate input from the user. In another embodiment of the present disclosure, device 130 can display a user interface associated with the function determined in operation 703 on the display of device 130. When a user input is received through the displayed user interface, device 130 can execute the determined function. However, when a user input is not received through the displayed user interface, device 130 may not execute the determined function. As described above, device 130 can determine the function based on the context data indicating the current context of device 130 and can recommend the determined function to the user through the user interface. Device 130 can execute the recommended function automatically or based on the presence or absence of a user input.

[0134] Figure 8 is a flowchart of a method for determining the recommended content executed by device 130 according to an embodiment of the present disclosure. In an embodiment of the present disclosure, Figure 8 The operation of Figure 7 can be included in the operation 703 of

[0135] Reference Figure 8 In operation 801, device 130 may determine the first recommended content based on the context data and the server knowledge graph 120. In an embodiment of the present disclosure, the context data may refer to data indicating the current context of device 130. In an embodiment of the present disclosure, device 130 may determine the first recommended content based on the degree of association between the server knowledge graph 120 and the context data indicating the current context. For example, when the context data indicates a context associated with baseball, device 130 may identify that an entity indicating baseball and an entity indicating baseball information are associated in the server knowledge graph 120. Device 130 may determine the baseball information as the first recommended content.

[0136] In operation 803, device 130 may determine the existence or non - existence of a pattern knowledge graph. For example, device 130 may determine whether a pattern knowledge graph generated based on the entities and the relationships between the entities in the device knowledge graph is stored in device 130. In an embodiment of the present disclosure, when there is a pattern knowledge graph, device 130 may perform operation 807. In an embodiment of the present disclosure, when there is no pattern knowledge graph, device 130 may perform operation 805.

[0137] In operation 805, device 130 may determine the first recommended content as the recommended content. That is, when there is no pattern knowledge graph, device 130 may not be able to update the device knowledge graph based on the pattern knowledge graph. Therefore, device 130 may determine the first recommended content determined based on the context data indicating the current context and the server knowledge graph as the recommended content.

[0138] In operation 807, device 130 may determine the second recommended content based on the context data and the updated device knowledge graph. That is, when there is a pattern knowledge graph, device 130 may update the device knowledge graph based on the pattern knowledge graph. Device 130 may determine the second recommended content based on the context data indicating the current context of device 130 and the updated device knowledge graph.

[0139] For example, device 130 may determine the second recommended content based on the degree of association between the updated device knowledge graph and the context data indicating the current context. In an embodiment of the present disclosure, device 130 may identify the entities associated with the context data based on the pattern knowledge graph included in the updated device knowledge graph. For example, when the context data indicates a context associated with baseball, device 130 may identify that an entity indicating baseball and an entity indicating a video playback application are associated with each other in the updated device knowledge graph including the pattern knowledge graph. Device 130 may determine the video playback application as the recommended content.

[0140] In operation 809, device 130 may select a recommended content from among a first recommended content and a second recommended content. In an embodiment of the present disclosure, device 130 may select the recommended content based on weights assigned to the first recommended content and the second recommended content. For example, device 130 may input the first recommended content and the second recommended content into a sorting algorithm. Different weights may be assigned to the first recommended content and the second recommended content by the sorting algorithm. In this case, among the content to which weights are assigned, the content to which a higher weight is assigned may be recommended with a higher ranking. In an embodiment of the present disclosure, the sorting algorithm may refer to an algorithm that assigns weights to specific inputs input to the algorithm according to a preset criterion.

[0141] For example, among the first recommended content determined based on the server knowledge graph 120 and the second recommended content determined based on the updated device knowledge graph, a weight higher than that of the first recommended content may be assigned to the second recommended content determined based on the pattern knowledge graph reflecting the user's behavior pattern. Device 130 may select the recommended content to which a higher weight is assigned as the final recommended content. For example, device 130 may select the second recommended content to which a weight higher than that of the first recommended content is assigned as the final recommended content. Device 130 may display a user interface associated with the final recommended content on a display of device 130.

[0142] Therefore, a technical effect is achieved. In some embodiments, the CPU represents the knowledge graph as a set in a memory. The set may include, for example, a set of associations {association1, association2, association3,...}. This is a non-limiting example. The establishment of the knowledge graph has been described above. For example, see Figure 5B (Inference of Pattern 1), Figure 6A 、 Figure 6B 、 Figure 6C non-limiting examples of (inference of associations based on instances, names, or structures), and also see the discussion in the above context, such as the sleep state and geographical location (at home, at a baseball field) that may be determined based on sensor input values. In some embodiments, when the current condition based on the input data has a match (e.g., high correlation or e.g., matching tags) with some elements of the knowledge graph (also considering the set), the algorithm sorts the associations and generates content recommendations. This is a non-limiting example.

[0143] Figure 9 Shows an example of content recommendation based on a server knowledge graph and content recommendation based on a device knowledge graph according to an embodiment of the present disclosure.

[0144] Refer to Figure 9, the content recommendation method 910 based on the server knowledge graph may refer to a method of recommending content based on the server knowledge graph 120 generated by the server 110. For example, information about "baseball game reservation" may be collected from the Mail application of the device 130, or information about the "baseball" schedule may be collected from the Calendar application of the device 130. In this case, the server 110 may identify the information about "baseball game reservation" and the information about the "baseball" schedule as a "baseball event".

[0145] The identified information may be connected to the entity indicating the "baseball event" in the server knowledge graph 120, and the entity indicating the "baseball event" may be connected to the entity indicating "baseball". By learning using big data, the entity indicating "baseball" in the server knowledge graph 120 may be connected to the entity indicating "News Category". That is, in the content recommendation method 910 based on the server knowledge graph, when a baseball event is identified, baseball news may be recommended.

[0146] According to an embodiment of the present disclosure, the content recommendation method 920 before user mode analysis may refer to a method of recommending content by the device 130 before analyzing the user's behavior pattern. That is, the content recommendation method 920 before user mode analysis may refer to a method of recommending content based on the device knowledge graph 140 before updating the mode knowledge graph. For example, the device recommendation knowledge generated based on the log history information of the device 130 and the server recommendation knowledge generated based on the server knowledge graph 120 are connected to generate the device knowledge graph 140. When a baseball event is identified based on the generated device knowledge graph 140, the device 130 may recommend content associated with baseball news.

[0147] According to an embodiment of the present disclosure, the content recommendation method 930 after user mode analysis may refer to a method of recommending content by the device 130 after analyzing the user's behavior pattern. That is, the content recommendation method 930 after user mode analysis may refer to a method of recommending content based on the updated device knowledge graph after updating the mode knowledge graph. For example, the mode knowledge graph 935 associated with the baseball event may be connected to the device knowledge graph 140. When a baseball event is identified, the mode knowledge graph 935 associated with the baseball event may indicate watching the baseball channel and using the YouTube application when watching the baseball channel. When a baseball event is identified, the device 130 may recommend the frequently watched YouTube baseball channels among the channels of the YouTube application based on the updated device knowledge graph in which the mode knowledge graph 935 is updated.

[0148] As referred to aboveFigure 9 As described above, when updating the device knowledge graph 140 based on the pattern knowledge graph 935 reflecting the user's behavior pattern, the device 130 may recommend user-customized content by using the updated device knowledge graph.

[0149] Figure 10A An example of a method for generating a device knowledge graph according to an embodiment of the present disclosure is shown.

[0150] Referring to Figure 10A , the server knowledge graph 1010 associated with a baseball game may be generated in advance by the server 110. For example, the server 110 may identify a baseball game as an event from baseball-related data (e.g., KBO, MLB, Ryu Hyun-jin, Choo Shin-soo, etc.). Entities indicating the baseball game event may be included in the server knowledge graph 1010. The server 110 may recommend baseball information when identifying a baseball game by learning using big data. For example, in the server knowledge graph 1010, entities indicating the baseball game event and entities indicating baseball information may be connected to each other. That is, when a baseball game is identified, baseball information may be determined as the recommended content. The server knowledge graph 1010 in FIG. 10 may refer to Figure 1 the server knowledge graph 120.

[0151] According to an embodiment of the present disclosure, the device 130 may generate a device knowledge graph 1 1020. For example, the device 130 may display the relationship of entities indicating a demand, an activity, or a profile with respect to an entity indicating a user. For example, information about the user's demand may be updated based on a form such as "the user has a demand". The above device knowledge graph 1 1020 may be predefined and stored in the device 130.

[0152] According to an embodiment of the present disclosure, the device 130 generates a device knowledge graph 2 1030 based on the server knowledge graph 1010, the device knowledge graph 1 1020, and current knowledge (e.g., log history information, etc.). In an embodiment of the present disclosure, the device knowledge graph 2 1030 may refer to Figure 1 the device knowledge graph 140.

[0153] According to an embodiment of the present disclosure, to generate a knowledge graph, device 130 may express data in a refined form by analyzing log history information generated by device 130, sensed values (e.g., real-time step count, etc.) accumulated in the database of device 130, multimedia file information (e.g., photos, videos, etc.), user behavior (e.g., searching for a specific word in an Internet browser or transmitting a message to a specific person), etc. However, these data fragments are in an atypical form and need to be converted into the knowledge graph form in order to derive meaningful user-related knowledge.

[0154] For example, when there is log history information indicating that the user of device 130 watched a baseball game from 20:00 to 21:00 by using the YouTube application, device 130 may generate a graph indicating that the user (USER) has watched (hasActivity) a baseball game (Activity). Additionally, device 130 may indicate that the user's activity is an activity associated with a baseball event by connecting the baseball event entity to the event entity connected to the activity entity. Additionally, device 130 may connect the baseball information entity to the need (Need) entity of the user (USER) based on the association between the baseball game entity and the baseball information entity in the server knowledge graph 1010. In this way, device 130 may reflect the knowledge associated with the recommendation of baseball information into the device knowledge graph 2 1030. That is, device 130 may update the device knowledge graph 1 1020 by using the knowledge of the server knowledge graph 1010, the user's log history information, etc. with the device knowledge graph 2 1030 based on the structure of the device knowledge graph 1 1020.

[0155] In an embodiment of the present disclosure, Figure 10B Another example of the device knowledge graph 2 1030 is shown. Refer to Figure 10B , as described above, when there is log history information indicating that the user watched a baseball game by using the YouTube application at a specific time, device 130 may generate a graph indicating that the user (USER) has watched (hasActivity) a baseball game (Activity). Since the user has watched the YouTube application, device 130 may connect the YouTube entity to the activity entity and connect the baseball entity to the event entity connected to the activity. However, the ontology schemes for modeling the user's behavior through the graph can be various and are not limited to Figure 10B the example of.

[0156] As described above, a knowledge graph associated with a user can be generated on a running device. Even when not continuously recording a large amount of log data, device 130 can manage user-related knowledge in a form that can be inferred with a small amount of resources.

[0157] Figure 10C Show the generation Figure 10A of the specific process of the device knowledge graph.

[0158] Refer to Figure 10C , the graph database 1040 can be included in device 130. The graph database 1040 can include the server knowledge graph 1010 and the device knowledge graph 1. The server knowledge graph 1010 is a knowledge graph constructed in server 110 and stored in device 130, and can refer to a knowledge graph indicating common knowledge or knowledge associated with recommendations. In addition, the device knowledge graph 1 1020 can refer to a knowledge graph constructed in device 130.

[0159] In an embodiment of the present disclosure, when the user of device 130 does not perform any actions, the device knowledge graph 1 1020 of device 130 may not be generated. Figure 10D shows an example of generating a device knowledge graph according to the user's behavior.

[0160] Refer to Figure 10D , when there is log history information indicating that the user of device 130 watches a game of the LA Dodgers through the YouTube application at a specific time (e.g., 20:00 to 21:00), device 130 can generate the device knowledge graph 1 1020. In this way, device 130 can confirm the connectivity between the device knowledge graph and the service knowledge graph. That is, the event entity of the device knowledge graph 1 1020 can be connected to the "LA Dodgers" entity of the server knowledge graph 1010. Device 130 can recommend baseball news to the user (USER) through the connection relationship from the "LA Dodgers" entity to the "Baseball News" entity. After the recommendation, device 130 can use queries based on the inference of the ontology scheme and usage rules. For example, device 130 can select recommended content by using queries to find entities directly or indirectly connected to the user entity through the edge of "hasSuggestion".

[0161] As described above, the behavior of the user of the learning device 130 can be learned and recorded in the device knowledge graph (e.g., device knowledge graph 1 1020), and the device knowledge graph and the server knowledge graph (e.g., server knowledge graph 1010) can be connected so that the device 130 recommends content to the user based on the connected knowledge graph (e.g., device knowledge graph 2 1030). However, the device 130 can recommend content by analyzing the user's behavior patterns, as described above with regard to recommending content based on the user's behavior. Therefore, Figure 11 An example of updating the device knowledge graph 140 based on a pattern knowledge graph indicating the behavior pattern of the user is shown.

[0162] Figure 11 Examples of a device knowledge graph and a pattern knowledge graph according to an embodiment of the present disclosure are shown.

[0163] Referring to Figure 11 , a device knowledge graph 1100 and a pattern knowledge graph 1150 are shown. In this case, Figure 11 the device knowledge graph 1100 of Figure 1 can refer to the device knowledge graph 140 of

[0164] In an embodiment of the present disclosure, the device knowledge graph 1100 may include entities generated based on log history information related to the operation of the device 130. For example, the device knowledge graph 1100 includes entities indicating a user, entities indicating a profile of the user, entities indicating an activity of the user, entities indicating an event, entities indicating a need of the user, entities indicating enjoyable, entities indicating emotion, entities indicating sports, and entities indicating news.

[0165] Referring to Figure 11 the device knowledge graph 1100, an entity indicating a specific event among the activities of the user of the device 130 can be connected to an entity indicating baseball, an entity indicating a hangout, and an entity indicating YouTube. This can mean that the event associated with the user is an event associated with baseball, an event associated with a hangout, or an event associated with YouTube.

[0166] According to an embodiment of the present disclosure, in the device knowledge graph 1100, an entity indicating baseball can be connected to an entity indicating the user's interest. This may mean that entities associated with baseball are classified into a type indicating the user's interest.

[0167] Additionally, according to an embodiment of the present disclosure, in the device knowledge graph 1100, an entity indicating a frequently visited place can be connected to an entity indicating the user's context. This may mean that entities associated with the frequently visited place are classified into a type indicating the user's context.

[0168] Additionally, according to an embodiment of the present disclosure, in the device knowledge graph 1100, an entity indicating YouTube can be connected to an entity indicating the user's behavior. This may mean that entities associated with YouTube are classified into a type indicating the user's behavior.

[0169] In an embodiment of the present disclosure, the pattern knowledge graph 1150 may include entities generated based on the user's behavior patterns. For example, the pattern knowledge graph 1150 may include an entity indicating a pattern, an entity indicating a first pattern (pattern 1), an entity indicating YouTube, an entity indicating baseball, an entity indicating an event, and an entity indicating an application.

[0170] For example, the entity indicating YouTube and the entity indicating baseball can be connected to the entity indicating the first pattern. This may mean that the first pattern is a pattern generated in the context associated with baseball. Additionally, the first pattern may indicate the user behavior of executing the YouTube application. Additionally, the entity indicating YouTube can be classified into a type indicating an application, and the entity indicating baseball can be classified into a type indicating an event.

[0171] According to an embodiment of the present disclosure, the device knowledge graph 1100 can be updated by connecting the pattern knowledge graph 1150 to an entity indicating the user among the entities included in the device knowledge graph 1100. That is, the device knowledge graph 1100 to which the pattern knowledge graph 1150 is connected can refer to the updated device knowledge graph.

[0172] According to an embodiment of the present disclosure, the data generated when generating the device knowledge graph 1100 may include user activity information containing user privacy data. The acquisition and use of this user privacy data comply with the provisions of relevant laws and regulations. The pattern knowledge graph 1150 may include user pattern information that does not contain user privacy data. For example, the user privacy data may include information about the user's schedule on device 130, information about the names of the applications used, and information about the search history.

[0173] As described above with reference to Figure 11As described above, the device knowledge graph 1100 can be updated based on the pattern knowledge graph 1150 that reflects the user's behavior patterns. The device 130 can provide functions to the user by using the updated device knowledge graph.

[0174] Figure 12 is a block diagram of the device 130 according to an embodiment of the present disclosure.

[0175] Referring Figure 12 , according to an embodiment of the present disclosure, the device 130 may include a user inputter 1210, an outputter 1220, a communicator 1230, and a processor 1240. The structure of the device 130 can generally be implemented by one or more processors using one or more memories. One or more memories may include instructions that implement the functions of the structure (e.g., the user inputter 1210, the outputter 1220, the communicator 1230, and generally other structures of the device) in the form of computer code. However, Figure 12 not all of the elements shown are necessary for the device 130. The device 130 may include more elements than Figure 12 those shown, or may include fewer elements than Figure 12 those shown. For example, Figure 13 is a detailed block diagram of the device 130 according to an embodiment of the present disclosure.

[0176] As Figure 13 shown, in addition to the user inputter 1210, the outputter 1220, the communicator 1230, and the processor 1240, the device 130 according to an embodiment of the present disclosure may further include a sensor 1310, an audio / video (A / V) inputter 1330, and a memory 1350.

[0177] The user inputter 1210 is a device that allows a user to input data to control the device 130. For example, the user inputter 1210 may include keys, dome switches, touchpads (such as capacitive touchpads, resistive touchpads, infrared beam sensing touchpads, surface acoustic wave touchpads, integrated strain gauge touchpads, or piezoelectric effect touchpads), scroll wheels, and roller switches, but is not limited thereto.

[0178] The user inputter 1210 may receive user input to provide recommended content based on the updated device knowledge graph.

[0179] The outputter 1220 may output an audio signal, a video signal, or a vibration signal. The outputter 1220 may include a display 1222, an audio outputter 1224, and a vibration motor 1226.

[0180] The display 1222 presents the information processed by the device 130. For example, the display may present a user interface to provide recommended content based on the updated device knowledge graph.

[0181] When the display 1222 and the touchpad form a hierarchical structure to constitute a touch screen, the display 1222 can also be used as an input device as well as an output device.

[0182] The audio output unit 1224 outputs the audio data received from the communicator 1230 or stored in the memory 1350. Additionally, the audio output unit 1224 outputs audio signals associated with the functions performed by the device 130 (e.g., call signal reception sound, message reception sound, and notification sound). The audio output unit 1224 may include a speaker, a buzzer, etc.

[0183] The vibration motor 1226 can output a vibration signal. For example, the vibration motor 1226 can output a vibration signal corresponding to the output of audio data or video data (e.g., call signal reception sound, message reception sound, etc.). Additionally, when a touch is input to the touch screen, the vibration motor 1226 can output a vibration signal.

[0184] The processor 1240 controls the overall operation of the device 130. That is, the processor 1240 can control one or more other components of the device 130 by executing the programs stored in the memory 1350. For example, the processor 1240 can control the operation of the device 130 by controlling the user input unit 1210, the output unit 1220, the sensor 1310, the communicator 1230, the A / V input unit 1330, etc.

[0185] Specifically, the processor 1240 can obtain log history information related to the operation of the device 130. The processor 1240 can receive the server knowledge graph 120 generated by the server 110 from the server 110. The processor 1240 can use the server knowledge graph 120 and the log history information to generate the device knowledge graph 140. The processor 1240 can generate a pattern knowledge graph associated with the behavior patterns of the users of the device 130 based on the entities and the relationships between the entities in the device knowledge graph 140. The processor 1240 can update the device knowledge graph 140 by adding the generated pattern knowledge graph to the device knowledge graph 140.

[0186] The processor 1240 can generate knowledge data by inputting the log history information into a knowledge graph generation model. The processor 1240 can generate the device knowledge graph 140 by updating at least a part of the server knowledge graph 120 based on the relationships between the knowledge data and the entities in the server knowledge graph 120.

[0187] The processor 1240 may identify entities associated with the user's behavior and entities associated with the user's context from the device knowledge graph 140. The processor 1240 may generate a pattern knowledge graph associated with the user's behavior pattern based on the entities associated with the behavior, the entities associated with the context, and the relationships between the entities associated with the behavior and the entities associated with the context.

[0188] The processor 1240 may determine the association relationship between the entities associated with the context and the entities associated with the behavior. The processor 1240 may generate a pattern knowledge graph based on the determined association relationship.

[0189] The processor 1240 may select at least one entity in the entities of the device knowledge graph 140 to which the pattern knowledge graph is to be connected. The processor 1240 may connect the pattern knowledge graph to the selected at least one entity.

[0190] The processor 1240 may obtain context data indicating the current context of the device 130. The processor 1240 may determine the function to be provided to the user by using the obtained context data and the pattern knowledge graph in the updated device knowledge graph. The processor 1240 may execute the determined function.

[0191] The processor 1240 may determine the recommended content to be provided to the user. To determine the recommended content, the processor 1240 may determine the first recommended content based on the context data indicating the user's current context and the server knowledge graph 120. The processor 1240 may determine the second recommended content based on the context data indicating the user's current context and the updated device knowledge graph 140. The processor 1240 may select at least one of the first recommended content or the second recommended content.

[0192] To determine the first recommended content, the processor 1240 may determine the first recommended content based on the degree of association between the server knowledge graph 120 and the context data indicating the current context.

[0193] To determine the second recommended content, the processor 1240 may determine the second recommended content based on the degree of association between the updated device knowledge graph and the context data indicating the current context.

[0194] To select the recommended content, among the weight assigned to the first recommended content and the weight assigned to the second recommended content, the processor 1240 may determine the content with the higher weight as the recommended content.

[0195] The sensor 1310 may detect the state of the device 130 or the state around the device 130, and transmit the detected information to the processor 1240.

[0196] The sensor 1310 may include at least one of the following: a geomagnetic sensor 1312, an acceleration sensor 1314, a temperature / humidity sensor 1316, an infrared sensor 1318, a gyro sensor 1320, a position sensor (e.g., GPS) 1322, an atmospheric pressure sensor 1324, a proximity sensor 1326, or an RGB sensor (illuminance sensor) 1328, but is not limited thereto. Since the functions of the corresponding sensors can be intuitively inferred from their names, detailed descriptions thereof will be omitted.

[0197] The communicator 1230 may include one or more elements for communicating with the server 110. For example, the communicator 1230 may include a short-range wireless communicator 1232, a mobile communicator 1234, and a broadcast receiver 1236.

[0198] The short-range wireless communicator 1232 may include a Bluetooth communicator, a Bluetooth Low Energy (BLE) communicator, a Near Field Communication communicator, a Wireless Local Area Network (WLAN) (Wi-Fi) communicator, a Zigbee communicator, an Infrared Data Association (IrDA) communicator, a Wi-Fi Direct (WFD) communicator, an Ultra Wideband (UWB) communicator, or an Ant+ communicator, but is not limited thereto.

[0199] The mobile communicator 1234 may transmit and receive wireless signals to and from at least one of a base station, an external terminal, or a server on a mobile communication network. Examples of the wireless signals may include various formats of data to support the transmission and reception of voice call signals, video call signals, or text or multimedia messages.

[0200] The broadcast receiver 1236 may receive a broadcast signal and / or broadcast-related information from the outside via a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. According to an implementation example, the device 130 may not include the broadcast receiver 1236.

[0201] The A / V inputter 1330 may receive an audio signal or a video signal, and may include a camera 1332, a microphone 1334, and so on. The camera 1332 may obtain an image frame such as a still image or a moving image through an image sensor in a video call mode or a shooting mode. The image captured by the image sensor may be processed by the processor 1240 or a separate image processor (not shown).

[0202] The image frame processed by the camera 1332 may be stored in the memory 1350 or may be transmitted to the outside through the communicator 1230. According to the configuration of the terminal, the camera 1332 may include two or more cameras.

[0203] The microphone 1334 can receive an external audio signal and process the external audio signal into electronic voice data. For example, the microphone 1334 can receive an audio signal from an external device or a person. The microphone 1334 can use various noise cancellation algorithms to cancel the noise generated during the reception of the external audio signal.

[0204] The memory 1350 can store programs for the processing and control of the processor 1240, and can store data input to the device 130 or data output from the device 130.

[0205] The memory 1350 can include at least one storage medium selected from the following: flash memory, hard disk, multimedia card micro memory, card type memory (e.g., SD or XD memory), random access memory (RAM), static random access memory (SRAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), programmable read only memory (PROM), magnetic memory, magnetic disk, and optical disk.

[0206] The memory 1350 can store the updated device knowledge graph in the database by adding the server knowledge graph 120, the device knowledge graph 140, and the pattern knowledge graph reflecting the behavior pattern of the user of the device, and can store the pattern knowledge graph. In addition, the memory 1350 can store log history information and knowledge data generated from the log history information.

[0207] The programs stored in the memory 1350 can be classified into multiple modules according to their functions. For example, the programs can be classified into a user interface (UI) module 1352, a touch screen module 1354, and a notification module 1356.

[0208] The UI module 1352 can provide a dedicated UI or a graphical user interface (GUI) that interoperates with the device 130 according to an application. The touch screen module 1354 can detect a touch gesture of a user on the touch screen and transmit information about the touch gesture to the processor 1240. According to an embodiment of the present disclosure, the touch screen module 1354 can identify and analyze a touch code. The touch screen module 1354 can be implemented as separate hardware including a controller.

[0209] Various sensors can be provided inside or near the touch screen to detect a touch on the touch screen or a hover above the touch screen. An example of a sensor for detecting a touch on the touch screen can be a tactile sensor. The tactile sensor can sense the contact of a specific object at or beyond human perception. The tactile sensor can detect various information such as the roughness of the contact surface, the hardness of the contact object, or the temperature of the contact point.

[0210] Additionally, an example of a sensor that detects touches on a touch screen can be a proximity sensor.

[0211] A proximity sensor is a sensor that detects the presence or absence of an object approaching a detection surface or an object existing near the sensor without mechanical contact by using electromagnetic force or infrared light. User touch gestures can include tapping, touch and hold, double-tapping, dragging, panning, flicking, drag and drop, swiping, and so on.

[0212] The notification module 1356 can output a signal for notifying that an event has occurred in the device 130. The notification module 1356 can output the notification signal in the form of a video signal through the display 1222. The notification module 1356 can output the notification signal in the form of an audio signal through the audio outputter 1224. The notification module 1356 can output the notification signal in the form of a vibration signal through the vibration motor 1226.

[0213] Figure 14 is a block diagram of the processor 1240 of the device 130 according to an embodiment of the present disclosure.

[0214] Reference Figure 14 , the device 130 according to an embodiment of the present disclosure can include a user inputter 1210, a processor, and a memory 1350. Additionally, the processor 1240 can include a device knowledge data generator 1410, a knowledge data manager 1420, and a function recommender 1430. Additionally, the structure of the device 130 can generally be implemented by one or more processors using one or more memories. The one or more memories can include instructions that implement the functions of the structure (e.g., the device knowledge data generator 1410, the knowledge data manager 1420, the function recommender 1430, and generally other structures of the device) in the form of computer code. However, Figure 14 not all of the elements shown are necessary for the device 130. The device 130 can include more elements than Figure 14 those shown, or can include fewer elements than Figure 14 those shown.

[0215] Reference Figure 14 , a user of the device 130 can input a request for a recommended function through the user inputter 1210. In an embodiment of the present disclosure, the recommended function can refer to a function among the various functions of the device 130 that is to be provided to the user of the device 130. For example, a user of the device 130 can input a request for a recommended function to the device 130 through a physical device such as a keyboard or a touchpad. Additionally, a user of the device 130 can execute the Bixby application and input a request for a recommended function to the device 130 by voice. A request for a recommended function can be input to the device 130 by various methods, and the present disclosure is not limited to the above methods.

[0216] The device knowledge data generator 1410 may obtain log history information related to the operation of the device 130. The log history information may be stored in the memory 1350. Alternatively, the log history information may be directly input into the device knowledge data generator 1410 without being stored in the memory 1350. In an embodiment of the present disclosure, the device knowledge data generator 1410 may generate knowledge data by inputting the log history information into a knowledge graph generation model. In this case, the knowledge data may refer to data expressed in text form.

[0217] In an embodiment of the present disclosure, the device knowledge data generator 1410 may obtain the server knowledge graph 120 generated by the server 110 from the memory 1350. Additionally, the device knowledge data generator 1410 may update at least a portion of the server knowledge graph 120 by combining the server knowledge graph 120 with the knowledge data. In this case, the updated server knowledge graph may refer to the device knowledge graph 140. The device knowledge data generator 1410 may store the device knowledge graph 140 in the memory 1350.

[0218] In an embodiment of the present disclosure, the device knowledge data generator 1410 may store the knowledge data generated based on the log history information in the memory 1350. In an embodiment of the present disclosure, the device knowledge data generator 1410 may store the device knowledge graph 140 in the memory 1350 in, for example, the text form of the knowledge data. In this case, the knowledge data in text form may be converted into the knowledge graph form and stored in the memory 1350.

[0219] According to an embodiment of the present disclosure, the knowledge data manager 1420 may obtain entities associated with the context of the user of the device 130 and entities associated with the behavior of the user from the device knowledge graph 140 stored in the memory 1350. Pattern knowledge graphs may be generated based on the entities associated with the context, the entities associated with the behavior, and the relationships between the entities associated with the context and the entities associated with the behavior. Alternatively, the knowledge data manager 1420 may generate pattern knowledge data in text form.

[0220] According to an embodiment of the present disclosure, the knowledge data manager 1420 may update the device knowledge graph 140 by adding the pattern knowledge graph to the device knowledge graph 140. The knowledge data manager 1420 may store the updated device knowledge graph in the memory 1350.

[0221] According to an embodiment of the present disclosure, the function recommender 1430 may obtain context data indicating the current context of the device 130. The function recommender 1430 may determine functions to be provided to the user by using the context data and the pattern knowledge graph included in the device knowledge graph. The function recommender 1430 may transmit information about the determined functions to the user inputter 1210. For example, the function recommender 1430 may transmit information about recommended content to be provided to the user to the user inputter 1210.

[0222] In an embodiment of the present disclosure, the information about the determined functions transmitted to the user inputter 1210 may be displayed on the outputter 1220. For example, the information about the determined functions may be visually displayed on the display 1222 of the device 130, may be transmitted as sound through the audio outputter 1224, or may be transmitted as vibration through the vibration motor 1226. However, the information about the determined functions may be transmitted to the user in various forms, and the present disclosure is not limited to the above examples.

[0223] Figure 15 is a block diagram of the device knowledge data generator 1410 of the device 130 according to an embodiment of the present disclosure.

[0224] In Figure 15 , the device knowledge data generator 1410 according to an embodiment of the present disclosure may include a knowledge data generator 1510 and a server knowledge graph updater 1520. However, Figure 15 not all of the elements of the device knowledge data generator 1410 shown are necessary for the device knowledge data generator 1410. The device knowledge data generator 1410 may include more elements than those shown in Figure 15 or may include fewer elements than those shown in Figure 15 .

[0225] Referring to Figure 15 , the knowledge data generator 1510 may obtain log history information related to the operation of the device 130. For example, the device 130 may generate knowledge data from log history information associated with a schedule in the calendar application stored in the device 130 (e.g., calendar, "Baseball game", 20190705:17:00, 20190705:21:00) or log history information associated with the search history of the YouTube application in the device 130 (App, YoutubeApp, "Baseball Highlights (Baseball Highlights) " 20190705:20:50 or App, YoutubeApp, "I like baseball", 20190705:21:18).

[0226] According to an embodiment of the present disclosure, the knowledge data generator 1510 may generate knowledge data by inputting log history information into a knowledge graph generation model. For example, the knowledge data may include data such as {User, hasActivity, CalendarActivity}, {CalendarActivity, hasTitle, "baseball game"}, {CalendarActivity, hasStartTime, 2019}, {User, hasActivity, AppUsage1}, {AppUsage1, hasApp, Youtube}, and {AppUsage1, hasVideoTitle, "BaseballHighlight"}. In this case, {User, hasActivity, CalendarActivity} may mean that the user has an activity described in the calendar application. Additionally, {CalendarActivity, hasTitle, "baseball game"} may mean that the name of the activity described in the calendar application is "baseball game".

[0227] According to an embodiment of the present disclosure, the server knowledge graph updater 1520 may generate the device knowledge graph 140 by updating at least a portion of the server knowledge graph 120 based on the relationship between the entities in the server knowledge graph 120 and the knowledge data. For example, the server knowledge graph updater 1520 may connect the knowledge data to the entities in the server knowledge graph 120. For example, the server knowledge graph updater 1520 may connect knowledge data such as {CalendarActivity, hasMeaning,Baseball_Event} and {AppUsage1, hasMeaning, Youtube_Baseball} to the entities in the server knowledge graph 120. In this case, {Baseball_Event} and {Youtube_Baseball} may mean that the entities in the server knowledge graph 120 are represented as text. For example, {CalendarActivity, hasMeaning,Baseball_Event} may mean that the activity described in the calendar application is a baseball game event. Additionally, {AppUsage1,hasMeaning, Youtube_Baseball} may mean that the application used by the user is the YouTube application associated with baseball.

[0228] According to an embodiment of the present disclosure, the server knowledge graph updated based on knowledge data may refer to the device knowledge graph 140. The server knowledge graph updater 1520 may store the generated device knowledge graph 140 in the memory 1350.

[0229] Figure 16 is a block diagram of the knowledge data manager 1420 of the device 130 according to an embodiment of the present disclosure.

[0230] Reference Figure 16 , according to an embodiment of the present disclosure, the knowledge data manager 1420 may include a user behavior information and user context information extractor 1610, a user behavior pattern manager 1620, and a recommendation function generator 1630. However, Figure 16 all the elements of the knowledge data manager 1420 shown are not necessary for the knowledge data manager 1420. The knowledge data manager 1420 may include more elements than Figure 16 those shown, or may include fewer elements than Figure 16 those shown.

[0231] Reference Figure 16 , the user behavior information and user context information extractor 1610 may obtain the behavior information and the context information of the user of the device 130 from the device knowledge graph 140 stored in the memory 1350. For example, the behavior information of the user may refer to information about the user's behavior, such as the user's application usage, URL access, and video playback on the device 130. Additionally, the context of the user may refer to information about the context in which the user of the device 130 is located in the past, present, or future, such as the schedule described in the calendar application of the device 130.

[0232] In an embodiment of the present disclosure, the behavior information and the context information of the user may be extracted from the device knowledge graph 140 and stored in the memory 1350. When the behavior information and the context information of the user are pre-stored in the memory 1350, the user behavior information and user context information extractor 1610 may request the stored behavior information and the stored context information of the user by transmitting an inquiry message to the memory 1350. In response to the requested inquiry message, the memory 1350 may transmit the stored behavior information and the stored context information of the user to the user behavior information and user context information extractor 1610. The user behavior information and user context information extractor 1610 may transmit the received behavior information and the received context information of the user to the user behavior pattern manager 1620.

[0233] According to an embodiment of the present disclosure, the user behavior pattern manager 1620 may generate a pattern knowledge graph based on the received behavior information of the user, the received context information of the user, and the relationship between the behavior information of the user and the context information of the user. That is, the user behavior pattern manager 1620 may generate a pattern knowledge graph based on the entities associated with the behavior, the entities associated with the context, and the relationship between the entities associated with the behavior and the entities associated with the context. For example, the user behavior pattern manager 1620 may generate knowledge data associated with the pattern, such as {User, hasPattern, Pattern1}, {Pattern1, hasUserContext, Baseball_Event}, and {Pattern1, hasUserAction, Youtube_Baseball}.

[0234] For example, {User, hasPattern, Pattern1} may indicate that the user of the device 130 has a behavior pattern of Pattern1. Additionally, {Pattern1, hasUserContext, Baseball_Event} may indicate that Pattern1 is a pattern generated when the user is in a context associated with a baseball game event. {Pattern1, hasUserAction, Youtube_Baseball} may indicate that the behavior of the user indicated by Pattern1 is to execute the YouTube application associated with baseball.

[0235] According to an embodiment of the present disclosure, the user behavior pattern manager 1620 may determine criteria (such as time, day, place, and user activity type) associated with the behavior of the user to generate a pattern. The user behavior pattern manager 1620 may count the behavior of the user of the device 130 based on the determined criteria.

[0236] When the value obtained by counting the behavior of the user in a specific context exceeds a certain threshold, the user behavior pattern manager 1620 may determine the corresponding behavior as a pattern. For example, when the user of the device 130 executes the YouTube application in the context of going to a baseball game, the user behavior pattern manager 1620 may count the number of times the user executes the YouTube application when going to a baseball game. When the counted number exceeds a certain threshold, the behavior of the user executing the YouTube application when going to a baseball game may be determined as a pattern. In an embodiment of the present disclosure, the pattern may be determined according to the probability, dispersion degree, etc. of the behavior of the user in a specific context.

[0237] The user behavior pattern manager 1620 may update the device knowledge graph 140 by generating a pattern knowledge graph based on knowledge data associated with the above patterns and adding the pattern knowledge graph to the device knowledge graph 140. In an embodiment of the present disclosure, the user behavior pattern manager 1620 may store the pattern knowledge graph or the updated device knowledge graph in the memory 1350.

[0238] According to an embodiment of the present disclosure, the user behavior pattern manager 1620 may transmit information about all or some entities of the generated pattern knowledge graph to the recommendation function generator 1630. For example, the user behavior pattern manager 1620 may prioritize the patterns included in the pattern knowledge graph according to a preset criterion, and may transmit only the patterns with a priority higher than the preset criterion to the recommendation function generator 1630.

[0239] According to an embodiment of the present disclosure, the recommendation function generator 1630 may determine a function to be recommended to a user of the device 130 based on the received patterns. For example, the recommendation function generator 1630 may obtain context data indicating the current context of the device 130 and determine the recommended function based on the degree of association between the patterns included in the pattern knowledge graph and the context data. The recommendation function generator 1630 may transmit the determined function to the function recommender 1430. The function recommender 1430 may select a function to be provided to the device 130 from among the recommended functions based on the server knowledge graph 120 (e.g., the first recommended content) and the recommended functions received from the recommendation function generator 1630 (e.g., the second recommended content).

[0240] Figure 17 is a block diagram of the server 110 according to an embodiment of the present disclosure.

[0241] Reference Figure 17 , according to an embodiment of the present disclosure, the server 110 may include a communicator 1710, a storage device 1720, and a processor 1730. Additionally, the structure of the server 110 may generally be implemented by one or more processors using one or more memories. One or more memories may include instructions in the form of computer code to implement the functions of the structure (e.g., the communicator 1710 and the storage device 1720 and generally other structures of the server 110). However, Figure 17 not all of the elements shown are necessary for the server 110. The server 110 may include more elements than Figure 17 those shown, or may include fewer elements than Figure 17 those shown.

[0242] The communicator 1710 may include one or more components for communicating with the device 130. For example, the communicator 1710 may include a short-range wireless communicator, a mobile communicator, and a broadcast receiver. Additionally, the communicator 1710 may transmit the server knowledge graph 120 to the device 130.

[0243] The storage device 1720 may store programs for the processing and control of the processor 1730, and may store data input to the server 110 or data output from the server 110. Additionally, the storage device 1720 may store the server knowledge graph 120. For each category, the storage device 1720 may store the server knowledge graph 120 in a database. For example, the storage device 1720 may store the first server knowledge graph 122 and the second server knowledge graph 124 updated in the first server knowledge graph 122.

[0244] The processor 1730 may control the overall operation of the server 110. That is, the processor 1730 may control one or more other components of the server 110 by executing programs stored in the storage device 1720. For example, the processor 1730 may control the overall operations of the communicator 1710 and the storage device 1720 by executing programs stored in the storage device 1720. The processor 1730 may control the overall operation of the server 110 by controlling the communicator 1710 and the storage device 1720.

[0245] Functions related to artificial intelligence (AI) according to an embodiment of the present disclosure are operated by a processor and a memory. The processor may include one or more processors. The one or more processors may be a general-purpose processor (such as a CPU, an access point (AP), or a digital signal processor (DSP)), a dedicated graphics processor (such as a GPU or a vision processing unit (VPU)), or a dedicated AI processor (such as a neural processing unit (NPU)). The one or more processors perform control according to predefined operation rules or an AI model stored in the memory to process input data. Alternatively, when the one or more processors are dedicated AI processors, the dedicated AI processors may be designed with a hardware structure dedicated to processing a specific AI model.

[0246] Predefined operation rules or an AI model are constructed through learning. Constructing through learning may mean training a basic AI model using multiple pieces of training data via a learning algorithm to construct a predefined operation rule or an AI model configured to perform a desired feature (or purpose). Such learning may be performed by the device itself on which the AI according to the present disclosure is executed, or may be performed by a separate server and / or system. Examples of learning algorithms may include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but the present disclosure is not limited to the above examples.

[0247] A method described in a claim or specification of the present disclosure according to an embodiment of the present disclosure can be implemented as hardware, software, or a combination of hardware and software.

[0248] When implemented as software, a computer-readable storage medium or a computer program product for storing one or more programs (software modules) can be provided. The one or more programs stored in the computer-readable storage medium or the computer program product are configured to be executable by one or more processors in an electronic device. The one or more programs include instructions that cause the electronic device to execute the method according to an embodiment of the present disclosure described in the claims or specification of the present disclosure.

[0249] One or more programs (software modules, software, etc.) can be stored in a RAM, a non-volatile memory including a flash memory, a ROM, an EEPROM, a magnetic disk storage device, a CD-ROM (Compact Disc Read-Only Memory), a DVD (Digital Versatile Disc), other types of optical storage devices, or a cassette tape. Alternatively, one or more programs can be stored in a memory provided by a combination of all or a part of these devices. Additionally, each memory can include a plurality of configured memories.

[0250] Additionally, one or more programs can be stored in an attachable storage device that can be accessed through a communication network such as the Internet, an intranet, a local area network (LAN), a wide LAN (WLAN), or a storage area network (SAN), or a combination thereof. These storage devices can be connected to a device implementing an embodiment of the present disclosure through an external port. Additionally, a stand-alone storage device on the communication network can access a device implementing an embodiment of the present disclosure.

[0251] As used herein, the term "computer program product" or "computer-readable medium" is used to refer to such media as a memory, a hard disk installed in a hard disk drive, and signals, etc. These "computer program products" or "computer-readable media" are devices for providing the following method: generating a device knowledge graph 140 by using a server knowledge graph 120 and log history information, generating a pattern knowledge graph associated with the behavior pattern of a user of a device 130 based on entities and relationships between entities in the device knowledge graph 140, and updating the device knowledge graph 140 by adding the generated pattern knowledge graph to the device knowledge graph 140.

[0252] In specific embodiments of the present disclosure, the elements included in the present disclosure are expressed in singular or plural forms according to the specific embodiments proposed in the present disclosure. However, the singular or plural forms of expression are appropriately selected according to the proposed circumstances for ease of explanation, rather than intending to limit the present disclosure to singular or plural elements. Even when an element is expressed in the plural form, a single element may be provided, and even when an element is expressed in the singular form, multiple elements may be provided.

[0253] The embodiments of the present disclosure described in the specification and the drawings are merely presented as specific examples for the purpose of easily explaining the technical content of the present disclosure and facilitating understanding of the present disclosure, rather than intending to limit the scope of the present disclosure. Therefore, the scope of the present disclosure should not be limited to the above-described embodiments of the present disclosure, but should be determined by the appended claims and their equivalents.

Claims

1. A method for a device to provide content based on a knowledge graph, the method comprising: Obtaining log history information related to the operation of the device; Generating a device knowledge graph based on the log history information, the device knowledge graph being represented in the Resource Description Framework (RDF) and including a first entity; Generating a pattern knowledge graph associated with the behavior pattern of the user of the device based on the relationship between the first entities in the device knowledge graph, wherein the pattern knowledge graph is in the form of an RDF structure and the RDF structure is used to identify which behavior the user frequently performs under various conditions and accordingly pattern the user's behavior; Updating the device knowledge graph by adding the pattern knowledge graph to the device knowledge graph; Updating a server knowledge graph based on the pattern knowledge graph, wherein the server knowledge graph is generated by the server based on data collected from multiple devices; Obtaining context data indicating the current context of the device; and Determining recommended content for the user based on the obtained context data, the pattern knowledge graph in the updated device knowledge graph, and the server knowledge graph.

2. The method according to claim 1, wherein Generating the device knowledge graph includes: Generating knowledge data by inputting the log history information into a knowledge graph generation model; and Generating the device knowledge graph by connecting at least one of the second entities in the server knowledge graph to the knowledge data based on the relationship between the second entities in the server knowledge graph and the knowledge data.

3. The method according to claim 1, wherein Generating the pattern knowledge graph includes: Identifying entities associated with the user's behavior and entities associated with the user's context from the device knowledge graph; and Generating the pattern knowledge graph associated with the user's behavior pattern based on the entities associated with the behavior, the entities associated with the context, and the relationship between the entities associated with the behavior and the entities associated with the context.

4. The method according to claim 3, wherein Generating the pattern knowledge graph further includes: Determining the association relationship between the entities associated with the context and the entities associated with the behavior; and Generating the pattern knowledge graph based on the determined association relationship.

5. The method according to claim 1, wherein Updating the device knowledge graph includes: Selecting at least one entity in the first entities in the device knowledge graph to which the pattern knowledge graph is to be connected; and Connecting the pattern knowledge graph to the selected at least one entity.

6. The method according to claim 1, wherein Updating the server knowledge graph includes: Selecting at least one entity in the second entities in the server knowledge graph to which the pattern knowledge graph is to be connected; and Connecting the pattern knowledge graph to the selected at least one entity.

7. The method according to claim 1, further comprising: Obtaining context data indicating the current context of the device; Determining a function to be provided to the user by using the obtained context data and the pattern knowledge graph in the updated device knowledge graph; And Execute the determined function, where executing the determined function includes providing content recommendations to the user based on the updated device knowledge graph.

8. The method according to claim 7, wherein Determining the function includes: determining content recommendations to be provided to the user, and Determining the content recommendations includes: Determining a first recommended content based on the obtained context data indicating the current context of the user and the server knowledge graph; Confirming the existence or non - existence of the pattern knowledge graph; and Selecting recommended content based on the result of the confirmation.

9. A device for providing content based on a knowledge graph, the device includes: A communication interface; A memory storing one or more instructions; And A processor configured to execute the one or more instructions to: Obtain log history information related to the operation of the device, Generate a device knowledge graph based on the log history information, the device knowledge graph being represented in the Resource Description Framework (RDF) and including a first entity, Generate a pattern knowledge graph associated with the behavior pattern of the user of the device based on the relationship between the first entities in the device knowledge graph, wherein the pattern knowledge graph is in the form of an RDF structure and the RDF structure is used to identify which behavior the user frequently performs under various conditions and accordingly pattern the user's behavior, Update the device knowledge graph by adding the pattern knowledge graph to the device knowledge graph, Update the server knowledge graph based on the pattern knowledge graph, where the server knowledge graph is generated by the server based on data collected from multiple devices, Wherein, the processor is further configured to execute the one or more instructions to: Obtain context data indicating the current context of the device; and Determine recommended content for the user based on the obtained context data, the pattern knowledge graph in the updated device knowledge graph, and the server knowledge graph.

10. The device according to claim 9, wherein, The processor is further configured to execute the one or more instructions to: Generate knowledge data by inputting the log history information into a knowledge graph generation model, and Generate the device knowledge graph by connecting at least one of the second entities in the server knowledge graph to the knowledge data based on the relationship between the second entity in the server knowledge graph and the knowledge data.

11. The device according to claim 9, wherein, The processor is further configured to execute the one or more instructions to: Identify entities associated with the user's behavior and entities associated with the user's context from the device knowledge graph, and Generate the pattern knowledge graph associated with the user's behavior pattern based on the entities associated with the behavior, the entities associated with the context, and the relationship between the entities associated with the behavior and the entities associated with the context.

12. The device according to claim 11, wherein, The processor is further configured to execute the one or more instructions to: Determine the association relationship between the entity associated with the context and the entity associated with the behavior, and Generate the pattern knowledge graph based on the determined association relationship.

13. The device according to claim 9, wherein, The processor is further configured to execute the one or more instructions to: select at least one entity from among the first entities in the device knowledge graph to which the pattern knowledge graph is to be connected, and connect the pattern knowledge graph to the at least one selected entity.

14. The device according to claim 9, wherein The processor is further configured to execute the one or more instructions to: select at least one entity from among the second entities in the server knowledge graph to which the pattern knowledge graph is to be connected, and connect the pattern knowledge graph to the at least one selected entity.

15. The device according to claim 9, wherein, The processor is further configured to execute the one or more instructions to: obtain context data indicating the current context of the device, determine a function to be provided to the user by using the obtained context data and the pattern knowledge graph in the updated device knowledge graph, and execute the determined function, wherein the processor is further configured to execute the determined function by providing content recommendations to the user based on the updated device knowledge graph.

Images