Method for managing activity information by user device, and user device using same

Abstract

Disclosed is a method for managing activity information by a user device, the method comprising: generating action information corresponding to a user's action, on the basis of data acquired by the user device; generating candidate activity information corresponding to a series of consecutive pieces of action information, on the basis of the acquired data or the action information; changing the candidate activity information in response to updates to the acquired data or the action information; determining activity information from the changed candidate activity information, on the basis of a condition for confirming the activity information; and storing and managing the activity information to construct a personalized knowledge graph.

Description

Method for managing activity information on user devices and user devices utilizing the same

[0001] It relates to a method for managing activity information on a user device and a user device that utilizes the same.

[0002] AI-based technologies are being utilized in diverse fields across industries. A variety of AI models are being developed and utilized across various sectors, and the application of AI-based solutions is rapidly increasing not only in manufacturing but also in robotics, transportation / logistics, healthcare, education, pharmaceuticals / biotechnology, and other industries. The adoption of such AI-based technologies is leading to enhanced competitiveness for both companies and countries.

[0003] To enhance the performance of AI models, knowledge bases can be used for AI model training and inference. One example of a knowledge base is a knowledge graph, which is a graph-based data structure. Interest in building and utilizing knowledge graphs is growing.

[0004] The present invention provides a method for managing user activity information on a user device based on data obtained from the user device and a user device utilizing the same.

[0005] According to one embodiment of the present disclosure, a method for managing activity information in a user device is provided. The method for managing activity information in a user device includes a step of generating action information corresponding to a user's action based on data acquired from the user device. Furthermore, the method for managing activity information in a user device includes a step of generating activity information candidates corresponding to a series of continuous action information based on the acquired data or the action information. Furthermore, the method for managing activity information in a user device includes a step of modifying the activity information candidates according to an update of the acquired data or the action information. Furthermore, the method for managing activity information in a user device includes a step of determining the activity information from the modified activity information candidates based on a condition for determining the activity information; and a step of storing and managing the activity information to build a personalized knowledge graph.

[0006] According to one embodiment of the present disclosure, a computer-readable recording medium having recorded thereon a program for executing the above-described method is provided.

[0007] According to one embodiment of the present disclosure, a user device for managing activity information is provided. The user device includes at least one processor and a memory storing at least one instruction, wherein when the at least one instruction is executed by the at least one processor, the at least one processor generates action information corresponding to a user's action based on data acquired from the user device. Furthermore, the at least one processor executes the at least one instruction to generate activity information candidates corresponding to a series of continuous action information based on the acquired data or the action information. Furthermore, the at least one processor executes the at least one instruction to modify the activity information candidates according to updates to the acquired data or the action information. Furthermore, the at least one processor executes the at least one instruction to determine the activity information from the modified activity information candidates based on a condition for determining the activity information, and stores and manages the activity information to build a personalized knowledge graph in the user device.

[0008] Figure 1 is an explanation of the multi-store model theory regarding the human memory system.

[0009] FIG. 2 is a diagram for explaining a multi-memory-based operation of a user device according to one embodiment of the present disclosure.

[0010] FIG. 3 is a flowchart illustrating a method for managing activity information in a user device according to one embodiment of the present disclosure.

[0011] FIG. 4a is a diagram for explaining action information in the form of a knowledge graph according to one embodiment of the present disclosure.

[0012] FIG. 4b is a diagram for explaining action information in the form of a knowledge graph according to one embodiment of the present disclosure.

[0013] FIG. 4c is a diagram for explaining action information in the form of a knowledge graph according to one embodiment of the present disclosure.

[0014] FIG. 5A is a diagram illustrating activity information and an activity information database according to one embodiment of the present disclosure.

[0015] FIG. 5b is a diagram illustrating activity information and an activity information database according to one embodiment of the present disclosure.

[0016] FIG. 6A is a diagram illustrating an example of activity information corresponding to an entire sequence according to one embodiment of the present disclosure.

[0017] FIG. 6b is a diagram illustrating a method for inferring activity information from activity information candidates corresponding to some sequences according to one embodiment of the present disclosure.

[0018] FIG. 7 is a flowchart illustrating a method for constructing a personalized database based on activity information in a user device (100) according to one embodiment of the present disclosure.

[0019] FIG. 8A is a diagram illustrating a plurality of activity information stored in a story memory according to one embodiment of the present disclosure.

[0020] FIG. 8b is a diagram illustrating anecdotal information stored in a personal semantic memory according to one embodiment of the present disclosure.

[0021] FIG. 9 is a diagram illustrating an example of generating routine information from anecdotal information stored in a personal semantic memory according to one embodiment of the present disclosure.

[0022] FIG. 10 is a diagram illustrating an example of expressing activity information stored in an episodic memory in the form of a knowledge graph by referencing a personalized knowledge graph stored in a personal semantic memory according to one embodiment of the present disclosure.

[0023] FIG. 11 is a diagram illustrating an example of managing similar anecdotal information with different patterns in a personal semantic memory according to one embodiment of the present disclosure.

[0024] FIG. 12 is a diagram illustrating an example of managing similar anecdotal information having multiple patterns of different probability distributions in a personal semantic memory according to one embodiment of the present disclosure.

[0025] FIG. 13 is a diagram for explaining a management method for parameter values ​​representing properties for activity information according to one embodiment of the present disclosure.

[0026] FIG. 14 is a diagram illustrating an AI platform based on a personalized knowledge graph according to one embodiment of the present disclosure.

[0027] FIG. 15 is a block diagram illustrating a user device according to one embodiment of the present disclosure.

[0028] FIG. 16 is a block diagram illustrating the configuration and operation of a user device according to one embodiment of the present disclosure.

[0029] FIG. 17 is a diagram illustrating a process for managing action information and activity information in a user device according to one embodiment of the present disclosure.

[0030] FIG. 18 is a diagram illustrating a process of building a personalized database based on activity information in a user device according to one embodiment of the present disclosure.

[0031] Hereinafter, terms used in this specification will be briefly described, and the present disclosure will be described in detail. In this disclosure, the expression "at least one of a, b, or c" may refer to "a," "b," "c," "a and b," "a and c," "b and c," "all of a, b, and c," or variations thereof.

[0032] The terms used in this disclosure are selected from widely used, common terms, taking into account the functions of the disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, in which case their meanings will be described in detail in the relevant description. Therefore, the terms used in this disclosure should not be defined simply as names, but rather based on the meanings of the terms and the overall content of the disclosure.

[0033] Singular expressions may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art described herein. Furthermore, terms containing ordinal numbers, such as "first" or "second," used herein may be used to describe various components, but such components should not be limited by such terms. Such terms are used solely to distinguish one component from another.

[0034] When a part of the specification is said to "include" a component, unless otherwise specifically stated, this does not exclude other components but rather implies the inclusion of other components. Furthermore, terms such as "part" and "module" used in the specification refer to a unit that processes at least one function or operation, which may be implemented in hardware, software, or a combination of hardware and software.

[0035] The artificial intelligence-related functions according to the present disclosure are operated via a processor and memory. The processor may be comprised of one or more processors. In this case, one or more processors may be a general-purpose processor such as a CPU, an AP, a Digital Signal Processor (DSP), a graphics-only processor such as a GPU or a Vision Processing Unit (VPU), or an artificial intelligence-only processor such as an NPU. One or more processors control the processing of input data according to predefined operating rules or artificial intelligence models stored in memory. Alternatively, if one or more processors are artificial intelligence-only processors, the artificial intelligence-only processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0036] The predefined operation rules or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that the basic artificial intelligence model is trained using a learning algorithm using a plurality of learning data, thereby creating a predefined operation rules or artificial intelligence model set to perform a desired characteristic (or purpose). This learning may be performed on the device itself on which the artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and / or system. Examples of the learning algorithm include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0037] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values, and performs neural network operations through operations between the operation results of the previous layer and the multiple weights. The multiple weights of the multiple neural network layers may be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated so that the loss value or cost value obtained from the artificial intelligence model is reduced or minimized during the learning process. The artificial neural network may include a deep neural network (DNN), and examples thereof include, but are not limited to, a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or deep Q-networks.

[0038] In this disclosure, the term "knowledge graph" refers to a knowledge base in graph form, which is a method for managing and exploring knowledge information, and which is based on a knowledge base that stores knowledge information and a graph that expresses it so that it can be analyzed in a network structure. A knowledge graph is a graph model that implements the knowledge accumulated in a knowledge base as a relationship between nodes and edges. A knowledge graph can be used to integrate data using a graph data model, topology, etc. To enable knowledge to be interconnected and integrated using a knowledge graph, a schema is implemented through an ontology, and a structure and dictionary (terminology) that can be shared with each other are used.

[0039] Semantic information, including common sense and factual knowledge, is organized as connections between nodes and edges, and by referencing this, various types of data can be converted into the form of a knowledge graph. Methods for representing knowledge graphs include, but are not limited to, Labeled Property Graphs (LPGs), in which nodes and edges can each have properties, and Resource Description Framework (RDF), which expresses relationships in a triple structure of subject-predicate-object. Knowledge graphs can be created by recognizing entities from various data, linking them to appropriate entities in an existing knowledge base (entity linking), and extracting relationships between entities.

[0040] Knowledge graphs can be leveraged to enhance the performance of artificial intelligence. They can be used in models such as graph neural networks (GNNs) and graph convolution neural networks (GCNs), and can be used to provide explanations for results in explainable artificial intelligence (XAI).

[0041] Below, with reference to the attached drawings, embodiments of the present disclosure are described in detail so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein.

[0042] The present disclosure will be described in detail with reference to the attached drawings below.

[0043] Figure 1 is an explanation of the multi-store model theory regarding the human memory system.

[0044] The multi-store model theory explains that the memory system is composed of multiple memory stores. According to the multi-store model theory, memory is composed of three structures: sensory memory, short-term memory, and long-term memory. Information is temporarily retained in sensory memory through perception and then disappears. Information in sensory memory that a person focuses on can be stored in short-term memory. Information in short-term memory that is repeated can be stored in long-term memory. Alan Baddeley proposed working memory as the counterpart to short-term memory, specifying it as a visuospatial notepad, a phonological circuit, and a central executive.

[0045] Referring to Figure 1, sensory memory stores information based on sensory input for only a very short period of time. The retention time of information varies depending on a person's sensory domain. For example, information input through sight can be retained for about one second, while information input through hearing can be retained for about two seconds. Short-term memory processes and stores information that has been focused on among the information stored in sensory memory. Sensory memory has limited capacity and retention period. If information in short-term memory is repeatedly learned, it can be transferred to long-term memory through encoding and stored there. Long-term memory has no limitations in capacity or retention period. Information stored in long-term memory can be retrieved and used.

[0046] FIG. 2 is a diagram for explaining a multi-memory-based operation of a user device (100) according to one embodiment of the present disclosure.

[0047] Following the multi-storage model of the human memory system described in FIG. 1, a user device (100) according to an embodiment of the present disclosure may include multiple memories for storing information. The user device (100) may be an electronic device capable of processing data. For example, the user device may be an electronic device such as a smartphone, smart glasses, a wearable device, a digital camera, a laptop, an AR (Augmented Reality) device, or a VR (Virtual Reality) device. The user device (100) may be equipped with various types of neural network models. For example, the user device may be equipped with at least one model such as a CNN (Convolution Neural Network), a GCN (Graph Convolution Neural Network), a GNN (Graph Neural Network), a DNN (Deep Neural Network), an RNN (Recurrent Neural Network), or a BRDNN (Bidirectional Recurrent Deep Neural Network), and may also use these models in combination.

[0048] Referring to FIG. 2, the user device (100) may be equipped with a first memory and a second memory to acquire and manage various types of data. The first memory corresponds to short-term memory and may include working memory. The first memory may store and manage information at a level corresponding to a person's short-term memory. The second memory corresponds to long-term memory and may include persistent memory. The second memory may store and manage information at a level corresponding to long-term memory. For example, the second memory may include a semantic memory that stores and manages semantic information corresponding to common sense or factual knowledge. The second memory may include a personal semantic memory that stores and manages a personalized knowledge base. The second memory may include an episodic memory that accumulates multiple activity information and manages multiple related activity information. Each of the first memory and the second memory may include multiple memories for different purposes.

[0049] Certain information can be derived from data acquired from the user device (100). The data acquired from the user device (100) may be stored in the user device (100) in the form of metadata for content such as text, photos, videos, music, etc., or data corresponding to time or place related to the content. The data acquired from the user device (100) may be stored in the user device (100) in the form of sensing data detected from at least one sensor. The data acquired from the user device (100) may be stored in the user device in the form of metadata for the application used, usage log data, log data such as the time or place of an event occurrence, etc.

[0050] The user device (100) can process unstructured data acquired from the user device (100) into structured information and store it. Referring to FIG. 2, the user device (100) can prepare semantic information in a structured form in advance in the second memory. Just as a person perceives data input through the sensory domain based on existing common sense or empirical facts, the user device (100) can process unstructured data acquired from the user device (100) into structured information and store it in the first memory by referring to the structured semantic information stored in the second memory. For example, the user device (100) can convert various types of unstructured data into the form of a knowledge graph and store it in the first memory.

[0051] For example, the user device (100) may collect information about the user's content consumption history, behavior, etc. from data about content played or applications used on the user device (100) and store the information in the first memory in the form of a knowledge graph. The user device (100) may infer environmental information or situational information from sensing data detected on the user device (100) and store the information in the first memory in the form of a knowledge graph. The user device (100) may infer information about the user's experience, patterns, habits, etc. from log data stored on the user device (100) and store the information in the first memory in the form of a knowledge graph.

[0052] Below, a technology for managing a user's personal information and building a personalized knowledge base based on data acquired from a user device (100) and a technology for storing and extracting data acquired from a user device (100) in a user-friendly manner by referring to the personalized knowledge base are described.

[0053] FIG. 3 is a flowchart illustrating a method for managing activity information in a user device (100) according to one embodiment of the present disclosure.

[0054] The user device (100) can manage the user's personal information and build a personalized knowledge base based on data acquired from the user device (100). For example, the user device (100) can manage the user's activity information based on data acquired from the user device (100).

[0055] Referring to FIG. 3, in step S310, the user device (100) may generate action information corresponding to a unit action of the user based on data acquired from the user device (100). The data acquired from the user device (100) may be data input from the user in the user device (100), data sensed by the user device (100), data received from the outside by the user device (100), data processed in the user device (100), etc. For example, the data input from the user in the user device (100) may be data input for control, selection, manipulation, output, etc. in the user device (100). The data detected by the user device (100) is sensing data detected by at least one sensor equipped in the user device (100), and may be sensing data detected by at least one of a camera, a LiDAR (Light Detection And Ranging) sensor, an infrared sensor, an ultrasonic sensor, a ToF (Time of Flight) sensor, a gyro sensor, a brightness sensor, a hall sensor, a motion sensor, a temperature sensor, a humidity sensor, a proximity sensor, a GPS sensor, etc. equipped in the user device (100). The data received by the user device (100) from the outside may be data received from an external device through a communication interface. The data processed in the user device (100) may be data related to an application or program executed in the user device (100) or data such as content processed by the execution of these.

[0056] The user device (100) can generate action information corresponding to a unit motion of the user from data acquired from the user device (100). The data acquired from the user device (100) may be related to an action actually performed by the user. The action actually performed by the user may be classified and processed at a level corresponding to a predetermined unit motion. The user device (100) may extract action information matching the unit motion of the user analyzed based on the data acquired from the user device (100) from the action information database of the knowledge graph. The action information corresponds to a value stored in the action information database of the knowledge graph to be matched to the unit motion of the user analyzed based on the data acquired from the user device (100). The action information may be information representing the user's movement that occurs over a relatively short period of time, such as several seconds to several minutes. The action information is a basic unit of a user movement event and may be mapped with context information such as the time of occurrence, location, weather, etc. The user device (100) can generate action information more related to the action actually performed by the user by considering the situation information along with the user's unit action. For example, the user device (100) can analyze the user's unit action called 'executing an e-book application' by obtaining data on the user's input event on the touch screen, the coordinates of the touch location on the touch screen, and whether the e-book application is executed. The user device (100) can extract the action information called 'start reading a book' that matches the analyzed user's unit action, 'executing an e-book application', by using the action information database of the knowledge graph.

[0057] For another example, the user device (100) can analyze the user's unit action, "the user moves," by obtaining data on whether Wi-Fi is connected / disconnected and changes in GPS signals. The user device (100) can then use the action information database of the knowledge graph to extract the action information, "leaving home," which matches the analyzed user's unit action, "the user moves."

[0058] A user's unit actions and action information can be mapped to many-to-many mappings. For example, the unit action "launching an e-book application" can be mapped to multiple action information items, such as "start reading a book," "start studying," and "take a break." Meanwhile, the action information "take a break" can be mapped to multiple unit actions, such as "launching an e-book application," "playing a video," and "playing a game."

[0059] The user device (100) can extract action information from the action information database of the knowledge graph for each user's unit action analyzed based on data acquired from the user device (100). For example, in the process of User A leaving home, getting into a car, and driving to work, the user device (100) can acquire various types of data and generate action information that can be matched to the user's unit action. Looking at it in chronological order, the user device (100) can analyze the user's movement (unit action) from the disconnection from the Wi-Fi router installed in the home and determine that the user has left the house (action). Accordingly, the user device (100) can generate first action information called 'leaving home.' The user device (100) can determine that the user has entered the car through a communication connection with the car. Accordingly, the user device (100) can generate second action information called 'getting in car.' The user device (100) can detect a change in the user's location information via GPS, thereby determining that the user is driving. Accordingly, the user device (100) can generate third action information, "driving (controlling car)." The user device (100) can detect that the user has arrived at the company by detecting, via GPS, that the user has entered a predetermined range from the company's previously registered location information. Accordingly, the user device (100) can generate fourth action information, "arriving at the company (arrving office)."

[0060] According to one embodiment of the present disclosure, the user device (100) can extract action information matching a user's unit action based on data acquired from the user device (100) from an action information database of a built-in knowledge graph. The user device (100) can determine the extracted action information as action information corresponding to the user's unit action. The action information database of the built-in knowledge graph is stored in a second memory (e.g., a semantic memory). The user device (100) can manage the generated action information in a first memory (e.g., a working memory).

[0061] For example, the user device (100) may compare an embedding vector of a user's unit action based on data acquired from the user device (100) with an embedding vector of at least one action information candidate in an action information database of a constructed knowledge graph. The user device (100) may classify action information for the user's unit action based on a comparison result between the embedding vectors. By classifying the action information for the user's unit action, action information matching the user's unit action may be extracted from the action information database of the constructed knowledge graph.

[0062] According to one embodiment of the present disclosure, the user device (100) can generate action information by inferring second action information based on time information or location information obtained from acquired data, while first action information has been generated. The user device (100) can manage the generated action information in a first memory (working memory).

[0063] For example, if the time the user arrives at work is during commuting hours, the user device (100) can infer that the user starts work at the company and generate action information titled "start work." In the action information database of the built-in knowledge graph, a relationship between the first action information titled "arrive at work," the time information titled "commute hours," and the second action information titled "start work" is predefined. At this time, in the process of generating action information, the action information can be generated by setting a confidence interval based on the user's history. If the user always performs action "X" at a specific time, the confidence interval for action "X" at that specific time can be set to a high value. On the other hand, if the user always arrives at work around 7:00 AM, but arrives at work at 10:00 AM, the confidence interval for "commute hours" can be set to a low value in order to generate action information titled "start work." For another example, if the user eats at a restaurant, the user device (100) can infer that the user eats out and generate action information titled "eat out." In the action information database of the constructed knowledge graph, the relationship between the first action information, 'eat a meal', the location information, 'restaurant', and the second action information, 'eat out', is predefined.

[0064] FIG. 4a, FIG. 4b, and FIG. 4c are diagrams for explaining action information in the form of a knowledge graph according to one embodiment of the present disclosure.

[0065] Data acquired from a user device (100) can be converted into action information based on semantic information in the form of a knowledge graph and stored in the working memory in the form of a knowledge graph. The semantic information is stored in the semantic memory in the form of a knowledge graph composed of connections between nodes and edges. The action information can be generated in the form of a knowledge graph composed of nodes and edges, and each node and edge can have attributes.

[0066] Referring to Figure 4a, the action information "leaving home" is represented in the form of a knowledge graph. The action information can be stored in the working memory in the form of "user did action." The action information "leaving home" can be composed of a node related to "user," a node related to "action," and a node related to "place," and each node can have a value for an attribute. Referring to Figure 4b, the action information "arriving at work" is represented in the form of a knowledge graph. The initially generated action information can be called a primitive action or a top action.

[0067] Referring to Figure 4c, in the action information database of the built-up knowledge graph, a relationship between the first action information "arrive at work," the time information "commute time," and the second action information "start work" can be predefined. Accordingly, if the user arrives at work during the commuting time, the action information "start work" can be dynamically generated and stored in the working memory by inferring that the user has started work at the company.

[0068] Referring back to FIG. 3, in step S320, the user device (100) may generate an activity information candidate corresponding to a sequence including a series of continuous action information based on the acquired data or generated action information. The activity information corresponds to an action corresponding to a sequence formed by chaining a series of action information. The activity information may refer to an action inferred by a set of action information. A single activity information may be generated based on a sequence of continuous action information. The activity information may have a start time and an end time, and may have start action information and end action information. The activity information candidate corresponds to incomplete activity information and corresponds to an action corresponding to a sequence reflecting the latest action information. The activity information has a start time and an end time, so that it can be distinguished whether it is currently in progress or a completed past event. An ongoing activity may be expressed in the form of 'user do activity', and a completed activity may be expressed in the form of 'user did activity'.

[0069] FIG. 5A and FIG. 5B are diagrams for explaining activity information and an activity information database according to one embodiment of the present disclosure.

[0070] Referring to Fig. 5a, the hierarchical structure of the activity information database is illustrated. The activity information database can determine the hierarchy in which activity information is arranged based on the hierarchical concepts between activity information. As illustrated in Fig. 5a, the first activity information corresponds to the highest concept and is primitive activity information. The second, third, and fourth activity information correspond to activity information specificized from the first activity information and correspond to lower concepts than the first activity information, and thus can be arranged in a lower hierarchy than the first activity information. The fifth, sixth, and seventh activity information correspond to activity information further specificized from the second activity information and correspond to lower concepts than the second activity information, and thus can be arranged in a lower hierarchy than the second activity information. Activity information including a sequence of action information can be determined from the activity information database based on data acquired from or action information generated by the user device (100).

[0071] The activity information of a higher-level concept and the activity information of a lower-level concept may have an inheritance relationship. The activity information of a higher-level concept may inherit the attributes and methods of the activity information of the higher-level concept to the activity information of the lower-level concept. The attributes or methods of the activity information of the higher-level concept may be applied identically to the attributes or methods of the activity information of the lower-level concept. In Fig. 5a, the attributes of the second activity information are applied identically to the attributes of the fifth, sixth, and seventh activity information, respectively, and when the attributes of the second activity information are updated, the attributes of the fifth, sixth, and seventh activity information, respectively, may also be updated.

[0072] Referring to FIG. 5b, in the case of the primitive activity information 'moving', it is defined in the semantic memory that it is activity information corresponding to a sequence that includes the start action information 'leaving a place' and the end action information 'arriving at a place'. Therefore, if the start action information 'leaving a place' occurs and the end action information 'arriving at a place' occurs, the activity information 'moving' can be determined. If the start action information 'leaving home' occurs and the end action information 'arriving at work' occurs, the activity information 'commuting to work' can be determined, and if the end action information 'arriving at school' occurs, the activity information 'commuting to study' can be determined.

[0073] If the start time attribute of the activity information called 'moving' is '2 PM', the attribute of the activity information called 'commuting to work', which is a sub-concept of the activity information called 'moving', can also be '2 PM'. If the activity information called 'moving' has an 'at place' relationship with the location information, the activity information called 'commuting to work', which is a sub-concept of the activity information called 'moving', can also have a 'from' relationship with the location of the location information and the attribute in its sub-relationship.

[0074] Referring back to FIG. 3, according to one embodiment of the present disclosure, the user device (100) may generate activity information candidates using the activity information database of the constructed knowledge graph based on acquired data or generated action information. For example, the activity information candidate generation model may generate activity information candidates based on the activity information database of the constructed knowledge graph using acquired data or generated action information as input. The activity information database of the constructed knowledge graph is stored in a second memory (personal semantic memory or semantic memory). The semantic memory may store an activity information database of the knowledge graph that is predefined based on semantic information (common sense or factual knowledge). The personal semantic memory may store an activity information database of a personalized knowledge graph that is inductively defined based on information related to an individual user's event. The inductive activity information is custom activity information that is inductively defined based on a sequence of actions that the user frequently performs. The user device (100) may manage the generated activity information candidates in a first memory (working memory).

[0075] According to one embodiment of the present disclosure, when one or more pieces of action information are generated based on the acquired data or the generated action information, the user device (100) may extract activity information including a sequence of one or more pieces of action information from an activity information database of a built-in knowledge graph. For example, when first action information is generated, the user device (100) may extract activity information including a sequence of the first action information from the activity information database of the built-in knowledge graph. The user device (100) may determine the extracted activity information as a candidate for activity information corresponding to the sequence of one or more pieces of action information.

[0076] In the example of action information described in S310 above, starting with the first action information, "leaving home," the user device (100) can extract activity information including a sequence of the first action information. Accordingly, the user device (100) can generate activity information, "moving," as an activity information candidate.

[0077] At step S330, the user device (100) may change the generated activity information candidate based on updates to acquired data or generated action information. The activity information candidate may be dynamically changed based on the updated sequence.

[0078] According to one embodiment of the present disclosure, the user device (100) can change activity information candidates using the activity information database of the built-in knowledge graph based on updates to acquired data or generated action information. For example, the activity information candidate generation model can update activity information candidates based on the activity information database using updated data or action information as input. The activity information database of the built-in knowledge graph is stored in a second memory (personal semantic memory or semantic memory). The user device (100) can manage changed activity information candidates in a first memory (working memory).

[0079] According to one embodiment of the present disclosure, the user device (100) can extract activity information including a sequence of action information to which updates are reflected, based on updates to data acquired or generated action information from the activity information database of the constructed knowledge graph. The user device (100) can determine the extracted activity information as an activity information candidate corresponding to the sequence of action information to which updates are reflected. The user device (100) can update the activity information candidate prior to the update of the data acquired or generated action information from the user device (100) with the extracted activity information.

[0080] For example, the user device (100) can extract activity information including a sequence of action information in which the update is reflected, by having the activity information of a higher concept be concretized into the activity information of a lower concept based on an activity information database of a hierarchical structure in which the hierarchy is determined according to the concept of upper and lower levels among the activity information, according to an update of data acquired or action information generated by the user device (100).

[0081] In a hierarchical activity information database, activity information of a higher concept may be placed in a higher layer, and activity information of a lower concept may be placed in a lower layer. A pointer (cursor) may be positioned for each of at least one activity information including a sequence of the latest action information. The activity information at which the pointer (cursor) is positioned becomes a candidate for activity information. Depending on the update of data acquired from the user device (100) or generated action information, the user device (100) may determine whether it is possible to move each of at least one pointer (cursor) pointing to the activity information to activity information of a lower concept in a lower layer. If the activity information at which the pointer (cursor) is positioned changes as the pointer (cursor) moves, the candidate activity information may be changed.

[0082] In the example of action information described in S310 above, as the first to third action information, i.e., 'leaving home', 'getting in car', and 'driving car', are updated, the user device (100) can change the activity information including the sequence of action information reflected up to the update of the third action information into the activity information candidate. Accordingly, the user device (100) can determine activity information such as 'commuting to school' and 'commuting to work' as the activity information candidate.

[0083] At step S340, the user device (100) can determine activity information from the changed activity information candidates based on the conditions for determining the activity information. Once the activity information is determined, the user device (100) can store and manage the determined activity information in a second memory (episodic memory). The user device (100) can store and manage the activity information to build a personalized knowledge graph. Once the activity information is determined, the user device (100) can delete the activity information candidates managed in the first memory (working memory) related to the determined activity information.

[0084] According to one embodiment of the present disclosure, the user device (100) can determine activity information from activity information candidates for which end action information is generated. In the example of the action information described in S310 above, as the first to fourth action information, i.e., 'leaving home', 'getting in car', 'controlling car', and 'arriving office', are updated, the user device (100) can extract activity information including a sequence of action information reflected up to the update of the fourth action information from the built-up activity information database. At this time, since the fourth action information is end action information of the activity information 'commuting to work', the user device (100) can determine the activity information as 'commuting to work'.

[0085] According to one embodiment of the present disclosure, the user device (100) can infer activity information from an activity information candidate whose probability corresponding to the similarity with the activity information based on the progress of the sequence among the changed activity information candidates satisfies a predetermined condition. The user device (100) can obtain the probability corresponding to the similarity with the activity information based on the progress of the sequence of action information in which an update to the entire sequence of activity information is reflected according to an update of data acquired or generated by the user device (100). If the probability corresponding to the similarity with the activity information exceeds a predetermined threshold, the user device (100) can infer the activity information from the activity information candidate.

[0086] FIG. 6A is a diagram illustrating an example of activity information corresponding to an entire sequence according to one embodiment of the present disclosure.

[0087] Referring to Fig. 6a, the activity information 'commuting to work' corresponds to the entire sequence including the action information of 'leaving home', 'getting in car', 'listening music', 'controlling car', and 'arriving office' (indicated by the solid box in Fig. 6b) that has the starting time attribute of '7:00 a.m. on Monday' (indicated by the dotted box in Fig. 6a). Once the ending action information of 'arriving at office' is generated, the activity information corresponding to the entire sequence, 'commuting to work', can be determined.

[0088] FIG. 6b is a diagram illustrating a method for inferring activity information from activity information candidates corresponding to some sequences according to one embodiment of the present disclosure.

[0089] Referring to FIG. 6B, according to the update of the action information called 'listening music', there are activity information candidates corresponding to some sequences including the time information of '7 AM on Monday', as well as the action information of 'leaving home', 'getting in car', and 'listening music'. The user device (100) can confirm that the probability corresponding to the similarity with the activity information corresponding to the entire sequence is 0.7 based on the progress of some sequences of the activity information candidates. If the probability corresponding to the similarity with the activity information exceeds a predetermined threshold, the user device (100) can infer the activity information of 'commuting to work'.

[0090] FIG. 7 is a flowchart illustrating a method for constructing a personalized database based on activity information in a user device (100) according to one embodiment of the present disclosure.

[0091] The user device (100) can manage the user's personal information based on data acquired from the user device (100) and build a personalized knowledge base based on the managed personal information. For example, the user device (100) can build a personalized knowledge base based on the user's activity information managed by the user device (100). The user device (100) references the personalized knowledge graph to store information based on data acquired from the user device (100) in a user-friendly manner, and the user can then easily extract the stored information.

[0092] Referring to FIG. 7, at step S710, the user device (100) can generate action information corresponding to the user's unit motion based on data acquired from the user device. Since step S710 of FIG. 7 can be applied as is to the description related to step S310 of FIG. 3 described above, a detailed description thereof will be omitted below.

[0093] In step S720, the user device (100) can determine activity information corresponding to a sequence including a series of continuous action information. Since step S720 of FIG. 7 can be directly applied to the descriptions related to steps S320 to S340 of FIG. 3 described above, a detailed description thereof will be omitted below.

[0094] The user device (100) can determine activity information corresponding to a sequence by using an activity information database inductively defined based on information related to the user's individual events stored in the personal semantic memory, or an activity information database predefined based on semantic information (common sense or factual knowledge) stored in the semantic memory. The activity information database of the personalized knowledge graph can be constructed in the personal semantic memory.

[0095] The user device (100) can use working memory to store and manage information to determine activity information. The working memory can be used to track activity information currently in progress based on a sequence of linked action information. When a start action occurs, the user device (100) loads and tracks activity information currently in progress into the working memory, and when an end action occurs, deletes completed activity information from the working memory.

[0096] The user device (100) can delete completed activity information from the working memory and store and manage completed activity information using episodic memory and personal semantic memory as in steps S730 and S740.

[0097] At step S730, the user device (100) may generate episode information based on multiple pieces of related activity information. The user device (100) may store completed activity information deleted from the working memory in the episode memory. The episode memory may store multiple pieces of activity information.

[0098] The user device (100) can generate anecdotal information based on a plurality of related activity information items among a plurality of activity information items stored in an anecdotal memory. The relationship between the activity information items can be determined based on the type of the activity information items or the sequence corresponding to the activity information items. For example, activity information items of the same type can be said to be related to each other. For another example, if a first sequence corresponding to first activity information and a second sequence corresponding to second activity information match above a predetermined standard, they can be said to be related to each other. If the start action information and the end action information of the first activity information are the same as the start action information and the end action information of the second activity information, and some of the action information constituting the sequence overlaps with each other, they can be said to be related to each other. In addition, if the attributes of information related to the activity information or the action information included in the activity information are similar to each other, the two pieces of activity information can be said to be related to each other. For example, in the case of commuting to work, since there is a set time to arrive at work, the time of departure from home is early in the morning, and the typical commuting time is consistent, the activity information called 'commuting to work' can be used as a criterion for judging the similarity of the time attribute between activity information.

[0099] Anecdotal information is information representing multiple related activity information. Anecdotal information can be generated by analyzing the rules, characteristics, and values ​​corresponding to the attributes of multiple related activity information, using activity information accumulated over a predetermined period as a population, deriving common attributes of multiple related activity information as a pattern, and determining a confidence value for the pattern. The pattern can be derived based on the sequence of multiple action information and the attributes of the action information, and can be expressed as a predetermined parameter value. Anecdotal information based on the parameter value corresponding to the pattern can be generated from multiple related activity information accumulated in the anecdotal memory, based on the common action information and the attributes of each action information (e.g., data regarding time, location, and content). The user device (100) can generate anecdotal information by deriving common attributes of multiple related activity information as a pattern and determining a confidence value for the pattern.

[0100] For example, if the first activity information and the second activity information have sequences that include start action information and end action information that match, and there is similarity between the time data and location data of the start action information and the time data and location data of the end action information, anecdotal information about the first activity information and the second activity information can be generated. In this case, the anecdotal information can define statistical values ​​representing the properties of the first activity information and the second activity information as patterns, and can have a reliability value for the patterns. The reliability value for the patterns can have a value between 0 and 1.

[0101] Meanwhile, anecdotal information with different patterns can be distinguished from one another. If there are first to ten activity pieces of the same type, and the first anecdotal information generated from the first to fifth activity pieces and the second anecdotal information generated from the sixth to tenth activity pieces have different patterns, the first and second anecdotal information of the same type can be distinguished from one another.

[0102] According to one embodiment of the present disclosure, when a predetermined period or a certain amount of activity information is stored in the anecdotal memory, the user device (100) can analyze a plurality of pieces of activity information to derive a pattern representing the attributes of the plurality of pieces of activity information, thereby generating anecdotal information. The user device (100) can train a predetermined analysis model with the plurality of pieces of activity information stored in the anecdotal memory, thereby generating anecdotal information having a pattern.

[0103] FIG. 8A is a diagram illustrating multiple activity information items stored in an anecdotal memory according to one embodiment of the present disclosure. FIG. 8B is a diagram illustrating anecdotal information stored in a personal semantic memory according to one embodiment of the present disclosure.

[0104] Referring to FIGS. 8A and 8B , the user device (100) can generate anecdotal information having a pattern by deriving a pattern and a reliability value representing the properties of a plurality of related activity information accumulated in the anecdotal memory for a predetermined period of time. The user device (100) can perform a statistical analysis on the activity information corresponding to the user's commuting to work for a predetermined period of time as a population to determine whether there are recurring rules or commonalities in the time or day of commuting, or what the average value is. The user device (100) can generate anecdotal information by deriving a pattern and a reliability value for the day and time of commuting from a plurality of activity information corresponding to commuting.

[0105] Referring to Fig. 8a, it can be seen that multiple activity information corresponding to commuting is stored in the episodic memory. The episodic memory can store the start and end times of each activity information. For example, referring to Fig. 8a, it can be seen that the user started commuting at 8:11:37 AM on November 12, 2022, and completed commuting at 9:02:42 AM. On December 22, 2022, the user started commuting at 8:07:18 AM and completed commuting at 8:59:32 AM. On December 23, 2022, the user started commuting at 8:12:45 AM and completed commuting at 9:05:12 AM.

[0106] Referring to Figure 8b, it can be seen that anecdotal information with a pattern is generated from multiple activity information corresponding to commuting. For example, referring to Figure 8b, it can be seen that a user begins commuting at 8:10:37 AM on a weekday and completes commuting at 9:03:22 AM. Anecdotal information with a confidence value of 0.92 is generated with the pattern ID "PCW01."

[0107] Referring again to FIG. 7, at step S740, the user device (100) can manage the generated anecdotal information as a personalized knowledge graph based on the patterns contained in the generated anecdotal information. The personalized knowledge graph provides a personalized knowledge base based on information related to personal events in graph format.

[0108] According to one embodiment of the present disclosure, the user device (100) can determine which pattern of the generated anecdotal information corresponds to which pattern of anecdotal information in a personalized knowledge graph pre-stored in a personal semantic memory. The user device (100) can determine whether the pattern of the generated anecdotal information corresponds to which pattern of anecdotal information in the personalized knowledge graph and within a predetermined range according to a reliability value for the pattern. For example, if a first parameter value derived from a pattern of first anecdotal information corresponds to a second parameter value derived from a pattern of second anecdotal information in the personalized knowledge graph within a predetermined range according to a reliability value, the user device (100) can process the pattern of the first anecdotal information as corresponding to the pattern of the second anecdotal information.

[0109] The user device (100) may store a personalized knowledge graph reflecting the generated anecdotal information in the personal semantic memory based on the judgment result. If the pattern of the generated anecdotal information corresponds to a pattern of any anecdotal information in the personalized knowledge graph, the user device (100) may update the anecdotal information of the corresponding pattern, which is pre-stored in the personal semantic memory, with the generated anecdotal information. For example, if first anecdotal information about commuting based on activity information corresponding to one month of commuting (e.g., commuting in January) is stored as a personalized knowledge graph in the personal semantic memory, if the pattern of second anecdotal information generated based on activity information corresponding to one month of commuting (e.g., commuting in February) corresponds to the pattern of the first anecdotal information in the personalized knowledge graph, the first anecdotal information may be updated with the second anecdotal information. As a result, updated anecdotal information based on activity information corresponding to two months of commuting (January and February commuting) can be stored as a personalized knowledge graph in the individual semantic memory.

[0110] If the pattern of the generated anecdotal information does not correspond to the pattern of any anecdotal information in the personalized knowledge graph, the user device (100) may register the generated anecdotal information as new anecdotal information in the personalized knowledge graph and store it in the personal semantic memory. For example, if first anecdotal information about commuting based on activity information corresponding to one month of commuting (e.g., commuting in January) is stored as a personalized knowledge graph in the personal semantic memory, if the pattern of second anecdotal information generated based on activity information corresponding to one month of commuting (e.g., commuting in February) does not correspond to the pattern of the first anecdotal information in the personalized knowledge graph, the second anecdotal information may be registered as new anecdotal information in the personalized knowledge graph and stored in the personal semantic memory. As a result, the first anecdotal information about commuting based on activity information corresponding to commuting in January and the second anecdotal information about commuting based on activity information corresponding to commuting in February can be stored separately as personalized knowledge graphs in the individual semantic memory.

[0111] The user device (100) can manage the compression of anecdotal information stored in a personal semantic memory by adjusting the degree of matching between the pattern of the generated anecdotal information and the pattern of anecdotal information in a personalized knowledge graph. The user device (100) can vary the compression level of anecdotal information stored in a personal semantic memory based on the degree of matching between the pattern of the generated anecdotal information and the pattern of anecdotal information in a personalized knowledge graph. By compressing the anecdotal information stored in the personal semantic memory, the user device (100) can improve the search speed for anecdotal information.

[0112] According to one embodiment of the present disclosure, the user device (100) can manage routine information including at least one anecdotal information as the personalized knowledge graph based on a pattern of each anecdotal information stored in the personal semantic memory.

[0113] The user device (100) can generate routine information based on the temporal recurrence of the pattern of each anecdotal information stored in the personal semantic memory. For example, if the first anecdotal information has a pattern of occurring every morning at 9:00 AM from Monday to Friday, and the second anecdotal information has a pattern of occurring every morning at 6:00 PM from Monday to Friday, routine information including the first and second anecdotal information can be generated.

[0114] The user device (100) can generate routine information based on the continuity of the sequence of action information contained in each anecdote information stored in the personal semantic memory. For example, if the first anecdote information includes a first sequence of action information and the second anecdote information includes a second sequence of action information, and if the continuity of the first and second sequences is recognized, routine information including the first and second anecdote information can be generated.

[0115] The user device (100) can generate routine information based on the likelihood that each anecdotal information stored in the personal semantic memory will occur under certain situational conditions. For example, if the first anecdotal information necessarily occurs in a specific location, routine information can be generated that uses the specific location as the condition for the occurrence of the first anecdotal information.

[0116] FIG. 9 is a diagram illustrating an example of generating routine information from anecdotal information stored in a personal semantic memory according to one embodiment of the present disclosure.

[0117] Referring to FIG. 9, anecdotal information about commuting to work is stored in a personal semantic memory with a pattern ID of 'PCW01', and anecdotal information about commuting home is stored in a personal semantic memory with a pattern ID of 'PCH02'. For example, the user device (100) can generate routine information based on the temporal repetitiveness of the patterns of anecdotal information with a pattern ID of 'PCW01' and anecdotal information with a pattern ID of 'PCH02' stored in the personal semantic memory. The anecdotal information of pattern ID 'PCW01' has a pattern of commuting to work starting at around 8:10 every morning and taking about 53 minutes, and the anecdotal information of pattern ID 'PCH02' has a pattern of commuting starting at around 6:05 p.m. every day and taking about 1 hour and 15 minutes, so it can be seen that routine information for commuting to and from work, including the anecdotal information of 'PCW01' and the anecdotal information of pattern ID 'PCH02', is created with routine ID 'R1'.

[0118] Referring to FIG. 9, if the anecdotal information of pattern ID 'PC01' stored in the personal semantic memory includes a first sequence of action information, the anecdotal information of pattern ID 'PW02' includes a second sequence of action information, and continuity between the first sequence and the second sequence is recognized, routine information for the entire sequence for which continuity is recognized, including the anecdotal information of 'PC01' and the anecdotal information of 'PW02', may be generated with routine ID 'R2'.

[0119] Referring to Fig. 9, if the anecdotal information of pattern ID 'PPM01' is necessarily generated at a specific location, routine information that makes the specific location a condition for the occurrence of the anecdotal information of pattern ID 'PPM01' may be generated with routine ID 'R3'.

[0120] FIG. 10 is a diagram illustrating an example of expressing activity information stored in an episodic memory in the form of a knowledge graph by referencing a personalized knowledge graph stored in a personal semantic memory according to one embodiment of the present disclosure.

[0121] The user device (100) can store and represent activity information stored in the anecdotal memory in the form of a knowledge graph by referencing the personalized knowledge graph stored in the personal semantic memory. Assuming the multiple activity information stored in the anecdotal memory of FIG. 8A discussed above and the anecdotal information and routine information stored in the personal semantic memory of FIG. 8B and FIG. 9, FIG. 10 represents the user's activity information on December 23, 2022, in the form of a knowledge graph.

[0122] Looking at some of the data in the anecdotal memory of Fig. 8a, it can be seen that the user on December 23, 2022, started work at around 8:12 a.m., commuted for about 53 minutes, and listened to music from 8:15 a.m. to 9:02 a.m. Although not shown in some of the data in the anecdotal memory of Fig. 8a, it is assumed that the user started work at around 6:05 p.m. and commuted home. Activity information about the user's commuting to and from work can be expressed in the form of a knowledge graph by referring to routine ID 'R1', which is routine information about commuting to and from work, including anecdotal information of pattern ID 'PCW01' and anecdotal information of pattern ID 'PCH02'. Meanwhile, activity information about listening to music, which is not processed as anecdotal information with a pattern, can be expressed in the form of a knowledge graph in the form of nodes and edges as activity information.

[0123] FIG. 11 is a diagram illustrating an example of managing similar anecdotal information with different patterns in a personal semantic memory according to one embodiment of the present disclosure.

[0124] Even if anecdotal information is generated from the same type of activity information, anecdotal information with different patterns must be managed separately. If the pattern of the generated anecdotal information does not correspond to the pattern of similar anecdotal information in the personalized knowledge graph, the user device (100) can set a validity period for each generated anecdotal information and the similar anecdotal information, thereby managing each anecdotal information in the personal semantic memory.

[0125] Referring to Fig. 11, it can be seen that the anecdotal information of pattern ID 'PCW01' has a pattern of starting work at around 8:10 AM on weekdays and arriving at work at around 9:03 AM, whereas the anecdotal information of pattern ID 'PCW10' has a pattern of starting work at around 7:30 AM on weekdays and arriving at work at around 7:55 AM. If there is a change in the activity information about the user's commuting due to reasons such as a change in the user's workplace or job, anecdotal information about commuting with different patterns can be generated, as illustrated in Fig. 11. Since the anecdotal information of pattern ID 'PCW01' is anecdotal information generated from activity information about past commuting from March 11, 2021 to April 21, 2023, the corresponding past period can be set as the valid period. Since the anecdotal information for the pattern ID 'PCW10' is anecdotal information generated from activity information on commuting from April 25, 2023 to the present, the period after April 25, 2023 can be set as the validity period.

[0126] Referring to FIG. 11, routine information for commuting to and from work can also be managed as a personalized knowledge graph in a personal semantic memory by dividing it into routine ID 'R1', which is past routine information from March 11, 2021 to April 21, 2023, and routine ID 'R10', which is current routine information.

[0127] FIG. 12 is a diagram illustrating an example of managing similar anecdotal information having multiple patterns of different probability distributions in a personal semantic memory according to one embodiment of the present disclosure.

[0128] If the generated anecdotal information has multiple patterns of different probability distributions, the user device (100) can manage anecdotal information corresponding to each of the multiple patterns in a personal semantic memory. For example, if the generated anecdotal information has first and second patterns of different normal distributions, the anecdotal information corresponding to the first pattern and the anecdotal information corresponding to the second pattern can be managed separately in a personal semantic memory.

[0129] Referring to Fig. 12, the anecdotal information of pattern ID 'PCH01' has a pattern of starting work at around 6:05 PM and arriving home at around 7:21 PM, while the anecdotal information of pattern ID 'PCH02' has a pattern of starting work at around 9:12 PM and arriving home at around 10:02 PM, and each pattern can have a normal distribution. In the case where there is activity information about work leaving that has two mixed patterns, such as when a user has days of the week when he or she works overtime and days when he or she does not, anecdotal information about commuting with different patterns can be created and managed in the personal semantic memory, as illustrated in Fig. 12.

[0130] Referring to FIG. 12, routine information for commuting to and from work can also be managed as a personalized knowledge graph in a personal semantic memory by dividing it into routine ID 'R1' containing anecdotal information about commuting home with pattern ID 'PCH01' and routine ID 'R2' containing anecdotal information about commuting home with pattern ID 'PCH02'.

[0131] FIG. 13 is a diagram for explaining a management method for parameter values ​​representing properties for activity information according to one embodiment of the present disclosure.

[0132] Activity information can have parameters representing attributes. These parameters can be either derived from patterns representing attributes for multiple related activity information or unrelated to patterns.

[0133] As illustrated in Figure 13, all parameters of activity information can be stored separately in a parameter database. Data from the parameter database can be deleted beyond a certain period of time. Parameters derived from patterns of anecdotal information generated based on multiple pieces of related activity information can be managed in a personal semantic memory. The user device (100) can manage parameter values ​​derived from patterns of anecdotal information as user preference information for activity information in a personal semantic memory.

[0134] FIG. 14 is a diagram illustrating an AI platform based on a personalized knowledge graph according to one embodiment of the present disclosure.

[0135] Referring to FIG. 14, it shows that a personalized knowledge graph-based AI platform is provided to a user through a process of building a personalized database based on a knowledge graph (S1410) and a process of providing a personalized AI service to a user through a knowledge graph-based service application (S1420).

[0136] In the process of building a personalized database based on a knowledge graph (S1410), the user device (100) can process user data acquired from the user device (100) into information in a standardized form by referring to a standardized knowledge graph. The user device (100) can convert various types of unstructured data into the form of a knowledge graph, build a personalized database based on a knowledge graph, and store it in storage.

[0137] In the process of providing personalized AI services to users through a knowledge graph-based service application (S1420), the user device (100) can provide various services using a personalized database stored in storage. The user device (100) can provide recommendation services, assistant services, QA (Quenton Answering) services, etc. using anecdotal information, routine information, or parameter values ​​stored in a personal semantic memory.

[0138] For example, if the user device (100) has information on the activity being performed by the user, the user device (100) can recommend music that the user enjoys listening to when the user is performing the activity through a recommendation service, or can provide relevant past experiences through an assistant service. The user device (100) can identify the user's behavioral pattern from a personalized database, and if the user shows a pattern that is different from the user's usual behavioral pattern, the user device (100) can provide a solution service related to the cause of the different pattern through a recommendation service. The user device (100) can provide a customized answer to the user's question based on a personalized database through a QA service. The user device (100) can utilize an episodic memory or personal semantic memory to perform a journaling function that manages and describes the user's daily routine or special events through an assistant service.

[0139] FIG. 15 is a block diagram illustrating a user device (100) according to one embodiment of the present disclosure. FIG. 16 is a block diagram for explaining the configuration and operation of a user device (100) according to one embodiment of the present disclosure.

[0140] Referring to FIG. 15, a user device (100) according to one embodiment may include, but is not limited to, a memory (110) and a processor (120), and general-purpose configurations may be further added. For example, as illustrated in FIG. 16, the user device (100) may further include, in addition to the memory (110) and the processor (120), a sensing unit (130), a communication unit (140), and an input / output unit (150). Hereinafter, each configuration will be described in detail with reference to FIGS. 15 and 16.

[0141] The memory (110) according to one embodiment can store a program for processing and controlling the processor (120), and can store data and information input to or generated from the user device (100). The memory (110) can store instructions, data structures, and program codes that can be read by the processor (120). Operations performed by the processor (120) can be implemented by executing instructions or program codes stored in the memory (110).

[0142] The memory (110) according to one embodiment may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), and may include a non-volatile memory including at least one of a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk, and a volatile memory such as a RAM (Random Access Memory) or an SRAM (Static Random Access Memory).

[0143] According to one embodiment, the memory (110) may store one or more instructions and / or programs that control the user device (100) to train a neural network model or utilize a neural network model.

[0144] According to one embodiment, the processor (120) can control operations or functions so that the user device (100) can perform tasks by executing instructions or programmed software modules stored in the memory (110). The processor (120) can be composed of hardware components that perform arithmetic, logic, and input / output operations and signal processing. The processor (120) can control the overall operations of the user device (100) by executing one or more instructions stored in the memory (110). The processor (120) can control the sensing unit (130), the communication unit (140), and the input / output unit (150) including at least one sensor by executing programs stored in the memory (110).

[0145] The processor (120) according to one embodiment may be configured with at least one of, for example, a Central Processing Unit, a Microprocessor, a Graphic Processing Unit, Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), an Application Processor, a Neural Processing Unit, or an AI-dedicated processor designed with a hardware structure specialized for processing an AI model, but is not limited thereto. Each processor constituting the processor (120) may be a dedicated processor for performing a predetermined function.

[0146] An artificial intelligence (AI) processor according to one embodiment may perform calculations and control to process tasks set for a user device (100) to perform using an artificial intelligence (AI) model. The AI ​​processor may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or may be manufactured as part of a general-purpose processor (e.g., a CPU or application processor) or a graphics-only processor (e.g., a GPU) and mounted on the user device (100).

[0147] The sensing unit (130) according to one embodiment may include a plurality of sensors configured to detect information about the surrounding environment of the user device (100). For example, the sensing unit (130) may include, but is not limited to, a camera (131), a temperature / humidity sensor (132), an infrared sensor (133), a barometric pressure sensor (134), a position sensor (135), a gyroscope sensor (136), etc. The function of each sensor can be intuitively inferred from its name by those skilled in the art, and thus is briefly described below.

[0148] The camera (131) according to one embodiment may include a stereo camera, a mono camera, a wide-angle camera, an around-view camera, or a 3D vision sensor. The lidar sensor (132) may detect the distance to an object and various physical properties by shining a laser on the target. The temperature / humidity sensor (132) may measure the temperature or humidity of the location where the user device (100) is located. The infrared sensor (133) may be either an active infrared sensor that detects changes by emitting infrared rays and blocking the light, or a passive infrared sensor that does not have a light emitter and only detects changes in infrared rays received from the outside. The barometric pressure sensor (134) may measure the barometric pressure of the location where the user device (100) is located. The location sensor (135) may detect the location of the user device (100). For example, the location sensor (135) may be a global positioning system (GPS). The gyroscope sensor (136) may detect angular velocity. The gyro sensor (136) can be used to measure the position of the user device (100) and set the moving direction of the user device (100).

[0149] The communication unit (140) may include one or more components that enable the user device (100) to communicate with an external device, such as a server or other electronic device. For example, the communication unit (140) may include, but is not limited to, a short-range wireless communication unit (141), a mobile communication unit (142), etc.

[0150] The short-range wireless communication unit (141) may include, but is not limited to, a Bluetooth communication unit, a BLE (Bluetooth Low Energy) communication unit, a near field communication unit, a WLAN (Wi-Fi) communication unit, a Zigbee communication unit, an Ant+ communication unit, a WFD (Wi-Fi Direct) communication unit, a UWB (ultra wideband) communication unit, an infrared (IrDA, infrared Data Association) communication unit, a microwave (uWave) communication unit, etc.

[0151] The mobile communication unit (142) transmits and receives wireless signals with at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signals may include various types of data, such as voice call signals, video call signals, or text / multimedia message transmission and reception.

[0152] The input / output unit (150) may include an input unit (151) and an output unit (153). The input / output unit (150) may be a separate input unit (151) and an output unit (153), or may be an integrated unit, such as a touch screen. The input / output unit (150) may receive input information from a user and provide output information to the user.

[0153] The input unit (151) may refer to a means for a user to input data for controlling the user device (100). For example, the input unit (151) may be a key pad, a touch panel (contact electrostatic capacitance type, pressure resistive film type, infrared detection type, surface ultrasonic conduction type, integral tension measurement type, piezo effect type, etc.), etc. In addition, the input unit (151) may include a jog wheel, a jog switch, etc., but is not limited thereto.

[0154] The output unit (153) can output an audio signal, a video signal, or a vibration signal, and the output unit (153) can include a display unit, an audio output unit, and a vibration motor. The display unit can display information processed in the user device (100). For example, the display unit can display a user interface for receiving a user's operation. When the display unit and the touchpad are configured as a touch screen in a layered structure, the display unit can be used as an input device in addition to an output device. The display unit can include at least one of a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, and a 3D display. Depending on the implementation form of the user device (100), the user device (100) can include two or more display units. The audio output unit can output audio data stored in the memory (110). The audio output unit can output an audio signal related to a function performed in the user device (100). The audio output section may include a speaker, a buzzer, etc.

[0155] According to one embodiment of the present disclosure, a user device (100) includes a memory (110) storing at least one instruction and at least one processor (120) operatively connected to the memory (110) to execute at least one instruction. The processor (120) may execute the at least one instruction to load and execute a command or code for a given module.

[0156] The processor (120) of the user device (100) according to one embodiment of the present disclosure can manage user activity information based on data acquired from the user device (100).

[0157] The processor (120) may execute at least one instruction to generate action information corresponding to a unit action of the user based on data acquired from the user device (100). According to one embodiment, the processor (120) may extract action information matching the unit action based on the data acquired from the user device (100) from an action information database of a pre-built knowledge graph. The processor (120) may compare an embedding vector of the unit action with an embedding vector of at least one action information candidate in the action information database, and classify action information for the unit action based on the comparison result, thereby extracting action information matching the unit action. The processor (120) may determine the extracted action information as action information corresponding to the unit action. According to one embodiment, the processor (120) may generate action information by inferring second action information based on time information or location information based on the acquired data in a state where first action information is generated.

[0158] The processor (120) may execute at least one instruction to generate an activity information candidate corresponding to a sequence including a series of continuous action information based on the acquired data or generated action information. According to one embodiment, when one or more pieces of action information are generated based on the acquired data or generated action information, the processor (120) may extract activity information including a sequence of one or more pieces of action information from an activity information database of a built-up knowledge graph. The processor (120) may determine the extracted activity information as an activity information candidate corresponding to the sequence of one or more pieces of action information.

[0159] The processor (120) may execute at least one instruction to change the generated activity information candidate according to an update of the acquired data or the generated action information. According to one embodiment, the processor (120) may extract activity information including a sequence of action information to which updates are reflected, according to an update of the acquired data or the generated action information, from an activity information database of a conventionally constructed knowledge graph. The processor (120) may extract activity information including a sequence of action information to which updates are reflected, based on an activity information database having a hierarchical structure in which a hierarchy arranged according to a concept of upper and lower levels between activity information is determined, by concretizing activity information of a higher level concept into activity information of a lower level concept according to an update of the acquired data or the generated action information. The processor (120) may determine the extracted activity information as an activity information candidate corresponding to the sequence of action information to which updates are reflected.

[0160] The processor (120) may execute at least one instruction to determine activity information from a candidate for changed activity information based on a condition for determining activity information. In one embodiment, the processor (120) may determine activity information from a candidate for activity information for which termination action information has been generated. In one embodiment, the processor (120) may determine activity information by inferring activity information from a candidate for changed activity information for which a probability corresponding to a similarity with the activity information based on the progress of the sequence satisfies a predetermined condition.

[0161] According to one embodiment of the present disclosure, the processor (120) of the user device (100) can build a personalized knowledge base based on the user's activity information managed in the user device (100).

[0162] The processor (120) may execute at least one instruction to generate action information corresponding to a unit action of the user based on data acquired from the user device (100). The processor (120) may execute at least one instruction to determine activity information corresponding to a sequence including a series of continuous action information. According to one embodiment, the processor (120) may determine activity information corresponding to the sequence by using an activity information database inductively defined based on information related to an individual user's event stored in a personal semantic memory or an activity information database predefined based on semantic information (common sense or factual knowledge) stored in a semantic memory.

[0163] The processor (120) may generate anecdotal information based on a plurality of pieces of related activity information by executing at least one instruction. According to one embodiment, the processor (120) may generate anecdotal information by deriving parameter values ​​representing common attributes of a plurality of pieces of related activity information as a pattern of anecdotal information and determining a reliability value for the pattern.

[0164] The processor (120) may execute at least one instruction to manage the generated anecdote information as a personalized knowledge graph based on a pattern of the generated anecdote information. According to one embodiment, the processor (120) may determine which pattern of the generated anecdote information corresponds to which pattern of anecdote information in a personalized knowledge graph pre-stored in a personal semantic memory. The processor (120) may determine whether the pattern of the generated anecdote information corresponds to which pattern of anecdote information in the personalized knowledge graph and whether it falls within a predetermined range based on a reliability value for the pattern. Based on the determination result, the processor (120) may store a personalized knowledge graph reflecting the generated anecdote information in the personal semantic memory. If the pattern of the generated anecdote information corresponds to a pattern of anecdote information in the personalized knowledge graph, the processor (120) may update the anecdote information of the corresponding pattern pre-stored in the personal semantic memory with the generated anecdote information. If the pattern of the generated anecdotal information does not correspond to a pattern of any anecdotal information in the personalized knowledge graph, the processor (120) can register the generated anecdotal information as new anecdotal information in the personalized knowledge graph and store it in the personal semantic memory.

[0165] According to one embodiment, the processor (120) can manage the compression of the anecdotal information stored in the personal semantic memory by adjusting the degree of matching between the pattern of the generated anecdotal information and the pattern of anecdotal information in the personalized knowledge graph. The processor (120) can vary the compression level of the anecdotal information stored in the personal semantic memory based on the degree of matching between the pattern of the generated anecdotal information and the pattern of anecdotal information in the personalized knowledge graph. By compressing the anecdotal information stored in the personal semantic memory, the processor (120) can improve the search speed for the anecdotal information in the user device (100).

[0166] According to one embodiment, the processor (120) can manage routine information including at least one anecdotal information as a personalized knowledge graph based on a pattern of each anecdotal information stored in a personal semantic memory.

[0167] According to one embodiment, if the pattern of the generated anecdotal information does not correspond to the pattern of the similar anecdotal information of the personalized knowledge graph, the processor (120) can set a validity period for each of the generated anecdotal information and the similar anecdotal information to manage each anecdotal information in the personal semantic memory.

[0168] According to one embodiment, when the generated anecdotal information has multiple patterns of different probability distributions, the processor (120) can manage anecdotal information corresponding to each of the multiple patterns in a personal semantic memory.

[0169] According to one embodiment, the processor (120) can manage parameter values ​​derived from a predetermined pattern of anecdotal information as preference information for the user's activity information in a personal semantic memory.

[0170] FIG. 17 is a diagram for explaining a process of managing action information and activity information in a user device (100) according to one embodiment of the present disclosure.

[0171] Referring to FIG. 17, an embedding vector of a user's unit action can be obtained by a first encoder from metadata about content such as text, photos, videos, and music acquired from a user device (100), or data corresponding to a time or place related to the content. An embedding vector of a user's unit action can be obtained by a second encoder from sensing data detected from at least one sensor in the user device (100). An embedding vector of a user's unit action can be obtained by a third encoder from metadata about an application used in the user device (100), usage log data, log data such as the time or place of an event occurrence, and the like.

[0172] The action information generation unit (action generator) can process unstructured data acquired from the user device (100) into structured information and store the information in the working memory by referring to the structured form of semantic information stored in the semantic memory. The action information generation unit can compare the embedding vector of the user's unit action acquired by at least one encoder with the embedding vector of at least one action information candidate in the action information database of the constructed knowledge graph. The action information generation unit can extract action information matching the user's unit action from the action information database of the constructed knowledge graph based on the comparison result between the embedding vectors. The action information generation unit can convert various types of unstructured data into the form of a knowledge graph and store the converted data in the working memory. The action information generation unit can store the generated action information in the working memory.

[0173] An activity information generator can determine whether activity information is determined from activity information candidates corresponding to a sequence of action information stored in a working memory. The activity information generator can extract activity information corresponding to a sequence of action information using an activity information database stored in a semantic memory or a personal semantic memory. When activity information is determined, the activity information generator can delete activity information candidates managed in the working memory related to the determined activity information and control the activity information to be stored and managed in an episodic memory. The activity information generator can track activity information currently in progress based on a sequence formed by chaining a series of action information and delete completed activity information from the working memory.

[0174] FIG. 18 is a diagram illustrating a process of building a personalized database based on activity information in a user device (100) according to one embodiment of the present disclosure.

[0175] Referring to FIG. 18, the user device (100) may have a first memory including a working memory. Action information generated through the first encoder, the second encoder, and the third encoder and a central executive may be stored in the working memory. The user device (100) may have a second memory, which is a permanent memory including an episodic memory, a personal semantic memory, and a semantic memory. A long-term memory manager may manage a database of policies or built-up knowledge graphs of the episodic memory, the personal semantic memory, and the semantic memory belonging to the second memory.

[0176] The user device (100) can delete completed activity information from the working memory and store and manage the completed activity information using the episode memory and personal semantic memory. The episode information generation unit can generate episode information based on a plurality of related activity information among the plurality of activity information accumulated in the episode memory. The episode information generation unit can generate episode information by deriving common attributes of the plurality of related activity information as a pattern and determining a reliability value for the pattern. Based on the pattern of the generated episode information, the generated episode information can be managed as a personalized knowledge graph in the personal semantic memory. Based on the pattern of each episode information stored in the personal semantic memory, routine information including at least one episode information can be managed as a personalized knowledge graph in the personal semantic memory. The user device (100) can build a personalized database based on the user's activity information managed in the user device (100). The episodic recaller can retrieve information stored in episodic memory, and the recognizer can retrieve information stored in personal semantic memory or semantic memory.

[0177] Meanwhile, embodiments of the present disclosure may also be implemented in the form of a recording medium containing computer-executable instructions, such as program modules, executed by a computer. Computer-readable media may be any available media that can be accessed by a computer, and include both volatile and nonvolatile media, removable and non-removable media. Furthermore, computer-readable media may include computer storage media and communication media. Computer storage media include both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Communication media may typically include computer-readable instructions, data structures, or other data in a modulated data signal, such as program modules.

[0178] Additionally, a computer-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" simply means a tangible device that does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is permanently stored in the storage medium and cases where data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0179] According to one embodiment, a method according to one embodiment of the present disclosure may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0180] According to one embodiment of the present disclosure, a computer-readable recording medium having recorded thereon a program for executing a method for managing activity information in the above user device and a method for constructing a database based on activity information is provided.

[0181] According to one embodiment of the present disclosure, a method for managing activity information in a user device (100) is provided. The method for managing activity information in the user device (100) may include a step (S310) of generating action information corresponding to a unit action of the user based on data acquired from the user device. In addition, the method for managing activity information in the user device (100) may include a step (S320) of generating an activity information candidate corresponding to a sequence including a series of continuous action information based on the acquired data or the generated action information. In addition, the method for managing activity information in the user device (100) may include a step (S330) of changing the generated activity information candidate according to an update of the acquired data or the generated action information. In addition, the method for managing activity information in the user device (100) may include a step (S340) of determining activity information from the changed activity information candidate based on a condition for determining the activity information.

[0182] Additionally, according to one embodiment of the present disclosure, the step of generating action information (S310) may include a step of extracting action information matching a unit action based on data acquired from a user device from an action information database of a constructed knowledge graph. Furthermore, the step of generating action information (S310) may include a step of determining the extracted action information as action information corresponding to the unit action.

[0183] Additionally, the step of extracting action information may include a step of comparing the embedding vector of the unit action with the embedding vector of at least one action information candidate within the action information database. Furthermore, the step of extracting action information may include a step of classifying the action information for the unit action based on the comparison result.

[0184] Additionally, according to one embodiment of the present disclosure, the step (S310) of generating action information may generate action information by inferring second action information based on time information or location information based on acquired data in a state where first action information is generated.

[0185] Additionally, according to one embodiment of the present disclosure, the step (S320) of generating activity information candidates may include a step of extracting activity information including a sequence of one or more action information from an activity information database of a built-up knowledge graph when one or more pieces of action information are generated based on acquired data or generated action information. Additionally, the step (S320) of generating activity information candidates may include a step of determining the extracted activity information as an activity information candidate corresponding to the sequence of one or more pieces of action information.

[0186] Additionally, according to one embodiment of the present disclosure, the step (S330) of modifying the generated activity information candidate may include a step of extracting activity information including a sequence of action information to which updates have been reflected, based on updates to acquired data or generated action information, from the activity information database of the constructed knowledge graph. Furthermore, the step (S330) of modifying the generated activity information candidate may include a step of determining the extracted activity information as an activity information candidate corresponding to the sequence of action information to which updates have been reflected.

[0187] In addition, the step of extracting activity information including a sequence of action information to which updates are reflected is based on an activity information database having a hierarchical structure in which a hierarchy arranged according to a concept of upper and lower levels among activity information is determined, and, according to an update of acquired data or generated action information, activity information of a higher concept is concretized into activity information of a lower concept, thereby extracting activity information including a sequence of action information to which updates are reflected.

[0188] Additionally, according to one embodiment of the present disclosure, activity information may include start action information and end action information. Furthermore, the step (S340) of determining activity information may determine activity information from activity information candidates for which end action information is generated.

[0189] In addition, according to one embodiment of the present disclosure, the step (S340) of determining activity information may infer activity information from an activity information candidate whose probability corresponding to the similarity with the activity information based on the progress of the sequence among the changed activity information candidates satisfies a predetermined condition.

[0190] According to one embodiment of the present disclosure, a computer-readable recording medium having recorded thereon a program for executing a method for managing activity information in the above user device (100) is provided.

[0191] According to one embodiment of the present disclosure, a user device (100) for managing activity information is provided. The user device (100) may include a memory (110) storing at least one instruction and at least one processor (120) operatively connected to the memory (110) and executing at least one instruction. In addition, the at least one processor (120) may execute at least one instruction to generate action information corresponding to a unit action of the user based on data acquired by the user device (100). In addition, the at least one processor (120) may execute at least one instruction to generate an activity information candidate corresponding to a sequence including a series of continuous action information based on the acquired data or the generated action information. In addition, the at least one processor (120) may execute at least one instruction to change the generated activity information candidate according to an update of the acquired data or the generated action information. Additionally, at least one processor (120) may execute at least one instruction to determine activity information from the changed activity information candidates based on a condition for determining activity information.

[0192] Additionally, according to one embodiment of the present disclosure, at least one processor (120) may execute at least one instruction to extract action information matching a unit action based on data acquired from a user device from an action information database of a constructed knowledge graph, and determine the extracted action information as action information corresponding to the unit action.

[0193] Additionally, at least one processor (120) may execute at least one instruction to compare an embedding vector of a unit operation with an embedding vector of at least one action information candidate in an action information database, and classify action information for the unit operation based on the comparison result.

[0194] Additionally, according to one embodiment of the present disclosure, at least one processor (120) may generate action information by executing at least one instruction, and inferring second action information based on time information or location information based on acquired data in a state where first action information is generated.

[0195] Additionally, according to one embodiment of the present disclosure, at least one processor (120) may execute at least one instruction to extract activity information including a sequence of one or more action information from an activity information database of a built-up knowledge graph when one or more action information pieces are generated based on acquired data or generated action information. Additionally, at least one processor (120) may execute at least one instruction to determine the extracted activity information as an activity information candidate corresponding to the sequence of one or more action information pieces.

[0196] Additionally, according to one embodiment of the present disclosure, at least one processor (120) may execute at least one instruction to extract activity information including a sequence of action information to which updates have been reflected, based on updates to acquired data or generated action information, from an activity information database of a constructed knowledge graph. Additionally, at least one processor (120) may execute at least one instruction to determine the extracted activity information as an activity information candidate corresponding to the sequence of action information to which updates have been reflected.

[0197] In addition, at least one processor (120) executes at least one instruction to extract activity information including a sequence of action information in which the update is reflected, by concretizing the activity information of the upper concept into the activity information of the lower concept according to the update of the acquired data or the generated action information based on the activity information database of the hierarchical structure in which the hierarchy arranged according to the upper and lower concepts between the activity information is determined.

[0198] Additionally, according to one embodiment of the present disclosure, activity information may include start action information and end action information. Furthermore, at least one processor (120) may execute at least one instruction to determine activity information from activity information candidates for which end action information is generated.

[0199] Additionally, at least one processor (120) can execute at least one instruction to infer activity information from an activity information candidate whose probability corresponding to the similarity with the activity information based on the progress of the sequence among the changed activity information candidates satisfies a predetermined condition.

[0200] Additionally, according to one embodiment of the present disclosure, at least one processor (120) may execute at least one instruction to generate anecdotal information based on a plurality of related activity information and manage the anecdotal information in a personal semantic memory.

[0201] One embodiment of the present disclosure can be configured to improve computer functionality (i.e., improve the functionality of the computer itself) by enhancing the performance of AI in analyzing user actions and activities. In particular, by constructing a knowledge graph, user data can be well organized for AI utilization.

[0202] The above description of the present disclosure is provided for illustrative purposes only, and those skilled in the art will readily appreciate that modifications to other specific forms can be made without altering the technical spirit or essential features of the present disclosure. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, components described as being single may be implemented in a distributed manner, and similarly, components described as being distributed may be implemented in a combined manner.

[0203] The scope of the present disclosure is indicated by the claims described below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present disclosure.

Claims

1. A step (S310) of generating action information corresponding to a user's action based on data acquired from a user device; A step (S320) of generating activity information candidates corresponding to a series of continuous action information based on the acquired data or the action information; A step of changing the activity information candidate according to the update of the acquired data or the action information (S330); A step (S340) of determining the activity information from the changed activity information candidates based on the conditions for determining the activity information; and A step of storing and managing the activity information to build a personalized knowledge graph on the user device; A method of managing activity information on a user device, including:

2. In paragraph 1, The step of generating the above action information (S310) is: A step of extracting action information matching the above-mentioned action based on the acquired data from the action information database of the constructed knowledge graph; and A method comprising a step of determining the extracted action information as action information corresponding to the operation.

3. In paragraph 1 or 2, The step of extracting the above action information is: A step of comparing the embedding vector of the above action with the embedding vector of at least one action information candidate in the action information database; and A method comprising a step of classifying action information for the operation according to the comparison result.

4. In any one of paragraphs 1 to 3, A method in which the above action information is generated by inferring second action information based on time information or location information based on the acquired data, based on the first action information being generated.

5. In any one of paragraphs 1 to 4, The step (S320) of generating the above activity information candidate is: A step of extracting activity information including a sequence of one or more action information from an activity information database of a knowledge graph constructed based on one or more action information generated according to the acquired data or the action information; and A method comprising a step of determining the extracted activity information as an activity information candidate corresponding to a sequence of one or more action information.

6. In any one of paragraphs 1 to 5, The step of changing the above activity information candidate (S330) is: A step of extracting activity information including a sequence of action information in which the update is reflected, according to an update of the acquired data or the action information, from an activity information database of a constructed knowledge graph; and A method comprising a step of determining the extracted activity information as an activity information candidate corresponding to a sequence of action information in which the update is reflected.

7. In any one of paragraphs 1 to 6, The above extraction step is, A method for extracting activity information including a sequence of action information in which the update is reflected, by having the activity information of a higher concept be concretized into the activity information of a lower concept based on the activity information database having a hierarchical structure in which the hierarchy is determined according to the upper and lower concepts between the activity information, according to the update of the acquired data or the action information.

8. In any one of paragraphs 1 to 7, The above activity information has start action information and end action information, The step (S340) of determining the above activity information is: A method for determining the activity information from the activity information candidates for which the above termination action information is generated.

9. In any one of paragraphs 1 to 8, The step (S340) of determining the above activity information is: A method for inferring the activity information from the changed activity information candidate, wherein the probability corresponding to the similarity with the activity information based on the progress of the series of continuous action information satisfies a predetermined condition.

10. A computer-readable recording medium having recorded thereon a program for executing any one of the methods of clauses 1 to 9.

11. At least one processor (120); and Contains a memory (110) storing at least one instruction, When the at least one instruction is executed by the at least one processor, the at least one processor (120) causes: The at least one processor (120) executes the at least one instruction, A user device (100) that generates action information corresponding to a user's action based on data acquired from a user device (100), generates an activity information candidate corresponding to a series of continuous action information based on the acquired data or the action information, changes the activity information candidate according to an update of the acquired data or the action information, determines the activity information from the changed activity information candidate based on a condition for confirming the activity information, and stores and manages the activity information to build a personalized knowledge graph in the user device (100).

12. In paragraph 11, The at least one instruction causes the at least one processor (120) to: A user device (100) that extracts action information matching the action based on the acquired data from the action information database of the constructed knowledge graph, and determines the extracted action information as action information corresponding to the action.

13. In paragraph 11 or 12, The at least one instruction causes the at least one processor (120) to: A user device (100) that generates the action information by inferring the second action information based on time information or location information based on the acquired data, based on the first action information generated.

14. In any one of paragraphs 11 to 13, The at least one processor (120) executes the at least one instruction, A user device (100) that extracts activity information including a sequence of the one or more action information from an activity information database of a knowledge graph that has been constructed based on the one or more action information generated according to the acquired data or the action information, and determines the extracted activity information as an activity information candidate corresponding to the sequence of the one or more action information.

15. In any one of paragraphs 11 to 14, The at least one processor (120) executes the at least one instruction, A user device (100) that extracts activity information including a sequence of action information to which the update is reflected, according to an update of the acquired data or the action information, from an activity information database of a constructed knowledge graph, and determines the extracted activity information as an activity information candidate corresponding to the sequence of action information to which the update is reflected.

Images