Method, computer device, and computer program for providing optimal routes using expert knowledge

Abstract

A method, computer device, and computer program are provided for providing an optimal route using expert knowledge. [Solution] A method for providing an optimal route using expert knowledge includes predicting the next action of a target user through an optimal route prediction model learned using a graph-type path that represents the user behavior trajectory on a session-by-session basis, and recommending the predicted action path as an optimal route, wherein the recommending step includes grasping the user intent based on the target user's previous behavior trajectory in the current session, and determining a path corresponding to the user intent as one of the optimal routes.

Description

[Technical Field]

[0001] The following description relates to a technique for finding patterns in user behavior and providing optimal routes. [Background technology]

[0002] In the current environment where a huge amount and variety of content is available, services that recommend content to users based on various criteria have appeared. A typical example of such services is the personalized recommendation service.

[0003] A personal recommendation service is a service that recommends content that is suitable for a user based on the user's activities (for example, content usage patterns, content purchase patterns, etc.), that is, personalized content.

[0004] Existing methods for such personalized recommendation services analyze a user's activity patterns based on their past activities, search for other users with similar patterns, and recommend to the user content that the other users have used or purchased.

[0005] Korean Patent Publication No. 10-1647364 (registered on August 4, 2016) discloses a technology that targets multiple user groups and recommends suitable content from among multiple pieces of content based on a score for each user group based on the interactions of multiple users with each piece of content, and a priority based on the user group to which the user belongs, thereby recommending content that is more suitable to the user. Summary of the Invention [Problem to be solved by the invention]

[0006] A method and apparatus are provided for modeling a user experience and recommending an optimal route to achieve a goal.

[0007] A method and apparatus are provided for recommending a route that reflects expert knowledge as the optimal route for achieving a user's goal. [Means for solving the problem]

[0008] Provided is a method executed on a computer device, the computer device including at least one processor configured to execute computer-readable instructions contained in a memory, the method including a step of predicting, by the at least one processor, a next action of a target user using an optimal path prediction model learned using a graph-form path that represents a user behavior trajectory on a session-by-session basis, and recommending the predicted action path as an optimal path, the step of recommending including a step of understanding a user intent based on the target user's previous behavior trajectory in the current session, and a step of determining a path corresponding to the user intent as one of the optimal paths.

[0009] According to one aspect, the determining step may determine a path based on expert knowledge by calling an expert AI model corresponding to the user intent from among expert AI models trained using paths determined as correct answer data in consideration of rules defined in expert knowledge.

[0010] According to another aspect, the optimal route prediction model may include an expert AI model for each expert category that has learned a path based on the expert knowledge of the corresponding category, and the determining step may determine a path based on the expert knowledge using the expert AI model of the category that is appropriate for the user intent.

[0011] According to another aspect, the graph-shaped path is composed of a state at each time stage, an action in the state, and a reward for the action, and the state is defined by the content of the service screen consumed by the user and may further include at least one of the type of service, environmental information associated with the user, personal information of the user, and a session category, the action is defined by user activity in the state, and the reward is defined by user satisfaction with the action.

[0012] According to another aspect, paths based on expert knowledge may be rewarded more highly than other paths.

[0013] According to yet another aspect, the determining step may include displaying an interface on a service screen of the current session to guide the expert knowledge-based path.

[0014] According to another aspect, the determining step may further include requesting user input associated with the user intent through an interface for guiding a path based on expert knowledge.

[0015] According to yet another aspect, the determining step may further include providing additional information corresponding to the user input as an expert knowledge-based path.

[0016] According to yet another aspect, the determining step may include defining a current situation of the target user and guiding a personalized path based on the current situation.

[0017] According to yet another aspect, the current situation of the target user may be defined by at least one of the target user's current session or user behavior history within a recent period of time, the user's environmental information, and the user's personal information.

[0018] A computer program is provided that is recorded on a computer-readable recording medium for executing the above method on a computer device.

[0019] Provided is a computer device including at least one processor configured to execute computer-readable instructions stored in a memory, wherein the at least one processor processes a process of predicting a target user's next action using an optimal path prediction model learned from a graph-form path that represents a user behavior trajectory on a session-by-session basis, and recommending the predicted action path as an optimal path, and the at least one processor grasps a user intent based on the target user's previous behavior trajectory in the current session, and determines a path corresponding to the user intent as one of the optimal paths. [Effects of the Invention]

[0020] Embodiments of the present invention utilize artificial intelligence models to discover patterns in user behavior and recommend optimal routes to user goals while eliminating unnecessary steps.

[0021] According to an embodiment of the present invention, by expressing and modeling user experiences as paths, it is possible to build a hyper-personalized model for predicting optimal routes, and to build a platform that is independent of specific domains or services.

[0022] According to an embodiment of the present invention, by recommending a route represented by expert knowledge as a route suited to the user intent, it is possible to guide the user to an optimal user experience to reach the user's goal more quickly. [Brief explanation of the drawings]

[0023] [Figure 1] FIG. 1 illustrates an example of a network environment in accordance with an embodiment of the present invention. [Figure 2]FIG. 1 is a block diagram illustrating an example of a computing device according to an embodiment of the present invention. [Figure 3] 1 is a flowchart illustrating an example method that may be performed by a computing device in accordance with an embodiment of the present invention. [Figure 4] FIG. 1 illustrates an example of a user experience scenario in accordance with an embodiment of the present invention. [Figure 5] FIG. 1 illustrates an example of a user experience scenario in accordance with an embodiment of the present invention. [Figure 6] 1 is a diagram illustrating an example of services connected in a network configuration according to an embodiment of the present invention. [Figure 7] FIG. 10 illustrates an example of a graph path representing user recorded session data in accordance with one embodiment of the present invention. [Figure 8] FIG. 10 is a diagram illustrating an example of a path configuration representing a user experience in one embodiment of the present invention. [Figure 9] 10 is an exemplary diagram illustrating a process of finding an optimal route using a model that has been trained by expressing a user experience as a path in accordance with an embodiment of the present invention; FIG. [Figure 10] 10 is an exemplary diagram illustrating a process of finding an optimal route using a model that has been trained by expressing a user experience as a path in accordance with an embodiment of the present invention; FIG. [Figure 11] 1 is a flowchart illustrating an example of a process for providing an optimal route using expert knowledge in an embodiment of the present invention. [Figure 12] FIG. 10 illustrates an example path configuration including expert experience in accordance with an embodiment of the present invention. [Figure 13] FIG. 1 illustrates an example of an expert-embedded AI model in accordance with an embodiment of the present invention. [Figure 14] FIG. 1 illustrates an example UA service scenario in accordance with an embodiment of the present invention. [Figure 15] FIG. 1 illustrates an example UA service scenario in accordance with an embodiment of the present invention. [Figure 16]FIG. 1 illustrates an example UA service scenario in accordance with an embodiment of the present invention. [Figure 17] FIG. 1 illustrates an example UA service scenario in accordance with an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0024] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0025] SUMMARY OF THE INVENTION Embodiments of the present invention relate to techniques for discovering patterns in user behavior and providing optimal routes.

[0026] Embodiments including those specifically disclosed in this specification can recommend optimal routes for achieving user goals on a platform that allows multiple services to be linked together.

[0027] An optimal route recommendation system according to an embodiment of the present invention may be realized by at least one computer device, and an optimal route recommendation method according to an embodiment of the present invention may be executed by at least one computer device included in the optimal route recommendation system. In this case, a computer program according to an embodiment of the present invention may be installed and executed in the computer device, and the computer device may execute the optimal route recommendation method according to an embodiment of the present invention under the control of the executed computer program. The above-mentioned computer program may be combined with the computer device and recorded on a computer-readable recording medium to execute the optimal route recommendation method on a computer.

[0028] FIG. 1 is a diagram showing an example of a network environment in one embodiment of the present invention. The network environment in FIG. 1 shows an example including multiple electronic devices 110, 120, 130, and 140, multiple servers 150 and 160, and a network 170. FIG. 1 is merely an example for explaining the invention, and the number of electronic devices and the number of servers are not limited to those shown in FIG. 1. Furthermore, the network environment in FIG. 1 is merely an example for explaining an environment applicable to this embodiment, and the environment applicable to this embodiment should not be limited to the network environment in FIG. 1.

[0029] The electronic devices 110, 120, 130, and 140 may be fixed or mobile terminals implemented by computers. Examples of the electronic devices 110, 120, 130, and 140 include smartphones, mobile phones, navigation systems, personal computers (PCs), notebook PCs, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), and tablets. While FIG. 1 illustrates a smartphone as an example of the electronic device 110, in embodiments of the present invention, the electronic device 110 may represent one of a variety of physical computer systems capable of communicating with other electronic devices 120, 130, and 140 and / or servers 150 and 160 via a network 170 using a substantially wireless or wired communication method.

[0030] The communication method is not limited, and may include not only a communication method utilizing a communication network (for example, a mobile communication network, a wired Internet, a wireless Internet, or a broadcast network) that can be included in network 170, but also short-range wireless communication between devices. For example, network 170 may include any one or more of a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), the Internet, etc. Furthermore, network 170 may include any one or more of a network topology including, but not limited to, a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or hierarchical network, etc.

[0031] Each of the servers 150, 160 may be implemented by one or more computing devices that communicate with the plurality of electronic devices 110, 120, 130, 140 via the network 170 and provide instructions, code, files, content, services, etc. For example, the server 150 may be a system that provides a service (such as a personalized recommendation service, for example) to the plurality of electronic devices 110, 120, 130, 140 connected via the network 170.

[0032] 2 is a block diagram showing an example of a computer device according to an embodiment of the present invention. Each of the electronic devices 110, 120, 130, and 140 and each of the servers 150 and 160 described above may be realized by a computer device 200 shown in FIG.

[0033] As shown in FIG. 2 , such a computer device 200 may include a memory 210, a processor 220, a communication interface 230, and an input / output interface 240. The memory 210 is a computer-readable storage medium and may include random access memory (RAM), read-only memory (ROM), and a permanent mass storage device such as a disk drive. The permanent mass storage device such as a ROM or a disk drive may be included in the computer device 200 as a separate permanent storage device distinct from the memory 210. The memory 210 may also store an operating system and at least one program code. Such software components may be loaded into the memory 210 from a computer-readable storage medium separate from the memory 210. Such separate computer-readable storage medium may include a computer-readable storage medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, or a memory card. In another embodiment, the software components may be loaded into the memory 210 through a communication interface 230, which is not a computer-readable storage medium. For example, the software components may be loaded into the memory 210 of the computing device 200 based on a computer program installed by a file received over the network 170 .

[0034] Processor 220 may be configured to process computer program instructions by performing basic arithmetic, logic, and input / output operations. The instructions may be provided to processor 220 by memory 210 or by communication interface 230. For example, processor 220 may be configured to execute instructions received according to program code stored in a storage device such as memory 210.

[0035] The communication interface 230 may provide a function for the computer device 200 to communicate with other devices (e.g., the above-mentioned storage device) via the network 170. For example, requests, instructions, data, files, etc. generated by the processor 220 of the computer device 200 in accordance with program code stored in a storage device such as the memory 210 may be transmitted to other devices via the network 170 under the control of the communication interface 230. Conversely, signals, instructions, data, files, etc. from other devices may be received by the computer device 200 via the communication interface 230 of the computer device 200 via the network 170. The signals, instructions, data, etc. received via the communication interface 230 may be transmitted to the processor 220 or the memory 210, and files, etc. may be recorded on a storage medium (e.g., the above-mentioned permanent storage device) that the computer device 200 may further include.

[0036] The input / output interface 240 may be a means for interfacing with the input / output device 250. For example, the input device may include a device such as a microphone, keyboard, or mouse, and the output device may include a device such as a display or speaker. As another example, the input / output interface 240 may be a means for interfacing with a device that integrates input and output functions into one, such as a touch screen. The input / output device 250 may be configured as a single device together with the computer device 200.

[0037] Also, in other embodiments, computing device 200 may include fewer or more components than those shown in Figure 2. However, most prior art components need not be explicitly shown in the figures. For example, computing device 200 may be implemented to include at least some of the input / output devices 250 described above, and may further include other components such as a transceiver, a database, etc.

[0038] Specific embodiments of a method and apparatus for providing an optimal route to achieve a user goal are described below.

[0039] As used herein, the term "path" refers to all user experiences on the Internet that are undertaken to achieve a user goal, and encompasses all search activities (search, exploration, navigation, etc.) on the Internet. That is, the term "path" does not simply refer to search activities, but may include all user experiences consisting of various interconnected services (search service, map service, shopping service, easy payment service, blog service, advertising service, etc.).

[0040] This embodiment can provide an artificial intelligence model that learns based on user behavior patterns to provide the optimal experience for the user, and can provide a model (Optimal Composable Experience Action Network, hereinafter referred to as the "OCEAN model") that easily generates and expands user experiences between services.

[0041] In this embodiment, a platform that is independent of a specific domain or service can be constructed through the OCEAN model. In this specification, the platform refers to an integrated platform including multi-domain and cross-platform for providing a network path where multiple services are linked.

[0042] The computer device 200 according to the present embodiment may provide a personalized recommendation service to a client through a dedicated application installed on the client or through connection to a web / mobile site associated with the computer device 200. A computer-implemented optimal route recommendation system may be configured in the computer device 200. For example, the optimal route recommendation system may be implemented in the form of an independently operating program, or may be implemented in the form of an in-app for a specific application so as to be operable on the specific application.

[0043] The processor 220 of the computer device 200 may be implemented with components for executing the following optimal route recommendation method. Depending on the embodiment, the components of the processor 220 may be selectively included in or excluded from the processor 220. Also, depending on the embodiment, the components of the processor 220 may be separated or combined to represent the functionality of the processor 220.

[0044] Such processor 220 and its components may control computer device 200 to perform the steps involved in the following optimal route recommendation method. For example, processor 220 and its components may be implemented to execute instructions from operating system code and at least one program code contained in memory 210.

[0045] Here, the components of the processor 220 may represent different functions that are performed by the processor 220 according to instructions provided by program code stored on the computer device 200 .

[0046] The processor 220 may read necessary instructions from the memory 210, which has been loaded with instructions related to the control of the computing device 200. In this case, the read instructions may include instructions for controlling the processor 220 to perform the following steps:

[0047] The steps included in the optimal route recommendation method described below may be performed in a different order than shown in the figures, some steps may be omitted, or additional processes may be included.

[0048] FIG. 3 is a flowchart illustrating an example method that may be performed by a computing device in accordance with an embodiment of the present invention.

[0049] 3, in step 310, the processor 220 may collect user historical session data for each user, including all behavioral trajectories in the session, on a session-by-session basis. The processor 220 may track user behavior in each session and collect all of the series of experiences made in the session as one piece of sample data to use as learning data for model learning.

[0050] In step 320, the processor 220 may represent the user recorded session data of each session as a graph path. The processor 220 may represent each of the user recorded session data for each session as at least one path. In this case, each path may be composed of a state, an action, and a reward. The state may represent the user's current situation at each time stage (the content the user is consuming on a screen provided by the service), the action may represent user activity in the current situation, such as clicking, scrolling, going back, or searching again, and the reward may represent the state obtained by the action, i.e., the user's satisfaction with the state-action pair.

[0051] In operation 330, the processor 220 may generate an OCEAN model, which is a model for optimal route prediction, by learning user experiences expressed as graph paths. The processor 220 may connect interoperable services in a network format based on multi-domain or cross-platform. User experiences collected through user logs for services may be expressed as graph paths in the network. In this case, the processor 220 may train the model using path history, actions for each state, and rewards for each path.

[0052] In step 340, the processor 220 may predict the user's next behavior using the OCEAN model for the user record session data collected from the target user's current session and recommend an optimal route on the network. The processor 220 may predict the next action for the target user who is using at least one of the services connected by the network, using the previous path history from the user log in the current session, and recommend the predicted behavior path as the optimal route.

[0053] The OCEAN model of this embodiment directly models user experience and may include user experiences for at least one or more services. The OCEAN model is built using a composable graph path structure, making it highly deterministic and allowing expert knowledge to be expressed as paths on the network. Furthermore, the OCEAN model is a hyper-personalized model that can guide users through a continuous experience to optimally achieve their goals.

[0054] This embodiment learns user recorded session data on a session-by-session basis in which the user defines the purpose, and can automatically find the purpose of the target user from among the multiple learned purposes based on the target user's behavior within the current session.

[0055] 4 and 5 are diagrams illustrating example user experience scenarios in accordance with one embodiment of the present invention.

[0056] Figure 4 illustrates a series of scenarios in which a user searches for dining tables in the search window to achieve a specific goal of "buying a dining table," clicks on a blog post from the search results screen, searches for the product name identified in the blog post, navigates to the product page, and places an order.

[0057] On the other hand, Figure 5 shows an example scenario that goes through more processes to achieve the same goal as Figure 4.

[0058] A user cycles through states and actions to achieve a goal, where a state may be defined as a current user state representing the content of a service screen, and an action may be defined as a user activity such as clicking, scrolling, going back, searching again, etc.

[0059] In this embodiment, interoperable services may be connected in a network configuration. In the network, a user's experience may be represented as a path on the network, and a model may be trained based on the path history, actions for each state, and rewards for each path. The trained model may predict the next user action and link the predicted actions to guide the user to achieve their goal via an optimal route.

[0060] For example, referring to FIG. 6, various services such as a search service, a map service, a shopping service, a simple payment service, a blog service, an advertisement service, etc. may be connected to one network as interoperable services.

[0061] Paths for user experience may be generated for various services, and an OCEAN model 600 may be constructed to predict the optimal route through comprehensive learning of all the paths of services connected in a network form.

[0062] FIG. 7 illustrates an example graph path representing user recorded session data in one embodiment of the present invention.

[0063] Users A, B, and C are all trying to purchase the same product, and although they all have the same goal to achieve, the process (user experience scenario) they take to reach that goal is different. Figure 7 shows the results of a network representation of the three users' experiences in achieving their goal, and as each user goes through a different process, a wide variety of paths are drawn. We can see that User A was unable to purchase the product he wanted, while User C, who had the most specific understanding of the product, was able to successfully purchase the product via the shortest route of the three.

[0064] In this embodiment, an OCEAN model 600 is generated that can predict the optimal route from among multiple routes by learning user experiences composed of graph paths, and this makes it possible to recommend the same route as user C to other users and provide guidance for achieving their goals.

[0065] FIG. 8 shows an example of a path configuration representing a user experience in one embodiment of the present invention.

[0066] Referring to FIG. 8, the network of the OCEAN model 600 may consist of one or more paths 800, each of which may consist of a state 801 at each time stage, an action 802 for the state 801, and a reward 803 for the action 802.

[0067] States 801 may be defined for all screens provided to users in all services that can be integrated, and all activities that users can take for each service (e.g., clicks, searches, browsing, etc.) may be defined as actions 802, and rewards 803 may be designed based on satisfaction with actions 802.

[0068] The state 801 may be defined to include all elements that can determine the user state. This may include not only the screen the user is currently using, but also the type of service (e.g., search, shopping, map, blog, etc.), environmental information (e.g., time, location, weather, season, etc.), personal information of the user (e.g., age, gender, preferences, etc.), and session category (e.g., interior, fashion, car, etc.). This makes it possible to provide a service that recommends personalized optimal routes.

[0069] The state 801 may be classified into service state, environment state, user state, session state, etc. depending on the element, and may be composed of various variables that represent the current situation, such as current query, age, current screen, location, time, etc.

[0070] Action 802 refers to a user action and may include a query (e.g., text, image, voice, etc.) and a gesture (e.g., view, click, scroll, back, favorite, etc.).

[0071] Reward 803 may represent a user's satisfaction with action 802. As one example, reward 803 may be determined by a direct method that receives direct feedback from the user to measure satisfaction with action 802. As another example, reward 803 may be determined by an indirect method that predicts satisfaction with action 802 based on additional actions such as dwell time on a page, purchasing a product, clicking on a page, etc.

[0072] All user logs for one session may be graphed, and the user experience may be expressed as a variety of paths 800. The user experience across all services that can be linked, without depending on a specific service, may be expressed as a path 800 consisting of a state 801, an action 802, and a reward 803. In this way, by modeling the user experience itself, it is possible to build a platform that is not dependent on a specific domain or service.

[0073] All sets of data that are passed through through trial and error to achieve the goal (i.e., including all successful and unsuccessful experiences) may be used as learning data.

[0074] For example, Table 1 shows an example of a user experience connected to a corrective knowledge panel service. In this case, the corrective knowledge panel service is a service that provides users with basic detailed information they desire, such as information about a region or a famous place, so that they can quickly view it. In other words, corrective search refers to a search type in which documents displayed on the search results screen for each query are pre-matched, and the corrective knowledge panel service refers to a service screen that contains corrective search results. User experiences related to such searches include correcting typos and entering queries that are more suitable for more desirable information. [Table 1]

[0075] A state 801, an action 802, and a reward 803 may be defined so that the model can be trained by collecting samples of user experiences as described above. [Table 2]

[0076] The reward 803 may be defined as the user's satisfaction with the action 802 .

[0077] As an example, the reward 803 may be determined based on the impression-to-click rate (CTR), and may be applied as a positive indicator if there are many user clicks on the impression.

[0078] As another example, the reward 803 may be determined based on the probability of clicking on a particular collection, text, point, etc., and may be applied as a positive indicator if the user clicks on similar locations many times, or as a negative indicator if the click locations are scattered.

[0079] As another example, the reward 803 may be determined based on the duration of stay, and may be applied as a positive indicator if the user stays in a certain state for a certain amount of time or more, and as a negative indicator if the user stays in a certain state for less than the certain amount of time.

[0080] Additionally, the reward 803 may be determined based on the difference in dwell time, and may be applied as a positive indicator if the dwell time in the current situation is longer than the previous state, and may be applied as a negative indicator if the dwell time in the current situation is shorter than the previous state.

[0081] As another example, the reward 803 may be determined based on scrolling, and may be applied as a positive indicator if there is little scrolling and the user stays in a specific area for a long time, and as a negative indicator if there is a lot of meaningless scrolling.

[0082] As another example, the reward 803 may be determined based on the action pair count, and may be applied as a positive indicator if the number of times the previous action and the current action appear together is above a certain level, and as a negative indicator if the number of times the previous action and the current action appear together is below a certain level.

[0083] As another example, the reward 803 may be determined based on an action transition probability, and may be applied as a positive indicator if the probability of transitioning from the previous action to the current action is above a certain level, and as a negative indicator if the probability is below the certain level.

[0084] As yet another example, reward 803 may be determined based on the probability that the current action is the last action of the session, and may be applied as a positive indicator if many users end the session in the current situation, and as a negative indicator if many users take other actions in the current situation. [Table 3]

[0085] If the reward for the final state of the session is higher than a certain level, it may be used as a successful experience, and if the reward for the final state of the session is lower than a certain level, it may be used as a failed experience.

[0086] In other words, for training the OCEAN model 600, user recorded session data including all behavioral trajectories in a given session can be used as training data on a session-by-session basis, and the training data can be appropriately processed and used depending on the service situation, model training method, etc. For example, in the case of a search service, a query can be defined as an action 802, and in the case of a shopping service, a click can be defined as an action 802. When language model training is applied, the state 801 and the action 802 can all be expressed in language and utilized.

[0087] The processor 220 may construct the OCEAN model 600 by learning a path 800 that represents the user's current situation at each time stage within a session as a state 801, the user's activities as actions 802, and the user's satisfaction as rewards 803.

[0088] As one example, the processor 220 may generate the OCEAN model 600 through reinforcement learning based on the state 801, the action 802, and the reward 803. As another example, the processor 220 may generate the OCEAN model 600 through training a language model that understands the state 801, the action 802, and the reward 803 themselves as language. As yet another example, the processor 220 may generate the OCEAN model 600 through neural network training on a path 800 of a graph structure made up of the state 801, the action 802, and the reward 803. Depending on the embodiment, the OCEAN model 600 may also be generated by ensemble learning using two or more models.

[0089] The process of model learning based on reinforcement learning is as follows:

[0090] In one definition for reinforcement learning, a Markov decision process (MDP) may be defined as a tuple (S, A, τ, r, γ, μ), where S and A are sets of states and actions. τ maps every s, a ∈ S×A to a conditional distribution τ(·|s, a) over states (S), and r maps any s, a ∈ S×A to r(s, a) ∈ [0, 1]. γ ∈ [0, 1] is a discount factor, and μ is the distribution over states (S).

[0091] Also, θ∈R d A policy parameterized by θ is the conditional distribution π for action (A) θ Define (·|s,h), where s∈S denotes the current situation and h denotes additional information for the condition (e.g., past state-action-reward history).

[0092] Furthermore, given MDP(S,A,τ,r,γ,μ) and policy π θ and the associated value function:

number

number

[0093] The goal of reinforcement learning is to find a policy that maximizes the value function V(θ).

[0094] The processor 220 may indirectly perform policy learning by predicting the maximum reward (Q function) for each state-action pair to extract the optimal policy. In this case, the learning algorithm may be based on temporal difference (TD) learning, and may utilize a model-based method (i.e., knowledge of the reward function and state transition distribution is required) such as value iteration or a model-free method (i.e., knowledge is not required) such as Q learning.

[0095] If the policy is differentiable, a gradient-based method may be used to optimize the value function V(θ). In this embodiment, the flat gradient problem caused by sparse signals can be solved by preconditioning / normalizing the ascending direction (natural gradient, TRPO, and PPO).

[0096] The processor 220 may directly model the sequence of states, actions, and rewards while considering offline reinforcement learning through sequence modeling. The user's behavior trajectory within a session may be configured as a static dataset from which a sequence model can be learned, and the decision transformer for sequence modeling is as shown in Equation (2).

number

[0097] where R t represents the returns-to-go (sum of future rewards) that can be specified at test time.

[0098] On the other hand, the trajectory transformer is as shown in equation (3).

number

[0099] Here, state, action, reward, R t If is discretized, a modified beam search may be used at test time to approximate the trajectory that maximizes the reward.

[0100] The OCEAN model 600 is trained by user experiences, including goals, and therefore contains useful resources for all tasks related to the service without the need for additional fine-tuning of the model, making it possible to personalize all services based on the user experience. However, if necessary, it is also possible to optimize the model service by adjusting the model, such as by fine-tuning.

[0101] Referring to FIG. 9, the processor 220 may represent each user experience that achieves the same goal as a path 800 composed of a state 801, an action 802, and a reward 803, and construct the OCEAN model 600 through reinforcement learning for the path 800.

[0102] There are various paths to achieving a goal, and the processor 220 can find the optimal path while predicting the next action using the OCEAN model 600. Referring to Figure 10, the processor 220 may use the OCEAN model 600 to find and recommend the optimal path to achieving a goal.

[0103] Within a session, the user's past actions and state history sequence (state[0], state[1], ..., state[n-1], action[0], action[1], ..., action[n-1]) that lead to the current situation are input to the OCEAN model 600, and the final state-action pair (state[n] based on action[n]) is the output of the OCEAN model 600. The n in action[n] represents the number of actions the user attempted to achieve a single goal.

[0104] The OCEAN model 600 considers not only the previous log but also all user behavior trajectories within the same session, i.e., all paths (states, actions, rewards) in the previous history, to predict the next action. For example, action[2] can be predicted using state[1], action[1], reward[1], action[3] can be predicted using state[1], state[2], action[1], action[2], reward[1], reward[2], and action[4] can be predicted using state[1], state[2], state[3], action[1], action[2], action[3], reward[1], reward[2], reward[3].

[0105] An embodiment for providing an optimal route using expert knowledge will be specifically described below.

[0106] In this embodiment, one of the services realized by the OCEAN model 600 is a universal advisor (hereinafter referred to as "UA") service.

[0107] UA services may be implemented in a bottom-up approach, where a collection of paths becomes an experience, a collection of experiences becomes an advisor, and a collection of advisors completes the universal advisor.

[0108] UA services can provide personalized information, appearing at key moments throughout the search flow to guide users toward their goals.

[0109] FIG. 11 is a flowchart illustrating an example process for using expert knowledge to provide an optimal route in accordance with an embodiment of the present invention.

[0110] Referring to FIG. 11 , in step 1110, the processor 220 may generate an expert AI model by learning centered on paths based on expert knowledge. For each expert category (e.g., interior design, fashion, cooking, law, etc.), the processor 220 may collect user experiences in which a goal is achieved based on the expert knowledge of the corresponding category. As an example, the processor 220 may track all behavioral trajectories in an expert's session as user experiences by the expert and collect all of the series of experiences performed in the session as a single piece of sample data to use as learning data for model learning. For example, for the goal of purchasing a table, the processor 220 may extract user record session data on the route a user corresponding to an interior design expert took to purchase the table, thereby collecting paths based on expert knowledge. As another example, the processor 220 may collect paths determined as correct answer data while taking into account rules defined by the expert knowledge. For example, to develop expert knowledge for an interior design expert, interior elements (materials, textures, colors, shapes), spatial styles (natural, modern, vintage, Scandinavian, etc.), and space types (living room, kitchen, bedroom, bathroom, entryway, etc.) may be defined. Rules may be defined as criteria for determining excellent interiors, such as the criteria by which the expert would change the space while considering each item. Paths based on the expert knowledge may be collected based on a set of images determined to be excellent interiors according to the interior design expert's rules. The processor 220 may collect paths for each expert category, classify the collected paths into the corresponding categories, and then learn the classified paths to generate an expert AI model for each expert category. Thus, the processor 220 may generate an expert AI model for a UA service by modeling a user experience that reflects the expert knowledge. The processor 220 includes expert experiences when expressing user experiences collected from user logs for the service as paths in a graph format, and therefore the expert experiences can also be expressed as paths on the same network.

[0111] In step 1120, the processor 220 may understand the search intent of the target user based on user record session data collected from the target user's current session. Among the states and actions extracted from the user log data within the session, the final state of the session may be defined as the user's potential goal. The target user's next action may be predicted using a model that learned the behavioral trajectory of sessions in which the goal was successfully achieved, and the target user's potential goal may be understood based on the prediction result from other past user behavior patterns. The processor 220 may determine the moment when a UA service is needed based on the target user's search intent. Once the target user's potential goal is understood, the processor 220 may check whether multiple paths to achieving the goal include an expert path.

[0112] In step 1130, when processor 220 recognizes that a UA service is needed based on the search intent of the target user, processor 220 may call an expert AI model appropriate for the search intent. For example, when processor 220 finds an interior design expert path among multiple past paths that achieved a latent goal recognized as the user's search intent, processor 220 may determine that a UA service is needed and call an interior design expert AI model that has learned the interior design expert path.

[0113] In step 1140, the processor 220 may recommend the path according to the expert AI model as one of the optimal routes for the target user. The processor 220 provides a UA service based on the search intent of the target user, and the UA service may use the expert AI model to guide the user to an expert path corresponding to the user's current situation.

[0114] The processor 220 can predict the target user's next action based on the history of the current session, and thereby grasp the user's potential goal that indicates the user's search intent. The processor 220 can request the user for information necessary to provide the next state based on the path to achieve the user's potential goal, and can provide the next state as the optimal path according to the user's input.

[0115] FIG. 12 is a diagram showing an example of the configuration of a path including expert experience in one embodiment of the present invention.

[0116] To provide UA services, not only general user experiences but also expert experiences are modeled as paths in a graph structure.

[0117] Referring to FIG. 12, the processor 220 may model both a general user experience 1201 of purchasing a light via a general product search path and an expert experience 1202 of purchasing a light via a path defined as expert knowledge as user experiences that achieve the common goal of purchasing a light.

[0118] When the processor 220 identifies "light purchase" as the target user's potential purpose, it may recommend a path based on general user experience 1201 to the target user, or it may recommend a path based on expert experience 1202 via the UA service.

[0119] The processor 220 may model the expert experience 1202 for the UA service to build an expert AI model. Referring to FIG. 13, the expert AI model 1300 may include AI models in various fields such as interior design (space), fashion, cooking, law, etc. The processor 220 may grasp the potential purpose of the target user and provide the UA service using an AI model appropriate for the purpose.

[0120] New paths that cannot be created using the general user experience 1201 can be created through the expert AI model 1300. In this case, the expert experience 1202 may be reflected in the OCEAN model 600 in a form in which a higher reward is applied than the general user experience 1201 during the modeling process.

[0121] The expert AI model 1300 may be constructed as an independent model for each category, and may operate in a manner of selecting and calling the expert AI model 1300 that is suitable for the user's search intent.

[0122] 14 to 17 are diagrams showing examples of UA service scenarios in one embodiment of the present invention.

[0123] Figure 14 shows an example of an interior-related UA service scenario.

[0124] When a user uses a search service with the search term "Scandinavian interior," the processor 220 understands the user's intent to transform their space into a Scandinavian-style interior based on the user's current situation (search term, history in the current session, etc.). If the user's search intent belongs to the expert category, the processor 220 may provide a UA service of the corresponding category. As an example, the processor 220 may determine that a UA service is needed when the user's current situation falls into the expert category.

[0125] The interior expert AI model is a model of the experience based on the knowledge of interior experts. A UA service based on the interior expert AI model may request an image of an interior space from a user, extract information about the interior space from the input image and detect objects within the space (understanding), analyze the interior space information of the input image and the object detection results by applying rules defined by the knowledge of interior experts (reasoning), and determine a recommendation solution appropriate for the analysis results (decision making), thereby providing recommendations based on expert knowledge.

[0126] Referring to FIG. 14, when the processor 220 determines that the user's search intent indicates that UA services from an interior design expert are needed, the processor 220 may display a UA interface 1401 on a search result screen 1400 corresponding to the search term "Scandinavian interior."

[0127] The processor 220 may request an image input related to an interior space from the user in order to discover and execute a path optimized for the user's search intent through the OCEAN model 600. When the UA interface 1401 is selected on the search result screen 1400, the processor 220 may provide an image input interface 1410 to request image input. The processor 220 may add the image input into the image input interface 1410 as the user's current situation, and may recommend interior results 1420 suitable for the space of the input image on the search result screen 1400 as a path corresponding to the user's state.

[0128] FIG. 15 shows an example of a UA service scenario related to shopping.

[0129] If a user enters the search term "mushroom light" with the intent to purchase a particular product but without knowing any specific information about the product, the user's intent to purchase the light may be understood based on the search term and the history of the current session.

[0130] Referring to FIG. 15, when the processor 220 determines that the user's search intent indicates that UA services from a shopping expert are needed, the processor 220 may display a UA interface 1501 on a search result screen 1500 corresponding to the search term "mushroom-shaped light."

[0131] The user experience for purchasing a light may be modeled to include not only general user experiences but also expert experiences. The processor 220 may request image input for a product to be purchased in order to discover and execute an expert path through the OCEAN model 600. When the UA interface 1501 is selected on the search result screen 1500, the processor 220 may provide an image input interface 1510 to request image input. The processor 220 may add the image input into the image input interface 1510 as the user's current situation and recommend shopping information (e.g., product details, product homepage, product purchase page, retailer information, etc.) 1520 for the product in the input image on the search result screen 1500 as a path corresponding to the user's state.

[0132] The processor 220 can eliminate unnecessary steps through the UA service and easily and quickly suggest the optimal route to achieve the user's goal, without the user having to go through multiple processes to search for product information.

[0133] The processor 220 can understand the user intent based on the user's current context, such as search terms and session history, and suggest cross-platform results according to the user intent. For example, if a user spills coffee on their clothes and enters the search term "how to remove coffee stains," the processor 220 can understand the user's potential goal of removing coffee stains and link a search service that provides text containing know-how on removing coffee stains, a shopping service for purchasing detergent for removing coffee stains, and a location service that provides a list of dry cleaners near the user's current location. In other words, the UA service can actively guide the user to achieve their goal by moving between multiple platforms, such as search services, shopping services, and location services.

[0134] As a UA service, processor 220 can also perform common sense reasoning: if a user uploads an image of a coffee-stained shirt instead of a search term, common sense reasoning from image analysis may identify the underlying goal of removing the stain and suggest a path to achieve this goal.

[0135] In the OCEAN model 600, user experience is modeled as a path in graph form. For example, it is possible to learn from search history data how users arrived at the final result for the search term "when I have a headache," and the final result selected by many users ultimately becomes common sense. By learning such user experience, when the search term "when I have a headache" is entered, it is possible to predict the final result based on the user's common sense. Numerous common sense examples such as "take a headache medicine when you have a headache," "eat when you are hungry," and "use an umbrella when it rains" can be reflected in the OCEAN model 600, and a UA service using the OCEAN model 600 can provide an optimized experience through common sense inference.

[0136] The processor 220 can also proactively provide notifications to users across multiple domains and platforms when they were unable to complete a goal in a previous session. For example, if a user was unable to purchase a desired product in a shopping service because it was out of stock, the UA service can monitor the status of the product and notify the user when the product is back in stock.

[0137] There are several criteria for determining whether a user has achieved their goal. For example, if a user purchases a product after searching, or if the user moves to another page after searching and stays on the page for a certain period of time, the user may be deemed to have achieved their goal.

[0138] In the process of building the OCEAN model 600, the final state in the user behavior history data (e.g., whether many users ended their session on the product search page or after purchasing a product) may be probabilistically learned. In other words, for each state, the probability that it is the final state in which the goal is achieved may be learned.

[0139] If the user ends the session in a state that is not highly likely to be the final state, the goal may be considered unsuccessful. The user's action sequence and current state may be continuously monitored to determine whether the user has achieved the goal.

[0140] Personalization is one of the key elements in guiding users along the optimal path to achieve their goals. How well a service can understand the user's state plays a key role in determining recommendations.

[0141] 16 and 17 show an example of a UA service scenario that takes into account the user state.

[0142] The processor 220 may recognize the user's current situation through the UA service and provide additional information. Figure 16 shows a product details screen 1600. If a user of Korean nationality currently residing in Korea is viewing a product whose shoe size is displayed in American standards, the processor 220 may determine that a UA service is needed and display a UA interface 1601 on the product details screen 1600.

[0143] If the user resides in the United States or has American nationality, the display of the UA interface 1601 on the product details screen 1600 is omitted.

[0144] When the UA interface 1601 is selected on the product details screen 1600, the processor 220 may provide Korean size standards or recommend sizes suitable for the user's personal information as additional information 1610 corresponding to the user's current situation on the product details screen 1600.

[0145] The more personal information, such as the user's shoe size, favorite color, favorite brand, etc., that is stored in the OCEAN model 600 as user status, the more optimal the path the processor 220 can provide UA services.

[0146] 17 shows a shopping service screen 1700. If the user has recently purchased a sofa and there has been an increase in searches for interior design-related keywords such as interior, lighting, rugs, etc., the user status may be defined based on such history. As the user uses the shopping service screen 1700, a UA interface 1701 including additional information related to the user status may be provided on the shopping service screen 1700 using an interior design expert AI model.

[0147] On the other hand, for a user who needs to lose weight and is searching for training wear for the purpose of exercise, the user status may be defined based on recent search terms, preferred brands, seasonal information, etc., and for such a user, related information may be recommended using at least one of a fashion expert AI model and a food expert AI model.

[0148] In other words, the processor 220 may define the user's current situation based on the user's behavioral history during the current session or a recent period of time, the user's environmental information (e.g., time, location, weather, season, etc.), and the user's personal information (e.g., age, gender, preferences, etc.), and may use this to guide a personalized path.

[0149] Therefore, the processor 220 can utilize expert AI models from various domains to provide more accurate and useful recommendations, and can recommend different paths for each user by understanding the user's state and providing information necessary for the current situation.

[0150] Conventionally, recommendation information has been provided based on rules defined by the service provider, but the UA service using the OCEAN model 600 of this embodiment can provide personalized recommendation information according to the user state defined based on the user's history through an AI model.

[0151] In other words, UA services using the OCEAN model 600 can predict goals based on user experience and guide optimal routes, providing results based on a wider variety of variables. By directly modeling not only general user experiences but also experiences based on expert knowledge, it is possible to respond to all situations, and by expanding the path, it is possible to connect with any number of new services.

[0152] As described above, according to an embodiment of the present invention, an artificial intelligence model can be used to find patterns in user behavior, eliminate unnecessary steps, and recommend an optimal route to the user's goal. In particular, according to an embodiment of the present invention, by representing and modeling the user experience as a graph path, a hyper-personalized model can be built to predict the optimal route, making it possible to build a platform that is independent of specific domains or services. Furthermore, according to an embodiment of the present invention, by recommending a route represented by expert knowledge as a route suited to the user intent, it is possible to guide the user through an optimal user experience to reach their goal more quickly.

[0153] The above-described devices may be realized using hardware components, software components, and / or a combination of hardware and software components. For example, the devices and components described in the embodiments may be realized using one or more general-purpose or special-purpose computers, such as a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or various devices capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the OS. The processing device may also access, record, manipulate, process, and generate data in response to the execution of the software. For convenience of understanding, a single processing device may be described, but those skilled in the art will understand that the processing device may include multiple processing elements and / or multiple types of processing elements. For example, the processing device may include multiple processors or one processor and one controller. Other processing configurations, such as parallel processors, are also possible.

[0154] Software may include a computer program, code, instructions, or a combination of one or more of these, and may configure a processing device to operate as desired or instruct the processing device, either individually or collectively. The software and / or data may be embodied in any type of machine, component, physical device, computer storage medium, or device to be interpreted by or to provide instructions or data to a processing device. The software may be distributed across computer systems connected by a network, and may be stored and executed in a distributed manner. The software and data may be stored on one or more computer-readable storage media.

[0155] Methods according to embodiments may be implemented in the form of program instructions executable by various computer means and recorded on a computer-readable medium. In this case, the medium may continuously record a computer-executable program or may temporarily record the program for execution or download. The medium may be various recording or storage means in the form of a single piece of hardware or multiple pieces of hardware combined together. It may be a medium directly connected to a computer system or distributed over a network. Examples of media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and media configured to record program instructions, such as ROMs, RAMs, and flash memories. Other examples of media include recording media or storage media managed by app stores that distribute applications, or by websites or servers that provide or distribute various software.

[0156] Although the embodiments have been described above based on limited examples and drawings, those skilled in the art will appreciate that various modifications and variations may be made in light of the above description. For example, the described techniques may be performed in an order different from that described, and / or the described system, structure, device, circuit, or other element may be coupled or combined in a manner different from that described, or may be substituted or replaced by other elements or equivalents, and still achieve suitable results.

[0157] Therefore, different embodiments are within the scope of the appended claims, provided that they are equivalent to the claims.

Claims

1. 1. A method implemented on a computer device, comprising: the computing device includes at least one processor configured to execute computer-readable instructions contained in a memory; The method comprises: predicting a next action of the target user through an optimal path prediction model learned by a graph-type path representing a user behavior trajectory in a session unit by the at least one processor, and recommending the predicted action path as an optimal path, wherein the graph-type path includes a general path based on the user experience of a general user and an expert path based on the user experience or expert knowledge of an expert; Including, The recommending step includes: understanding a user intent based on a previous behavioral trajectory of the target user in a current session; and determining a path corresponding to the user intent as one of the optimal paths; Including, The method, wherein the determining step includes, if a path for achieving the understood user intent includes at least one expert path, invoking an expert AI model corresponding to the user intent and using the expert AI model to determine a path based on expert knowledge.

2. The determining step comprises: Calling an expert AI model corresponding to the user intent from among a plurality of expert AI models trained by paths determined as correct data in consideration of rules defined in expert knowledge, and determining a path based on expert knowledge. The method of claim 1 ,

3. The optimal route prediction model is For each expert category, an expert AI model is included that has learned a path based on the expert knowledge of the corresponding category, The determining step comprises: Determining a path based on the expert knowledge through an expert AI model of a category appropriate for the user intent. The method of claim 1 ,

4. The determining step comprises: Displaying an interface for guiding the path based on the expert knowledge on a service screen of the current session. The method of claim 1 , comprising:

5. The determining step comprises: soliciting user input associated with the user intent via the interface for guiding the expert knowledge-based path; The method of claim 4 further comprising:

6. The determining step comprises: providing additional information corresponding to the user input as the expert knowledge-based path; The method of claim 5 further comprising:

7. The determining step comprises: Defining a current situation of the target user and guiding a personalized path based on the current situation. The method of claim 1 , comprising:

8. The current situation of the target user is defined by at least one of the target user's current session or user behavior history within a recent certain period, user environment information, and user personal information. The method of claim 7, characterized in that

9. A computer program recorded on a computer-readable recording medium for causing a computer device to execute the method according to any one of claims 1 to 8.

10. 1. A computer device comprising: at least one processor configured to execute computer-readable instructions contained in the memory; Including, The at least one processor A process of predicting the target user's next action through an optimal path prediction model learned from a graph-type path that represents the user's behavioral trajectory for each session, and recommending the predicted action path as the optimal path. Process The path in the form of a graph includes a general path based on the user experience of a general user and an expert path based on the user experience or expert knowledge of an expert, The at least one processor Understanding user intent based on a previous behavioral trajectory of the target user in a current session; determining a path corresponding to the user intent as one of the optimal routes; The determining step includes, when a path for achieving the understood user intent includes at least one expert path, invoking an expert AI model corresponding to the user intent and using the expert AI model to determine a path based on expert knowledge.

Images