Vehicle and method thereof for providing personalized recommendation service

Abstract

Disclosed is a vehicle providing a personalized recommendation service. The vehicle according to one embodiment of the present invention comprises: a microphone for receiving a voice command of a first user; an output interface; and a processor which acquires a first user embedding vector based on a first behavior data set corresponding to a first behavior area of the first user, acquires a second user embedding vector corresponding to a second behavior area of a second user having at least a predetermined similarity to the first user embedding vector on the basis of an analysis result of the voice command, acquires one or more attribute elements having at least a predetermined attribute score among a plurality of attribute elements matched to the acquired second user embedding vector, and outputs a recommendation service based on the acquired one or more attribute elements and the analysis result through the output interface.

Description

How to provide vehicles and their personalized recommendation services

[0001] The present disclosure relates to a vehicle, and more specifically, to a vehicle capable of providing personalized recommendation services based on various data about a user.

[0002] Existing in-vehicle recommendation systems rely solely on limited data collected within the vehicle (e.g., driving routes, usage history of vehicle functions). This results in limitations, as they fail to adequately grasp the user's overall lifestyle patterns, interests, and external activities.

[0003] For example, it is difficult for the in-vehicle system to access information such as a user's favorite places at home or the office, web surfing history, and shopping patterns.

[0004] Therefore, in-vehicle recommendations are limited to merely past driving records or patterns of visiting specific locations, showing limitations in predicting the user's current situation or potential preferences. This fails to meet the user's true needs, causing a decrease in recommendation accuracy and satisfaction.

[0005] Furthermore, existing recommendation systems use a method of directly matching data by identifying the user IDs of each system when combining in-vehicle and in-home data. Since unique user identification information is utilized in this process, there is a risk that personal information will be exposed. If this data is not pseudonymized or is processed in an environment with weak security, user privacy could be severely violated.

[0006] The purpose of the present disclosure is to provide a personalized recommendation service that reflects the user's overall behavioral patterns.

[0007] The purpose of the present disclosure is to provide a personalized recommendation service by reflecting not only in-vehicle data but also the user's behavioral patterns within the home.

[0008] The purpose of the present disclosure may be to improve recommendation accuracy for other areas of user behavior.

[0009] The purpose of the present disclosure may be to enable personalized recommendations while protecting privacy through embedding vector-based data processing without combining user identifiers (IDs).

[0010] A vehicle according to one embodiment of the present disclosure may include: a microphone for receiving a voice command from a first user; an output interface; and a processor for obtaining a first user embedding vector based on a first behavioral data set corresponding to a first behavioral area of ​​a first user, obtaining a second user embedding vector corresponding to a second behavioral area of ​​a second user having a certain degree of similarity or higher to the first user embedding vector based on the analysis result of the voice command, obtaining one or more attribute elements having a certain attribute score or higher among a plurality of attribute elements matched to the obtained second user embedding vector, and outputting a recommendation service based on the obtained one or more attribute elements and the analysis result through the output interface.

[0011] In a non-transient recording medium storing computer-readable instructions that cause the device to perform operations when executed by a device according to one embodiment of the present disclosure, the operations may include: a step of obtaining a first user embedding vector based on a first behavioral data set corresponding to a first behavioral area of ​​a first user; a step of receiving a voice command of the first user; a step of obtaining a second user embedding vector corresponding to a second behavioral area of ​​a second user having a certain similarity or greater to the first user embedding vector based on an analysis result of the received voice command; a step of obtaining one or more attribute elements having a certain attribute score or greater among a plurality of attribute elements matched to the obtained second user embedding vector; and a step of outputting a recommendation service based on the obtained one or more attribute elements and the analysis result.

[0012] According to an embodiment of the present disclosure, the user can receive accurate and rich recommendation services.

[0013] According to an embodiment of the present disclosure, user satisfaction can be improved by increasing the level of personalization of the service while protecting the user's privacy.

[0014] According to an embodiment of the present disclosure, recommendation accuracy in other user behavior domains can be dramatically improved, and it can be of great help in solving data sparsity and cold start problems.

[0015] According to an embodiment of the present disclosure, user satisfaction with recommendations can be increased by continuously improving the recommendation algorithm through the collection of user feedback.

[0016] FIG. 1 is a block diagram for explaining the components of an artificial intelligence device according to one embodiment of the present disclosure.

[0017] FIG. 2 is a drawing for explaining the configuration of an artificial intelligence server according to one embodiment of the present disclosure.

[0018] FIG. 3 is a drawing illustrating the process of deriving customer attributes according to one embodiment of the present disclosure.

[0019] FIG. 4 is a flowchart illustrating a method for obtaining user attributes of a device according to one embodiment of the present disclosure.

[0020] FIGS. 5a to 5c are drawings illustrating a process of extracting user behavior information based on a self-search method according to an embodiment of the present disclosure.

[0021] FIGS. 6a and 6b are drawings illustrating a process of extracting a user embedding vector based on LBM according to an embodiment of the present disclosure.

[0022] FIGS. 7a and 7b are drawings illustrating the process of extracting user embedding vectors based on a behavior indicator map according to an embodiment of the present disclosure.

[0023] FIGS. 8a to 8d are drawings showing behavior indicator maps and clustered results based on the behavior indicator maps according to another embodiment of the present disclosure.

[0024] FIG. 9 is a drawing showing a plurality of attributes that a user may have according to one embodiment of the present disclosure.

[0025] FIGS. 10a and FIGS. 10b are drawings illustrating a process of deriving user attribute information from a user embedding vector according to one embodiment of the present disclosure.

[0026] FIGS. 11a and FIGS. 11b are drawings illustrating a process of deriving user attribute information from a user embedding vector according to another embodiment of the present disclosure.

[0027] FIG. 12 is a diagram illustrating a process of deriving user attribute information based on a user embedding vector according to another embodiment of the present disclosure.

[0028] FIG. 13 is a flowchart illustrating a personalized recommendation service method of a device according to one embodiment of the present disclosure.

[0029] FIGS. 14 to 16d are drawings for explaining a method for providing a personalized recommendation service for a vehicle according to one embodiment of the present disclosure.

[0030] FIG. 17 is a diagram showing user behaviors categorized by clustering user embedding vectors according to one example of the present invention.

[0031] FIG. 18 is a drawing for explaining the architecture of a system according to one embodiment of the present disclosure.

[0032] Artificial intelligence refers to the field of researching artificial intelligence or the methodologies to create it, while machine learning refers to the field of researching methodologies to define the various problems addressed in the field of artificial intelligence and to solve them.

[0033] Machine learning is also defined as an algorithm that improves the performance of a task through consistent experience.

[0034] An Artificial Neural Network (ANN) is a model used in machine learning that can refer to a model capable of problem-solving, composed of artificial neurons (nodes) that form a network through the connection of synapses.

[0035] An artificial neural network can be defined by connection patterns between neurons in different layers, a learning process that updates model parameters, and an activation function that generates output values.

[0036] An artificial neural network may include an input layer, an output layer, and optionally one or more hidden layers. Each layer may include one or more neurons, and the artificial neural network may include synapses connecting the neurons. In an artificial neural network, each neuron may output a function value of an activation function for input signals, weights, and biases input through the synapses.

[0037] Model parameters refer to parameters determined through learning, including synaptic connection weights and neuron biases. Hyperparameters refer to parameters that must be set before training in a machine learning algorithm, including the learning rate, number of iterations, mini-batch size, and initialization function.

[0038] The objective of training an artificial neural network can be viewed as determining model parameters that minimize the loss function. The loss function can be used as an indicator to determine optimal model parameters during the training process of an artificial neural network.

[0039] Machine learning can be classified into supervised learning, unsupervised learning, and reinforcement learning depending on the learning method.

[0040] Supervised learning refers to a method of training an artificial neural network with labels provided for the training data; a label can refer to the correct answer (or result value) that the artificial neural network must infer when training data is input into it.

[0041] Unsupervised learning can refer to a method of training an artificial neural network without being given labels for the training data.

[0042] Reinforcement learning can refer to a learning method that trains an agent defined within an environment to select an action or sequence of actions that maximizes the cumulative reward in each state.

[0043] Machine learning implemented as a deep neural network (DNN) containing multiple hidden layers among artificial neural networks is also called deep learning, and deep learning is a part of machine learning.

[0044] In the following, machine learning is used to include deep learning.

[0045] FIG. 1 is a block diagram for explaining the components of an artificial intelligence device according to one embodiment of the present disclosure.

[0046] The artificial intelligence device (100) can be implemented as a stationary device or a mobile device, such as a TV, projector, mobile phone, smartphone, laptop, digital broadcasting terminal, PDA (personal digital assistants), PMP (portable multimedia player), navigation, tablet PC, wearable device, set-top box (STB), DMB receiver, radio, washing machine, refrigerator, desktop computer, digital signage, robot, vehicle, etc.

[0047] Referring to FIG. 1, the artificial intelligence device (100) may include a communication interface (110), an input interface (120), a learning processor (130), a sensor (140), an output interface (150), a memory (170), and a processor (180).

[0048] The communication interface (110) can transmit and receive data with external devices, such as other artificial intelligence devices or AI servers (200), using wired or wireless communication technology. For example, the communication interface (110) can transmit and receive sensor information, user input, learning models, control signals, etc., with external devices.

[0049] The communication technologies used by the communication interface (110) include GSM (Global System for Mobile communication), CDMA (Code Division Multi Access), LTE (Long Term Evolution), 5G, WLAN (Wireless LAN), Wi-Fi (Wireless-Fidelity), Bluetooth, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), ZigBee, NFC (Near Field Communication), etc.

[0050] The input interface (120) can acquire various types of data.

[0051] The input interface (120) may include a camera (121) for capturing images, a microphone (122) for receiving audio signals, and a user input interface (123) for receiving information from a user.

[0052] A camera (121) or a microphone (122) can be treated as a sensor, and a signal obtained from the camera (121) or the microphone (122) can be named as sensing data or sensor information.

[0053] The input interface (120) can obtain input data to be used when obtaining an output using training data and a training model for model training. The input interface (120) may also obtain unprocessed input data, in which case the processor (180) or the learning processor (130) can extract input features as a preprocessing step for the input data.

[0054] The camera (121) processes image frames, such as still images or video, obtained by an image sensor in video call mode or shooting mode. The processed image frames may be displayed on a display (151) or stored in memory (170).

[0055] The microphone (122) processes external acoustic signals into electrical voice data. The processed voice data can be utilized in various ways depending on the function (or application running) being performed by the artificial intelligence device (100). Meanwhile, various noise removal algorithms can be applied to the microphone (122) to remove noise generated during the process of receiving external acoustic signals.

[0056] The user input interface (123) is for receiving information from a user, and when information is input through the user input interface (123), the processor (180) can control the operation of the artificial intelligence device (100) to correspond to the input information.

[0057] The user input interface (123) may include mechanical input means (or mechanical keys, such as buttons, dome switches, jog wheels, jog switches, etc. located on the front / rear or side of the artificial intelligence device (100)) and touch input means.

[0058] As an example, a touch input means may consist of a virtual key, soft key, or visual key displayed on a touchscreen through software processing, or a touch key placed on a part other than the touchscreen.

[0059] The learning processor (130) can train a model composed of an artificial neural network using training data. The trained artificial neural network can be called a learning model. The learning model can be used to infer a result value for new input data other than the training data, and the inferred value can be used as a basis for judgment to perform some action.

[0060] The learning processor (130) can perform AI processing together with the learning processor (240) of the AI ​​server (200).

[0061] The learning processor (130) may include memory integrated into or implemented in the artificial intelligence device (100). The learning processor (130) may also be implemented using memory (170), external memory directly coupled to the artificial intelligence device (100), or memory maintained in an external device.

[0062] The sensor (140) can acquire at least one of internal information of the artificial intelligence device (100), surrounding environment information of the artificial intelligence device (100), and user information using various sensors.

[0063] The sensor (140) may include one or more of a proximity sensor, an illuminance sensor, an accelerometer, a magnetic sensor, a gyroscope, an inertial sensor, an RGB sensor, an IR sensor, a fingerprint recognition sensor, an ultrasonic sensor, a light sensor, a microphone, a lidar sensor, and a radar sensor.

[0064] The output interface (150) can generate output related to visual, auditory, or tactile senses.

[0065] The output interface (150) may include a display (151) for outputting an image, an audio output interface (152) for outputting audio, a haptic device (153) for outputting tactile information, and a light output interface (154) for outputting light.

[0066] The display (151) displays (outputs) information processed by the artificial intelligence device (100). For example, the display (151) can display information on the execution screen of an application running on the artificial intelligence device (100), or UI (User Interface) and GUI (Graphic User Interface) information based on such execution screen information.

[0067] The display (151) can be implemented as a touch screen by forming a layered structure with the touch sensor or by being formed as an integral unit. The touch screen functions as a user input interface (123) that provides an input interface between the artificial intelligence device (100) and the user, and at the same time, can provide an output interface between the artificial intelligence device (100) and the user.

[0068] The audio output interface (152) can output audio data received from the communication interface (110) or stored in the memory (170) in call signal reception, call mode or recording mode, voice recognition mode, broadcast reception mode, etc.

[0069] The audio output interface (152) may include at least one of a receiver, a speaker, and a buzzer.

[0070] The haptic device (153) generates various tactile effects that the user can feel. A typical example of the tactile effect generated by the haptic device (153) can be vibration.

[0071] The light output interface (154) outputs a signal to indicate the occurrence of an event using the light of the light source of the artificial intelligence device (100). Examples of events occurring in the artificial intelligence device (100) may include receiving a message, receiving a call signal, a missed call, an alarm, a schedule notification, receiving an email, receiving information through an application, etc.

[0072] The memory (170) can store data that supports various functions of the artificial intelligence device (100). For example, the memory (170) can store input data, training data, training models, training history, etc. obtained from the input interface (120).

[0073] The processor (180) can determine at least one executable operation of the artificial intelligence device (100) based on information determined or generated using a data analysis algorithm or a machine learning algorithm.

[0074] The processor (180) can control the components of the artificial intelligence device (100) to perform a determined operation.

[0075] To this end, the processor (180) can request, search, receive, or utilize data from the learning processor (130) or memory (170), and can control the components of the artificial intelligence device (100) to execute a predicted operation or a preferred operation among the at least one executable operation.

[0076] If the processor (180) requires the connection of an external device to perform a determined operation, it can generate a control signal to control the external device and transmit the generated control signal to the external device.

[0077] The processor (180) can obtain intent information regarding user input and determine the user's requirements based on the obtained intent information.

[0078] The processor (180) can obtain intent information corresponding to user input by using at least one of a Speech To Text (STT) engine for converting voice input into a string or a Natural Language Processing (NLP) engine for obtaining intent information of natural language.

[0079] At least one of the STT engine or NLP engine may be composed of an artificial neural network, at least a portion of which is trained according to a machine learning algorithm. Additionally, at least one of the STT engine or NLP engine may be trained by a learning processor (130), trained by a learning processor (240) of an AI server (200), or trained by distributed processing thereof.

[0080] The processor (180) can collect history information, including the operation details of the artificial intelligence device (100) or user feedback regarding the operation, and store it in memory (170) or a learning processor (130), or transmit it to an external device such as an AI server (200). The collected history information can be used to update the learning model.

[0081] The processor (180) can control at least some of the components of the artificial intelligence device (100) to run an application stored in memory (170).

[0082] The processor (180) can operate two or more of the components included in the artificial intelligence device (100) in combination with each other to operate the application.

[0083] FIG. 2 is a drawing for explaining the configuration of an artificial intelligence server according to one embodiment of the present disclosure.

[0084] Referring to FIG. 2, the AI ​​server (200) may refer to a device that trains an artificial neural network using a machine learning algorithm or uses a trained artificial neural network.

[0085] The AI ​​server (200) may be composed of multiple servers to perform distributed processing and may be defined as a 5G network device. The AI ​​server (200) may be included as part of the configuration of the artificial intelligence device (100) to perform at least some of the AI ​​processing together.

[0086] The AI ​​server (200) may include a communication interface (210), memory (230), a learning processor (240), and a processor (260).

[0087] The communication interface (210) can transmit and receive data with an external device, such as an artificial intelligence device (100).

[0088] The memory (230) may include a model memory (231). The model memory (231) may store a model (or artificial neural network, 231a) that is being learned or has been learned through the learning processor (240).

[0089] The learning processor (240) can train the artificial neural network (231a) using training data. The training model may be used while mounted on the AI ​​server (200) of the artificial neural network, or it may be used while mounted on an external device such as an artificial intelligence device (100).

[0090] The learning model may be implemented in hardware, software, or a combination of hardware and software. If part or all of the learning model is implemented in software, one or more instructions constituting the learning model may be stored in memory (230).

[0091] The processor (260) can use a learning model to infer a result value for new input data and generate a response or control command based on the inferred result value.

[0092] In the following, the artificial intelligence device (100) may be referred to as the device (100).

[0093] One or more processors (180) may be provided.

[0094] In the following, customers may be referred to as users.

[0095] The process of deriving customer attributes of FIG. 3 and the method of obtaining user attributes of FIG. 4 described below are described as being performed by the processor (180) of the device (100) shown in FIG. 1, but may also be performed by the processor (260) of the AI ​​server (200) of FIG. 2.

[0096] FIG. 3 is a drawing illustrating the process of deriving customer attributes according to one embodiment of the present disclosure.

[0097] The device (100) can collect a user data set (310). The user data set (310) may include multiple behavior data sets (311, 312, 313, 314, 315) collected from multiple behavior domains.

[0098] A behavioral domain may be a unit of data collection and analysis that classifies a user's behavior across various devices, services, or physical environments by type.

[0099] Multiple behavioral domains may include a management domain of home appliances, a usage domain of home appliances, a content usage domain, a movement domain related to the user's location movement, and a usage domain of the user's smartphone.

[0100] Each of the multiple behavioral data sets may include source data representing actions performed by the user in each behavioral domain.

[0101] The device (100) can extract user behavior information (320) based on the collected user data set (310).

[0102] The device (100) can obtain a sequence of actions (331) based on the extracted user's action information (320).

[0103] The device (100) can obtain a user embedding vector (330) representing customer characteristics from a behavior sequence (331) through an LBM (332).

[0104] The device (100) can store behavior indicators generated based on extracted user behavior information (320) in a behavior indicator DB (341). The device (100) can generate a behavior indicator map (342) based on the behavior indicators and can obtain a user embedding vector (340) from the behavior indicator map (342) through an encoder (343).

[0105] The device (100) can store the acquired user embedding vectors (330, 340) in the customer characteristic DB (350). The customer characteristic DB (350) can store multiple user embedding vectors corresponding to each of the multiple customers. The customer characteristic DB (350) can be loaded into the memory (170) of FIG. 1 or the memory (230) of FIG. 2.

[0106] The device (100) can obtain customer attribute information from user embedding vectors (330, 340) through attribute extraction models (361, 362) and can store the obtained attribute information in a customer attribute DB (370). The customer attribute DB (370) can store multiple customer attributes corresponding to each of multiple customers. The customer attribute DB (370) can be loaded into the memory (170) of FIG. 1 or the memory (230) of FIG. 2.

[0107] The conversational agent (380) can receive a question from an administrator and can obtain customer attribute information and customer behavior information based on the interpretation result of the received question. The conversational agent (380) can obtain an analysis result for the question from the attribute information and behavior information through LLM. The analysis result may include one or more of the customer attributes, interest categories, or behavior patterns specified in the question.

[0108] Various embodiments of the present disclosure will be described below based on FIG. 3.

[0109] FIG. 4 is a flowchart illustrating a method for obtaining user attributes of a device according to one embodiment of the present disclosure.

[0110] The processor (180) of the device (100) can collect user data sets (S401).

[0111] In one embodiment, the user data set (310) may include a plurality of behavior data sets (311 to 315) corresponding to each of a plurality of behavior domains of one or more users. The processor (180) may collect source data containing customer behavior information for each of the plurality of behavior domains.

[0112] The user data set (310) may include data representing behavioral information that reflects the user's intention and user type.

[0113] Multiple behavioral areas may include two or more areas among a management area of ​​a home appliance, a usage area of ​​a home appliance, a content usage area, a movement area related to the user's location movement, or a usage area of ​​the user's smartphone.

[0114] The management area of ​​the home appliance may be an area representing actions performed for the management of the home appliance. The first action data set (311) corresponding to the management area of ​​the home appliance may include data corresponding to each of the actions related to the management of the home appliance, such as purchasing, repairing, consulting, and accessing a website.

[0115] The usage area of ​​the home appliance may be an area representing behaviors related to the use of the home appliance. The second behavior data set (312) corresponding to the usage area of ​​the home appliance may include data corresponding to each of the function execution behavior and operation setting behavior of the home appliance.

[0116] The content usage area may be an area representing the user's behavior of using or consuming content. The third behavior data set (313) corresponding to the content usage area may include data corresponding to each of the behaviors of viewing content, searching for content, downloading content, and deleting content.

[0117] The user's movement area may be an area representing actions performed when the user moves their location. The fourth action data set (314) corresponding to the user's movement area may include data corresponding to the act of setting a destination via navigation and the act of setting a favorite place via navigation, respectively.

[0118] The usage area of ​​the smartphone may be an area representing actions of the user operating or using the smartphone. The fifth behavior data set (315) corresponding to the usage area of ​​the smartphone may include data corresponding to each of the actions of installing an application installed on the smartphone, executing an application, updating an application, and deleting an application.

[0119] In one embodiment, the processor (180) may receive a user behavior data set from a server (200) or an external device through a communication interface (110). The external device may be either a user's home appliance or a smartphone.

[0120] The processor (180) can collect multiple user data sets corresponding to each of the multiple users.

[0121] The processor (180) can extract user behavior information based on the collected user data set (S403).

[0122] Behavioral information may include records regarding specific actions actually performed by the user. Behavioral information may include at least one of a plurality of unit actions or a plurality of behavioral indicators corresponding to each of the plurality of unit actions. A unit action may represent a minimum unit of action that is mutually distinguishable, such as "washing machine repair" or "air conditioner purchase." A behavioral indicator may be an indicator for relatively comparing the importance between unit actions.

[0123] The processor (180) can extract multiple behavior information from each of the multiple behavior data sets (311 to 315).

[0124] In one embodiment, the processor (180) can extract user behavior information from the user data set (310) based on a self-discovery method.

[0125] The self-exploration method may be a method of automatically understanding the schema of a user data set (310) through LLM and generating a SQL (Structured Query Language) query (or query statement) to extract user behavior information. The self-exploration method may be a method of applying or executing an SQL query statement to a user data set to extract user behavior information.

[0126] FIGS. 5a to 5c are drawings illustrating a process of extracting user behavior information based on a self-search method according to an embodiment of the present disclosure.

[0127] Referring to FIG. 5a, the first behavior data set (311) may include a plurality of DB tables (501 to 504). Each DB table may include information corresponding to a column name.

[0128] The first DB table (501) may be a table containing main information about customers. The first DB table (501) may include a customer ID item for identifying a customer, a product ID item for identifying a product, and a purchase ID item for identifying the purchase of a product.

[0129] The second DB table (502) may be a table containing personal information of customers. The second DB table (502) may include a customer ID item identifying a customer, a name item identifying a customer's name, and an address item identifying a customer's address.

[0130] The third DB table (503) may be a table containing information about products. The third DB table (503) may include a product ID item identifying a product, a product family item indicating a product family, a model name item indicating a product model name, and a release date item indicating a product release date.

[0131] The fourth DB table (504) may be a table containing product purchase information. The fourth DB table (504) may include a customer ID item identifying the purchase, a place of purchase item indicating the place of purchase, a purchase date item indicating the date of purchase of the product, and a quantity item indicating the number of products purchased.

[0132] The processor (180) can generate a prompt for selecting data related to the user's behavior from the first behavior data set (311). The processor (180) can receive text input or voice input for generating the prompt through the user input interface (123).

[0133] LLM(510) may return a data selection code as input to the prompt. The data selection code may be either an SQL query or Python code. An SQL query may be a structured query written to retrieve or manipulate desired data in a database. An SQL query may be referred to as an SQL query code.

[0134] The prompt may include the role of the LLM (510), the purpose of data selection, a description of the data, and the content of the work request.

[0135] FIG. 5b is a diagram showing an example of a prompt (540) that is entered into the LLM (510).

[0136] The prompt (540) may include a first prompt element (541), a second prompt element (542), a third prompt element (543), and a fourth prompt element (544).

[0137] The first prompt element (541) may be an element that specifies the role of the LLM (510).

[0138] The second prompt element (542) may be an element that specifies the purpose of the business objective or data selection of the LLM (510).

[0139] The third prompt element (543) may be an element containing a description of the data that the LLM (510) is to analyze.

[0140] The fourth prompt element (544) may be an element containing the request for the task to be performed by the LLM (510).

[0141] The LLM (510) can generate an SQL query (520) as illustrated in FIG. 5c in response to an input prompt (540).

[0142] The processor (180) can obtain user behavior information (320) by applying the SQL query (520) output from the LLM (510) to the first behavior data set (311) stored in memory (170).

[0143] The processor (180) can interpret conditions specified in the SQL query (520) and can query each DB table and each DB table's column name included in the first behavior data set (311) to extract data that matches the interpreted conditions. The processor (180) can combine the extracted data to generate user behavior information (320).

[0144] User behavior information (320) is result data extracted by executing an SQL query (520) on a first behavior data set (311), and may include specific records regarding specific actions actually performed by individual customers. The behavior information (320) may include a behavior ID identifying a behavior (or unit behavior), a customer ID identifying a customer, unit behavior information, and detailed behavior information.

[0145] Unit behavior information may include information indicating whether a customer has purchased a product. For example, unit behavior information may include information about the behavior of purchasing a washing machine.

[0146] Detailed behavioral information may be information indicating when, what product, where, and how many units a customer purchased. For example, detailed behavioral information may include information about the behavior of purchasing 3 ABC washing machine models at Best Shop on January 1, 2024.

[0147] The behavior information (320) can be referred to as a behavior information table.

[0148] The processor (180) can extract user behavior information (320) by executing an SQL query (520) output through the LLM (510).

[0149] The extracted user behavior information (320) can be used to generate a behavior sequence or derive behavior indicators.

[0150] The processor (180) can extract behavior information corresponding to each of the second to fifth behavior data sets (312 to 315) through such a process.

[0151] Again, Figure 4 is explained.

[0152] The processor (180) can obtain a user embedding vector based on the extracted user behavior information (S405).

[0153] In one embodiment, the user embedding vector may be a vector for representing the characteristics of a user. The user embedding vector may be referred to as a user embedding.

[0154] In one embodiment, the user embedding vector can be used as an input variable for an artificial intelligence model that derives user attributes.

[0155] In another embodiment, the user embedding vector can be used to search for other users having characteristics similar to those of a specific user.

[0156] In one embodiment, a user embedding vector may be extracted from behavior information corresponding to a single behavior domain. The processor (180) may extract multiple user embedding vectors corresponding to each of multiple behavior domains. Each user embedding vector may be used to calculate an attribute score of an attribute matched to the corresponding behavior domain.

[0157] In another embodiment, the user embedding vector can be extracted from behavior information corresponding to a plurality of behavior domains.

[0158] In other words, user embedding vectors can be generated based on a single behavior domain or based on multiple behavior domains.

[0159] In one embodiment, the processor (180) can extract a user embedding vector from the user's behavior information through a Large Behavior Model (LBM).

[0160] LBM is a model based on LLM that converts all sequential actions performed by a user into text and sequentially inputs the converted text into the LLM to extract user embedding vectors.

[0161] FIGS. 6a and 6b are drawings illustrating a process of extracting a user embedding vector based on LBM according to an embodiment of the present disclosure.

[0162] Referring to Fig. 6a, the learning process of LBM (332) is illustrated.

[0163] LBM (332) may be stored in memory (170).

[0164] LBM (332) may be a model based on BERT (Bidirectional Encoder Representations from Transformer). A BERT-based LBM (332) may be a model that focuses on the input sentence itself to determine what characteristics the sentence has. A BERT-based LBM (332) can analyze the relationships between words in both directions of the sentence (forward and backward directions of the sentence).

[0165] LBM (332) can be learned by masking some words in the input text and then guessing those words (self-supervised learning).

[0166] For example, when training text (610) with some words (buy, water purifier) ​​masked is input into LBM (332), LBM (332) can be trained by inferring the masked words (buy, water purifier).

[0167] Referring to FIG. 6b, a process is illustrated in which a behavior sequence (620) corresponding to the management area of ​​a home appliance is input into a learned LBM (332) to output a user embedding vector (330).

[0168] The action sequence (620) is generated based on the user's action information (320) and may be a text-type sequence that sequentially expresses unit actions.

[0169] The unit actions constituting the action sequence (620) can be sequentially input into the LBM (332).

[0170] The action sequence (620) may be either text representing a unit action of the user's action information (320) or text representing a detailed action.

[0171] The LBM (332) can output a user embedding vector (330) representing the characteristics of the user from the action sequence (620). The user embedding vector (330) is a high-dimensional vector consisting of N numbers, which are the node values ​​of the last layer constituting the LBM (332). N can be 768, but this is merely an example and may vary depending on the type of the LBM (332).

[0172] As such, according to an embodiment of the present disclosure, the user embedding vector generated through LBM can implicitly represent various user behaviors as the context between the sequence of user behaviors is reflected.

[0173] Again, Figure 4 is explained.

[0174] In another embodiment, the processor (180) can extract a user embedding vector from the user's behavior information through a behavior indicator map.

[0175] FIGS. 7a and 7b are drawings illustrating the process of extracting user embedding vectors based on a behavior indicator map according to an embodiment of the present disclosure.

[0176] FIG. 7a may be a drawing illustrating an embodiment of step S405 of FIG. 4.

[0177] Referring to FIG. 7a, the processor (180) can extract a behavior indicator map based on user behavior information (S701).

[0178] The behavior indicator map may be a map showing behavior indicators for each unit behavior of the user by time period. A unit behavior may represent a minimum unit of behavior that is mutually distinguishable, such as “washing machine repair” or “air conditioner purchase.” A unit behavior may correspond to a unit behavior item included in the user’s behavior information (320).

[0179] Behavioral indicators can serve as metrics to relatively compare the importance of different unit actions. For example, if the unit action of dispensing water from a water purifier occurs 10 times a day and the unit action of doing laundry with a washing machine occurs 10 times a day, the frequency is the same at 10 times, but the action of doing laundry more frequently would be more impactful in identifying customer characteristics.

[0180] Behavioral indicators may be normalized indicators that account for the relativity between heterogeneous unit behaviors. Behavioral indicators may be weighted sums of four unit indicators: Recency (R), Frequency (F), Monetary (M), and Period (P). RFMP indicators may be an example of behavioral indicators. Behavioral indicators may also be referred to as RFMP indicators. Each unit indicator may have a range from 0 to 100.

[0181] The recency (R) indicator may be an indicator of how recently a unit action occurred. The recency (R) indicator can be 100 when the unit action occurred most recently. However, the recency (R) indicator may approach 100 as the action occurs more recently, and may approach 0 as the unit action occurs in the past.

[0182] The frequency (F) indicator may be an indicator of how often a unit behavior occurs. The frequency (F) indicator can be 100 when the unit behavior occurs at maximum frequency. The frequency (F) indicator can approach 100 as it approaches maximum frequency and can approach 0 as it approaches minimum frequency.

[0183] The Amount (M) indicator may be an indicator representing how much cost / time / energy is required for a unit action. The Amount (M) indicator can be 100 when the unit action occurs at maximum intensity. The Amount (M) indicator can approach 100 as it approaches maximum intensity, and can approach 0 as it approaches minimum intensity.

[0184] The amount (M) indicator may represent any one of the time intensity (M_T) indicator, the energy intensity (M_E) indicator, or the cost intensity (M_P), or represent a comprehensive amount evaluation indicator calculated based on a combination of the three indicators.

[0185] The period (P) indicator may be an indicator of how periodically a unit action occurs. The period (P) indicator can be 100 when the unit action occurs at a constant period. The period (P) indicator can approach 100 when the unit action occurs in a manner that approaches a constant period.

[0186] The processor (180) can calculate an average value indicating how many times the entire user performs a unit action on average for a specific unit action, and can normalize the action indicators by taking the average value into account. For example, if the water dispensing action through the previously described water purifier is 10 times and the washing action through the washing machine is 10 times, the action indicator for the water dispensing action may be smaller than the action indicator for the washing action.

[0187] When the processor (180) derives behavior indicators for each unit of action, it can statistically analyze the derived behavior indicators for a set period of time. The set period of time may be one day, one month, or one year, but this is merely an example.

[0188] The processor (180) can generate a grayscale image by representing the behavior indicators by time period in the form of a heat map, and can generate the generated grayscale image as a behavior indicator map.

[0189] Referring to FIG. 7b, a process for deriving a unit action indicator corresponding to each unit action based on user action information (320) is illustrated. The unit action indicator may be any one of a recency (R) indicator, a frequency (F) indicator, a monetary (M) indicator, or a period (P) indicator.

[0190] The processor (180) can use behavior information (320) to calculate a recency (R), frequency (F), amount (M), and period (P) indicators corresponding to each of multiple unit behaviors that occurred within a specific period for a specific user.

[0191] For example, the processor (180) can obtain a recency (R) indicator of 80, a frequency (F) indicator of 65, an amount (M) indicator of 90, and a period (P) indicator of 70 corresponding to the unit action of <washing machine purchase> for a user with customer ID C1 from 2019 to 2025.

[0192] The processor (180) can calculate RFMP indicators by applying weights (w1 to w4) to each acquired unit indicator. That is, the processor (180) can obtain 73 by calculating the RFMP indicator corresponding to the unit action <purchase washing machine> as shown in the following [Equation 1].

[0193] [Mathematical Formula 1]

[0194] RFMP = w1*R + w2*F + w3*M + w4*P

[0195] Each of the weights (w1 to w4) may be the same or different from one another. The weights (w1 to w4) may be values ​​that vary depending on the administrator's settings.

[0196] Each of the weights (w1 to w4) can be a fixed value.

[0197] Additionally, the processor (180) can obtain a recency (R) indicator of 90, a frequency (F) indicator of 10, an amount (M) indicator of 13, and a period (P) indicator of 8 for a user with customer ID C1 from 2019 to 2025, corresponding to a unit action called <refrigerator repair>.

[0198] The processor (180) can obtain an RFMP index of 31 according to [Equation 1].

[0199] The processor (180) can obtain multiple behavior indicators corresponding to each of the multiple unit behaviors, and can generate a behavior indicator map (342) as shown in FIG. 7c using the obtained multiple behavior indicators.

[0200] The behavior indicator map (342) may be a two-dimensional table-shaped heat map having a horizontal axis representing time and a behavior indicator axis representing behavior indicators corresponding to unit behavior.

[0201] A first action indicator (73) corresponding to the unit action of <buying a washing machine> can be mapped to the first coordinate (701) of the action indicator map (342), and a second action indicator (31) corresponding to the unit action of <repairing a refrigerator> can be mapped to the second coordinate (702).

[0202] The behavioral pattern of the user can be intuitively identified through the behavioral indicator map (342). Specifically, the behavioral indicator map (342) may be a map for intuitively identifying which behaviors the user exhibits at which time of day.

[0203] The processor (180) can generate a user embedding vector from the extracted behavior indicator map (S703).

[0204] In one embodiment, the processor (180) may generate a user embedding vector (340) by inputting a behavior indicator map (342) into an encoder (343) as shown in FIG. 7d. The number of dimensions of the user embedding vector (340) may be 32, but this is merely an example.

[0205] The encoder (343) may be a model having a dimensionality reduction function. Since the number of unit actions will be very large, the size of the action indicator map (342) may become very large. The encoder (343) may perform an encoding method to reduce the dimensionality of the action indicator map (342).

[0206] In one embodiment, any one of the encoding methods may be an AutoEncoder method, a PCA (Principal Component Analysis) method, or a tSNE (t-Distributed Stochastic Neighbor Embedding) method.

[0207] The AutoEncoder method can be a neural network-based dimensionality reduction method that encodes input data into a low-dimensional latent space in a non-linear manner and is trained to reconstruct it.

[0208] PCA (Principal Component Analysis) is a method that calculates principal components based on the direction with the greatest variance of the data and transforms them into a lower-dimensional space through linear projection.

[0209] The tSNE (t-Distributed Stochastic Neighbor Embedding) method may be a non-linear visualization method that calculates the similarity between high-dimensional data as a probability distribution and reconstructs the calculated probability distribution based on the t-distribution in a low-dimensional space to preserve local similarity.

[0210] The user embedding vector (340) may have the same or different number of dimensions as the user embedding vector (330) generated in FIG. 6b.

[0211] The user embedding vector (330) generated in FIG. 6b may be referred to as a first type user embedding vector or a first user embedding vector, and the user embedding vector (340) generated in FIG. 7d may be referred to as a second type user embedding vector or a second user embedding vector. The number of dimensions of the second user embedding vector may be smaller than the number of dimensions of the first user embedding vector.

[0212] FIGS. 8a to 8d are drawings showing behavior indicator maps and clustered results based on the behavior indicator maps according to another embodiment of the present disclosure.

[0213] Referring to FIG. 8a, RFMP-based behavioral indicator maps (800-1 to 800-n) of n people possessing a cooktop, refrigerator, and water purifier are shown. n may be 293.

[0214] Each of the multiple behavior indicator maps (800-1 to 800-n) may be a map corresponding to each of n users. The time axis of each behavior indicator map may be an axis in units of one hour, and the behavior indicator axis may be an axis representing a behavior indicator corresponding to each behavior unit.

[0215] The processor (180) can generate multiple user embedding vectors corresponding to each of the multiple behavior indicator maps (800-1 to 800-n) through the encoder (343).

[0216] The processor (180) can cluster the generated multiple user embedding vectors into 10 vector groups (811 to 820) as shown in FIG. 8b.

[0217] Referring to FIG. 8c, 10 average behavior indicator maps (avg C_0 to avg C_9) corresponding to each of the 10 vector groups (811 to 820) are shown.

[0218] The processor (180) can generate 10 average behavior indicator maps (avg C_0 to avg C_9) by averaging the behavior indicator maps of multiple users belonging to each of the 10 vector groups (811 to 820). For example, the first average behavior indicator map (avg C_0) may correspond to the first vector group (811), and the second average behavior indicator map (avg C_1) may correspond to the second vector group (812).

[0219] The average behavior indicator map may be a map that visualizes the average behavioral patterns of users belonging to each vector group. For example, the second average behavior indicator map (avg C_1) may represent a cluster of users with high energy consumption. Services for energy-saving measures may be recommended to this user cluster. Additionally, the tenth average behavior indicator map (avg C_9) may represent a cluster of users who frequently eat late-night snacks. Services for improving dietary habits may be recommended to this user cluster.

[0220] Accordingly, according to an embodiment of the present disclosure, representative vector groups characterizing user behavior patterns can be derived by grouping multiple user embedding vectors together based on their similarity. This enables the efficient application of customized services, recommendations, or control policies tailored to different user clusters. Furthermore, it has the effect of allowing for the analysis of user preferences or behavioral tendencies based on the characteristics of each cluster.

[0221] Meanwhile, referring to FIG. 8d, the processor (180) can generate a plurality of comparison behavior indicator maps (diff c_0 to diff c_9) by calculating the difference between each of the 10 average behavior indicator maps (avg C_0 to avg C_9) and the overall user average behavior indicator map.

[0222] The overall user average behavior indicator map may be a map representing the average of the plurality of behavior indicator maps (800-1 to 800-n) of FIG. 8a.

[0223] The first color (821) may be a color representing an action that the user group does frequently, and the second color (822) may be a color representing an action that the user group does not frequently do.

[0224] As such, according to an embodiment of the present disclosure, by utilizing a comparative behavioral indicator map, it is possible to intuitively identify behavioral patterns that a customer group belonging to a specific cluster relatively prefers or avoids compared to the average customer. This enables the effective identification of the characteristic usage tendencies of each customer group and contributes to the derivation of differentiated recommendation, marketing, and control policies.

[0225] Again, Figure 4 is explained.

[0226] The processor (180) can obtain user attribute information based on the acquired user embedding vector (S407).

[0227] Attribute information may include multiple attribute scores corresponding to each of the multiple attributes.

[0228] Each attribute score can represent a score that quantitatively expresses the likelihood of possessing the corresponding attribute. Attribute scores may be referred to as attribute indices.

[0229] In one embodiment, the attribute score may have a range of -1.0 to +1.0. As the attribute index approaches +1.0, it may indicate that the consistency with the corresponding attribute is higher, and as the attribute index approaches -1.0, it may indicate that the consistency with the corresponding attribute is lower.

[0230] In another embodiment, the attribute score may have a range from 0 to 100. As the attribute index approaches 100, it may indicate that the consistency with the corresponding attribute is higher, and as the attribute index approaches 0, it may indicate that the consistency with the corresponding attribute is lower.

[0231] Multiple attributes can be classified as intention attributes, type attributes, or fact attributes.

[0232] FIG. 9 is a drawing showing a plurality of attributes that a user may have according to one embodiment of the present disclosure.

[0233] Referring to FIG. 9, an attribute classification table (900) is shown.

[0234] The attribute classification table (900) may be stored in memory (170).

[0235] The attribute classification table (900) may include three top attributes. The three top attributes may include an intention attribute, a type attribute, and a fact attribute.

[0236] Each of the intent attribute, type attribute, and fact attribute may include intermediate attributes. Each intermediate attribute may include one or more sub-attributes.

[0237] The attribute classification table (900) may include sub-attributes corresponding to behavioral domains associated with each intermediate attribute.

[0238] The Intent attribute may be an attribute representing the user's intent. The intermediate attributes of the Intent attribute may include an imminent purchase intent attribute, a new subscription / subscription cancellation intent attribute, and a potential purchase intent attribute. The potential purchase intent attribute may include an attribute indicating whether the pet is owned.

[0239] The Type attribute may be an attribute representing the user's type. The median attributes of the Type attribute may include household type, household economy, life event, lifestyle / interest, behavior pattern, driving pattern, and carbon emission attributes. For example, the lifestyle / interest attribute could be a home cooking attribute indicating that the user enjoys cooking at home.

[0240] The Fact attribute may be an attribute representing quantitative indicators or fixed information about the user. The intermediate attributes of the Fact attribute may include behavioral indicator attributes and demographic attributes. The sub-attributes of the behavioral indicator attribute may be RFMP indicator attributes for unit behaviors.

[0241] Again, Figure 4 is explained.

[0242] In one embodiment, the processor (180) can derive user attribute information from user embedding vectors through supervised learning.

[0243] FIGS. 10a and FIGS. 10b are drawings illustrating a process of deriving user attribute information from a user embedding vector according to one embodiment of the present disclosure.

[0244] The embodiment of FIG. 10a may be a method used when user embedding vectors and attribute answer sheets corresponding to each user embedding vector are rich.

[0245] Referring to FIG. 10a, the attribute derivation model (361) may be a model based only on artificial neurons learned through supervised learning. The attribute derivation model (361) may be a model that infers user attribute information from user embedding vectors. The attribute information may also be represented as a vector.

[0246] The training data set for supervised learning of the attribute derivation model (361) may include user embedding vectors for training and attribute labels (or attribute answer sheets) representing correct data.

[0247] The attribute derivation model (361) can be trained to minimize a loss function representing the difference between the output attribute and the attribute label using a training user embedding vector (10001) as input.

[0248] The output of the attribute derivation model (361) may be an attribute score, and the attribute label may also be an attribute score. The attribute derivation model (361) may be a model that outputs an attribute score corresponding to a single attribute. In this case, the attribute label may include the correct attribute score. Accordingly, multiple attribute derivation models may be used to obtain multiple attribute scores corresponding to the user. That is, each attribute derivation model may be a model that outputs an attribute score corresponding to a specific attribute.

[0249] The attribute derivation model (361) may be a model that outputs multiple attribute scores corresponding to multiple attributes. In this case, the attribute label may include multiple correct attribute scores.

[0250] The trained attribute derivation model (361) can infer user attribute information from the input user embedding vector (1011). The user attribute information may include upper attributes, middle attributes, and lower attributes. The user attribute information may include only one or more lower attributes.

[0251] The attribute derivation model (361) can have its performance verified through testing, and if the reliability is above a certain level, it can infer the user's attribute information during the application phase.

[0252] Referring to FIG. 10b, the processor (180) can obtain a first attribute score (0.7) corresponding to a first attribute (single-person household attribute) from a user embedding vector (1021) through a first attribute derivation model (361a), and can obtain a second attribute score (-0.3) corresponding to a second attribute (pet ownership attribute) from a user embedding vector (1021) through a second attribute derivation model (361b). That is, a number of attribute derivation models corresponding to the number of attributes to be derived can be used.

[0253] In another embodiment, the processor (180) can derive user attribute information from the user embedding vector through virtual behavior data.

[0254] This method can be used when the correct answer key for an attribute is almost impossible to obtain.

[0255] FIGS. 11a and FIGS. 11b are drawings illustrating a process of deriving user attribute information from a user embedding vector according to another embodiment of the present disclosure.

[0256] Referring to Fig. 11a, for the type attribute <game enthusiast>, a virtual action sequence (1101) corresponding to the attribute can be input into the LBM (332), and the LBM (332) can output a virtual user embedding vector (1102).

[0257] Afterwards, an actual action sequence (1111) representing the user's actual action can be input into the LBM (332), and the LBM (332) can output an actual user embedding vector (1112).

[0258] The attribute derivation model (361-1) can measure the similarity between a virtual user embedding vector (1102) and a real user embedding vector (1112), and can output the attribute score of the user based on the similarity.

[0259] The similarity between the virtual user embedding vector (1102) and the real user embedding vector (1112) may be either distance-based Euclidean similarity between the vectors or direction-based cosine similarity between the vectors.

[0260] The attribute derivation model (361-1) can output an attribute score that numerically indicates how much a user possesses a specific attribute, such as "game fanatic," based on the measured similarity. The attribute score can quantitatively express the likelihood or strength of the user possessing the attribute. The processor (180) can obtain the attribute score corresponding to the specific attribute as the user's attribute information based on the measured similarity.

[0261] Referring to FIG. 11b, if one wishes to derive an attribute of liking late-night snacks, a virtual behavior indicator map (1121) corresponding to the attribute may be input into an encoder (343), and the encoder (343) may output a virtual user embedding vector (1122). The virtual behavior indicator map (1121) may be generated to reflect the late-night time period of the home appliance's usage time.

[0262] Afterwards, a real behavior indicator map (1131) reflecting the user's actual behavior can be input into the encoder (343), and the encoder (343) can output a real user embedding vector (1132).

[0263] The attribute derivation model (361-2) can measure the similarity between a virtual user embedding vector (1122) and a real user embedding vector (1132), and can output the attribute score of the user based on the similarity.

[0264] The similarity between the virtual user embedding vector (1122) and the real user embedding vector (1132) may be either distance-based Euclidean similarity between the vectors or direction-based cosine similarity between the vectors.

[0265] The attribute derivation model (361-2) can output an attribute score that numerically indicates how much a user possesses the attribute of liking late-night snacks based on the measured similarity. The attribute score can quantitatively express the likelihood or strength of the user possessing the attribute.

[0266] In another embodiment, the processor (180) can derive user attribute information based on a representative embedding vector obtained through user embedding vectors.

[0267] FIG. 12 is a diagram illustrating a process of deriving user attribute information based on a user embedding vector according to another embodiment of the present disclosure.

[0268] Referring to FIG. 12, the processor (180) can extract a representative embedding vector (1201) based on a plurality of attribute embedding vectors (1200-1 to 1200-n) that correspond to a plurality of users and are each matched to a plurality of predetermined attribute scores of a specific attribute. Each of the plurality of attribute embedding vectors (1200-1 to 1200-n) may have an attribute score of a known attribute matched to it.

[0269] In one embodiment, the processor (180) can calculate the average value of each of the multiple dimensions for a plurality of attribute embedding vectors (1200-1 to 1200-n) and obtain a representative embedding vector (1201) having the calculated average values.

[0270] In another embodiment, the processor (180) can calculate the centroid of a plurality of attribute embedding vectors (1200-1 to 1200-n) through an arithmetic mean in a vector space, and can obtain the calculated centroid as a representative embedding vector (1201).

[0271] The attribute derivation model (361-3) can measure the similarity between the representative embedding vector (1201) and the actual user embedding vector (1211), and can output the attribute score of the user based on the similarity.

[0272] The similarity between the representative embedding vector (1201) and the actual user embedding vector (1211) may be either distance-based Euclidean similarity between the vectors or direction-based cosine similarity between the vectors.

[0273] The attribute derivation model (361-3) can output an attribute score based on the measured similarity. The attribute score can quantitatively express the likelihood or strength that the user possesses the corresponding attribute.

[0274] The processor (180) can provide a service that matches the acquired attribute information (S409).

[0275] The service may include one or more of an interactive analytics service, a user-customized home appliance control service, a content recommendation service, or an energy management service.

[0276] The conversational analysis service may be a service that provides the results of the analysis of the question in a conversational format through a conversational agent (380).

[0277] In one embodiment, the processor (180) can obtain attributes inferred through an attribute derivation model (361) learned through supervised learning as user attribute information.

[0278] In another embodiment, the processor (180) includes attribute information that includes attribute scores output through attribute derivation models (361-1, 361-2, 361-3), and if the attribute score is greater than or equal to a certain score, the processor can obtain an attribute corresponding to the embedding vector of the comparison target as the user's attribute information.

[0279] In one embodiment, the processor (180) can output a service corresponding to the attribute information through the output interface (150).

[0280] In another embodiment, the processor (180) can transmit a service corresponding to the attribute information to an external device or a user's terminal through the communication interface (110).

[0281] FIG. 13 is a flowchart illustrating a personalized recommendation service method of a device according to one embodiment of the present disclosure.

[0282] One or more processors (180) may be provided.

[0283] Referring to FIG. 13, the processor (180) of the device (100) can obtain a first user embedding vector based on a first action data set corresponding to a first action area (S1301).

[0284] The process of obtaining a first user embedding vector based on a first behavior data set may be applied to the embodiments of FIGS. 5a to 7b.

[0285] The processor (180) can measure the similarity between a plurality of user embedding vectors corresponding to a second user's second behavioral area and a first user embedding vector (S1303).

[0286] The processor (180) can obtain either distance-based Euclidean similarity or direction-based cosine similarity between vectors as the similarity between vectors.

[0287] Multiple user embedding vectors can be stored in a customer characteristic DB (350) included in memory (170). The customer characteristic DB (350) can be provided separately from memory (170) or device (100).

[0288] The processor (180) can measure the similarity between the first user embedding vector and the plurality of user embedding vectors corresponding to the first user and other users, and the first user embedding vector and other user embedding vectors corresponding to the first behavior area and other behavior areas.

[0289] The customer characteristic DB (350) may include multiple sets of embedding vectors corresponding to each of the multiple behavioral domains. That is, multiple user embedding vectors may be stored in each behavioral domain by matching the sets of embedding vectors.

[0290] The customer characteristic DB (350) may include multiple user embedding vectors extracted from the same behavior domain and different behavior domains.

[0291] The customer characteristic DB (350) may include a plurality of user embedding vectors extracted from the first user's behavior data set and the second user's behavior data set.

[0292] Each of the multiple user embedding vectors corresponding to the second behavior domain may be a vector obtained based on the second behavior data set of the second user.

[0293] In one example, the first action area may be the management area of ​​a home appliance, and the second action area may be the movement area, but this is merely an example.

[0294] The processor (180) can determine whether there exists a second user embedding vector among a plurality of user embedding vectors that has a similarity greater than a certain degree to the first user embedding vector (S1305).

[0295] The processor (180) can extract a second user embedding vector of a second user based on a second behavioral domain different from the first behavioral domain that formed the basis of the first user embedding vector from the customer characteristic DB (350) through similarity comparison.

[0296] When the processor (180) determines that the similarity between the first user embedding vector and the second user embedding vector is greater than a certain similarity, it can obtain one or more attribute elements that have a certain attribute score or more among a plurality of attribute elements matched to the second user embedding vector (S1307).

[0297] The processor (180) can obtain one or more attribute elements having a certain attribute score or more among a plurality of attribute elements matched to a second user embedding vector from the customer attribute DB (370).

[0298] A plurality of attribute elements matched to the second user embedding vector and information of each attribute element can be obtained according to the embodiments of FIGS. 9 to 12.

[0299] The processor (180) can extract the attribute element with the largest attribute score among the multiple attribute elements matched to the second user embedding vector.

[0300] The processor (180) can provide a recommendation service based on one or more acquired attribute elements (S1309).

[0301] The processor (180) can provide a recommendation service using behavior data matched to one or more acquired attribute elements.

[0302] As another example, the processor (180) can generate a recommendation service based on behavior data matched to the attribute element with the largest attribute score among a plurality of attribute elements matched to the second user embedding vector.

[0303] FIGS. 14 to 16d are drawings for explaining a method for providing a personalized recommendation service for a vehicle according to one embodiment of the present disclosure.

[0304] In particular, FIGS. 14 to 16d may be embodiments in which the embodiment of FIG. 13 is applied to a vehicle.

[0305] In FIGS. 14 to 16d, the device (100) may be a vehicle (100).

[0306] Referring to FIG. 14, the processor (180) of the vehicle (100) can obtain a first user embedding vector based on a first action data set corresponding to a first action area (S1401).

[0307] The process of obtaining a first user embedding vector based on a first behavior data set may be applied to the embodiments of FIGS. 5a to 7b.

[0308] The processor (180) can receive a voice command spoken by the first user (S1403).

[0309] The processor (180) can receive voice commands spoken by the first user in the vehicle (100) through the microphone (122).

[0310] The processor (180) can extract a second action area based on the analysis result of the received voice command (S1405).

[0311] The processor (180) can convert the voice command into text data using a Speech To Text (STT) engine. The processor (180) can obtain an analysis result from the converted text data using a Natural Language Processing (NLP) engine. The analysis result may include the first user's intent.

[0312] The processor (180) can extract a behavior area that matches the analysis result of a voice command among a plurality of behavior areas. For example, if the analysis result of the voice command indicates an intention to recommend a place to visit, the behavior area may be the user's movement area.

[0313] As another example, if the analysis result of a voice command indicates an intention to recommend a refrigerator, the behavioral domain may be the management domain of home appliances.

[0314] The second behavioral domain may be a different domain from the first behavioral domain that formed the basis of the first user embedding vector of step S1401. For example, if the first behavioral domain is a management domain of a home appliance, the second behavioral domain may be a movement domain.

[0315] The processor (180) can measure the similarity between a plurality of user embedding vectors corresponding to a second behavioral area of ​​a second user and a first user embedding vector (S1407).

[0316] The processor (180) can obtain either distance-based Euclidean similarity or direction-based cosine similarity between vectors as the similarity between vectors.

[0317] The description of step S1407 is replaced with the relevant description of step S1303 of FIG. 13.

[0318] The processor (180) can determine whether there exists a second user embedding vector among a plurality of user embedding vectors that has a similarity greater than a certain degree to the first user embedding vector (S1409).

[0319] The processor (180) can extract a second user embedding vector of a second user based on a second behavioral domain different from the first behavioral domain that formed the basis of the first user embedding vector from the customer characteristic DB (350) through similarity comparison.

[0320] When the processor (180) determines that the similarity between the first user embedding vector and the second user embedding vector is greater than a certain similarity, it can obtain one or more attribute elements that have a certain attribute score or more among a plurality of attribute elements matched to the second user embedding vector (S1411).

[0321] The processor (180) can obtain one or more attribute elements having a certain attribute score or more among a plurality of attribute elements matched to a second user embedding vector from the customer attribute DB (370).

[0322] A plurality of attribute elements matched to the second user embedding vector and information of each attribute element can be obtained according to the embodiments of FIGS. 9 to 12.

[0323] The processor (180) can extract the attribute element with the largest attribute score among the multiple attribute elements matched to the second user embedding vector.

[0324] The processor (180) can output a recommendation service based on the analysis results of one or more acquired attribute elements and voice commands (S1413).

[0325] The processor (180) can generate a recommendation service based on the analysis results of behavior data and voice commands matched to one or more acquired attribute elements, and can output the generated recommendation service through the output interface (150).

[0326] The recommendation service may be a service that reflects behavioral data related to the second user's second behavior domain. For example, the recommendation service may include one or more of a destination recommendation service, a waypoint recommendation service, a content recommendation service, or an activity recommendation service.

[0327] The processor (180) can obtain a recommendation service by inputting the analysis results of one or more acquired attribute elements and voice commands into the conversational agent (380).

[0328] Meanwhile, if the processor (180) determines that there is no second user embedding vector among the plurality of user embedding vectors that has a similarity of at least a certain degree to the first user embedding vector, it can output a recommendation service based on the analysis results of the behavior data set and voice command corresponding to the second behavior area of ​​the first user (S1415).

[0329] For example, if the analysis result of a voice command indicates a place recommendation intention, and the data corresponding to the behavior ID with the largest RFMP indicator based on a behavior data set corresponding to the first user's second behavior area indicates a visit to an outdoor store, information about high-rated outdoor stores near the destination can be obtained as a recommendation service.

[0330] FIG. 15 is a diagram illustrating a method for providing a recommendation service based on a second user embedding vector similar to a first user embedding vector corresponding to a first user's first behavioral area according to one embodiment of the present disclosure.

[0331] Referring to FIG. 15, the attribute table (1510), the first action table (1520), and the second action table (1530) are illustrated.

[0332] The attribute table (1510), the first action table (1520), and the second action table (1530) may be stored in memory (170).

[0333] The attribute table (1510) can be stored in the customer attribute DB (370) of FIG. 3, and the first behavior table (1520) and the second behavior table (1530) can be stored in the behavior indicator DB (341).

[0334] The attribute table (1510) may be a table containing attribute scores of attribute elements corresponding to the behavioral domain of the customer (or user).

[0335] The attribute table (1510) may be a table that maps a behavior domain, a customer ID, a user embedding vector (or user embedding), and a plurality of attribute elements.

[0336] The behavior domain may include a movement domain (map) and a home appliance management domain (cdp).

[0337] A customer ID can be an identifier for identifying customers in the corresponding behavioral domain.

[0338] The first behavior table (1520) may be a table containing RFMP indicators for the customer's behavior area.

[0339] The first behavior table (1520) may be a table mapping customer ID, behavior ID, recency (R) indicator, frequency (F) indicator, amount (M) indicator, period (P) indicator and RFMP indicator.

[0340] A behavior ID can be an identifier that identifies unit behaviors corresponding to a specific behavioral domain of a customer.

[0341] The second behavior table (1530) may be a table containing a behavior name and a behavior description for a customer's behavior area. The behavior name may be the name of a behavior ID. For example, if the unit behavior is a visit to place 1, the behavior name may be place 1. The behavior description may be natural language text that describes the unit behavior in detail. For example, if the behavior name is place 1, the behavior description is<place 1을 자주 방문함> It could be.

[0342] The second behavior table (1530) may be a table mapping behavior domains, behavior IDs, behavior names, behavior embeddings, and behavior descriptions. The behavior embedding may be a vector representing the semantic features of a unit behavior. The behavior embedding may be a vector generated by at least one of RFMP indicators, behavior names, or behavior descriptions.

[0343] In FIG. 15, it is assumed that the first user boards the vehicle (100) and utters the voice command "Recommend a place to go nearby."

[0344] The processor (180) can obtain the analysis result of the spoken voice command as <recommendation of nearby places to visit>.

[0345] The processor (180) can extract a behavior area that matches the analysis result of a voice command among a plurality of behavior areas as a movement area (map) related to the user's location movement.

[0346] The processor (180) can extract the <home appliance purchase cost-effectiveness attribute> with the largest attribute score (89 points) among a plurality of attribute elements corresponding to the management area (cdp) of the home appliance of the first user (u1) through the attribute table (1510). The processor (180) can obtain the first user embedding vector (1511) that formed the basis of the attribute score of the <home appliance purchase cost-effectiveness attribute>.

[0347] The processor (180) can extract a second user embedding vector (1512) having a certain degree of similarity to a first user embedding vector (1511) among a plurality of user embedding vectors corresponding to different behavioral areas of other users from an attribute table (1510) or a customer characteristic DB (350). The second user embedding vector (1512) may be a vector corresponding to the movement area (map) of the second user (u2).

[0348] The second user embedding vector (1512) may have 100 points of attribute value for an attribute element called <luxury POI visit attribute> mapped to it.

[0349] The processor (180) can extract the action ID (map_b1, 1521) mapped to the customer ID (map_u2) for the movement area of ​​the second user (u2) through the first action table (1520).

[0350] The processor (180) can obtain one or more of the action names or action descriptions mapped to the action ID (map_b1, 1521) extracted through the second action table (1530).

[0351] The processor (180) can generate a visit place recommendation service based on the acquired action name or action description. For example, the processor (180)<place 1> Text recommending a visit to a place called can be generated, and the generated text can be displayed on the display (151) of the vehicle (100).

[0352] In this way, according to an embodiment of the present disclosure, an attribute-based recommendation service for similar types of users can be provided through similar user embedding vectors of other users corresponding to different behavioral domains of the user.

[0353] Accordingly, recommendation accuracy in other user behavior domains can be dramatically improved, and it can be of great help in solving data sparsity and cold start problems.

[0354] FIGS. 16a to 16d are drawings illustrating a process of providing a visit location recommendation service based on different behavioral areas of different users in accordance with a user's voice command in a vehicle according to an embodiment of the present disclosure.

[0355] Referring to FIG. 16a, the display (151) of the vehicle (100) may display a voice recognition initial screen (1610). The voice recognition initial screen (1610) may include cafe recommendation items (1611), rest place recommendation items (1612), and nearby activity recommendation items (1613).

[0356] The voice recognition initial screen (1610) may further include a voice recognition icon (1614) for activating voice recognition.

[0357] The processor (180) of the vehicle (100) can activate the voice recognition service when it selects the voice recognition icon (1614) or recognizes the activation word.

[0358] The processor (180) can display a speech waiting screen (1620) waiting for a user's speech on the display (151) as shown in FIG. 16b, as the speech recognition icon (1614) is selected.

[0359] The user can utter a voice command (1621) with the intention of searching for a dinner place near the Las Vegas Convention Center.

[0360] The processor (180) can display a voice recognition confirmation screen (1630) containing text (1631) corresponding to a spoken voice command (1621) on the display (151).

[0361] The processor (180) can extract a movement area matched to the intent of a voice command (1621) according to the embodiment of FIGS. 14 and 15. The processor (180) can extract a second user embedding vector mapped to a movement area of ​​another user having a certain degree of similarity or greater than a first user embedding vector corresponding to a specific user action area (e.g., home appliance management area) from an attribute table (1510) or a customer characteristic DB (350).

[0362] The processor (180) can extract attribute elements mapped to the extracted second user embedding vector and can extract an action ID corresponding to the attribute elements extracted from the first action table (1520).

[0363] The processor (180) may extract one or more action names or action descriptions corresponding to an action ID extracted from the second action table (1530). The processor (180) may obtain a place to visit near the Las Vegas Convention Center based on the analysis result of the voice command (1621) and one or more of the extracted action names or action descriptions. The processor (180) may generate recommendation text containing information about the obtained place to visit and may display a voice recognition recommendation screen (1640) containing the recommendation text (1641) on the display (151).

[0364] In this way, according to an embodiment of the present disclosure, an accurate and rich recommendation service can be provided to a user based on behavioral data of other users having similar tendencies.

[0365] Meanwhile, the processor (180) can receive user feedback regarding the recommended text (1641) and can adjust the first user embedding vector according to the received feedback. For example, if the processor (180) receives positive feedback, it can adjust the first user embedding vector so that the first user embedding vector becomes closer to the second user embedding vector.

[0366] As another example, if the processor (180) receives negative feedback, it can adjust the first user embedding vector so that the first user embedding vector is further away from the second user embedding vector.

[0367] As such, according to an embodiment of the present disclosure, as the user embedding vector is continuously adjusted to reflect real-time user feedback, personalization accuracy and suitability over time can be improved.

[0368] FIG. 17 is a diagram showing user behaviors categorized by clustering user embedding vectors according to one example of the present invention.

[0369] Referring to FIG. 17, a plurality of zones (1710 to 1740) are arranged in vector space.

[0370] Each of the multiple zones (1710 to 1740) may be a zone that clusters multiple user embedding vectors based on different behavioral regions.

[0371] Zone 1 (1710) may be a zone representing a group of customers exhibiting kids, child behavior patterns.

[0372] The second zone (1720) may be a zone representing a group of customers showing seniors' home appliance usage and luxury destination travel patterns.

[0373] The third zone (1730) may be a zone representing a group of customers exhibiting sports behavior patterns.

[0374] The fourth zone (1740) may be a zone representing a group of customers exhibiting travel behavior patterns and cost-effective home appliance usage behavior patterns.

[0375] Through clustering as in Fig. 17, the device (100) can generate customized recommendation services tailored to customers in each zone. For example, the device (100) can provide a service recommending premium domestic and international travel packages or high-end health functional foods to customers in the second zone (1720).

[0376] In addition, the device (100) can provide a service that recommends cost-effective, practical travel package products to customers in the fourth zone (1740).

[0377] As such, according to the embodiments of the present disclosure, customer behavior types are clearly classified and potential intentions of each type are predicted, thereby maximizing the accuracy of recommendations and marketing.

[0378] FIG. 18 is a drawing for explaining the architecture of a system according to one embodiment of the present disclosure.

[0379] Referring to FIG. 18, the system (1800) may include an AI server (200), a device (100), and an external server (1830). The external server (1830) may also be composed of the components of FIG. 2.

[0380] The AI ​​server (200) can be referred to as a LEAD platform server.

[0381] The AI ​​server (200) may include an enterprise knowledge database (1811), an embedding model (1812), and a downstream model (1813).

[0382] It may include a knowledge database (1811), an embedding model (1812), and a downstream model (1813).

[0383] The knowledge database (1811), embedding model (1812), and downstream model (1813) may be included in the memory (230) of FIG. 2. The memory (230) may be referred to as the database.

[0384] The knowledge database (1811) can store multiple behavior data sets corresponding to multiple behavior domains.

[0385] The embedding model (1812) may include a model that generates multiple user embedding vectors based on multiple behavior data sets. The embedding model (1812) may include the LBM (332) and encoder (343) of FIG. 3.

[0386] The downstream model (1813) can obtain attribute scores of attribute elements using multiple user embedding vectors output from the embedding model (1812), or perform user clustering by type and user classification by specific category.

[0387] The device (100) may include a voice recognition service app (1821), a voice processor (1822), an STT engine (1823), a TTS engine (1824), a first storage (1825), a second storage (1826), a vector searcher (1827), and an AI connector (1829).

[0388] A voice recognition service app (1821) may be an application that provides a voice recognition service by receiving user input, such as voice commands, and outputting a response according to the received user input. The front end manages the user interface, and the back end manages request processing and data flow.

[0389] The voice processor (1822) can process the user's voice command. The voice processor (1822) can generate voice data corresponding to the voice command and transmit the generated voice data to the STT engine (1823).

[0390] The STT engine (1823) can convert voice data into text data.

[0391] The TTS engine (1824) can convert response text data into voice data and can transmit the converted voice data to a voice processor (1822). The voice processor (1822) can transmit the received voice data to an app (1821).

[0392] The first storage (1825) can store relational data (e.g., user profile).

[0393] The second storage (1826) can store user embedding vectors and behavior embeddings received from the AI ​​server (200).

[0394] The first storage (1825) and the second storage (1826) may be included in the memory (170) of FIG. 1.

[0395] The vector searcher (1827) may include a configuration loader (1827a) and a searcher (1827b).

[0396] The configuration loader (1827a) can read a configuration file (1828) that stores settings and operating parameters to define and initialize the operation method of the searcher (1827b).

[0397] The searcher (1827b) can search for a second user embedding vector of a second user that has a certain degree of similarity to the first user embedding vector of a first user who uttered a voice command. The second user embedding vector may be a vector corresponding to an action region matched to the analysis result of the voice command.

[0398] The AI ​​connector (1829) may be an interface that relays communication with an external server (1830). The AI ​​connector (1829) may be included in the communication interface (110) of FIG. 1.

[0399] The external server (1830) may include an LLM (1831) and a database (1832).

[0400] The LLM (1831) can receive a prompt from the AI ​​connector (1829) and output a response result for the received prompt. The response result can be delivered to the AI ​​connector (1829).

[0401] The database (1832) can store larger or dynamically changing operational metadata or system instructions in addition to the settings stored in the configuration file (1828).

[0402] The functions of the elements disclosed in this disclosure may be implemented using circuits or processing circuits comprising general-purpose processors, special-purpose processors, integrated circuits, ASICs (Application-Specific Integrated Circuits), conventional circuits, and / or combinations thereof. A processor may be defined as a processing circuit or circuit comprising transistors and other circuits.

[0403] In the present invention, circuits, units, or means may be hardware designed or programmed to perform a specified function. The hardware may be the hardware disclosed in the present invention or other known hardware programmed or configured to perform a specified function. Where the hardware is a processor that can be considered a type of circuit, the circuits, means, or units may be a combination of hardware and software, and the software may constitute the hardware and / or the processor.

[0404] The above-described disclosure can be implemented as computer-readable code on a medium on which a program is recorded. A computer-readable medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include a Hard Disk Drive (HDD), a Solid State Disk (SSD), a Silicon Disk Drive (SDD), ROM, RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc. Additionally, the computer may include a processor (180) of an artificial intelligence device.

Claims

1. Regarding vehicles, A microphone for receiving voice commands from the first user; Output interface; and A processor comprising: acquiring a first user embedding vector based on a first behavioral data set corresponding to a first behavioral domain of a first user; acquiring a second user embedding vector corresponding to a second behavioral domain of a second user having a certain similarity or higher to the first user embedding vector based on the analysis result of the voice command; acquiring one or more attribute elements having a certain attribute score or higher among a plurality of attribute elements matched to the acquired second user embedding vector; and outputting a recommendation service based on the acquired one or more attribute elements and the analysis result through the output interface. vehicle.

2. In Paragraph 1, The above processor Extracting the second behavior area that matches the analysis result based on the analysis result of the voice command among a plurality of behavior areas vehicle.

3. In Paragraph 1, The above processor Acquiring a second behavior data set corresponding to the second behavior area of ​​the second user and serving as the basis for the generation of the second user embedding vector, and generating the recommendation service based on the acquired second behavior data set and the analysis result. vehicle.

4. In Paragraph 3, The above processor Extracting the attribute element with the largest attribute score among a plurality of attribute elements matched to the second user embedding vector, obtaining behavioral data for unit behaviors matched to the extracted attribute element, and generating the recommendation service based on the obtained behavioral data and the analysis result. vehicle.

5. In Paragraph 1, The above processor One or more of the above-mentioned attribute elements and the above-mentioned analysis results are input into an interactive agent to obtain the above-mentioned recommendation service. vehicle.

6. In Paragraph 1, It further includes memory for storing multiple user embedding vectors, and The above processor Comparing the first user embedding vector with the plurality of user embedding vectors, and obtaining the second user embedding vector having a certain similarity or greater according to the comparison result. vehicle.

7. In Paragraph 1, The above recommended service including one or more of a destination recommendation service, a waypoint recommendation service, a content recommendation service, or an activity recommendation service vehicle.

8. In Paragraph 1, The above-mentioned first action area is It is an area representing actions performed for the management of home appliances, and The above second action area is an area representing behaviors related to the user's location movement vehicle.

9. In a non-transient recording medium storing computer-readable instructions that, when executed by a device, cause said device to perform operations, The above operations are A step of obtaining a first user embedding vector based on a first behavior data set corresponding to a first behavior domain of the first user; A step of receiving a voice command from the first user; A step of obtaining a second user embedding vector corresponding to a second behavioral domain of a second user having a certain degree of similarity or greater than that of the first user embedding vector based on the analysis result of the received voice command; A step of obtaining one or more attribute elements having a certain attribute score or more among a plurality of attribute elements matched to the second user embedding vector obtained above; and A step comprising outputting a recommendation service based on one or more acquired attribute elements and the analysis result. Non-temporary recording media.

10. In Paragraph 9, The above operations are The method further includes the step of extracting the second behavior area corresponding to the analysis result based on the analysis result of the voice command among a plurality of behavior areas. Non-temporary recording media.

11. In Paragraph 9, The above operations are A step of obtaining a second behavior data set corresponding to the second behavior area of ​​the second user and forming the basis for the generation of the second user embedding vector; and The method further includes the step of generating the recommendation service based on the second behavioral data set obtained above and the analysis result. Non-temporary recording media.

12. In Paragraph 11, The step of creating the above recommendation service is A step of extracting the attribute element with the largest attribute score among a plurality of attribute elements matched to the second user embedding vector, and A step comprising obtaining behavioral data for unit behaviors matched to extracted attribute elements, and generating the recommendation service based on the obtained behavioral data and the analysis results. Non-temporary recording media.

13. In Paragraph 9, The above operations are The method further includes the step of obtaining the recommendation service by inputting one or more of the acquired attribute elements and the analysis results into an interactive agent. Non-temporary recording media.

14. In Paragraph 9, The step of obtaining the second user embedding vector above A step of comparing the above-mentioned first user embedding vector with a plurality of previously stored user embedding vectors and The step of obtaining the second user embedding vector having a certain similarity or greater according to the comparison result Non-temporary recording media.

15. In Paragraph 9, The above recommended service including one or more of a destination recommendation service, a waypoint recommendation service, a content recommendation service, or an activity recommendation service Non-temporary recording media.

Images