Server for providing related context information in real time while transmitting and receiving chat message to and from user equipment and operation method thereof

Abstract

According to various embodiments, a server for providing related context information in real time while transmitting and receiving a chat message to and from a user equipment by using an intelligent agent may comprise: a communication module; and a processor, wherein the processor is configured to execute instructions for: causing the user equipment to display at least one chat message transmitted and received to and from a large language model (LLM)-based intelligent agent on a chat window screen; in response to identifying first context information related to a travel itinerary from a first user message received from the user equipment, causing the user equipment to display the first context information on a travel itinerary display window screen; and while outputting a first agent message including at least a part of the first context information and first search result information corresponding to a specific query on the chat window screen through the intelligent agent, causing the user equipment to display, on a context display window screen, at least one first object for second context information corresponding to the first search result information. Other embodiments are also possible.

Description

A server and its operating method for providing real-time contextual information while transmitting and receiving chat messages with a user device

[0001] Various embodiments of the present disclosure relate to a server and its operating method that provides real-time relevant context information while transmitting and receiving chat messages with a user device.

[0002] Various embodiments of the present disclosure relate to a server that provides package travel products while transmitting and receiving chat messages with a user device through an intelligent agent, and a method of operating the same.

[0003] Various embodiments of the present disclosure relate to an electronic device and a method of operating the same for generating training data to be used in training a large-scale language model.

[0004] Recently, with technological advancements in artificial intelligence, especially natural language understanding, the development and utilization of conversational AI agent systems have been increasing. These systems allow users to operate machines and obtain desired services through more human-friendly methods, such as conversations using natural language in the form of voice and / or text, moving beyond traditional machine-centric command input / output methods. Accordingly, in various fields, including (but not limited to) online consultation centers and online shopping malls, users can now request desired services from conversational AI agent systems and obtain desired results through natural language conversations in the form of voice and / or text.

[0005] A server according to the present disclosure can provide a service that allows a user to plan a travel itinerary in an interactive manner and receive recommendations for personalized travel products by combining a large language model (LLM), travel-related information, and user interface (UI) technology.

[0006] The server according to the present disclosure can, while transmitting and receiving chat messages with a user device through an intelligent agent, check a package travel product in which the user device freely combines travel products, reasonably determine a discount price for the package travel product, and provide the package travel product.

[0007] An electronic device according to the present disclosure can provide a method for generating virtual text data specialized for various domains suitable for training a large-scale language model using rules for generating training data based on probability and state, and automatically expanding the training data by utilizing the large-scale language model.

[0008] According to various embodiments, a server for providing real-time context information while transmitting and receiving chat messages with a user device using an intelligent agent includes a communication module and a processor, wherein the processor is configured to execute instructions for causing the user device to display at least one chat message transmitted and received with an intelligent agent based on a large language model (LLM) on a chat window screen, and in response to identifying first context information related to a travel itinerary from a first user message received from the user device, causing the user device to display the first context information on a travel itinerary display screen, and outputting a first agent message including at least a portion of the first context information and first search result information corresponding to a specific query through the intelligent agent on the chat window screen, while causing the user device to display at least one first object for second context information corresponding to the first search result information on the context display screen.

[0009] According to various embodiments, a method for operating a server that provides real-time related context information while transmitting and receiving chat messages with a user device using an intelligent agent may include causing the user device to display at least one chat message transmitted and received with an intelligent agent based on a large language model (LLM) on a chat window screen, causing the user device to display first context information related to a travel itinerary on a travel itinerary display window screen in response to identifying first context information related to a travel itinerary from a first user message received from the user device, and causing the user device to display at least one first object for second context information corresponding to the first search result information on the context display window screen while outputting a first agent message including at least a portion of the first context information and first search result information corresponding to a specific query on the chat window screen through the intelligent agent.

[0010] According to various embodiments, a computer program product comprising one or more programs configured to be executed by one or more processors of a computer system, the one or more programs including instructions for: causing a user device to display at least one chat message transmitted and received with an intelligent agent based on a large language model (LLM) on a chat window screen; causing the user device to display the first context information on a travel itinerary display window screen in response to identifying first context information related to a travel itinerary from a first user message received from the user device; and causing the user device to display at least one first object for second context information corresponding to the first search result information on a context display window screen while outputting a first agent message including at least a portion of the first context information and first search result information corresponding to a specific query on the chat window screen through the intelligent agent.

[0011] According to various embodiments, a server providing a package travel product while transmitting and receiving a chat message to and from a user device through an intelligent agent includes a communication module and a processor, and the processor is configured to execute instructions that cause the user device to display at least one chat message transmitted and received with an intelligent agent based on a large language model (LLM) on a chat window screen, identify a request for creating a package travel product combining at least two travel products from a user message received from the user device, determine price information of the package travel product using first context information related to a travel itinerary, and cause the user device to display an agent message including the price information of the package travel product on the chat window screen through the intelligent agent.

[0012] According to various embodiments, a method of operating a server providing a package travel product while transmitting and receiving a chat message to and from a user device through an intelligent agent may include: causing the user device to display at least one chat message transmitted and received with an intelligent agent based on a large language model (LLM) on a chat window screen; identifying a request for creating a package travel product combining at least two travel products from a user message received from the user device; determining price information of the package travel product using first context information related to a travel schedule; and causing the user device to display an agent message including the price information of the package travel product on a chat window screen through the intelligent agent.

[0013] According to various embodiments, a computer program product comprising one or more programs configured to be executed by one or more processors of a computer system, wherein the one or more programs may include instructions for: causing a user device to display at least one chat message transmitted and received with an intelligent agent based on a large language model (LLM) on a chat window screen; identifying a request for creating a package travel product combining at least two travel products from a user message received from the user device; determining price information of the package travel product using first context information related to a travel itinerary; and causing the user device to display an agent message including the price information of the package travel product on the chat window screen through the intelligent agent.

[0014] According to various embodiments, an electronic device for generating training data to be used for training a large-scale language model includes a display and a processor, wherein the processor is configured to generate, using an application for generating the training data, at least one node among a storage node, a switch node, a branch node, and an output node, generate a node flow that sets a transition relationship between the at least one node and a variable-related value for each of the at least one node, and generate the training data according to a result of executing the node flow in response to a request for generating the training data, wherein the training data may include prompt data corresponding to a virtual user message and completion data corresponding to a virtual agent message.

[0015] According to various embodiments, a method of operating an electronic device for generating training data to be used for training a large-scale language model includes an operation of generating at least one node among a storage node, a switch node, a branch node, and an output node using an application for generating the training data, an operation of generating a node flow that sets a transition relationship between the at least one node and a variable-related value for each of the at least one node, and an operation of generating the training data according to a result of executing the node flow in response to a request for generating the training data, wherein the training data may include prompt data corresponding to a virtual user message and completion data corresponding to a virtual agent message.

[0016] According to various embodiments, a computer program product comprising one or more programs configured to be executed by one or more processors of a computer system, wherein the one or more programs include instructions for: using an application for generating training data to be used for training a large-scale language model, generating at least one node among a storage node, a switch node, a branch node, and an output node; generating a node flow that sets a transition relationship between the at least one node and a variable-related value for each of the at least one node; and generating the training data according to a result of executing the node flow in response to a request for generating the training data, wherein the training data may include prompt data corresponding to a virtual user message and completion data corresponding to a virtual agent message.

[0017] The present disclosure provides information suitable for the user's request by smoothly conducting a conversation with the user through an intelligent agent based on a large-scale language model, and by providing contextual information related to the conversation with the user in real time through an intuitive UI, thereby providing the effect of increasing user convenience by displaying content that is easy for the user to recognize.

[0018] The present disclosure can provide the effect of stimulating the user's desire to travel and increasing the cost-effectiveness of the travel itinerary by determining appropriate discount price information when the user requests a package travel product by freely combining travel products during a conversation with an intelligent agent.

[0019] In order to train a large-scale language model, a large number of user messages and a large number of agent messages configured in pairs are required. According to the present disclosure, by automatically generating a large amount of training data configured with virtual user messages and virtual agent messages without requiring the user to directly input user messages or agent messages, the training efficiency of a large-scale language model can be improved.

[0020] FIG. 1 illustrates a block diagram of a user device and a server according to various embodiments of the present disclosure.

[0021] FIG. 2 is a flowchart illustrating an operation in which a server provides relevant context information in real time during a conversation with a user device through an intelligent agent according to various embodiments.

[0022] FIGS. 3A to 3D are diagrams illustrating a first embodiment in which a server displays an object for context information in real time during a conversation with a user device through an intelligent agent, according to various embodiments.

[0023] FIGS. 4A to 4D are diagrams illustrating a second embodiment in which a server displays an object for context information in real time during a conversation with a user device through an intelligent agent, according to various embodiments.

[0024] FIGS. 5A to 5G are diagrams for explaining an embodiment in which a server changes and displays an object for context information according to a change in the content of a user message during a conversation with a user device through an intelligent agent, according to various embodiments.

[0025] FIG. 6 is a diagram illustrating an embodiment in which a server generates a user message to be input to an intelligent agent based on user input confirmed through a context display window screen according to various embodiments.

[0026] FIG. 7 is a drawing for explaining an embodiment in which a server according to various embodiments provides multiple contents through a context display window screen divided into multiple areas.

[0027] FIG. 8 is a flowchart illustrating an operation of a server providing a package travel product according to various embodiments.

[0028] FIGS. 9A to 9C illustrate a first embodiment in which a server provides a package travel product according to various embodiments.

[0029] FIGS. 10A and 10B illustrate a first embodiment in which a server provides a package travel product according to various embodiments.

[0030] FIG. 11 is a flowchart illustrating an operation of an electronic device to generate training data to be used for training a large-scale language model, according to various embodiments.

[0031] FIG. 12 is a diagram showing a GUI for each type of node used to generate training data according to various embodiments.

[0032] FIG. 13 is a diagram showing a node flow object that implements node flow in GUI form according to various embodiments.

[0033] FIG. 14 is a diagram illustrating a training data set according to various embodiments.

[0034] FIG. 15a is a diagram illustrating a node flow object for generating training data for an LLM for translation according to various embodiments.

[0035] FIG. 15b is a diagram illustrating an example of a training data entry of an LLM for translation generated by the node flow object of FIG. 15a according to various embodiments.

[0036] FIG. 16a is a diagram illustrating a node flow object for generating training data for a summary LLM according to various embodiments.

[0037] FIG. 16b is a diagram illustrating an example of a training data entry of a summary LLM generated by the node flow object of FIG. 16a according to various embodiments.

[0038] Hereinafter, various embodiments of the present document will be described with reference to the attached drawings. It should be understood that the embodiments and the terms used therein are not intended to limit the technology described in the present document to a specific embodiment, but rather include various modifications, equivalents, and / or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar components. The singular expression may include plural expressions unless the context clearly indicates otherwise. In this document, expressions such as "A or B" or "at least one of A and / or B" may include all possible combinations of the items listed together. Expressions such as "first," "second," "first," or "second," may modify the corresponding components regardless of order or importance, and are only used to distinguish one component from another, but do not limit the corresponding components. When it is said that a component (e.g., a first component) is “(functionally or communicatively) connected” or “connected” to another component (e.g., a second component), said component may be directly connected to said other component, or may be connected via another component (e.g., a third component).

[0039] In this document, "configured to" may be used interchangeably with, for example, "suitable for," "capable of," "modified to," "made to," "capable of," or "designed to," either in hardware or software. In some contexts, the phrase "a device configured to" may mean that the device is "capable of" doing something together with other devices or components. For example, the phrase "a processor configured to perform A, B, and C" may mean a dedicated processor (e.g., an embedded processor) for performing the operations, or a general-purpose processor (e.g., a CPU or application processor) that can perform the operations by executing one or more software programs stored in a memory device.

[0040] A user device or electronic device according to various embodiments of the present document may include, for example, at least one of a smartphone, a tablet PC, a desktop PC, a laptop PC, a netbook computer, a workstation, and a server.

[0041] Referring to FIG. 1, a user device (100) and a server (101) in various embodiments are described. The user device (100) may include a communication module (110), a processor (120), a memory (130), and a display (140). In some embodiments, the user device (100) may omit at least one of the components or additionally include other components.

[0042] The communication module (110) can establish communication between, for example, the user device (100) and an external device (e.g., a first external electronic device (102), a second external electronic device (104), or a server (101)). For example, the communication module (110) can be connected to a network (180) via wireless communication or wired communication to communicate with the external device (e.g., a second external electronic device (104) or a server (101)).

[0043] The wireless communication may include, for example, cellular communication using at least one of LTE, LTE-A (LTE Advance), CDMA (code division multiple access), WCDMA (wideband CDMA), UMTS (universal mobile telecommunications system), WiBro (Wireless Broadband), or GSM (Global System for Mobile Communications). In one embodiment, the wireless communication may include, for example, at least one of WiFi (wireless fidelity), Bluetooth, Bluetooth low energy (BLE), Zigbee, near field communication (NFC), Magnetic Secure Transmission, radio frequency (RF), or body area network (BAN). In one embodiment, the wireless communication may include GNSS. The GNSS may be, for example, GPS (Global Positioning System), Glonass (Global Navigation Satellite System), Beidou Navigation Satellite System (hereinafter "Beidou"), or Galileo, the European global satellite-based navigation system. Hereinafter, in this document, "GPS" may be used interchangeably with "GNSS." Wired communication may include at least one of, for example, USB (universal serial bus), HDMI (high definition multimedia interface), RS-232 (recommended standard 232), power line communication, or POTS (plain old telephone service).The network (180) may include at least one of a telecommunications network, for example, a computer network (e.g., a LAN or WAN), the Internet, or a telephone network.

[0044] The processor (120) may include one or more of a central processing unit, an application processor, or a communication processor (CP). The processor (120) may, for example, perform operations or data processing related to control and / or communication of at least one other component of the user device (100).

[0045] The memory (130) may include volatile and / or non-volatile memory. The memory (130) may store, for example, commands or data related to at least one other component of the user device (100).

[0046] The display (140) may include, for example, a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a micro electro mechanical systems (MEMS) display, or an electronic paper display. The display (140) may, for example, display various contents (e.g., text, images, videos, icons, and / or symbols) to the user. The display (160) may include a touch screen and may receive touch, gesture, proximity, or hovering inputs using, for example, an electronic pen or a part of the user's body.

[0047] Each of the first and second external electronic devices (102, 104) may be the same or a different type of device as the user device (100). According to various embodiments, all or part of the operations executed in the user device (100) may be executed in another one or more electronic devices (e.g., electronic devices (102, 104) or server (101). According to one embodiment, when the user device (100) needs to perform a certain function or service automatically or upon request, the user device (100) may request at least some functions related thereto from another device (e.g., electronic devices (102, 104) or server (101)) instead of executing the function or service by itself or in addition. The other electronic device (e.g., electronic devices (102, 104) or server (101)) may execute the requested function or additional function and transmit the result to the user device (100). The user device (100) may process the received result as is or additionally to provide the requested function or service. For this purpose, for example, cloud computing, distributed computing, or client-server computing technology may be used.

[0048] The server (101) may include a communication module (111), a processor (121), and a memory (131). In some embodiments, the server (101) may omit at least one of the components or additionally include other components. The communication module (111), the processor (121), and the memory (131) may perform the same functions as the communication module (110), the processor (120), and the memory (130) within the user device (100), respectively.

[0049]

[0050] FIG. 2 is a flowchart illustrating an operation in which a server (e.g., server (101) of FIG. 1) provides relevant context information in real time during a conversation with a user device through an intelligent agent, according to various embodiments.

[0051] FIGS. 3A to 3D are diagrams for explaining a first embodiment in which a server (101) displays an object for context information in real time during a conversation with a user device (e.g., the user device (100) of FIG. 1) through an intelligent agent, according to various embodiments.

[0052] FIGS. 4A to 4D are diagrams for explaining a second embodiment in which a server (101) displays an object for context information in real time during a conversation with a user device (100) through an intelligent agent according to various embodiments.

[0053] According to various embodiments, the server (101) may operate an application composed of a plurality of execution screens or a website composed of a plurality of web pages, and communicate with a user device (e.g., an electronic device (100, 102, 104) of FIG. 1) (e.g., a PC, a laptop, a smartphone, etc.) via a network (162, 164), process a request received from the user device (100) regarding the application or the web page, and transmit the requested information to the user device (100). The server (101) may exchange conversations with the user device (100) through an intelligent agent, and transmit source code that enables the user device (100) to display each execution screen of a dedicated application or website that provides related context information, and the user device (101) may receive the source code and display the execution screen requested by the user of the user device (100) through the dedicated application or a web browser. According to one embodiment, the server (101) may include the same types of components as the components of the electronic device (100) of FIG. 1. According to one embodiment, the components referred to as user devices (100) in the present disclosure may refer to a user account that accesses a platform provided by the server (101) through the user device.

[0054]

[0055] In operation 201, according to various embodiments, the server (101) (e.g., the processor (121) of FIG. 1) may cause the user device (100) to display at least one chat message transmitted and received with an intelligent agent based on a large language model (LLM) on a chat window screen.

[0056] According to one embodiment, the large-scale language model may include, for example, a transformer-based neural network model, and may be used in (1) natural language understanding (NLU) fields that understand the meaning of sentences and perform natural language understanding tasks such as keyword extraction, sentiment analysis, and information retrieval, (2) natural language generation (NLG) fields that generate natural language in various forms such as sentences, paragraphs, summaries, and translations, (3) question answering fields that generate answers to given questions, (4) machine translation fields that perform translation tasks between multiple languages, and (5) text summarization fields that concisely summarize long texts to increase the value of information. In addition to the examples described above, the large-scale language model may include various language models that can be easily realized by those skilled in the art.

[0057] According to one embodiment, the server (101) may execute an intelligent agent upon a request from the user device (100), and may transmit and receive chat messages with the user device (100) through the intelligent agent. For example, referring to FIG. 3A, the server (101) may cause the user device (100) to display a chat window screen (310) that displays a chat message between the user device (100) and the intelligent agent through the display (140). According to one embodiment, an intelligent agent linked with a large-scale language model may input a user message obtained from the user device (100) into the large-scale language model, and generate an agent message processed in the form of a reply to the user message. The server (101) may transmit the agent message to the user device (100), and may display the chat message between the user device (100) and the intelligent agent on the chat window screen (310) of the user device (100). For example, referring to FIG. 3A, a chat window screen (310) may include a text input field (311) for receiving text input from a user, and the server (101) may display a user message (312) input by a user through the text input field (311) and an agent message (313) in response to a chat message (312) from an intelligent agent through the chat window screen (310).

[0058] According to one embodiment, the counterparty transmitting and receiving chat messages with the user device (100) may be not only an intelligent agent, but also another user device (104) (e.g., another general user or an employee in charge of customer service (CS) of an application operated by the server (101), and this is only one example and is not limited to the above example. According to one embodiment, the server (101) may perform an operation of analyzing chat messages between the user devices (100, 104) and displaying related context information according to the present disclosure.

[0059]

[0060] In operation 203, according to various embodiments, the server (101) (e.g., the processor (121) of FIG. 1) may, in response to identifying first context information related to a travel itinerary from a first user message received from the user device (100), cause the user device (100) to display the first context information on a travel itinerary display window screen.

[0061] According to one embodiment, the server (101) may identify first context information related to a travel itinerary from a first user message. According to one embodiment, the first context information related to the travel itinerary may include at least one of travel region information, travel date information, or travel number information. The travel region information may include information by continent, country, city, administrative district, or a combination thereof registered in the server (101). The travel date information may include at least one of travel date information (e.g., 2023.M1.D1 ~ M2.D2), travel period information corresponding to the travel date information (e.g., m nights or m nights n days), travel day information corresponding to the travel date information (e.g., travel start day of the week, travel end day of the week, days of the week during the travel period), or travel month information belonging to the travel date information (e.g., at least one of January to December). The travel number information may include information regarding the number of travelers, such as “n adults” and “m children,” and may include information regarding the ages and number of travelers. The examples of the first context information related to the travel schedule described above are only examples and are not limited to the above examples, and the types / items thereof may be freely changed according to the settings of the user of the user device (100) or the administrator of the server (101).

[0062] According to one embodiment, in response to identifying the first context information through the intelligent agent, the server (101) may cause the user device (100) to display the first context information on the travel itinerary display screen. For example, referring to FIG. 3B, the server (101) may identify travel region information (321) (e.g., Gangneung), travel date information (322) (e.g., 11.4 (Sat) - 11.5 (Sun), 1 night), and travel number information (323) (e.g., 2 adults, 1 child) as the first context information from the first user message (312a) (e.g., "I'm planning to go to Gangneung with my husband, myself, and a child. Let's go on November 4th for 1 night and 2 days"), and cause the user device (100) to display the first context information on the travel itinerary display screen (320).

[0063]

[0064] In operation 205, according to various embodiments, the server (101) (e.g., the processor (121) of FIG. 1) may cause the user device (100) to display at least one first object for second context information corresponding to the first search result information on the context display window screen while outputting a first agent message including at least a portion of the first context information and first search result information corresponding to a specific query on the dialog box screen through the intelligent agent.

[0065] According to one embodiment, the server (101) can check first search result information determined based on at least a portion of the first context information and a specific query by the intelligent agent. According to one embodiment, the intelligent agent can obtain the first search result information by searching for at least a portion of the first context information and a specific query regarding the current conversation session from a database (e.g., a travel information-related DB) linked to a large-scale language model or a large-scale language model trained using the database. The database storing the travel-related information can be implemented in the form of a component within the server (101) or can be implemented in the form of a separate component external to the server (101). Specifically, referring to FIG. 3b, the intelligent agent can check first search result information (e.g., first accommodation information, second accommodation information) by using travel region information (e.g., Gangneung) of the first context information and a specific query (e.g., accommodation information) regarding the current conversation session. Here, a specific query may represent a conversation topic (i.e., a search keyword) that the intelligent agent wishes to confirm in a conversation with the user device (100), and may be determined by the intelligent agent's settings or by the analysis results of messages obtained from the user device (100). According to one embodiment, the specific query may be transportation information, accommodation information, tourist attraction information, restaurant information, tour information, or ticket information. For example, the specific query in FIGS. 3A to 3D may be "accommodation information."

[0066] According to one embodiment, the server (101) may input a first user message into a large-scale language model via an intelligent agent, thereby generating a first agent message including first search result information as a response message obtained. For example, referring to FIG. 3b, the server (101) may cause the user device (100) to display a first agent message (313a) including first search result information on a chat window screen (310).

[0067] According to one embodiment, the second context information may include at least one of location information (e.g., address information), web link information (e.g., web page URL), or app link information (e.g., app page URL) corresponding to the first search result information. According to one embodiment, the second context information corresponding to the first search result information may further include application execution information that will represent the second context information. For example, the second context information may include address information and map app execution information. As another example, the second context information may include web link information and web browser execution information. The examples of the second context information described above are merely examples, and are not limited to the above examples, and the types / items thereof may be freely changed according to the settings of the administrator of the server (101).

[0068] According to one embodiment, the server (101) may cause the user device (100) to display a first agent message on the chat window screen (310) through the intelligent agent, while displaying at least one first object for second context information on the context display window screen. For example, referring to FIG. 3B, the server (101) may cause the user device (100) to display a first agent message (313a) on the chat window screen (310) through the intelligent agent, while executing a map app on the context display window screen (330) to display location indicators (331, 332) for second context information (e.g., location information of the first accommodation and location information of the second accommodation).

[0069] According to one embodiment, the server (101) may cause the user device (100) to display at least one additional message that assists the first agent message through the intelligent agent. For example, referring to FIGS. 3B and 3C , the server (101) may cause the user device (100) to display additional messages (314a, 315a, 314b, 315b) after outputting the first agent message (313a). According to one embodiment, the at least one additional message may be implemented in the form of a widget for either the first format or the second format. For example, referring to FIG. 3C , the additional messages (314a, 314b) of the first format may be configured to include a name, a review, a location, a price, and a review, and the additional messages (314b, 315b) of the second format may be configured to include a name, a thumbnail image, a price, and a review. According to one embodiment, the server (101) may, in response to the user device (100) selecting at least one additional message, display a detailed page regarding the selected additional message. For example, referring to FIGS. 3C and 3D , the server (101) may, in response to the user device (100) selecting an additional message (314a or 314b), display a detailed page including multiple information items regarding the additional message (314a or 314b) through the context display window screen (330). That is, the server (101) may confirm a request (e.g., user interaction regarding the additional message (314a or 314b)) of the user device (100) generated through the conversation window screen (310), and perform an action corresponding to the request (e.g., displaying a detailed information screen) through the context display window screen (330).

[0070]

[0071] In operation 207, according to various embodiments, the server (101) (e.g., the processor (121) of FIG. 1 ) may, in response to identifying third context information related to a travel location condition from a second user message received from the user device (100), output a second agent message including at least a portion of the first context information, the third context information, and second search result information corresponding to a specific query on a dialog window screen (310), while causing the user device (100) to display at least one second object for fourth context information corresponding to the second search result information on a context display window screen (330).

[0072] According to one embodiment, the server (101) can identify third context information related to a travel location condition from the second user message. According to one embodiment, the third context information related to the travel location condition can include at least one of a review score condition, a price range condition, a location condition, a place characteristic condition, an age range condition, a gender condition, a number of people condition, an accommodation type (e.g., hotel / motel, etc.) condition, an accommodation grade (e.g., hotel star rating) condition, or an amenity (e.g., availability of breakfast / Wi-Fi, etc.) condition, and the above-described example is only one example, and the present invention is not limited thereto, and the type / item thereof can be freely changed by the settings of a user of the user device (100) or an administrator of the server (101). For example, referring to FIG. 4a, the server (101) can identify a price condition (e.g., less than 150,000 won) as third context information from a second user message (312b) (e.g., "Good, but it's a bit expensive. Are there any places that cost less than 150,000 won per night?").

[0073] According to one embodiment, the server (101) may, in response to identifying third context information through the intelligent agent, cause the user device (100) to display the third context information on the travel itinerary display screen.

[0074] According to various embodiments, the server (101) (e.g., the processor (121) of FIG. 1) may, in response to identifying the third context information, output a second agent message including at least a portion of the first context information, the third context information, and second search result information corresponding to a specific query to the dialog box screen (310).

[0075] According to one embodiment, the server (101) can verify second search result information determined based on at least a portion of the first context information, third context information, and a specific query by the intelligent agent. For example, referring to FIG. 4A, the intelligent agent can verify second search result information (e.g., third accommodation information, fourth accommodation information) using travel region information (e.g., Gangneung) of the first context information, price conditions (e.g., less than 150,000 won) of the third context information, and a specific query (e.g., accommodation information) regarding the current conversation session.

[0076] According to one embodiment, the server (101) may input a second user message into a large-scale language model via an intelligent agent, thereby generating a second agent message including second search result information as a response message obtained. For example, referring to FIG. 4A, the server (101) may cause the user device (100) to display a second agent message (313b) including second search result information on the chat window screen (310).

[0077] According to one embodiment, the server (101) may cause the user device (100) to display at least one second object for fourth context information corresponding to the second search result information on the context display screen (330) while outputting the second agent message on the chat window screen (310). The description regarding the fourth context information may be applied to the description regarding the second context information in operation 205. For example, referring to FIG. 4A, the server (101) may cause the user device (100) to display the second agent message (313b) on the chat window screen (310) through the intelligent agent while displaying location indicators (431, 432) for the fourth context information (e.g., location information of the third accommodation and location information of the fourth accommodation) within the running map app screen on the context display screen (330).

[0078] According to various embodiments, the server (101) may, in response to identifying a specific query from a third user message received from the user device (100), check at least one context information and third search result information corresponding to another specific query. For example, referring to FIG. 4B, the server (101) may, in response to identifying a specific query (e.g., activity information) from a third user message (312c) (e.g., “Are there any activities that can be done with children?”), check third search result information (e.g., first activity information and second activity information) determined by the intelligent agent based on the travel area information (e.g., Gangneung) of the first context information, the age condition (e.g., children) of the third context information, and the specific query (e.g., activity information).

[0079] According to one embodiment, the server (101) may input a third user message into a large-scale language model via an intelligent agent, thereby generating a third agent message including third search result information as a response message obtained. For example, referring to FIG. 4b, the server (101) may cause the user device (100) to display a third agent message (313c) including third search result information on the chat window screen (310).

[0080] According to one embodiment, the server (101) may cause the user device (100) to display at least one object for fifth context information corresponding to the third search result information on the context display screen (330) while outputting the third agent message on the chat window screen (310). The description of the fifth context information may be applied to the description of the second context information in operation 205. For example, referring to FIG. 4b, the server (101) may cause the user device (100) to display the third agent message (313c) on the chat window screen (310) through the intelligent agent, while displaying location indicators (441, 442) for the fifth context information (e.g., location information of the first activity and location information of the second activity) within the running map app screen on the context display screen (330).

[0081] According to one embodiment, in response to identifying new search result information, the server (101) may switch a first screen including an object for context information corresponding to previous search result information to a second screen including an object for context information corresponding to the new search result information. That is, whenever new search result information is identified, the server (101) may cause the user device (100) to immediately switch a first screen including an object for context information corresponding to previous search result information (e.g., context display window screen (330) of FIG. 4a) to a second screen including an object for context information corresponding to the new search result information (e.g., context display window screen (330) of FIG. 4b).

[0082] According to one embodiment, the server (101) may cause the user device (100) to display at least one additional message that assists the agent message through the intelligent agent. For example, referring to FIGS. 4B and 4C , the server (101) may cause the user device (100) to display additional messages (414a, 415a, 414b, 415b) after the user device (100) outputs the third agent message (313c). According to one embodiment, the at least one additional message may be implemented in the form of a widget for one of the first format or the second format, such as the additional messages (314a, 315a, 314b, 315b). According to one embodiment, the server (101) may cause the user device (100) to display a detail page regarding the selected additional message in response to the selection of at least one additional message. For example, referring to FIGS. 4c and 4d , in response to the user device (100) selecting an additional message (415b), the server (101) may display a detailed page including multiple information items regarding the additional message (415a or 415b) through the context display window screen (330). That is, the server (101) may confirm a request (e.g., user interaction regarding the additional message (415a or 415b)) of the user device (100) generated through the dialog window screen (310), and perform an action corresponding to the request (e.g., displaying a detailed information screen) through the context display window screen (330).

[0083]

[0084] FIGS. 5A to 5G are drawings for explaining an embodiment in which a server (e.g., server (101) of FIG. 1) changes and displays an object for context information according to a change in the content of a user message during a conversation with a user device (e.g., user device (100) of FIG. 1) through an intelligent agent, according to various embodiments.

[0085] According to various embodiments, the server (101) (e.g., the processor (121) of FIG. 1) may cause the user device (100) to display at least one chat message transmitted and received with an intelligent agent based on a large language model (LLM) on a chat window screen. For example, referring to FIG. 5A, the server (101) may receive a first user message (512a) (e.g., "I'm going to Busan or Gyeongju alone, so can you tell me what there is to do? Let's start with Busan") from the user device (100), and cause the user device (100) to display the first user message (512a) on the chat window screen (310).

[0086] According to one embodiment, in response to identifying first context information related to a travel itinerary from a first user message received from the user device (100), the server (101) may cause the user device (100) to display the first context information on the travel itinerary display window screen (320). For example, the server (101) may identify travel region information (521) (e.g., Busan) as the first context information from the first user message (512a), and cause the user device (100) to display the first context information on the travel itinerary display window screen (320).

[0087] According to one embodiment, the server (101) may input a first user message into a large-scale language model via an intelligent agent, thereby generating a first agent message including first search result information as a response message obtained. For example, referring to FIG. 5A, the server (101) may cause the user device (100) to display a first agent message (513a) including first search result information on the chat window screen (310).

[0088] According to various embodiments, in response to identifying a specific query from a second user message received from the user device (100), the server (101) may verify at least one context information and second search result information corresponding to the specific query. For example, referring to FIG. 5b, in response to identifying a specific query (e.g., tour information) from a second user message (512b) (e.g., “The exhibition or performance looks good”), the server (101) may verify second search result information (e.g., first tour information and second tour information) determined by the intelligent agent based on the travel region information (e.g., Busan) of the first context information and the specific query (e.g., tour information).

[0089] According to one embodiment, the server (101) may input a second user message into a large-scale language model via an intelligent agent, thereby generating a second agent message including second search result information as a response message obtained. For example, referring to FIG. 5b, the server (101) may cause the user device (100) to display a second agent message (513b) including second search result information on the chat window screen (310).

[0090] According to one embodiment, the server (101) may cause the user device (100) to display at least one object for context information corresponding to the second search result information on the context display screen (330) while outputting the second agent message on the chat window screen (310). For example, referring to FIG. 5b, the server (101) may cause the user device (100) to display the second agent message (513b) on the chat window screen (310) through an intelligent agent, while displaying location indicators (531, 532) for context information (e.g., location information of the first tour and location information of the second tour) corresponding to the second search result information on the context display screen (330) within the running map app screen.

[0091] According to one embodiment, in response to identifying the first context information and a specific query together, the server (101) may cause the user device (100) to display the first context information on the travel itinerary display screen (320). For example, referring to FIGS. 5A and 5B , the server (101) may identify the first context information (e.g., Busan) from the first user message (512a) and the specific query (e.g., tour information) from the second user message (513b), and then cause the user device (100) to display the first context information (521) (e.g., Busan) on the travel itinerary display screen (320). In addition to the action of identifying the specific query from the user message in the above example, the action may include an action identified from an agent message by the intelligent agent (e.g., accommodation information of message 313).

[0092] According to various embodiments, the server (101) (e.g., the processor (121) of FIG. 1) may, in response to identifying other first context information related to the travel itinerary from a third user message received from the user device (100), check third search result information corresponding to the other first context information and the latest specific query. For example, referring to FIG. 5c, the server (101) may, in response to identifying context information (e.g., Gyeongju) related to the travel itinerary from a third user message (512c) (e.g., “What’s Gyeongju?”), check third search result information (e.g., third tour information and fourth tour information) determined by the intelligent agent based on the other first context information (e.g., Gyeongju) and the most recently identified specific query (e.g., tour information).

[0093] According to one embodiment, the server (101) may input a third user message into a large-scale language model via an intelligent agent, thereby generating a third agent message including third search result information as a response message obtained. For example, referring to FIG. 5c, the server (101) may cause the user device (100) to display a third agent message (513c) including third search result information on the chat window screen (310).

[0094] According to one embodiment, the server (101) may cause the user device (100) to display at least one object for context information corresponding to the third search result information on the context display screen (330) while outputting the third agent message on the chat window screen (310). For example, referring to FIG. 5c, the server (101) may cause the user device (100) to display the third agent message (513c) on the chat window screen (310) through the intelligent agent, while displaying location indicators (533, 534) for context information (e.g., location information of the third tour and location information of the fourth tour) corresponding to the third search result information on the context display screen (330) within the running map app screen.

[0095] According to various embodiments, in response to identifying another specific query from a fourth user message received from the user device (100), the server (101) may verify at least one contextual piece of information and fourth search result information corresponding to the other specific query. For example, referring to FIG. 5d, in response to identifying another specific query (e.g., accommodation information) from the fourth user message (512d) (e.g., “The price is much better in Gyeongju. Is there accommodation in Gyeongju?”), the server (101) may verify fourth search result information (e.g., first accommodation information and second accommodation information) determined by the intelligent agent based on travel area information (e.g., Gyeongju) and the other specific query (e.g., accommodation information) of contextual information related to the travel itinerary.

[0096] According to one embodiment, the server (101) may input the fourth user message into a large-scale language model via an intelligent agent, thereby generating a fourth agent message including the fourth search result information as a response message obtained. For example, referring to FIG. 5d , the server (101) may cause the user device (100) to display a second agent message (513d) including the fourth search result information on the chat window screen (310).

[0097] According to various embodiments, the server (101) may, in response to identifying a plurality of contextual information from a fifth user message received from the user device (100), check the plurality of contextual information and fifth search result information corresponding to a specific query. For example, referring to FIG. 5e, the server (101) may identify travel date information (e.g., 12.2 (Sat) - 12.4 (Mon), 2 nights) and travel number information (e.g., 1 adult) as context information related to the travel itinerary from the fifth user message (512e) (e.g., "I'm going to go on December 2 and stay for 2 nights and 3 days, and I'm going alone. Umm, a motel or a guesthouse is cheap, so either one would be good. My budget is 70,000 won per night?"), and identify a location characteristic condition (e.g., motel or guesthouse) and a price condition (e.g., 70,000 won or less) as context information related to the travel location condition, and in response to this, the server may confirm fifth search result information (e.g., third accommodation information and fourth accommodation information) determined by the intelligent agent based on the aforementioned context information and a specific query (e.g., accommodation information).

[0098] According to one embodiment, the server (101) may input the fifth user message into a large-scale language model via an intelligent agent, thereby generating a fifth agent message including fifth search result information as a response message obtained. For example, referring to FIG. 5e, the server (101) may cause the user device (100) to display a fifth agent message (513e) including fifth search result information on the chat window screen (310).

[0099] According to one embodiment, in response to identifying additional context information related to a travel itinerary from a fifth user message received from the user device (100), the server (101) may cause the user device (100) to additionally display context information on the travel itinerary display screen (320). For example, in response to identifying travel date information (523) and traveler number information (524) as additional context information related to a travel itinerary from the fifth user message (512e), the server (101) may cause the user device (100) to additionally display the additional context information on the travel itinerary display screen (320).

[0100] According to various embodiments, the server (101) (e.g., the processor (121) of FIG. 1 ) may, in response to identifying context information related to a travel location condition from a sixth user message, verify at least some of the context information related to the travel itinerary, context information related to the travel location condition, and sixth search result information corresponding to a specific query. For example, referring to FIG. 5f , the server (101) may, in response to identifying a review score condition (e.g., 4.8 or higher) as context information related to the travel location condition from a sixth user message (512f) (e.g., “Everything is good, but can you rate it again with a rating of 4.8 or higher?”), verify fifth search result information (e.g., fifth accommodation information and sixth accommodation information) determined by the intelligent agent based on the context information related to the travel itinerary, the context information related to the travel location condition, and the specific query (e.g., accommodation information).

[0101] According to one embodiment, the server (101) may input the sixth user message into a large-scale language model via an intelligent agent, thereby generating a sixth agent message including the sixth search result information as a response message obtained. For example, referring to FIG. 5f, the server (101) may cause the user device (100) to display the sixth agent message (513f) including the sixth search result information on the chat window screen (310).

[0102] According to various embodiments, the server (101) (e.g., the processor (121) of FIG. 1) may, in response to identifying an additional search condition expression from a user message in a dialog window screen (310) after displaying an object for context information corresponding to specific search result information in a context display window screen (330), check search result information corresponding to the first context information, the third context information, the specific query, and the condition information specified by the additional search condition expression. According to one embodiment, the condition information specified by the additional search condition expression may be condition information (e.g., a region range, a price range, etc.) specified by the user device (100) through the context display window screen (330). For example, after displaying an object (e.g., location of accommodation) for context information corresponding to specific search result information on a context display window screen (330) as in FIG. 5f, the server (101) may, in response to identifying an additional search condition expression (e.g., "within here") from a user message (512g) (e.g., "Find me here") in a dialogue window screen (310) as in FIG. 5g, check search result information corresponding to first context information (e.g., travel area / date / number of people), third context information (e.g., accommodation type / cost / review score condition), a specific query (e.g., accommodation information), and condition information specified by the additional search condition expression (e.g., within the area range displayed on the context display window screen (330)). In the example described above, the user of the user device (100) can input a user message including additional search condition expressions in the dialog box screen (310) after changing the content displayed in the context display window screen (330) (e.g., moving the area range displayed in the map app screen) in order to intuitively input the condition information desired by the user.

[0103] According to one embodiment, referring to FIG. 5g, the server (101) may input a user message (512g) into a large-scale language model through an intelligent agent, and generate an agent message (513g) including the search result information of FIG. 5g as a response message, and display the generated response message on a dialogue window screen (310).

[0104] According to one embodiment, referring to FIG. 5g, the server (101) may cause the user device (100) to display an agent message (513g) on ​​the chat window screen (310) through the intelligent agent, while displaying a location indicator for context information (e.g., location information of accommodation A and location information of accommodation B) corresponding to the search result information on the context display window screen (330) within the map app screen running.

[0105] According to one embodiment, the server (101) may display a screen including an object for context information corresponding to an agent message displayed on the chat window screen (310) on the context display window screen (330) as the user scrolls through the details of user messages and agent messages in the chat window screen (310) in response to a user input to the user device (100). For example, when the agent message (513c) of FIG. 5c is displayed on the chat window screen (310) of the user device (100) and the map app screen of FIG. 5c is displayed on the context display window screen (330), when the agent message (513b) of FIG. 5b is displayed on the chat window screen (310) by a user input (e.g., scrolling up) to the user device (100), the server (101) can cause the user device (100) to switch the first screen (e.g., the map app screen of FIG. 5c) displayed on the context display window screen (330) to a second screen (e.g., the map app screen of FIG. 5b). For another example, when the agent message (513c) of FIG. 5c is displayed on the chat window screen (310) of the user device (100) and the map app screen of FIG. 5c is displayed on the context display window screen (330), when the agent message (513e) of FIG. 5e is displayed on the chat window screen (310) by a user input (e.g., scrolling down) to the user device (100), the server (101) can cause the user device (100) to switch the first screen (e.g., the map app screen of FIG. 5c) displayed on the context display window screen (330) to a second screen (e.g., the map app screen of FIG. 5e).

[0106]

[0107] FIG. 6 is a diagram for explaining an embodiment in which a server (e.g., server (101) of FIG. 1) generates a user message to be input to an intelligent agent based on user input confirmed through a context display window screen according to various embodiments.

[0108] According to various embodiments, the server (101) (e.g., the processor (121) of FIG. 1) may cause a user device (e.g., the user device (100) of FIG. 1) to display an indicator for specifying context information related to a travel location condition on a context display window screen (330).

[0109] According to one embodiment, referring to FIG. 6, a user may move a location indicator (631) displayed on a context display screen within a display of a user device (100) to position it at a specific point (632) within a map app screen, and the server (101) may generate a user message (612a) (e.g., “Are there any activities worth doing near the point of interest?”) with information about the specific point and input it into a large-scale language model linked with an intelligent agent. According to one embodiment, the information about the specific point may include location information within a predetermined distance from the specific point, and the predetermined distance may be freely changed by an administrator of the server (101) or a user of the user device (100).

[0110] According to one embodiment, in response to identifying contextual information related to a travel location condition from a user message, the server (101) may verify at least one piece of contextual information and search result information corresponding to a specific query. For example, referring to FIG. 6, after performing the operation of FIG. 4c, the server (101) may identify contextual information regarding a specific point (632) using a location indicator (631), and verify search result information (e.g., first activity information and second activity information) corresponding to the contextual information and a specific query (e.g., activity).

[0111] According to one embodiment, the server (101) may input the user message into a large-scale language model via an intelligent agent, thereby generating an agent message including the search result information as a response message obtained. For example, referring to FIG. 6, the server (101) may cause the user device (100) to display an agent message (613a) including the search result information on the chat window screen (310).

[0112]

[0113] FIG. 7 is a drawing for explaining an embodiment in which a server (server (101) of FIG. 1) provides multiple contents through a context display window screen divided into multiple areas according to various embodiments.

[0114] According to one embodiment, referring to FIG. 7, the server (101) may input a user message (712a) into a large-scale language model via an intelligent agent, and generate an agent message (713a) as a response message that includes context information and search result information corresponding to a specific query. The server (101) may cause the user device (100) to display the agent message (713a) on the chat window screen (310).

[0115] According to one embodiment, in response to identifying a plurality of app execution information from context information corresponding to search result information, the server (101) divides the context display window screen (330) into a plurality of areas for executing the plurality of apps, and executes the apps corresponding to the search result information through each independent area. For example, referring to FIG. 7, after performing the operation of FIG. 4c, the server (101) may display an agent message (713a) for a user message (712a) on the conversation window screen (310), and in response to identifying a plurality of app execution information (e.g., YouTube link, web page link) from context information corresponding to the search result information included in the agent message (713a), divide the context display window screen (330) into two areas, display a first content through a YouTube app in the upper area (e.g., Twin Animal Farm video review), and display a second content through a web browser (e.g., Twin Animal Farm blog review) in the lower area.

[0116]

[0117] FIG. 8 is a flowchart for explaining an operation of a server (e.g., server (101) of FIG. 1) providing a package travel product according to various embodiments.

[0118] FIGS. 9A to 9C illustrate a first embodiment in which a server (101) provides a package travel product according to various embodiments.

[0119] FIGS. 10A and 10B illustrate a first embodiment in which a server (101) provides a package travel product according to various embodiments.

[0120] In operation 801, according to various embodiments, the server (101) (e.g., the processor (121) of FIG. 1) may cause a user device (e.g., the user device (100) of FIG. 1) to display at least one chat message transmitted and received with a large-scale language model-based intelligent agent on a chat window screen. In one embodiment, the server (101) may perform operation 801 using the method described in operation 201 of FIG. 1.

[0121] In operation 803, according to various embodiments, the server (101) (e.g., the processor (121) of FIG. 1) may identify a request to create a package travel product combining at least two travel products from a user message received from the user device (100). According to one embodiment, the server (101) may identify the request to create a package travel product as a specific query.

[0122] In one embodiment, the server (101) can identify information about at least two travel products from a user message and a request for a package tour product combining the at least two travel products. For example, referring to FIG. 9A, the server (101) can identify the name of a first travel product (e.g., Gangneung Seaview Pension) and the name of a second travel product (e.g., Twin Zoo) from a user message (911), identify a request for creating a package tour product combining the travel products as a specific query, and input the request to an intelligent agent. For another example, referring to FIG. 10A, the server (101) can identify the name of a first travel product (e.g., Gyeongju Dotori House) and the name of a second travel product (e.g., Gyeongju Night View Tour) from a user message (1011), identify a request for creating a package tour product combining the travel products as a specific query, and input the request to an intelligent agent.

[0123] According to one embodiment, the server (101) can check the history of previous chat messages sent and received between the user device (100) and the intelligent agent, and identify information about at least two travel products to be combined into a package travel product from the history of the previous chat messages.

[0124] According to various embodiments, the server (101) (e.g., the processor (121) of FIG. 1) may determine to automatically create a package travel product under certain conditions even if it does not receive a request for creating a package travel product from the user device (100).

[0125] According to one embodiment, the server (101) may input at least one chat message transmitted and received between the user device (100) and the intelligent agent into a transformer-based predictive model to determine at least two travel products to be recommended to the user device (100). For example, the server (101) may input a plurality of chat messages (e.g., chat messages from agent message (313) of FIG. 3A to agent message (415b) of FIG. 4C) transmitted and received between the user device (100) and the intelligent agent for a first time period or a first number of times from the time of identifying a specific query or third context information into a transformer-based predictive model to determine the name of the first travel product (e.g., Gangneung Seaview Pension) and the name of the second travel product (e.g., Twin Zoo) to be recommended to the user device (100). As another example, the server (101) may input a plurality of chat messages (e.g., chat messages from the user message (312b) of FIG. 4A to the agent message (415b) of FIG. 4C) transmitted and received between the user device (100) and the intelligent agent for a second time period or a second number of times from the time of identifying a specific query or third context information into a transformer-based prediction model, thereby determining the name of the first travel product (e.g., Gangneung Seaview Pension) and the name of the second travel product (e.g., Twin Zoo) to be recommended to the user device (100).

[0126] According to one embodiment, a transformer-based predictive model for recommending at least two travel products may be learned based on correlations between (i) a plurality of chat messages transmitted and received between a plurality of user devices and an intelligent agent for a first time period or a first number of times from the time point at which a specific query or third context information is identified, and (ii) package travel products purchased by the plurality of user devices. Meanwhile, the transformer-based predictive model may be implemented within a large-scale language model or as a separate model.

[0127] In operation 805, according to various embodiments, the server (101) (e.g., the processor (121) of FIG. 1) may determine price information of a package travel product by using first context information related to the travel itinerary.

[0128] According to various embodiments, the server (101) (e.g., the processor (121) of FIG. 1) may determine price information of a package travel product using travel date information and travel number information as first context information.

[0129] According to one embodiment, the server (101) can identify discount weights corresponding to at least two types of travel products. According to one embodiment, the server (101) can store a table mapping discount weights by type of travel product in a memory (e.g., memory (131) of FIG. 1), as shown in [Table 1] below, and can identify discount weights corresponding to each type of travel product from the table. For example, the types of travel products can include transportation (transportation) information, accommodation information, tourist attraction information, restaurant information, tour information, ticket information, activity packages, rental car services, travel insurance, local SIM cards or portable WiFi, gift certificates or vouchers, transportation passes, special event packages, personalized guided tours, airport transfer services, and experience products such as cooking or trying on hanbok, etc. The above examples are merely illustrative and are not limiting, and the administrator of the server (101) can designate discount weights for various types.

[0130] Travel product discount weighting, accommodation A, tourist attraction B, restaurant C, ticket N

[0131] According to one embodiment, the types of travel products can be classified into multiple classification systems (e.g., large classification / medium classification / small classification, etc.) as shown in [Table 2] below, and the administrator of the server (101) can specify a discount weight for each specific classification system.

[0132] Large category, medium category, small category, discount weight, accommodation, hotel, 1-star to 2-star A, 3-star or higher B, motel - C, guesthouse - D, restaurant, Japanese restaurant - E, Korean restaurant, stew specialty restaurant, grill specialty restaurant

[0133] According to one embodiment, the server (101) can designate different discount weights according to the travel date information and the number of travelers for each type of travel product.

[0134] According to one embodiment, the server (101) may calculate cost information by applying travel date information and travel number information to price information of at least two travel products. For example, referring to FIG. 9B, the server (101) may calculate first cost information (e.g., 105,000 won) by applying travel date information (e.g., 1 night) confirmed from the travel schedule display screen (320) to price information (e.g., 105,000 won) of a first travel product (e.g., Gangneung Seaview Pension), and may calculate second cost information (e.g., 27,000 won) by applying travel number information (e.g., 3 people) confirmed from the travel schedule display screen (320) to price information (e.g., 9,000 won) of a second travel product (e.g., Twins Zoo). For another example, referring to FIG. 10b, the server (101) can calculate first cost information (e.g., 105,000 won) by applying travel date information (e.g., 1 night) confirmed from the travel schedule display screen (320) to price information (e.g., 105,000 won) of the first travel product (e.g., Gyeongju Dotori House), and can calculate second cost information (e.g., 27,000 won) by applying travel number information (e.g., 3 people) confirmed from the travel schedule display screen (320) to price information (e.g., 9,000 won) of the second travel product (e.g., Twin Zoo).

[0135] In one embodiment, the server (101) can calculate discount price information by applying discount weights for each type to the price information of at least two travel products. For example, referring to FIG. 9b, the server (101) can calculate package discount price information (e.g., KRW 30,450) by applying discount weights to the first and second travel products. For another example, referring to FIG. 10b, the server (101) can calculate package discount price information (e.g., KRW 17,850) by applying discount weights to the first and second travel products.

[0136] In one embodiment, the server (101) can calculate price information for a package travel product by combining the original price and discount price information. For example, referring to FIG. 9b , the server (101) can calculate price information (e.g., KRW 101,550) for a package travel product by combining the original price and discount price information for a first travel product and a second travel product. For another example, referring to FIG. 10b , the server (101) can calculate price information (e.g., KRW 60,150) for a package travel product by combining the original price and discount price information for a first travel product and a second travel product.

[0137] According to one embodiment, after the server (101) identifies a request to create a package travel product, if the travel date information or the number of travelers information does not exist in the first context information, the server (101) may output an agent message requesting the corresponding information on the dialogue window screen (310) through the intelligent agent. For example, after the server (101) identifies a request to create a package travel product, if the travel date information or the number of travelers information cannot be confirmed from the travel itinerary summary window screen (320), the server (101) may output an agent message requesting the unconfirmed information to the user device (100) through the intelligent agent.

[0138] In operation 807, according to various embodiments, the server (101) (e.g., the processor (121) of FIG. 1) may cause the user device (100) to display an agent message including price information of a package travel product on a chat window screen (310) through an intelligent agent.

[0139] According to one embodiment, an agent message including price information of a package travel product may include a first sub-agent message representing a response message to a user message, a second sub-agent message representing discount details, and a third sub-agent message configured to realize a function of purchasing a package travel product. For example, referring to FIGS. 9b and 9c , the agent message may include a first sub-agent message (912a), a second sub-agent message (912b), and a third sub-agent message (912c), and as shown in FIGS. 9b and 9c , the server (101) may cause the user device (100) to display the first to third sub-agent messages in the order of the first to third sub-agent messages on the chat window screen (310). For another example, referring to FIGS. 10a and 10b , the agent message may include a first sub-agent message (1012a), a second sub-agent message (1012b), and a third sub-agent message (1012c). According to one embodiment, the second sub-agent message may include the names of at least two travel products, the package period, the respective cost information (detailed amount) and total cost information (total amount) of the at least two travel products, the discount price information of the package travel product, the final price information obtained by subtracting the discount price information from the total cost information (i.e., the price information of the package travel product). According to one embodiment, the third sub-agent message may include thumbnails of at least two travel products, review rating information, the respective cost information, the total cost information, the discount price information of the package travel product, the final price information obtained by subtracting the discount price information from the total cost information, and an object set to execute a payment screen for purchasing the package travel product (e.g., purchase package product).

[0140] According to one embodiment, the server (101) may output an agent message including price information of a package travel product on a chat window screen (310), and may cause the user device (100) to highlight and display a movement route between package travel products within a map app screen on a context display window screen (330). According to one embodiment, the server (101) may cause the user device (100) to display an execution screen (e.g., a travel product detail information page, a package product payment page, etc.) corresponding to the specific object on the context display window screen (330) in response to the user of the user device (100) selecting a specific object (e.g., a travel product thumbnail, a package product purchase button, etc.) of the agent message on the chat window screen (310).

[0141]

[0142] FIG. 11 is a flowchart illustrating an operation of an electronic device (e.g., a user device (100) of FIG. 1) to generate training data to be used for training a large-scale language model, according to various embodiments.

[0143] FIG. 12 is a diagram showing a GUI for each type of node used to generate training data according to various embodiments.

[0144] FIG. 13 is a diagram showing a node flow object that implements node flow in GUI form according to various embodiments.

[0145] FIG. 14 is a diagram illustrating a training data set according to various embodiments.

[0146] According to various embodiments, the server (101) may store / manage a large-scale language model, and transmit source code to the user device (100) that enables each execution screen of a dedicated application or website for transmitting and receiving messages with the user device (100) through an intelligent agent based on the large-scale language model to display the source code on the user device (100), and the user device (100) may receive the source code and display the execution screen requested by the user of the user device (100) through the dedicated application or web browser.

[0147] In operation 1101, according to various embodiments, the electronic device (100) (e.g., the processor (120) of FIG. 1) may generate at least one node among a storage node, a switch node, and an output node using an application for generating training data to be used for training a large language model (LLM).

[0148] According to one embodiment, the large-scale language model is a type of generative AI, and may include, for example, a transformer-based neural network model, and may be used in (1) natural language understanding (NLU) fields that understand the meaning of sentences and perform natural language understanding tasks such as keyword extraction, sentiment analysis, and information retrieval, (2) natural language generation (NLG) fields that generate natural language in various forms such as sentences, paragraphs, summaries, and translations, (3) question answering fields that generate answers to given questions, (4) machine translation fields that perform translation tasks between multiple languages, and (5) text summarization fields that concisely summarize long texts to increase the value of information. In addition to the examples described above, the large-scale language model may include various language models that can be easily realized by those skilled in the art.

[0149] In one embodiment, the at least one node may be generated in the form of program code or a graphical user interface through the application. In one embodiment, the application may be an application (e.g., Visual Studio) that provides an integrated development environment capable of programming in various languages.

[0150] According to one embodiment, the electronic device (100) can display an object corresponding to a node used for generating training data on an application in the form of a GUI through a display (e.g., display (140) of FIG. 1).

[0151] In one embodiment, a storage node used to generate training data may be a component for storing at least one candidate storage variable value for a storage variable. Each of the at least one candidate storage variable value may be selected based on a preset probability value, and each probability value may be set by the user to be the same or different from each other.

[0152] According to one embodiment, a switch node used for generating training data may be a component for setting a condition variable indicating a branching condition. According to one embodiment, referring to FIG. 12, a switch node object (1211) that implements a switch node in a GUI form may include (i) a node selection object for selecting a type of node (e.g., Switch), (ii) a condition variable value field indicating a condition variable (e.g., has_greeted), and (iii) a button for adding a candidate state value of a branch node (e.g., add case).

[0153] According to one embodiment, a branch node used for generating training data may be a component for setting a candidate state value that can be branched based on the current state of a condition variable of a switch node. According to one embodiment, referring to FIG. 12, a branch node object (1212) that implements a branch node in the form of a GUI may include (i) a candidate state value (e.g., true), (ii) an object for modifying / removing the candidate state value, and (iii) a field for adding another candidate state value, and the branch node object (1212) may be implemented as a set (1210) with a switch node object (1211). The electronic device (100) may transition to a branch node that has a value that matches the current condition variable of the switch node. For example, if a switch node is set to a condition variable (e.g., apple_count), a first branch node connected to the switch node is set to a first candidate state value (e.g., 0, 1, 2), and a second branch node connected to the switch node is set to a second candidate state value (e.g., 3), the electronic device (100) may transition to the first branch node when the value of apple_count is the first candidate state value, transition to the second branch node when it is the second candidate state value, and stop execution and generate an error when it is any other value.

[0154] In one embodiment, an output node used for generating training data may be a component for outputting at least one candidate output variable value for an output variable. Each of the at least one candidate output variable value may be selected based on a preset probability value, and each probability value may be set to be the same or different from each other by a user. In one embodiment, the output node may be configured to output the type of the output node, the output variable name, and the content (i.e., a value selected from among the candidate output variable values). According to one embodiment, referring to FIG. 12, an output node object (1220, 1230) that implements an output node in the form of a GUI may include (i) an object that can select the type of the output node (e.g., assistant), (ii) an output variable value field indicating an output variable (e.g., GreetingAgain, Greeting), (iii) a candidate output variable value (e.g., "Hi again! I'm {name}." / "Hello again, this is {name}!", "Hi!" / "Hello!"), (iv) an object for modifying / removing the candidate output variable value, and (v) a field for adding another candidate output variable value. According to one embodiment, the type of the output node may be one of prompt data corresponding to a virtual user message (e.g., user) or completion data corresponding to a virtual agent message (e.g., assistant). According to one embodiment, the type of the output node may be set to various types by the user of the electronic device (100) other than the prompt data or the completion data.

[0155] According to various embodiments, the training data generated according to the present disclosure may include data to be used for training a conversational LLM, data to be used for training a translation LLM, and data to be used for training a summarizing LLM, and may include data to be used for training LLMs for various purposes without being limited to the examples described above. According to one embodiment, the training data may have a pair of properties, such as prompt data and completion data, for each data entry, or may have a single property (e.g., text) or three or more properties (e.g., question / correct answer / wrong answer, etc.).

[0156] If each node for generating training data and the node flow including it are implemented in GUI form, it is possible to provide convenience for easily generating training data by intuitively recognizing the node's transition relationship, transition probability, and each setting value.

[0157] According to one embodiment, at least one node used for generating training data may further include a void node that does not perform any operation, a start node, and an end node, and the start node and the end node may be expressed as a void node or as a node independent of the void node, which may be variously changed according to the user's settings. According to one embodiment, the end node may include an object (preview function) that enables displaying the result of generating training data according to the node flow from the start node to the end node. According to one embodiment, the type of the at least one node described above is only one example, and various types of nodes for generating training data may be included in addition to a storage node, a switch node, an output node, and a void node according to the settings of a user of the electronic device (100) or an administrator of the server (101).

[0158] According to one embodiment, the electronic device (100) may generate at least one node based on a user input. For example, the electronic device (100) may receive a user input for generating a specific node (e.g., an output node) and generate the specific node based on the user input.

[0159]

[0160] In operation 1103, according to various embodiments, the electronic device (100) (e.g., the processor (120) of FIG. 1) may generate a node flow that sets a transition relationship between at least one node and a variable-related value for each of at least one node. According to one embodiment, the node flow may define an execution order of a series of nodes for generating training data, and specifically, may indicate a transition relationship and a transition probability between each node from a start node (e.g., the first node) to an end node (e.g., the last node) and a variable-related value within each node. According to one embodiment, the electronic device (100) may implement and display an object corresponding to the node flow in a GUI format on an application through the display (140). For example, FIG. 13 illustrates a draft for implementing a node flow in a GUI format as part of a node flow for generating training data of a conversational LLM.

[0161] According to various embodiments, an electronic device (100) (e.g., processor (120) of FIG. 1) may establish a transition relationship between at least one node.

[0162] According to one embodiment, the electronic device (100) can generate a node flow capable of generating training data by setting a transition relationship and a probability value for the transition relationship that point from one node (e.g., a starting node) to another node (e.g., an arrival node). The starting node and the arrival node merely represent relative concepts between the two nodes and do not represent fixed concepts, and each node except the starting node and the ending node can be an arrival node and also a new starting node. According to one embodiment, the types of the starting node and the arrival node can be the same or different.

[0163] According to one embodiment, referring to FIG. 13, the electronic device (100) may set a transition relationship (1301) from a storage node (e.g., Set: name) to a storage node (e.g., Set: has_greeted) and a probability value (e.g., 1) for the transition relationship (1302).

[0164] According to one embodiment, referring to FIG. 13, the electronic device (100) may set a transition relationship (402) from a storage node (e.g., Set: has_greeted) to a switch node (e.g., Switch: has_greeted) and a probability value (e.g., 1) for the transition relationship (402).

[0165] According to one embodiment, referring to FIG. 13, the electronic device (100) may set a transition relationship (1303) from a branch node (e.g., True) to a storage node (e.g., Set: has_greeted) and a probability value (e.g., 0.05) for the transition relationship (1303).

[0166] According to one embodiment, referring to FIG. 13, the electronic device (100) may set a transition relationship (1304) from a storage node (e.g., Set: has_greeted) to an output node (e.g., User: Greeting) and a probability value (e.g., 1) for the transition relationship (1304).

[0167] According to one embodiment, referring to FIG. 13, the electronic device (100) may set a transition relationship (1305) from an output node (e.g., User: Greeting) to a switch node (e.g., Switch: has_greeted) and a probability value (e.g., 0.5) for the transition relationship (1305).

[0168] According to one embodiment, referring to FIG. 13, the electronic device (100) may set a transition relationship (1306) from a branch node (e.g., True) to an output node (e.g., Assistant: GreetingAgain) and a probability value (e.g., 0.00001) for the transition relationship (1306).

[0169] According to one embodiment, referring to FIG. 13, the electronic device (100) may set a transition relationship (1307) from a branch node (e.g., False) to a void node (e.g., Void: Final) and a probability value (e.g., 0.999) for the transition relationship (1307).

[0170] In one embodiment, a transition relationship from a switch node to a branch node may not have a probability value.

[0171] According to one embodiment, the node flow may include at least one output node for generating prompt data corresponding to a virtual user message and at least one output node for generating completion data corresponding to a virtual agent message. For example, referring to FIG. 13, the node flow may include a first output node (1330) for generating prompt data corresponding to a virtual user message, a second output node (1340) for generating completion data corresponding to the virtual first agent message, and a third output node (1350) for generating completion data corresponding to the virtual second agent message. Meanwhile, the user message may refer to a message obtained from a user, and the agent message may refer to a message that is input into a large-scale language model, processed in the form of a response to the user message, and output by an intelligent agent (or intelligent assistant) linked with the large-scale language model. The electronic device (100) according to the present disclosure can automatically generate a large number of virtual user messages and virtual agent messages using node flow without directly receiving training user messages and training agent messages from a user.

[0172] According to various embodiments, the electronic device (100) (e.g., the processor (120) of FIG. 1) may set a variable-related value for each of at least one node.

[0173] According to one embodiment, the electronic device (100) may obtain, from a user, variable-related values ​​for a storage node, including storage variables and candidate storage variable values. For example, referring to FIG. 13 , the electronic device (100) may obtain, from a user, storage variables (e.g., name) and candidate storage variable values ​​(e.g., John, Karl, Alice, Alex) for a storage node (1310). According to one embodiment, the electronic device (100) may set a probability value for selecting a candidate storage variable value for each candidate storage variable value based on a user input.

[0174] According to one embodiment, the electronic device (100) may obtain a condition variable as a variable-related value for a switch node from a user. For example, referring to FIG. 13, the electronic device (100) may obtain a condition variable (e.g., has_greeted) for a switch node (1320) from a user.

[0175] According to one embodiment, the electronic device (100) may obtain a candidate state value of a condition variable for a switch node (1320) as a variable-related value for a branch node from a user. For example, referring to FIG. 13 , the electronic device (100) may obtain a candidate state value (e.g., True) of a condition variable (e.g., has_greeted) for a branch node (1321) from a user.

[0176] According to one embodiment, the electronic device (100) may obtain, from the user, variable-related values ​​for an output node, such as output variables and candidate output variable values. For example, referring to FIG. 13, the electronic device (100) may obtain, from the user, an output variable (e.g., User: Greeting) and candidate output variable values ​​(e.g., Hi, hi, Hello, hello) for an output node (1330). According to one embodiment, the electronic device (100) may set a probability value for selecting a candidate output variable value for each candidate output variable value based on a user input.

[0177] According to one embodiment, the electronic device (100) can freely perform the setting of transition relationships and variable-related values ​​regardless of the order of operations. For example, the electronic device (100) can set variable-related values ​​for each node after setting transition relationships between nodes, or can set variable-related values ​​for each node after setting transition relationships between nodes.

[0178]

[0179] In operation 1105, according to various embodiments, the electronic device (100) (e.g., the processor (120) of FIG. 1) may generate training data according to the result of executing the node flow in response to a request for generation of training data.

[0180] According to one embodiment, after the establishment of transition relationships between each node in the node flow and the establishment of variable-related values ​​for each node are completed, the electronic device (100) may obtain a request for generating training data (i.e., a request for executing a node flow) from a user, and generate training data according to a result of executing the node flow in response to the request. For example, the electronic device (100) may identify a result value of executing a node while starting from a start node and proceeding (moving) to the next node according to a transition probability set for each transition relationship, and may generate training data according to a result value of executing each node after executing through the last output node to the end node.

[0181] According to one embodiment, the electronic device (100) may obtain the number of training data entries to be generated from the user as a request for generating training data. For example, referring to FIG. 14, if the electronic device (100) receives a request from the user to generate 10 training data entries, the electronic device (100) may generate 10 training data entries, in which case the first training data entry represents [{"type":"user", "name":"Greeting", "content":"hello"}, {"type":"assistant", "name":"Greeting", "content":"Hi!"}]. As another example, if the electronic device (100) receives a request from the user to generate 100 training data entries, the electronic device (100) may generate 100 training data entries.

[0182] According to one embodiment, each training data may include prompt data corresponding to a virtual user message and completion data corresponding to a virtual agent message. According to one embodiment, the electronic device (100) may output the type, output variable (name), and content (i.e., a value selected from among candidate output variable values) of an output node corresponding to each of the prompt data and the completion data of the training data according to the execution of the node flow. For example, referring to FIG. 14, in the first training data, the prompt data may be {"type":"user", "name":"Greeting", "content":"hello"}, and the completion data may be {"type":"assistant", "name":"Greeting", "content":"Hi!"}.

[0183] According to one embodiment, when a candidate output variable value includes a specific storage variable, the electronic device (100) may generate training data with candidate output variable values ​​that include the candidate storage variable value selected for the specific storage variable, upon execution of the node flow. For example, referring to FIG. 12, when a candidate output variable value of an output node object (1220) (e.g., "Hi again! I'm {name}.) includes a specific storage variable (e.g., name), the electronic device (100) may generate training data with candidate output variable values ​​that include the candidate storage variable value selected for the specific storage variable (e.g., Alex), upon execution of the node flow.

[0184] According to one embodiment, the electronic device (100) can generate training data in a JSON (JavaScript Object Notation) structured format. The JSON format is merely an example and is not limited thereto, and training data can be generated in various formats easily achievable by those skilled in the art. For example, the training data can be converted and stored in file formats such as XML (eXtensible Markup Language), CSV (Comma-Separated Values), and TXT.

[0185]

[0186] FIG. 15a is a diagram illustrating a node flow object for generating training data for an LLM for translation according to various embodiments.

[0187] FIG. 15b is a diagram illustrating an example of a training data entry of an LLM for translation generated by the node flow object of FIG. 15a according to various embodiments.

[0188] According to various embodiments, the electronic device (100) (e.g., the processor (120) of FIG. 1) may receive an input from a user to generate at least one node. According to one embodiment, the electronic device (100) may generate an output node object for generating a system message according to a user input, an output node object for generating a virtual user message as prompt data, or an output node object for generating a virtual agent message as completion data, in order to generate training data of an LLM for translation. For example, referring to FIG. 15A, the electronic device (100) may generate a start node object (1501), a first output node object (1511) for generating a system message, second-first to second-third output node objects (1521, 1522, 1523) for generating a virtual user message, third-first to third-third output node objects (1531, 1532, 1533) for generating a virtual agent message, and an end node object (1502) according to a user input.

[0189] According to one embodiment, a virtual user message as training data for an LLM for translation may mean an original message to be translated, and a virtual agent message may mean a translated message that is a translation of the original message.

[0190] According to one embodiment, each of the output node objects for generating a virtual user message and / or a virtual agent message can be linked to an LLM trained to output similar expressions, and the electronic device (100) can expand candidate output variable values ​​by a predetermined number using the LLM according to a user request. For example, if only two data (e.g., “Hello”, “Hello”) are input as candidate output variable values ​​in the output node object (1521) for generating a virtual user message, when the user selects an expand button (Expand), the electronic device (100) can input the two data into the LLM to expand similar data (e.g., “Hello~”, “Hello”, “Hello”, etc.) by a predetermined number (e.g., 11) and input them into the output node object (1521). Here, the predetermined number can be freely changed and set by the user.

[0191] According to one embodiment, the electronic device (100) may receive an input from a user to set a transition relationship and a transition probability between at least one node. For example, referring to FIG. 15A, the electronic device (100) may set a transition relationship and a transition probability (e.g., 33% each) from a first output node object (1511) to second-first to second-third output node objects (1521, 1522, 1523) according to a user input. For another example, referring to FIG. 15A, the electronic device (100) may set a transition relationship and a transition probability (e.g., 100% each) from second-first to second-third output node objects (1521, 1522, 1523) to third-first to third-third output node objects (1531, 1532, 1533) according to a user input. As another example, the electronic device (100) can set a transition relationship and a transition probability (e.g., 100% each) from the 3-1 to 3-3 output node objects (1531, 1532, 1533) to the end node object (1502) according to user input.

[0192] According to one embodiment, the training data of the LLM for translation may be applied with the contents of the training data of the LLM for conversation described in FIGS. 11 to 14, and the training data of the LLM for conversation may also be applied with the contents of the training data of the LLM for translation described above.

[0193] According to one embodiment, referring to FIG. 15b, an output node object for generating a virtual user message of training data of an LLM for translation may be set to output one prompt data (e.g., an original message) and one completion data (e.g., a translated message) upon execution of a node flow object, so that the electronic device (100) may generate a training data entry such as FIG. 15b as a result of executing the node flow of FIG. 15a.

[0194]

[0195] FIG. 16a is a diagram illustrating a node flow object for generating training data for a summary LLM according to various embodiments.

[0196] FIG. 16b is a diagram illustrating an example of a training data entry of a summary LLM generated by the node flow object of FIG. 15a according to various embodiments.

[0197] According to various embodiments, the electronic device (100) (e.g., the processor (120) of FIG. 1) may receive an input from a user to generate at least one node. According to one embodiment, the electronic device (100) may generate an output node object for generating a system message according to a user input, an output node object for generating a virtual user message as prompt data, or an output node object for generating at least one of a virtual agent message as completion data, in order to generate training data of a summary LLM. For example, referring to FIG. 16A, the electronic device (100) may generate a start node object (1601), a first output node object (1611) for generating a system message, second-first to second-second output node objects (1621, 1622) for generating a virtual user message, third-first to third-second output node objects (1631, 1632) for generating a virtual agent message, and an end node object (1602) according to a user input.

[0198] According to one embodiment, a virtual user message as training data of a summary LLM may mean an original message that is the target of summary, and a virtual agent message may mean a summary message that summarizes the original message.

[0199] According to one embodiment, each output node object for generating a virtual user message and / or a virtual agent message may be linked to an LLM trained to output a new message according to a predetermined criterion (e.g., an action of outputting the same content in a different manner), and the electronic device (100) may use the LLM to expand candidate output variable values ​​by a predetermined number according to a user request. The above-described predetermined criterion (e.g., an action of outputting the same content in a different manner) may include various actions such as maintaining the content of a sentence but changing the sentence structure / arrangement, maintaining the core expression of a sentence but changing the particles of a sentence, or adding / removing an exclamation mark. For example, if only two data (1621a, 1621b) are input as candidate output variable values ​​in an output node object (1621) for generating a virtual user message, when the user selects the expand button (Expand), the electronic device (100) can input the two data into the LLM and input new data (e.g., new review messages) that are identical to the contents of the two data (1621a, 1621b) but written in a different manner by expanding them by a predetermined number (e.g., 3) to the output node object (1621). Here, the predetermined number can be freely changed and set by the user.

[0200] According to one embodiment, the electronic device (100) may receive an input from a user to set a transition relationship and a transition probability between at least one node. For example, referring to FIG. 16A, the electronic device (100) may set a transition relationship and a transition probability (e.g., 50% each) from a first output node object (1611) to the second-first to second-second output node objects (1621, 1622) according to a user input. For another example, referring to FIG. 16A, the electronic device (100) may set a transition relationship and a transition probability (e.g., 100% each) from the second-first to second-second output node objects (1621, 1622) to the third-first to third-second output node objects (1631, 1632) according to a user input. As another example, the electronic device (100) can set a transition relationship and a transition probability (e.g., 100% each) from the 3-1 to 3-2 output node objects (1631, 1632) to the end node object (1602) according to user input.

[0201] According to one embodiment, the training data of the summary LLM may be applied with the contents of the training data of the conversational LLM described in FIGS. 11 to 14, and the training data of the conversational LLM may also be applied with the contents of the training data of the summary LLM described above.

[0202] According to one embodiment, referring to FIG. 16b, an output node object for generating a virtual user message of training data of a summary LLM may be set to output one prompt data (e.g., an original message) and one completion data (e.g., a summary message) upon execution of a node flow object, so that the electronic device (100) may generate a training data entry such as FIG. 16b as a result of executing the node flow of FIG. 16a.

[0203]

[0204] According to various embodiments, a server for providing real-time context information while transmitting and receiving chat messages with a user device using an intelligent agent includes a communication module and a processor, wherein the processor is configured to execute instructions for causing the user device to display at least one chat message transmitted and received with an intelligent agent based on a large language model (LLM) on a chat window screen, and in response to identifying first context information related to a travel itinerary from a first user message received from the user device, causing the user device to display the first context information on a travel itinerary display screen, and outputting a first agent message including at least a portion of the first context information and first search result information corresponding to a specific query through the intelligent agent on the chat window screen, while causing the user device to display at least one first object for second context information corresponding to the first search result information on the context display screen.

[0205] According to various embodiments, the processor may be configured to identify at least one of travel area information, travel date information, or travel number information as the first context information from the first user message.

[0206] According to various embodiments, the processor may be configured to obtain first search result information from a database linked to the large-scale language model using at least a portion of the first context information and the specific query, and to identify, as the second context information, at least one of location information, web link information, or app link information corresponding to the first search result information.

[0207] According to various embodiments, the processor may be further configured to execute instructions that, in response to identifying third context information related to a travel location condition from a second user message received from the user device, cause the intelligent agent to identify at least a portion of the first context information, the third context information, and second search result information determined based on the specific query, and output a second agent message including the second search result information as a reply message to the second user message on the dialog window screen, while causing the user device to display at least one second object for fourth context information corresponding to the second search result information on the context display window screen.

[0208] According to various embodiments, the processor may be further configured to execute instructions that cause the user device to display the third context information on the travel itinerary display screen in response to identifying the third context information.

[0209] According to various embodiments, the processor may be further configured to execute instructions that, in response to identifying another specific query from a third user message received from the user device, identify at least one context information and third search result information corresponding to the other specific query, and output a third agent message including the third search result information as a response message to the third user message on the dialog window screen, while causing the user device to display at least one third object for fifth context information corresponding to the third search result information on the context display window screen.

[0210] According to various embodiments, a method for operating a server that provides real-time related context information while transmitting and receiving chat messages with a user device using an intelligent agent may include causing the user device to display at least one chat message transmitted and received with an intelligent agent based on a large language model (LLM) on a chat window screen, causing the user device to display first context information related to a travel itinerary on a travel itinerary display window screen in response to identifying first context information related to a travel itinerary from a first user message received from the user device, and causing the user device to display at least one first object for second context information corresponding to the first search result information on the context display window screen while outputting a first agent message including at least a portion of the first context information and first search result information corresponding to a specific query on the chat window screen through the intelligent agent.

[0211] According to various embodiments, a computer program product comprising one or more programs configured to be executed by one or more processors of a computer system, the one or more programs including instructions for: causing a user device to display at least one chat message transmitted and received with an intelligent agent based on a large language model (LLM) on a chat window screen; causing the user device to display the first context information on a travel itinerary display window screen in response to identifying first context information related to a travel itinerary from a first user message received from the user device; and causing the user device to display at least one first object for second context information corresponding to the first search result information on a context display window screen while outputting a first agent message including at least a portion of the first context information and first search result information corresponding to a specific query on the chat window screen through the intelligent agent.

[0212] According to various embodiments, a server providing a package travel product while transmitting and receiving a chat message to and from a user device through an intelligent agent includes a communication module and a processor, and the processor is configured to execute instructions that cause the user device to display at least one chat message transmitted and received with an intelligent agent based on a large language model (LLM) on a chat window screen, identify a request for creating a package travel product combining at least two travel products from a user message received from the user device, determine price information of the package travel product using first context information related to a travel itinerary, and cause the user device to display an agent message including the price information of the package travel product on the chat window screen through the intelligent agent.

[0213] According to various embodiments, the first context information includes travel area information, travel date information, and travel number information, and the processor may be configured to determine price information of the package travel product using the travel date information and the travel number information.

[0214] According to various embodiments, the processor may be configured to identify discount weights corresponding to the types of the at least two travel products, calculate cost information by applying the travel date information and the number of travelers information to the price information of the at least two travel products, calculate discount price information by applying the type-specific discount weights to the price information of the at least two travel products, and calculate price information of the package travel product by adding the cost information and the discount price information.

[0215] According to various embodiments, each of the at least two types of travel products may include at least one of transportation information, accommodation information, tourist attraction information, restaurant information, tour information, or ticket information.

[0216] According to various embodiments, the server further includes a memory storing a table in which the types of travel products are classified into a plurality of classification systems and discount weights are assigned to each of the plurality of classification systems, and the processor may be configured to identify the discount weights corresponding to the classifications of the at least two travel products from the table.

[0217] According to various embodiments, the processor may be configured to request travel date information or travel number information through the intelligent agent if, after identifying a request for creation of the package travel product, there is no travel date information or travel number information in the first context information.

[0218] According to various embodiments, a method of operating a server providing a package travel product while transmitting and receiving a chat message to and from a user device through an intelligent agent may include: causing the user device to display at least one chat message transmitted and received with an intelligent agent based on a large language model (LLM) on a chat window screen; identifying a request for creating a package travel product combining at least two travel products from a user message received from the user device; determining price information of the package travel product using first context information related to a travel schedule; and causing the user device to display an agent message including the price information of the package travel product on a chat window screen through the intelligent agent.

[0219] According to various embodiments, a computer program product comprising one or more programs configured to be executed by one or more processors of a computer system, wherein the one or more programs may include instructions for: causing a user device to display at least one chat message transmitted and received with an intelligent agent based on a large language model (LLM) on a chat window screen; identifying a request for creating a package travel product combining at least two travel products from a user message received from the user device; determining price information of the package travel product using first context information related to a travel itinerary; and causing the user device to display an agent message including the price information of the package travel product on the chat window screen through the intelligent agent.

[0220] According to various embodiments, an electronic device for generating training data to be used for training a large-scale language model includes a display and a processor, wherein the processor is configured to generate, using an application for generating the training data, at least one node among a storage node, a switch node, a branch node, and an output node, generate a node flow that sets a transition relationship between the at least one node and a variable-related value for each of the at least one node, and generate the training data according to a result of executing the node flow in response to a request for generating the training data, wherein the training data may include prompt data corresponding to a virtual user message and completion data corresponding to a virtual agent message.

[0221] According to various embodiments, the at least one node and the node flow are implemented in the form of a graphical user interface, and the node flow may include at least one output node for generating the prompt data and at least one output node for generating the completion data.

[0222] According to various embodiments, the processor may be configured to obtain, as variable-related values ​​for the storage node, a storage variable and a candidate storage variable value, as variable-related values ​​for the switch node, a condition variable, as variable-related values ​​for the branch node, a candidate state value of the condition variable, and as variable-related values ​​for the output node, an output variable and a candidate output variable value.

[0223] According to various embodiments, the processor may be configured to generate the training data with candidate output variable values ​​that include a candidate storage variable value selected for the specific storage variable, upon execution of the node flow, when the candidate output variable value includes a specific storage variable.

[0224] According to various embodiments, the processor may be configured to output a type, an output variable, and content of an output node corresponding to each of the prompt data and the completion data, according to execution of the node flow.

[0225] According to various embodiments, the processor is configured to obtain a number of training data to be generated from a user as a request for generating the training data, and the training data can be generated in a JSON structured format.

[0226] According to various embodiments, a method of operating an electronic device for generating training data to be used for training a large-scale language model includes an operation of generating at least one node among a storage node, a switch node, a branch node, and an output node using an application for generating the training data, an operation of generating a node flow that sets a transition relationship between the at least one node and a variable-related value for each of the at least one node, and an operation of generating the training data according to a result of executing the node flow in response to a request for generating the training data, wherein the training data may include prompt data corresponding to a virtual user message and completion data corresponding to a virtual agent message.

[0227] According to various embodiments, a computer program product comprising one or more programs configured to be executed by one or more processors of a computer system, wherein the one or more programs include instructions for: using an application for generating training data to be used for training a large-scale language model, generating at least one node among a storage node, a switch node, a branch node, and an output node; generating a node flow that sets a transition relationship between the at least one node and a variable-related value for each of the at least one node; and generating the training data according to a result of executing the node flow in response to a request for generating the training data, wherein the training data may include prompt data corresponding to a virtual user message and completion data corresponding to a virtual agent message.

[0228]

[0229] The term "module" or "part" used in this document includes a unit composed of hardware, software, or firmware, and can be used interchangeably with terms such as logic, logic block, component, or circuit, for example. The "module" or "part" can be an integrally configured component or a minimum unit or a part thereof that performs one or more functions. The "module" or "part" can be implemented mechanically or electronically, and can include, for example, an ASIC (application-specific integrated circuit) chip, FPGAs (field-programmable gate arrays), or a programmable logic device, known or to be developed in the future, that performs certain operations, and can be executed by the processor (120). At least a part of the device (e.g., modules or functions thereof) or method (e.g., operations) according to various embodiments can be implemented as instructions stored in a computer-readable storage medium (e.g., memory (130)) in the form of a program module. When the above command is executed by a processor (e.g., processor (120)), the processor can perform a function corresponding to the command. The computer-readable recording medium may include a hard disk, a floppy disk, a magnetic medium (e.g., a magnetic tape), an optical recording medium (e.g., a CD-ROM, a DVD, a magneto-optical medium (e.g., a floptical disk), a built-in memory, etc. The command may include a code generated by a compiler or a code executable by an interpreter. A module or program module according to various embodiments may include at least one or more of the above-described components, some of which may be omitted, or other components may be further included. Operations performed by a module, a program module, or other components according to various embodiments may be executed sequentially, in parallel, iteratively, or heuristically, or at least some operations may be executed in a different order, omitted, or other operations may be added.

[0230] The embodiments disclosed in this document are presented for the purpose of explaining and understanding the disclosed technical content, and do not limit the scope of the present disclosure. Therefore, the scope of the present disclosure should be interpreted to include all modifications or various other embodiments based on the technical concepts of the present disclosure.

Claims

1. In a server that provides real-time contextual information while sending and receiving chat messages with a user device using an intelligent agent, Communication module, and Contains a processor, The above processor, The user device displays at least one chat message transmitted and received with an intelligent agent based on a large language model (LLM) on a chat window screen, In response to identifying first context information related to a travel itinerary from a first user message received from the user device, the user device displays the first context information on a travel itinerary display screen, and The user device is configured to execute instructions to display at least one first object for second context information corresponding to the first search result information on the context display window screen while outputting a first agent message including at least a portion of the first context information and first search result information corresponding to a specific query on the dialog window screen through the intelligent agent. Server.

2. In paragraph 1, The above processor, From the first user message, the first context information is set to identify at least one of travel area information, travel date information, or travel number information. Server.

3. In paragraph 1, The above processor, Obtaining first search result information from a database linked to the large-scale language model using at least some of the first context information and the specific query, and As the second context information, at least one of location information, web link information, or app link information corresponding to the first search result information is set to be identified. Server.

4. In paragraph 1, The above processor, In response to identifying third context information related to a travel location condition from a second user message received from the user device, the intelligent agent verifies at least a portion of the first context information, the third context information, and second search result information determined based on the specific query, and As a reply message to the second user message, a second agent message including the second search result information is output on the dialog window screen, and instructions are further set to be executed to cause the user device to display at least one second object for fourth context information corresponding to the second search result information on the context display window screen. Server.

5. In paragraph 4, The above processor, In response to identifying the third context information, the user device is further configured to execute instructions that cause the third context information to be displayed on the travel itinerary display screen. Server.

6. In paragraph 5, The above processor, In response to identifying another specific query from a third user message received from the user device, identifying at least one context information and third search result information corresponding to the other specific query, and As a reply message to the third user message, a third agent message including the third search result information is output on the dialog window screen, and instructions are further set to be executed to cause the user device to display at least one third object for fifth context information corresponding to the third search result information on the context display window screen. Server.

7. In a method of operating a server that provides real-time contextual information while sending and receiving chat messages with a user device using an intelligent agent, An action that causes the user device to display at least one chat message transmitted and received with an intelligent agent based on a large language model (LLM) on a chat window screen; In response to identifying first context information related to a travel itinerary from a first user message received from the user device, an action causing the user device to display the first context information on a travel itinerary display screen, and An operation of causing the user device to display at least one first object for second context information corresponding to the first search result information on the context display window screen while outputting a first agent message including at least a portion of the first context information and first search result information corresponding to a specific query on the dialog window screen through the intelligent agent, How the server works.

Images