Electronic device and control method therefor

Abstract

This electronic device comprises: a memory for storing instructions; and at least one processor, wherein, when collectively or individually executed by the at least one processor, the instructions instruct the electronic device to: acquire text data on the basis of a user input including a user query; acquire, using an artificial intelligence model, response data by processing the text data when the response data to the text data cannot be acquired on the basis of a keyword corresponding to the acquired text data and the response data to the text data is not included in a query history database; and output a response to the user input on the basis of the response data.

Description

Electronic device and method of controlling the same

[0001] The present disclosure relates to an invention concerning an electronic device and a method for controlling the same, and more specifically, includes an electronic device and a method for controlling the same that acquire response data by augmenting LLM using a metadata database.

[0002] Recently, algorithms for natural language processing have been developing rapidly.

[0003] However, since the size of the LLM (Large Language Model) was large, there was a problem where delay occurred when acquiring response data.

[0004] Therefore, it is necessary to explore ways to reduce the time required to obtain response data as the metadata database is updated.

[0005] According to one or more embodiments of the present disclosure, an electronic device acquires response data by augmenting an LLM using a metadata database, and a control method for the same is included.

[0006] According to one aspect of the present disclosure, an electronic device comprises a memory for storing instructions; and at least one processor. When the instructions are executed collectively or individually by the at least one processor, the electronic device acquires text data based on user input including a user query, and if response data for the text data is not obtainable based on a keyword corresponding to the acquired text data and the response data for the text data is not included in a query history database, the electronic device processes the text data with an artificial intelligence model to acquire the response data and outputs a response to the user input based on the response data.

[0007] When the above instructions are executed collectively or individually by the at least one processor, the electronic device may obtain a weight for at least one record based on a weight value assigned to at least one keyword corresponding to a preset field in the text data, wherein the preset field includes at least one of a movie title, a director's name, a type of genre, an actor's name, and a production year, and the at least one record is stored in a metadata database having a field that matches the preset field corresponding to the at least one keyword, and may obtain the response data having a weight greater than or equal to a threshold value based on the at least one record.

[0008] The above query history database may include i) a feature vector corresponding to the text data and ii) the response data, the feature vector is an embedding vector for the text data, and whether the response data is included in the query history database can be identified based on the feature vector that matches one of the feature vectors among the plurality of feature vectors stored in the query history database.

[0009] The above feature vector can be matched with one of the plurality of feature vectors if the cosine similarity value between the above feature vector and one of the plurality of feature vectors is greater than or equal to a threshold value.

[0010] The above response data may include at least one of data obtained through an artificial intelligence model based on the above text data or augmented data for updating the above metadata database.

[0011] The memory may store a prompt for obtaining the augmented data from the artificial intelligence model, and when the instructions are executed collectively or individually by the at least one processor, the electronic device may input the text data and the prompt into the artificial intelligence model to obtain the augmented data.

[0012] The augmented data may include at least one of genre, title, summary, production year, and rating information, and when the instructions are executed collectively or individually by the at least one processor, the electronic device may enable the query history database to update the augmented data.

[0013] When the above instructions are executed collectively or individually by the at least one processor, the electronic device may transmit the response data to an external device, receive a selection for at least one part of the response data from the external device, and match the text data with the selection and store it in the query history database.

[0014] When the above instructions are executed collectively or individually by the at least one processor, the electronic device may transmit response data corresponding to a preset number of records having the highest weight in the metadata database to an external device if there are no records having a weight greater than the threshold value.

[0015] When the above instructions are executed collectively or individually by the at least one processor, if the electronic device cannot identify the keyword corresponding to the text data, it may process the text data with the artificial intelligence model to obtain the response data and transmit the response data to an external device.

[0016] According to one aspect of the present disclosure, a control method for an electronic device comprises: a step of acquiring text data based on user input including a user query; a step of acquiring response data by processing the text data with an artificial intelligence model if response data for the text data is not obtainable based on keywords corresponding to the acquired text data and the response data for the text data is not included in a query history database; and a step of outputting a response to the user input based on the response data.

[0017] The step of obtaining the response data may include: obtaining a weight for at least one record based on a weight value assigned to at least one keyword corresponding to a pre-set field among the text data, wherein the pre-set field includes at least one of a movie title, a director's name, a type of genre, an actor's name, and a production year, and the at least one record is stored in a metadata database having a field that matches the pre-set field corresponding to the at least one keyword; and obtaining the response data having a weight greater than or equal to a threshold value based on the at least one record.

[0018] The above query history database may include i) a feature vector corresponding to the text data and ii) the response data, wherein the feature vector is an embedding vector for the text data, and whether the response data is included in the query history database can be identified based on the feature vector that matches one of the feature vectors among a plurality of feature vectors stored in the query history database.

[0019] The above feature vector can be matched with one of the plurality of feature vectors if the cosine similarity value between the above feature vector and one of the plurality of feature vectors is greater than or equal to a threshold value.

[0020] The above response data may include at least one of data obtained through an artificial intelligence model based on the above text data or augmented data for updating the above metadata database.

[0021] A prompt for acquiring the augmented data can be stored in memory from the artificial intelligence model, and the augmented data can be acquired by inputting the text data and the prompt into the artificial intelligence model.

[0022] The augmented data may include at least one of genre, title, summary, production year, and rating information, and the control method may further include the step of updating the query history database with the augmented data.

[0023] The above control method may further include the steps of: transmitting the response data to an external device; receiving a selection for at least one part of the response data from the external device; and matching the text data with the selection and storing it in the query history database.

[0024] The above control method may further include the step of transmitting response data corresponding to a preset number of records having the highest weight in the metadata database to an external device if there are no records having a weight greater than or equal to the threshold value.

[0025] According to one aspect of the present disclosure, a non-transient computer-readable recording medium having at least one instruction recorded thereon is configured such that when the at least one instruction is executed collectively or individually by at least one processor, the at least one processor acquires text data based on user input including a user query, and if response data for the text data is not obtainable based on keywords corresponding to the acquired text data and the response data for the text data is not included in a query history database, the processor processes the text data with an artificial intelligence model to acquire the response data and outputs a response to the user input based on the response data.

[0026] The above-described aspects and other aspects, features, and advantages of specific embodiments of the present disclosure can be more clearly understood by referring together with the accompanying drawings and the description below.

[0027] FIG. 1 is a drawing for explaining a system for obtaining response data corresponding to user input according to one embodiment of the present disclosure, and

[0028] FIG. 2 is a block diagram for explaining the configuration of an electronic device according to one embodiment of the present disclosure, and

[0029] FIG. 3 is a flowchart illustrating a case in which keywords included in text data according to one embodiment of the present disclosure cannot be recognized, and

[0030] FIG. 4 is a drawing for illustrating a metadata database according to one embodiment of the present disclosure, and

[0031] FIG. 5 is a flowchart illustrating a process for obtaining response data when response data having a weight greater than or equal to a threshold value can be obtained from a metadata database according to one embodiment of the present disclosure.

[0032] FIG. 6 is a flowchart illustrating an embodiment according to one embodiment of the present disclosure in which response data having a weight greater than or equal to a threshold value cannot be obtained, and

[0033] FIG. 7 is a drawing for illustrating a query history database according to one embodiment of the present disclosure, and

[0034] FIG. 8 is a drawing for explaining a method for obtaining a cosine similarity value between vectors according to one embodiment of the present disclosure, and

[0035] FIG. 9 is a flowchart illustrating an example in which a matching feature vector is not identified in a query history database according to one embodiment of the present disclosure, and

[0036] FIG. 10 is a drawing for explaining a method of acquiring augmented data according to one embodiment of the present disclosure, and

[0037] FIG. 11 is a drawing for explaining an embodiment of displaying response data on a user terminal device, and

[0038] FIG. 12 is a block diagram for explaining the configuration of a user terminal device according to one embodiment of the present disclosure, and

[0039] FIG. 13 is a flowchart illustrating a method for controlling an electronic device according to one embodiment of the present disclosure.

[0040] The embodiments described herein are subject to various modifications and may have various forms; specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope of specific embodiments and should be understood to include various modifications, equivalents, and / or alternatives of the embodiments of the present disclosure. In relation to the description of the drawings, similar reference numerals may be used for similar components.

[0041] In this disclosure, descriptions of related known functions or configurations may be omitted to avoid unnecessarily obscuring the gist of this disclosure.

[0042] Additionally, the following embodiments may be modified in various other forms, and the scope of the technical concept of the present disclosure is not limited to the following embodiments. Rather, these embodiments are provided to make the present disclosure more faithful and complete and to fully convey the technical concept of the present disclosure to those skilled in the art.

[0043] The terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit the scope of the rights. The singular expression includes the plural expression unless the context clearly indicates otherwise.

[0044] In the present disclosure, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of such features (e.g., numerical values, functions, actions, or components such as parts) and do not exclude the presence of additional features.

[0045] In the present disclosure, expressions such as “A or B,” “at least one of A or / and B,” or “one or more of A or / and B” may include all possible combinations of items listed together. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” may refer to cases including (1) A, (2) B, or (3) both A and B.

[0046] Expressions such as "first," "second," "first," or "second" used in this disclosure may modify various components regardless of order and / or importance, and are used only to distinguish one component from another and do not limit said components.

[0047] Where it is stated that a certain component (e.g., a first component) is "(operatively or communicatively) coupled with / to" or "connected to" another component (e.g., a second component), it should be understood that the said certain component may be directly connected to the said other component or connected through another component (e.g., a third component).

[0048] On the other hand, when it is stated that a certain component (e.g., a first component) is "directly connected" or "directly coupled" to another component (e.g., a second component), it may be understood that no other component (e.g., a third component) exists between said certain component and said other component.

[0049] As used in this disclosure, the expression “configured to” may be replaced, depending on the context, with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” may not necessarily mean only “specifically designed to” in hardware.

[0050] Instead, in some situations, the expression “device configured to do something” may mean that the device is “capable of doing something” together with other devices or components. For example, the phrase “processor configured (or set) to perform A, B, and C” may mean a dedicated processor for performing those operations (e.g., an embedded processor), or a generic-purpose processor (e.g., a CPU or application processor) capable of performing those operations by executing one or more software programs stored in a memory device.

[0051] In the embodiments, a 'module' or 'part' performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Additionally, a plurality of 'modules' or a plurality of 'parts' may be integrated into at least one module and implemented by at least one processor, except for the 'module' or 'part' that needs to be implemented in specific hardware.

[0052] Meanwhile, the various elements and areas in the drawings are depicted schematically. Accordingly, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0053] Hereinafter, various embodiments of the present invention will be described in detail using the attached drawings.

[0054] FIG. 1 is a diagram illustrating a system capable of obtaining response data to a user inquiry according to an embodiment of the present disclosure. As illustrated in FIG. 1, the system may include an electronic device (100) and a user terminal device (200). Here, the electronic device (100) may be implemented as a server as illustrated in FIG. 1, but this is merely one embodiment and may be implemented as various electronic devices such as a tablet, a laptop, or a TV. In addition, the user terminal device (200) may be implemented as a TV as illustrated in FIG. 1, but this is merely one embodiment and may be implemented as various terminal devices such as a tablet, a laptop, or a smartphone.

[0055] The user terminal device (200) receives user input including user inquiries (e.g., user voice, user touch, etc.) and can transmit text data corresponding to the received user input to the electronic device (100).

[0056] The electronic device (100) can obtain text data from a user terminal device (200) that says, "I am looking for a movie about a mass murder case. There were many scenes with a lot of snow, and there were also scenes of driving cars."

[0057] The electronic device (100) can identify keywords such as “a scene with heavy snow,” “mass murder,” or “a movie about a murder case” among a plurality of text segments included in text data, and then identify whether there is a matching movie among a plurality of movies stored in a metadata database based on the identified keywords.

[0058] In this case, the 'metadata database' may be a database containing information about media data. For example, if the metadata database contains detailed information about a movie, the metadata database may include 'movie title', 'movie identification number', 'movie director', 'resolution information', 'subtitle information', or 'audio information'. Meanwhile, for the convenience of explanation, the present disclosure will be described below on the premise that the metadata database is a database containing information about a movie. Additionally, the metadata database may be referred to as the first database.

[0059] If the electronic device (100) can identify a matching movie in a metadata database based on keywords included in text data entered by the user, it can transmit information about the matched movie to the user terminal device (200).

[0060] The electronic device (100) can convert text data entered by a user into a feature vector when it is identified that no matching movie exists in the metadata database. At this time, the electronic device (100) can identify whether there is a matching vector among the feature vectors converted from the text data and the multiple feature vectors stored in the query history database. Here, the 'query history database' may be a database that stores matching feature vectors corresponding to text data and response data for text data. Additionally, the 'query history database' may be referred to as a second database.

[0061] For example, the electronic device (100) can identify whether there is a matching feature vector among the feature vectors stored in the query history database and the feature vector converted from “I am looking for a movie about a mass murder. There were many scenes with a lot of snow, and there were also car driving scenes.”

[0062] At this time, if it is identified that there is a matching feature vector, the electronic device (100) can transmit response data corresponding to the matched feature vector in the query history database to the user terminal device (200).

[0063] If the response data corresponding to the matched feature vector is 'Movie AAA', 'Movie BBB', and 'Movie CCC', the electronic device (100) can transmit information about 'Movie AAA', 'Movie BBB', or 'Movie CCC' to the user terminal device (200). As illustrated in FIG. 1, the user terminal device (200) can display information about the received movie on the display (230).

[0064] At this time, if the user selects ‘Movie AAA’, the electronic device (100) can match the converted feature vector of “I am looking for a movie about a mass murder case. There were many scenes with a lot of snow and also scenes of driving” with the data for ‘Movie AAA’ and store (or update) it in the query history database.

[0065] However, if the electronic device (100) fails to identify a matching feature vector in the query history database, it can obtain response data using LLM.

[0066] For example, the electronic device (100) can obtain ‘reinforcement data’ and ‘recommended movie data’ (or response data) by inputting text and prompts such as “I am looking for movies about mass murder. There were many scenes with a lot of snow, and there were also car driving scenes” into the LLM. The electronic device (100) can update the obtained ‘reinforcement data’ in the metadata database. ‘Reinforcement data’ is data for updating the metadata database.

[0067] As illustrated in FIG. 1, the 'enhancement data' may be information about a 'movie AAA'. Specifically, the 'enhancement data' may include the identification number, genre, director, release date, rating, lead actor, summary, and information about the lead actor of the 'movie AAA'. Meanwhile, a method for obtaining the enhancement data will be described below with reference to FIG. 10.

[0068] The electronic device (100) can provide information about 'movie AAA' if another user enters similar text data as it updates the query history database and metadata database.

[0069] FIG. 2 is a block diagram for explaining the configuration of an electronic device (100).

[0070] The configuration illustrated in FIG. 2 is merely an example of various embodiments, and additional or different configurations may be added. As illustrated in FIG. 2, the electronic device (100) may include a communication interface (110), a memory (120), and a processor (130). The configuration illustrated in FIG. 2 is merely an example, and it goes without saying that some configurations may be changed or added depending on the configuration of the electronic device (100).

[0071] First, the communication interface (110) is configured to communicate with various types of external devices according to various types of communication methods. In particular, the communication interface (110) can transmit response data to the user terminal device (200). Additionally, the communication interface (110) may receive information about the selected data when the user selects one of the response data from the user terminal device (200).

[0072] A wireless communication module may be a module that communicates wirelessly with an external device. For example, the wireless communication module may include at least one module among a Wi-Fi module, a Bluetooth module, an infrared communication module, an Ultra Wide-Band (UWB) module, or other communication modules.

[0073] A wired communication module may be a module that communicates with an external device via a wire. For example, a wired communication module may include at least one of a Local Area Network (LAN) module, an Ethernet module, a pair cable, a coaxial cable, or a fiber optic cable.

[0074] The memory (120) can store instructions or information related to the operating system (OS) for controlling the overall operation of the components of the electronic device (100) and the components of the electronic device (100). In particular, the memory (120) can store information on a method for identifying at least one keyword corresponding to a pre-set field among a plurality of text segments included in text data. Additionally, the memory (120) may store a metadata database or a query history database.

[0075] Meanwhile, it goes without saying that the metadata database or query history database may be stored on an external device.

[0076] The memory (120) may also store response data obtained by inputting text data into the LLM or augmented data for updating the metadata database.

[0077] Meanwhile, for the sake of convenience of explanation, the following description assumes that the LLM model is stored in the memory (120), but this is merely one embodiment, and the LLM model can be stored in an external device and can be received from an external device.

[0078] Memory (120) can be implemented in various forms such as volatile memory (e.g., DRAM (dynamic RAM), SRAM (static RAM), or SDRAM (synchronous dynamic RAM), non-volatile memory (e.g., OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD).

[0079] The processor (130) may include one or more processors. Specifically, the one or more processors may include one or more of a CPU (Central Processing Unit), GPU (Graphics Processing Unit), APU (Accelerated Processing Unit), MIC (Many Integrated Core), DSP (Digital Signal Processor), NPU (Neural Processing Unit), hardware accelerator, or machine learning accelerator. The processor (130) may control one or any combination of other components of an electronic device and may perform operations or information processing related to communication. The one or more processors may execute one or more programs or instructions stored in memory. For example, the one or more processors may perform a method according to one embodiment of the present disclosure by executing one or more instructions stored in memory.

[0080] One or more processors may be implemented as a single-core processor comprising one core, or as one or more multicore processors comprising multiple cores (e.g., homogeneous multicore or heterogeneous multicore). When one or more processors are implemented as multicore processors, each of the multiple cores included in the multicore processor may include internal processor memory such as cache memory or on-chip memory, and a common cache shared by multiple cores may be included in the multicore processor. Additionally, each of the multiple cores included in the multicore processor (or some of the multiple cores) may independently read and execute program instructions for implementing a method according to one embodiment of the present disclosure, or all (or some) of the multiple cores may be linked together to read and execute program instructions for implementing a method according to one embodiment of the present disclosure.

[0081] In particular, the processor (130) can acquire text data based on user input including a user query, identify whether response data for the text data can be acquired based on keywords corresponding to the acquired text data, and if it is identified that response data cannot be acquired based on the keywords, identify whether response data for the text data is included in the query history database, and if it is identified that response data is not included in the query history database, process the text data with an artificial intelligence model to acquire response data and output a response to the user input based on the response data.

[0082] Additionally, the processor (130) may obtain (calculate) a weight for at least one record based on a weight value assigned to at least one keyword, and obtain response data having a weight greater than or equal to a threshold value based on at least one record. Here, the field may include at least one of a movie title, a director's name, a type of genre, an actor's name, and a production year.

[0083] Additionally, the processor (130) can identify the existence of response data corresponding to text data based on the query history database when a feature vector corresponding to text data matches one of the multiple feature vectors stored in the query history database. The query history database contains a matching feature vector corresponding to text data and response data for text data, and the feature vector can be an embedding vector for text data. The processor (130) can identify that a match has occurred if the cosine similarity value between the acquired feature vector and one of the multiple feature vectors included in the query history database is greater than or equal to a threshold value. At this time, the response data may include at least one of data acquired through an artificial intelligence model based on text data or augmented data for updating a metadata database. Additionally, the memory stores a prompt corresponding to the acquisition of augmented data, and the processor (130) can acquire augmented data by inputting text data and the prompt into the artificial intelligence model.

[0084] The processor (130) can update the acquired augmented data by storing it in a query history database, and the augmented data may include at least one of genre, title, summary content, production year, and rating information. The processor (130) can transmit response data to an external device, receive a selection for at least one part of the response data from the external device, and match text data with the selection and store it in the query history database.

[0085] If there are no records with a weight greater than or equal to a threshold value, the processor (130) can transmit response data corresponding to a preset number of records with the highest weight in the metadata database to an external device. Additionally, if the processor (130) cannot identify keywords corresponding to text data, it can obtain response data through an artificial intelligence model based on text data and transmit the obtained response data to an external device.

[0086] Meanwhile, each of the above-described operations will be described later with reference to FIGS. 3 to FIGS. 10.

[0087] First, FIG. 3 is a flowchart illustrating a case where keywords included in text data according to one embodiment of the present disclosure cannot be recognized.

[0088] First, the electronic device (100) can acquire text data (S310). At this time, the text data may be data obtained from the user terminal device (200) containing the user's voice and converted into text. At this time, the voice signal can be received using the microphone (240) of the user terminal device (200). However, this is merely one embodiment, and it is obvious that the remote control can be equipped with a microphone so that the remote control can receive the user's voice signal.

[0089] In addition, it is possible to input a voice signal to the user terminal device (200) by running a remote control application on the user terminal device (200).

[0090] Meanwhile, the remote control can communicate with the electronic device (100) using a communication interface, and it goes without saying that the type of communication interface used by the electronic device (100) and the type of communication interface used by the remote control may be the same or different.

[0091] The electronic device (100) can identify whether a keyword can be identified among a plurality of text segments included in text data (S320). Specifically, the electronic device (100) can identify at least one keyword corresponding to a pre-set field among a plurality of text segments included in text data, and can obtain a weight corresponding to the text data by assigning a weight value to each of the at least one identified keyword. In addition, the electronic device (100) can identify response data having a weight greater than or equal to a threshold value based on a metadata database. At this time, the field may include at least one of a movie title, a director's name, a type of genre, an actor's name, and a production year.

[0092] For example, the electronic device (100) can obtain text data containing the content "Recommend any interesting movie." At this time, the electronic device (100) can identify whether it can identify a field corresponding to one of 'movie title', 'director's name', 'type of genre', 'actor's name', or 'production year' in a plurality of text segments included in the text data.

[0093] If the electronic device (100) cannot identify keywords in the text data, it can input the text data into the LLM to obtain response data (S330). For example, if the electronic device (100) inputs text data "Recommend any fun movie" into the LLM, it can obtain response data such as "1. Action / Thriller - Movie AAA, Movie BBB, 2. Comedy - Movie CCC 3. Animation - Movie DDD".

[0094] After the electronic device (100) acquires the response data, it can transmit it to the user terminal device (200) (S340).

[0095] The electronic device (100) can store text data and selected data in a metadata database when the user selects one of the response data.

[0096] Specifically, the electronic device (100) can obtain input data from a user who has selected one of a plurality of response data from a user terminal device (200). At this time, the electronic device (100) can store text data and the response data selected by the user in a metadata database. At this time, the metadata database will be described later with reference to FIG. 4.

[0097] As illustrated in FIG. 4, the electronic device (100) can store a database containing various information about media data.

[0098] *85 As illustrated in Fig. 4, the metadata database may include information about the identification number (contents_id), title, director, or leading actor of multiple media data.

[0099] However, this is merely one example, and the metadata database may include information regarding the screening time, release year, country, and viewing rating of the video.

[0100] FIG. 5 is a flowchart illustrating the process of obtaining response data when response data having a weight greater than or equal to a threshold value can be obtained from a metadata database according to one embodiment of the present disclosure.

[0101] Meanwhile, S510 to S520 correspond to S310 to S320 of FIG. 3. Therefore, for details regarding additional implementation of S510 to S520, the description of S310 to S320 may be referenced.

[0102] The electronic device (100) can obtain a weight corresponding to the text data by assigning a weight value to each of at least one identified keyword (S530).

[0103] Specifically, the electronic device (100) can identify at least one keyword corresponding to a pre-set field among a plurality of text segments included in text data.

[0104] For example, the electronic device (100) can set the fields to title, director, and genre. At this time, the electronic device (100) can set a weight value of 9 for the 'title' field, 7 for the 'director' field, and 5 for the 'genre' field.

[0105] The electronic device (100) can obtain text data such as "Recommend an animated movie among the movies made by director AAA."

[0106] The electronic device (100) can assign a value of 0 to the 'Title' field of 'Video 1' in the metadata database, assign a value of 0.9 to the 'Director' field, and assign a weight value of 0.6 to the genre. In this case, the weight assigned to 'Video 1' can be identified as '9*0 + 7*0.6 + 5*0.6', that is, 7.2.

[0107] The electronic device (100) can obtain a weight summed for each data included in the metadata database by the method described above. The electronic device (100) can obtain text data having a weight greater than or equal to a threshold value (S540).

[0108] The electronic device (100) can transmit response data having a weight greater than or equal to a threshold value to the user terminal device (200) (S550). For example, if the threshold value is '7', the electronic device (100) can transmit 'Image 1' to the user terminal device (200) because the summed weight of 'Image 1' is '7.1'.

[0109] The above-described embodiment describes a case where response data having a weight greater than or equal to a threshold value can be obtained from a metadata database, but an embodiment in which response data having a weight greater than or equal to a threshold value cannot be obtained will be described below. This will be described below with reference to FIG. 6.

[0110] First, S610 to S630 of FIG. 6 correspond to S510 to S530 of FIG. 5. Therefore, for details regarding additional implementation of S610 to S630, the description of S510 to S530 may be referenced.

[0111] The electronic device (100) can identify that there is no text data with a weight greater than or equal to a threshold value (S640).

[0112] At this time, the electronic device (100) can convert text data into a feature vector (S650). The feature vector can be an embedding vector for the text data.

[0113] The electronic device (100) can identify whether there is a matching vector among a plurality of feature vectors stored in the feature vector and query history database (S660).

[0114] At this time, the query history database will be described later with reference to Fig. 7.

[0115] FIG. 7 is a drawing for illustrating a query history database according to one embodiment of the present disclosure.

[0116] The query history database can include feature vectors corresponding to text data and response data matched to the text data.

[0117] For example, if the text data is "a car chase movie with a scene of heavy snowfall," the text data can be converted into a feature vector of [0.15, 0.22, 0.35].

[0118] At this time, the response data for "a car chase movie with a scene of heavy snowfall" can be "Movie AAA", and the identification number of "Movie AAA" can be "123456a9c". The electronic device (100) can store the feature vector [0.15, 0.22, 0.35] and the response data (contents_id) of "123456a9c" in the query history database by matching them.

[0119] Meanwhile, for the sake of convenience of explanation, the above description was made on the premise that one feature vector and one response data are matched; however, this is merely one embodiment, and it goes without saying that multiple feature vectors and one response data can be matched.

[0120] The electronic device (100) can identify cosine similarity values ​​between feature vectors to identify whether there is a matching vector among a plurality of feature vectors stored in a feature vector and query history database.

[0121] This will be described later with reference to Fig. 8.

[0122] FIG. 8 is a diagram illustrating a method for obtaining a cosine similarity value between vectors according to one embodiment of the present disclosure.

[0123] Multiple feature vectors stored in the query history database can be represented as multiple vectors as shown in Fig. 8.

[0124] For example, the feature vector obtained by the electronic device (100) may be a feature vector corresponding to "a movie with a warm atmosphere." At this time, the electronic device (100) can identify that the similarity value between the obtained feature vector and the feature vector corresponding to "a romance movie" among the feature vectors stored in the query history database is greater than or equal to a threshold value. Therefore, the electronic device (100) can identify that the feature vector for "a movie with a warm atmosphere" and the feature vector for "a romance movie" are matched.

[0125] On the other hand, the electronic device (100) can identify that the cosine similarity value is below a threshold value because the angle between the feature vector for "a movie with a warm atmosphere" and the feature vector for "a movie containing a murder case" is large. At this time, the electronic device (100) can identify that the feature vector for "a movie with a warm atmosphere" and the feature vector for "a movie containing a murder case" do not match.

[0126] Meanwhile, for the sake of convenience of explanation, a method for obtaining Cosine Similarity values ​​to identify whether multiple feature vectors stored in the acquired feature vector and query history database match has been described in detail; however, this is merely one example, and it goes without saying that similarity can be determined using Euclidean similarity or vector similarity.

[0127] Returning to FIG. 6, if the electronic device (100) identifies that there is a matching vector in the query history database by the method described above, it can obtain response data for the matched vector in the query history database (S670).

[0128] Specifically, when the electronic device (100) identifies a vector that matches the acquired feature vector in the query history database, it can obtain response data corresponding to the matched vector from the query history database.

[0129] For example, the electronic device (100) can identify that a vector matching the acquired feature vector is [0.28, 0.45, 0.62]. At this time, the electronic device (100) can identify that the response data corresponding to the matching vector is "987654dde" or "abcdef312", as illustrated in FIG. 7.

[0130] The electronic device (100) can transmit response data with a weight less than a threshold value and response data obtained from a query history database to a user terminal device (200) (S680).

[0131] For example, if the movies for the previously acquired response data "987654dde" and "abcdef312" are "Movie AAA" and "Movie BBB", the electronic device (100) can transmit information about "Movie AAA" and "Movie BBB" to the user terminal device (200).

[0132] In addition, the electronic device (100) may acquire a preset number of response data having a weight less than a threshold value from a metadata base and transmit the acquired response data to a user terminal device (200) in order to provide various response data to the user. At this time, the acquired response data may be provided in order to provide various response data, although the accuracy of the user's request may be reduced.

[0133] Meanwhile, an example in which matching feature vectors are identified in a query history database has been described in detail. Below, a case in which a feature vector matching a feature vector corresponding to text data in a query history database is not identified will be described later with reference to FIG. 9.

[0134] FIG. 9 is a flowchart illustrating an example in which a matching feature vector is not identified in a query history database according to one embodiment of the present disclosure.

[0135] First, S905 to S920 overlap with the contents of S610 to S640 of FIG. 6. Therefore, for details regarding the additional implementation of S905 to S920, the description of S610 to S640 may be referenced.

[0136] First, the electronic device (100) can identify whether there is a matching vector among the converted vectors and the vectors stored in the query history database (S925). Meanwhile, this is described in detail in FIG. 6. Therefore, for details regarding additional implementation, the description of FIG. 6 may be referenced.

[0137] If the electronic device (100) identifies that no matching vector exists, it can input text data and a stored prompt into the LLM (S930). Meanwhile, for the sake of convenience of explanation, the following description assumes that the LLM is stored in memory (120), but this is merely one embodiment and it is obvious that it may be stored in an external device.

[0138] At this time, the electronic device (100) can obtain response data by inputting text data and a stored prompt into the LLM (S935). This will be described later with reference to FIG. 10.

[0139] Specifically, the memory (120) stores a prompt for acquiring augmented data, and can acquire augmented data by inputting text data and the prompt into the LLM.

[0140] As illustrated in FIG. 10, the electronic device (100) can obtain text data from the user terminal device (200) that says, "I am looking for a movie about a mass murder case. There were many scenes with a lot of snow, and there were also scenes of driving cars."

[0141] At this time, the memory (120) may store a prompt in the format [{"Title: "1st priority content title", "contents_id": "corresponding contents_id"}]. For convenience of explanation, the first prompt will be described below.

[0142] In addition, a prompt [{Title: "Top 1 Content Title", "contents_id":"Relevant contents_id", "Genre": "Genre of the relevant content", "Viewer Score":"User score on IMDB"}, "Summary": "A summary of the movie content in 100 characters or less",..] 쪋.] can be stored for updating the metadata database based on the user's response results. For the convenience of explanation, this will be described below as the second prompt.

[0143] For example, the electronic device (100) can input text data, the first prompt and the second prompt shown in FIG. 9, into the LLM, such as “I am looking for a movie about a mass murder case. There were many scenes with a lot of snow, and there were also scenes of driving cars.”

[0144] At this time, the electronic device (100) can obtain response data in the format of the first prompt and response data in the format of the second prompt, satisfying the condition of “I am looking for a movie about a mass murder case. There were many scenes with a lot of snow, and there were also scenes of driving cars.”

[0145] Again, returning to FIG. 9, the electronic device (100) can transmit response data with a weight less than a threshold and response data obtained using LLM to a user terminal device (200) (S940).

[0146] Meanwhile, this will be described in detail later with reference to Fig. 11.

[0147] The electronic device (100) can transmit multiple response data to the user terminal device (200). At this time, the electronic device (100) can first transmit response data having a weight less than a threshold value to the user terminal device (200). At this time, the user terminal device (200) can display the response data having a weight less than a threshold value through the display (230).

[0148] Afterward, the electronic device (100) can input text data and a prompt into the LLM and transmit the acquired response data to the user terminal device (200). The user terminal device (200) can display the response data acquired through the LLM via the display (230).

[0149] The electronic device (100) may first transmit response data having a weight below a threshold value to the user terminal device (200). For example, the response data having a weight below a threshold value may be 'Movie AAA', 'Movie BBB', and 'Movie CCC'. At this time, as shown on the left side of FIG. 11, the user terminal device (200) may display information about 'Movie AAA', 'Movie BBB', and 'Movie CCC' on the display (230). The order in which the movies are displayed may be in order of highest weight.

[0150] The electronic device (100) can transmit response data having a weight less than a threshold value to the user terminal device (200), and then transmit information about 'movie aaa' and 'movie bbb', which are response data obtained by inputting text data and a prompt into the LLM, to the user terminal device (200). At this time, as shown on the right side of FIG. 11, information about 'movie aaa' and 'movie bbb' may be displayed on the far left, and information about 'movie AAA' and 'movie BBB' may be displayed on the right. Meanwhile, it goes without saying that the order in which information about the movies is displayed may vary.

[0151] FIG. 12 is a block diagram for explaining the configuration of a user terminal device (200) according to one embodiment of the present disclosure.

[0152] The configuration illustrated in FIG. 12 is merely an example of various embodiments, and some configurations may be changed or added. As illustrated in FIG. 12, the user terminal device (200) may include a communication interface (210), memory (220), display (230), microphone (240), user interface (250), input / output interface (260), speaker (270), and processor (280). The configuration illustrated in FIG. 12 is merely an example, and it goes without saying that some configurations may be deleted or added depending on the configuration of the user terminal device (200).

[0153] Meanwhile, regarding details of the implementation of configurations that overlap with the configuration shown in FIG. 2, such as the communication interface (210), memory (220), and processor (280) among the configurations of the user terminal device (200), one may refer to the description of FIG. 2.

[0154] The communication interface (210) is configured to communicate with various types of external devices according to various types of communication methods. In particular, the communication interface (210) can transmit information about text entered by a user to the electronic device (100). Meanwhile, this is merely one embodiment, and the electronic device (100) can convert a voice signal entered by a user into text and transmit the data converted into text.

[0155] Meanwhile, the communication interface (210) receives response data from the electronic device (100), and if the user selects one of the received response data, it may transmit information about the selected response data to the electronic device (100).

[0156] The memory (220) can store instructions or information related to the operating system (OS) for controlling the overall operation of the components of the user terminal device (200) and the components of the user terminal device (200). In particular, the memory (220) can store data entered by the user.

[0157] The display (230) can display various information. Specifically, the display (230) can display response data received from the electronic device (100). Meanwhile, as described in detail in FIG. 11, the display (230) can display multiple response data having weights below a threshold value and subsequently display response data obtained using LLM. However, this is only one embodiment, and it is obvious that the display (230) can simultaneously display response data having weights below a threshold value and response data obtained using LLM.

[0158] The display (230) can be implemented as various types of displays such as an LCD (Liquid Crystal Display), an OLED (Organic Light Emitting Diodes) display, and a PDP (Plasma Display Panel). The display may also include a driving circuit, a backlight unit, etc., which can be implemented in forms such as an a-si TFT (amorphous silicon thin film transistor), an LTPS (low temperature poly silicon) TFT, and an OTFT (organic TFT). The display can be implemented as a touch screen combined with a touch sensor, a flexible display, a 3D display, a three-dimensional display, etc. According to various embodiments of the present disclosure, the display (230) may include not only a display panel that outputs an image, but also a bezel that houses the display panel.

[0159] The microphone (240) may refer to a module that acquires sound and converts it into an electrical signal, and may be a condenser microphone, ribbon microphone, moving coil microphone, piezoelectric element microphone, carbon microphone, or MEMS (Micro Electro Mechanical System) microphone. Additionally, it may be implemented in omnidirectional, bidirectional, unidirectional, subcardioid, supercardioid, or hypercardioid modes.

[0160] According to one embodiment, the microphone (240) can receive a voice command from a user. According to one example, the microphone (240) can acquire a sound corresponding to the voice command and convert it into an audio signal.

[0161] The user interface (250) may be implemented as a device such as a button, touchpad, mouse, and keyboard, or as a touch screen capable of performing the aforementioned display function and operation input function. Here, the button may be a various type of button, such as a mechanical button, touchpad, or wheel, formed in any area such as the front, side, or back of the main body exterior of the user terminal device (200). Data for inputting one of a plurality of response data can be obtained through the user interface (250).

[0162] The input / output interface (260) is configured to input or output at least one of audio and video signals. For example, the input / output interface (260) may be HDMI (High Definition Multimedia Interface), but this is merely an example of an embodiment, and it may be any one of MHL (Mobile High-Definition Link), USB (Universal Serial Bus), DP (Display Port), Thunderbolt, VGA (Video Graphics Array) port, RGB port, D-SUB (D-subminiature), or DVI (Digital Visual Interface). Depending on the implementation example, the input / output interface (260) may include a port for inputting and outputting only audio signals and a port for inputting and outputting only video signals as separate ports, or it may be implemented as a single port for inputting and outputting both audio and video signals.

[0163] The speaker (270) is configured to output various audio data, as well as various notification sounds or voice messages, after various processing operations such as decoding, amplification, and noise filtering have been performed by the audio processing unit. In particular, the speaker (270) can output voice for receiving user input. Of course, the speaker (270) can also output voice even when providing response data before receiving user input. The configuration for outputting audio can be implemented as a speaker, but this is merely one embodiment, and it can be implemented as an output terminal capable of outputting audio data.

[0164] The processor (280) can receive response data from the electronic device (100) using the communication interface (210). At this time, after receiving the response data, the response data can be displayed through the display (230). If a user inputs one of the data in response data displayed, the input data can be transmitted to the electronic device (100).

[0165] FIG. 13 is a flowchart illustrating a method for controlling an electronic device according to one embodiment of the present disclosure.

[0166] First, if the electronic device (100) fails to obtain response data for the text based on a keyword corresponding to the acquired text data, it can identify whether response data for the text data exists based on a query history database (S1310). At this time, at least one keyword corresponding to a pre-set field among a plurality of text segments in the text data can be identified, and a weight can be assigned to the identified keyword. This has been described in detail above.

[0167] If the electronic device (100) identifies that there is no response data corresponding to the text data, it can obtain response data through an artificial intelligence model based on the text data (S1320). Meanwhile, although an embodiment for obtaining response data using an LLM stored in the memory of the electronic device (100) has been described above, this is merely one embodiment, and it is obvious that the electronic device (100) can obtain response data from an external device. At this time, the response data may include at least one of data obtained by inputting text data into the LLM or augmented data for updating a metadata database.

[0168] According to the present disclosure, the electronic device (100) can store text data entered by a user and response data to the text data in a metadata database. In addition, since the process of inputting text data into an LLM model according to the user's input result is not performed repeatedly, the computation process can be performed efficiently even with limited resources.

[0169] Additionally, methods according to various embodiments of the present disclosure may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user (20) devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., downloadable app) may be temporarily stored or temporarily created in a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0170] A method according to various embodiments of the present disclosure may be implemented as software comprising instructions stored on a machine-readable storage medium (e.g., a computer). The machine may include a server device or an electronic device according to the disclosed embodiments, which is a device capable of calling instructions stored from the storage medium and operating according to the called instructions.

[0171] Meanwhile, a device-readable storage medium may be provided in the form of a non-transitory readable recording medium. Here, 'non-transitory readable recording medium' simply means that it is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where information is stored semi-permanently and cases where it is stored temporarily. For example, a 'non-transitory storage medium' may include a buffer in which information is stored temporarily.

[0172] When the above instruction is executed by a processor, the processor may perform the function corresponding to the instruction directly or by using other components under the control of the processor. The instruction may include code generated or executed by a compiler or an interpreter.

[0173] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure.

Claims

1. In an electronic device, Memory for storing instructions; Includes at least one processor; and When the above instructions are executed collectively or individually by the at least one processor, the electronic device, Acquire text data based on user input including user queries, and If response data for the text data cannot be obtained based on keywords corresponding to the acquired text data, and the response data for the text data is not included in the query history database, the text data is processed by an artificial intelligence model to obtain the response data, and An electronic device that outputs a response to the user input based on the above response data.

2. In Paragraph 1, When the above instructions are executed collectively or individually by the at least one processor, the electronic device, A weight for at least one record is obtained based on a weight value assigned to at least one keyword corresponding to a pre-set field among the above text data, and Here, the aforementioned pre-configured field includes at least one of a movie title, director's name, genre type, actor's name, and production year, and the above at least one record is stored in a metadata database having a field that matches the pre-configured field corresponding to the above at least one keyword, and An electronic device for obtaining the response data having a weight greater than or equal to a threshold value based on at least one record.

3. In Paragraph 1, The above query history database includes i) a feature vector corresponding to the above text data and ii) the above response data, wherein the feature vector is an embedding vector for the above text data, and An electronic device in which whether the above response data is included in the above query history database is identified based on the feature vector that matches one of the multiple feature vectors stored in the above query history database.

4. In Paragraph 3, The above feature vector is an electronic device that matches one of the plurality of feature vectors if the cosine similarity value between the above feature vector and one of the plurality of feature vectors is greater than or equal to a threshold value.

5. In Paragraph 2, The electronic device comprising at least one of the above response data obtained through an artificial intelligence model based on the above text data or augmented data for updating the above metadata database.

6. In Paragraph 5, The above memory stores a prompt for obtaining the augmented data from the above artificial intelligence model, and The above instructions, when executed collectively or individually by the at least one processor, are an electronic device that inputs the text data and the prompt into the artificial intelligence model to acquire the augmented data.

7. In Paragraph 6, The above augmented data includes at least one of genre, title, summary, production year, and rating information, and The above instructions, when executed collectively or individually by the at least one processor, enable the electronic device to update the query history database with the augmented data.

8. In Paragraph 1, When the above instructions are executed collectively or individually by the at least one processor, the electronic device, Transmit the above response data to an external device, and Receive a selection for at least one part of the response data from the above external device, and An electronic device that matches the above text data with the above selection and stores it in the above query history database.

9. In Paragraph 2, An electronic device that, when the above instructions are executed collectively or individually by the at least one processor, transmits response data corresponding to a preset number of records having the highest weight in the metadata database to an external device if there are no records having a weight greater than the threshold value.

10. In Paragraph 1, When the above instructions are executed collectively or individually by the at least one processor, the electronic device, If the keyword corresponding to the above text data cannot be identified, the text data is processed by the artificial intelligence model to obtain the response data, and An electronic device that transmits the above response data to an external device.

11. A method for controlling an electronic device, A step of obtaining text data based on user input including a user query; If response data for the text data cannot be obtained based on keywords corresponding to the acquired text data and the response data for the text data is not included in the query history database, the step of processing the text data with an artificial intelligence model to obtain the response data; and A control method comprising the step of outputting a response to the user input based on the above response data.

12. In Paragraph 11, The step of obtaining the above response data is, A step of obtaining a weight for at least one record based on a weight value assigned to at least one keyword corresponding to a pre-set field among the text data, wherein the pre-set field includes at least one of a movie title, a director's name, a type of genre, an actor's name, and a production year, and the at least one record is stored in a metadata database having a field that matches the pre-set field corresponding to the at least one keyword; and A control method comprising the step of obtaining response data having a weight greater than or equal to a threshold value based on at least one record.

13. In Paragraph 11, The above query history database includes i) a feature vector corresponding to the above text data and ii) the above response data, wherein the feature vector is an embedding vector for the above text data, and A control method in which whether the above response data is included in the above query history database is identified based on the feature vector that matches one of the multiple feature vectors stored in the above query history database.

14. In Paragraph 13, A control method in which the above feature vector is matched with one of the plurality of feature vectors if the cosine similarity value between the above feature vector and one of the plurality of feature vectors is greater than or equal to a threshold value.

15. In Paragraph 12, A control method comprising at least one of the above response data obtained through an artificial intelligence model based on the above text data or augmented data for updating the above metadata database.

Images