Apparatus, method and program for providing content search service

Abstract

There is provided a server including a communication device configured to be communicatively connected to an electronic device, a database for storing search information for content search by category, and a processor configured to provide a search result corresponding to a content search request of the electronic device, and the processor obtains search information for searching for at least one content from the electronic device, determines additional required information based on the obtained search information and requests the additional required information from the electronic device, determines final search information based on additional information obtained from the electronic device in response to a request for the additional required information, and provides a search result to the electronic device based on the final search information.

Description

APPARATUS, METHOD AND PROGRAM FOR PROVIDING CONTENT SEARCH SERVICE

[0001] The present invention relates to an apparatus, a method, and a program for providing a content search service capable of providing optimal search results to a user requesting a content search.

[0002] As online and mobile video consumption rapidly increases due to high-speed mobile communications, the widespread use of smartphones, and the like, over-the-top (OTT) services, which are online streaming platforms on the web and mobile in addition to existing TV-based broadcasting platforms such as terrestrial broadcasting, cable broadcasting, satellite broadcasting, IPTV, and the like, are expanding.

[0003] The size of the OTT market is increasing every year, and as mobile communication technology advances, the demand for OTT services is expected to further increase.

[0004] Accordingly, in OTT services, the development of services that are not provided by existing broadcasting platforms, or services capable of giving users enjoyment while using content, is being developed.

[0005] The present invention is directed to providing an optimal search result with high accuracy and reliability to a user requesting a content search using final search information determined based on additional information from the user.

[0006] The present invention is directed to providing an optimal search result to a user by retraining and updating a content search model based on content finally selected by the user from a search result and final search information used in the search result.

[0007] According to one aspect, there is provided a server including a communication device configured to communicate with an electronic device, a database for storing search information for content search by category, and a processor configured to provide a search result corresponding to a content search request of the electronic device, wherein the processor obtains search information for searching for at least one content from the electronic device, determines additional required information based on the obtained search information, requests the additional required information from the electronic device, determines final search information based on additional information obtained from the electronic device in response to a request for the additional required information, and provides a search result to the electronic device based on the final search information.

[0008] According to an embodiment, wherein the processor obtains search request data input to the electronic device based on a content search request received from the electronic device, and extracts the search information from the search request data.

[0009] According to an embodiment, wherein the processor extracts a search term from text data included in the search request data and obtains the search information corresponding to the search term.

[0010] According to an embodiment, wherein the processor extracts text data from image data included in the search request data and obtains the search information from the text data.

[0011] According to an embodiment, wherein the processor calculates a relevance score for each of a plurality of text segments extracted from the image data with respect to objects represented in the image data, selects a text segment with highest relevance score, and obtains the search information from the selected text segment.

[0012] According to an embodiment, wherein the processor converts sound data included in the search request data into text data, extracts a search term from the text data, and obtains the search information corresponding to the search term.

[0013] According to an embodiment, wherein the processor performs a content search based on the search information, determines an initial search result based on the content search, and determines the additional required information for specifying the final search information based on the determined initial search result.

[0014] According to an embodiment, wherein the processor determines the additional required information when the amount of content information corresponding to the initial search result is greater than or equal to a preset reference amount of information.

[0015] According to an embodiment, wherein the processor provides the initial search result to the electronic device when the amount of content information corresponding to the initial search result is less than the preset reference amount of information.

[0016] According to an embodiment, wherein the processor selects a search category based on at least one of a content genre, an actor, and a synopsis corresponding to the initial search result, and determines the additional required information based on the selected search category.

[0017] According to an embodiment, wherein the processor selects the search category further based on a user's search history for at least one of the content genre, the actor, and the synopsis corresponding to the initial search result.

[0018] According to an embodiment, wherein the processor selects the search category including a higher-level category and a subcategory of the higher-level category when the subcategory corresponding to the higher-level category exists.

[0019] According to an embodiment, wherein the processor provides a search category included in the additional required information and the initial search result to the electronic device so that a main search icon corresponding to the search category is displayed over the initial search result on the electronic device.

[0020] According to an embodiment, wherein the processor receives, from the electronic device, a user input for selecting the main search icon corresponding to the search category, and, when a subcategory corresponding to the selected main search icon exists, provides the subcategory to the electronic device so that a sub-search icon corresponding to the subcategory is displayed with the main search icon on the electronic device.

[0021] According to an embodiment, wherein the processor receives, from the electronic device, a user input for selecting the sub-search icon, provides items included in a subcategory of the selected sub-search icon to the electronic device so that an input field for a search weight for each item is displayed on the electronic device, and receives the search weight for each item input through the input field.

[0022] According to an embodiment, wherein the processor provides a search category included in the additional required information and the initial search result to the electronic device so that a speech bubble corresponding to the search category is displayed over the initial search result on the electronic device.

[0023] According to an embodiment, wherein the processor receives, from the electronic device, a user's response to the additional required information, performs an additional content search based on the user's response, determines an additional search result based on the additional content search, and, when the final search information is not determined based on the additional search result, requests further additional required information for determining the final search information from the electronic device.

[0024] According to an embodiment, wherein the processor performs a content search based on the additional information, determines an intermediate search result based on the content search, determines further additional required information based on the intermediate search result when the amount of information in the intermediate search result is greater than or equal to a preset reference amount of information, requests the further additional required information from the electronic device, and determines the final search information based on further additional information obtained from the electronic device in response to a request for the further additional required information.

[0025] According to an embodiment, wherein the processor provides at least one of main information about finally retrieved content and a thumbnail, a replay video, or a preview video of the finally retrieved content to the electronic device based on the search result.

[0026] According to an embodiment, wherein the processor inputs the final search information into a pre-trained search model, and obtains the search result including content that a user wants to search for.

[0027] According to an embodiment, wherein the processor obtains content information classified by category, obtains metadata of content corresponding to the content information, trains the search model based on the content information and the metadata, obtains a training result, retrains the search model based on the training result, and updates the search model.

[0028] According to an embodiment, wherein the processor receives, from the electronic device, a user input for selecting specific content from the search result, analyzes a correlation between attributes of the selected content and the final search information, retrains the search model based on the correlation, and updates the search model.

[0029] According to an embodiment, wherein the processor obtains at least one search category corresponding to the selected content, obtains at least one search term included in the final search information, matches the search category of the content with the search term of the final search information, and analyzes the correlation between the attributes of the selected content and the final search information based on the matching result.

[0030] According to another aspect, there is provided a method for providing a content search service of a server, including: obtaining search information for searching for at least one content from an electronic device; determining additional required information based on the obtained search information; requesting the additional required information from the electronic device; obtaining additional information in response to a request for the additional required information from the electronic device; determining final search information based on the additional information; and providing a search result to the electronic device based on the final search information.

[0031] According to another aspect, there is provided a non-transitory computer-readable storage medium storing instructions that, when executed by a processor of a server, cause the server to perform a method for providing a content search service, the method comprising: obtaining search information for searching for at least one content from an electronic device; determining additional required information based on the obtained search information; requesting the additional required information from the electronic device; obtaining additional information in response to a request for the additional required information from the electronic device; determining final search information based on the additional information; and providing a search result to the electronic device based on the final search information.

[0032] According to another aspect, there is provided an electronic device comprising: a communication device configured to communicate with a server; a display configured to display a content search result provided from the server; and a processor configured to control the communication device and the display, wherein the processor transmits a content search request including search request data to the server, receives, from the server, a request for additional required information, determined based on the search request data, displays a user response input window on the display in response to the request for the additional required information, transmits additional information based on a user input through the user response input window to the server, receives, from the server, a search result for the content search request determined based on the additional information, and displays the search result on the display.

[0033] According to another aspect, there is provided a method for a content search service of an electronic device, the method comprising: transmitting a content search request including search request data to a server; receiving, from the server, a request for additional required information determined based on the search request data; displaying a user response input window in response to the request for the additional required information; transmitting, to the server, additional information based on a user input through the user response input window; receiving, from the server, a search result for the content search request determined based on the additional information; and displaying the search result.

[0034] According to another aspect, there is provided a non-transitory computer-readable storage medium storing instructions that, when executed by a processor of an electronic device, cause the electronic device to perform a method for a content search service, the method comprising: transmitting a content search request including search request data to a server; receiving, from the server, a request for additional required information determined based on the search request data; displaying a user response input window in response to the request for the additional required information; transmitting, to the server, additional information based on a user input through the user response input window; receiving, from the server, a search result for the content search request determined based on the additional information; and displaying the search result.

[0035] It is possible to provide an optimal search result with high accuracy and reliability to a user requesting a content search using final search information determined based on additional information from the user.

[0036] It is possible to provide an optimal search result with high accuracy and reliability to a user requesting a content search using final search information determined based on additional information from the user, for example, by employing methods for generating (or determining) 'additional required information' and determining 'final search information' based on natural language processing (NLP) or vector search, especially for ambiguous or semantic queries

[0037] It is possible to provide an optimal search result to a user by retraining and updating a content search model based on content finally selected by the user from a search result and final search information used in the search result.

[0038] It is possible to provide an even more accurate and personalized optimal search result to the user by retraining and updating a content search model based on analyzing the correlation (e.g., using positive / negative pairs as in fine-tuning) between the content finally selected by the user from a search result and the final search information (e.g., query vector) used in the search result.

[0039] The present disclosure may be readily understood by the combination of the following detailed description and the accompanying drawings, wherein reference numerals refer to structural elements.

[0040] FIG. 1 is a diagram for describing a content search service providing system according to an embodiment.

[0041] FIG. 2 is a diagram for describing a server according to an embodiment.

[0042] FIG. 3 is a diagram for describing an electronic device communicatively connected to the server according to an embodiment.

[0043] FIG. 4 is a diagram for describing a database of the server according to an embodiment.

[0044] FIG. 5 is a diagram for describing a process of providing a content search service according to an embodiment.

[0045] FIGS. 6a to 6c are views for describing a process of acquiring additional information for determining final search information according to an embodiment.

[0046] FIG. 7 is a view for describing a process of acquiring additional information for determining final search information according to another embodiment.

[0047] FIG. 8 is a diagram for describing a process of generating a search result using a search model according to an embodiment.

[0048] FIGS. 9 and 10 are diagrams for describing a process of retaining and updating a search model according to an embodiment.

[0049] FIG. 11 is a view for describing a process of displaying a search result according to an embodiment.

[0050] FIG. 12 is a flowchart for describing a method for providing a content search service of a server according to an embodiment.

[0051] FIG. 13 is a flowchart for describing a method for displaying a content search service of an electronic device that is communicatively connected to a server, according to an embodiment.

[0052] According to one aspect, a server may include a communication device communicatively connected to an electronic device, a database for storing search information for content search by category, and a processor for providing a search result corresponding to a content search request of the electronic device. The processor may obtain search information for searching for at least one content from the electronic device, determine additional required information based on the obtained search information, request the additional required information from the electronic device, determine final search information based on additional information obtained from the electronic device in response to a request for the additional required information, and provide a search result to the electronic device based on the final search information.

[0053] Hereinafter, various embodiments will be described in detail with reference to the drawings. The embodiments described below may be modified and implemented in various different forms. In order to more clearly describe characteristics of the embodiments, detailed descriptions of matters widely known to those skilled in the art to which the following embodiments belong will be omitted.

[0054] Meanwhile, when an element is referred to as being "connected" to another element in the present specification, it includes not only a circumstance when the element is "directly connected" to the other element, but also a circumstance when the element is "connected" to the other element with another element interposed therebetween. In addition, when an element "includes" another element, unless described to the contrary, this means that the element does not exclude other elements, but may further include other components.

[0055] In addition, terms, such as "first" and "second", including an ordinal number used herein may be used for describing various elements, but the elements should not be limited by the terms. The terms are only used to distinguish one element from another element.

[0056] In the present specification, a server may determine final search information based on additional information from a user, and generate and provide an optimal content search result with high accuracy and reliability to a user requesting a content search using the determined final search information.

[0057] As used herein, "final search information" refers to information representing the search intent that has been progressively specified and refined through one or more interactions between the server and the user (e.g., responses to additional questions) following the user's initial search request. It may take various forms, such as a set of simple keywords, a weighted structured query, or a query vector representing semantic similarity, and is the conclusive information used as input to a search model to derive the final search result. Thus, 'final' implies not only the last in sequence but also the result of a process of refinement through user interaction.

[0058] In the present specification, an "electronic device" may be, but is not limited to, a smartphone, a tablet PC, a PC, a TV, a smart TV, a mobile phone, a personal digital assistant (PDA), a laptop, a non-mobile computing device, or the like. An application for providing a content search service may be distributed and installed on an electronic device. The electronic device may execute the application for providing the content search service, receive an optimal content search result, and display it to a user requesting the content search through the application.

[0059] In the present specification, "content" may refer to information or materials provided through the Internet, computer communication, and the like. "Content" may refer to information or materials, such as text, symbols, voice, sound, images, video, and the like that are digitally produced and then processed or distributed. For example, the content provided through the server may be video content.

[0060] FIG. 1 is a diagram for describing a content search service providing system according to an embodiment.

[0061] As illustrated in FIG. 1, the content search service providing system may include a server 100 that provides a search result corresponding to a content search request of an electronic device 200, and the electronic device 200 that is communicatively connected to the server 100 to receive a final search result corresponding to additional information from the server 100 and displays the final search result for the content search.

[0062] For example, the electronic device 200 may include both a stationary device (standing device) such as a personal computer (PC), a network TV, a hybrid broadcast broadband TV (HBBTV), a smart TV, an Internet protocol TV (IPTV), and the like, and a mobile device (mobile device or handheld device) such as a smartphone, a tablet PC, a notebook, a personal digital assistant (PDA), and the like.

[0063] In addition, the network for connecting the communication between the electronic device 200 and the server 100 includes both wired and wireless networks, and is a general term for a communication network for supporting various communication standards or protocols for pairing or / and data transmission and reception between the electronic device 200 and the server 100.

[0064] These wired / wireless networks include all communication networks to be currently or in the future supported by the standard, and may support one or more communication protocols for them.

[0065] The wired / wireless networks may be formed by a network and a communication standard or protocol for wired connection, such as Universal Serial Bus (USB), Composite Video Banking Sync (CVBS), Component, S-Video (analog), Digital Visual Interface (DVI), High Definition Multimedia Interface (HDMI), RGB, and D-SUB, and a network and a communication standard or protocol for wireless connection, such as Bluetooth, Radio Frequency Identification (RFID), infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Digital Living Network Alliance (DLNA), Wireless LAN (WLAN) (Wi-Fi), Wireless broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed Downlink Packet Access (HSDPA), Long Term Evolution / LTE-Advanced (LTE / LTE-A), and Wi-Fi direct.

[0066] Meanwhile, the server 100 may obtain search information for searching for at least one content from the electronic device 200, generate / determine / select additional required information based on the search information and request the additional required information from the electronic device 200, determine final search information based on additional information from a user corresponding to the additional required information, when acquiring the additional information from the electronic device 200, and provide the search result to the electronic device 200 based on the final search information.

[0067] In one embodiment, computationally intensive tasks such as processing user's search request data (e.g., voice, image), executing the logic for determining 'additional required information', determining the 'final search information', generating search results using the search model, and retraining the model may primarily be performed by the server 100. The electronic device 200 may be responsible for providing the user interface (UI), receiving user input, communicating with the server 100, and displaying the received search results.

[0068] Here, when acquiring the search information, the server 100 may, upon receiving the content search request from the electronic device 200, obtain search request data input to the electronic device 200 corresponding to the content search request, and obtain the search information by extracting it from the search request data.

[0069] In addition, when the additional required information is requested, the server 100 may perform a content search based on the search information, provide an initial search result to the electronic device 200, generate the additional required information for specifying the final search information based on the initial search result, and request the additional required information from the electronic device 200.

[0070] Here, the server 100 may check a content genre corresponding to the initial search result, select a search category for classifying at least one of the content genre, an actor, and a synopsis, and generate the additional required information based on the selected search category.

[0071] In addition, when the search result is provided based on the final search information, the server 100 may input the final search information into a pre-trained search model, obtain a search result including content that the user wants to search for, and provide the search result to the electronic device 200.

[0072] Here, when the search model is trained, the server 100 may obtain content information classified by category, obtain metadata of content corresponding to the content information, train the search model based on the content and metadata, obtain a training result, retrain the search model based on the training result, and update the search model.

[0073] In some cases, when a user input for selecting specific content from the search result is received from the electronic device 200, the server 100 may analyze a correlation between a configuration of the selected content and the final search information, and update the search model by retraining the search model based on the correlation.

[0074] In this way, the apparatus for providing a content search service may provide an optimal search result with high accuracy and reliability to the user requesting the content search using the final search information determined based on the additional information from the user.

[0075] In addition, the apparatus for providing the content search service may provide the optimal search result to the user by retraining and updating the content search model based on content finally selected by the user from the search result and final search information used in the search result.

[0076] FIG. 2 is a diagram for describing the server according to an embodiment.

[0077] As illustrated in FIG. 2, the server 100 may include a communication device 110 communicatively connected to an electronic device, a database 120 for storing search information for content search by category, and a processor 130 for providing a search result corresponding to a content search request from the electronic device.

[0078] Here, the processor 130 of the server 100 may include one or more cores, and may include a processor for data analysis and deep learning, such as a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), and a tensor processing unit (TPU) of a computing device.

[0079] In addition, the communication device 110 of the server 100 may include a wireless Internet module, a short-range communication module, a location information module, and the like.

[0080] The wireless Internet module refers to a module for wireless Internet access, and is configured to transmit and receive wireless signals in a communication network according to wireless Internet technologies.

[0081] Wireless internet technologies include, for example, Wireless LAN (WLAN), Wireless-Fidelity (Wi-Fi), Wireless Fidelity Direct (Wi-Fi Direct), Digital Living Network Alliance (DLNA), Wireless Broadband (WiBro), World Interoperability for Microwave Access (WiMAX), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), and the like, and the wireless internet module transmits and receives data according to at least one wireless internet technology within a scope including but not limited to the Internet technologies listed above.

[0082] From the viewpoint that wireless Internet access by WiBro, HSDPA, HSUPA, GSM, CDMA, WCDMA, LTE, or LTE-A is performed through the mobile communication network, the wireless Internet module that performs wireless Internet access through the mobile communication network may be understood as one type of the mobile communication module.

[0083] The short-range communication module is a module for short-range communication, and may support short-range communication using at least one of Bluetooth, radio frequency identification (RFID), Infrared Data Association (IrDA), ultra wideband (UWB), ZigBee, near field communication (NFC), Wireless-Fidelity (Wi-Fi), Wi-Fi Direct, and wireless Universal Serial Bus (wireless USB) technologies.

[0084] The location information module is a module for acquiring a location (or current location) of the server, and representative examples thereof include a Global Positioning System (GPS) module or a Wireless Fidelity (WiFi) module.

[0085] In addition, the database 120 of the server 100 may include at least one storage medium of a flash memory type memory, a hard disk type memory, a multimedia card micro type memory, a card type memory (e.g., SD memory, XD memory, or the like), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disc.

[0086] In addition, the server 100 may operate in relation to web storage that performs a storage function of the database 120 on the Internet. The description of the database described above is merely exemplary, and is not intended to be limiting.

[0087] For example, the database 120 may include a plurality of sub-databases for storing search information for content search for each higher-category, and each sub-database may include a plurality of subcategories included in the higher-level category.

[0088] Meanwhile, the processor 130 may obtain search information for searching for at least one content from the electronic device, generate additional required information based on the search information and request the additional required information from the electronic device, determine final search information based on additional information from the user corresponding to the additional required information upon acquiring, from the electronic device, the additional information from the user, and provide the search result to the electronic device based on the final search information.

[0089] Here, when acquiring the search information, the processor 130 may, upon receiving the content search request from the electronic device 200, obtain search request data input to the electronic device corresponding to the content search request, and obtain the search information by extracting it from the search request data.

[0090] For example, when the search request data input to the electronic device is text, the processor 130 may extract a search term from the text and obtain the search information corresponding to the search term.

[0091] For another example, when the search request data input to the electronic device is an image, the processor 130 may extract text from the image and obtain the search information from the text.

[0092] Here, when there a plurality of text segments are extracted, the processor 130 may estimate a similarity to objects represented in the image among the plurality of extracted text segments, select a text segment having the highest similarity, and obtain the search information from the selected text segment.

[0093] Here, when a plurality of text segments are extracted, the processor 130 may estimate the similarity between each of the plurality of extracted text segments and objects represented in the image, wherein the estimation involves calculating a quantified relevance score, may select the text segment with the highest similarity, i.e., the highest relevance score, and may obtain the search information from the selected text segment. The calculation of the relevance score may be implemented using various Information Retrieval techniques, including, for example, TF-IDF (Term Frequency-Inverse Document Frequency) similarity between text features and image object features, BM25 (Best Matching 25) scores, or cosine similarity between deep learning-based cross-modal embedding vectors.

[0094] The relevance score may be calculated using, for example, (1) BM25 scores by treating the extracted text segment as a query and text labels of the objects within the image as documents, or (2) cosine similarity between deep learning-based cross-modal embedding vectors.

[0095] The calculation of the relevance score may be implemented using various Information Retrieval techniques, including, for example, cross-modal analysis methods. Specifically, (1) each of the extracted text segments may be converted into a text vector using a natural language processing model, and (2) visual features of the objects within the image may be converted into object vectors using an image recognition model (e.g., an object detection model). (3) After mapping these text vectors and object vectors into a common embedding space, (4) the relevance score can be calculated by computing, for instance, the cosine similarity or Euclidean distance between a specific text vector and each object vector in this space. The text segment yielding the highest score is then selected to obtain the search information.

[0096] For still another example, when the search request data input to the electronic device is sound or voice, the processor 130 may convert the sound or voice into text, extract a search term from the text, and obtain the search information corresponding to the search term.

[0097] For example, if a user requests a search by providing background music (BGM), the processor may extract 'search information' such as the music's genre ('classical', 'pop') or mood ('tense', 'sad') using music recognition technology in addition to speech recognition. Subsequently, the processor may refine the search intent by requesting 'additional required information' such as "Are you looking for a specific scene (e.g., battle scene, farewell scene) in a movie where this music was used?"

[0098] In addition, when the additional required information is requested, the processor 130 may perform a content search based on the search information, provide an initial search result to the electronic device, generate the additional required information for specifying the final search information based on the initial search result, and request the additional required information from the electronic device.

[0099] For example, after providing the initial search result, the processor 130 may check whether the amount of content information corresponding to the initial search result is greater than or equal to a preset reference amount of information, and generate the additional required information when the amount of content information corresponding to the initial search result is greater than or equal to the reference amount of information. The reason is that when the amount of content information corresponding to the initial search result is excessive, inaccurate content information items will also be included, and accordingly, the additional required information for search is generated to extract only content information with high reliability and accuracy.

[0100] Here, the processor 130 may not generate the additional required information when the amount of content information corresponding to the initial search result is less than the reference amount of information.

[0101] The aforementioned 'preset reference amount of information' may be a fixed value, but it can also be dynamically adjusted based on the user's search context, characteristics of the target content, or the server's load status.

[0102] If the processor 130, despite analyzing the 'initial search result', may determine that it is difficult to generate meaningful 'additional required information' (e.g., discriminating categories) (e.g., if the number of search results is below the reference amount or metadata variance is low), the processor may skip the step of requesting 'additional required information'. Alternatively, it may provide a message to the user such as "No further refinement information available. Would you like to browse the current results?" and consider the current search information as the 'final search information' to provide the results. This reduces unnecessary interaction and completes the search process.

[0103] In addition, after providing the initial search result, the processor 130 may check at least one of a content genre, an actor, and a synopsis corresponding to the initial search result, select a search category for classifying the content genre, and generate the additional required information based on the selected search category.

[0104] Here, when selecting the search category, the processor 130 may extract a user's search history for at least one of the content genre, the actor, and the synopsis corresponding to the initial search result, and select the search category for classifying at least one of the content genre, the actor, and the synopsis based on the user's search history.

[0105] In some cases, when selecting the search category, the processor 130 may check whether a subcategory corresponding to the selected search category exists when the selected search category is a higher-level category, and select the search category including both the higher-level category and the corresponding subcategory when the subcategory exists.

[0106] In addition, when the additional required information is a search category, the processor 130 may generate the search category in the form of a main search icon, and provide the main search icon to the electronic device so that the main search icon corresponding to the search category is displayed over the initial search result, thereby requesting the additional required information.

[0107] Here, when a user input for selecting the main search icon corresponding to the search category is received from the electronic device, the processor 130 may check whether a subcategory corresponding to the selected search icon exists, generate a sub-search icon corresponding to the subcategory when the subcategory exists, and provide the sub-search icon to the electronic device so that the sub-search icon is displayed to be matched with the main search icon, thereby requesting the additional required information.

[0108] In some cases, when a user input for selecting the sub-search icon is received from the electronic device, the processor 130 may generate an input field for a search weight of the user for each item included in a subcategory of the selected sub-search icon and provide the items included in the subcategory and the input field for the search weight of the user for each item to the electronic device, thereby requesting the additional required information including the search weight of the user input through the input field for the search weight.

[0109] For example, the processor 130 may recognize that, when the search weight of the user input through the input field for the search weight is at a higher level, the accuracy of the user's knowledge with respect to an item of a subcategory corresponding to the higher level is at the higher level, recognize that, when the search weight of the user is at a middle level, the accuracy of the user's knowledge with respect to an item of a subcategory corresponding to the middle level is at the middle level, and recognize that, when the search weight of the user is at a lower level, the accuracy of the user's knowledge with respect to an item of a subcategory corresponding to the lower level is at the lower level.

[0110] In addition, when the additional required information is the search category, the processor 130 may generate the search category in the form of a speech bubble and provide the speech bubble to the electronic device so that the speech bubble corresponding to the search category is displayed over the initial search result, thereby requesting the additional required information.

[0111] Here, when a user's response to the additional required information is received from the electronic device, the processor 130 may perform a content search based on the user's response and generate a search result, and when the final search information is not determined based on the search result, the processor 130 may further generate additional required information for determining the final search information in the form of the speech bubble and provide the additional required information to the electronic device.

[0112] For example, the processor 130 may not determine the final search information when the amount of information in the search result is greater than or equal to a preset reference amount of information, and may determine the final search information when the amount of information in the search result is less than the preset reference amount of information.

[0113] In addition, when determining the final search information, the processor 130 may perform a content search based on the additional information from the user and generate a search result, generate the additional required information based on the search result when the amount of information in the search result is greater than or equal to a preset reference amount of information and further request the additional required information from the electronic device, and determine the final search information based on additional information from the user corresponding to the additional required information when further acquiring the additional information from the electronic device.

[0114] In yet another embodiment, the processor 130 may determine (or refine) the 'final search information' by analyzing user behavior on the electronic device 200 as implicit feedback, even without explicit additional information input from the user.

[0115] For example, after a user searches for 'sci-fi movie' and is presented with the 'initial search result' 420 list, the processor 130 may analyze behaviors such as (1) hovering the mouse cursor over the thumbnail of movie A for a long duration (dwell time), or (2) quickly scrolling past movies B and C but stopping the scroll at movie D (scroll depth / stop).

[0116] The processor 130 may analyze both positive implicit signals (e.g., dwell / stop on movies A, D) and negative implicit signals (e.g., quickly scrolling past movies B, C). Common metadata (e.g., 'space setting', 'director Z') may be extracted from the positively signaled movies A and D, and these are used as features to adjust the ranking of the search results. Since the user's explicitly entered search term ("sci-fi movie") is considered the strongest indication of intent, a weight controlling the influence on the result ranking is assigned to the features ('space setting', 'director Z') extracted from implicit feedback (e.g., corresponding to a modest boost, such as a 10-30% increase in relevance score, for other sci-fi movies possessing these features) when adjusting the 'final search information' or re-ranking the search results. Concurrently, features associated with the negatively signaled movies B and C may be assigned lower scores or penalties to lower their ranking. This allows for the gradual improvement of search results by reflecting inferred preferences based on implicit feedback without significantly distorting the user's explicit search intent, even without direct user responses.

[0117] In addition, when providing the search result based on the final search information, the processor 130 may provide at least one of main information about finally retrieved content and a thumbnail, a replay video, or a preview video of the finally retrieved content.

[0118] Furthermore, if the user emphasized a specific type of information (e.g., a particular scene, a specific character) during the process of determining the 'final search information', the processor may arrange or prioritize the display of corresponding information types (e.g., preview videos, character close-up thumbnails) more prominently when displaying the final search result (S150, S260).

[0119] In addition, when the search result is provided based on the final search information, the processor 130 may input the final search information into a pre-trained search model, obtain a search result including content that the user wants to search for, and provide the search result to the electronic device.

[0120] Here, when the search model is trained, the processor 130 may obtain content information classified by category, obtain metadata of content corresponding to the content information, store the metadata in the database 120, train the search model based on the content and metadata, obtain a training result, retrain the search model based on the training result, and update the search model.

[0121] The metadata (FIG. 4) of the content used for training the search model may be pre-processed or transformed. For example, text-based metadata (e.g., synopsis, actor names) may be converted into vector representations through NLP techniques (e.g., tokenization, embedding), and image-based metadata (e.g., poster features) may be extracted as feature vectors using methods like Convolutional Neural Networks (CNNs). Such prepared metadata allows the search model to more effectively learn similarities between content and more accurately understand user search intent.

[0122] In some cases, when a user input for selecting specific content from the search result is received from the electronic device, the processor 130 may analyze a correlation between a configuration of the selected content and the final search information, retrain the search model based on the correlation, and update the search model.

[0123] Here, when analyzing the correlation between the configuration of the selected content and the final search information, the processor 130 may obtain at least one search category corresponding to the selected content, obtain at least one search term included in the final search information, match the search category of the content with the search term of the final search information, and analyze the correlation between the configuration of the selected content and the final search information based on the matching result.

[0124] In addition, the search model including the aforementioned neural network model may be a deep neural network. Throughout the present specification, the terms "neural network", "network function," and "neural network" may be used interchangeably. The deep neural network (DNN) may refer to a neural network that includes a plurality of hidden layers in addition to an input layer and an output layer. The deep neural network may be used to identify latent structures of data. That is, latent structure of photos, text, videos, voices, and music (e.g., what objects are in the photo, what the content and emotion of the text are, what the content and emotion of the voice are, and so on) may be identified. The deep neural network may include a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a Q-network, a U-network, a Siamese network, or the like.

[0125] The convolutional neural network is a type of deep neural network, and includes neural networks that contain convolutional layers. The convolutional neural network is a type of multilayer perceptron designed to use minimal preprocessing. The CNN may include one or more convolutional layers and artificial neural network layers combined with them. The CNN may additionally utilize weights and pooling layers. Due to this structure, the CNN may fully utilize input data with a two-dimensional structure. The convolutional neural network may be used to recognize an object in an image. The convolutional neural network may process image data by representing the image data as a matrix with dimensions. For example, in the case of image data encoded in RGB (red-green-blue), each of the R, G, and B colors may be represented as a two-dimensional (e.g., when the image is a two-dimensional image) matrix. That is, a color value of each pixel of the image data may be an element of the matrix, and the size of the matrix may be the same as the size of the image. Therefore, the image data may be represented as three two-dimensional matrices (three-dimensional data array).

[0126] In the convolutional neural network, a convolutional process (an input and an output of a convolutional layer) may be performed by multiplying elements of a convolutional filter and those of the image at each position while moving the convolutional filter. The convolutional filter may be formed as an n*n matrix. The convolutional filter may be generally formed as a fixed-type filter that is smaller than the total number of pixels in the image. That is, when an m*m image is input to a convolutional layer (for example, the convolutional layer with a convolutional filter of size n*n), a matrix representing n*n pixels containing each pixel of the image may be multiplied with the convolutional filter on an element-by-element basis (that is, multiplication of respective elements of the matrices). By the multiplication with the convolutional filter, components matching the convolutional filter may be extracted from the image. For example, a 3*3 convolutional filter for extracting vertical line components from the image may be formed as [[0,1,0], [0,1,0], [0,1,0]]. When the 3*3 convolutional filter is applied to the input image to extract the vertical line components from the image, the vertical line components matching the convolutional filter may be extracted from the image and output. The convolutional layer may apply the convolutional filter to each matrix for each channel representing the image (that is, R, G, B colors in the case of an R, G, B coded image). The convolutional layer may apply the convolutional filter to the input image and extract features from the input image matching the convolutional filter. Filter values of a convolutional filter (that is, values of respective elements of the matrix) may be updated by backpropagation during the training process of the convolutional neural network.

[0127] A subsampling layer may be connected to the output of the convolutional layer to simplify the output of the convolutional layer, thereby reducing memory usage and computational load. For example, when the output of the convolutional layer is input to a pooling layer having a 2*2 max pooling filter, the image may be compressed by outputting a maximum value contained in each patch for each 2*2 patch at each pixel of the image. The aforementioned pooling may be a method of outputting the minimum value from the patch or an average value of the patch, and any pooling method may be included in the present invention.

[0128] The convolutional neural network may include one or more convolutional layers and subsampling layers. The convolutional neural network may extract features from the image by repeatedly performing a convolutional process and a subsampling process (e.g., max pooling as described above). Through the repeated convolutional and subsampling processes, the neural network may extract global features of the image.

[0129] The output of the convolutional layer or the subsampling layer may be input to a fully connected layer. The fully connected layer is a layer in which all neurons in one layer are connected to all neurons in a neighboring layer. The fully connected layer may refer to a structure in the neural network where all nodes in each layer are connected to all nodes in another layer.

[0130] FIG. 3 is a diagram for describing the electronic device communicatively connected to the server according to an embodiment.

[0131] As illustrated in FIG. 3, the electronic device 200 may include a communication device 210 communicatively connected to the server, a display 220 that displays a content search result provided from the server, and a controller 230 that controls the communication device 210 and the display 220.

[0132] Here, the communication device 210 may include a wireless Internet module, a short-range communication module, a location information module, and the like.

[0133] In addition, the display 220 may output any type of information generated or determined by the controller 230 and any type of information received by the communication device 210.

[0134] For example, the display 220 may include at least one of a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display, and a 3D display. Some of the display modules may be configured as a transparent type or a light-transmitting type so that the outside may be viewed through the display modules. This may be referred to as a transparent display module, and a representative example of the transparent display module is a transparent OLED (TOLED) or the like.

[0135] In addition, the electronic device may further include an input unit for receiving user input, and the input unit may include keys and / or buttons on a user interface for receiving user input, or physical keys and / or buttons. A computer program according to embodiments of the present disclosure may be executed based on user input through the input unit.

[0136] In addition, the input unit may receive a signal by detecting a user's button operation or touch input, or may receive a user's voice or motion through a camera or microphone and convert it into an input signal. For this purpose, speech recognition technology or motion recognition technology may be used.

[0137] In addition, the input unit may be implemented as an external input device connected to the electronic device. For example, the input device may be at least one of a touchpad, a touch pen, a keyboard, or a mouse for receiving user input, but these are merely exemplary, and the present invention is not limited thereto.

[0138] For example, the input unit may recognize user touch input. In some cases, the input unit may have the same configuration as an output unit. The input unit may be configured as a touch screen implemented to receive a user's selection input. The touch screen may use any one of a capacitive touch method, an infrared light detection method, a surface acoustic wave (SAW) method, a piezoelectric method, or a resistive film method. The detailed description of the touch screen described above is merely an example according to an embodiment of the present disclosure, and various touch screen panels may be employed in the platform server 100. The input unit, which is configured as the touch screen, may include a touch sensor. The touch sensor may be configured to convert changes in pressure applied to a specific portion of the input unit, electrostatic capacitance occurring at a specific portion of the input unit, or the like into an electrical input signal. The touch sensor may be configured to detect not only a touch position and a touch area, but also a pressure at the time of touch. When there is a touch input to the touch sensor, a corresponding signal(s) is(are) sent to a touch controller. The touch controller may process the signal(s) and then transmit corresponding data to the processor. This allows the processor to recognize which region of the input unit has been touched.

[0139] Meanwhile, the controller 230 may include one or more cores, and may include a processor for data analysis and deep learning, such as a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), and a tensor processing unit (TPU) of a computing device.

[0140] Here, the controller 230 may control the communication device 210 to transmit a content search request including search request data to the server when search request data for requesting a content search is input, control the display 220 to display a user response input window corresponding to a request for additional required information when the request for the additional required information is received from the server, control the communication device 210 to transmit additional information input from the user through the user response input window to the server, and control the display 220 to display a final search result for the content search when a final search result corresponding to the additional information is received from the server.

[0141] The controller 230 may control the communication device 210 to transmit, to the server, the content search request together with the search request data when the user input including the search request data and the content search request is received.

[0142] For example, the search request data may be at least one of text, image, sound, and voice.

[0143] In addition, when receiving the initial search result corresponding to the search request data from the server, the controller 230 may control the display 220 to display the initial search result.

[0144] Subsequently, when a request for additional required information is received from the server, the controller 230 may control the display 220 to display the user response input window corresponding to the request for additional required information.

[0145] Here, when the additional required information is a search category, the user response input window may include a main search icon corresponding to the search category over the initial search result.

[0146] In this case, when a user input for selecting the main search icon corresponding to the search category is received, the controller 230 may control the display 220 to check whether a subcategory corresponding to the selected search icon exists and display a sub-search icon corresponding to the subcategory to be matched with the main search icon when the subcategory exists.

[0147] In one embodiment, the user response input window may provide various UIs depending on the form of the 'additional required information'. For example, if the 'additional required information' is in the form of a multiple-choice question (e.g., "Is the setting a futuristic city?"), [Yes] / [No] buttons may be displayed. If it involves selecting items within a specific category (e.g., selecting characters), it may be presented as icons 430, 440 as in FIGS. 6b, 6c, or a scrollable list. Furthermore, a speech bubble interface allowing free text input 520 may be provided as in FIG. 7. This helps the user respond in the most convenient manner.

[0148] In addition, when a user input for selecting the sub-search icon is received, the controller 230 may control the display 220 to display an input field for a search weight of the user for each item included in the subcategory of the selected sub-search icon.

[0149] Here, the controller 230 may control the communication device 210 to transmit the search weight of the user to the server when the user input is received in the input field for the search weight.

[0150] For example, the controller 230 may transmit, to the server, the search weight of the user that is input as an higher level when the user inputs the higher level into the input field for the search weight, transmit, to the server, the search weight of the user that is input as a middle level when the user inputs the middle level into the input field for the search weight, and transmit, to the server, the search weight of the user that is input as a lower level when the user inputs the lower level into the input field for the search weight.

[0151] In some cases, when the additional required information is a search category, the user response input window may include a speech bubble corresponding to the search category over the initial search result.

[0152] In this case, the controller 230 may control the display 220 to display a speech bubble containing text requesting additional required information corresponding to the search category.

[0153] Subsequently, the controller 230 may control the display 220 to display a speech bubble containing text for a user's response corresponding to the speech bubble containing the text requesting additional required information when the user's response is input, and control the communication device 210 to transmit, to the server, text data for the user's response.

[0154] Next, when a search result is received from the server, the controller 230 may control the display 220 to display the search result including at least one of the main information about finally retrieved content and a thumbnail, a replay video, and a preview video of the finally retrieved content.

[0155] FIG. 4 is a diagram for describing a database of the server according to an embodiment.

[0156] As illustrated in FIG. 4, the database 120 of the server may store search information for content search by category.

[0157] For example, the database 120 may include a plurality of sub-databases that store search information for content search for each higher-level category.

[0158] Here, each sub-database may include a plurality of subcategories included in the higher-level category.

[0159] For example, a first sub-database 122 is a database including search categories related to posters, and may include subcategories including poster background color, background, text, and the like.

[0160] A second sub-database 124 is a database that includes search categories related to back ground music (BGM), and may include subcategories including a music genre, an instrumental sound, a vocal gender, a pace, and the like.

[0161] A third sub-database 126 is a database that includes search categories related to people, and may include subcategories including race, skin color, eye color, hair color, language spoken, accessories worn, and other characteristics.

[0162] A fourth sub-database 128 is a database that includes search categories related to synopses and may include subcategories including character names, actor names, locations, eras, plots, and the like.

[0163] A fifth sub-database 129 is a database that includes search categories related to scenes and may include subcategories including mountains, water, deserts, seas, lakes, cities, and the like.

[0164] The aforementioned sub-databases are merely exemplary, and the server may request additional information required for the final search information from the electronic device based on various search categories stored in the database.

[0165] The categories and subcategories shown in FIG. 4 are merely examples of search information that can be utilized in the present invention. Various other forms of information useful for content search and recommendation, such as user review texts, content viewing log data, social media mention information, etc., may be stored in the database 120 and utilized in generating 'additional required information' or determining 'final search information'

[0166] FIG. 5 is a diagram for describing a process of providing a content search service according to an embodiment.

[0167] As illustrated in FIG. 5, when a user's search information 300 including text, images, sound, voice, and the like and a content search request are input, the electronic device 200 may transmit, to the server 100, the content search request together with the search information 300.

[0168] In addition, the server 100 may perform a content search, when the search information 300 received from the electronic device 200 is text, by extracting a search term from the text; when the search information 300 is an image, by extracting text from the image; and when the search information 300 is sound or voice, by converting the sound or voice to text.

[0169] For example, when the search request data input to the electronic device 200 is text, the server 100 may extract a search term from the text and obtain the search information corresponding to the search term.

[0170] For another example, when the search request data input to the electronic device 200 is an image, the server 100 may extract text from the image and obtain the search information from the text.

[0171] Here, when there a plurality of text segments are extracted, the server 100 may estimate a similarity to objects included in the image among the plurality of extracted text segments, select a text segment having the highest similarity, and obtain the search information from the selected text segment.

[0172] For still another example, when the search request data input to the electronic device 200 is sound or voice, the server 100 may convert the sound or voice into text, extract a search term from the text, and obtain the search information corresponding to the search term.

[0173] For example, when the search request data input to the electronic device is an image, the processor 130 may (1) apply Optical Character Recognition (OCR) to extract text information (e.g., 'THE REVENGE' from a poster) and (2) apply an Object Detection model to recognize key objects (e.g., 'man with gun', 'medieval armor').

[0174] The processor 130 may obtain the extracted text ('THE REVENGE') as the 'search information'. Alternatively, the processor may present the detected object information ('gun', 'medieval armor') to the user as 'additional required information' (e.g., "Are 'gun' or 'medieval armor' relevant in this image?") and determine the 'final search information' based on the user's selection, thereby combining multi-modal inputs to improve search accuracy.

[0175] Subsequently, the server 100 may perform a content search based on the search information, provide an initial search result to the electronic device 200, generate additional required information for determining final search information based on the initial search result, and request the additional required information from the electronic device 200.

[0176] Next, when a request for the additional required information is received from the server 100, the electronic device 200 may display a user response input window corresponding to the request for the additional required information.

[0177] Here, when the additional required information is a search category, the user response input window may include a main search icon corresponding to the search category over the initial search result.

[0178] In some cases, when the additional required information is the search category, the user response input window may include a speech bubble corresponding to the search category over the initial search result.

[0179] In addition, when a user's response corresponding to the additional required information is input, the electronic device 200 may transmit, to the server, the additional information for the user's response.

[0180] Subsequently, the server 100 may perform a content search based on the additional information from the user and generate a search result, further generate the additional required information based on the search result when the amount of information in the search result is greater than or equal to a preset reference amount of information and further request the additional required information from the electronic device 200, and determine the final search information based on additional information from a user corresponding to the additional required information when further acquiring the additional information from the electronic device 200.

[0181] Next, when the search result is received from the server 100, the electronic device 200 may display the final search information including at least one of the main information about finally retrieved content and a thumbnail, a replay video, and a preview video of the finally retrieved content.

[0182] FIGS. 6a to 6c are views for describing a process of acquiring additional information for determining final search information according to an embodiment.

[0183] As illustrated in FIG. 6a, when a content search request is received from the electronic device, the server may obtain search request data 410 input into an input field of a display screen 400 of the electronic device corresponding to the content search request, extract search information from the search request data 410, perform a content search based on the search information, and provide an initial search result 420 to the electronic device.

[0184] For example, when the search information includes a term "REVENGE", the server may search for content classified into search categories related to "REVENGE" and provide content information related to "REVENGE".

[0185] Subsequently, the server may generate additional required information to determine final search information based on the initial search result 420 and request the additional required information from the electronic device.

[0186] For example, after providing the initial search result 420, the server may check whether the amount of content information corresponding to the initial search result 420 is greater than or equal to a preset reference amount of information, and generate the additional required information when the amount of content information corresponding to the initial search result 420 is greater than or equal to the reference amount of information.

[0187] The reason is that when the amount of content information corresponding to the initial search result is excessive, inaccurate content information items will also be included, and accordingly, the additional required information for search is generated to extract only content information with high reliability and accuracy.

[0188] Here, the server may not generate the additional required information when the amount of content information corresponding to the initial search result 420 is less than the reference amount of information.

[0189] After providing the initial search result 420, the server may check a content genre corresponding to the initial search result 420, select a search category for classifying the content genre, and generate the additional required information based on the selected search category.

[0190] Here, when selecting the search category, the server may extract a user's search history for the content genre corresponding to the initial search result, and select the search category for classifying the content genre based on the user's search history.

[0191] In some cases, when selecting the search category, the server may check whether a subcategory corresponding to the selected search category exists when the selected search category is a higher-level category, and select the search category including both the higher-level category and the corresponding subcategory when the subcategory exists.

[0192] In another embodiment, when generating the 'additional required information', the processor 130 may refer to the user's profile information or past content viewing history. For instance, if a user searches for 'romance movie' and their profile includes a 'preference for sci-fi' tag, the processor, in addition to analyzing the 'initial search result', may prioritize generating a personalized query like "Are you looking for romance with sci-fi elements?" as the 'additional required information'. This helps approach the user's potential intent more quickly, enhancing search efficiency.

[0193] In another embodiment, when generating the 'additional required information', the processor 130 may ask for information the user wishes to exclude. For example, if a user searches for 'fantasy movie' and the initial search result contains a mix of 'High Fantasy', 'Urban Fantasy', and 'Children's Fantasy', the processor may generate and request a negative query, such as "Any genres to exclude? [High Fantasy] [Urban Fantasy] [Children's Fantasy]".

[0194] If the user selects 'Children's Fantasy', the processor 130 may incorporate this into the determination of the 'final search information', potentially forming a query like {"genre": "fantasy", "NOT": {"subgenre": "children's"}}. This is used to effectively remove results the user explicitly does not want.

[0195] In one embodiment, even after determining the 'final search information' and generating the search results, the processor 130 may request 'additional required information' to clarify the sorting criteria for the results. For example, if there are multiple 'action movies' starring 'Actor AAA', the processor may present a question like "How should we sort the results? (A) Most Recent, (B) Highest Rating, (C) By Director".

[0196] If the user selects (B) 'Highest Rating', the processor 130 may re-sort and display the final search results 810 based on ratings. This helps the user find the desired information more easily within the search results.

[0197] In another embodiment, the processor 130 may suggest relevant search scopes as 'additional required information' based on inferred user intent. For example, if a user searches for a specific drama title ('AAA') and then shows a pattern of exploring related actor information ('BBB'), the next time they search for a similar drama, the processor may generate an icon or speech bubble requesting to expand the scope, such as "Search for other works by the related actors too?" or "Show recent news related to this drama?". This enriches the content discovery experience.

[0198] As illustrated in FIG. 6b, when the additional required information is a search category, the server may generate the search category in the form of a main search icon, and provide a main search icon 430 to the electronic device so that the main search icon 430 corresponding to the search category is displayed over the initial search result 420, thereby requesting the additional required information.

[0199] Here, when a user input for selecting the main search icon 430 corresponding to the search category is received from the electronic device, the server may check whether a subcategory corresponding to the selected main search icon 430 exists, generate a sub-search icon 440 corresponding to the subcategory when the subcategory exists, and provide the sub-search icon 440 to the electronic device so that the sub-search icon 440 is displayed to be matched with the main search icon 430, thereby requesting the additional required information.

[0200] Next, as illustrated in FIG. 6c, when a user input for selecting the sub-search icon 440 is received from the electronic device, the server may generate an input field 460 for a search weight of the user for each item 450 included in a subcategory of the selected sub-search icon 440 and provide the items 450 included in the subcategory and the input field 460 for the search weight of the user for each item 450 to the electronic device, thereby requesting the additional required information including the search weight of the user input through the input field 460 for the search weight.

[0201] For example, the server may recognize that, when the search weight of the user input through the input field 460 for the search weight is at a higher level, the accuracy of the user's knowledge with respect to an item 450 of a subcategory corresponding to the higher level is at the higher level, recognize that, when the search weight of the user is at a middle level, the accuracy of the user's knowledge with respect to an item 450 of a subcategory corresponding to the middle level is at the middle level, and recognize that, when the search weight of the user is at a lower level, the accuracy of the user's knowledge with respect to an item 450 of a subcategory corresponding to the lower level is at the lower level.

[0202] For example, referring to FIG. 6c, when the search weights are set to AAA at the higher level and BBB at the middle level in the subcategory of characters, the server may provide a search result by giving priority to the content featuring AAA over the content featuring BBB. Since the user's feedback information is reflected in the search weights, the server may provide the search result that correspond to the user's intention.

[0203] FIG. 7 is a view for describing a process of acquiring additional information for determining final search information according to another embodiment.

[0204] As illustrated in FIG. 7, when a content search request is received from the electronic device, the server may obtain search request data 410 input into an input field of the display screen 400 of the electronic device corresponding to the content search request, extract search information from the search request data 410, perform a content search based on the search information, and provide an initial search result to the electronic device.

[0205] Subsequently, when the additional required information is the search category, the server may generate the search category in the form of a speech bubble and provide a speech bubble 510 to the electronic device so that the speech bubble 510 corresponding to the search category is displayed over the initial search result, thereby requesting the additional required information.

[0206] Here, when a user's response 520 to the additional required information is received from the electronic device, the server may perform a content search based on the user's response 520 and generate a search result, and when the final search information is not determined based on the search result, the server may further generate additional required information for determining the final search information in the form of the speech bubble and provide the additional required information to the electronic device.

[0207] For example, the server may not determine the final search information when the amount of information in the search result is greater than or equal to a preset reference amount of information, remains undecided, and may determine the final search information when the amount of information in the search result is less than the preset reference amount of information.

[0208] In addition, when determining the final search information, the server may perform a content search based on the additional information from the user and generate a search result 530, generate the additional required information based on the search result 530 when the amount of information in the search result 530 is greater than or equal to a preset reference amount of information and further request the additional required information from the electronic device, and determine the final search information based on additional information from a user corresponding to the additional required information when further acquiring the additional information from the electronic device.

[0209] In the conversational interface 510 of FIG. 7, when the processor 130 receives the user's response 520 (e.g., "It's a movie with actor AAA and actor CCC that was screened last year."), it may determine the 'final search information' using Natural Language Processing (NLP) techniques.

[0210] Specifically, the processor 130 may apply a Named Entity Recognition (NER) model to the response 520 to extract 'Actor AAA' and 'Actor CCC' as PERSON entities and 'last year' as a DATE entity. The processor 130 then may map these extracted entities to metadata fields, such as {"actor": ["AAA", "CCC"], "release_year": "2024"}, to determine the 'final search information'.

[0211] In one embodiment, the process of determining the final search information may be repeatedly performed. For example, after a user searches for 'action movie' (initial search information), the server may ask for 'additional required information' such as 'Character features?' 510. If the user responds 'blonde female protagonist' 520 (first additional information), the server generates a 'first search result'.

[0212] If the amount of information in the 'first search result' still exceeds the reference amount, the server may again generate and request 'additional required information' based on the 'first search result' (e.g., "Is the setting a futuristic city or contemporary?"). If the user responds 'futuristic city' (second additional information), the server determines the 'final search information' by combining the 'initial search information', the 'first additional information', and the 'second additional information', such as {"genre": "action", "protagonist_feature": "blonde female", "setting": "futuristic city"}. This multi-turn conversation allows for progressive narrowing of the search scope.

[0213] As described above, it has been described that the server requests the additional required information in the form of the speech bubble 510, but the present disclosure is not limited thereto and the server may interactively request the additional required information in various forms.

[0214] FIG. 8 is a diagram for describing a process of generating a search result using a search model according to an embodiment.

[0215] As illustrated in FIG. 8, the server may pre-train a neural network model for content search and store it in a database.

[0216] As used herein, the 'search model' may encompass any type of system or algorithm that derives search results based on the final search information, including, but not limited to, deep learning-based neural network models, rule-based inference systems, statistical information retrieval models (e.g., vector space models), or knowledge graph-based systems.

[0217] Here, when the server determines a final search information, the server may input the final search information into a pre-trained search model 700 and obtain a search result including the content the user wants to search for.

[0218] The server may obtain metadata of the content selected by the user, train the search model based on the metadata of the content, obtain a training result, retrain the search model based on the training result, and update the search model.

[0219] In addition, the server may retrain the search model and update the search model based on the correlation between the specific content selected by the user from the provided search result and the final search information.

[0220] FIGS. 9 and 10 are diagrams for describing a process of retaining and updating a search model according to an embodiment.

[0221] As illustrated in FIG. 9, first, the server may obtain content 612 classified by category 614 (620).

[0222] Subsequently, the server may obtain metadata of content corresponding to the content 612 and store the metadata in a database (630).

[0223] Next, the server may train the search model based on the content and metadata and obtain a training result (640).

[0224] In addition, the server may retrain the search model based on the training result and update the search model (650).

[0225] Meanwhile, as illustrated in FIG. 10, the server may receive a user input for selecting specific content 612 in the search results from the electronic device.

[0226] Subsequently, the server may analyze the configuration of a search category 614 of the specific content 612.

[0227] Next, the server may analyze the configuration of a search term 665 of final search information 660.

[0228] In addition, the server may analyze the correlation between the configuration of the category 614 of specific content 612 and the search term 665 of the final search information 660, retrain the search model based on the correlation, and update the search model 700.

[0229] For example, the server may obtain at least one search category 614 corresponding to the specific content 612 finally selected by the user, obtain at least one search term 665 included in the final search information 660, match the search category 614 of the specific content 612 with the search term 665 of the final search information 660, and analyze the correlation between the configuration of the selected specific content 612 and the final search information 660 based on the matching result.

[0230] In one embodiment, when a user searches for "Find the scene where XXX duels in drama OOO" 660, the processor 130 may convert this text into a query vector.

[0231] This query vector, as the 'initial search information', may retrieve too many scenes (e.g., 'war scenes', 'argument scenes', 'sword practice scenes').

[0232] The processor 130 then may generate 'additional required information' to resolve this semantic ambiguity, such as "What kind of 'duel' are you looking for? (A) Sword fight, (B) Gun fight, (C) Verbal argument."

[0233] If the user selects (A) 'Sword fight', the processor 130 may determine the 'final search information' by creating a 'final query vector', for example by adding or averaging a modifier vector corresponding to 'sword fight' with the 'initial query vector'. This 'final query vector' is input into the search model 700 to provide the scenes 810 with the highest similarity, which are the user's intended scenes, as the final search result.

[0234] Subsequently, the 'search model update' may be performed as follows. When the user finally selects specific content 612 from the results 810 based on the 'final search information' (the second query vector), the processor may analyze this interaction (query -> additional info -> final selection) as a 'correlation'.

[0235] Specifically, the processor 130 may construct a 'positive pair' (using the second query vector as an anchor and the vector of the selected content 612) and 'negative pairs' (using the vectors of the unselected results). The search model 700 is then fine-tuned using these pairs to update the model, adjusting the vector space so that the query vector and the positive pair vector are closer (higher similarity).

[0236] The methods described above for generating 'additional required information' and determining 'final search information' (e.g., rule-based, statistics-based, NLP-based, vector-based) may be used independently or in combination depending on the system implementation. For example, it is possible to first extract key entities from a user's natural language response using an NLP-based method, and then determine the final search information by assigning statistical weights to these entities.

[0237] FIG. 11 is a view for describing a process of displaying a search result according to an embodiment.

[0238] As illustrated in FIG. 11, when the search result is received from the server, the electronic device may display, on the display screen, the search result 800 including at least one of the main information about finally retrieved content and a thumbnail 810, a replay video, and a preview video of the finally retrieved content.

[0239] In some cases, the server may display the search result 800, based on the final search information, further including the user search request information 660 on the display screen.

[0240] FIG. 12 is a flowchart for describing a method for providing a content search service of a server according to an embodiment.

[0241] As illustrated in FIG. 12, the server may obtain search information for searching for at least one content from the electronic device (S110).

[0242] Here, when a content search request is received from the electronic device, the server may obtain search request data input to the electronic device corresponding to the content search request, and obtain the search information by extracting it from the search request data.

[0243] Then, the server may generate additional required information based on the search information and request the additional required information to the electronic device (S120).

[0244] Here, the server may perform a content search based on the search information, provide an initial search result to the electronic device, generate additional required information for determining final search information based on the initial search result, and request the additional required information from the electronic device.

[0245] In this case, after providing the initial search result, the server may check at least one of a content genre, an actor, and a synopsis corresponding to the initial search result, select a search category for classifying at least one of a content genre, an actor, and a synopsis, and generate the additional required information based on the selected search category.

[0246] Next, the server may obtain additional information from the user corresponding to the additional required information from the electronic device (S130).

[0247] In addition, the server may determine the final search information based on the additional information (S140).

[0248] Subsequently, the server may provide a search result to the electronic device based on the final search information (S150).

[0249] Here, the server may input the final search information into the pre-trained search model, obtain a search result including the content the user wants to search for, and provide the search result to the electronic device.

[0250] In this way, the server may provide an optimal search result with high accuracy and reliability to the user requesting the content search using the final search information determined based on the additional information from the user.

[0251] In addition, the server may provide an optimal search result to the user by retraining and updating the content search model based on content finally selected by the user from the search result and the final search information used in the search result.

[0252] FIG. 13 is a flowchart for describing a method for displaying a content search service of an electronic device communicatively connected to a server according to an embodiment.

[0253] As illustrated in FIG. 13, the electronic device may receive search request data for a content search request from the user (S210).

[0254] Then, the electronic device may transmit the content search request including search request data to the server (S220).

[0255] Here, the electronic device may transmit, to the server, the content search request together with the search request data when a user input including the search request data and the content search request is received.

[0256] For example, the search request data may be at least one of text, an image, sound, and voice.

[0257] Next, the electronic device may receive a request for the additional required information from the server 100 and display a user response input window corresponding to the request for the additional required information (S230).

[0258] Then, the electronic device may transmit, to the server, the additional information of the user input through the user response input window (S240).

[0259] Here, the electronic device may display an initial search result corresponding to the search request data when receiving the initial search result from the server, and may display the user response input window corresponding to a request for the additional required information when receiving the request for the additional required information from the server.

[0260] For example, when the additional required information is a search category, the user response input window may include a main search icon corresponding to the search category over the initial search result.

[0261] In this case, when a user input for selecting the main search icon corresponding to the search category is received, the electronic device may check whether a subcategory corresponding to the selected search icon exists and display a sub-search icon corresponding to the subcategory to be matched with the main search icon when the subcategory exists.

[0262] In addition, when a user input for selecting the sub-search icon is received, the electronic device may display an input field for a search weight of the user for each item included in the subcategory of the selected sub-search icon.

[0263] Here, the electronic device may transmit the search weight of the user to the server when the user input is received in the input field for the search weight.

[0264] For example, the electronic device may transmit, to the server, the search weight of the user that is input as an higher level when the user inputs the higher level into the input field for the search weight, transmit, to the server, the search weight of the user that is input as a middle level when the user inputs the middle level into the input field for the search weight, and transmit, to the server, the search weight of the user that is input as a lower level when the user inputs the lower level into the input field for the search weight.

[0265] In some cases, when the additional required information is a search category, the user response input window may include a speech bubble corresponding to the search category over the initial search result.

[0266] In this case, the electronic device may display a speech bubble containing text requesting additional required information corresponding to the search category.

[0267] Subsequently, the electronic device may display a speech bubble containing text for a user's response corresponding to the speech bubble containing the text requesting additional required information when the user's response is input and transmit, to the server, text data for the user's response.

[0268] Next, the electronic device may receive a final search result corresponding to the additional information from the server (S250).

[0269] In addition, the electronic device may display the final search result for content search (S260).

[0270] Here, when the search result is received from the server, the electronic device may display the search result including at least one of the main information about finally retrieved content and a thumbnail, a replay video, and a preview video of the finally retrieved content.

[0271] In this way, the present disclosure may provide an optimal search result with high accuracy and reliability to the user requesting the content search using the final search information determined based on the additional information from the user.

[0272] The present disclosure may provide an optimal search result to the user by retraining and updating the content search model based on content finally selected by the user from the search result and final search information used in the search result.

[0273] The user interface forms such as icons and speech bubbles shown in the drawings are merely examples, and various forms of graphical user interfaces (GUIs), voice user interfaces (VUIs), or chatbot interfaces for interacting with the user to obtain 'additional information' may be implemented within the scope of the present invention.

[0274] The above-described methods for calculating relevance scores (e.g., cosine similarity, BM25) or natural language processing (e.g., NER) are merely examples for implementing the invention, and it will be apparent to those skilled in the art that various other analysis, inference, and learning algorithms and models may be applied without departing from the spirit and scope of the invention.

[0275] The electronic device and the server described in the present disclosure may be implemented by a hardware component, a software component, and / or a combination of a hardware component and a software component. In addition, the present disclosure may be provided in the form of a computer program stored in a computer readable storage medium to perform a method for operating the electronic device and the server. In addition, the present disclosure may be written as a program that executable on a computer, and may be implemented in a general-purpose digital computer that operates such a program using the computer-readable storage medium.

[0276] The computer-readable storage medium may be a read-only memory (ROM), a random-access memory (RAM), a flash memory, CD-ROMs, CD-Rs, CD+Rs, CD-RWs, CD+RWs, DVD-ROMs, DVD-Rs, DVD+Rs, DVD-RWs, DVD+RWs, DVD-RAMs, BD-ROMs, BD-Rs, BD-R LTHs, BD-REs, magnetic tapes, floppy disks, magneto-optical data storage devices, optical data storage devices, hard disks, or solid-state disks (SSDs), and may be any device capable of storing instructions or software, related data, data files, and data structures, and providing instructions or software, related data, data files, and data structures to a processor or a computer so that the processor or the computer may execute the instructions.

[0277] Although the embodiments have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by a person having ordinary skill in the art using the basic concept of the present invention defined in the following claims also fall within the scope of the present invention.

Claims

1.A server comprising:a communication device configured to communicate with an electronic device;a database for storing search information for content search by category; anda processor configured to provide a search result corresponding to a content search request of the electronic device,wherein the processor obtains search information for searching for at least one content from the electronic device, determines additional required information based on the obtained search information, requests the additional required information from the electronic device, determines final search information based on additional information obtained from the electronic device in response to a request for the additional required information, and provides a search result to the electronic device based on the final search information.2.The server of claim 1, wherein the processor obtains search request data input to the electronic device based on a content search request received from the electronic device, and extracts the search information from the search request data.3.The server of claim 2, wherein the processor extracts a search term from text data included in the search request data and obtains the search information corresponding to the search term.4.The server of claim 2, wherein the processor extracts text data from image data included in the search request data and obtains the search information from the text data.5.The server of claim 4, wherein the processor calculates a relevance score for each of a plurality of text segments extracted from the image data with respect to objects represented in the image data, selects a text segment with highest relevance score, and obtains the search information from the selected text segment.6.The server of claim 2, wherein the processor converts sound data included in the search request data into text data, extracts a search term from the text data, and obtains the search information corresponding to the search term.7.The server of claim 1, wherein the processor performs a content search based on the search information, determines an initial search result based on the content search, and determines the additional required information for specifying the final search information based on the determined initial search result.8.The server of claim 7, wherein the processor determines the additional required information when the amount of content information corresponding to the initial search result is greater than or equal to a preset reference amount of information.9.The server of claim 8, wherein the processor provides the initial search result to the electronic device when the amount of content information corresponding to the initial search result is less than the preset reference amount of information.10.The server of claim 7, wherein the processor selects a search category based on at least one of a content genre, an actor, and a synopsis corresponding to the initial search result, and determines the additional required information based on the selected search category.11.The server of claim 10, wherein the processor selects the search category further based on a user's search history for at least one of the content genre, the actor, and the synopsis corresponding to the initial search result.12.The server of claim 10, wherein the processor selects the search category including a higher-level category and a subcategory of the higher-level category when the subcategory corresponding to the higher-level category exists.13.The server of claim 7, wherein the processor provides a search category included in the additional required information and the initial search result to the electronic device so that a main search icon corresponding to the search category is displayed over the initial search result on the electronic device.14.The server of claim 13, wherein the processor receives, from the electronic device, a user input for selecting the main search icon corresponding to the search category, and, when a subcategory corresponding to the selected main search icon exists, provides the subcategory to the electronic device so that a sub-search icon corresponding to the subcategory is displayed with the main search icon on the electronic device.15.The server of claim 14, wherein the processor receives, from the electronic device, a user input for selecting the sub-search icon, provides items included in a subcategory of the selected sub-search icon to the electronic device so that an input field for a search weight for each item is displayed on the electronic device, and receives the search weight for each item input through the input field.16.The server of claim 7, wherein the processor provides a search category included in the additional required information and the initial search result to the electronic device so that a speech bubble corresponding to the search category is displayed over the initial search result on the electronic device.17.The server of claim 7, wherein the processor receives, from the electronic device, a user's response to the additional required information, performs an additional content search based on the user's response, determines an additional search result based on the additional content search, and, when the final search information is not determined based on the additional search result, requests further additional required information for determining the final search information from the electronic device.18.The server of claim 1, wherein the processor performs a content search based on the additional information, determines an intermediate search result based on the content search, determines further additional required information based on the intermediate search result when the amount of information in the intermediate search result is greater than or equal to a preset reference amount of information, requests the further additional required information from the electronic device, and determines the final search information based on further additional information obtained from the electronic device in response to a request for the further additional required information.19.The server of claim 1, wherein the processor provides at least one of main information about finally retrieved content and a thumbnail, a replay video, or a preview video of the finally retrieved content to the electronic device based on the search result.20.The server of claim 1, wherein the processor inputs the final search information into a pre-trained search model, and obtains the search result including content that a user wants to search for.21.The server of claim 20, wherein the processor obtains content information classified by category, obtains metadata of content corresponding to the content information, trains the search model based on the content information and the metadata, obtains a training result, retrains the search model based on the training result, and updates the search model.22.The server of claim 20, wherein the processor receives, from the electronic device, a user input for selecting specific content from the search result, analyzes a correlation between attributes of the selected content and the final search information, retrains the search model based on the correlation, and updates the search model.23.The server of claim 22, wherein the processor obtains at least one search category corresponding to the selected content, obtains at least one search term included in the final search information, matches the search category of the content with the search term of the final search information, and analyzes the correlation between the attributes of the selected content and the final search information based on the matching result.24.A method for providing a content search service of a server, comprising:obtaining search information for searching for at least one content from an electronic device;determining additional required information based on the obtained search information;requesting the additional required information from the electronic device;obtaining additional information in response to a request for the additional required information from the electronic device;determining final search information based on the additional information; andproviding a search result to the electronic device based on the final search information.25.A non-transitory computer-readable storage medium storing instructions that, when executed by a processor of a server, cause the server to perform a method for providing a content search service, the method comprising:obtaining search information for searching for at least one content from an electronic device;determining additional required information based on the obtained search information;requesting the additional required information from the electronic device;obtaining additional information in response to a request for the additional required information from the electronic device;determining final search information based on the additional information; andproviding a search result to the electronic device based on the final search information.26.An electronic device comprising:a communication device configured to communicate with a server;a display configured to display a content search result provided from the server; anda processor configured to control the communication device and the display,wherein the processor transmits a content search request including search request data to the server, receives, from the server, a request for additional required information, determined based on the search request data, displays a user response input window on the display in response to the request for the additional required information, transmits additional information based on a user input through the user response input window to the server, receives, from the server, a search result for the content search request determined based on the additional information, and displays the search result on the display.27.A method for a content search service of an electronic device, the method comprising:transmitting a content search request including search request data to a server;receiving, from the server, a request for additional required information determined based on the search request data;displaying a user response input window in response to the request for the additional required information;transmitting, to the server, additional information based on a user input through the user response input window;receiving, from the server, a search result for the content search request determined based on the additional information; anddisplaying the search result.28.A non-transitory computer-readable storage medium storing instructions that, when executed by a processor of an electronic device, cause the electronic device to perform a method for a content search service, the method comprising:transmitting a content search request including search request data to a server;receiving, from the server, a request for additional required information determined based on the search request data;displaying a user response input window in response to the request for the additional required information;transmitting, to the server, additional information based on a user input through the user response input window;receiving, from the server, a search result for the content search request determined based on the additional information; anddisplaying the search result.

Images