Electronic device for providing natural language-based assistance and method therefor

Abstract

Provided are an electronic device for providing natural language-based assistance and a method therefor. The electronic device comprises: a display; at least one processor comprising processing circuitry; and a memory comprising one or more storage media storing instructions, wherein the instructions, when executed by the at least one processor, may cause the electronic device to: receive, from a user, an input query based on a natural language; generate an input embedding representation by encoding the input query; determine, from configuration items, an embedding-based search result related to the input query by comparing the input embedding representation with configuration embedding representations based on text information of the configuration items configuring the electronic device; and display the embedding-based search result on an assistance screen of the display.

Description

Electronic device and method for providing natural language-based assistance

[0001] The following embodiments relate to an electronic device and a method for providing natural language-based assistance.

[0002] Electronic devices can provide various functions. Electronic devices can provide various setting items for configuring the various functions of the electronic device. Users can configure the various functions of the electronic device according to their needs by using these various setting items.

[0003] The information described above may be provided as related art for the purpose of aiding understanding of this document. None of the foregoing is to be claimed as prior art related to this document, nor is it to be used to determine prior art.

[0004] According to one embodiment, the electronic device includes a display, at least one processor including a processing circuit, and a memory including one or more storage media for storing instructions. When instructions are executed by at least one processor, the electronic device may receive a natural language-based input query from a user, encode the input query to generate an input embedding representation, compare the input embedding representation with setting embedding representations based on text information of setting items that configure the electronic device, determine an embedding-based search result related to the input query from the setting items, and display the embedding-based search result on an assist screen of the display.

[0005] According to one embodiment, the electronic device includes a display, at least one processor including a processing circuit, and a memory including one or more storage media for storing instructions. When instructions are executed by at least one processor, the electronic device may receive a natural language-based input query from a user, encode the input query to generate an input embedding representation, compare the input embedding representation with setting embedding representations based on text information of setting items that set the electronic device to determine an embedding-based search result related to the input query from the setting items, determine a text keyword according to the input query, compare the text keyword with text information of the setting items to determine a text-based search result related to the input query from the setting items, and display the embedding-based search result and the text-based search result on an assist screen of the display.

[0006] According to one embodiment, a method performed by an electronic device may include receiving a natural language-based input query from a user, encoding the input query to generate an input embedding representation, comparing the input embedding representation with setting embedding representations based on text information of setting items that set the electronic device to determine an embedding-based search result related to the input query from the setting items, and displaying the embedding-based search result on an assist screen of a display.

[0007] FIG. 1 is a drawing illustrating the configuration of an electronic device according to one embodiment.

[0008] FIG. 2 is a diagram illustrating the configuration of a program in an exemplary manner according to one embodiment.

[0009] FIG. 3 is a drawing that exemplarily illustrates the configuration of an assist screen according to one embodiment.

[0010] FIG. 4 is a flowchart exemplarily illustrating an embedding-based search operation according to one embodiment.

[0011] FIG. 5 is a diagram exemplarily illustrating the operation of forming a text-based database and an embedding-based database according to one embodiment.

[0012] FIG. 6 is a diagram illustrating, in an exemplary manner, an operation of comparing an input embedding representation and a set embedding representation according to one embodiment.

[0013] FIG. 7 is a flowchart exemplarily illustrating a text-based search operation according to one embodiment.

[0014] FIG. 8 is a diagram exemplarily illustrating an operation of comparing text keywords and text information according to one embodiment.

[0015] FIG. 9 is a diagram exemplarily showing pre-assist information of an assist screen according to one embodiment.

[0016] FIG. 10 is a drawing exemplarily showing the search results of an assist screen according to one embodiment.

[0017] FIG. 11 is a diagram exemplarily illustrating alignment results based on similarity and rank according to one embodiment.

[0018] FIG. 12 is a diagram exemplarily illustrating the alignment results of the embedding-based search results and text-based search results of each setting group according to one embodiment.

[0019] FIG. 13 is a flowchart exemplarily illustrating an operation to determine the rank of each search result according to one embodiment.

[0020] FIG. 14 is a diagram exemplifying a weighting operation based on the linguistic characteristics of an input query according to one embodiment.

[0021] FIG. 15 is a diagram illustrating an exemplary language model that outputs the priority of search results according to one embodiment.

[0022] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are given the same reference numeral regardless of the drawing number, and redundant descriptions thereof will be omitted.

[0023] FIG. 1 is a diagram illustrating the configuration of an electronic device according to one embodiment. Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through the server (108).

[0024] According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display (160), audio module (170), sensor (176), interface (177), connection terminal (178), haptic module (179), camera (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor (176), camera (180), or antenna module (197)) may be integrated into a single component (e.g., display (160)).

[0025] The processor (120) may be implemented as one or more IC (integrated circuit (or circuitry)) chips and may perform various data processing operations. The processor (120) may include at least one electrical circuit and may process instructions (or programs (140), data, etc.) stored in memory (130) individually or collectively in a distributed manner. The processor (120) may include a processor assembly comprising one or more processing circuits. The processor (120) may include any processing circuit that is operative to control the performance and operation of one or more components of the electronic device (101) (e.g., memory (130), display (160), camera (180), communication module (190), and / or sensor (176)).

[0026] The processor (120) can control at least one other component (e.g., hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., program (140)), for example, and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., sensor (176) or communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., central processing unit or application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., graphics processing unit, neural processing unit (NPU), image signal processor, sensor hub processor, or communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0027] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display (160), sensor (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0028] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related instructions. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0029] According to one embodiment, the electronic device includes a display, at least one processor including a processing circuit, and a memory including one or more storage media for storing instructions. When instructions are executed by at least one processor, the electronic device may receive a natural language-based input query from a user, encode the input query to generate an input embedding representation, compare the input embedding representation with setting embedding representations based on text information of setting items that configure the electronic device, determine an embedding-based search result related to the input query from the setting items, and display the embedding-based search result on an assist screen of the display.

[0030] When instructions are executed by at least one processor, the electronic device can determine text keywords according to an input query, compare the text keywords with text information of setting items, determine text-based search results related to the input query from the setting items, and display the text-based search results on an assist screen.

[0031] When instructions are executed by at least one processor, the electronic device may determine the ranks of first setting items of an embedding-based search result and second setting items of a text-based search result based on embedding-based ranking conditions and text-based ranking conditions, and sort the first setting items and second setting items based on the ranks to display the first setting items and second setting items on an assist screen.

[0032] The embedding-based ranking condition includes a threshold regarding the similarity between the input embedding representation and the set embedding representations, and the text-based ranking condition may include a matching state between text keywords and text information.

[0033] The assist screen includes search result areas by setting group, and when instructions are executed by at least one processor, the electronic device may cause the first setting items and the second setting items to be sorted based on ranks within the search result area to which each of the first setting items and the second setting items belongs among the search result areas.

[0034] When instructions are executed by at least one processor, the electronic device may execute a neural network-based language model based on an input query, one or more of embedding-based search results and text-based search results, and context information of the electronic device to generate priority information for one or more of the embedding-based search results and text-based search results, and display one or more of the embedding-based search results and text-based search results on an assist screen based on the priority information.

[0035] When instructions are executed by at least one processor, the electronic device may be configured to assign a higher weight to either the embedding-based search result or the text-based search result based on the linguistic characteristics of the input query.

[0036] The assist screen may include one or more of a first assist area including a recommendation query based on context information of an electronic device, a second assist area including a recent input query, and a third assist area including a preset tag.

[0037] When instructions are executed by at least one processor, the electronic device may be configured to collect configuration items of the electronic device, determine text information of the configuration items, and generate configuration embedding representations based on the text information of the configuration items.

[0038] When instructions are executed by at least one processor, the electronic device may be configured to periodically or non-periodically update one or more configuration items, update one or more corresponding configuration embedding representations of configuration embedding representations based on the one or more updated configuration items, and determine an embedding-based search result based on the one or more updated corresponding configuration embedding representations.

[0039] Text information of configuration items includes one or more identifiers, keys, permissions, titles, and descriptions of each configuration item, and configuration embedding representations can be generated based on the descriptions of the text information of each configuration item.

[0040] Input embedding representations and configuration embedding representations are numeric values ​​in which input query and configuration items are mapped to a feature space, and the numeric values ​​may depend on the similarity between the input query and configuration items.

[0041] According to one embodiment, the electronic device includes a display, at least one processor including a processing circuit, and a memory including one or more storage media for storing instructions. When instructions are executed by at least one processor, the electronic device may receive a natural language-based input query from a user, encode the input query to generate an input embedding representation, compare the input embedding representation with setting embedding representations based on text information of setting items that set the electronic device to determine an embedding-based search result related to the input query from the setting items, determine a text keyword according to the input query, compare the text keyword with text information of the setting items to determine a text-based search result related to the input query from the setting items, and display the embedding-based search result and the text-based search result on an assist screen of the display.

[0042] When instructions are executed by at least one processor, the electronic device can determine text keywords according to an input query, compare the text keywords with text information of setting items, determine text-based search results related to the input query from the setting items, and display the text-based search results on an assist screen.

[0043] When instructions are executed by at least one processor, the electronic device may determine the ranks of the first setting items of the embedding-based search results and the second setting items of the text-based search results based on the embedding-based ranking conditions and the text-based ranking conditions, and sort the first setting items and the second setting items based on the ranks to display the first setting items and the second setting items on an assist screen.

[0044] The embedding-based ranking condition includes a threshold regarding the similarity between the input embedding representation and the set embedding representations, and the text-based ranking condition may include a matching state between text keywords and text information.

[0045] The assist screen includes search result areas by setting group, and when instructions are executed by at least one processor, the electronic device may be configured to sort the first setting items and the second setting items based on ranks within the search result area to which each of the first setting items and the second setting items belongs among the search result areas.

[0046] When instructions are executed by at least one processor, the electronic device may execute a neural network-based language model based on an input query, one or more of embedding-based search results and text-based search results, and context information of the electronic device to generate priority information for one or more of the embedding-based search results and text-based search results, and display one or more of the embedding-based search results and text-based search results on an assist screen based on the priority information.

[0047] When instructions are executed by at least one processor, the electronic device may be configured to assign a higher weight to either the embedding-based search result or the text-based search result based on the linguistic characteristics of the input query.

[0048] According to one embodiment, a method performed by an electronic device may include receiving a natural language-based input query from a user, encoding the input query to generate an input embedding representation, comparing the input embedding representation with setting embedding representations based on text information of setting items that set the electronic device to determine an embedding-based search result related to the input query from the setting items, and displaying the embedding-based search result on an assist screen of a display.

[0049] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0050] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0051] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0052] A display (160) (e.g., a display) can visually provide information to an external (e.g., a user) outside of the electronic device (101). The display (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0053] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0054] The sensor (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor. For example, the sensor (176) may include an inertial measurement unit (IMU).

[0055] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0056] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0057] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0058] The camera (180) can capture still images and video. According to one embodiment, the camera (180) may include one or more lenses, one or more image sensors, one or more image signal processors, or one or more flashes.

[0059] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0060] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0061] A communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication circuits. The communication module (190) may include one or more communication processors (CP) that operate independently of a processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local region network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct or IrDA (infrared data relation)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips).The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network, such as a first network (198) or a second network (199), using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0062] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) can support a Peak data rate (e.g., 20 Gbps or more) for eMBB realization, loss coverage (e.g., 164 dB or less) for mMTC realization, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for URLLC realization.

[0063] An antenna module (197) can transmit a signal or power to an external source (e.g., an external electronic device) or receive it from an external source. According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0064] According to one embodiment, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0065] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0066] According to one embodiment, commands or data may be transmitted or received between an electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199).

[0067] Each of the external electronic devices (102, 104) and the server (108) may be of the same or different type as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104) or the server (108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the request may perform at least part of the requested function or service, or additional functions or services related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request.

[0068] FIG. 2 is a diagram illustrating the configuration of a program according to one embodiment. According to one embodiment, the program (140) may include an operating system (142), middleware (144), or an application (146) executable on the operating system (142) for controlling one or more resources of an electronic device (101). The operating system (142) may include, for example, Android™, iOS™, Windows™, Symbian™, Tizen™, or Bada™. At least some of the programs (140) may be preloaded into the electronic device (101) at manufacturing time, for example, or downloaded or updated from an external electronic device (e.g., electronic device (102 or 104), or server (108)) when used by a user.

[0069] The operating system (142) can control the management (e.g., allocation or reclamation) of one or more system resources (e.g., processes, memory, or power) of the electronic device (101). The operating system (142) may additionally or substantially include one or more driver programs for driving other hardware devices of the electronic device (101), e.g., an input module (150), an audio output module (155), a display (160), an audio module (170), a sensor (176), an interface (177), a haptic module (179), a camera (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197).

[0070] Middleware (144) may provide various functions to an application (146) so that functions or information provided from one or more resources of an electronic device (101) can be used by the application (146). Middleware (144) may include, for example, an application manager (201), a window manager (203), a multimedia manager (205), a resource manager (207), a power manager (209), a database manager (211), a package manager (213), a connectivity manager (215), a notification manager (217), a location manager (219), a graphics manager (221), a security manager (223), a call manager (225), or a voice recognition manager (227).

[0071] The application manager (201) can, for example, manage the life cycle of the application (146). The window manager (203) can, for example, manage one or more GUI resources used on the screen. The multimedia manager (205) can, for example, identify one or more formats required for the playback of media files and perform encoding or decoding of the corresponding media files among the media files using a codec that matches the selected corresponding format. The resource manager (207) can, for example, manage the source code of the application (146) or the memory space of the memory (130). The power manager (209) can, for example, manage the capacity, temperature, or power of the battery (189) and, using the relevant information, determine or provide relevant information required for the operation of the electronic device (101). According to one embodiment, the power manager (209) can interact with the BIOS (basic input / output system) (not shown) of the electronic device (101).

[0072] The database manager (211) can, for example, create, search, or modify a database to be used by the application (146). The package manager (213) can, for example, manage the installation or update of the application distributed in the form of a package file. The connectivity manager (215) can, for example, manage a wireless or direct connection between the electronic device (101) and an external electronic device. The notification manager (217) can, for example, provide a function to notify the user of the occurrence of a specified event (e.g., an incoming call, a message, or an alarm). The location manager (219) can, for example, manage location information of the electronic device (101). The graphics manager (221) can, for example, manage one or more graphic effects or related user interfaces to be provided to the user.

[0073] The security manager (223) may, for example, provide system security or user authentication. The telephony manager (225) may, for example, manage voice call functions or video call functions provided by the electronic device (101). The voice recognition manager (227) may, for example, transmit user voice data to the server (108) and receive from the server (108) a command corresponding to a function to be performed on the electronic device (101) based on at least part of the voice data, or text data converted based on at least part of the voice data. According to one embodiment, the middleware (244) may dynamically delete some existing components or add new components. According to one embodiment, at least part of the middleware (144) may be included as part of the operating system (142) or implemented as separate software different from the operating system (142).

[0074] The application (146) may include, for example, a home (251), a dialer (253), an SMS / MMS (255), an IM (instant message) (257), a browser (259), a camera (261), an alarm (263), a contact (265), a voice recognition (267), an email (269), a calendar (271), a media player (273), an album (275), a watch (277), a health (279) (e.g., measuring biometric information such as exercise volume or blood sugar), or an environmental information (281) (e.g., measuring atmospheric pressure, humidity, or temperature information). According to one embodiment, the application (146) may further include an information exchange application (not shown) capable of supporting information exchange between the electronic device (101) and an external electronic device. The information exchange application may include, for example, a notification relay application configured to transmit information (e.g., a call, a message, or an alarm) designated to an external electronic device, or a device management application configured to manage the external electronic device. The notification relay application may transmit notification information corresponding to a designated event (e.g., receiving mail) generated in another application of the electronic device (101) (e.g., an email application (269)) to the external electronic device. Additionally or alternatively, the notification relay application may receive notification information from the external electronic device and provide it to the user of the electronic device (101).

[0075] A device management application can control the power (e.g., turn-on or turn-off) or function (e.g., brightness, resolution, or focus) of an external electronic device or a part of its components (e.g., a display module or camera module of the external electronic device) that communicates with the electronic device (101). The device management application can additionally or substantially support the installation, deletion, or updating of applications running on the external electronic device.

[0076] FIG. 3 is a drawing that exemplarily illustrates the configuration of an assist screen according to one embodiment.

[0077] An electronic device (e.g., the electronic device (101) of FIG. 1) may provide various setting items for setting the electronic device. A setting item may be a setting unit that classifies setting values ​​for setting the functions of the electronic device. For example, a setting item may include, but is not limited to, one or more of connection setting items (e.g., WiFi, Bluetooth, NFC (Near Field Communication), UWB (ultrawide band), mobile network, hotspot, tethering), inter-device connection setting items (quick share, call / text on other devices, camera sharing, multi-control, wearable device), display setting items (e.g., brightness, brightness optimization, color optimization, screen mode, font size, screen resolution), sound and vibration setting items (e.g., ringtone, notification sound, system sound, volume, vibration pattern, vibration intensity), notification setting items (e.g., application notification, lock screen notification, notification popup style, do not disturb), battery and performance setting items (e.g., battery usage, power saving mode, high performance mode, charging settings), application setting items (e.g., application management, application permissions, default application settings), and individual application setting items (e.g., function settings for internet applications).

[0078] Referring to FIG. 3, the assist screen (300a) may include prior assist information (310) and an input window (320). According to one embodiment, a user may enter a natural language-based input query (321) into the input window (320). In the case of keyword search (e.g., rule-based search), the user must enter a setting item of the desired function or a synonym of the setting item to search for the desired setting item. As the setting items become more complex, it may be difficult for the user to obtain the desired setting item. According to one embodiment, the user can easily obtain search results (330) regarding setting items suitable for the user by using a natural language-based input query (321). For example, if the electronic device is a smartphone and the user wishes to receive a call received on the smartphone on a tablet computer, the user may enter an input query (321) such as "Let me answer the call on the tablet" into the input window (320). If the search method is limited to keyword search, setting items that do not match the user's intent, such as "phone," may be derived as search results (330). If natural language search of one embodiment (e.g., embedding-based search) is used, setting items that match the user's intent, such as "make phone calls / text messages on other devices," may be derived as search results (330). If an input query (321) is entered via voice recognition, a search result (330) suitable for the input query (321) may be provided using natural language search.

[0079] Natural language search can be performed through an assist screen (e.g., assist screen (300a, 300b)). For example, the assist screen (300a) may include prior assist information (310) before a search using an input query (321) is performed, and the assist screen (300b) may include a search result (330) after a search using an input query (321) is performed.

[0080] The pre-assist information (310) may include various information to assist the user before a search is performed. According to one embodiment, the pre-assist information (310) may include assist areas (e.g., a first assist area (311), a second assist area (312), and a third assist area (313)).

[0081] According to one embodiment, the first assist area (311) may include a recommendation query based on context information of the electronic device. The electronic device may collect context information using the configuration of the electronic device (e.g., sensor (176), battery (189), audio module (170), camera (180)). For example, the context information may include, but is not limited to, one or more of the following: device connection status (e.g., state of being connected to multiple devices via Bluetooth), running application (e.g., music application), current time (e.g., evening time), current day of the week (e.g., weekday, weekend), current date (e.g., holiday), current illuminance (e.g., low light), current location (e.g., country where the electronic device is located), ambient noise, storage space status (e.g., insufficient storage space), communication status (e.g., message reception), battery status (e.g., low battery status), and the user's vital signs (e.g., heart rate, sleep status, step count).

[0082] According to one embodiment, the first assist area (311) may provide recommendation queries related to connections and operations between devices based on context information. For example, when a user is using multiple devices or when a device connected to an electronic device is detected, recommendation queries related to context information may be provided. For example, recommendation queries may include, but are not limited to, "Let me answer a call on the tablet," "Use a laptop keyboard together with a connected mobile phone," "How do I share the screen to a TV?", and "Manage notifications on a smartwatch."

[0083] According to one embodiment, the first assist area (311) may provide translation and language-related recommendation queries based on context information. For example, if the use of a browser or a translation application is detected, recommendation queries related to language settings such as "I want to translate and view an English site," "Turn on automatic translation," "How to add a translation language," and "Set up a foreign language keyboard" may be provided.

[0084] According to one embodiment, the first assist area (311) may provide a recommendation query related to the terminal state based on context information. For example, when entering the assist screen (300a) when the battery is low or when a specific mode (e.g., power saving mode) is not activated, a recommendation query based on the terminal state may be provided. For example, the recommendation query may include, but is not limited to, "to save battery," "block app background data," "enable power saving mode," and "check battery usage."

[0085] According to one embodiment, the first assist area (311) may provide display and interface-related recommendation queries based on context information. For example, if a user uses the screen for a long time or a low-light environment is detected, display-related recommendation queries may be provided. For example, recommendation queries may include, but are not limited to, "screen settings to reduce eye strain," "to dim the screen," "turn on night mode," and "blue light filter settings."

[0086] According to one embodiment, the first assist area (311) may provide recommendation queries related to general usability improvement based on context information. For example, frequently asked queries regarding general device usage may be recommended in natural language. For example, recommendation queries may include, but are not limited to, "how to turn off notifications," "lower the sound of incoming calls," "how to use Galaxy AI," and "speed up app launch."

[0087] According to one embodiment, the second assist area (312) may include a recent input query (321). For example, the recent input query (321) may include natural language such as "how to use AI (artificial intelligence)," "let me answer the phone on the tablet," or "in another language." Input queries (321) that have been used once by the user may tend to be used again. The user can conveniently use natural language search by utilizing the recent input query (321) in the second assist area (312) without re-entering the same input query (321) into the input window (320).

[0088] According to one embodiment, the third assist area (313) may include a pre-set tag. Certain tags may be assigned to the setting items. The setting items may be grouped based on the tags. For example, the tags may include, but are not limited to, one or more of "in the evening," "customize visuals," "pen," "use with one hand," "privacy," and "emergency." When a user selects a specific tag among the tags of the third assist area (313), the setting items having that tag may be displayed as search results (330).

[0089] According to one embodiment, the search result (330) may include setting items (e.g., a first setting item (331), a second setting item (332), a third setting item (333)). The electronic device may search for setting items using a natural language-based input query (321) and display setting items corresponding to the input query (321) as the search result (330). The setting items may be classified by setting group. For example, WiFi, Bluetooth, NFC, UWB, mobile network, hotspot, and tethering may be classified as a connection setting group, and brightness, brightness optimization, color optimization, screen mode, font size, and screen resolution may be classified as a display setting group. According to one embodiment, the setting items of the search result (330) may be displayed according to the setting group to which the setting items belong.

[0090] According to one embodiment, the electronic device may perform a natural language-based search (e.g., an embedding-based search) and / or a keyword-based search (e.g., a text-based search). For example, the electronic device may perform a natural language-based search to derive a search result (330), perform a keyword-based search to derive a search result (330), or perform both a natural language-based search and a keyword-based search to derive a search result (330). According to one embodiment, the electronic device may selectively perform the more suitable of the natural language-based search and the input query (321) based on the characteristics (e.g., linguistic characteristics) of the input query (321). According to one embodiment, the electronic device may perform both a natural language-based search and a keyword-based search, but may give higher weight to the search result (330) that is more suitable to the characteristics of the input query (321).

[0091] FIG. 4 is a flowchart exemplarily illustrating an embedding-based search operation according to one embodiment. Referring to FIG. 4, in operation (410), an electronic device (e.g., the electronic device (101) of FIG. 1) may receive a natural language-based input query (e.g., the input query (321) of FIG. 3) from a user. For example, the input query may correspond to natural language or may contain natural language.

[0092] In operation (420), the electronic device may generate an input embedding representation by encoding an input query. The electronic device may generate an embedding representation of the input query when an input query is entered by a user. The electronic device may generate a corresponding embedding representation whenever an input query is entered by a user. The input embedding representation may be a numeric value determined by mapping the input query to a feature space. The feature space may be an n-dimensional space. n may be a natural number greater than or equal to 1. Encoding may include an operation to generate a numeric value corresponding to a text-based input query. For example, the embedding representation may be a vector representation, but is not limited thereto. In this case, the numeric value of the input embedding representation may be a vector value. The electronic device may convert all or part of the input query into an input embedding representation. For example, the electronic device may generate an input embedding representation by performing vectorization on all or part of the input query, but is not limited thereto.

[0093] In operation (430), the electronic device can compare the input embedding representation with the setting embedding representations of the setting items. The electronic device can collect setting items for various functions of the electronic device (e.g., connection setting items, inter-device connection setting items, display setting items, sound and vibration setting items, notification setting items, battery and performance setting items, application setting items, and individual application setting items). The electronic device can create a database based on the setting items. Each setting item may have text information. For example, the text information may include one or more of the identification (ID), key, authority, title, and description of the corresponding setting item.

[0094] Configuration embedding representations can be numeric values ​​determined by mapping configuration items to a feature space. The numeric values ​​of input embedding representations and configuration embedding representations may depend on the similarity between the input query and the configuration items. For example, the higher the similarity between the input query and the configuration items, the more similar the input embedding representations and configuration embedding representations may have similar numeric values.

[0095] The electronic device may convert all or part of the text information of the configuration items into configuration embedding representations. For example, the electronic device may generate configuration embedding representations by performing vectorization on all or part of the text information of each configuration item, but is not limited thereto. For example, the electronic device may generate configuration embedding representations by vectorizing the description of each configuration item, but is not limited thereto. The electronic device may compare the input embedding representations with the configuration embedding representations by measuring the similarity between the numerical values ​​(e.g., vector values) of the input embedding representations and the numerical values ​​(e.g., vector values) of the configuration embedding representations. For example, the similarity may include, but is not limited to, cosine similarity (e.g., cosine distance).

[0096] In operation (440), the electronic device can determine an embedding-based search result related to an input query from the setting items. Based on the comparison result of operation (430), the electronic device can extract setting items of a setting embedding expression that have a high similarity to the input embedding expression. For example, the electronic device may extract setting items that have a similarity greater than a threshold, or extract a predetermined number of setting items in order of high similarity, but is not limited thereto.

[0097] In operation (450), the electronic device may display the embedding-based search results on an assist screen (e.g., the assist screen (300b) of FIG. 3) of a display (e.g., the display (160) of FIG. 1). The assist screen may include search result areas by setting group. For example, a setting group may include, but is not limited to, one or more of a connection setting group, a device-to-device connection setting group, a display setting group, a sound and vibration setting group, a notification setting group, a battery and performance setting group, an application setting group, and an individual application setting group.

[0098] For example, the assist screen may include a first search result area corresponding to a first setting group, a second search result area corresponding to a second setting group, and a third search result area corresponding to a third setting group. Each search result area may include one or more setting items belonging to a corresponding setting group. For example, the first search result area may include one or more setting items belonging to the first setting group, the second search result area may include one or more setting items belonging to the second setting group, and the third search result area may include one or more setting items belonging to the third setting group.

[0099] FIG. 5 is a diagram illustrating, in an exemplary manner, the operation of forming a text-based database and an embedding-based database according to one embodiment. Referring to FIG. 5, an electronic device (e.g., the electronic device (101) of FIG. 1) can collect setting items (510) (e.g., connection setting items, inter-device connection setting items, display setting items, sound and vibration setting items, notification setting items, battery and performance setting items, application setting items, and individual application setting items) for various functions of the electronic device.

[0100] The electronic device can determine text information for each of the setting items (510). The electronic device can determine text information for the setting items (510) based on the information of the setting items (510) obtained while collecting the setting items (510). The electronic device can create a text-based database (520) based on the text information of the setting items (510). For example, the text information may have a key-value format. For example, the key-value format may include a JSON (JavaScript Object Notation) format, but is not limited thereto. For example, the text-based database (520) may be a SQL (Structured Query Language) database, but is not limited thereto.

[0101] For example, text information may include one or more of the ID, key, authority, title, and description of the corresponding setting item. The ID may represent identification information of the setting item. The key may represent a menu providing the setting item. The authority may represent the entity (e.g., application) providing the setting item. The title may represent the subject of the setting item provided to the user. The description may represent the detailed function of the setting item. Table 1 below may show examples of text information for setting items (510) of a text-based database (520).

[0102]

[0103]

[0104] The electronic device can generate configuration embedding expressions based on text information of configuration items (510). The electronic device can generate an embedding-based database (530) containing configuration embedding expressions. The electronic device can convert all or part of the text information of configuration items (510) into configuration embedding expressions. For example, the electronic device can generate configuration embedding expressions by encoding (e.g., vectorizing) all or part of the text information of each configuration item, but is not limited thereto. For example, the electronic device can convert one or more of the ID, key, authority, title, and description of each configuration item into configuration embedding expressions. For example, the electronic device can generate configuration embedding expressions based on the description of the text information of each configuration item.

[0105] The electronic device checks whether the text-based database (520) and the embedding-based database (530) have already been created, and if the text-based database (520) and the embedding-based database (530) have not already been created, it can create the text-based database (520) and the embedding-based database (530). The text-based database (520) and the embedding-based database (530) can be formed before an input query (e.g., the input query (321) of FIG. 3) is entered.

[0106] The text-based database (520) and the embedding-based database (530) may be updated periodically or non-periodically after being initially formed. The periodic and / or non-periodic timing at which the text-based database (520) and the embedding-based database (530) are updated may be the same or different. For example, the initial timing at which the text-based database (520) and the embedding-based database (530) are formed may include, but is not limited to, the time of production of the electronic terminal by the producer, the time of initial operation of the electronic terminal by the user, and the time of initial execution of the assist screen (e.g., the assist screen (300a) of FIG. 3) by the user. For example, the update cycle at which the text-based database (520) and the embedding-based database (530) are updated may include, but is not limited to, one week, one month, and one quarter. For example, non-periodic times when the text-based database (520) and the embedding-based database (530) are updated may include, but are not limited to, when a user request is made, when an assist screen (e.g., the assist screen (300a) of FIG. 3) is executed, and when a new application is installed.

[0107] The electronic device may update one or more of the setting items (510) periodically or non-periodically. For example, updates regarding previously collected setting items (510) and / or newly collected setting items (510) may be performed. The electronic device may update one or more corresponding setting embedding expressions of setting embedding expressions based on the one or more updated setting items (510). When the previously collected setting items (510) are updated, the corresponding setting embedding expressions may be updated. When newly collected setting items (510) exist, corresponding setting embedding expressions may be generated. The electronic device may determine embedding-based search results based on the one or more updated corresponding setting embedding expressions.

[0108] According to one embodiment, the electronic device may include an embedding processing module that converts all or part of the text information of the setting items (510) into a setting embedding representation. The embedding processing module may include a software module and / or a hardware module. The embedding processing module may generate a setting embedding representation by mapping all or part of the text information into a feature space. The electronic device may provide all or part of the text information of the setting items (510) to the embedding processing module. The embedding processing module may generate setting embedding representations based on the provided text information. The embedding processing module may generate an embedding-based database (530) based on the setting embedding representations. When the embedding-based database (530) is generated, the embedding processing module may notify the electronic device that the embedding-based database (530) has been generated.

[0109] According to one embodiment, consistent and accurate search results can be derived by using a text-based database (520) and an embedding-based database (530). For example, by using setting embedding expressions generated based on text data such as "tips and user manuals," accurate and fast search results can be derived even if a user enters a natural language-based input query such as "mobile usage tips" or "settings guide." According to one embodiment, an improved search environment can be provided by intelligently matching semantic associations between input queries and setting items.

[0110] FIG. 6 is a diagram illustrating, in an exemplary manner, an operation of comparing an input embedding representation and a configuration embedding representation according to one embodiment. Referring to FIG. 6, an embedding-based database (630) (e.g., the embedding-based database (530) of FIG. 5) may include configuration embedding representations (e.g., a first configuration embedding representation (631), a second configuration embedding representation (632), and a third configuration embedding representation (633)). The first configuration embedding representation (631), the second configuration embedding representation (632), and the third configuration embedding representation (633) may correspond to a first configuration item, a second configuration item, and a third configuration item, respectively. An electronic device (e.g., the electronic device (101) of FIG. 1) may generate an input embedding representation (622) based on an input query (e.g., the input query (321) of FIG. 3). The electronic device may compare the input embedding representation (622) and the configured embedding representations by measuring the similarity between the numerical values ​​(e.g., vector values) of the input embedding representation (622) and the numerical values ​​(e.g., vector values) of the configured embedding representations. For example, the similarity may include, but is not limited to, cosine similarity (e.g., cosine distance).

[0111] For example, an input query such as "using a laptop keyboard together with a connected mobile phone" may be entered. The input query may indicate a user's intention to use a keyboard connected to an electronic device (e.g., electronic device (101) of FIG. 1) simultaneously with another electronic device (e.g., electronic device (102), electronic device (104) of FIG. 1). Keywords such as "laptop," "keyboard," "connection," "mobile phone," and "use together" may be extracted from the input query. Using an embedding-based database (630), a "multi-control" setting item may be provided as a search result.

[0112] For example, an input query such as "Let me answer calls on the tablet" may be entered. The input query may indicate a user's intention to set up the tablet to answer calls. Keywords such as "tablet", "phone", and "answer" may be extracted from the input query. Using an embedding-based database (630), the "call / text on other devices" setting item may be provided as a search result.

[0113] For example, an input query such as "I want to translate and view an English site" may be entered. The input query may indicate a user's intention to translate a website by activating the internet translation function. Keywords such as "English", "site", "translate", and "view" may be extracted from the input query. Using an embedding-based database (630), a "browsing assist" setting item may be provided as a search result.

[0114] According to one embodiment, the electronic device may include an embedding processing module that generates an input embedding representation (622) and configuration embedding representations, and compares the input embedding representation (622) with the configuration embedding representations. The embedding processing module may include a software module and / or a hardware module. When an input query is input, the electronic device may generate an input embedding representation (622) using the embedding processing module. The embedding processing module may encode the input query into the input embedding representation (622). The embedding processing module may perform a natural language-based search by comparing the input embedding representation (622) with the configuration embedding representations, and select configuration items for the search result (e.g., the search result (330) of FIG. 3).

[0115] The embedding processing module can generate configuration embedding representations based on text data of configuration items. For example, an electronic device can transmit text data containing the key, authority, and description of each configuration item to the embedding processing module. The embedding processing module can encode the description of the text data of each configuration item into a configuration embedding representation.

[0116] The embedding processing module can determine the similarity between the input embedding representation (622) and the configuration embedding representations by comparing the numerical values ​​of the input embedding representation (622) with the numerical values ​​of the configuration embedding representations. The embedding processing module may select configuration items having a similarity greater than a threshold, or select a predetermined number of configuration items in order of highest similarity, but is not limited thereto. The embedding processing module may notify the electronic device of the selected configuration items. For example, the embedding processing module may output the key, authority, and similarity of each of the selected configuration items. The similarity may represent the similarity between the input embedding representation (622) and the configuration embedding representations of each selected configuration item.

[0117] The electronic device can provide search results based on selected setting items. The electronic device can identify selected setting items using keys and / or permissions. The electronic device can sort selected setting items within an assist screen (e.g., the assist screen (300b) of FIG. 3) using the similarity of the selected setting items.

[0118] FIG. 7 is a flowchart exemplifying a text-based search operation according to one embodiment. Referring to FIG. 7, in operation (710), an electronic device (e.g., the electronic device (101) of FIG. 1) can determine a text keyword based on an input query (e.g., the input query (321) of FIG. 3). The text keyword may include all or part of the input query. For example, the text keyword may include the entire text of the input query or some text extracted from the input query. According to one embodiment, the electronic device can parse the input query and extract some text of high importance from the input query. For example, if an input query "Let me answer the phone on the tablet" is entered, "Let me answer the phone on the tablet" may be determined as the text keyword, or "tablet" and "answer the phone" extracted from "Let me answer the phone on the tablet" may be determined as the text keywords.

[0119] In operation (720), the electronic device can compare a text keyword with text information of the setting items. The electronic device can create a text-based database (e.g., the text-based database (520) of FIG. 5) based on the text information of the setting items. The electronic device can compare a text keyword with text information of each setting item in the text-based database. The electronic device can compare a text keyword with all or part of the text information of each setting item. For example, the text information may include one or more of the ID, key, authority, title, and description of the setting item. For example, the electronic device can compare a text keyword with one or more of the ID, key, authority, title, and description of each setting item. For example, the electronic device can compare a text keyword with the title of each setting item, but is not limited thereto. For example, if the input query is "Let me receive calls on the tablet," "tablet" and "phone" can be extracted as keywords, and a setting item from a text-based database with the key "device_integration_call," the title "Call connection between devices," and the description "Settings to enable receiving calls on a tablet or other device." can be selected as a search result.

[0120] In operation (730), the electronic device can determine text-based search results related to the input query from the setting items. Based on the comparison result of operation (720), the electronic device can extract setting items of text information that have high similarity to the text keyword. For example, the electronic device can extract setting items that have similarity greater than a threshold, or extract a predetermined number of setting items in order of high similarity, but is not limited thereto.

[0121] In operation (740), the electronic device may display text-based search results on an assist screen (e.g., the assist screen (300b) of FIG. 3) of a display (e.g., the display (160) of FIG. 1). According to one embodiment, the electronic device may display embedding-based search results and text-based search results on the assist screen. For example, the embedding-based search results and the text-based search results may each include one or more setting items.

[0122] According to one embodiment, the electronic device can determine the rank of each of the first setting items of the embedding-based search results and the second setting items of the text-based search results based on a ranking condition. The rank may indicate the accuracy of the search results. A small rank value may be assigned to a search result having high accuracy. For example, if rank values ​​[1] to

[0013] are assigned to the search results, the rank value [1] may indicate the most accurate search result, but is not limited thereto. The ranking condition may represent a predefined condition for assigning a rank to the search results.

[0123] The ranking condition may include an embedding-based ranking condition for assigning a rank to an embedding-based search result and a text-based ranking condition for assigning a rank to a text-based search result. For example, the embedding-based ranking condition may include a threshold regarding the similarity between an input embedding expression and a set embedding expression. For example, the text-based ranking condition may include a matching state between a text keyword and text information. For example, the matching state may include, but is not limited to, a complete match, a partial match at a starting position, and a partial match at another position. The electronic device may sort the first set items and the second set items in the search result (e.g., the search result (330) in the assist screen (300b) of FIG. 3) based on the ranks of the first set items and the second set items.

[0124] The search results may include search result areas for each setting group. For example, a setting group may include, but is not limited to, one or more of a connection setting group, a device-to-device connection setting group, a display setting group, a sound and vibration setting group, a notification setting group, a battery and performance setting group, an application setting group, and an individual application setting group. The search results may include a search result area corresponding to each setting group. The electronic device may display a mixture of first setting items and second setting items for each setting group. For example, if first setting items and second setting items belonging to a first setting group are derived as search results, the electronic device may display the first setting items and second setting items in a first search result area corresponding to the first setting group.

[0125] The electronic device can sort first setting items and second setting items in each setting group based on the ranks of the first setting items and second setting items. For example, if a specific first setting item of the first setting items of the first setting group corresponds to the most accurate search result among the first setting items and second setting items, the smallest rank value (e.g., [1]) may be assigned to that first setting item, and that first setting item may be displayed at the top of the first search result area.

[0126] FIG. 8 is a diagram illustrating, in an exemplary manner, an operation of comparing text keywords and text information according to one embodiment. Referring to FIG. 8, a text-based database (820) (e.g., the text-based database (520) of FIG. 5) may include text information (e.g., first text information (821), second text information (822), third text information (823)). The first text information (821), second text information (822), and third text information (823) may correspond to a first setting item, a second setting item, and a third setting item, respectively. An electronic device (e.g., the electronic device (101) of FIG. 1) may generate a text keyword (825) based on an input query (e.g., the input query (321) of FIG. 3). The electronic device may compare the text keyword (825) and the text information by measuring the similarity between the text of the text keyword (825) and the text of the text information. For example, similarity may include, but is not limited to, text matching states.

[0127] FIG. 9 is a diagram illustrating, in an exemplary manner, pre-assist information of an assist screen according to one embodiment. Referring to FIG. 9, the assist screen (900a) (e.g., the assist screen (300a) of FIG. 3) may include pre-assist information (910) (e.g., the pre-assist information (310) of FIG. 3). The pre-assist information (910) may include assist areas (e.g., a first assist area (911), a second assist area (912), and a third assist area (913)).

[0128] According to one embodiment, the first assist area (911) (e.g., the first assist area (311) of FIG. 3) may include a recommendation query based on context information of an electronic device (e.g., the electronic device (101) of FIG. 1). For example, the context information may include, but is not limited to, one or more of the following: device connection status (e.g., state of being connected to multiple devices via Bluetooth), running application (e.g., music application), current time (e.g., evening time), current day of the week (e.g., weekday, weekend), current date (e.g., holiday), current illuminance (e.g., low illuminance), current location (e.g., country where the electronic device is located), ambient noise, storage space status (e.g., insufficient storage space), communication status (e.g., message reception), battery status (e.g., low battery status), and the user's vital signs (e.g., heart rate, sleep status, step count). For example, FIG. 9 illustrates an example in which the first assist area (911) includes, but is not limited to, "change time," "lower the sound of incoming calls," and "how to use AI" based on context information.

[0129] According to one embodiment, the second assist area (912) (e.g., the second assist area (312) of FIG. 3) may include recent input queries. The recent input queries may be based on natural language. For example, FIG. 9 illustrates an example where the second assist area (912) includes "how to use AI (artificial intelligence)," "let me answer a call on the tablet," and "in another language," but is not limited thereto. Input queries that a user has used once (e.g., input query (321) of FIG. 3) may tend to be reused. By using the recent input queries in the second assist area (912), the user can conveniently use natural language search without re-entering the same input query into the input window (920) (e.g., input window (320) of FIG. 3).

[0130] According to one embodiment, the third assist area (913) (e.g., the third assist area (313) of FIG. 3) may include preset tags. Certain tags may be assigned to the setting items. The setting items may be grouped based on the tags. For example, FIG. 9 illustrates an example in which the third assist area (913) includes tags such as "in the evening," "make visuals as I please," "pen," "use with one hand," "privacy," and "emergency," but is not limited thereto. When a user selects a specific tag among the tags of the third assist area (913), the setting items having that tag may be displayed as search results (e.g., search results (330) of FIG. 3).

[0131] According to one embodiment, at least one of the first assist area (911), the second assist area (912), and the third assist area (913) may not be displayed. For example, if there is no recent search history, the second assist area (912) may not be displayed. For example, when the assist function is executed by a user and the assist screen (900a) is displayed, the prior assist information (910) may include the first assist area (911) and the third assist area (913). When the input window (920) is selected by the user or when an input query is entered into the input window (920), the second assist area (912) may be displayed in conjunction with the input window (920). For example, an item in the second assist area (912) may be determined in conjunction with the input query entered into the input window (920).

[0132] FIG. 10 is a drawing exemplarily illustrating search results of an assist screen according to one embodiment. Referring to FIG. 10, the assist screen (e.g., the assist screen (300b) of FIG. 3) may include search results (1030) (e.g., the search results (330) of FIG. 3). The search results (1030) may include search result areas by setting group (e.g., a first search result area (1035), a second search result area (1036), and a third search result area (1037)). For example, setting items of the first setting group (10351), the second setting group (10361), and the third setting group (10371) may be displayed respectively in the first search result area (1035), the second search result area (1036), and the third search result area (1037). For example, a settings group may include, but is not limited to, one or more of a connection settings group, a device-to-device connection settings group, a display settings group, a sound and vibration settings group, a notification settings group, a battery and performance settings group, an application settings group, and individual application settings groups.

[0133] Each search result area may include one or more setting items belonging to a corresponding setting group. In the example of FIG. 10, an input query such as "Let me answer calls on the tablet" (e.g., input query (321) of FIG. 3) may be entered. "Calls received more than twice" and "Call" may be searched as setting items of the notification setting group, "Make calls / texts on other devices" may be searched as setting items of the inter-device connection setting group, and "Play vibration sound when a call comes in" may be searched as setting items of the sound and vibration setting group.

[0134] The search result (1030) may include a fourth search result area (1038) representing an application group (10381). The application group (10381) may include applications (e.g., the phone application of FIG. 10). According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1) may search for applications other than setting items using an input query. The electronic device may perform an application search corresponding to the setting item search of one embodiment (e.g., embedding-based search, keyword-based search). The electronic device may create a database regarding the applications of the electronic device and perform text-based comparison and / or embedding-based comparison using an input query to derive applications that match the input query. The derived applications may belong to the application group (10381).

[0135] According to one embodiment, assistance such as Tables 2 through 6 below may be provided in terms of various form factors. Table 2 below may show examples of electronic devices having a foldable display. For example, through natural language utterance, the user can easily find and apply settings that make the most of the tri-fold form factor.

[0136]

[0137]

[0138]

[0139] Table 3 below shows examples of smartphones.

[0140]

[0141] Table 4 below shows examples of tablets.

[0142]

[0143] Table 5 below shows examples of smartwatches.

[0144]

[0145] Table 6 below shows examples of smart televisions.

[0146]

[0147] According to one embodiment, assistance such as Tables 7 to 10 below may be provided in terms of various applications and services. Table 7 below may show examples regarding operating system settings.

[0148]

[0149] Table 8 below shows examples of smart home services.

[0150]

[0151] Table 9 below shows examples of health management applications.

[0152]

[0153] Table 10 below shows examples of multimedia applications.

[0154]

[0155] According to one embodiment, keyword-based search and / or embedding-based search may be performed based on machine translation even if the database (e.g., text-based database (520) of FIG. 5, embedding-based database (530) of FIG. 6, text-based database (820) of FIG. 8) is configured in a language different from the input query. For example, an electronic device may perform a search by translating an input query in a first language into a second language of the database using a language model (e.g., a large language model (LLM)), and then provide the search results in the second language to the user after translating them into the first language using the language model.

[0156] According to one embodiment, assistance such as that shown in Tables 11 to 13 below may be provided in various environments. Table 11 below may show examples regarding metaverse environments.

[0157]

[0158] Table 12 below shows examples of autonomous vehicles.

[0159]

[0160] Table 13 below shows examples of AR (augmented reality) / VR (virtual reality) environments.

[0161]

[0162] According to one embodiment, an electronic device can provide a visual guide regarding search results using a language model (e.g., LLM). For example, when an input query "dazzling" is entered into a language model, the output of the language model as shown in Table 14 below can be obtained.

[0163]

[0164] Since "toggleOn" is "true," the electronic device can determine that it intends for the user to turn on the Comfortable Screen View feature. The electronic device can provide visual guidance appropriate to this context. For example, visual guidance highlighting the toggle button for the Comfortable Screen View feature may be provided.

[0165] For example, if the input query "The text is too small" is input into the language model, output such as Table 15 below can be obtained.

[0166]

[0167] Since "increase" is "true," the electronic device can recognize the user's intention to increase the font size. The electronic device can provide visual guidance appropriate to this context. For example, visual guidance indicating the direction of adjustment can be provided on the slider of the text size adjustment function.

[0168] FIG. 11 is a diagram illustrating, in an exemplary manner, sorting results based on similarity and rank according to one embodiment. Referring to FIG. 11, an assist screen (1100b) (e.g., the assist screen (300b) of FIG. 3, the assist screen (1000b) of FIG. 10) may include a search result (1130) (e.g., the search result (330) of FIG. 3, the search result (1030) of FIG. 10). The search result (1130) may include a first setting group (1135) and a second setting group (1136), and an application group (1137).

[0169] According to one embodiment, an electronic device (e.g., the electronic device (101) of FIG. 1) may perform an embedding-based search and / or a text-based search. An embedding-based search result may be determined based on the embedding-based search. A text-based search result may be determined based on the text-based search. The search result may include a setting item. A setting item of the embedding-based search result may be called a first setting item, and a setting item of the text-based search result may be called a second setting item.

[0170] According to one embodiment, the electronic device can determine the rank of each of the first setting items of the embedding-based search results and the second setting items of the text-based search results based on a ranking condition. The rank may indicate the accuracy of the search results. A small rank value (e.g., [1]) may be assigned to search results having high accuracy. The ranking condition may indicate a predefined condition for assigning a rank to the search results. The electronic device can sort the first setting items and the second setting items in the search results (1130) based on the ranks of the first setting items and the second setting items.

[0171] The ranking condition may include an embedding-based ranking condition for assigning a rank to an embedding-based search result and a text-based ranking condition for assigning a rank to a text-based search result. For example, the embedding-based ranking condition may include a threshold regarding the similarity between an input embedding representation and a set embedding representation. The electronic device may determine a similarity value between an input embedding representation and each set embedding representation and compare each similarity value with a threshold. For example, the similarity value may include, but is not limited to, cosine similarity (e.g., cosine distance). Instead of the similarity value, a normalized value of the similarity value may be used, but is not limited to. For example, a first range, a second range, and a third range may be identified using a first threshold and a second threshold. For example, the first threshold may represent 0.9, and the second threshold may represent 0.4. For example, the first range may represent a similarity range exceeding the top 10%, the third range may represent a similarity range below the bottom 40%, and the second range may represent the remaining similarity range. For example, a rank 4 may be assigned to a setting item in the first range, a rank 5 to a setting item in the second range, and a rank 6 to a setting item in the third range, but is not limited thereto.

[0172] For example, text-based ranking conditions may include a matching state between text keywords and text information (e.g., title). Matching may mean that the texts match. For example, matching states may include, but are not limited to, complete matching, partial matching at the starting position, partial matching at other positions, and similar matching. Complete matching may indicate that the text keyword and the text information are identical. Partial matching at the starting position may indicate that the text keyword partially matches the starting position of the text information. Partial matching at other positions may indicate that the text keyword partially matches a position other than the starting position of the text information. The inclusion of the text keyword in the text information may correspond to partial matching at other positions. Similar matching may indicate that the text keyword and the text information do not match each other, but that one or more synonyms, identical initial consonants, translations, and transliterations are formed between the text keyword and the text information. For example, a rank 1 may be assigned to a complete match, a rank 3 to a partial match of the starting position, a rank 7 to a partial match of another position, and a rank 11 to a similar match, but is not limited thereto.

[0173] In the example of FIG. 11, an embedding-based search based on an input query "call forwarding settings" may be performed. The search results (1130) may include "other call settings", "call settings", "answer and hang up", "text call", and "phone". "Other call settings", "call settings", "answer and hang up", and "text call" may be indicated as a first setting group (1135), and "phone" may be indicated as a second setting group (1136). "Other call settings", "call settings", "answer and hang up", "text call", and "phone" may correspond to the first setting items according to the embedding-based search.

[0174] The electronic device can convert an input query into an input embedding representation and determine the similarity between the input embedding representation and each setting embedding representation by comparing the input embedding representation with each setting embedding representation. In the example of FIG. 11, the similarity between the input embedding representation and the setting embedding representation of "Other Call Settings" may be [0.82484907], the similarity between the input embedding representation and the setting embedding representation of "Call Settings" may be [0.80599385], the similarity between the input embedding representation and the setting embedding representation of "Answer and End Call" may be [0.7874946], the similarity between the input embedding representation and the setting embedding representation of "Text Call" may be [0.7504264], and the similarity between the input embedding representation and the setting embedding representation of "Phone" may be [0.7344289]. Each similarity value may be unnormalized data. Each similarity value in Fig. 11 can be compared with a threshold after normalization. The electronic device can determine the rank of "Other call settings" by applying the threshold of the embedding-based ranking condition to each similarity value [4], the rank of "Call settings" [5], the rank of "Answering and hanging up calls" [5], the rank of "Text calls" [6], and the rank of "Phone" [6].

[0175] The electronic device can sort first setting items and second setting items in each setting group based on the ranks of the first setting items and second setting items. For example, the electronic device can sort "Other call settings," "Call settings," "Answer and hang up calls," and "Text calls" in the first setting group (1135) in rank order. A smaller rank value may be assigned as the search result becomes more accurate. For example, the rank value of [1] may represent the most accurate search result, but is not limited thereto.

[0176] The electronic device can search for applications in addition to setting items. Searched applications (e.g., "Phone") may be displayed as application groups (1137). Searched application items may be assigned a lower rank value (e.g., rank

[0013] ) compared to setting items.

[0177] FIG. 12 is a diagram exemplarily illustrating the alignment results of embedding-based search results and text-based search results of each setting group according to one embodiment. Referring to FIG. 12, an assist screen (1200b) (e.g., the assist screen (300b) of FIG. 3, the assist screen (1000b) of FIG. 10, the assist screen (1100b)) may include a search result (1230) (e.g., the search result (330) of FIG. 3, the search result (1030) of FIG. 10, the search result (1130) of FIG. 11). The search result (1230) may include first setting items (12352, 12353) which are embedding-based search results and second setting items (12351, 12361) which are text-based search results.

[0178] An electronic device (e.g., the electronic device (101) of FIG. 1) can determine the ranks of first setting items (12352, 12353) based on embedding-based ranking conditions and determine the ranks of second setting items (12351, 12361) based on text-based ranking conditions. The electronic device can sort the first setting items (12352, 12353) and the second setting items (12351, 12361) based on the ranks of the first setting items (12352, 12353) and the second setting items (12351, 12361). The electronic device can sort the first setting items (12352, 12353) and the second setting items (12351, 12361) in order of having the highest rank.

[0179] According to one embodiment, the assist screen (1200b) may include search result areas (e.g., first search result area (1237), second search result area (1238)) for each setting group (e.g., first setting group (1235), second setting group (1236)). The electronic device may classify the first setting items (12352, 12353) and the second setting items (12351, 12361) by setting group and display them in the search result areas. The electronic device may display the first setting items (12352, 12353) and the second setting items (12351, 12361) within the search result area to which each of the first setting items (12352, 12353) and the second setting items (12351, 12361) belongs among the search result areas, such as the first setting items (12352, 12353) and can be sorted based on the ranks of the second setting items (12351, 12361).

[0180] In the example of FIG. 12, the first setting items (12352, 12353) and the second setting item (12351) may belong to the first setting group (1235), and the second setting item (12361) may belong to the second setting group (1236). The electronic device may display the first setting items (12352, 12353) and the second setting item (12351) in the first search result area (1237) and display the second setting item (12361) in the second search result area (1238). The electronic device may sort the first setting items (12352, 12353) and the second setting item (12351) within the first search result area (1237) based on the ranks of the first setting items (12352, 12353) and the second setting item (12351). For example, if the ranks of the second setting item (12351), the first setting item (12352), and the second setting item (12353) are rank 1, rank 4, and rank 5, respectively, the electronic device can display the second setting item (12351), the first setting item (12352), and the second setting item (12353) in the order of the first search result area (1237).

[0181] For example, Table 16 below shows embedding-based search results based on an input query "phone on tablet".

[0182]

[0183] Table 17 below shows text-based search results based on the input query "phone on tablet".

[0184]

[0185] Table 18 below shows the ranks of the embedding-based search results in Table 16.

[0186]

[0187] Table 19 below shows the ranks of the text-based search results in Table 17.

[0188]

[0189] Table 5 below may show sorting results based on Tables 16 through 19. Search results (1230) may be displayed based on Table 20.

[0190]

[0191] FIG. 13 is a flowchart exemplarily illustrating an operation for determining the rank of each search result according to one embodiment. The rank values ​​shown in FIG. 13 (e.g., rank 1, rank 3, rank 4, rank 5, rank 6, rank 7, rank 11, rank 13) are examples and are not limited thereto. Referring to FIG. 13, in operation (1310), an electronic device (e.g., the electronic device (101) of FIG. 1) can determine whether a ranking target (e.g., a setting item, an application item) is an embedding-based search result. If the ranking target is not an embedding-based search result, in operations (1320 to 1340), the electronic device can determine the rank of the ranking target based on text-based ranking conditions. If the ranking target is an embedding-based search result, in operations (1370, 1380), the electronic device can determine the rank of the ranking target based on the embedding-based ranking condition.

[0192] Text-based ranking conditions may include a matching state between text keywords and text information (e.g., title) of a ranking target. A match may mean that the text matches. For example, a matching state may include, but is not limited to, an exact match, a partial match at the starting position, a partial match at different positions, and a similar match.

[0193] In operation (1320), the electronic device can determine whether the text keyword and text information correspond to a perfect match. A perfect match may indicate that the text keyword and text information are identical. If the text keyword and text information correspond to a perfect match, the electronic device can determine the rank of the ranking target to be 1.

[0194] In operation (1330), the electronic device can determine whether the text keyword and text information correspond to a partial match of the starting position. A partial match of the starting position may indicate that the text keyword partially matches the starting position of the text information. If the text keyword and text information correspond to a partial match of the starting position, the electronic device can determine the rank of the ranking target to be 3.

[0195] In operation (1340), the electronic device can determine whether the text keyword and text information correspond to a partial match at a different location. A partial match at a different location may indicate that the text keyword is partially matched at a location other than the starting location of the text information. The inclusion of the text keyword in the text information may correspond to a partial match at a different location. If the text keyword and text information correspond to a partial match at a different location, the electronic device can determine the rank of the ranking target to be 7.

[0196] In operation (1350), the electronic device can determine whether the ranking target belongs to an application group. If the ranking target is an application item, the ranking target may belong to an application group. If the ranking target belongs to an application group, the electronic device can determine the rank of the ranking target to be 13.

[0197] If the ranking target does not belong to an application group, the text keyword and text information may correspond to a similar match. A similar match indicates that the text keyword and text information do not match each other, but that one or more synonyms, identical initial consonants, translations, and transliterations are formed between the text keyword and text information. If the ranking target does not belong to an application group, the electronic device may determine the rank of the ranking target to be 11.

[0198] The embedding-based ranking condition may include a threshold regarding the similarity between the input embedding representation and the set embedding representations. The electronic device may determine a similarity value between the input embedding representation and the set embedding representation of the ranking target and compare the similarity value with the threshold. For example, the similarity value may include, but is not limited to, cosine similarity (e.g., cosine distance). In operation (1360), the electronic device may normalize the similarity value and use the normalized value of the similarity value instead of the similarity value to determine the rank of the ranking target.

[0199] For example, MinMaxScaling may be used for normalization, but is not limited thereto. The electronic device may determine similarity values ​​by comparing the input embedding representation with the setting embedding representations of the ranking targets (e.g., setting items). The electronic device may normalize the similarity values ​​of each ranking target using the similarity values. For example, the similarity values ​​may be normalized such that the maximum value among the similarity values ​​becomes the first value (e.g., 1) and the minimum value becomes the second value (e.g., 0).

[0200] For example, the threshold for similarity may include a first threshold and a second threshold. The electronic device may identify a first range, a second range, and a third range using the first threshold and the second threshold. For example, the first threshold may represent 0.9 and the second threshold may represent 0.4. For example, the first range may represent a similarity range exceeding the top 10%, the third range may represent a similarity range below the bottom 40%, and the second range may represent the remaining similarity range.

[0201] In operation (1370), the electronic device can determine whether the ranking target falls within the first range. If the ranking target falls within the first range, the electronic device can determine the rank of the ranking target to be 4. If the ranking target does not fall within the first range, in operation (1380), the electronic device can determine whether the ranking target falls within the second range. If the ranking target falls within the second range, the electronic device can determine the rank of the ranking target to be 5. If the ranking target does not fall within the second range, the electronic device can determine the rank of the ranking target to be 6. That the ranking target does not fall within the second range may mean that the ranking target falls within the third range.

[0202] FIG. 14 is a diagram illustrating an exemplary weighting operation based on the linguistic characteristics of an input query according to one embodiment. Referring to FIG. 14, an electronic device (e.g., the electronic device (101) of FIG. 1) can analyze the linguistic characteristics (14211) of an input query (1421) (e.g., the input query (321) of FIG. 3). The linguistic characteristics (14211) can indicate whether the input query (1421) is close to natural language or close to keywords.

[0203] The electronic device can determine the linguistic characteristics (14211) of the input query (1421) based on the number of characters of the input query (1421) and one or more natural language scores. The electronic device can determine that the input query (1421) is closer to natural language as the number of characters of the input query (1421) increases. For example, the electronic device can determine that the input query (1421) is closer to natural language if the number of characters of the input query (1421) is greater than a threshold. The electronic device can determine the natural language score based on whether the input query (1421) is closer to a sentence or closer to a word. For example, the electronic device can determine the natural language score of the input query (1421) based on the number of particles of the input query (1421) and one or more sentence forms. The electronic device can determine that the input query (1421) is closer to natural language if the natural language score of the input query (1421) is greater than a threshold.

[0204] The electronic device may assign a higher weight to either the embedding-based search result (1450) or the text-based search result (1460) based on the linguistic characteristics (14211) of the input query (1421). If the input query (1421) is close to natural language, the electronic device may assign a higher weight to the embedding-based search result (1450). If the input query (1421) is close to keywords, the electronic device may assign a higher weight to the text-based search result (1460). For example, the electronic device may apply a first weight (w1) to the embedding-based search result (1450) and a second weight (w2) to the text-based search result (1460). If the input query (1421) is close to natural language, the electronic device may set the first weight (w1) higher than the second weight (w2). If the input query (1421) is close to a keyword, the electronic device can set the second weight (w2) higher than the first weight (w1).

[0205] FIG. 15 is a diagram illustrating, in an exemplary manner, a language model that outputs the priority of search results according to one embodiment. Referring to FIG. 15, an electronic device may execute a neural network-based language model (1500) based on an input query (1521) (e.g., input query (321) of FIG. 3, input query (1421) of FIG. 14), a search result (1525) (e.g., search result (330) of FIG. 3, search result (1030) of FIG. 10, search result (1130) of FIG. 11, search result (1230) of FIG. 12), and context information (1527) to generate priority information (1501) regarding the search result (1525). For example, the language model (1500) may be a generative language model or a large language model (LLM), but is not limited thereto. LLMs can include relatively more parameters (e.g., over 10 billion) than existing general language models. LLMs can use Transformer AI neural network structures based on an attention mechanism.

[0206] According to one embodiment, the training of the LLM may include pre-training and / or fine-tuning. Pre-training may include a process of training the LLM to acquire general language knowledge using a large amount of text data. For example, pre-training may include self-supervised learning that predicts the next word using the previous word sequence of a text sequence. Fine-tuning may include a process of training the LLM to be suitable for a specific domain (e.g., chatbot, AI assistant, translation, summary generation, question answering) and / or task. Fine-tuning may include a process of further training the LLM (e.g., supervised learning, adaptive learning) using a dataset corresponding to the specific domain and / or task based on the pre-trained model. The LLM may perform tasks based on text input containing natural language referred to as a prompt.

[0207] According to one embodiment, fine-tuning may be omitted during the training of the LLM. To improve performance for a desired task, the user can control the prompts input to the LLM. For example, the user can control the prompts to additionally provide examples of the task and / or guidance for performing the task, such as in-context learning, zero-shot learning, and / or few-shot learning. Examples of publicly available LLMs include BERT (Bidirectional Encoder Representations from Transformer) and GPT (generative pre-trained transformer).

[0208] The term 'LLM' may refer to the language neural network model itself, but it may also refer to models of LLM-based applications (e.g., chatbots, AI assistants, translation, summary generation, text classification, sentence generation). For example, an LLM-based chatbot or LLM-based translator such as ChatGPT may also be referred to as 'LLM'.

[0209] 'LLM' may include an inference engine utilizing an LLM neural network model. For example, 'inputting an input prompt into the LLM' may mean 'inputting an input prompt into an LLM-based inference engine.' For instance, 'the output of the LLM for the input prompt' may refer to the output information of the last neural network layer of the LLM obtained when the input prompt is input into the LLM-based inference engine, and / or output information modified through additional processing.

[0210] The search result (1525) may include one or more of an embedding-based search result and a text-based search result. The embedding-based search result may include one or more first setting items, and the text-based search result may include one or more second setting items.

[0211] Context information may include, but is not limited to, one or more of the following: device connection status (e.g., connected to multiple devices via Bluetooth), running application (e.g., music application), current time (e.g., evening), current day of the week (e.g., weekday, weekend), current date (e.g., public holiday), current illumination (e.g., low light), current location (e.g., country where the electronic device is located), ambient noise, storage space status (e.g., low storage space), communication status (e.g., message reception), battery status (e.g., low battery), and user's vital signs (e.g., heart rate, sleep status, step count). For example, if the input query (1521) includes "sound," the search result (1525) includes sound-related setting items for "device-to-device connection" and sound-related setting items for "sound and vibration," and the context information (1527) indicates a state where multiple devices are connected via Bluetooth, the language model (1500) can generate priority information (1501) such that the sound-related setting item for "device-to-device connection" has a higher priority than the sound-related setting item for "sound and vibration."

[0212] Priority information (1501) may indicate the priority of one or more of the first setting items and second setting items of the search result (1525). Based on the priority information (1501), the electronic device may display the search result (1525) on an assist screen (e.g., the assist screen (300b) of FIG. 3, the assist screen (1000b) of FIG. 10, the assist screen (1100b) of FIG. 11, and the assist screen (1200b) of FIG. 12). For example, the electronic device may highlight one or more of the setting items of the search result (1525) that have a high priority on the assist screen. For example, the electronic device may highlight a predetermined number of setting items in order of priority based on the priority information (1501).

[0213] The electronic device according to one embodiment disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone, a tablet computer), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiment of this document is not limited to the aforementioned devices.

[0214] One embodiment of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as "coupled" or "connected" to another (e.g., 2nd) component, with or without the terms "functionally" or "communicationly," it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0215] The term "module" as used in an embodiment of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to an embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0216] One embodiment of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0217] According to one embodiment, the method according to one embodiment disclosed herein may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0218] According to one embodiment, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to one embodiment, one or more of the components or operations among the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to one embodiment, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0219] The embodiments described above may be implemented as hardware components, software components, and / or combinations of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are also possible.

[0220] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, or computer storage medium or device so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on computer-readable recording media.

[0221] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination, and the program instructions recorded on the medium may be those specifically designed and configured for the embodiment or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0222] The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

Claims

1. In an electronic device, display; At least one processor including a processing circuit; and The memory includes one or more storage media for storing instructions, and When the above instructions are executed by the at least one processor, the electronic device, Receive natural language-based input queries from the user, and Encode the above input query to generate an input embedding representation, and The above input embedding expression is compared with setting embedding expressions based on text information of setting items that configure the electronic device, and an embedding-based search result related to the input query is determined from the setting items. Displaying the above embedding-based search results on the assist screen of the above display Electronic device.

2. In Paragraph 1, When the above instructions are executed by the at least one processor, the electronic device, Determine text keywords based on the above input query, and Compare the above text keyword with the above text information of the above setting items to determine text-based search results related to the above input query from the above setting items, and Displaying the above text-based search results on the above assist screen, Electronic device.

3. In Paragraph 2, When the above instructions are executed by the at least one processor, the electronic device, Based on embedding-based ranking conditions and text-based ranking conditions, the ranks of the first setting items of the embedding-based search results and the second setting items of the text-based search results are determined, and Sort the first setting items and the second setting items based on the above ranks to display the first setting items and the second setting items on the assist screen. Electronic device.

4. In Paragraph 3, The above embedding-based ranking condition is It includes a threshold for the similarity between the above-mentioned input embedding representation and the above-mentioned set embedding representations, and The above text-based ranking conditions are including the matching state between the above text keyword and the above text information, Electronic device.

5. In Paragraph 3 or 4, The above assist screen is Includes search result areas by setting group, When the above instructions are executed by the at least one processor, the electronic device, The first setting items and the second setting items are sorted based on the ranks within the search result area to which each of the first setting items and the second setting items belongs among the search result areas. Electronic device.

6. In any one of paragraphs 2 through 5, When the above instructions are executed by the at least one processor, the electronic device, A neural network-based language model is executed based on the above input query, one or more of the above embedding-based search results and the above text-based search results, and context information of the above electronic device to generate one or more priority information of the above embedding-based search results and the above text-based search results, and Displaying one or more of the embedding-based search results and text-based search results on the assist screen based on the above priority information, Electronic device.

7. In any one of paragraphs 2 through 6, When the above instructions are executed by the at least one processor, the electronic device, Based on the linguistic characteristics of the above input query, weights are assigned to one or more of the above embedding-based search results and the above text-based search results. Electronic device.

8. In any one of paragraphs 1 through 7, The above assist screen is One or more of a first assist area including a recommendation query based on context information of the electronic device, a second assist area including a recent input query, and a third assist area including a preset tag. Electronic device.

9. In any one of paragraphs 1 through 8, When the above instructions are executed by the at least one processor, the electronic device, Collecting the above setting items of the above electronic device, Determining the text information of the above setting items, and Generating the above-mentioned setting embedding expressions based on the above-mentioned text information of the above-mentioned setting items, Electronic device.

10. In any one of paragraphs 1 through 9, When the above instructions are executed by the at least one processor, the electronic device, One or more of the above setting items are updated periodically or non-periodically, and Based on one or more of the above-mentioned updated setting items, one or more corresponding setting embedding expressions of the above-mentioned setting embedding expressions are updated periodically or non-periodically, and Determining the embedding-based search result based on one or more of the above-mentioned updated corresponding configuration embedding representations, Electronic device.

11. In any one of paragraphs 1 through 10, The text information of the above setting items Includes one or more of the identifier, key, authority, title, and description of each of the above setting items, The above configuration embedding representations are Generated based on the description of the text information for each of the above setting items, Electronic device.

12. In any one of paragraphs 1 through 11, The above input embedding representations and the above setting embedding representations are Numerical values ​​determined by mapping the above input query and the above setting items into a feature space, and The above numerical values ​​are Dependent on the similarity between the above input query and the above setting items, Electronic device.

13. In electronic devices, display; At least one processor including a processing circuit; and The memory includes one or more storage media for storing instructions, and When the above instructions are executed by the at least one processor, the electronic device, Receive natural language-based input queries from the user, and Encode the above input query to generate an input embedding representation, and The above input embedding expression is compared with setting embedding expressions based on text information of setting items that configure the electronic device, and an embedding-based search result related to the input query is determined from the setting items. Determine text keywords based on the above input query, and Compare the above text keyword with the above text information of the above setting items to determine text-based search results related to the above input query from the above setting items, and Displaying the above embedding-based search results and the above text-based search results on the assist screen of the above display Electronic device.

14. In Paragraph 13, When the above instructions are executed by the at least one processor, the electronic device, Based on embedding-based ranking conditions and text-based ranking conditions, the ranks of the first setting items of the embedding-based search results and the second setting items of the text-based search results are determined, and Sort the first setting items and the second setting items based on the above ranks to display the first setting items and the second setting items on the assist screen. Electronic device.

15. In Paragraph 14, The above embedding-based ranking condition is It includes a threshold for the similarity between the above-mentioned input embedding representation and the above-mentioned set embedding representations, and The above text-based ranking conditions are including the matching state between the above text keyword and the above text information, Electronic device.

Images