Electronic devices and search methods

Abstract

A method for operating an electronic device according to one embodiment may comprise an operation of receiving a search word from a user. The operation method may comprise an operation of estimating a keyboard layout of a keyboard utilized for inputting the search word. The operation method may comprise an operation of selecting an index word in which a distance to the search word is less than a threshold value, on the basis of the keyboard layout. The operation method may include an operation of obtaining a search result corresponding to each of the index word and the search word. The operation method may include an operation of returning the search result in response to the search word.

Description

Electronic devices and search methods

[0001] Embodiments of the present invention relate to an electronic device and a search method.

[0002] Typo-correction search utilizes an automatic correction mechanism to return correct search results when a user enters a search term containing a typo or spelling error. Typo-correction search may be a technology required by search engines to improve the user experience and enhance the accuracy of search results.

[0003] Typo-correction search technologies can utilize pre-built dictionaries or language models to provide correct word or correction suggestions. Typo-correction search technologies can utilize correction mechanisms based on the edit distance (e.g., Levenshtein distance) between the input search term and the candidate word.

[0004] The above information may be provided as background art to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above-described matters constitute prior art related to the present disclosure.

[0005] An operating method of an electronic device according to one embodiment may include an operation of receiving a search word from a user. The operating method may include an operation of estimating a keyboard layout of a keyboard used to input the search word. The operating method may include an operation of selecting an index word whose proximity to the search word is less than a threshold value based on the keyboard layout. The operating method may include an operation of obtaining search results corresponding to each of the index word and the search word. The operating method may include an operation of returning the search results corresponding to the search word.

[0006] An electronic device according to one embodiment may include one or more processors. The electronic device may include a memory storing instructions. The instructions, when individually or collectively executed by the one or more processors, may cause the electronic device to receive a search term from a user. The instructions, when individually or collectively executed by the one or more processors, may cause the electronic device to estimate a keyboard layout of a keyboard used to input the search term. The instructions, when individually or collectively executed by the one or more processors, may cause the electronic device to select an index term whose proximity to the search term is less than a threshold value based on the keyboard layout. The instructions, when individually or collectively executed by the one or more processors, may cause the electronic device to obtain search results corresponding to each of the index term and the search term. The above instructions, when individually or collectively executed by the one or more processors, may cause the electronic device to return the search results in response to the search term.

[0007] FIG. 1 is a block diagram of an electronic device within a network environment, according to one embodiment.

[0008] FIGS. 2 and 3 are schematic block diagrams of an electronic device according to one embodiment.

[0009] FIGS. 4A and 4B are drawings for explaining a type of keyboard according to one embodiment.

[0010] FIGS. 5A to 5D are drawings for explaining a keyboard layout according to one embodiment.

[0011] FIG. 6 is a diagram for explaining a keyboard layout estimation method according to one embodiment.

[0012] FIGS. 7A and 7B are diagrams for explaining a keyboard layout estimation method according to one embodiment.

[0013] FIG. 8 is a diagram illustrating an index dictionary according to one embodiment.

[0014] FIGS. 9A to 9C are diagrams for explaining an adjacent distance calculation operation according to one embodiment.

[0015] FIG. 10 is an example of search results returned by an electronic device according to one embodiment.

[0016] FIG. 11 illustrates a flowchart of a method of operating an electronic device according to one embodiment.

[0017] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are assigned the same reference numerals regardless of the drawing numbers, and redundant descriptions thereof will be omitted.

[0018]

[0019] FIG. 1 is a block diagram of an electronic device (101) within a network environment (100), according to one embodiment.

[0020] Referring to FIG. 1, in a network environment (100), an electronic device (101) may communicate with an electronic device (102) via a first network (198) (e.g., a short-range wireless communication network), or may communicate with at least one of an electronic device (104) or a server (108) via a network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) via the server (108). According to one embodiment, the electronic device (101) may include a processor (120), a memory (130), an input module (150), an audio output module (155), a display module (160), an audio module (170), a sensor module (176), an interface (177), a connection terminal (178), a haptic module (179), a camera module (180), a power management module (188), a battery (189), a communication module (190), a subscriber identification module (196), or an antenna module (197). In some embodiments, the electronic device (101) may omit at least one of these components (e.g., the connection terminal (178)), or may have one or more other components added. In some embodiments, some of these components (e.g., the sensor module (176), the camera module (180), or the antenna module (197)) may be integrated into one component (e.g., the display module (160)).

[0021] The processor (120) may, for example, execute software (e.g., a program (140)) to control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) and perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the processor (120) may store commands or data received from other components (e.g., a sensor module (176) or a communication module (190)) in a volatile memory (132), process the commands or data stored in the volatile memory (132), and store result data in a non-volatile memory (134).

[0022] According to one embodiment, the processor (120) may be implemented as a circuit (e.g., a processing circuit) such as a system on chip (SoC) or an integrated circuit (IC). The processor (120) may include one or more processors. For example, the processor (120) may include a combination of one or more processors such as a CPU, a GPU, an MPU, an AP, and a CP.

[0023] According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently or together with the main processor (121). For example, when the electronic device (101) includes the main processor (121) and the auxiliary processor (123), the auxiliary processor (123) may be configured to use less power than the main processor (121) or to be specialized for a given function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as a part thereof.

[0024] The auxiliary processor (123) may control at least a portion of functions or states associated with at least one component (e.g., a display module (160), a sensor module (176), or a communication module (190)) of the electronic device (101), for example, on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (123) (e.g., an image signal processor or a communication processor) may be implemented as a part of another functionally related component (e.g., a camera module (180) or a communication module (190)). In one embodiment, the auxiliary processor (123) (e.g., a neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning. This learning can be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or can be performed through a separate server (e.g., server (108)). The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include multiple artificial neural network layers.The artificial neural network may be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to, or alternatively to, a hardware structure, an artificial intelligence model may include a software structure.

[0025] The memory (130) can store various data used by at least one component (e.g., processor (120) or sensor module (176)) of the electronic device (101). The data can include, for example, software (e.g., program (140)) and input data or output data for commands related thereto.

[0026] According to one embodiment, the memory (130) may include one or more memories. The instructions stored in the memory (130) may be stored in one memory. The instructions stored in the memory (130) may be divided and stored in multiple memories. The instructions stored in the memory (130) may be individually or collectively executed by the processor (120) to cause the electronic device (101) (e.g., the electronic device (201) of FIG. 2) to perform and / or control the search method described with reference to FIGS. 2 to 11. The instructions stored in the memory (130) may be individually or collectively executed by multiple processors to cause the electronic device (101) (e.g., the electronic device (201) of FIG. 2) to perform and / or control the search method described with reference to FIGS. 2 to 11. According to one embodiment, the memory (130) may include a volatile memory (132) or a nonvolatile memory (134).

[0027] The program (140) may be stored as software in the memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0028] The input module (150) can receive commands or data to be used in a component of the electronic device (101) (e.g., a processor (120)) from an external source (e.g., a user) of the electronic device (101). The input module (150) can include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0029] The audio output module (155) can output audio signals to the outside of the electronic device (101). The audio output module (155) can include, for example, a speaker or a receiver. The speaker can be used for general purposes, such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. In one embodiment, the receiver can be implemented separately from the speaker or as part of the speaker.

[0030] The display module (160) can visually provide information to an external party (e.g., a user) of the electronic device (101). The display module (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display module (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0031] The audio module (170) can convert sound into an electrical signal, or vice versa, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150), output sound through the sound output module (155), or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphone) directly or wirelessly connected to the electronic device (101).

[0032] The sensor module (176) can detect the operating status (e.g., power or temperature) of the electronic device (101) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. According to one embodiment, the sensor module (176) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0033] The interface (177) may support one or more designated protocols that may be used to directly or wirelessly connect the electronic device (101) with an external electronic device (e.g., the electronic device (102)). In one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0034] The connection terminal (178) may include a connector through which the electronic device (101) may be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0035] A haptic module (179) can convert electrical signals into mechanical stimuli (e.g., vibration or movement) or electrical stimuli that a user can perceive through tactile or kinesthetic sensations. In one embodiment, the haptic module (179) can include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

[0036] The camera module (180) can capture still images and videos. According to one embodiment, the camera module (180) may include one or more lenses, image sensors, image signal processors, or flashes.

[0037] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least a part of a power management integrated circuit (PMIC).

[0038] A battery (189) may power at least one component of the electronic device (101). In one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0039] The communication module (190) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may operate independently from the processor (120) (e.g., application processor) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (194) (e.g., a local area network (LAN) communication module, or a power line communication module). Among these communication modules, the corresponding communication module can communicate with an external electronic device (104) via a first network (198) (e.g., a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network (199) (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules can be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can verify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) by using subscriber information (e.g., an international mobile subscriber identity (IMSI)) stored in the subscriber identification module (196).

[0040] The wireless communication module (192) can support 5G networks and next-generation communication technologies following the 4G network, such as NR access technology (new radio access technology). The NR access technology can support high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and connection of multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency communications)). The wireless communication module (192) can support, for example, a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate. The wireless communication module (192) can support various technologies for securing performance in a high-frequency band, such as beamforming, massive multiple-input and multiple-output (MIMO), full dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), an external electronic device (e.g., the electronic device (104)), or a network system (e.g., the second network (199)). According to one embodiment, the wireless communication module (192) can support a peak data rate (e.g., 20 Gbps or more) for eMBB realization, a loss coverage (e.g., 164 dB or less) for mMTC realization, or a U-plane latency (e.g., 0.5 ms or less for downlink (DL) and uplink (UL), or 1 ms or less for round trip) for URLLC realization.

[0041] The antenna module (197) can transmit or receive signals or power to or from an external device (e.g., an external electronic device). In one embodiment, the antenna module (197) may include an antenna including a radiator formed of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). In one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as the first network (198) or the second network (199), may be selected from the plurality of antennas by, for example, the communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. In some embodiments, in addition to the radiator, another component (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as a part of the antenna module (197).

[0042] In one embodiment, the antenna module (197) may form a mmWave antenna module. In one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent a first side (e.g., a bottom side) of the printed circuit board and capable of supporting a designated high-frequency band (e.g., a mmWave band), and a plurality of antennas (e.g., an array antenna) disposed on or adjacent a second side (e.g., a top side or a side side) of the printed circuit board and capable of transmitting or receiving signals in the designated high-frequency band.

[0043] At least some of the above components can be interconnected and exchange signals (e.g., commands or data) with each other via a communication method between peripheral devices (e.g., a bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)).

[0044] According to one embodiment, commands or data may be transmitted or received between the electronic device (101) and an external electronic device (104) via a server (108) connected to a second network (199). Each of the external electronic devices (102 or 104) may be the same or a different type of device as the electronic device (101). According to one embodiment, all or part of the operations executed in the electronic device (101) may be executed in one or more of the external electronic devices (102, 104, or 108). For example, when the electronic device (101) is to perform a certain function or service automatically or in response to a request from a user or another device, the electronic device (101) may, instead of or in addition to executing the function or service itself, request one or more external electronic devices to perform the function or at least a part of the service. One or more external electronic devices that receive the request may execute at least a portion of the requested function or service, or an additional function or service related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may process the result as is or additionally and provide it as at least a portion of a response to the request. For this purpose, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used, for example. The electronic device (101) may provide an ultra-low latency service by using distributed computing or mobile edge computing, for example. In another embodiment, the external electronic device (104) may include an Internet of Things (IoT) device. The server (108) may be an intelligent server utilizing machine learning and / or a neural network. According to one embodiment, the external electronic device (104) or the server (108) may be included in the second network (199).The electronic device (101) can be applied to intelligent services (e.g., smart home, smart city, smart car, or healthcare) based on 5G communication technology and IoT-related technology.

[0045]

[0046] FIGS. 2 and 3 are schematic block diagrams of an electronic device according to one embodiment.

[0047] According to one embodiment, the electronic device (201) may support a keyboard fuzzy function. The keyboard fuzzy function may be designed to prevent a user from accidentally pressing adjacent keys. The keyboard fuzzy function may be designed considering the keyboard type.

[0048] In one embodiment, the electronic device (201) may be an extension of the keyboard fuzzy function that was limited to the QWERTY keyboard. The electronic device (201) may estimate the layout of the keyboard. Based on the estimated layout, the electronic device (201) may select an index word (e.g., one of the index words processed for search) whose proximity to the search word (e.g., a user input) is less than a threshold value. The electronic device (201) may utilize the selected index word to return search results corresponding to the user input.

[0049] Depending on the keyboard type (e.g., QWERTY, QWERTZ, Azerty, 3*4, Cheonjiin), the physical arrangement of the keys on the keyboard may vary. Various keyboard types exist worldwide, and keyboard types may be designed to accommodate the languages ​​and characters used in specific countries or regions. Keyboard type can aid typo-correction technology in understanding user input.

[0050] In one embodiment, the keyboard layout may have different values ​​depending on the keyboard type. The keyboard layout may include a vector corresponding to the keys that make up the keyboard. The vector may include information about the physical location of the keys within the keyboard and / or attribute information based on the type of keyboard.

[0051] In one embodiment, the proximity distance may be the sum of one or more L1 distances. The L1 distance may refer to the Manhattan distance. The L1 distance may be calculated on a per-character basis. The electronic device (201) may calculate the L1 distance between the characters constituting the search term and the characters constituting the index term based on the vectors included in the keyboard layout. The electronic device (201) may calculate the proximity distance between the search term and the index term by collecting the L1 distances.

[0052] According to one embodiment, the electronic device (201) may be implemented as at least one of a smartphone, a tablet personal computer, a mobile phone, a speaker (e.g., an AI speaker), a video phone, an e-book reader, a desktop personal computer, a laptop personal computer, a netbook computer, a workstation, a server, a personal digital assistant (PDA), a portable multimedia player (PMP), an MP3 player, a mobile medical device, a camera, or a wearable device.

[0053] Referring to FIG. 2, according to one embodiment, an electronic device (201) may include an input module (210) (e.g., input module (150) of FIG. 1). The electronic device (201) may include one or more processors (220) (e.g., processor (120) of FIG. 1). The electronic device (201) may include a memory (230) (e.g., memory (130) of FIG. 1).

[0054] According to one embodiment, the input module (210) may receive commands or data to be used in a component (e.g., processor (120)) of the electronic device (201) from an external source (e.g., a user) of the electronic device (201). The input module (210) may include a keyboard. The input module (210) may include a hardware keyboard and / or a software keyboard. The processor (220) (e.g., an application processor) may access the memory (230) and execute one or more instructions. The processor (220) may execute a search engine (221). The memory (230) may store data used by at least one component (e.g., processor (220)) of the electronic device (201). The memory (230) may store a dictionary (231).

[0055] According to one embodiment, the processor (220) may be implemented as a circuit (e.g., a processing circuit) such as a system on chip (SoC) or an integrated circuit (IC). The processor (220) may include one or more processors. For example, the processor (220) may include a combination of one or more processors such as a CPU, a GPU, an MPU, an AP, and a CP.

[0056] According to one embodiment, the memory (230) may include one or more memories. The instructions stored in the memory (230) may be stored in a single memory. The instructions stored in the memory (230) may be divided and stored in a plurality of memories. The instructions stored in the memory (230) may be individually or collectively executed by the processor (220) to cause the electronic device (201) to perform and / or control the search method described with reference to FIGS. 2 to 11. The instructions stored in the memory (230) may be individually or collectively executed by a plurality of processors to cause the electronic device (201) to perform and / or control the search method described with reference to FIGS. 2 to 11.

[0057] In one embodiment, the search engine (221) may receive a search term from a user and return search results. The search engine (221) may return an application running within the electronic device (201) as a search result. However, the search results returned by the search engine (221) are not limited to applications. The search engine (221) may also return documents and / or images in response to the search term.

[0058] According to one embodiment, the search engine (221) can correct search terms received from a user through a search term correction module (222). The search terms processed through the search term correction module (222) can be index terms (e.g., components of an index term dictionary (231) including index terms processed for search). The search engine (221) can perform a search based on the search terms received from the user or the corrected search terms (e.g., index terms) through the search module (224). The search engine (221) can update (and / or create) an index term dictionary (231) through the index module (223).

[0059] Referring to FIG. 3, according to one embodiment, a search term correction module (300) (e.g., search term correction module (222) of FIG. 2) may include a first search term correction module (310) and a second search term correction module (320). The first search term correction module (310) may be for supporting a keyboard fuzzy function. The first search term correction module (310) may estimate a keyboard layout (e.g., see FIGS. 5A to 5D). The second search term correction module (320) may be for supporting other search term correction functions (e.g., a synonym correction function, a morpheme-based correction function, a translation result correction function, a phoneme separation function, an initial consonant-based correction function, an edit distance-based correction function, and a similar pronunciation correction function).

[0060] According to one embodiment, the keyboard image analysis module (311) can analyze a keyboard image. The keyboard image analysis module (311) may be performed when the electronic device (201) has screen capture authority. The keyboard image analysis module (311) can capture the screen displayed by the electronic device (201). The keyboard image analysis module (311) can obtain a keyboard layout from a keyboard image included in the screen based on a first neural network model. The operation of the keyboard image analysis module (311) will be described in detail with reference to FIG. 6.

[0061] According to one embodiment, the keyboard input analysis module (312) can analyze input (e.g., text) entered through the keyboard. The keyboard input analysis module (312) may be performed when the electronic device (201) does not have screen capture permission. The keyboard input analysis module (312) can obtain the type of the keyboard from at least one of a first sequence (e.g., inputs to the keyboard, classified in chronological order) (e.g., see 701 of FIG. 7A) or a second sequence (e.g., search words that change according to inputs to the keyboard, classified in chronological order) (e.g., see 702 of FIG. 7A) associated with a search word based on a second neural network model. The keyboard input analysis module (312) can obtain a keyboard layout based on the type of the keyboard. The operation of the keyboard input analysis module (312) will be described in detail with reference to FIGS. 7A and 7B.

[0062] According to one embodiment, the keyboard layout management module (313) can manage the keyboard layout output by the keyboard image analysis module (311) and / or the keyboard input analysis module (312).

[0063] According to one embodiment, the index term determination module (314) may select an index term whose proximity distance to a search term (e.g., user input) is less than a threshold based on the keyboard layout. The keyboard layout may include a vector corresponding to keys constituting the keyboard. The vector may include information about the physical location of the keys within the keyboard and / or attribute information based on the type of the keyboard. The proximity distance may be the sum of one or more L1 distances. The L1 distance may refer to the manhattan distance. The L1 distance may be calculated for each character. The electronic device (201) may calculate the L1 distance between characters constituting the search term and characters constituting the index term based on the vector included in the keyboard layout. The electronic device (201) may calculate the proximity distance between the search term and the index term by collecting the L1 distances.

[0064] According to one embodiment, the operation of the search term correction module (300) may be based on an instant search method in which search results are updated for each input on the keyboard. The operation of the search term correction module (300) may be triggered when the number of characters constituting the search term is greater than a threshold. The operation of the search term correction module (300) may be triggered when no search results corresponding to the search term exist. The operation of the search term correction module (300) may be performed as a fallback mechanism according to the search policy of the electronic device (201).

[0065]

[0066] FIGS. 4A and 4B are drawings for explaining a type of keyboard according to one embodiment, and FIGS. 5A to 5D are drawings for explaining a keyboard layout according to one embodiment.

[0067] Referring to FIG. 4A, according to one embodiment, various English keyboard types can be identified. Keyboard type (401) may be a qwerty type. Keyboard type (402) may be a qwertz type. Keyboard type (403) may be an azerty type. Keyboard type (404) may be a 3*4 type.

[0068] Referring to FIG. 4B, according to one embodiment, various Korean keyboard types can be identified. Keyboard type (405) may be a QWERTY type. Keyboard type (406) may be a Chunjiin type. Keyboard type (407) may be a Chunjiin Plus type. Keyboard type (408) may be a one-handed Moakey type.

[0069] Referring to FIG. 5A, according to one embodiment, a keyboard layout (502) of a qwerty type English keyboard (501) can be confirmed. The keyboard layout (502) can include information (e.g., X, Y) about the physical location of a key within the keyboard. For example, the q key can be located one space below a key located at the top left of the qwerty type English keyboard (501) (e.g., the 1 key). The physical location of the q key can be expressed as (0,1). For example, the w key can be located one space to the right and one space below a key located at the top left of the qwerty type English keyboard (501) (e.g., the 1 key). The physical location of the W key can be expressed as (1, 1). For example, the a key can be located half a space to the right and two spaces below a key located at the top left of the qwerty type English keyboard (501) (e.g., the 1 key). The physical location of the a key can be expressed as (0.5, 2).

[0070] Referring to FIG. 5B, according to one embodiment, a keyboard layout (504) of a qwerty type English keyboard (503) supporting Latin can be confirmed. The keyboard layout (504) may include not only information about the physical location of a key within the keyboard (e.g., X, Y), but also attribute information of the key (e.g., Z, ZX, ZY). For example, the ae key may require additional input compared to the a key described above in FIG. 5A. The ae key may be located at the upper left of the displayed Latin input interface (503-1) by maintaining a touch on the a key. The ae key may be expressed as (0.5, 2, 1, 0, 0) through the physical location information of the key (e.g., X, Y) and attribute information of the key (e.g., Z, ZX, ZY). For example, The key can be positioned one space to the right of ae in the displayed Latin input interface (503-1) by holding the a key touch. A key can be expressed as (0.5, 2, 1, 1, 0) through the physical location information of the key (e.g., X, Y) and the attribute information of the key (e.g., Z, ZX, ZY).

[0071] Referring to FIG. 5C, according to one embodiment, a keyboard layout (506) of a qwerty type Korean keyboard (505) can be confirmed. The keyboard layout (506) may include information (e.g., X, Y) about the physical location of a key within the keyboard. For example, the ㅂ key may be located one space below a key located at the upper left corner (e.g., the 1 key) of the qwerty type Korean keyboard (505). The physical location of the ㅂ key may be expressed as (0,1). For example, the ㅈ key may be located one space to the right and one space below a key located at the upper left corner (e.g., the 1 key) of the qwerty type Korean keyboard (505). The physical location of the ㅈ key may be expressed as (1, 1).

[0072] Referring to FIG. 5d, according to one embodiment, the keyboard layout (508) of the Cheonjiin type Korean keyboard (507) can be confirmed. The keyboard layout (508) may include attribute information (e.g., X, Y, Z) of characters, rather than the physical location of the keys. The attribute information of characters in the keyboard layout (508) may be attribute information for inputting characters (e.g., consonants or vowels). For example, in order to input the vowel ㅏ, the ㅣ key located at the upper left of the Cheonjiin type Korean keyboard (507) and the ㅣ key located one space to the right are used. You may need to enter the key. For example, to enter the vowel ㅑ, press the ㅣ key located at the top left of the Cheonjiin type Korean keyboard (507) once, and then press the ㅣ key located one space to the right. You may need to press the key twice. Therefore, referring to the keyboard layout (508), The vowels ㅏ and ㅑ, which require different key presses, can differ only by 1 in their Z values. For keyboard types where a single character (e.g., a consonant or a vowel) is completed by combining multiple keys and / or multiple inputs, the keyboard layout can be expressed as attribute information of the character.

[0073]

[0074] FIG. 6 is a diagram for explaining a keyboard layout estimation method according to one embodiment.

[0075] Referring to FIG. 6, according to one embodiment, an electronic device (e.g., the electronic device (201) of FIG. 2) can analyze a keyboard image (601-1) to obtain a keyboard layout (603). The electronic device (201) can capture a screen (601) that it displays. Based on a first neural network model (602), the electronic device (201) can obtain a keyboard layout (603) from the keyboard image (601-1) included in the screen (601). The keyboard layout (603) can include vectors corresponding to keys constituting the keyboard.

[0076] In one embodiment, the first neural network model (602) may refer to an overall model in which artificial neurons (nodes) that form a network through the combination of synapses change the strength of the synaptic connections through learning, thereby possessing problem-solving capabilities. The first neural network model (602) may be trained based on learning data in which a keyboard image and keyboard layout are mapped.

[0077] In one embodiment, the neurons of the first neural network model (602) may include a combination of weights or biases. The first neural network model (602) may include one or more layers composed of one or more neurons or nodes. The first neural network model (602) may infer a desired outcome from any input by changing the weights of the neurons through learning.

[0078] According to one embodiment, the first neural network model (602) may include a deep neural network. The first neural network model (602) may include a convolutional neural network (CNN), a recurrent neural network (RNN), a perceptron, a multilayer perceptron, a feed forward (FF), a radial basis network (RBF), a deep feed forward (DFF), a long short term memory (LSTM), a gated recurrent unit (GRU), an auto encoder (AE), a variational auto encoder (VAE), a denoising auto encoder (DAE), a sparse auto encoder (SAE), a markov chain (MC), a hopfield network (HN), a boltzmann machine (BM), a restricted boltzmann machine (RBM), a deep belief network (DBN), a deep convolutional network (DCN), a deconvolutional network (DN), a deep convolutional inverse graphics network (DCIGN), and a generative adversarial network (GAN). The operation of FIG. 6 described above may be performed when the electronic device (201) has the authority to capture the screen.

[0079]

[0080] FIGS. 7A and 7B are diagrams for explaining a keyboard layout estimation method according to one embodiment.

[0081] In one embodiment, an electronic device (e.g., electronic device (201) of FIG. 2) needs to estimate a keyboard layout even if it does not have screen capture permission. The electronic device (201) can estimate the keyboard layout by analyzing text entered via the keyboard.

[0082] Referring to FIG. 7A, according to one embodiment, sequences (701 and 702) associated with a search word (e.g., user input) (e.g., face) can be identified. A keyboard for inputting the search word (e.g., face) may be a qwerty type keyboard. A first sequence (701) associated with the search word (e.g., face) may be a sequence of inputs to the keyboard in chronological order (e.g., f, a, v, backspace, c, e). A second sequence (702) associated with the search word (e.g., face) may be a sequence of search words that change according to inputs to the keyboard in chronological order (e.g., f, fa, fav, fa, fac, face). The electronic device (201) may obtain the first sequence (701) and the second sequence (702) based on the search word (e.g., face).

[0083] Referring to FIG. 7B, the electronic device (201) can obtain a keyboard type based on sequences (703) to which a first sequence (701) and a second sequence (702) are mapped based on a second neural network model (704). The electronic device (201) can estimate a keyboard layout (705) based on the keyboard type. The second neural network model (704) may be learned based on learning data to which the first sequence, the second sequence, and the keyboard type are mapped. The second neural network model (704) may be based on a character-level recurrent neural network (RNN).

[0084] According to one embodiment, the electronic device (201) may utilize a typo pattern (e.g., typo identification based on backspace input recognition). However, in some cases, a typo pattern may not be required. In the above-described FIG. 7B, the typo pattern included in the sequences (703) to which the first sequence (701) and the second sequence (702) are mapped is utilized, but the present invention is not limited thereto. For example, in the case of a 3*4 type English keyboard, the search word (e.g., face) can be sufficiently identified using only the second sequence (e.g., d, e, f, fa, faa, fab, fac, facd, face). In a type where one character is determined through multiple inputs, such as a 3*4 type English keyboard, the second sequence alone can be sufficiently identified. The electronic device (201) can estimate the keyboard layout based on the second sequence.

[0085] According to one embodiment, the method for estimating the keyboard layout may not be limited to utilizing a neural network model. The keyboard layout may also be acquired directly within the electronic device (201). For example, the operating system of the electronic device (201) (e.g., the operating system (142) of FIG. 1 ) may manage settings related to the keyboard layout, and the electronic device (201) may acquire the keyboard layout using any library or any application programming interface (API).

[0086]

[0087] FIG. 8 is a diagram illustrating an index dictionary according to one embodiment.

[0088] Referring to FIG. 8, according to one embodiment, examples of index terms (e.g., 801 and 802) stored in an index term dictionary (e.g., the index term dictionary (231) of FIG. 2) can be identified. The index term dictionary may be composed of index terms processed for search. The index terms (801) may be a set of index terms associated with a search target (e.g., a Facebook application). The index terms (802) may be a set of index terms associated with a search target (e.g., a Carrot Market application).

[0089] According to one embodiment, the index terms (801) may include an original index term (e.g., Facebook) of the search target (e.g., Facebook application), a lowercase index term (e.g., facebook) of the search target (e.g., Facebook application), an index term for keyboard fuzzing (e.g., facebook) of the search target (e.g., Facebook application), an index term corresponding to a translation result (e.g., phonetic conversion) of the search target (e.g., Facebook), an index term corresponding to a related search term of the search target (e.g., Facebook application), and an index term corresponding to an app category of the search target (e.g., Facebook application) (e.g., social).

[0090] According to one embodiment, the index terms (802) may include an original index term (e.g., Carrot Market) of a search target (e.g., Carrot Market), an index term corresponding to a morphological analysis result of the search target (e.g., Carrot and Market), an index term corresponding to a translated result (e.g., English name) of the search target (e.g., Carrot Market application) (e.g., carrot market), an index term corresponding to a phoneme separation result of the search target (e.g., Carrot Market application) (e.g., ㄷㅏㅇㄱㅡㄴㅁㅏㅋㅔㅅ), an index term corresponding to an initial consonant of the search target (e.g., Carrot Market application) (e.g., ㄷㄱㅁㅋ), and an index term corresponding to a related search term of the search target (e.g., Carrot Market application) (e.g., used goods transaction). In the case of Korean text, in order to support a keyboard fuzzy function, an index term corresponding to a phoneme separation result may necessarily be included in the index term dictionary.

[0091]

[0092] FIGS. 9A to 9C are diagrams for explaining an adjacent distance calculation operation according to one embodiment.

[0093] According to one embodiment, an electronic device (e.g., the electronic device (201) of FIG. 2) may select an index word whose proximity distance to a search word (e.g., a user input) is less than a threshold based on a keyboard layout. The proximity distance may be the sum of one or more L1 distances. The L1 distance may refer to the manhattan distance. The L1 distance may be calculated for each character. The keyboard layout may include vectors corresponding to keys constituting the keyboard. The electronic device (201) may calculate the L1 distance between characters constituting the search word and characters constituting the index word based on the vectors included in the keyboard layout. The electronic device (201) may collect the L1 distances to obtain the proximity distance between the search word and the index word.

[0094] Referring to FIG. 9A, according to one embodiment, the electronic device (201) can calculate the proximity distance between a search word (e.g., rave) and an index word (e.g., facebook). The keyboard illustrated in FIG. 9A may be a qwerty type keyboard. The L1 distance between a character (e.g., r) (902) constituting the search word (e.g., rave) and a character (e.g., f) (901) constituting the index word (e.g., facebook) may be 1.5 (e.g., 1 in the y-axis and 0.5 in the x-axis). The L1 distance between a character (e.g., a) constituting the search word (e.g., rave) and a character (e.g., a) constituting the index word (e.g., facebook) may be 0. The L1 distance between a character (e.g., v) (912) constituting a search term (e.g., rave) and a character (e.g., c) (911) constituting an index term (e.g., facebook) may be 1. The L1 distance between a character (e.g., e) constituting a search term (e.g., rave) and a character (e.g., e) constituting an index term (e.g., facebook) may be 0. The electronic device (201) may calculate the proximity distance (e.g., 2.5) between the search term (e.g., rave) and the index term (e.g., facebook) by collecting the L1 distances (e.g., 1.5, 0, 1, and 0). For the same search term-index term pair, the proximity distance may differ depending on the keyboard type (or keyboard layout). Therefore, even if the same search term is entered into the electronic device (201), the search term correction result (e.g., index term) may differ depending on the keyboard type used to enter the search term. Even if the same search term is entered into the electronic device (201), the returned search results may differ depending on the keyboard type.

[0095] Referring to FIG. 9B, according to one embodiment, the electronic device (201) may calculate the proximity distance between a search word (e.g., 댱근) and an index word (e.g., carrot). The keyboard illustrated in FIG. 9B may be a keyboard of the Cheonjiin type (e.g., 406 of FIG. 4B). The search word (e.g., 댱근) and the index word (e.g., carrot) may differ only in the vowel ㅏ and the vowel ㅑ. The L1 distance between the character (e.g., ㅑ) constituting the search word (e.g., 댱근) and the character (e.g., ㅏ) constituting the index word (e.g., carrot) may be 1. As described above through FIG. 5D, in order to input the vowel ㅏ, the ㅣ key located at the upper left of the Cheonjiin type Korean keyboard and the ㅣ key located one space to the right may be used. You may need to enter the key (921). For example, to enter the vowel ㅑ, press the ㅣ key located at the top left of the Cheonjiin type Korean keyboard once, and then press the ㅣ key located one space to the right. You may need to enter the key (921) twice. Therefore, The L1 distance between vowels ㅏ and ㅑ, which have different input counts of key (921), can be 1. The adjacency distance between a search word (e.g., 댱근) and an index word (e.g., 단고) can be 1.

[0096] Referring to FIG. 9C, according to one embodiment, the electronic device (201) may calculate the proximity distance between a search word (e.g., "dunggeun") and an index word (e.g., "carrot"). The keyboard illustrated in FIG. 9C may be a one-handed moakey type keyboard (e.g., 408 of FIG. 4B). The search word (e.g., "dunggeun") and the index word (e.g., "carrot") may differ only in the vowel ㅓ and the vowel ㅏ. The L1 distance between the character (e.g., ㅓ) constituting the search word (e.g., "dunggeun") and the character (e.g., ㅏ) constituting the index word (e.g., "carrot") may be 1. In order to input the vowel ㅓ, it may be necessary to drag to the left on the one-handed moakey type keyboard (e.g., 931) (e.g., 941). To input the vowel ㅏ, you may need to drag to the right on a one-handed Moakey keyboard (e.g., 931) (e.g., 942). Therefore, the L1 distance between the vowels ㅓ and ㅏ, which have different drag directions, may be 1. The adjacency distance between a search term (e.g., 뱀갑갑) and an index term (e.g., 갑갑갑) may be 1.

[0097]

[0098] FIG. 10 is an example of search results returned by an electronic device according to one embodiment.

[0099] Referring to FIG. 10, according to one embodiment, an electronic device (201) (e.g., the electronic device (101) of FIG. 1) may return search results (e.g., 1001, 1002, 1003) (e.g., application search results) corresponding to a search term (e.g., cane). The search results (e.g., 1001, 1002, 1003) may be displayed in order of decreasing proximity. If the proximity distances are the same, the search results may be displayed according to an existing policy (e.g., sorted alphabetically).

[0100] According to one embodiment, the electronic device (201) may be based on an instant search method (e.g., a method in which search results are updated for each input on the keyboard). To prevent missearch, the electronic device (201) may perform the keyboard purge function described above with reference to FIGS. 2 to 9 when the number of characters constituting the search term is greater than a threshold value (e.g., 4).

[0101] According to one embodiment, the electronic device (201) may perform a keyboard purge function when no search result corresponding to the search word exists, according to a fallback mechanism.

[0102]

[0103] FIG. 11 illustrates a flowchart of a method of operating an electronic device according to one embodiment.

[0104] Actions 1110 to 1150 may be performed sequentially, but are not necessarily performed sequentially. For example, the order of each action (1110 to 1150) may be changed, and at least two actions may be performed in parallel.

[0105] According to one embodiment, operations 1110 to 1150 may be understood to be performed in a processor (e.g., processor (220) of FIG. 2) of an electronic device (e.g., electronic device (201) of FIG. 2).

[0106] In operation 1110, an electronic device according to one embodiment may receive a search term from a user.

[0107] In operation 1120, an electronic device according to one embodiment may estimate a keyboard layout of a keyboard utilized to input a search word.

[0108] In operation 1130, an electronic device according to one embodiment may select an index word whose proximity to the search word is less than a threshold value based on the keyboard layout.

[0109] In operation 1140, an electronic device according to one embodiment can obtain search results corresponding to each of the index word and the search word.

[0110] In operation 1150, an electronic device according to one embodiment may return search results in response to a search term.

[0111]

[0112] According to one embodiment, a method of operating an electronic device (e.g., electronic device 101 of FIG. 1 or electronic device 201 of FIG. 2) may include an operation of receiving a search word from a user. The method may include an operation of estimating a keyboard layout of a keyboard used to input the search word. The method may include an operation of selecting an index word whose proximity distance to the search word is less than a threshold based on the keyboard layout. The method may include an operation of obtaining search results corresponding to each of the index word and the search word. The method may include an operation of returning the search results corresponding to the search word.

[0113] In one embodiment, the operation of estimating the keyboard layout may include an operation of capturing a screen displayed by the electronic device. The operation of estimating the keyboard layout may include an operation of obtaining the keyboard layout from a keyboard image included in the screen based on a first neural network model.

[0114] In one embodiment, the operation of estimating the keyboard layout may include an operation of obtaining the type of the keyboard from at least one of a first sequence or a second sequence associated with the search term based on a second neural network model. The operation of estimating the keyboard layout may include an operation of obtaining the keyboard layout based on the type of the keyboard. The first sequence may be an operation of dividing inputs to the keyboard according to a time sequence. The second sequence may be an operation of dividing search terms that change according to the inputs to the keyboard according to a time sequence.

[0115] In one embodiment, the keyboard layout may include a vector corresponding to a key constituting the keyboard. The vector may include at least one of information regarding the physical location of the key within the keyboard; or attribute information based on the type of the keyboard.

[0116] In one embodiment, the proximity distance may be the sum of one or more L1 distances, each of which may be an L1 distance between characters constituting the search term and characters constituting the index term, based on a vector included in the keyboard layout.

[0117] According to one embodiment, the index word may be selected from an index word dictionary composed of index words processed for searching.

[0118] According to one embodiment, the search results may be displayed in order of decreasing proximity.

[0119] In one embodiment, the operation of returning the search results may be based on an instant search method in which the search results are updated for each input to the keyboard.

[0120] According to one embodiment, the operating method may be triggered when the number of characters constituting the search word is greater than a threshold value or when no search result corresponding to the search word exists.

[0121] According to one embodiment, the method of operation may be performed as a fallback mechanism according to a search policy of the electronic device.

[0122] According to one embodiment, the index word dictionary may include at least one of an original index word of the search target, an index word corresponding to a result of morphological analysis of the search target, an index word corresponding to a translation result of the search target, an index word corresponding to a result of grapheme segmentation of the search target, an index word corresponding to an initial consonant of the search target, an index word corresponding to a category of the search target, or an index word corresponding to a related search word of the search target.

[0123] An electronic device (e.g., electronic device 101 of FIG. 1 or electronic device 201 of FIG. 2) according to one embodiment may include one or more processors (e.g., processor 120 of FIG. 1 or processor 220 of FIG. 2). The electronic device may include a memory (e.g., memory 130 of FIG. 1 or memory 230 of FIG. 2) that stores instructions. The instructions, when individually or collectively executed by the one or more processors, may cause the electronic device to receive a search word from a user. The instructions, when individually or collectively executed by the one or more processors, may cause the electronic device to estimate a keyboard layout of a keyboard used to input the search word. The instructions, when individually or collectively executed by the one or more processors, may cause the electronic device to select an index word whose proximity to the search word is less than a threshold value based on the keyboard layout. The instructions, when individually or collectively executed by the one or more processors, may cause the electronic device to obtain search results corresponding to each of the index term and the search term. The instructions, when individually or collectively executed by the one or more processors, may cause the electronic device to return the search results corresponding to the search term.

[0124] According to one embodiment, the instructions, when individually or collectively executed by the one or more processors, may cause the electronic device to capture a screen displayed by the electronic device (101; 201). The instructions, when individually or collectively executed by the one or more processors, may cause the electronic device to obtain the keyboard layout from a keyboard image included in the screen based on a first neural network model.

[0125] According to one embodiment, the instructions, when individually or collectively executed by the one or more processors, may cause the electronic device to obtain the type of the keyboard from at least one of a first sequence or a second sequence associated with a search term based on a second neural network model. The instructions, when individually or collectively executed by the one or more processors, may cause the electronic device to obtain the keyboard layout based on the type of the keyboard. The first sequence may be a sequence that categorizes inputs to the keyboard according to a time sequence. The second sequence may be a sequence that categorizes search terms that change according to the inputs to the keyboard according to a time sequence.

[0126] In one embodiment, the keyboard layout may include a vector corresponding to a key constituting the keyboard. The vector may include at least one of information regarding the physical location of the key within the keyboard; or attribute information based on the type of the keyboard.

[0127] In one embodiment, the proximity distance may be the sum of one or more L1 distances. Each of the one or more L1 distances may be an L1 distance between characters constituting the search term and characters constituting the index term, based on a vector included in the keyboard layout.

[0128] According to one embodiment, the index word may be selected from an index word dictionary composed of index words processed for searching.

[0129] According to one embodiment, the search results may be displayed in order of decreasing proximity distance.

[0130] According to one embodiment, the electronic device may be based on an instant search method in which search results are updated for each input to the keyboard.

[0131] According to one embodiment, the index word dictionary may include at least one of an original index word of the search target, an index word corresponding to a result of morphological analysis of the search target, an index word corresponding to a translation result of the search target, an index word corresponding to a result of grapheme segmentation of the search target, an index word corresponding to an initial consonant of the search target, an index word corresponding to a category of the search target, or an index word corresponding to a related search word of the search target.

[0132]

[0133] Electronic devices according to the various embodiments disclosed in this document may take various forms. Electronic devices may include, for example, portable communication devices (e.g., smartphones), computer devices, portable multimedia devices, portable medical devices, cameras, wearable devices, or home appliances. Electronic devices according to the embodiments of this document are not limited to the aforementioned devices.

[0134] The various embodiments of this document and the terminology used therein are not intended to limit the technical features described in this document to specific embodiments, but should be understood to include various modifications, equivalents, or substitutes of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of the items, unless the context clearly indicates otherwise. In this document, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" can include any one of the items listed together in the corresponding phrase among those phrases, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used merely to distinguish one component from another, and do not limit the components in any other respect (e.g., importance or order). When a component (e.g., a first component) is referred to as "coupled" or "connected" to another (e.g., a second component), with or without the terms "functionally" or "communicatively," it means that the component can be connected to the other component directly (e.g., wired), wirelessly, or through a third component.

[0135] The term "module" used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integral component, or a minimum unit or part of such a component that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0136] Various embodiments of the present document may be implemented as software (e.g., a program) including one or more instructions stored in a storage medium (e.g., built-in memory or external memory) readable by a machine (e.g., an electronic device). For example, a processor (e.g., a processor) of the machine (e.g., an electronic device) may call at least one instruction among the one or more instructions stored from the storage medium and execute it. This enables the machine to operate to perform at least one function according to the at least one instruction called. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily in the storage medium.

[0137] According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smart phones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

[0138] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (e.g., a module or a program) may be integrated into a single component. In such a case, the integrated component may perform one or more functions of each of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. According to various embodiments, the operations performed by a module, program, or other component may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

Claims

1. In the operating method of an electronic device (101; 201), The action of receiving a search term from a user; An action to estimate the keyboard layout of the keyboard used to input the above search term; An operation of selecting an index word whose proximity distance to the search word is less than a threshold value based on the above keyboard layout; An operation of obtaining search results corresponding to each of the above index words and the above search words; and An action that returns the search results in response to the above search term. A method of operating an electronic device (101; 201), comprising:

2. In paragraph 1, The action of estimating the above keyboard layout is: An operation of capturing a screen displayed by the above electronic device (101; 201); and An operation of obtaining the keyboard layout from the keyboard image included in the screen based on the first neural network model. A method of operation, comprising:

3. In either of paragraphs 1 and 2, The action of estimating the above keyboard layout is: An operation of obtaining the type of the keyboard from at least one of the first sequence or the second sequence associated with the search word based on the second neural network model; and An operation for obtaining the keyboard layout based on the type of the above keyboard. Including, The above first sequence is a sequence of inputs to the keyboard, classified in chronological order. The second sequence above is a search word that changes according to the inputs to the keyboard, divided into chronological order. How it works.

4. In any one of paragraphs 1 to 3, The above keyboard layout is, Contains a vector corresponding to the keys that make up the above keyboard, The above vector is, Information about the physical location of said key within said keyboard; or Attribute information based on the type of the above keyboard A method of operation, comprising at least one of:

5. In any one of paragraphs 1 to 4, The above adjacent distances are, is the sum of one or more L1 distances, Each of the above one or more L1 distances, The L1 distance between the characters constituting the search word and the characters constituting the index word, based on the vectors included in the above keyboard layout. How it works.

6. In any one of paragraphs 1 to 5, The above keywords are, Selected from an index dictionary consisting of index words processed for searching, How it works.

7. In the electronic device (101; 201), One or more processors (120; 220); and Memory for storing instructions (130; 230) Including, The above instructions, when individually or collectively executed by the one or more processors (120; 220), cause the electronic device (101; 201) to: Receive search terms from users, Estimate the keyboard layout of the keyboard used to input the above search term, Based on the above keyboard layout, index words whose proximity to the search word is less than a threshold value are selected, Obtain search results corresponding to each of the above index words and search words, In response to the above search term, return the above search results. Electronic devices (101; 201).

8. In paragraph 7, The above instructions, when individually or collectively executed by the one or more processors (120; 220), cause the electronic device (101; 201) to: Capturing the screen displayed by the above electronic device (101; 201), Based on the first neural network model, the keyboard layout is obtained from the keyboard image included in the screen. Electronic devices (101; 201).

9. In either of paragraphs 7 and 8, The above instructions, when individually or collectively executed by the one or more processors (120; 220), cause the electronic device (101; 201) to: Based on the second neural network model, the type of the keyboard is obtained from at least one of the first sequence or the second sequence associated with the search word, Based on the type of the above keyboard, obtain the above keyboard layout, The above first sequence is a sequence of inputs to the keyboard, classified in chronological order. The second sequence above is a search word that changes according to the inputs to the keyboard, divided into chronological order. Electronic devices (101; 201).

10. In any one of paragraphs 7 to 9, The above keyboard layout is, Contains a vector corresponding to the keys that make up the above keyboard, The above vector is, Information about the physical location of said key within said keyboard; or Attribute information based on the type of the above keyboard An electronic device (101; 201) comprising at least one of:

11. In any one of paragraphs 7 to 10, The above adjacent distances are, is the sum of one or more L1 distances, Each of the above one or more L1 distances, The L1 distance between the characters constituting the search word and the characters constituting the index word, based on the vectors included in the above keyboard layout. Electronic devices (101; 201).

12. In any one of paragraphs 7 to 11, The above keywords are, Selected from an index dictionary consisting of index words processed for searching, Electronic devices (101; 201).

13. In any one of paragraphs 7 to 12, The above search results are, It is displayed in order of decreasing adjacent distance. Electronic devices (101; 201).

14. In any one of paragraphs 7 to 13, The above electronic device (101; 201) It is based on an instant search method where search results are updated with each input to the above keyboard. Electronic devices (101; 201).

15. In any one of paragraphs 7 to 14, The above index dictionary is, An original index word of the search target, an index word corresponding to the result of morphological analysis of the search target, an index word corresponding to the result of translation of the search target, an index word corresponding to the result of grapheme segmentation of the search target, an index word corresponding to the initial consonant of the search target, an index word corresponding to the category of the search target, or an index word corresponding to a related search word of the search target. An electronic device (101; 201) comprising at least one of:

Images