Multi-language keyboard input detection for translanguaging

Abstract

Multi-language keyboard input mode. A method comprises receiving a string of keyboard inputs in a user interface associated with an electronic processing system when the electronic processing system is in a mode that allows the keyboard inputs to represent two or more different human languages associated with a corresponding two or more different sets of characters. A human language is selected by the electronic processing system for the string of keyboard inputs out of the two or more different human languages. The electronic processing system displays text in a display associated with the electronic processing system in the selected human language.

Description

MULTI-LANGUAGE KEYBOARD INPUT DETECTION FOR TRANSLANGUAGINGInventors:Zongfang Lin Hong Heather Yu Xiyun Song Yubin Zhou Zhiqiang Lao Liang PengFIELD

[0001] The disclosure generally relates to input techniques for keyboard input to a processing system.BACKGROUND

[0002] Electronic processing systems such as laptop computers, desktop computers, cellular telephones, notepad computers, and Internet of Things (loT) devices etc. allow users to input via user interfaces such keyboards, keypads, etc. Such user interfaces could include a “physical” user interface such as a physical keyboard or a “virtual” user interface such as a virtual keypad (also referred to as a virtual keyboard) shown on a display screen. FIG. 1 shows an example of a conventional keyboard layout for a physical keyboard. This example keyboard has the QWERTY layout with keys having letters suitable for English. The QWERTY layout in FIG. 1 is just one example. Similar keyboards such as the QWERTZ layout are suitable for languages such as German. Modifications may be made to QWERTY layout in FIG. 1 to accommodate diacritics and special character requirements. In general, the QWERTY layout and variations may be suitable for Latin-script alphabets such as English, German, French, Italian, etc. The keyboard layout contains additional keys for entering input other than letters, such as numbers and function keys. The type and location of the function keys may vary depending on the physical keyboard, computer system, operating system, etc. Each key on the keyboard is associated with a unique scan code. The scan code for each key is depicted in FIG. 1 , although scan codes are typically not depicted on the actual physical keyboard. When the userpresses a particular key, the electronic processing system receives the associated scan code. For example, the “Q” key has the scan code “10”. A virtual keypad based on the QWERTY layout may contain all of the letters suitable for Latin-script alphabets, but possibly omitting some of the other keys to save space on a display screen such as a cellular telephone display.

[0003] Many users speak and type in languages having characters other than Latin-script alphabets. Therefore, keyboards have been devised for languages having characters other than Latin-script alphabets. For example, keyboards have been developed for languages such as Chinese, Korean, Japanese, Arabic, etc. Some electronic processing systems allow the user to input different languages at different times. To do so the computer system may first require that the user place the system into a specific language mode. Then, when a scan code is received from the keyboard, the electronic processing system assigns a language specific meaning to the scan code. For example, the keys of the QWERTY layout in FIG. 1 may be used at one point in time to enter letters in the Latin-script alphabet, but at another time to enter symbols of the Korean alphabet. For example, when in a Korean language input mode the keys in FIG. 1 for each letter of the Latin-script alphabet may be interpreted as a particular basic vowel or basic consonant of the Korean alphabet. As a specific example, the user selection of the “Q” key results in the scan code “10”, which when in an English mode may be interpreted by the electronic processing system as a “Q”, but when in a Korean mode may be interpreted as a basic consonant in the Korean alphabet. As another example, the QWERTY keyboard in FIG. 1 could be used to enter Pinyin characters. Pinyin may be referred to as the Chinese phonetic alphabet. Pinyin allows Mandarin Chinese sounds to be entered by the selection of the Pinyin characters. The electronic processing system may then present the user with a selection of candidate words in traditional Chinese characters to choose from. Note that although FIG. 1 shows the Latin-script alphabet, the keys could instead, or in addition, show characters in Pinyin, the Korean alphabet, etc.

[0004] While such electronic processing systems allow the user to enter text in more than one language, the user explicitly switches between the language modes. For example, the user may be required to select a “hotkey” or use a “click selection”to switch the keyboard input language mode. As a specific example, the user may need to press two specific keys simultaneously to switch the keyboard input language mode. As another specific example, the user might select a button on a taskbar being displayed on the display screen. Such explicit actions can be annoying to the user and inconvenient when switching between input languages often. Such frequent language mode switching can interrupt the user’s train of thoughts or flow of ideas. Constantly alternating between language modes can result in errors and mental strain for users.SUMMARY

[0005] One general aspect includes a method for providing a multi-language keyboard input mode. The method comprises receiving a string of keyboard inputs in a user interface associated with an electronic processing system when the electronic processing system is in a mode that allows the keyboard inputs to represent two or more different human languages associated with a corresponding two or more different sets of characters. The method comprises selecting, by the electronic processing system, a human language for the string of keyboard inputs out of the two or more different human languages. The method comprises displaying, by the electronic processing system, text in a display associated with the electronic processing system in the selected human language.

[0006] Implementations may include the method where receiving the string of keyboard inputs in the user interface comprises receiving a scan code for each keyboard input in the string; and selecting the human language for the string of keyboard inputs comprises processing characters in the two or more different human languages to which the scan codes map.

[0007] Implementations may include any of the foregoing methods wherein selecting, by the electronic processing system, the human language for the string of keyboard inputs out of the two or more different human languages comprises: accessing, from non-transitory memory, two or more dictionaries that correspond to the two or more human languages; and selecting a first human language responsiveto the string of keyboard inputs mapping to a word in a first dictionary of the two or more dictionaries, the first dictionary corresponding to the first human language.

[0008] Implementations may include any of the foregoing methods wherein selecting, by the electronic processing system, the human language for the string of keyboard inputs out of the two or more different human languages comprises: inferring a word in a first human language responsive to inputting the string of keyboard inputs to an artificial intelligence language model.

[0009] Implementations may include any of the foregoing methods wherein selecting, by the electronic processing system, the human language for the string of keyboard inputs out of the two or more different human languages comprises: accessing, from non-transitory memory, two or more dictionaries that correspond to the two or more human languages; determining whether the string of keyboard inputs map to a word in any of the two or more dictionaries; inputting the string of keyboard inputs to an artificial intelligence language model responsive to the string of keyboard inputs not mapping to any words in the two or more dictionaries; and inferring a word in a first human language responsive to inputting the string of keyboard inputs to the artificial intelligence language model.

[0010] Implementations may include any of the foregoing methods wherein selecting, by the electronic processing system, the human language for the string of keyboard inputs out of the two or more different human languages comprises: accessing, from non-transitory memory, two or more dictionaries that correspond to the two or more human languages; determining whether the string of keyboard inputs map to a word in each of two of the two or more dictionaries that correspond to a first human language and a second human language; and selecting between the first human language and the second human language based on a context in which the keyboard input is received responsive to the string of keyboard inputs mapping to a word in each of two of the two or more dictionaries.

[0011] Implementations may include any of the foregoing methods wherein selecting, by the electronic processing system, the human language for the string of keyboard inputs out of the two or more different human languages comprises:accessing, from non-transitory memory, a first vocabulary for a user for a first human language; accessing, from non-transitory memory, a second vocabulary for the user for a second human language; and selecting between the first human language and the second human language based on a matching between the string of keyboard inputs and content in the first vocabulary and the second vocabulary.

[0012] Implementations may include any of the foregoing methods wherein selecting, by the electronic processing system, the human language for the string of keyboard inputs out of the two or more different human languages comprises: accessing, from non-transitory memory, a first pool of words in a first human language present in documents and applications in present use on the electronic processing system; accessing, from non-transitory memory, a second pool of words in a second human language present in the documents and the applications in present use on the electronic processing system; and selecting between the first human language and the second human language based on a matching between the string of keyboard inputs and content in the first pool and the second pool.

[0013] Implementations may include any of the foregoing methods wherein selecting, by the electronic processing system, the human language for the string of keyboard inputs out of the two or more different human languages comprises: accessing, from non-transitory memory, topics in a first human language present in documents and applications in present use on the electronic processing system; accessing, from non-transitory memory, topics in a second human language present in the documents and the applications in present use on the electronic processing system; and selecting between the first human language and the second human language based on the topics in the first human language present and the second human language.

[0014] Implementations may include any of the foregoing methods wherein selecting, by the electronic processing system, the human language for the string of keyboard inputs out of the two or more different human languages comprises: determining a first probability that the string of keyboard inputs corresponds to a first human language of the two or more human languages; determining a secondprobability that the string of keyboard inputs corresponds to a second human language of the two or more human languages; and selecting between the first human language and the second human language based on the first probability and the second probability.

[0015] Implementations may include any of the foregoing methods wherein selecting, by the electronic processing system, the human language for the string of keyboard inputs out of the two or more different human languages comprises: determining a probability that the string of keyboard inputs corresponds to a first human language of the two or more human languages; and selecting the first human language responsive to the probability being greater than a threshold.

[0016] Implementations may include any of the foregoing methods wherein selecting, by the electronic processing system, the human language for the string of keyboard inputs out of the two or more different human languages comprises: determining one or more candidate words in a first human language of the two or more human languages for the string of keyboard inputs; determining one or more candidate words in a second human language of the two or more human languages for the string of keyboard inputs; determining a probability for each of one or more candidate words in first human language for the string of keyboard inputs; and determining a probability for each of one or more candidate words in second human language for the string of keyboard inputs.

[0017] Implementations may include any of the foregoing methods wherein selecting, by the electronic processing system, the human language for the string of keyboard inputs out of the two or more different human languages comprises: selecting, by the electronic processing system, a candidate word of the one or more candidate words in the first human language and the one or more candidate words in the second human language having a highest probability.

[0018] Implementations may include any of the foregoing methods further comprising: receiving a user selection of a weighting between the first human language and the second human language; and determining the probability for each of the one or more candidate words in the first human language and determining theprobability for each of the one or more candidate words in the second human language based on the user selection of the weighting between the first human language and the second human language.

[0019] Implementations may include any of the foregoing methods further comprising: determining, by the electronic processing system, a weighting between the first human language and the second human language for a user; and determining the probability for each of the one or more candidate words in the first human language and determining the probability for each of the one or more candidate words in the second human language based on the weighting for the user.

[0020] Implementations may include any of the foregoing methods further comprising fixing the weighting responsive to a user input to fix the weighting.

[0021] Implementations may include any of the foregoing methods further comprising dynamically updating the weighting responsive to implicit user actions.

[0022] Implementations may include any of the foregoing methods further comprising updating the weighting responsive to an explicit user request to adjust the weighting.

[0023] Implementations may include any of the foregoing methods further comprising displaying, on a display associated with the electronic processing system, one or more candidates in the first human language and one or more candidates in the second human language; and receiving a user selection of one of the one or more candidates in the first human language and the one or more candidates in the second human language.

[0024] Implementations may include any of the foregoing methods further comprising displaying the one or more candidates in the first human language and one or more candidates in the second human language in order of their probabilities.

[0025] Implementations may include any of the foregoing methods wherein selecting the human language for the string of keyboard inputs out of the two or moredifferent human languages comprises automatically translating from a language of the string of keyboard inputs to another language.

[0026] Implementations may include any of the foregoing methods further comprising accessing a user selection of the two or more languages for the mode that allows the keyboard inputs to represent the two or more different human languages and entering the mode responsive to accessing the user selection of the two or more languages for the mode.

[0027] Implementations may include any of the foregoing methods further comprising accessing a user selection of a plurality of languages to be used in a single language mode and selecting between the plurality of languages as the two or more different human languages associated with the corresponding two or more different sets of characters.

[0028] Implementations may include any of the foregoing methods further comprising accessing user preferences and determining the two or more different human languages based on the user preferences.

[0029] One general aspect includes an electronic processing system for multilanguage keyboard input. The system comprises an input / output interface; a storage medium comprising computer program instructions; and one or more processors coupled to communicate with the input / output interface and the storage medium. The one or more processors execute the instructions to: receive a string of keyboard inputs in the input / output interface in a mode that allows the keyboard inputs to represent two or more different human languages associated with a corresponding two or more different sets of characters; select a human language for the string of keyboard inputs out of the two or more different human languages; and display text in a display associated with the input / output interface in the selected human language.

[0030] One general aspect includes a non-transitory computer-readable medium storing computer instructions for multi-language keyboard input, that when executed by one or more processors, cause the one or more processors to: receive a string of keyboard inputs in a mode that allows the keyboard inputs to represent two or moredifferent human languages associated with a corresponding two or more different sets of characters; select a human language for the string of keyboard inputs out of the two or more different human languages; and display text in a display screen in the selected human language.

[0031] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. The claimed subject matter is not limited to implementations that solve any or all disadvantages noted in the Background.BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Aspects of the present disclosure are illustrated by way of example and are not limited by the accompanying Figures for which like references indicate the same or similar elements.

[0033] FIG. 1 illustrates an example of a conventional keyboard layout for a physical keyboard.

[0034] FIG. 2 illustrates an embodiment of an electronic processing system that provides a multi-language input interface.

[0035] FIG. 3 illustrates an embodiment of an interface that allows selection of a multi-language mode.

[0036] FIG. 4 illustrates an example of how settings for dual- or multi-languages may be specified in a tag structure.

[0037] FIGs. 5A and 5B depicts conventional tag structures for specifying languages.

[0038] FIG. 6 is a flowchart of one embodiment of a process of determining a language mode of input.

[0039] FIG. 7 is a flowchart of another embodiment of a process of determining a language mode of input.

[0040] FIG. 8 is a flowchart of an embodiment of a process of providing for multilanguage input.

[0041] FIG. 9A illustrates an example of keys selected by the user, with each key selection being expressed as one of the Latin-script alphabet characters.

[0042] FIG. 9B illustrates an example that combines both Chinese and English inputs.

[0043] FIGs. 9C and 9D illustrate an embodiment of automatic translation during a multi-language input.

[0044] FIG. 10 is a diagram illustrating application level and system level processing during an embodiment of multi-language input processing.

[0045] FIG. 11 is a flowchart of one embodiment of a process of multi-language input detection.

[0046] FIG. 12 is a flowchart of one embodiment of a process of multi-language input detection with translation.

[0047] FIG. 13A is a flowchart of one embodiment of a process of developing language models for multi-language input detection.

[0048] FIG. 13B is a flowchart of one embodiment of a process of fine-tuning an existing pre-trained model.

[0049] FIG. 14 is a flowchart of one embodiment of a process of using a language model to classify and predict a language input to the electronic processing system.

[0050] FIG. 15 is a flowchart of an embodiment of a process of context aware input string processing and prediction.

[0051] FIG. 16 illustrates examples of types of data that may be accessed by the multi-language processing for an example in which the English and Chinese are two possible input languages.

[0052] FIG. 17 illustrates an example of weighting languages for multi-language input detection.

[0053] FIG. 18 is a flowchart of an embodiment of a process of generating candidate words in the potential input languages.

[0054] FIG. 19 is a flowchart of an embodiment of a process of outputting a word based on the candidates generated in the process of FIG. 18.

[0055] FIG. 20 is a schematic diagram of a general-purpose computer system upon which embodiments of multi-language processing as described herein may be performed.WRITTEN DESCRIPTION

[0056] The present disclosure and embodiments present systems and methods for multi-language keyboard input. The term “keyboard” as used throughout this disclosure has a broad meaning and may comprise any interface that allows characters associated with languages to be entered. The term “keyboard” includes both physical keyboards and virtual keyboards. A “keyboard”, as the term is used herein, is not required to include a full set of keys including alphabetic, numeric, punctuation, and function leys. A keyboard, especially a virtual keyboard, can hide some keys or an entire row of keys. For example, a virtual keyboard may dynamically show the keys or row of keys per settings or finger movements. An embodiment of a virtual keyboard can indicate multi-language status. For example, the virtual keyboard may indicate whether multi-language keyboard input is presently enabled. If multilanguage keyboard input is presently enabled the virtual keyboard may indicate which languages are candidates for the keyboard input.

[0057] FIG. 2 illustrates an embodiment of an electronic processing system 200 that provides a multi-language keyboard input interface. Specific systems may utilize all of the components shown, or only a subset of the components, and levels of integration may vary from device to device. Furthermore, electronic processing system 200 may contain multiple instances of a component, such as multiple processing units, processors, memories, transmitters, receivers, etc. The electronic processing system 200 may include a central processing unit (CPU) 210, a memory 220, a mass storage device 230, and an I / O interface 260 connected to a bus 270. The bus 270 may be one or more of any type of several bus architectures including a memory bus or memory controller, a peripheral bus, or the like. A network interface 250 enables the electronic processing system to communicate over a network 280 with other processing devices.

[0058] The I / O interface 260 includes some form of keyboard (physical or virtual) that allows a user to enter characters for language input. The I / O interface 260 may include a display 235, which may be used to display characters in response to user input of a physical keyboard or virtual keyboard. The display 235 may be integrated into the electronic system 200, such as a cellphone display, laptop display screen, notepad computer display screen, etc. However, the display 235 may be attached to the electronic system 200 such as a computer monitor attached to a desktop computer via a cable. The I / O interface 260 may optionally include a physical keyboard 233 such as the example physical keyboard depicted in FIG. 1. The physical keyboard 233 may be integrated into the electronic system 200, such as a laptop keyboard. However, the physical keyboard 233 may be attached to the electronic system 200, such as a physical keyboard attached by a cable to a desktop computer system. However, a physical keyboard is not a requirement. In addition to, or as an alternative to a physical keyboard 233, the I / O interface 260 may include a virtual keyboard. A virtual keyboard may be implemented in display 235, such as touch sensitive display.

[0059] The mass storage device 230 may comprise any type of storage device configured to store data, programs, and other information and to make the data, programs, and other information accessible via the bus 270. The mass storage device 230 may comprise, for example, one or more of a solid-state drive, hard disk drive, amagnetic disk drive, an optical disk drive, or the like. The mass storage device 230 may store instructions for implementing multi-language processing 222. These instructions 222 may be loaded into the memory 220 for execution on the CPU 210. The mass storage device 230 may store data that is used by the multi-language processing 222. The context repository 224 includes context such as user preferences, preferred languages, weights to be given to each language, etc. The language dictionaries 226 may be used by multi-language processing 222 to detect languages input into I / O interface 260. In an embodiment, the multi-language processing 222 performs a machine learning algorithm to develop language modes, which may be stored in the language model repository 262.

[0060] The CPU 210 may comprise any type of electronic data processor. Memory 220 may comprise any type of system memory such as static random-access memory (SRAM), dynamic random-access memory (DRAM), synchronous DRAM (SDRAM), read-only memory (ROM), a combination thereof, or the like. In an embodiment, memory 220 may include ROM for use at boot-up, and DRAM for program and data storage for use while executing programs. In embodiments, the memory 220 is non- transitory. In one embodiment, at least during runtime the memory 220 has computer readable instructions that are executed by the CPU 210 to implement embodiments of the disclosed technology, including multi-language processing 222. All or a subset of the instructions for multi-language processing 222 may be loaded into volatile memory when the electronic processing system 200 is powered on. The multi-language processing 222 may be divided into different functions (or modules), each of which may be implemented by computer readable instructions. Example functions (or modules) include context modeling 242, context manager 244, and preference manager 246. The input buffer 248 is used to store values associated with keyboard inputs. The input buffer 248 could store the scan codes for each key stroke. Alternatively, the scan codes may be mapped to characters in a specific language, wherein the characters may be stored in the input buffer 248. The electronic processing system 200 may store mappings from the scan codes to the characters. Different mappings may be used depending on a present selection of a language associated with keyboard input.

[0061] In an embodiment the user is allowed to explicitly select a multi-language mode. FIG. 3 illustrates an embodiment of an interface 300 that allows selection of a multi-language mode. The interface 300 may be presented to the user on the display 235 in response to the user choosing to establish language settings for the electronic processing system 200. The interface 300 is a very simplified example that has only three language choices. However, a more typical interface will allow the selection of many more languages. The interface 300 has Pinyin icon 302 for selecting a Pinyin mode, an English icon 304 for selecting a US English mode (ENG US), and a dual US English / Pinyin icon 306 for selecting a dual US English / Pinyin mode. When the electronic processing system 200 is in the Pinyin mode scan codes received from the keyboard will be interpreted as Pinyin characters. When the electronic processing system 200 is in the US English mode scan codes received from the keyboard will be interpreted as US English characters. When the electronic processing system 200 is in an embodiment of the dual Pinyin / US English mode scan codes received from the keyboard could be interpreted as either Pinyin characters or as US English characters depending on factors such as the context and user preferences. The interface 300 shows one example format. In general, any interface that allows selection of multiple languages may be used. More generally, the user may select among a large number of languages. Furthermore, more than two languages may be selected in a multilanguage mode.

[0062] It is not required that the user explicitly select a multi-language mode. In another embodiment, the interface 300 in FIG. 3 does not have an option for explicit selection of multiple languages. For example, the user is not necessarily required to select the dual US English / Pinyin option 306. In one embodiment, if the user selects both the Pinyin option 302 and the English option 304 then the electronic processing system 200 provides for a dual English / Pinyin mode without the user explicitly selecting the dual English / Pinyin mode. In an embodiment, the system does not explicitly offer the ‘language selection’ option at all. Thus, it is not required that the system allow the user to select languages by an action such as, but not limited to, hotkeys or click selection. In an embodiment, the keyboard input is interpreted aseither a first language character or a second language character depending on factors such as the context and user preferences.

[0063] FIG. 4 illustrates an example of how settings for dual- or multi-languages may be specified in a tag structure. For example, the tag structure 402 may be used when the user selects dual P iny in / Eng lish. The tag structure 402 in FIG. 4 is compliant with the tag structure defined in the Internal Engineering Task Force (IETF) Best Current Practice (BCP) 47. FIG. 5A depicts a conventional tag structure 504 for English (en). For example, the tag structure 504 may be used when the user selects English. FIG. 5B depicts a conventional tag structure 506 for Chinese (zh). For example, the tag structure 506 may be used when the user selects Chinese (Pinyin). The tag structures 402, 504, 506 may be used for input locale setting to specify the input language and method for input devices such as the keyboard layout. The input locale (also called the input language) is a per-process setting that describes the input language and input method (for example, the keyboard).

[0064] FIG. 6 is a flowchart of one embodiment of a process of determining a language mode of input. The process may be performed by the electronic processing system 200, but is not limited thereto. Step 602 includes accessing a language type for the input. Note that the user could have different language types for different inputs. For example, the user could have a different language type for the physical keyboard than for a virtual keyboard. Step 602 may include determining what language(s) were selected by the user in an interface such as interface 300. Step 602 may include accessing a tag structure such as one of the tag structures 402, 504, 506. Step 604 includes a determination of whether the language type is a Multi-language input. If so, then in step 606 multi-language keyboard input detection is performed. Otherwise conventional single language keyboard input detection is performed in step 608.

[0065] FIG. 7 is a flowchart of another embodiment of a process of determining a language mode of input. The process may be performed by the electronic processing system 200, but is not limited thereto. Step 702 includes detecting the number of languages preferred by the user. This step does not require the user to explicitly selecta multi-language mode. In one embodiment, the system 200 accesses language preferences that include options to select single languages (as in the options 302, 304 in FIG. 3), but does not have the option for a dual-language or multi-language (as in the option 306 in FIG. 3). One embodiment includes detecting such language preferences in user system settings. Step 704 includes a determination of whether more than one language is preferred by the user. If so, then in step 706 multi-language keyboard input detection is performed. Otherwise conventional single language keyboard input detection is performed in step 708.

[0066] FIG. 8 is a flowchart of an embodiment of a process 800 of providing for multi-language input. In an embodiment, the electronic processing system 200 performs process 800. Prior to process 800 the user may have selected a multilanguage mode; however, it is not required that the user expressly select a multilanguage mode. Process 800 may be performed in an embodiment of step 606 in FIG. 6. Process 800 may be performed in an embodiment of step 706 in FIG. 7. In process 800, the electronic processing system 200 is in a mode that allows the inputs in a user interface to represent two or more different sets of characters that are associated with a corresponding two or more different human languages.

[0067] Step 802 includes receiving a string of inputs in a user interface associated with the electronic processing system 200. The user interface may include a physical keyboard or a virtual keyboard. The user may provide the string of inputs by selecting keys in the user interface. For example, the user may select keys on the example QWERTY keyboard shown in FIG. 1. The electronic processing system 200 may receive a scan code for each key selection.

[0068] Step 804 includes selecting a language for the string. In an embodiment, the system 200 predicts what language the user intended. As an example, the electronic processing system 200 predicts whether the user intended the string of inputs to represent Chinese or English. In one embodiment, the electronic processing system 200 predicts what character set the user intended. For example, the electronic processing system 200 predicts whether the user intended the string to include Pinyin characters or Latin-script alphabet characters.

[0069] Step 806 includes displaying text in the selected human language. The term “text” means characters used to write a human language. This text may include characters in the character set the system 200 predicted the user intended to input. For example, if the electronic processing system 200 predicts that the user intended the character set to be Latin-script alphabet characters, the electronic processing system 200 could display text in Latin-script alphabet characters. However, the text could include characters other than the character set the system 200 predicted the user intended to input. For example, if the electronic processing system 200 predicts that the user intended the character set to be Pinyin, the electronic processing system 200 could display a text in standard Chinese characters. The text could include a list of candidate words in one or more languages. For example, the text could include a list of words in standard Chinese characters based on the Pinyin characters.

[0070] FIG. 9A illustrates an example of limitations of a conventional input technique that only allows a single language at a time. The text 902 in FIG. 9A is what might appear on the display as the user is typing those Latin-script alphabet characters. If the system is in an English language only mode then the system will interpret all of the input as English. In this example, the Latin-script alphabet characters begin with “It’s based on,” which the user types in with the intent of the system interpreting the key inputs as Latin-script alphabet characters. However, the Latin-script alphabet characters end with “jiqixuexi.” However, in this example, the user’s intent of entering jiqixuexi is to phonetically input Chinese words for “machine learning.” If dual language is not enabled then the electronic device may simply output the Latin-script alphabet characters “jiqixuexi.” However, an embodiment of the electronic processing system 200 when in a dual language mode automatically predicts that the user intended the string “jiqixuexi” to be interpreted as Chinese language input. Then, the electronic processing system 200 displays the text for “machine learning” in standard Chinese characters (not depicted in FIG. 9A) instead of jiqixuexi. Moreover, in this dual-language input mode the user is not required to perform any explicit action to switch between English and Chinese.

[0071] FIG. 9B illustrates an example that combines both Chinese and English inputs. The input begins with Chinese characters 904 and ends with Latin-scriptalphabet characters “slide” 904. If the input mode is Chinese only then the system may interpret the keystrokes for “slide” as Pinyin characters, which may result in a failure to display any reasonable text. However, an embodiment of the electronic processing system 200 when in a dual language mode automatically predicts that the user intended the input string “slide” to be a form of the word slide in some language. Thus, the system may output the word “slide” on the display. Optionally, the system could also present the user with candidate words 908 as well. In FIG. 9B, seven candidate words 908 are presented. The user is allowed to select one of the candidates 908 for the input “slide” 906.

[0072] In some embodiments, the multi-language input detection performs automatic translation from a language entered by the user to another language. An example will be discussed in which the user enters some text in Chinese and others in English. The multi-language input detection recognizes that the English should be translated to Chinese. FIGs. 9C and 9D show such an example of automatic translation during multi-language input. In this example, the translation occurs midsentence. FIG. 9C shows that the user types in the English word “book” 920 in the middle of a Chinese sentence. This may occur if, for example, the user is primarily entering text in Chinese but has forgotten the proper Chinese word for “book.” Thus, rather than typing in, for example, Pinyin characters for “book” the user may type in the four Latin-script characters “b” “o” “o” “k”. The system 200 having multi-language processing 222 recognizes that the user likely would want the entire sentence to appear in Chinese text and automatically translates the English word “book” into Chinese. FIG. 9D shows the result in which the English word “book” has been translated to Chinese characters 930. In an embodiment, the user can select the text being displayed and translate from the displayed language to another preferred language.

[0073] FIG. 10 is a diagram illustrating application level and system level processing during an embodiment of multi-language input processing. The diagram in general is divided into actions of the user 1002, application 1004, and OS / kernel 1006. The application 1004 may allocate a buffer (input buffer) 248, which will be used to store values associated with the keyboard input. The user actions in this exampleare user selection of two different keys (K1 , K2) on a keyboard (physical or virtual). Each key selection may result in a character being added to the input buffer 248. For example, character C1 may be added to the input buffer 248 in response to K1 selection 1012 and character C2 may be added to the input buffer 248 in response to K2 selection 1032. The step 1024 of multi-language string processing may analyze the contents of the input buffer 248 to determine what text string (and in what language) to output in step 1026. More generally, step 1024 of multi-language string processing may analyze any values that represent the keyboard input. In one aspect the scan codes are mapped to characters in whatever languages are presently in use (e.g., English and Chinese). Then, step 1024 of multi-language string processing analyzes the English characters and the Chinese characters (e.g., Pinyin) to determine what text to output.

[0074] The following describes details of the interrupt handling. The user may press a key (K1 ) on the keyboard, which triggers a keyboard interrupt 1014. The keyboard interrupt is processed by interrupt handler 1016. The interrupt handler 1016 may perform the following: 1 ) save the context of job X; 2) get KTs scan code S1 ; 3) write the scan code S1 to the input buffer; 4) increment an input buffer pointer; and 5) resume job X. Step 1018 is a mapping of the scan code S1 to a character C1 . The scan code map 1020 contains the mapping between each scan code and a character. There may be different mappings for languages that use different character sets. In step 1018, the mapping may be to the character set of an initially predicted language, which may be changed later during the step of multi-language string processing 1024. Step 1022 includes a text input OS event initiation, which triggers multi-language string processing in step 1024. As noted, step 1024 could change the characters in the input buffer 248. User selection of key (K2) on the keyboard is similar, except the scan code (S2) for K2 is mapped to C2 in step 1019.

[0075] FIG. 11 is a flowchart of one embodiment of a process 1100 of multilanguage input detection. The process 1100 may be performed by the electronic processing system 200. Step 1102 includes receiving and buffering keyboard input. The keyboard input may be processed as described with respect to FIG. 10, wherein the input buffer 248 contains characters based on the scan codes for the selectedkeys. Thus, the input buffer 248 will contain a string of characters. In an embodiment, the characters are for an initially predicted language. Alternatively, the input buffer 248 may simply contain the scan codes, which may be mapped to characters in any of the user’s preferred languages.

[0076] Step 1104 includes classifying and predicting the string in the input buffer 248. Step 1104 may include predicting a language for the string in the input buffer 248. Step 1104 may include accessing a language model repository 262 and / or language dictionaries 226. In an embodiment, the system 200 will first determine if the string is found in a dictionary for any of the presently allowed input languages. Note that when looking for the string in a dictionary for a particular language, the string is mapped to characters used in the particular language. However, if the string is not found in a dictionary then the string may be input to a language model to infer a word in one of the presently allowed input languages. In an embodiment, step 1104 includes generating one or more candidate words in one or more languages. A probability may be estimated for each candidate word. Further details of generating candidate words and probabilities are discussed below.

[0077] Step 1106 includes a determination of whether to correct the input. The determination of whether to correct may be based on information in the context repository 224, such as user preferences. The context modeling 242, context manager 244, and / or preference manager 246 may be used in step 1106. If a correction to the inputs is desired then in step 1108 the content in the input buffer may be corrected. If a correction is performed the step 1104 may be repeated. One example of correcting the input is to correct a misspelling. Another example correction is to change the characters to characters used in a different language. For example, if the input buffer 248 initially contained Latin-script characters these may be changed to Pinyin characters. The example in FIG. 9A may be used as one example of a correction. In this example, initially the input buffer 248 may be populated with Latin- script characters for “jiqixuexi” due to the use of Latin-script characters at the beginning. However, the system 200 may determine that the user’s intent was for “jiqixuexi” to phonetically represent “machine learning” in Chinese. Therefore, thedevice may correct the input buffer 248 to replace the Latin-script characters jiqixuexi with Chinese Pinyin characters for “machine learning.”

[0078] Step 1110 includes generating an output. In an embodiment, the output for English may be the same as the characters in the input buffer 248. In an embodiment, if the input buffer 248 contains Pinyin characters then the system 200 generates standard Chinese characters for the Pinyin characters.

[0079] Step 1112 includes a determination of whether the input has ended. If there is further input, then step 1102 is performed again. When the input has ended, the process ends.

[0080] FIG. 12 is a flowchart of one embodiment of a process 1200 of multilanguage input detection with translation. The process 1200 is similar to process 1100, but adds steps 1240 and 1242. Step 1240 includes a determination of whether to translate the string in the input buffer 248. For example, with reference to the example in FIG. 9C, the string in the input buffer may include the English characters for “book”. However, the user may have been primarily entering Chinese (e.g., Pinyin) characters. If translation is desired then step 1242 includes translating the string in the input buffer 248. For example, book is translated to the Chinese word for book. Step 1110 includes generating the output. FIG. 9D shows an example output for step 1110.

[0081] In some embodiments, machine learning is used to develop one or more language models, which may be stored in the language model repository 262. These language models could be developed by the electronic processing system 200. However, the language models could be developed by another system and then loaded onto the electronic processing system 200. For example, the language models could be provided to the electronic processing system 200 over the network 280 or loaded onto the electronic processing system 200 prior to shipping the electronic processing system 200 for sale.

[0082] FIG. 13A is a flowchart of one embodiment of a process 1300 of developing language models for multi-language input detection. The process 1300 may bereferred to as a classification stage of machine learning. Process 1300 describes a general overview of how language models for multi-language input detection may be developed. Developing models for machine learning (or artificial intelligence) are well- known and hence process 1300 will not be described in great detail. Step 1302 includes training and data collection. The training data may include examples of inputs for each language for which the model is being trained. A specific input string could be for only one language or could be for more than one language (see examples in FIGs. 9A 9B, 9C inputs having more than one language). Step 1302 may include defining language types (e.g., English Chinese, Korean, etc.). Step 1302 may include collecting a training dataset of bi-lingual or multi-lingual input strings. Optionally, existing datasets may be used instead of, or in addition to, collecting datasets.

[0083] Step 1304 includes data preprocessing. Step 1304 may include data filtering and tokenization. Step 1304 may also include training and validation data separation. Step 1306 includes data annotation. Step 1306 may include defining labels and data labeling. Labeling data is known to those of ordinary skill in the fields of machine training such as Artificial Intelligence models. Step 1308 include feature extraction. The feature extraction may be based on N-gram, word embedding, phrase embedding, etc.

[0084] Step 1310 includes model training and evaluation. Step 1310 may include training the language model on the training dataset(s). Step 1310 may include evaluating the language model on an evaluation dataset. Step 1312 includes model tuning. Step 1313 includes compressing the model. Step 1314 includes storing the language model in the language model repository 262.

[0085] FIG. 13B is a flowchart of one embodiment of a process 1350 of fine-tuning an existing pre-trained model. Step 1352 includes loading a pre-trained model. Step 1354 includes preparing data. Preparing the data may include processing and formatting user interaction data for training. The user interaction data may include, but is not limited to, user typing and selection history. Step 1356 includes setting training parameters. Setting training parameters may include, but is not limited to, configuration of hyperparameters and training settings. Step 1358 includes fine-tuningthe model. Step 1358 includes a training process in which the model is adapted to a specific task. Step 1360 includes evaluating the model. Step 1360 may include assessing and validating model performance. Step 1362 includes saving the model to the repository. Step 1362 therefore preserves the fine-tuned model for future use. Optionally the fine-tuned model may be compressed prior to storage in the repository.

[0086] FIG. 14 is a flowchart of one embodiment of a process 1400 of using a language model to classify and predict a language input to the electronic processing system 200. The language model developed in process 1300 may be used in process 1400. Process 1400 may be used in what is commonly referred to as an “interface” stage” of artificial intelligence. Process 1400 may be performed on word, phrase, sentence, or other level. Step 1402 includes data preprocessing. The data preprocessing may include data filtering, data segmentation or separation, tokenization, training, validation, etc. In one embodiment, step 1402 includes mapping scan codes to characters in the preferred languages. Step 1404 includes feature extraction. Feature extraction includes identifying and extracting relative information or patterns from the pre-processed input data. Feature extraction is well-known to those of ordinary skill and will not be described in detail. Step 1406 includes classification and prediction using the language model. Step 1406 may include inputting the input string (after mapping the characters in the preferred language) to the language model to “infer” a result. The result may include one or more candidate words in each of one or more preferred languages. In some embodiments, a probability is inferred for each candidate word. The probability may refer to a probability that user intended the input string to be that candidate word in that language.

[0087] FIG. 15 is a flowchart of an embodiment of a process of context aware input string processing and prediction. In this example, the two possible input languages are Chinese and English. The process operates on a string of inputs from, for example, the input buffer 248. Step 1502 includes word line segmentation and preprocessing of the input string. Step 1504 includes extracting features. These features may include, for example, N-gram, word embedding, and phrase embedding that may map to the characters that appear in the Pinyin dictionary 226a or the Englishdictionary 226b. However, N-gram, word embedding, and phrase embedding are examples of possible features. Various different features used in natural language processing may be used. Step 1506 includes word level classification of the input. The Pinyin dictionary 226a and English dictionary 226b are used for the word level classification. Step 1508 includes a determination of whether the input is found in one or both dictionaries 226a, 226b. If the input is not found in one of the dictionaries 226a, 226b, then the language model (LM) processing is performed in step 1510 to predict a language for the input. The LM processing uses the language model in the language model repository 262. Next, the word level processing step 1506 takes action based on whether the input could be either English or Pinyin (step 1512), Pinyin (step 1514), English (step 1516) or a symbol (step 1518). If the input could be either English or Pinyin then context processing is performed in step 1520 to select between English and Pinyin. If the input is predicted to be Pinyin (step 1522 is y) then standard Chinese characters are generated for the Pinyin in step 1524. Then, the standard Chinese characters are output in step 1526. However, if the input is predicted to be English (step 1524 is n) then English characters will be output in step 1526 Returning now to step 1514, if it is determined that the input is Pinyin (step 1514 is y), then standard Chinese characters are generated for the Pinyin in step 1524. Then, the standard Chinese characters are output in step 1526. Returning now to step 1516, if it is determined that the input is English (step 1516 is y) then English characters are output in step 1526. Returning now to step 1518, if it is determined that the input is a symbol (step 1518 is y) then the symbol is output in step 1526.

[0088] In some embodiments, the multi-language processing accesses data pertaining to context and user preferences to predict the language and / or word being entered in the input. FIG. 16 shows examples of types of data that may be accessed by the multi-language processing 222 for an example in which the user has selected English and Chinese as two possible input languages. This data includes the user’s English vocabulary 1602, the user’s Chinese vocabulary 1404, all English vocabulary 1606, all Chinese vocabulary 1608, a pool of English words used in current applications or documents 1610, a pool of Chinese words used in current applications or documents 1612, and topics in current applications or documents 1614. The user’sEnglish vocabulary 1602 and the user’s Chinese vocabulary 1404 could include, but are not limited to, special dictionaries such as dictionaries of technical words, legal words, etc. The all English vocabulary 1606 and the all Chinese vocabulary 1608 could include, but are not limited to, standard language dictionaries (e.g., the Oxford English Dictionary, etc.). The current applications or documents refers to applications and / or documents currently in use on or by the electronic processing system 200. Note that this could include an Internet- or cloud-based application or document in use on or by the electronic processing system 200.

[0089] In some embodiments, a weighting between two or more languages is used to facilitate the prediction of the language. This weighting may be used by the multilanguage processing 222 to predict the input language and / or candidate words. FIG. 17 illustrates an example of weighting languages for multi-language input detection. In FIG. 17, User A has a 30% weighting to Chinese and a 90% weighting to English, whereas User B has a 55% weighting to Chinese and a 45% weighting to English. The weighting may be fixed or dynamic. In a fixed weighting embodiment, the system receives a manual user selection of the weight, as well as a user selection to lock the weight to the selected weight value. In a dynamic weighting embodiment, the system receives an initial user selection of the weight and automatically updates the weight based on implicit user actions such as user typing and selection history. An implicit user action is an action from which a weighting preference may be inferred, but is not an explicit request from the user to set or adjust the weighting. In a dynamic weighting embodiment, the user is allowed to make an explicit manual adjustment to the weight at any time. In one embodiment, the users are allowed to select the weighting. In one embodiment, the user can “slide” the bar 1702a, 1702b to adjust the weight. In one embodiment, the system 200 automatically adjusts the weighting based on user preferences and / or profiles. For example, the system 200 may base the weighing on the user’s input language history, the user’s history of languages in accessed documents, applications, etc. The weighting can be set per application. For example, the weighting for an application for online Web meetings can be different than the weighting for a software development tool.

[0090] FIG. 18 is a flowchart of an embodiment of a process 1800 of generating candidate words in the potential input languages. In this example, the potential input languages are English and Chinese, but other languages are possible. Note that process 1800 may use, but is not limited to, the sources depicted in FIG. 16. Step 1802 includes accessing an input string from, for example, the input buffer 248. This input string could include the scan codes from the keyboard, which may be mapped to the characters of the potential input languages. If the input string has already been mapped to characters of one of the potential input languages, the system 200 may then map the content of the input buffer 248 to characters used in other potential input languages.

[0091] Step 1804 includes predicting English word(s) based on the user’s English vocabulary 1602. Step 1806 includes predicting Chinese word(s) based on the user’s Chinese vocabulary 1604. Step 1808 includes predicting English word(s) based on all English vocabulary 1606. Step 1810 includes predicting Chinese word(s) based on the all Chinese vocabulary 1608. Step 1812 includes predicting English word(s) based on a present context. Step 1812 may be based on the pool of English words used in current applications and documents 1610. Step 1812 may be based on the topics in current applications and documents 1610. Step 1814 includes predicting Chinese word(s) based on a present context. Step 1814 may be based on the pool of Chinese words used in current applications and documents 1612. Step 1814 may be based on the topics in current applications and documents 1610. Step 1816 includes generating English word candidates with probabilities based on the English word(s) predicted in steps 1804, 1808, and 1812. Step 1820 includes generating Chinese word candidates with probabilities based on the Chinese word(s) predicted in steps 1806, 1810, and 1814. Step 1822 includes weighing the English and Chinese word candidates based on the user preferences. Step 1822 may use a weighting provided by the user as discussed in connection with FIG. 17. Step 1824 includes updating the English and Chinese word candidates with probabilities based on the user weighting.

[0092] FIG. 19 is a flowchart of an embodiment of a process 1900 of outputting a word based on the candidates generated in process 1800. Step 1902 includes a determination of whether the candidate with the highest probability has a probabilitygreater than a threshold. If so, then this candidate is “auto-confirmed” and added in step 1904 to a document (or other application) to which the user is inputting text. Step 1906 includes a determination of whether there is more input. If not, then the process ends. Otherwise, in step 1908 the input buffer 248 is cleared and the system waits for the next input. Step 1910 includes returning to candidate word generation (e.g., process 1800).

[0093] Returning to the discussion of step 1902, if the candidate with the highest probability does not have a probability greater than the threshold, then in step 1912 a list of candidate words is displayed in order of the probabilities. Step 1914 includes a determination of whether the user has selected one of the candidates or has continued typing. If the user has selected one of the candidate words, then in step 1916 the selected candidate word is added to the document to which the user is inputting text. The process 1900 then continues at step 1906 as already described. If the user does not select one of the candidate words then in step 1910 candidate generation continues to be performed (e.g., process 1800 is performed).

[0094] FIG. 20 is a schematic diagram of a general-purpose computer system 2000 upon which embodiments of multi-language processing as described herein may be performed. The general-purpose computer system 2000 includes a processor 2002 (which may be referred to as a central processor unit or CPU) that is in communication with memory devices including secondary storage 2004, and memory, such as ROM 2006 and RAM 2008, input / output (I / O) devices 2010, and a network 2012, such as the Internet or any other well-known type of network, that may include network connectivity devices, such as a network interface. Although illustrated as a single processor, the processor 2002 is not so limited and may comprise multiple processors. The processor 2002 may be implemented as one or more CPU chips, cores (e.g., a multi-core processor), FPGAs, ASICs, and / or DSPs, and / or may be part of one or more ASICs. The processor 2002 may be configured to implement any of the schemes described herein. The processor 2002 may be implemented using hardware, software, or both.

[0095] The secondary storage 2004 is typically comprised of one or more solid state drives (SSDs) or the like used for non-volatile storage of data and as an overflow data storage device if the RAM 2008 is not large enough to hold all working data. The secondary storage 2004 may be used to store programs that are loaded into the RAM 2008 when such programs are selected for execution. The ROM 2006 is used to store instructions and perhaps data that are read during program execution. The ROM 2006 is a non-volatile memory device that typically has a small memory capacity relative to the larger memory capacity of the secondary storage 2004. The RAM 2008 is used to store volatile data and perhaps to store instructions. Access to both the ROM 2006 and the RAM 2008 is typically faster than to the secondary storage 2004. At least one of the secondary storage 2004 or RAM 2008 may be configured to store multi-language processing instructions 222, context repository 224, language dictionaries 226, language model 262, or other information disclosed herein.

[0096] It is understood that by programming and / or loading executable instructions onto the general-purpose computer system 2000, at least one of the processor 2002 or the memory (e.g., ROM 2006, RAM 2008) are changed, transforming the general- purpose network component or computer system 2000 in part into a particular machine or apparatus, e.g., a router, having the novel functionality taught by the present disclosure. Similarly, it is understood that by programming and / or loading executable instructions onto the general-purpose network component or computer system 2000, at least one of the processor 2002, the ROM 2006, and the RAM 2008 are changed, transforming the general-purpose network component or computer system 2000 in part into a particular machine or apparatus, e.g., a router, having the novel functionality taught by the present disclosure. It is fundamental to the electrical engineering and software engineering arts that functionality that can be implemented by loading executable software into a computer can be converted to a hardware implementation by well-known design rules. Decisions between implementing a concept in software versus hardware typically hinge on considerations of stability of the design and numbers of units to be produced rather than any issues involved in translating from the software domain to the hardware domain. Generally, a design that is still subject to frequent change may be preferred to be implemented in software, because re-spinning a hardware implementation is more expensive than re-spinning a software design. Generally, a design that is stable that will be produced in large volume may be preferred to be implemented in hardware, for example in an ASIC, because for large production runs the hardware implementation may be less expensive than the software implementation. Often a design may be developed and tested in a software form and later transformed, by well-known design rules, to an equivalent hardware implementation in an application specific integrated circuit that hardwires the instructions of the software. In the same manner as a machine controlled by a new ASIC is a particular machine or apparatus, likewise a computer that has been programmed and / or loaded with executable instructions may be viewed as a particular machine or apparatus.

[0097] The technology described herein can be implemented using hardware, firmware, software, or a combination of these. The software used is stored on one or more of the processor readable storage devices described above to program one or more of the processors to perform the functions described herein. The processor readable storage devices can include computer readable media such as volatile and non-volatile media, removable and non-removable media. By way of example, and not limitation, computer readable media may comprise computer readable storage media and communication media. Computer readable storage media may be implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer readable storage media include RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. A computer readable medium or media does (do) not include propagated, modulated or transitory signals.

[0098] Communication media typically embodies computer readable instructions, data structures, program modules or other data in a propagated, modulated or transitory data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means asignal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as RF and other wireless media. Combinations of any of the above are also included within the scope of computer readable media.

[0099] In alternative embodiments, some or all of the software can be replaced by dedicated hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs), Applicationspecific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), special purpose computers, etc. In one embodiment, software (stored on a storage device) implementing one or more embodiments is used to program one or more processors. The one or more processors can be in communication with one or more computer readable media / storage devices, peripherals and / or communication interfaces.

[0100] Aspects of the present disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable instruction execution apparatus, create a mechanism for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0101] For the purposes of this document, it should be noted that the dimensions of the various features depicted in the figures may not necessarily be drawn to scale.

[0102] For purposes of this document, reference in the specification to “an embodiment,” “one embodiment,” “some embodiments,” or “another embodiment” may be used to describe different embodiments or the same embodiment.

[0103] For the purposes of this document, a connection may be a direct connection or an indirect connection (e.g., via one or more other parts). In some cases, when an element is referred to as being connected or coupled to another element, the element may be directly connected to the other element or indirectly connected to the other element via intervening elements. When an element is referred to as being directly connected to another element, then there are no intervening elements between the element and the other element. Two devices are “in communication” if they are directly or indirectly connected so that they can communicate electronic signals between them.

[0104] Although the present disclosure has been described with reference to specific features and embodiments thereof, it is evident that various modifications and combinations can be made thereto without departing from the scope of the disclosure. The specification and drawings are, accordingly, to be regarded simply as an illustration of the disclosure as defined by the appended claims, and are contemplated to cover any and all modifications, variations, combinations, or equivalents that fall within the scope of the present disclosure.

[0105] It is understood that the present subject matter may be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this subject matter will be thorough and complete and will fully convey the disclosure to those skilled in the art. Indeed, the subject matter is intended to cover alternatives, modifications, and equivalents of these embodiments, which are included within the scope and spirit of the subject matter as defined by the appended claims. Furthermore, in the following detailed description of the present subject matter, numerous specific details are set forth in order to provide a thorough understanding of the present subject matter. However, it will be clear to those of ordinary skill in the art that the present subject matter may be practiced without such specific details.

[0106] The description of the present disclosure has been presented for purposes of illustration and description but is not intended to be exhaustive or limited to the disclosure in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the disclosure. The aspects of the disclosure herein were chosen and described in order to best explain the principles of the disclosure and the practical application, and to enable others of ordinary skill in the art to understand the disclosure with various modifications as are suited to the particular use contemplated.

[0107] For purposes of this document, each process associated with the disclosed technology may be performed continuously and by one or more computing devices. Each step in a process may be performed by the same or different computing devices as those used in other steps, and each step need not necessarily be performed by a single computing device.

[0108] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

CLAIMSWhat is claimed is:1 . A method for providing a multi-language input mode, the method comprising: receiving a string of keyboard inputs in a user interface associated with an electronic processing system when the electronic processing system is in a mode that allows the keyboard inputs to represent two or more different human languages associated with a corresponding two or more different sets of characters; selecting, by the electronic processing system, a human language for the string of keyboard inputs out of the two or more different human languages; and displaying, by the electronic processing system, text in a display associated with the electronic processing system in the selected human language.

2. The method of claim 1 , wherein: receiving the string of keyboard inputs in the user interface comprises receiving a scan code for each keyboard input in the string; and selecting the human language for the string of keyboard inputs comprises processing characters in the two or more different human languages to which the scan codes map.

3. The method of claim 1 or 2, wherein selecting, by the electronic processing system, the human language for the string of keyboard inputs out of the two or more different human languages comprises: accessing, from non-transitory memory, two or more dictionaries that correspond to the two or more human languages; and selecting a first human language responsive to the string of keyboard inputs mapping to a word in a first dictionary of the two or more dictionaries, the first dictionary corresponding to the first human language.

4. The method of claim 1 or 2, wherein selecting, by the electronic processing system, the human language for the string of keyboard inputs out of the two or more different human languages comprises: inferring a word in a first human language responsive to inputting the string of keyboard inputs to an artificial intelligence language model.

5. The method of claim 1 or 2, wherein selecting, by the electronic processing system, the human language for the string of keyboard inputs out of the two or more different human languages comprises: accessing, from non-transitory memory, two or more dictionaries that correspond to the two or more human languages; determining whether the string of keyboard inputs map to a word in any of the two or more dictionaries; inputting the string of keyboard inputs to an artificial intelligence language model responsive to the string of keyboard inputs not mapping to any words in the two or more dictionaries; and inferring a word in a first human language responsive to inputting the string of keyboard inputs to the artificial intelligence language model.

6. The method of any of claims 1 to 2, wherein selecting, by the electronic processing system, the human language for the string of keyboard inputs out of the two or more different human languages comprises: accessing, from non-transitory memory, two or more dictionaries that correspond to the two or more human languages; determining whether the string of keyboard inputs map to a word in each of two of the two or more dictionaries that correspond to a first human language and a second human language; and selecting between the first human language and the second human language based on a context in which the keyboard input is received responsive to the string of keyboard inputs mapping to a word in each of two of the two or more dictionaries.

7. The method of any of claims 1 to 6, wherein selecting, by the electronic processing system, the human language for the string of keyboard inputs out of the two or more different human languages comprises: accessing, from non-transitory memory, a first vocabulary for a user for a first human language; accessing, from non-transitory memory, a second vocabulary for the user for a second human language; and selecting between the first human language and the second human language based on a matching between the string of keyboard inputs and content in the first vocabulary and the second vocabulary.

8. The method of any of claims 1 to 7, wherein selecting, by the electronic processing system, the human language for the string of keyboard inputs out of the two or more different human languages comprises: accessing, from non-transitory memory, a first pool of words in a first human language present in documents and applications in present use on the electronic processing system; accessing, from non-transitory memory, a second pool of words in a second human language present in the documents and the applications in present use on the electronic processing system; and selecting between the first human language and the second human language based on a matching between the string of keyboard inputs and content in the first pool and the second pool.

9. The method of any of claims 1 to 8, wherein selecting, by the electronic processing system, the human language for the string of keyboard inputs out of the two or more different human languages comprises: accessing, from non-transitory memory, topics in a first human language present in documents and applications in present use on the electronic processing system;accessing, from non-transitory memory, topics in a second human language present in the documents and the applications in present use on the electronic processing system; and selecting between the first human language and the second human language based on the topics in the first human language present and the second human language.

10. The method of any of claims 1 to 9, wherein selecting, by the electronic processing system, the human language for the string of keyboard inputs out of the two or more different human languages comprises: determining a first probability that the string of keyboard inputs corresponds to a first human language of the two or more human languages; determining a second probability that the string of keyboard inputs corresponds to a second human language of the two or more human languages; and selecting between the first human language and the second human language based on the first probability and the second probability.11 . The method of any of claims 1 to 10, wherein selecting, by the electronic processing system, the human language for the string of keyboard inputs out of the two or more different human languages comprises: determining a probability that the string of keyboard inputs corresponds to a first human language of the two or more human languages; and selecting the first human language responsive to the probability being greater than a threshold.

12. The method of any of claims 1 to 10, wherein selecting, by the electronic processing system, the human language for the string of keyboard inputs out of the two or more different human languages comprises: determining one or more candidate words in a first human language of the two or more human languages for the string of keyboard inputs; determining one or more candidate words in a second human language of the two or more human languages for the string of keyboard inputs;determining a probability for each of one or more candidate words in first human language for the string of keyboard inputs; and determining a probability for each of one or more candidate words in second human language for the string of keyboard inputs.

13. The method of claim 12, wherein selecting, by the electronic processing system, the human language for the string of keyboard inputs out of the two or more different human languages comprises: selecting, by the electronic processing system, a candidate word of the one or more candidate words in the first human language and the one or more candidate words in the second human language having a highest probability.

14. The method of claim 12, further comprising: receiving a user selection of a weighting between the first human language and the second human language; and determining the probability for each of the one or more candidate words in the first human language and determining the probability for each of the one or more candidate words in the second human language based on the user selection of the weighting between the first human language and the second human language.

15. The method of claim 12, further comprising: determining, by the electronic processing system, a weighting between the first human language and the second human language for a user; and determining the probability for each of the one or more candidate words in the first human language and determining the probability for each of the one or more candidate words in the second human language based on the weighting for the user.

16. The method of claim 14 or 15, further comprising: fixing the weighting responsive to a user input to fix the weighting.

17. The method of claim 16, further comprising: dynamically updating the weighting responsive to implicit user actions.

18. The method of claim 17, further comprising: updating the weighting responsive to an explicit user request to adjust the weighting.

19. The method of any of claims 12 or 14 to 18, further comprising: displaying, on a display associated with the electronic processing system, one or more candidates in the first human language and one or more candidates in the second human language; and receiving a user selection of one of the one or more candidates in the first human language and the one or more candidates in the second human language.

20. The method of claim 19, further comprising: displaying the one or more candidates in the first human language and one or more candidates in the second human language in order of their probabilities.21 . The method of any of claims 1 to 20, wherein selecting the human language for the string of keyboard inputs out of the two or more different human languages comprises automatically translating from a language of the string of keyboard inputs to another language.

22. The method of any of claims 1 to 21 , further comprising: accessing a user selection of the two or more languages for the mode that allows the keyboard inputs to represent the two or more different human languages; and entering the mode responsive to accessing the user selection of the two or more languages for the mode.

23. The method of any of claims 1 to 22, further comprising: accessing a user selection of a plurality of languages to be used in a single language mode; andselecting between the plurality of languages as the two or more different human languages associated with the corresponding two or more different sets of characters.

24. The method of any of claims 1 to 23, further comprising: accessing user preferences; and determining the two or more different human languages based on the user preferences.

25. An electronic processing system for multi-language keyboard input, the system comprising: an input / output interface; a storage medium comprising computer program instructions; and one or more processors coupled to communicate with the input / output interface and the storage medium, wherein the one or more processors execute the instructions to: receive a string of keyboard inputs in the input / output interface in a mode that allows the keyboard inputs to represent two or more different human languages associated with a corresponding two or more different sets of characters; select a human language for the string of keyboard inputs out of the two or more different human languages; and display text in a display associated with the input / output interface in the selected human language.

26. The system of claim 25, wherein the one or more processors further execute the instructions to: receive a scan code for each keyboard input in the string; and process characters in the two or more different human languages to which the scan codes map to select the human language for the string of keyboard inputs.

27. The system of claim 25 or 26, wherein the instructions that cause the one or more processors to select the human language for the string of keyboard inputs outof the two or more different human languages further comprise instructions that cause the one or more processors to: access, from non-transitory memory, two or more dictionaries that correspond to the two or more human languages; and select a first human language responsive to the string of keyboard inputs mapping to a word in a first dictionary of the two or more dictionaries, the first dictionary corresponding to the first human language.

28. The system of claim 25 or 26, wherein the instructions that cause the one or more processors to select the human language for the string of keyboard inputs out of the two or more different human languages further comprise instructions that cause the one or more processors to: infer a word in a first human language responsive to inputting the string of keyboard inputs to an artificial intelligence language model.

29. The system of claim 25 or 26, wherein the instructions that cause the one or more processors to select the human language for the string of keyboard inputs out of the two or more different human languages further comprise instructions that cause the one or more processors to: access, from non-transitory memory, two or more dictionaries that correspond to the two or more human languages; determine whether the string of keyboard inputs map to a word in any of the two or more dictionaries; input the string of keyboard inputs to an artificial intelligence language model responsive to the string of keyboard inputs not mapping to any words in the two or more dictionaries; and infer a word in a first human language responsive to inputting the string of keyboard inputs to the artificial intelligence language model.

30. The system of claim 25 or 26, wherein the instructions that cause the one or more processors to select the human language for the string of keyboard inputs outof the two or more different human languages further comprise instructions that cause the one or more processors to: access, from non-transitory memory, two or more dictionaries that correspond to the two or more human languages; determine whether the string of keyboard inputs map to a word in each of two of the two or more dictionaries that correspond to a first human language and a second human language; and selecting between the first human language and the second human language based on a context in which the keyboard input is received responsive to the string of keyboard inputs mapping to a word in each of two of the two or more dictionaries.31 . The system of any of claims 25 to 30, wherein the instructions that cause the one or more processors to select the human language for the string of keyboard inputs out of the two or more different human languages further comprise instructions that cause the one or more processors to: access, from non-transitory memory, a first vocabulary for a user for a first human language; access, from non-transitory memory, a second vocabulary for the user for a second human language; and select between the first human language and the second human language based on a matching between the string of keyboard inputs and content in the first vocabulary and the second vocabulary.

32. The system of any of claims 25 to 31 , wherein the instructions that cause the one or more processors to select the human language for the string of keyboard inputs out of the two or more different human languages further comprise instructions that cause the one or more processors to: access, from non-transitory memory, a first pool of words in a first human language present in documents and applications in present use on the electronic processing system;access, from non-transitory memory, a second pool of words in a second human language present in the documents and the applications in present use on the electronic processing system; and select between the first human language and the second human language based on a matching between the string of keyboard inputs and content in the first pool and the second pool.

33. The system of any of claims 25 to 32, wherein the instructions that cause the one or more processors to select the human language for the string of keyboard inputs out of the two or more different human languages further comprise instructions that cause the one or more processors to: access, from non-transitory memory, topics in a first human language present in documents and applications in present use on the electronic processing system; access, from non-transitory memory, topics in a second human language present in the documents and the applications in present use on the electronic processing system; and select between the first human language and the second human language based on the topics in the first human language present and the second human language.

34. The system of any of claims 25 to 33, wherein the instructions that cause the one or more processors to select the human language for the string of keyboard inputs out of the two or more different human languages further comprise instructions that cause the one or more processors to: determine a first probability that the string of keyboard inputs corresponds to a first human language of the two or more human languages; determine a second probability that the string of keyboard inputs corresponds to a second human language of the two or more human languages; and select between the first human language and the second human language based on the first probability and the second probability.

35. The system of any of claims 25 to 34, wherein the instructions that cause the one or more processors to select the human language for the string of keyboard inputs out of the two or more different human languages further comprise instructions that cause the one or more processors to: determine a probability that the string of keyboard inputs corresponds to a first human language of the two or more human languages; and select the first human language responsive to the probability being greater than a threshold.

36. The system of any of claims 25 to 34, wherein the instructions that cause the one or more processors to select the human language for the string of keyboard inputs out of the two or more different human languages further comprise instructions that cause the one or more processors to: determine one or more candidate words in a first human language of the two or more human languages for the string of keyboard inputs; determine one or more candidate words in a second human language of the two or more human languages for the string of keyboard inputs; determine a probability for each of one or more candidate words in first human language for the string of keyboard inputs; and determine a probability for each of one or more candidate words in second human language for the string of keyboard inputs.

37. The system of claim 36, wherein the instructions that cause the one or more processors to select the human language for the string of keyboard inputs out of the two or more different human languages further comprise instructions that cause the one or more processors to: select a candidate word of the one or more candidate words in the first human language and the one or more candidate words in the second human language having a highest probability.

38. The system of claim 36, wherein the one or more processors further execute the instructions to:receive a user selection of a weighting between the first human language and the second human language; and determine the probability for each of the one or more candidate words in the first human language and determining the probability for each of the one or more candidate words in the second human language based on the user selection of the weighting between the first human language and the second human language.

39. The system of claim 36, wherein the one or more processors further execute the instructions to: determine a weighting between the first human language and the second human language for a user; and determining the probability for each of the one or more candidate words in the first human language and determining the probability for each of the one or more candidate words in the second human language based on the weighting for the user.

40. The system of claim 38 or 39, wherein the one or more processors further execute the instructions to: fix the weighting responsive to a user input to fix the weighting.41 . The system of claim 40, wherein the one or more processors further execute the instructions to: dynamically update the weighting responsive to implicit user actions.

42. The system of claim 41 , wherein the one or more processors further execute the instructions to: update the weighting responsive to an explicit user request to adjust the weighting.

43. The system of any of claims 39, 41 , or 42, wherein the one or more processors further execute the instructions to:display, on the display associated with the input / output interface, one or more candidates in the first human language and one or more candidates in the second human language; and receive a user selection of one of the one or more candidates in the first human language and the one or more candidates in the second human language.

44. The system of claim 43, wherein the one or more processors further execute the instructions to: display the one or more candidates in the first human language and one or more candidates in the second human language in order of their probabilities.

45. The system of any of claims 25 to 44, wherein selecting the human language for the string of keyboard inputs out of the two or more different human languages comprises automatically translating from a language of the string of keyboard inputs to another language.

46. The system of any of claims of any of claims 25 to 45, wherein the one or more processors further execute the instructions to: access a user selection of the two or more languages for the mode that allows the keyboard inputs to represent the two or more different human languages; and enter the mode responsive to accessing the user selection of the two or more languages for the mode.

47. The system of any of claims of any of claims 25 to 45, wherein the one or more processors further execute the instructions to: access a user selection of a plurality of languages to be used in a single language mode; and select between the plurality of languages as the two or more different human languages associated with the corresponding two or more different sets of characters.

48. The system of any of claims of any of claims 25 to 45, wherein the one or more processors further execute the instructions to:access user preferences; and determine the two or more different human languages based on the user preferences.

49. A non-transitory computer-readable medium storing computer instructions for multi-language keyboard input, that when executed by one or more processors, cause the one or more processors to: receive a string of keyboard inputs in a mode that allows the keyboard inputs to represent two or more different human languages associated with a corresponding two or more different sets of characters; select a human language for the string of keyboard inputs out of the two or more different human languages; and display text in a display screen in the selected human language.

50. The non-transitory computer-readable medium of claim 49, wherein the one or more processors further execute the instructions to: receive a scan code for each keyboard input in the string; and process characters in the two or more different human languages to which the scan codes map to select the human language for the string of keyboard inputs.51 . The non-transitory computer-readable medium of claim 49 or 50, wherein the instructions that cause the one or more processors to select the human language for the string of keyboard inputs out of the two or more different human languages further comprise instructions that cause the one or more processors to: access, from non-transitory memory, two or more dictionaries that correspond to the two or more human languages; and select a first human language responsive to the string of keyboard inputs mapping to a word in a first dictionary of the two or more dictionaries, the first dictionary corresponding to the first human language.

52. The non-transitory computer-readable medium of claim 49 or 50, wherein the instructions that cause the one or more processors to select the human language forthe string of keyboard inputs out of the two or more different human languages further comprise instructions that cause the one or more processors to: infer a word in a first human language responsive to inputting the string of keyboard inputs to an artificial intelligence language model.

53. The non-transitory computer-readable medium of claim 49 or 50, wherein the instructions that cause the one or more processors to select the human language for the string of keyboard inputs out of the two or more different human languages further comprise instructions that cause the one or more processors to: access, from non-transitory memory, two or more dictionaries that correspond to the two or more human languages; determine whether the string of keyboard inputs map to a word in any of the two or more dictionaries; input the string of keyboard inputs to an artificial intelligence language model responsive to the string of keyboard inputs not mapping to any words in the two or more dictionaries; and infer a word in a first human language responsive to inputting the string of keyboard inputs to the artificial intelligence language model.

54. The non-transitory computer-readable medium of medium of claim 49 or 50, wherein the instructions that cause the one or more processors to select the human language for the string of keyboard inputs out of the two or more different human languages further comprise instructions that cause the one or more processors to: access, from non-transitory memory, two or more dictionaries that correspond to the two or more human languages; determine whether the string of keyboard inputs map to a word in each of two of the two or more dictionaries that correspond to a first human language and a second human language; and selecting between the first human language and the second human language based on a context in which the keyboard input is received responsive to the string of keyboard inputs mapping to a word in each of two of the two or more dictionaries.

55. The non-transitory computer-readable medium of any of claims 49 to 54, wherein the instructions that cause the one or more processors to select the human language for the string of keyboard inputs out of the two or more different human languages further comprise instructions that cause the one or more processors to: access, from non-transitory memory, a first vocabulary for a user for a first human language; access, from non-transitory memory, a second vocabulary for the user for a second human language; and select between the first human language and the second human language based on a matching between the string of keyboard inputs and content in the first vocabulary and the second vocabulary.

56. The non-transitory computer-readable medium of any of claims 49 to 55, wherein the instructions that cause the one or more processors to select the human language for the string of keyboard inputs out of the two or more different human languages further comprise instructions that cause the one or more processors to: access, from non-transitory memory, a first pool of words in a first human language present in documents and applications in present use on an electronic processing system; access, from non-transitory memory, a second pool of words in a second human language present in the documents and the applications in present use on an electronic processing system; and select between the first human language and the second human language based on a matching between the string of keyboard inputs and content in the first pool and the second pool.

57. The non-transitory computer-readable medium of any of claims 49 to 56, wherein the instructions that cause the one or more processors to select the human language for the string of keyboard inputs out of the two or more different human languages further comprise instructions that cause the one or more processors to: access, from non-transitory memory, topics in a first human language present in documents and applications in present use on the electronic processing system;access, from non-transitory memory, topics in a second human language present in the documents and the applications in present use on an electronic processing system; and select between the first human language and the second human language based on the topics in the first human language present and the second human language.

58. The non-transitory computer-readable medium of any of claims 49 to 57, wherein the instructions that cause the one or more processors to select the human language for the string of keyboard inputs out of the two or more different human languages further comprise instructions that cause the one or more processors to: determine a first probability that the string of keyboard inputs corresponds to a first human language of the two or more human languages; determine a second probability that the string of keyboard inputs corresponds to a second human language of the two or more human languages; and select between the first human language and the second human language based on the first probability and the second probability.

59. The non-transitory computer-readable medium of any of claims 49 to 58, wherein the instructions that cause the one or more processors to select the human language for the string of keyboard inputs out of the two or more different human languages further comprise instructions that cause the one or more processors to: determine a probability that the string of keyboard inputs corresponds to a first human language of the two or more human languages; and select the first human language responsive to the probability being greater than a threshold.

60. The non-transitory computer-readable medium of any of claims 49 to 59, wherein the instructions that cause the one or more processors to select the human language for the string of keyboard inputs out of the two or more different human languages further comprise instructions that cause the one or more processors to:determine one or more candidate words in a first human language of the two or more human languages for the string of keyboard inputs; determine one or more candidate words in a second human language of the two or more human languages for the string of keyboard inputs; determine a probability for each of one or more candidate words in first human language for the string of keyboard inputs; and determine a probability for each of one or more candidate words in second human language for the string of keyboard inputs.61 . The non-transitory computer-readable medium of claim 60, wherein the instructions that cause the one or more processors to select the human language for the string of keyboard inputs out of the two or more different human languages further comprise instructions that cause the one or more processors to: select a candidate word of the one or more candidate words in the first human language and the one or more candidate words in the second human language having a highest probability.

62. The non-transitory computer-readable medium of claim 60, wherein the one or more processors further execute the instructions to: receive a user selection of a weighting between the first human language and the second human language; and determine the probability for each of the one or more candidate words in the first human language and determining the probability for each of the one or more candidate words in the second human language based on the user selection of the weighting between the first human language and the second human language.

63. The non-transitory computer-readable medium of claim 60, wherein the one or more processors further execute the instructions to: determine a weighting between the first human language and the second human language for a user; anddetermining the probability for each of the one or more candidate words in the first human language and determining the probability for each of the one or more candidate words in the second human language based on the weighting for the user.

64. The non-transitory computer-readable medium of claim 62 or 63, wherein the one or more processors further execute the instructions to fix the weighting responsive to a user input to fix the weighting.

65. The non-transitory computer-readable medium of claim 64, wherein the one or more processors further execute the instructions to: dynamically update the weighting responsive to implicit user actions.

66. The non-transitory computer-readable medium of claim 65, wherein the one or more processors further execute the instructions to update the weighting responsive to an explicit user request to adjust the weighting.

67. The non-transitory computer-readable medium of claim 63, 65, or 66, wherein the one or more processors further execute the instructions to: display, on the display screen, one or more candidates in the first human language and one or more candidates in the second human language; and receive a user selection of one of the one or more candidates in the first human language and the one or more candidates in the second human language.

68. The non-transitory computer-readable medium of claim 67, wherein the one or more processors further execute the instructions to: display the one or more candidates in the first human language and one or more candidates in the second human language in order of their probabilities.

69. The non-transitory computer-readable medium of any of claims 49 to 68, wherein selecting the human language for the string of keyboard inputs out of thetwo or more different human languages comprises automatically translating from a language of the string of keyboard inputs to another language.

70. The non-transitory computer-readable medium of any of claims 49 to 68, wherein the one or more processors further execute the instructions to: access a user selection of the two or more languages for the mode that allows the keyboard inputs to represent the two or more different human languages; and enter the mode responsive to accessing the user selection of the two or more languages for the mode.

71. The non-transitory computer-readable medium of any of claims 49 to 68, wherein the one or more processors further execute the instructions to: access a user selection of a plurality of languages to be used in a single language mode; and select between the plurality of languages as the two or more different human languages associated with the corresponding two or more different sets of characters.

72. The non-transitory computer-readable medium of any of claims 49 to 68, wherein the one or more processors further execute the instructions to: access user preferences; and determine the two or more different human languages based on the user preferences.

Images