A text input method, terminal and storage medium

Abstract

The present invention discloses a text input method, a terminal and a storage medium, which are used to solve the problem that the text input method is relatively single. In the embodiment of the present invention, the terminal collects the voice data of the user; recognizes the voice data, and matches the recognized text data with the user-defined phrase; after determining that the match is successful, the information content corresponding to the phrase matching the text data is input into the text input area in the display page. In the text input method of the embodiment of the present invention, the user only needs to input the voice data matching the custom phrase, and the terminal can automatically input the information content corresponding to the locally stored custom phrase into the text input area, and no longer needs to input the complete content to be input into the text input area, thereby simplifying the text input method, providing a more flexible text input method, and can improve the efficiency of text input and enhance the user experience.

Description

Technical Field

[0001] The present invention relates to the field of terminal technology, and in particular to a text input method, a terminal and a storage medium. Background Art

[0002] With the popularization of smart terminals, smart terminal applications have almost become standard equipment for Internet products in all walks of life, greatly facilitating people's daily lives. More and more operations can be completed on smart terminals.

[0003] When using smart terminals, users often need to input text in the text input interface. For example, when using the instant messaging application of the smart terminal, users need to input text content to be sent to the other party, or users fill in personal text information on the web page of the smart terminal. In addition, current smart terminals all support intelligent voice input. When users need to input text in the text input interface, the smart terminal can collect the user's voice data and convert the collected voice data into text data for input. The current voice input method can only convert the user's voice data into text data for input, and the input method is relatively simple. Summary of the invention

[0004] The present invention provides a text input method, terminal and storage medium, which are used to solve the problem that the current voice input method of smart terminals can only convert the user's voice data into text data for input, and the input method is relatively single. The technical solution of the present invention is as follows:

[0005] According to a first aspect of an exemplary embodiment, there is provided a terminal, comprising a display screen, a processor and a microphone;

[0006] The microphone is configured to collect voice data of the user;

[0007] The processor is configured to recognize the voice data, match the recognized text data with a user-defined phrase; after determining that the match is successful, input the information content corresponding to the phrase matching the text data into a text input area in the display page; wherein the number of characters in the user-defined phrase is less than the number of characters in the information content;

[0008] The display screen is configured to display a display page including a text input area, and to display information content corresponding to a phrase matching the text data in the text input area.

[0009] In the above-mentioned embodiment, the text input method provided by the embodiment of the present invention, when the user needs to input text information in the display page, the terminal collects the voice data input by the user through the microphone, and converts the collected voice data into text data; the terminal locally stores the user-defined phrase, matches the converted text data with the user-defined phrase, and after determining that the match is successful, inputs the information content corresponding to the user-defined phrase that matches the text data stored locally into the text input area in the display page. In this way, the user only needs to input the voice data that matches the customized phrase, and the terminal can automatically input the information content corresponding to the locally stored customized phrase into the text input area. The user only needs to input the customized phrase by voice, and the user is no longer required to input the complete information content. Since the number of characters in the user-defined phrase is less than the number of characters in the corresponding information content, the text input method is simplified, a more flexible text input method is provided, and the efficiency of text input can be improved, thereby enhancing the user experience.

[0010] In some exemplary embodiments, the processor is specifically configured to:

[0011] Determining whether the text data contains a user-defined phrase;

[0012] If so, it is determined that the text data matches the user-defined phrase successfully.

[0013] In the above embodiment, when judging whether the converted text data matches the user-defined phrase, it is determined whether the text data contains the user-defined phrase. After the text data contains a match with the user-defined phrase, it is determined that the converted text data successfully matches the user-defined phrase, thereby providing a method for judging whether the text data converted from the user's voice data matches the user-defined phrase.

[0014] In some exemplary embodiments, the processor is specifically configured to obtain the user-defined phrase and the information content corresponding to the phrase in the following manner:

[0015] In response to the user's input operation in the phrase editing interface, the phrase and information content input by the user in the phrase editing interface are obtained, and the obtained phrase and information content are bound and stored locally.

[0016] In the above embodiment, users can customize phrases and corresponding information content in the phrase editing interface. The terminal obtains the phrases and information content input by the user, binds them, and stores them locally in the terminal. Therefore, users can flexibly customize the correspondence between phrases and information content through the phrase editing interface of the terminal, thereby realizing personalized definition of words and corresponding information content.

[0017] In some exemplary embodiments, the process is further configured to:

[0018] If the voice data includes a re-entered control instruction, before the information content corresponding to the phrase matching the text data is entered into the text input area in the display page, the text information already entered in the text input area is deleted.

[0019] In the above embodiment, when the voice data contains a re-entered control instruction, when the information content corresponding to the text data converted from the voice data is entered into the text input area, it is necessary to delete the text information already entered in the text input area, and then enter the information content corresponding to the text data into the text input area, so as to ensure accurate input of the text.

[0020] In some exemplary embodiments, the user-defined phrase includes at least one of the user's address information, mobile phone number information, ID number information, company name information, invoice header information, and invoice tax number information.

[0021] According to a second aspect of an exemplary embodiment, there is provided a text input method, comprising:

[0022] Collect user's voice data;

[0023] Recognizing the voice data, and matching the recognized text data with a user-defined phrase;

[0024] After determining that the match is successful, the information content corresponding to the phrase matching the text data is input into the text input area in the display page; wherein the number of characters in the user-defined phrase is less than the number of characters in the information content.

[0025] In some exemplary embodiments, matching the recognized text data with a user-defined phrase includes:

[0026] Determining whether the text data contains a user-defined phrase;

[0027] If so, it is determined that the text data successfully matches the user-defined phrase.

[0028] In some exemplary implementations, the user-defined phrase and the information content corresponding to the phrase are obtained in the following manner:

[0029] In response to the user's input operation in the phrase editing interface, the phrase and information content input by the user in the phrase editing interface are obtained, and the obtained phrase and information content are bound and stored locally.

[0030] In some exemplary embodiments, if the voice data includes a re-entered control instruction, before inputting the information content corresponding to the phrase matching the text data into the text input area in the display page, the method further includes:

[0031] Delete the text information entered in the text input area.

[0032] According to a third aspect of an exemplary embodiment, there is provided a text input device, comprising:

[0033] An acquisition module is used to collect user's voice data;

[0034] A determination module, configured to recognize the voice data and match the recognized text data with a user-defined phrase;

[0035] The input module is used to input the information content corresponding to the phrase matching the text data into the text input area in the display page after determining that the match is successful; wherein the number of characters in the user-defined phrase is less than the number of characters in the information content.

[0036] According to a fourth aspect of the exemplary implementation, a computer storage medium is provided, wherein the computer storage medium stores computer program instructions, and when the instructions are executed on a computer, the computer executes the method according to the second aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0038] Figure 1 A schematic diagram of the structure of a terminal provided by an embodiment of the present invention;

[0039] Figure 2 A schematic diagram of a software architecture of a terminal provided by an embodiment of the present invention is exemplarily shown;

[0040] Figure 3 The following is a schematic diagram of a user interface of a terminal provided by an embodiment of the present invention;

[0041] Figure 4 A schematic diagram of a user interface of another terminal provided by an embodiment of the present invention is exemplarily shown;

[0042] Figure 5 The flowchart of the text input method provided by the embodiment of the present invention is exemplarily shown;

[0043] Figure 6 The following is a schematic diagram of a command editing page of a voice assistant provided by an embodiment of the present invention;

[0044] Figure 7 The following is a schematic diagram showing a display page of a first voice assistant application provided by an embodiment of the present invention;

[0045] Figure 8 The following is a schematic diagram showing a display page of a second voice assistant application provided by an embodiment of the present invention;

[0046] Fig. 9 The following is a schematic diagram showing a display page of a third voice assistant application provided by an embodiment of the present invention;

[0047] Fig.10 The text input interface provided by the embodiment of the present invention is exemplarily shown;

[0048] Fig.11 A schematic diagram of a text input interface for creating a new note provided by an embodiment of the present invention is exemplarily shown;

[0049] Fig.12 Another schematic diagram of a text input interface for creating a new note provided by an embodiment of the present invention is exemplarily shown;

[0050] Fig.13 The following is a schematic diagram of a chat interface of a first instant messaging application provided by an embodiment of the present invention;

[0051] Fig.14 A schematic diagram of a chat interface of a second instant messaging application provided by an embodiment of the present invention is exemplarily shown;

[0052] Fig.15 The third instant messaging application provided in the embodiment of the present invention is exemplarily shown as a chat interface diagram;

[0053] Fig.16 The fourth instant messaging application chat interface diagram provided by the embodiment of the present invention is exemplarily shown;

[0054] Fig.17 The flowchart of the overall method of text input in an embodiment of the present invention is exemplarily shown;

[0055] Fig.18 The schematic diagram of the structure of the terminal provided by the embodiment of the present invention is exemplarily shown;

[0056] Fig.19 The schematic diagram of the structure of the text input device provided by the embodiment of the present invention is exemplarily shown. DETAILED DESCRIPTION

[0057] The following will be combined with the accompanying drawings to clearly and thoroughly describe the technical solutions in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0058] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0059] The following are explanations of some of the terms that appear in this article:

[0060] 1. The term "voice assistant" in the embodiments of the present invention refers to an application on an intelligent terminal device, which realizes intelligent interaction between the user and the terminal device through intelligent dialogue and instant question and answer.

[0061] 2. The term "terminal" in the embodiments of the present invention may be a mobile phone, a computer, a digital broadcast terminal, a message transceiver, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0062] 3. In the embodiments of the present invention, the term "server" refers to a device that provides computing services. Since the server needs to respond to service requests and perform processing, generally speaking, the server should have the ability to undertake and guarantee services.

[0063] Figure 1 A schematic structural diagram of the terminal 100 is shown.

[0064] The embodiment is described in detail below by taking the terminal 100 as an example. It should be understood that: Figure 1 The terminal 100 shown is only an example and the terminal 100 may have more Figure 1 The more or less components shown in the figure can be combined with two or more components, or can have different component configurations. The various components shown in the figure can be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and / or application specific integrated circuits.

[0065] Figure 1FIG. 1 is a block diagram showing a hardware configuration of the terminal 100 according to an exemplary embodiment. Figure 1 As shown, the terminal 100 includes: a radio frequency (RF) circuit 110, a memory 120, a display unit 130, a camera 140, a sensor 150, an audio circuit 160, a wireless fidelity (Wi-Fi) module 170, a processor 180, a Bluetooth module 181, and a power supply 190 and other components.

[0066] The RF circuit 110 can be used to receive and send signals during the process of sending and receiving information or making calls. It can receive downlink data from the base station and hand it over to the processor 180 for processing; it can send uplink data to the base station. Generally, the RF circuit includes but is not limited to antennas, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer and other devices.

[0067] The memory 120 can be used to store software programs and data. The processor 180 executes various functions and data processing of the terminal 100 by running the software programs or data stored in the memory 120. The memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. The memory 120 stores an operating system that enables the terminal 100 to run. In the present application, the memory 120 can store an operating system and various application programs, and can also store code for executing the method described in the embodiment of the present application.

[0068] The display unit 130 may be used to receive input digital or character information and generate signal input related to user settings and function control of the terminal 100. Specifically, the display unit 130 may include a touch screen 131 disposed on the front of the terminal 100, which may collect user touch operations on or near it, such as clicking a button, dragging a scroll box, etc.

[0069] The display unit 130 may also be used to display information input by the user or information provided to the user and a graphical user interface (GUI) of various menus of the terminal 100. Specifically, the display unit 130 may include a display screen 132 disposed on the front of the terminal 100. The display screen 132 may be configured in the form of a liquid crystal display, a light emitting diode, etc. The display unit 130 may be used to display various graphical user interfaces described in the present application.

[0070] The touch screen 131 may be covered on the display screen 132, or the touch screen 131 and the display screen 132 may be integrated to realize the input and output functions of the terminal 100, and the integrated touch screen may be referred to as a touch display screen. In the present application, the display unit 130 may display the application and the corresponding operation steps.

[0071] The camera 140 can be used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then passed to the processor 180 for conversion into a digital image signal.

[0072] The terminal 100 may further include at least one sensor 150, such as an acceleration sensor 151, a distance sensor 152, a fingerprint sensor 153, and a temperature sensor 154. The terminal 100 may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, a light sensor, and a motion sensor.

[0073] The audio circuit 160, the speaker 161, and the microphone 162 can provide an audio interface between the user and the terminal 100. The audio circuit 160 can transmit the electrical signal converted from the received audio data to the speaker 161, which is converted into a sound signal for output. The terminal 100 can also be configured with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 162 converts the collected sound signal into an electrical signal, which is received by the audio circuit 160 and converted into audio data, and then the audio data is output to the RF circuit 110 to be sent to, for example, another terminal, or the audio data is output to the memory 120 for further processing. In this application, the microphone 162 can obtain the user's voice.

[0074] Wi-Fi is a short-range wireless transmission technology. The terminal 100 can help users send and receive emails, browse web pages, and access streaming media through the Wi-Fi module 170, which provides users with wireless broadband Internet access.

[0075] The processor 180 is the control center of the terminal 100. It uses various interfaces and lines to connect various parts of the entire terminal. It executes various functions of the terminal 100 and processes data by running or executing software programs stored in the memory 120 and calling data stored in the memory 120. In some embodiments, the processor 180 may include one or more processing units; the processor 180 may also integrate an application processor and a baseband processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the baseband processor mainly processes wireless communications. It is understandable that the above-mentioned baseband processor may not be integrated into the processor 180. In the present application, the processor 180 can run the operating system, application programs, user interface display and touch response, as well as the processing method described in the embodiments of the present application. In addition, the processor 180 is coupled to the display unit 130.

[0076] The Bluetooth module 181 is used to exchange information with other Bluetooth devices having Bluetooth modules through the Bluetooth protocol. For example, the terminal 100 can establish a Bluetooth connection with a wearable electronic device (such as a smart watch) that also has a Bluetooth module through the Bluetooth module 181 to exchange data.

[0077] The terminal 100 also includes a power supply 190 (such as a battery) for supplying power to various components. The power supply can be logically connected to the processor 180 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption. The terminal 100 can also be configured with a power button for turning on and off the terminal, as well as locking the screen and other functions.

[0078] Figure 2 It is a software structure block diagram of the terminal 100 according to an embodiment of the present invention.

[0079] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.

[0080] The application layer can include a series of application packages.

[0081] like Figure 2 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0082] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0083] like Figure 2 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0084] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0085] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0086] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0087] The phone manager is used to provide communication functions of the terminal 100, such as management of call status (including connection, disconnection, etc.).

[0088] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0089] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, the terminal vibrates, the indicator light flashes, etc.

[0090] Android Runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0091] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0092] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0093] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0094] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0095] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0096] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0097] A 2D graphics engine is a drawing engine for 2D drawings.

[0098] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0099] The following is an illustrative description of the software and hardware workflow of the terminal 100 in conjunction with the capture and photo shooting scene.

[0100] When the touch screen 131 receives a touch operation, the corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into an original input event (including touch coordinates, timestamp of the touch operation, and other information). The original input event is stored in the kernel layer. The application framework layer obtains the original input event from the kernel layer and identifies the control corresponding to the input event. For example, if the touch operation is a touch single-click operation and the control corresponding to the single-click operation is a control of the camera application icon, the camera application calls the interface of the application framework layer to start the camera application, and then starts the camera driver by calling the kernel layer to capture a static image or video through the camera 140.

[0101] The terminal 100 in the embodiment of the present application may be a mobile phone, a tablet computer, a wearable device, a laptop computer, a television, etc.

[0102] Figure 3 is used to show the terminal (for example Figure 1Schematic diagram of a user interface on a terminal 100). In some specific implementations, a user can open a corresponding application by touching an application icon on the user interface, or can open a corresponding folder by touching a folder icon on the user interface.

[0103] Or, if Figure 4 Schematic diagram of the user interface of the terminal shown. In some specific implementations, the user can call the voice assistant on the terminal to control the opening of the corresponding application or the opening of the corresponding folder through voice commands.

[0104] Current terminals are pre-installed with voice assistants, and users can call the voice assistant on the terminal and input voice commands to control the terminal to complete specified operations.

[0105] However, in the current voice input scenario of the terminal, the terminal collects the user's voice data through a microphone, and converts the collected voice data into text data through a voice assistant. For example, when a user needs to enter his or her mobile phone number into the text input area on the terminal, he or she needs to enter his or her complete mobile phone number into the terminal in the form of voice data, and the terminal converts the voice data into an 11-digit number and enters it into the text input area. However, for some commonly used information, each time the commonly used information needs to be entered, the user can only enter the complete commonly used information in the form of voice each time, and the terminal converts the commonly used information in the form of voice into text and enters it into the text input area; for example, for the user's home address, ID card number and other information, the user may need to enter the commonly used text information in multiple scenarios, and the content corresponding to the home address is generally a long text. When entering this type of commonly used text information by voice input, the user needs to repeat the voice data with more content each time. This voice input is inefficient and greatly reduces the user experience.

[0106] Based on the above problems, an embodiment of the present invention provides a text input method, which can realize automatic text input with the help of a voice assistant.

[0107] like Figure 5 As shown, a text input method according to an embodiment of the present invention comprises the following steps:

[0108] Step S501: Collecting user's voice data;

[0109] Step S502: Recognize the voice data, and match the recognized text data with a user-defined phrase;

[0110] Step S503: after determining that the match is successful, input the information content corresponding to the phrase matching the text data into the text input area in the display page; wherein the number of characters in the user-defined phrase is less than the number of characters in the information content.

[0111] The text input method provided by the embodiment of the present invention is that when a user needs to input text information in a display page, the terminal collects the voice data input by the user through a microphone and converts the collected voice data into text data; the terminal locally stores a user-defined phrase, matches the converted text data with the user-defined phrase, and after determining that the match is successful, inputs the information content corresponding to the user-defined phrase that matches the text data stored locally into the text input area in the display page. In this way, the user only needs to input the voice data that matches the customized phrase, and the terminal can automatically input the information content corresponding to the locally stored customized phrase into the text input area. The user only needs to input the customized phrase by voice, and the user is no longer required to input the complete information content. Since the number of characters in the user-defined phrase is less than the number of characters in the corresponding information content, the text input method is simplified, a more flexible text input method is provided, and the efficiency of text input can be improved, thereby enhancing the user experience.

[0112] During implementation, in an embodiment of the present invention, in the display page for text input, the user can provide a defined physical button to trigger the voice assistant startup command; or the user can also trigger the voice assistant startup command through a preset trigger gesture in the display page for text input; or in an embodiment of the present invention, the user can also trigger the voice assistant startup command through other predefined methods, which are not limited here.

[0113] The user-defined phrases and corresponding information contents in the embodiment of the present invention may be stored locally in the terminal.

[0114] An optional implementation is that the user can customize the correspondence between the phrase and the information content through the phrase editing page on the terminal, and the terminal binds the customized phrase input by the user through the phrase editing page with the information content and stores it locally in the terminal.

[0115] The user-defined phrase in the embodiment of the present invention includes part or all of the following information:

[0116] The user's address information, mobile phone number information, ID number information, company name information, invoice header information, and invoice tax number information.

[0117] It should be noted that the embodiment of the present invention can perform operations such as adding, deleting, and modifying the correspondence between the locally stored phrases and information content.

[0118] For example, Figure 6 The phrase editing page shown, in which the user enters a user-defined phrase in the text input area corresponding to the phrase, and enters the information content corresponding to the phrase in the text input area corresponding to the input text; assuming that the user-defined phrase is "My Address" and the information content entered by the user is "No. 001, Songling Road, Laoshan District".

[0119] like Figure 7 The custom phrase display page shown includes the correspondence between multiple phrases and information contents defined by the user: the information content corresponding to the phrase "my address" is "No. 001, Songling Road, Laoshan District"; the information content corresponding to the phrase "my name" is "Harry"; the information content corresponding to the phrase "my ID number" is "370XXXXXXXXXXXXXXX"; the information content corresponding to the phrase "my mobile phone number" is "150XXXXXXXX".

[0120] The user can re-edit the phrase or information content through the edit icon after each phrase or information content to modify the correspondence between the stored phrase and the information content; in addition, the user can add the correspondence between the phrase and the information content through the new icon in the display page; or, the user can delete the correspondence between the stored phrase and the information content by long pressing.

[0121] In addition, in the correspondence between the self-defined phrases and information contents in the embodiment of the present invention, one phrase may correspond to one or more information contents;

[0122] When a user customizes the correspondence between a phrase and information content through the phrase editing page, one or more information contents can be set for a customized phrase; for example, when the phrase is "mobile phone number", the user can bind the phrase to one or more mobile phone numbers.

[0123] It should be noted that, in the case where a phrase corresponds to multiple information contents, the embodiment of the present invention allows the user to select a valid information content from the multiple information contents; for example, Figure 8 In the custom phrase display page shown, the phrase "My mobile phone number" corresponds to two information contents, namely "150XXXXXXXX" and "137XXXXXXXX"; the user can click the "select" icon corresponding to the information content to select valid information content from the two information contents. For example, if the user selects "150XXXXXXXX", "150XXXXXXXX" is the valid information content corresponding to "My mobile phone number".

[0124] After the terminal stores the correspondence between the phrase and the information content, the embodiment of the present invention can realize automatic text input through a voice assistant. The specific scheme is as follows:

[0125] The terminal collects the user's voice data through a microphone, and the voice assistant converts the collected voice data into text data; the terminal determines whether the text data contains a user-defined phrase, and if so, determines that the text data successfully matches the user-defined phrase.

[0126] In implementation, when the terminal determines whether the text data contains the user-defined phrase, it compares the key words in the text data with the user-defined phrase. When the terminal key words in the text data are consistent with the user-defined phrase, it is determined that the text data matches the user-defined phrase successfully.

[0127] The terminal may determine the keywords contained in the text data in the following manner:

[0128] The terminal sends the converted text data to the remote server through the voice assistant application. The remote server performs semantic recognition on the text data, determines the control instructions and keywords contained in the text data, and returns the determined control instructions and keywords to the terminal; thereby, the terminal can obtain the control instructions and keywords contained in the collected user's voice data.

[0129] After determining that the text data successfully matches the user-defined phrase, the terminal determines the information content corresponding to the phrase matching the text data based on the stored correspondence between the phrase and the information content;

[0130] It should be noted that, when it is determined that a phrase matching the text data corresponds to multiple information contents based on the correspondence between the phrase and the information contents, valid information contents among the multiple information contents are used as the information contents to be input into the text input area.

[0131] In addition, in the embodiment of the present invention, for the situation where a phrase corresponds to an information content, the user can also choose whether to regard the information content corresponding to the phrase as valid information content; if the information content corresponding to the phrase matching the text data is determined to be valid information content based on the stored correspondence between the phrase and the information content, the information content corresponding to the phrase is used as the information content to be input into the text input area; otherwise, if it is determined that the information content corresponding to the phrase matching the text data is not valid information content, it is determined that the information content corresponding to the phrase matching the text data has not been determined from the correspondence between the phrase and the information content.

[0132] For example, Fig. 9In the custom phrase display page shown, the correspondence between multiple custom phrases stored in the terminal and the information content is as follows: the information content corresponding to the phrase "my address" is "No. 001, Songling Road, Laoshan District", and the user selects the information content, then the information content is valid information content; the information content corresponding to the phrase "my name" is "Harry", and the user selects the information content, then the information content is valid information content; the information content corresponding to the phrase "my ID number" is "370XXXXXXXXXXXXXXX", and the user does not select the information content, then the information content is not valid information content; the information content corresponding to the phrase "my mobile phone number" is "150XXXXXXXX", "137XXXXXXXX", and the information content selected by the user is "150XXXXXXXX", then "150XXXXXXXX" is valid information content, and "137XXXXXXXX" is not valid information content.

[0133] When the collected user's voice data is converted into text data, assuming that the phrase matching the text data is "my mobile phone number", it is determined that the information content corresponding to the phrase is "150XXXXXXXX"; assuming that the phrase matching the text data is "my name", it is determined that the information content corresponding to the phrase is "Harry"; assuming that the phrase matching the text data is "my ID number", it is determined that the phrase has no corresponding information content.

[0134] In the embodiment of the present invention, when the terminal matches the recognized text data with the user-defined phrase, if it is determined that the match fails, the terminal sends the text data or keywords in the text data to the remote server through the voice assistant application, and the remote server determines the information content and scenario tags corresponding to the keywords in the text data;

[0135] Among them, the scenario tag is used to indicate the applicable scenario of the information content corresponding to the keyword.

[0136] Context labels include but are not limited to:

[0137] Conversation scene tags, information query scene tags, audio and video playback scene tags, text message sending scene tags, phone call scene tags, message reminder scene tags, and translation scene tags.

[0138] The remote server determines the information content and context labels corresponding to the keywords in the text data in the following manner:

[0139] The remote server performs semantic recognition on keywords in the text data according to the text semantic understanding CMD_WRITE method, and obtains the information content corresponding to the keyword and the context label of the identified applicable scenario of the information content.

[0140] After the remote server identifies the information content and context tags corresponding to the keywords in the text data, it sends the identification results to the terminal; after the terminal determines that the context tags match the display page of the current text input, it inputs the determined information content into the text input area in the current display page.

[0141] In addition, after determining that the scenario tag does not match the display page of the current text input, the terminal uses the keywords in the text data as the information content that needs to be input into the text input area.

[0142] For example, assuming that the current text input display page is the display page of the terminal's memo application, after the voice assistant is turned on in the display page, if the text data converted from the collected voice data is "Enter a meeting tomorrow morning", the text data is matched with the user-defined phrase, and it is determined that there is no successful match from the stored user-defined phrases. The keyword "meeting tomorrow morning" in the text data is semantically recognized through the remote server, and the information content returned to the terminal by the remote server is "What time is the meeting tomorrow morning" and the scenario label is the dialogue scenario. The terminal determines that the scenario label returned by the remote server does not match the current text input interface, and the keyword "meeting tomorrow morning" in the text data is used as the information content that needs to be entered into the text input area.

[0143] The terminal of the embodiment of the present invention obtains the control instruction contained in the voice data according to the converted text data; if the control instruction contained in the voice data is input, the determined information content is directly input into the text input area; if the control instruction contained in the voice data is re-input, the text information already input in the text input area is deleted, and the determined information content is input into the text input area.

[0144] The following introduces the text input methods for different control instructions:

[0145] Method 1: The voice data contains input control instructions, and the determined information content is input into the text input area.

[0146] In this manner, if there is text information already input in the text input area, when the determined information content is input into the text input area, there is no need to delete the information content already input in the text input area.

[0147] If the text input area does not contain the input text information, the determined information content is directly input into the text input area;

[0148] If the text input area contains input text information, the input text information and the determined information content are combined and input into the text input area according to the input position selected by the user in the text input area.

[0149] In implementation, the Accessibility Service provided by Android can simulate operations such as clicks and inputs. First, obtain the form node object Accessibility NodeInfo of the text input area in the display interface. Then, call the accessibilityNodeInfo.findFocus(AccessibilityNodeInfo.FOCUS_INPUT) method to obtain the input control nodeInfo of the cursor focus in the current text input area. Call the nodeInfo.getText() method to obtain the text information that has been entered in the text input area.

[0150] It should be noted that when there is no text information input in the text input area, there may be prompt information in the text input area; and the prompt information in the text input area is not the text information that has been input. Fig.10 In the text input display page shown, if the text input area includes prompt information "for example, my address", when the user enters determined information content in the text input area, the prompt information is automatically deleted.

[0151] When the nodeInfo.getText() method is called to obtain the text information that has been entered in the text input area, the hint information in the text input area will also be obtained as the text information that has been entered; therefore, the embodiment of the present invention also needs to call the nodeInfo.isShowingHintText() method to determine whether the text information that has been entered in the current text input area is hint information. If the nodeInfo.getText() method is called to obtain the entered text information, and the nodeInfo.isShowingHintText() method is called to determine that the current text input area contains hint information, then it is determined that the current text input area does not contain the entered text information; if the nodeInfo.getText() method is called to obtain the entered text information, and the nodeInfo.isShowingHintText() method is called to determine that the current text input area does not contain hint information, then it is determined that the current text input area contains the entered text information.

[0152] If the text input area does not contain the entered text information, the current text input area is empty. You can use ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE to specify the parameters of the text information to be set:

[0153] bundle.putCharSequence(AccessibilityNodeInfo.ACTION_ARGUMENT_SE T_TEXT_CHARSEQUENCE,"XXXX");

[0154] Call the following method function to input the determined information content into the text input area:

[0155] nodeInfo.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT,bundle).

[0156] If the text input area contains input text information, it is necessary to determine the cursor position in the current text input area, and determine the input position selected by the user in the text input area according to the cursor position in the text input area;

[0157] In implementation, the index x of the cursor start position is obtained through nodeInfo.getTextSelectionStart(), and the cursor start position is determined to be the start position of the information content to be input;

[0158] After the cursor start position is determined, the character string before the cursor in the text input area, the determined information content, and the character string after the cursor are combined and input into the text input area.

[0159] Call the following method function to input the determined information content into the text input area:

[0160] nodeInfo.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT,bundle).

[0161] And call the following two method functions to determine the parameters of the cursor start position and end position:

[0162] arguments.putInt(AccessibilityNodeInfo.ACTION_ARGUMENT_SELECTIO N_START_INT,x);

[0163] arguments.putInt(AccessibilityNodeInfo.ACTION_ARGUMENT_SELECTIO N_END_INT,y).

[0164] Finally, call the following method function to complete the cursor position specification:

[0165] nodeInfo.performAction(AccessibilityNodeInfo.ACTION_SET_SELECTION,arguments);

[0166] The above process realizes the repositioning of the cursor position in the text input area, and the final effect presented in the text input area is to insert the determined information content at the original cursor position.

[0167] For example, Fig.11 The display page of creating a new note shown, after the user triggers the command to start the voice assistant in the display page, the terminal collects the user's voice data through the microphone, converts the collected voice data into text data through the voice assistant, and displays the recognized text data in the display area corresponding to the voice assistant, assuming that it is "Enter my address";

[0168] The terminal obtains the correspondence between the user-defined phrase and the text information stored locally, and matches the converted text data with the user-defined phrase; if the match is successful, and the information content corresponding to the phrase matching the text data is determined to be "No. 001, Songling Road, Laoshan District", the text input area of ​​the "No. 001, Songling Road, Laoshan District" display page is displayed, and the new note page after input is as follows Fig.12 As shown, the voice assistant prompts the user with a confirmation message "I have helped you enter" in the corresponding display area.

[0169] For example, Fig.13 The chat interface of the instant messaging application shown in the figure, in which the text message entered by the user is "Call me tomorrow"; after the user triggers the command to start the voice assistant in the chat interface, the terminal collects the user's voice data, recognizes the collected voice data as text data through the voice assistant, and displays the recognized text data in the display area corresponding to the voice assistant; assuming that it is "Enter my mobile phone number";

[0170] The terminal obtains the correspondence between the user-defined phrase and the text information stored locally, and matches the converted text data with the user-defined phrase; if the match is successful, and the information content corresponding to the phrase matching the text data is determined to be "150XXXXXXXX", then "150XXXXXXXX" and the input text information "Call me tomorrow" are combined and entered into the text input area, and the new note page after input is as follows: Fig.14 As shown, the voice assistant prompts the user with a confirmation message "I have helped you enter" in the corresponding display area.

[0171] Method 2: The voice data includes a re-input control instruction, the text information already input in the text input area is deleted, and the determined information content is input into the text input area.

[0172] In this method, the determined information content is used to replace the text information input in the text input area.

[0173] In implementation, calling the nodeInfo.getText() method can obtain the text information entered in the text input area.

[0174] When the nodeInfo.getText() method is called to obtain the text information that has been entered in the text input area, the hint information in the text input area will also be obtained as the original text information; therefore, the embodiment of the present invention also needs to call the nodeInfo.isShowingHintText() method to determine whether the text information that has been entered in the current text input area is hint information. If the nodeInfo.getText() method is called to obtain the entered text information, and the nodeInfo.isShowingHintText() method is called to determine that the current text input area contains hint information, then it is determined that the current text input area does not contain the entered text information; if the nodeInfo.getText() method is called to obtain the entered text information, and the nodeInfo.isShowingHintText() method is called to determine that the current text input area does not contain hint information, then it is determined that the current text input area contains the entered text information.

[0175] In this manner, after determining that the text input area contains the input text information, the original text information in the text input area is deleted, and the following method function is called to input the information content to be determined into the text input area:

[0176] nodeInfo.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT,bundle).

[0177] For example, Fig.15 In the chat interface of the instant messaging application shown, the text information entered by the user in the text input area of ​​the chat interface is "No. 001, Huachang Road, Hedong District"; after the user triggers the command to start the voice assistant in the chat interface, the voice assistant collects the user's voice data, recognizes the collected voice data as text data, and displays the recognized text data in the display area corresponding to the voice assistant, assuming that it is "Re-enter my address";

[0178] The terminal obtains the correspondence between the user-defined phrase and the text information stored locally, and matches the converted text data with the user-defined phrase; if it is determined that the match is successful, and it is determined that the information content corresponding to the phrase matching the text data is "No. 001, Songling Road, Laoshan District", the text information "No. 001, Huachang Road, Hedong District" entered in the text input area of ​​the chat interface is deleted, and "No. 001, Songling Road, Laoshan District" is entered into the text input area of ​​the chat interface. The chat interface after input is as follows. Fig.16 As shown, the voice assistant prompts the user with a confirmation message "I have re-entered it for you" in the corresponding display area of ​​the voice assistant.

[0179] like Fig.17 As shown, the overall method flow chart of text input in an embodiment of the present invention can be applied to a terminal, and includes the following steps:

[0180] Step S1701: Collecting user's voice data;

[0181] Step S1702: converting the collected voice data into text data through a voice assistant;

[0182] Step S1703: matching the converted text data with the user-defined phrases stored locally;

[0183] Step S1704, determine whether the match is successful; if so, execute step S1705, if not, execute step S1706;

[0184] Step S1705: obtaining information content corresponding to the phrase matching the text data, and inputting the obtained information content into the text input area in the display page;

[0185] Step S1706: sending the text data to the remote server, and receiving content information and scenario tags corresponding to the text data returned by the remote server;

[0186] Step S1707, determine whether the scenario tag corresponding to the text data matches the current display page; if so, execute step S1708, otherwise, execute step S1709;

[0187] Step S1708: input the content information corresponding to the text data returned by the remote server into the text input area in the display page;

[0188] Step S1709: input the keywords in the text data into the text input area in the display page.

[0189] Based on the same inventive concept, a terminal is also provided in an embodiment of the present invention. Since the principle of solving the problem by the terminal is similar to that of the text input method in the embodiment of the present invention, the implementation of the terminal can refer to the implementation of the method, and the repeated parts will not be repeated.

[0190] like Fig.18 As shown, an embodiment of the present invention provides a terminal, including a display screen 1801, a processor 1802 and a microphone 1803;

[0191] The microphone 1803 is configured to collect voice data of the user;

[0192] The processor 1802 is configured to recognize the voice data, match the recognized text data with a user-defined phrase; after determining that the match is successful, input the information content corresponding to the phrase matching the text data into a text input area in the display page; wherein the number of characters in the user-defined phrase is less than the number of characters in the information content;

[0193] The display screen 1801 is configured to display a display page including a text input area, and to display information content corresponding to a phrase matching the text data in the text input area.

[0194] In some exemplary embodiments, the processor 1802 is specifically configured to:

[0195] Determining whether the text data contains a user-defined phrase;

[0196] If so, it is determined that the text data matches the user-defined phrase successfully.

[0197] In some exemplary implementations, the processor 1802 is specifically configured to obtain the user-defined phrase and the information content corresponding to the phrase in the following manner:

[0198] In response to the user's input operation in the phrase editing interface, the phrase and information content input by the user in the phrase editing interface are obtained, and the obtained phrase and information content are bound and stored locally.

[0199] In some exemplary embodiments, the processor 1802 is further configured to:

[0200] If the voice data includes a re-entered control instruction, before the information content corresponding to the phrase matching the text data is entered into the text input area in the display page, the text information already entered in the text input area is deleted.

[0201] In some exemplary embodiments, the user-defined phrase includes at least one of the user's address information, mobile phone number information, ID number information, company name information, invoice header information, and invoice tax number information.

[0202] like Fig.19 As shown, an embodiment of the present invention provides a text input device, including:

[0203] Acquisition module 1901, used to collect user's voice data;

[0204] The determination module 1902 is used to recognize the voice data and match the recognized text data with a user-defined phrase;

[0205] Input module 1903 is used to input the information content corresponding to the phrase matching the text data into the text input area in the display page after determining that the match is successful; wherein the number of characters in the user-defined phrase is less than the number of characters in the information content.

[0206] In some exemplary implementations, the determining module 1902 is specifically configured to:

[0207] Determining whether the text data contains a user-defined phrase;

[0208] If so, it is determined that the text data successfully matches the user-defined phrase.

[0209] In some exemplary implementations, the determining module 1902 is specifically configured to:

[0210] Obtain the user-defined phrase and the information content corresponding to the phrase in the following manner:

[0211] In response to the user's input operation in the phrase editing interface, the phrase and information content input by the user in the phrase editing interface are obtained, and the obtained phrase and information content are bound and stored locally.

[0212] In some exemplary embodiments, the input module 1903 is further configured to:

[0213] If the voice data includes a re-entered control instruction, before the information content corresponding to the phrase matching the text data is entered into the text input area in the display page, the text information already entered in the text input area is deleted.

[0214] In some exemplary embodiments, the user-defined phrase includes at least one of the user's address information, mobile phone number information, ID number information, company name information, invoice header information, and invoice tax number information.

[0215] An embodiment of the present disclosure provides a storage medium. When instructions in the storage medium are executed by a processor, the processor is enabled to execute the method for acquiring data of an extended storage device implemented by a mobile terminal in the above process.

[0216] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0217] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0218] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0219] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0220] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A terminal, characterized in that: Includes display, processor and microphone; The microphone is configured to collect voice data of the user; The processor is configured to recognize the voice data, match the recognized text data with a user-defined phrase; after determining that the match is successful, input the information content corresponding to the phrase matching the text data into a text input area in the display page; wherein the number of characters in the user-defined phrase is less than the number of characters in the information content; The display screen is configured to display a display page including a text input area, and to display information content corresponding to a phrase matching the text data in the text input area; The processor is further configured to, after determining that the match fails, send the text data or a keyword in the text data to a remote server, so that the remote server determines the information content and the scenario tag corresponding to the keyword in the text data, and sends the information content and the scenario tag to the terminal; The processor is further configured to input the information content into the text input area after determining that the scenario tag matches the display page of the current text input.

2. The terminal according to claim 1, characterized in that The processor is specifically configured to: Determining whether the text data contains a user-defined phrase; If so, it is determined that the text data matches the user-defined phrase successfully.

3. The terminal according to claim 2, characterized in that The processor is specifically configured to obtain the user-defined phrase and the information content corresponding to the phrase according to the following method: In response to the user's input operation in the phrase editing interface, the phrase and information content input by the user in the phrase editing interface are obtained, and the obtained phrase and information content are bound and stored locally.

4. The terminal according to claim 1, characterized in that The processor is further configured to: If the voice data includes a re-entered control instruction, before the information content corresponding to the phrase matching the text data is entered into the text input area in the display page, the text information already entered in the text input area is deleted.

5. The terminal according to any one of claims 1 to 4, characterized in that: The user-defined phrase includes at least one of the user's address information, mobile phone number information, ID card number information, company name information, invoice header information, and invoice tax number information.

6. A text input method, characterized in that: The method includes: Collect user's voice data; Recognizing the voice data, and matching the recognized text data with a user-defined phrase; After determining that the match is successful, inputting the information content corresponding to the phrase matching the text data into the text input area in the display page; wherein the number of characters in the user-defined phrase is less than the number of characters in the information content; or After determining that the match fails, the text data or keywords in the text data are sent to a remote server so that the remote server determines the information content and scenario tags corresponding to the keywords in the text data; the information content and scenario tags sent by the remote server are received; and after determining that the scenario tags match the display page of the current text input, the information content is input into the text input area.

7. The method according to claim 6, characterized in that The matching of the recognized text data with the user-defined phrases includes: Determining whether the text data contains a user-defined phrase; If so, it is determined that the text data successfully matches the user-defined phrase.

8. The method according to claim 7, characterized in that Obtain the user-defined phrase and the information content corresponding to the phrase in the following manner: In response to the user's input operation in the phrase editing interface, the phrase and information content input by the user in the phrase editing interface are obtained, and the obtained phrase and information content are bound and stored locally.

9. The method according to claim 6, characterized in that If the voice data includes a re-input control instruction, before inputting the information content corresponding to the phrase matching the text data into the text input area in the display page, the method further includes: Delete the text information entered in the text input area.

10. A computer storage medium, characterized in that: The computer storage medium stores computer program instructions, and when the instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 6 to 9.

Images