Emoji input method, apparatus, device, storage medium and product

Abstract

The application discloses an emoji input method, device, equipment, storage medium and product, and belongs to the field of human-computer interaction. The method comprises the following steps: obtaining voice information in a voice-to-text input process; displaying at least one recommended emoji on a candidate bar based on the voice information, wherein the at least one recommended emoji comprises a first emoji; and in response to an on-screen operation on the first emoji, the first emoji is on-screened to a text input area. The user can directly select a recommended emoji on the candidate bar and on-screen the emoji, without triggering an emoji tool entry and jumping back and forth between related pages, so that the number of interaction steps is greatly reduced, the input efficiency of the emoji is improved, and the coherence of voice input is also improved.

Description

Technical Field

[0001] This application relates to the field of human-computer interaction, and in particular to a method, apparatus, device, storage medium, and product for inputting emoticons. Background Technology

[0002] With the rapid development of smartphones and mobile internet, traditional Pinyin and handwriting input methods can no longer meet users' demands for input speed and convenience. Voice input, as an efficient method, can convert users' speech into text and is gradually becoming one of the mainstream input methods.

[0003] When using voice input methods, users often need to add emojis to text. In related technologies, users convert their voice messages into text using the voice input method, then access the emoji entry in the input method toolbar, find suitable emojis from the emoji module's relevant page, add them to the text box, and then return to the voice input interface to continue voice input.

[0004] However, the methods for adding emojis in related technologies require a lot of human-computer interaction, which is quite cumbersome. Summary of the Invention

[0005] This application provides a method, apparatus, device, storage medium, and product for inputting emojis, the technical solution of which is as follows.

[0006] According to one aspect of this application, a method for inputting emojis is provided, the method comprising:

[0007] During the speech-to-text input process, voice information is acquired;

[0008] Based on the voice information, at least one recommended emoji is displayed in the candidate bar, and the at least one recommended emoji includes a first emoji;

[0009] In response to the screen-up operation of the first emoji, the first emoji is displayed on the text input area.

[0010] According to one aspect of this application, an emoji input device is provided, the device comprising:

[0011] The acquisition module is used to acquire voice information during the speech-to-text input process;

[0012] The display module is used to display at least one recommended emoji on the candidate bar based on the voice information, wherein the at least one recommended emoji includes a first emoji;

[0013] The screen display module is used to respond to the screen display operation for the first emoji and display the first emoji on the text input area.

[0014] According to another aspect of this application, a computer device is provided, comprising: a processor and a memory, wherein the memory stores at least one computer program, the at least one computer program being loaded and executed by the processor to implement the emoji input method as described above.

[0015] According to another aspect of this application, a computer storage medium is provided, wherein at least one computer program is stored in the computer-readable storage medium, the at least one computer program being loaded and executed by a processor to implement the emoji input method as described above.

[0016] According to another aspect of this application, a computer program product is provided, comprising a computer program stored in a computer-readable storage medium; the computer program is read from and executed by a processor of a computer device from the computer-readable storage medium, causing the computer device to perform the emoji input method as described above.

[0017] The beneficial effects of the technical solutions provided in this application include at least the following:

[0018] During the user's voice input process, the terminal device (or the input method program installed on the terminal device) converts the user's voice information into text content and displays at least one recommended emoji in the candidate bar that is related to the emotional intent expressed by the text content. The user can directly select the appropriate recommended emoji from the candidate bar to display, without having to trigger the emoji tool entry separately and jump back and forth to the relevant page. This greatly reduces the number of interaction steps, thereby improving the efficiency of emoji input and also improving the continuity of voice input. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is an architecture diagram of a computer system provided in one embodiment of this application;

[0021] Figure 2 This is a schematic diagram illustrating the addition of emojis in a related technology provided in one embodiment of this application;

[0022] Figure 3 This is a schematic diagram of an emoji input method provided in one embodiment of this application;

[0023] Figure 4 This is a flowchart of an embodiment of the emoji input method provided in this application;

[0024] Figure 5 This is a flowchart of an embodiment of the emoji input method provided in this application;

[0025] Figure 6 This is a schematic diagram of an emoji input method provided in one embodiment of this application;

[0026] Figure 7 This is a flowchart of an embodiment of the emoji input method provided in this application;

[0027] Figure 8 This is a schematic diagram of an emoji input method provided in one embodiment of this application;

[0028] Figure 9 This is a flowchart of an embodiment of the emoji input method provided in this application;

[0029] Figure 10 This is a schematic diagram of an emoji input method provided in one embodiment of this application;

[0030] Figure 11 This is a schematic diagram of an emoji input method provided in one embodiment of this application;

[0031] Figure 12 This is a flowchart of an embodiment of the emoji input method provided in this application;

[0032] Figure 13 This is a flowchart of an embodiment of the emoji input method provided in this application;

[0033] Figure 14 This is a schematic diagram of an emoji input method provided in one embodiment of this application;

[0034] Figure 15 This is a flowchart of an embodiment of the emoji input method provided in this application;

[0035] Figure 16 This is a flowchart of an embodiment of the emoji input method provided in this application;

[0036] Figure 17 This is a schematic diagram of an emoji input method provided in one embodiment of this application;

[0037] Figure 18This is a flowchart of an embodiment of the emoji input method provided in this application;

[0038] Figure 19 This is a schematic diagram of an emoji input method provided in one embodiment of this application;

[0039] Figure 20 This is a schematic diagram of an emoji input method provided in one embodiment of this application;

[0040] Figure 21 This is a schematic diagram of the system architecture of an emoji input method provided in one embodiment of this application;

[0041] Figure 22 This is a structural block diagram of an emoji input device provided in one embodiment of this application;

[0042] Figure 23 This is a schematic diagram of the structure of a computer device provided in one embodiment of this application. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0044] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0045] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0046] It should be understood that although the terms first, second, etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, a first parameter may also be referred to as a second parameter, and similarly, a second parameter may also be referred to as a first parameter. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0047] It should be noted that this application may display prompt interfaces, pop-ups, or output voice prompts before and during the collection of user data. These prompt interfaces, pop-ups, or voice prompts are used to inform the user that their data is being collected. This ensures that the application only begins the steps for collecting user data after receiving confirmation from the user regarding the prompt interface or pop-up; otherwise (i.e., without user confirmation), the steps for collecting user data end, meaning no user data is collected. In other words, all user data collected in this application is collected with the user's consent and authorization, and the collection, use, and processing of related user data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0048] First, let me introduce the relevant terms used in this application:

[0049] Voice input method: This refers to a type of input method that uses Automatic Speech Recognition (ASR) technology to achieve text input. Voice input methods primarily use speech-to-text technology to convert the user's spoken input into text, thereby enabling text input.

[0050] Candidate bar: This refers to the area in the input method panel used to display candidate content. The candidate bar is typically located at the top of the input method program. For example, the candidate bar displays recommended emoticons suggested by the input method program. Optionally, when the user inputs voice, the input method program will display a series of related recommended emoticons in the candidate bar based on the user's current voice input.

[0051] Emojis: Emojis are graphic characters used to express emotions, intentions, or feelings. Optionally, emojis include at least one of Emojis, emoticons, and combinations of emoticons. Emojis are commonly used in chat, social media, or text messaging to complement text in expressing emotions or amplifying tone.

[0052] Input method program: An input tool used on a terminal device. Users can input text using an input method program, such as entering messages in chat software, entering search keywords on web pages, and entering text in documents. Input method programs typically provide multiple input methods, such as Pinyin input, handwriting input, and voice input, allowing users to choose the appropriate method according to their preferences and needs. This application embodiment mainly uses voice input as an example for description.

[0053] Figure 1A schematic diagram of the architecture of a computer system provided in one embodiment of this application is shown. The computer system may include: a terminal 100 and a server 120.

[0054] Terminal 100 can be an electronic device such as a mobile phone, tablet computer, in-vehicle terminal (vehicle system), wearable device, personal computer (PC), intelligent voice interaction device, smart home appliance, or unmanned vending terminal. A client application for the target application can be installed and run on terminal 100. This target application can be an application that supports an input method program; this application is not limited in this respect. Furthermore, this application does not limit the form of the target application, including but not limited to applications (Apps), mini-programs, etc., installed on terminal 100, and it can also be in the form of a webpage.

[0055] Server 120 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services such as cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and other basic cloud computing services. Server 120 can be a backend server for the aforementioned target application, used to provide backend services to the clients of the target application.

[0056] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Based on the cloud computing business model, cloud technology encompasses network technology, information technology, integration technology, management platform technology, and application technology. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0057] In some embodiments, the server described above can also be implemented as a node in a blockchain system. Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and cryptographic algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.

[0058] Terminal 100 and server 120 can communicate via a network, such as a wired or wireless network.

[0059] The emoji input method provided in this application embodiment can be executed by a computer device, which refers to an electronic device with data computing, processing, and storage capabilities. Figure 1 Taking the implementation environment of the scheme shown as an example, the method can be executed by the terminal 100 (such as the client execution method of the target application installed and running in the terminal 100), or by the server 120, or by the interaction and cooperation between the terminal 100 and the server 120. This application does not limit this.

[0060] When users use voice input methods, the methods convert the user's speech into text. However, voice input methods cannot accurately understand the user's emotional expressions, thus creating a need to add emojis to the text. Related technologies, in conjunction with references... Figure 2 On the voice input interface 200, after the user converts the voice message to be sent into text using the voice input method, such as... Figure 2 As shown in Figure (a), the user then triggers the emoji entry 201 in the input method toolbar. Optionally, in response to the triggering operation on the emoji entry 201, as follows: Figure 2 As shown in Figure (b), the emoticon module related page 202 is displayed. The emoticon module related page 202 displays an emoticon area 203, which includes a preset number of emoticons. Users can select one or more emoticons from the emoticon area 203 to add to the text as needed. Figure 2 As shown in Figure (c), the user adds the selected emoji 204 to the text box 205. After adding the emoji, the user decides whether to return to the voice input interface to continue voice input based on their personal needs. However, the method of adding emojis in related technologies requires a large amount of human-computer interaction, which is relatively cumbersome.

[0061] Based on this, in order to solve the problems existing in the related technologies, this application provides a method for inputting emojis. Figure 3This illustration shows a schematic diagram of an emoji input method provided in an exemplary embodiment of this application. The method is executed by a computer device, with the computer device serving as the... Figure 1 The following explanation uses terminal 100 as an example. The steps for installing and running the input method program on terminal 100 are briefly described below:

[0062] 1) Display the input window containing the voice control 10.

[0063] The voice control 10 is a control displayed on the input method program. An input method program is an input tool used on a terminal device. Input method programs typically provide multiple input methods, such as Pinyin input, handwriting input, and voice input. This embodiment mainly uses voice input as an example for illustration.

[0064] Optionally, in response to a trigger operation on the voice control 10, the voice input method in the input method program is activated, and the user can input voice content through the voice control 10. The input method program then converts the user's voice content into text content.

[0065] Optionally, the input window can be a chat interface in an instant messaging application, a playback interface on a video platform, a comment interface on an audio platform, or a comment interface on a social media platform. This application embodiment uses a chat interface of an instant messaging application as an example for illustration.

[0066] 2) In response to a voice input operation on the voice control 10, at least one recommended emoji is displayed in the candidate bar 20.

[0067] The candidate bar 20 refers to the area in the input method program used to display recommended emojis. Recommended emojis are emojis recommended by the input method program based on the text content corresponding to the input voice information. Emojis are graphic characters used to express emotions, intentions, or feelings. Optionally, emojis include at least one of Emojis, emoticons, and combined emojis. Emojis are commonly used in chat, social media, or text messages to help express emotions or strengthen tone in text content.

[0068] In some embodiments, the recommended emojis displayed in the candidate bar 20 include two types of emojis. The first type of recommended emojis is obtained by the input method program based on a cloud-based emoji library, and the second type of recommended emojis is obtained by the input method program based on the user's local emoji library. The cloud-based emoji library is an emoji library stored in a cloud database. The local emoji library is an emoji library stored by the user on their terminal device. Optionally, the local emoji library includes a user dictionary and a list of frequently used emojis. The user dictionary is an emoji library that the user has downloaded, and the list of frequently used emojis is a list of emojis that the user uses frequently.

[0069] For example, such as Figure 3 As shown in Figure (a), the input window is an example of a chat interface in an instant messaging program. A voice control 10 is displayed on the chat interface; the voice control 10 is a control in the input method program. In response to a voice input operation on the voice control 10, the input method program converts the input voice content 11 into text content 12, for example, the text content 12 is "Today, I met a good friend," and displays at least one recommended emoticon in the candidate bar 20. The at least one recommended emoticon is an emoticon associated with the emotional state of the text content 12.

[0070] 3) In response to the screen operation of the first emoji 21, the first emoji 21 is screened onto the text input area.

[0071] In some embodiments, the candidate bar 20 includes at least one recommended emoji. The at least one recommended emoji includes a first emoji 21, which is a recommended emoji selected by the user.

[0072] In some embodiments, in response to a first interactive operation on a target emoji 21, a single first emoji 21 is displayed on the text input area. The first interactive operation is an operation used to trigger the display of the single first emoji 21. For example, the first interactive operation is a click operation, where clicking the first emoji 21 displays the first emoji 21 on the text input area.

[0073] In some embodiments, in response to a second interactive operation on a first emoji 21, at least one first emoji 21 is displayed on the text input area. The second interactive operation is used to trigger an increase or decrease in the number of target emojis 21. Optionally, the second interactive operation is a continuous, uninterrupted operation. For example, in response to a selection operation on a first emoji 21, a right swipe operation increases the number of first emojis 21; or a left swipe operation decreases the number of first emojis 21, displaying at least one first emoji 21 on the text input area.

[0074] In some embodiments, in response to a third interactive operation on the first emoji 21, the first emoji 21 is displayed on the screen at a preset position in the text input area. The third interactive operation is used to trigger the addition of the first emoji 21 to the preset position in the text input area. Optionally, the third interactive operation is a continuous, uninterrupted operation. For example, in response to a selection operation on the first emoji 21, an upward swipe operation wakes up the cursor 40, and a leftward or rightward swipe operation controls the position of the cursor 40 in the text input area, thus displaying the first emoji 21 on the screen at the preset position in the text input area.

[0075] In some embodiments, the second and third interactive operations can be triggered together. Optionally, after determining the number of first emojis 21 based on the second interactive operation, the third interactive operation is used to display a preset number of first emojis 21 at preset positions in the text input area.

[0076] For example, such as Figure 3 As shown in Figure (a), the candidate bar 20 includes a first emoji 21. After selecting the first emoji 21, in response to a rightward swipe operation on the first emoji 21, as shown... Figure 3 As shown in Figure (b), the number of first emojis 21 is increased to 3, and a list 30 of the number of emojis corresponding to the first emojis 21 is displayed. Optionally, in response to an upward swipe operation on the list 30 of the number of emojis, as shown in Figure (b), the following steps are taken: Figure 3 As shown in Figure (c), the cursor 40 is displayed after being woken up. By sliding the cursor 40, it can be moved to a preset position. Releasing the slider will then... Figure 3 As shown in Figure (d), the emojis in the emoji list 30 are displayed at preset positions in the text input area. For example, the three first emojis 21 are displayed at the position between "Today," and "Met a good friend."

[0077] Figure 4 This is a flowchart illustrating an exemplary embodiment of an emoji input method provided in this application. The method is executed by a computer device, which may be... Figure 1 The terminal 100 shown is equipped with an input method program. The method includes at least a portion of steps 210, 220, and 230:

[0078] Step 210: Acquire voice information during the speech-to-text input process;

[0079] Voice information refers to the audio content that a user sends to a terminal device via voice input.

[0080] In some embodiments, the terminal device is equipped with a voice acquisition component, which is used to receive voice information input by the user. For example, the voice acquisition component is a microphone array, consisting of multiple microphones, capable of acquiring voice information emitted by the user.

[0081] Optionally, the voice acquisition component can be activated in ways including but not limited to: triggering the activation of the virtual display control of the voice acquisition component; triggering the activation of a physical button that is bound to the voice acquisition component; or activating the voice acquisition component by waking it up with a voice command.

[0082] For example, the voice acquisition component can be activated by triggering the virtual display control of the voice acquisition component. Optionally, an input method program is installed on the terminal device, and a voice control (virtual display control) is displayed on the input method program. In response to a trigger operation on the voice control, the input method program acquires the user's voice information during the speech-to-text input process.

[0083] For example, activating the voice acquisition component includes triggering a physical button that is bound to the voice acquisition component. Optionally, a physical button bound to the voice acquisition component is configured in the terminal device. For example, the physical button is a volume control button on the terminal device. Optionally, an input method program is installed on the terminal device. The input method program can acquire the voice information received by the terminal device. In response to the pressing operation of the physical button, the input method program acquires the voice information input by the user during the speech-to-text input process.

[0084] For example, the voice acquisition component can be activated via a voice command. Optionally, an input method program is installed on the terminal device, which supports a voice assistant, and the voice acquisition component has the function of receiving voice commands. In response to the wake-up operation for a voice command, the input method program acquires the user's voice information during the speech-to-text input process. For example, the user's voice command to the voice assistant might be, "Please help me write a paragraph, the content of which is XXX."

[0085] It should be noted that the above are merely illustrative examples, and this application does not limit the way in which the input method program obtains voice information.

[0086] In some embodiments, during the speech-to-text input process, firstly, the user's voice information is acquired through the voice acquisition component in the terminal device, and then the input method program converts the voice content in the voice information into text content through voice recognition technology.

[0087] Step 220: Based on the voice information, display at least one recommended emoji in the candidate bar, wherein the at least one recommended emoji includes a first emoji;

[0088] The candidate bar refers to the area in the input method program used to display candidate content. Optionally, the candidate content includes multiple options, and the candidate types of the multiple options include at least one of the following: recommended emojis, recommended text content, and a combination of recommended emojis and recommended text content.

[0089] In some embodiments, the candidate content includes multiple candidate options, and the candidate type of the multiple candidate options includes recommended emojis. Optionally, at least one recommended emoji is displayed in the candidate bar. Recommended emojis are emojis obtained by the input method program based on the text content corresponding to the input voice information. Emojis refer to graphic characters used to express emotions, intentions, or feelings. Optionally, emojis include, but are not limited to, at least one of Emojis, emoticons, and combined emojis. Emojis are used to assist text content in expressing emotions or strengthening tone.

[0090] Optionally, recommended emojis are associated with the user's emotional state. For example, if the user's emotional state is positive, the recommended emojis are positive, such as [smiley face], [laughing], and [hug] emojis; if the user's emotional state is negative, the recommended emojis are negative, such as [disappointed], [sad], and [crying] emojis. By displaying associated emojis in the candidate bar, the input method program can help users express their emotions more accurately and richly.

[0091] For example, the user inputs the voice message "wojintianzailubiankandaoleyiduoxiaohua", and the corresponding text content is "I saw a small flower on the roadside today". At least one recommended emoji is displayed in the candidate bar. The recommended emoji is one corresponding to a positive emotion, such as a smiley face emoji or other emojis that express happiness.

[0092] In some embodiments, the candidate content includes multiple candidate options, and the candidate type of the multiple candidate options includes recommended text content. Optionally, at least one piece of recommended text content is displayed in the candidate bar. Recommended text content refers to the text content displayed in the candidate bar. Optionally, the recommended text content may be exactly the same as the text content corresponding to the voice information, or it may not be exactly the same as the text content corresponding to the voice information. For example, the recommended text content is an optimized expression of the text content corresponding to the voice information.

[0093] For example, the user inputs the voice message "wojintianzailubiankandaoleyiduoxiaohua", and the corresponding text content is "I saw a small flower on the roadside today". At least one recommended text content is displayed in the candidate bar. For instance, the recommended text content at candidate position A is "I saw a small flower on the roadside today", and the recommended text content at candidate position B is "I saw a beautiful small flower on the roadside today". The recommended text content at candidate position B is an optimized expression of the text content.

[0094] In some embodiments, the candidate content includes multiple candidate options, and the candidate types of these multiple candidate options include a combination of recommended emojis and recommended text content. Optionally, at least one combination of recommended emojis and recommended text content is displayed in the candidate bar. "Combination" refers to the input method program simultaneously providing a combination option of emojis and text content in the candidate bar, allowing users to use both text and emojis when expressing themselves. The recommended emojis are emojis obtained by the input method program based on the text content corresponding to the input voice information. The recommended text content is the text content obtained by the input method program based on the voice information. The recommended text content may be exactly the same as the text content corresponding to the voice information, or it may not be exactly the same as the text content corresponding to the voice information.

[0095] For example, the user inputs the voice message "wojintianzhenkaixin", and the corresponding text content is "I'm really happy today". The candidate bar displays a combination of recommended emojis and recommended text content, for example, the combination is displayed as "I'm really happy today [smiley face]".

[0096] Step 230: In response to the on-screen operation for the first emoji, on-screen the first emoji to the text input area.

[0097] In some embodiments, at least one recommended emoji is displayed in the candidate bar. Optionally, the at least one recommended emoji includes a first emoji. The first emoji is the recommended emoji selected by the user in the candidate bar.

[0098] In some embodiments, in response to a loading operation for a first emoji, the first emoji is loaded onto the text input area. The loading operation refers to the user moving the selected element (the first emoji) from the candidate area of ​​the candidate bar to the text input area. Optionally, the text input area is an editable area displaying text content.

[0099] Optionally, the first emoji is displayed on the text input area. The display position of the first emoji in the text input area includes an automatic display position and a manual display position. The automatic display position is where the first emoji automatically appears in the text input area after being triggered. The manual display position is where the first emoji appears in the text input area following the user's touch operation after being triggered.

[0100] The following section explains the automatic placement of the first emoji.

[0101] In some embodiments, during the speech-to-text input process, in response to the operation of displaying the first emoticon, the first emoticon is automatically displayed at the default position of the text input area, that is, the automatic display position is the default position; or, in response to the operation of displaying the first emoticon, the first emoticon is automatically displayed at the keyword position of the text input area, that is, the automatic display position is the keyword position; or, in response to the operation of displaying the first emoticon, the first emoticon is automatically displayed at another preset position of the text input area, that is, the automatic display position is another preset position.

[0102] The default position is the end of the text content corresponding to the voice message. The keyword position is the position adjacent to the keyword in the text content corresponding to the voice message. Other preset positions are areas that can be set in advance. For example, if the text content corresponding to the user's voice message is two sentences, the other preset position is the middle of the two sentences.

[0103] For example, the user inputs the voice message "wojintianhehaopengyoujianmianlewomenyiqichifan", and the corresponding text content is "I met my good friend today, and we ate together". Optionally, the first emoji displayed in the candidate bar is the [happy] emoji.

[0104] For example, in response to the on-screen operation for the first emoji, the [happy] emoji is automatically on-screened at the end of the text content, and the text input area displays "I met my good friend today, and we ate together [happy]".

[0105] For example, in response to the operation of displaying the first emoji, the [Happy] emoji is automatically displayed in the text content next to the keyword. Taking the keyword "good friend" as an example, the text input area displays "I met my good friend [Happy] today, and we ate together."

[0106] For example, in response to the on-screen operation of the first emoji, the [happy] emoji is automatically on-screened in another preset position in the text content. The other preset position is the middle position between two sentences, and the text input area displays "I met my good friend today [happy], we ate together".

[0107] In summary, the method provided in this application, during the user's voice input process, converts the user's voice information into text content and displays at least one recommended emoticon in the candidate bar that is related to the emotional intent expressed by the text content. The recommended emoticon displayed in the candidate bar is related to the user's emotional intent, and the user can directly select the recommended emoticon from the candidate bar to display, without having to separately trigger the emoticon tool entry and jump back and forth to related pages. This greatly reduces the number of interaction steps, improves the efficiency of emoticon input, and also improves the continuity of voice input.

[0108] In some embodiments, the candidate content includes multiple candidate options, and the candidate type of the multiple candidate options includes at least one of the following: recommended emoji, recommended text content, and a combination of recommended emoji and recommended text content. Optionally, when the candidate type of the multiple candidate options includes recommended emoji, it can also be understood that at least the recommended emoji is displayed among the multiple candidate options. Optionally, when the candidate type of the multiple candidate options includes recommended text content, it can also be understood that at least the recommended text content is displayed among the multiple candidate options. Optionally, when the candidate type of the multiple candidate options includes a combination of recommended emoji and recommended text content, it can also be understood that at least the combination of recommended emoji and recommended text content is displayed among the multiple candidate options.

[0109] The following sections will introduce the different candidate types, that is, the content displayed in the candidate column.

[0110] The candidate bar displays at least one recommended emoji.

[0111] In some embodiments, during the speech-to-text input process, the input method program acquires the user's voice information and displays the corresponding text content in the text input area. Optionally, recommended emojis corresponding to the text content are displayed in the candidate bar.

[0112] Figure 5 This is a flowchart of an emoji input method provided in an exemplary embodiment of this application. Step 220 above can be replaced by steps 221 and 222.

[0113] Step 221: Display the text content in the text input area. The text content is obtained by converting the speech information to text.

[0114] Among them, the text input area is an editable area for displaying text content. For example, the text input area is an input box or a text box.

[0115] In some embodiments, the text content corresponding to the voice information is displayed in the text input area, and the text content is obtained by the input method program based on voice recognition technology; or, a voice recognition model is installed in the input method program, and the text content is obtained by the input method program based on the voice recognition model. This application does not limit this.

[0116] In some embodiments, after the input method program obtains the voice information, it converts the audio content in the voice information into text content based on voice recognition technology. Among them, the text content is obtained after performing speech-to-text on the voice information. Exemplarily, the voice information input by the user is "wojintianzailubiankanda oleyiduoxiaohua," and the text content corresponding to the voice information is "我今天在路边看到了一朵小花".

[0117] Step 222: Display at least one recommended emoji in at least one first candidate item in the candidate bar.

[0118] Among them, the candidate bar refers to the area in the input method program for displaying candidate content.

[0119] Optionally, the candidate content includes multiple candidate items, the multiple candidate items include first candidate items, and at least one recommended emoji is displayed in at least one first candidate item in the candidate bar. Among them, the first candidate item is the candidate item in the candidate bar for displaying recommended emojis.

[0120] In some embodiments, the recommended emoji is an emoji obtained by the input method program based on the text content corresponding to the input voice information. Among them, an emoji refers to a graphical character used to express emotions, intentions, or moods. Optionally, emojis include, but are not limited to, at least one of Emoji (expressions), kaomoji, and combined emojis. Emojis are used to assist the text content in expressing emotions or strengthening the tone.

[0121] Exemplarily, with reference to Figure 6 , on the voice input interface 300, during the process of speech-to-text, the input method program obtains the voice information 301 input by the user, converts the voice information 301 into text content, and displays the text content in the text input area 302. For example, the text content displayed in the text input area 302 is "今天,见到了好朋友". Optionally, the input method program displays 5 recommended emojis related to the text content in the candidate bar 303, and the recommended emojis include Emoji (expressions) and kaomoji.

[0122] Optionally, at least one recommended emoji displayed in the candidate bar is displayed in a random arrangement, or at least one recommended emoji displayed in the candidate bar is displayed in a sequential order. This application does not limit this.

[0123] In one possible implementation, at least one recommended emoji displayed in the candidate bar is randomly arranged, and each of the at least one recommended emoji is associated with the emotion expressed by the text content.

[0124] In one possible implementation, at least one recommended emoji is displayed in the candidate bar. Optionally, the at least one recommended emoji is sorted from highest to lowest according to its match with the text content. Optionally, the top n recommended emojis in the ranking are identified as the first emoji and highlighted, where n is a positive integer. For example, the first recommended emoji in the ranking is identified as the first emoji and highlighted with a bright background color to indicate that the first emoji has a high match with the current text content.

[0125] For example, in conjunction with the reference Figure 6 The candidate bar 303 displays five recommended emojis. Optionally, the five recommended emojis displayed in the candidate bar 303 are arranged according to their matching degree with the text content. For example, if the recommended emoji 303a ranked first in the candidate bar 303 is considered to have the highest matching degree with the text content, then recommended emoji 303a is displayed with a highlighted background color.

[0126] In some embodiments, at least one recommended emoji is related to at least one of the following factors: the semantics of the voice message, the tone of the voice message, the emotion of the voice message, the background sound of the voice message, the semantics of the text content, and the keywords in the text content.

[0127] In some embodiments, at least one recommended emoji is semantically related to the voice information. Here, the semantics of the voice information refers to the actual meaning expressed in the speech. For example, if the voice content mentions emotional words such as "happy" or "joyful," the input method program may recommend emojis related to happiness or joy to quickly reflect the user's emotional state.

[0128] In some embodiments, at least one recommended emoji is related to the tone of the voice message. The tone of the voice message refers to the attitude expressed in the voice message. The tone of the voice message includes, but is not limited to, at least one of the following: questioning tone, surprised tone, calm tone, sad tone, and angry tone. The user's current attitude is determined by analyzing the tone of the voice message, and corresponding emojis are recommended in the candidate bar accordingly. For example, during speech-to-text conversion, if the user's voice message is full of surprise, such as "Really?", the input method program may display [surprised] or [questioning] emojis in the candidate bar.

[0129] In some embodiments, at least one recommended emoji is associated with the emotion of the voice message. The emotion of the voice message refers to the emotional state expressed in the speech. Optionally, the emotional state is used to represent the user's current emotional type; for example, emotional states include happy, excited, thrilled, humorous, sad, angry, and depressed states. Emotional keywords are words related to the user's emotions. For example, during speech-to-text conversion, if the user is currently in a depressed mood and the converted text is "Didn't do well on today's exam," the input method program might display an emoji related to "[sad]" in the candidate bar.

[0130] In some embodiments, at least one recommended emoticon is associated with the background noise of the voice message. The background noise of the voice message refers to sound elements other than the user's voice. For example, background noise might be ambient sounds, music, wind, traffic noise, etc., used to enhance the atmosphere. Optionally, the background noise of the voice message is used to assist the input method program in recognizing the user's emotions. For example, during speech-to-text conversion, if there are keyboard sounds in the user's environment, the input method program might display emoticons of the "[Busy]" category in the candidate bar. As another example, during speech-to-text conversion, if the user's background noise includes soft music, it may indicate that the user's current mood is relatively peaceful, and the input method program might display emoticons of the "[Happy]" category in the candidate bar.

[0131] In some embodiments, at least one recommended emoji is semantically related to the text content. Here, the semantics of the text content refers to the actual meaning expressed by the text content after the speech information is converted into text. Optionally, after the input method program converts the speech information into text content using speech recognition technology, it can analyze the semantics of the text content to recommend emojis that match the theme or emotion of the text content. For example, if the user's voice input is converted into text content such as "Congratulations on getting first place today," the input method program might recognize the word "congratulations" and display emojis like [cheering] or [fireworks] in the candidate bar.

[0132] In some embodiments, at least one recommended emoji is associated with keywords (or sentiment keywords) in the text content. Keywords in the text content refer to key words or phrases that appear in the text and can indicate the user's emotions or intentions. In other embodiments, keywords may also be called sentiment keywords. For example, after a user's voice input is converted to text content, such as "I'm so tired today," the input method program might recognize the phrase "so tired" and use it as a sentiment keyword, displaying emojis of the "[tired]" category in the candidate bar.

[0133] In some embodiments, at least one recommended emoji is associated with the personalized state of the input window. For example, if the input window displays a user-set personalized state of "busy," the input method program obtains the personalized state of the input window and displays an emoji of the "[busy]" category in the candidate bar. As another example, if the input window displays a user-set personalized state of "listening to music," the input method program obtains the personalized state of the input window and displays an emoji of the "[happy]" category in the candidate bar.

[0134] In some embodiments, at least one recommended emoji is associated with the user's current geographic location. Optionally, the input method program obtains the user's current geographic location, determines the user's status based on the location, and displays recommended emojis in the candidate bar according to the user's status. For example, if the input method program finds that the user's current geographic location is a gym, it displays emojis of the [sweating] category in the candidate bar.

[0135] In some embodiments, at least one recommended emoji may be related to one of the following factors: the semantics of the voice information, the tone of the voice information, the emotion of the voice information, the background sound of the voice information, the semantics of the text content, and the keywords of the text content; or, it may be related to at least two of the above factors (the semantics of the voice information, the tone of the voice information, the emotion of the voice information, the background sound of the voice information, the semantics of the text content, and the keywords of the text content). Optionally, if at least one recommended emoji displayed in the candidate bar is related to the above factors, at least one recommended emoji is displayed in the candidate bar according to the weight ratio of each factor.

[0136] The above are merely illustrative examples, and this application does not limit the factors related to the recommended emojis displayed in the candidate bar. For example, in other embodiments, at least one recommended emoji displayed in the candidate bar is related to the text box type. This application does not limit this.

[0137] In this embodiment, users can quickly input text content via voice, while at least one recommended emoji is displayed in the candidate bar, improving the efficiency of expressing personal emotions and intentions. Specifically, the input method program can refer to factors related to the recommended emojis, analyzing the semantics and tone of the voice information, as well as the semantics and keywords of the text content, to make the recommended emojis more accurate and improve input accuracy. Furthermore, this intelligent recommendation enhances the richness of user expression and the efficiency of communication, meeting users' personalized expression needs.

[0138] The candidate bar displays recommended emojis and text content separately.

[0139] In some embodiments, during the speech-to-text input process, the input method program acquires the user's voice information, displays at least one recommended emoji in at least one first candidate option in the candidate bar, and displays text content in a second candidate option in the candidate bar.

[0140] Figure 7 This is a flowchart of an emoji input method provided in an exemplary embodiment of this application. Step 220 above can be replaced by step 223.

[0141] Step 223: Display at least one recommended emoji in at least one first candidate in the candidate bar; and display text content in the second candidate.

[0142] The candidate bar refers to the area in the input method program used to display candidate content. The text content is obtained by converting the voice information to text. Recommended emoticons are emoticons generated by the input method program based on the text content corresponding to the input voice information.

[0143] Optionally, the candidate content includes multiple candidate options, including a first candidate option and a second candidate option. At least one recommended emoji is displayed in at least one first candidate option in the candidate column; and text content is displayed in the second candidate option in the candidate column.

[0144] The first candidate is the one displayed as a recommended emoji in the candidate bar. The second candidate is the one displayed as text content in the candidate bar.

[0145] In some embodiments, the text content displayed in the candidate column may also be referred to as recommended text content.

[0146] Optionally, the text content displayed in the candidate column may be exactly the same as the text content corresponding to the voice information; or, the text content displayed in the candidate column may not be exactly the same as the text content corresponding to the voice information. For example, the text content displayed in the candidate column may be an optimized expression of the text content corresponding to the voice information.

[0147] For example, the user inputs the voice message "wojintianzailubiankandaoleyiduoxiaohua", and the corresponding text content is "I saw a small flower on the roadside today". In candidate position A of the candidate bar, the recommended text content displayed is "I saw a small flower on the roadside today", and in candidate position B of the candidate bar, the recommended text content displayed is "I saw a beautiful small flower on the roadside today". The recommended text content in candidate position B is an optimized expression of the text content.

[0148] In some embodiments, when the candidate bar includes a first candidate for displaying recommended emojis and a second candidate for displaying text content, the display layout of the candidate bar can take various forms. The display layout of the candidate bar refers to the way the first and second candidates are optimized and arranged to make them clearer, easier to read, and more aesthetically pleasing. Optionally, the display layout of the candidate bar includes, but is not limited to, at least one of grid layout, list layout, and sliding tabs.

[0149] For example, the candidate bar display layout includes a grid layout, arranging the first and second candidates in a grid, with the icons or names of the first and second candidates occupying a fixed-size square. Users can view the first and second candidates by scrolling or paginating. For example, the candidate bar display layout includes a list layout, presenting the first and second candidates in a vertical list format, arranged sequentially. Users can browse the first and second candidates by scrolling. For example, the candidate bar display layout includes a sliding tab, placing a sliding tab in the candidate bar. Based on the sliding tab, recommended emojis for the first candidate and text content for the second candidate are displayed. Users can switch between viewing the first and second candidates by sliding or clicking the sliding tab.

[0150] The above are merely illustrative examples, and this application does not limit the display layout of the candidate column.

[0151] For example, in conjunction with the reference Figure 8In the voice input interface 400, during the speech-to-text process, the input method program obtains the user-input voice information 401, for example, voice information 401 is "wozhailubiank andaoyizhimao". At least one recommended emoticon is displayed in the first candidate option 402a of the candidate bar 402, and text content is displayed in the second candidate option 402b, where the at least one recommended emoticon is associated with the emotion expressed in the text content. Optionally, the text content displayed in the second candidate option 402b (also referred to as recommended text content) can be the text content corresponding to voice information 401, for example, displayed as "I saw a cat on the roadside"; the text content displayed in the second candidate option 402b (also referred to as recommended text content) can be an optimized expression of the text content corresponding to voice information 401, for example, displayed as "I saw a very cute cat on the roadside today".

[0152] In one possible implementation, when both recommended emojis and text content are displayed simultaneously in the candidate bar, in response to a text content upload operation, the text content is uploaded to the text input area, and the recommended emojis remain in the candidate bar. This can also be understood as the candidate bar not disappearing due to the text content upload operation. Optionally, in response to a recommended emoji upload operation, the selected recommended emoji is uploaded to the text input area.

[0153] For example, in conjunction with the reference Figure 8 In the candidate bar 402, candidate recommended emojis and text content are displayed simultaneously. In response to the screen operation of the text content "I saw a very cute cat on the roadside today" displayed in the second candidate 402b, "I saw a very cute cat on the roadside today" is screened onto the text input area, and at least one recommended emoji continues to be displayed in the first candidate 402a in the candidate bar 402.

[0154] In some embodiments, at least one recommended emoji is related to at least one of the following factors: the semantics of the voice message, the tone of the voice message, the emotion of the voice message, the background sound of the voice message, the semantics of the text content, and the keywords in the text content. Wherein, the semantics of the voice message refers to the actual meaning expressed in the voice message. The tone of the voice message refers to the attitude expressed in the voice message. The emotion of the voice message refers to the emotional state expressed in the voice message. The semantics of the text content refers to the actual meaning expressed in the text content after the voice message is converted into text content. The keywords in the text content refer to keywords or key phrases appearing in the text content; keywords can indicate the user's emotion or intention.

[0155] For details regarding at least one of the above factors, please refer to the relevant description in step 222, which will not be repeated here.

[0156] In this embodiment, when processing speech-to-text, the input method program displays two types of text content in the candidate bar. When the text content displayed in the candidate bar is exactly the same as the text content corresponding to the speech information, the user can immediately see that what they said has been accurately converted into text, improving the consistency and accuracy of input. The text content in the candidate bar can also be an optimized expression, which the user can directly select without manual editing. Users can directly select the exact same or optimized text content from the candidate bar to quickly complete the input, improving the overall input efficiency.

[0157] The candidate bar displays a combination of recommended emojis and text content.

[0158] In some embodiments, during the speech-to-text input process, the input method program obtains the user's voice information and displays recommended emojis and combinations of recommended emojis in the third candidate option in the candidate bar.

[0159] Figure 9 This is a flowchart of an emoji input method provided in an exemplary embodiment of this application. Step 220 above can be replaced by step 224.

[0160] Step 224: Display a combination of text content and recommended emojis in the third candidate option in the candidate bar.

[0161] In some embodiments, the candidate content includes multiple candidate options, which include a first candidate option, a second candidate option, and a third candidate option. At least one recommended emoji is displayed in at least one first candidate option in the candidate bar, text content is displayed in the second candidate option in the candidate bar, and a combination of text content and recommended emoji is displayed in the third candidate option in the candidate bar.

[0162] The first candidate is the one displayed as the recommended emoji. The second candidate is the one displayed as the text content. The third candidate is the one displayed as a combination of the recommended emoji and the text content.

[0163] In some embodiments, a combination of text content and recommended emojis is displayed in the third candidate option of the candidate bar. The text content in the combination may be exactly the same as the text content corresponding to the voice information; or, the text content in the combination may not be exactly the same as the text content corresponding to the voice information. For example, the text content in the combination may be an optimized expression of the text content corresponding to the voice information. This application does not limit this.

[0164] Optionally, the third candidate in the candidate bar displays a combination of preset recommended emojis and text content. The preset recommended emojis can be considered as the recommended emojis that best match the text content.

[0165] In some embodiments, when the candidate bar includes a first candidate for displaying a recommended emoji, a second candidate for displaying text content, and a third candidate for displaying a combination of recommended emojis and text content, the display layout of the candidate bar can take various forms. Optionally, the display layout of the candidate bar includes, but is not limited to, at least one of a grid layout, a list layout, and a sliding tab.

[0166] For example, in conjunction with the reference Figure 10 In the voice input interface 500, during the speech-to-text process, the input method program acquires the user's voice input 501, for example, the voice input 501 is "wozhailubiank andaoyizhimao". At least one recommended emoticon is displayed in the first candidate option 502a of the candidate bar 502, text content is displayed in the second candidate option 502b, and a combination of recommended emoticons and text content is displayed in the third candidate option 502c. For example, the text content displayed in the second candidate option 502b is "I saw a cat on the roadside", and the combination of recommended emoticons and text content displayed in the third candidate option 502c is "I saw a cat on the roadside [happy]".

[0167] In one possible implementation, when the candidate bar simultaneously displays recommended emojis, text content, and a combination of recommended emojis and text content, in response to a text content upload operation, the text content is uploaded to the text input area, while the recommended emojis and combinations of recommended emojis and text content remain in the candidate bar. This can also be understood as the candidate bar not disappearing due to the text content upload operation. Optionally, in response to a recommended emoji upload operation, the selected recommended emoji is uploaded to the text input area.

[0168] In one possible implementation, when the text content corresponding to the voice information is already displayed in the text input area, in response to the on-screen operation for the combination of recommended emojis and text content, the text content displayed in the text input area is replaced with the combination of recommended emojis and text content.

[0169] For example, in conjunction with the reference Figure 11 In the voice input interface 600, during the voice-to-text process, the text content corresponding to the voice information is displayed in the text input area 601, such as... Figure 11As shown in Figure (a), the text content is displayed as "I saw a cat on the street". At least one recommended emoji is displayed in the first candidate 602a of the candidate bar 602, and a combination of recommended emoji and text content is displayed in the third candidate 602b, for example, the combination of recommended emoji and text content is "I saw a cat on the street [happy]". Optionally, in response to an on-screen operation for the combination of recommended emoji and text content, such as... Figure 11 As shown in Figure (b), the text content in the text input area 601 is replaced with a combination of recommended emojis and text content.

[0170] In some embodiments, at least one recommended emoji is related to at least one of the following factors: the semantics of the voice information, the tone of the voice information, the emotion of the voice information, the semantics of the text content, and the keywords in the text content. Wherein, the semantics of the voice information refers to the actual meaning expressed in the voice. The tone of the voice information refers to the attitude expressed in the voice information. The emotion of the voice information refers to the emotional state expressed in the voice. The semantics of the text content refers to the actual meaning expressed in the text content after the voice information is converted into text content. The keywords in the text content refer to keywords or key phrases appearing in the text content; keywords can indicate the user's emotion or intention.

[0171] For details regarding at least one of the above factors, please refer to the relevant description in step 222, which will not be repeated here.

[0172] In this embodiment, by displaying a combination of recommended emojis and text content in the candidate bar, the recommended emojis in the combination can be considered as the recommended emojis with the highest matching degree with the text content, and the text content in the combination can be the text content corresponding to the voice information or optimized text content. By displaying a combination of text content and recommended emojis in the candidate bar, information delivery becomes more vivid and intuitive, providing users with more choices and meeting the personalized needs of different users.

[0173] Furthermore, users can directly select pre-combined recommended emojis and text content from the candidate bar, eliminating the need to select text content and recommended emojis separately. By completing the input in one step, time is saved and input efficiency is improved.

[0174] At least one source type for recommended emojis

[0175] In some embodiments, at least one recommended emoji displayed in the candidate bar includes two types of emojis. Optionally, the first type of recommended emoji is obtained by the input method program based on a cloud-based emoji library. This can also be understood as the first type of recommended emoji being obtained by the server based on the cloud-based emoji library, and the server returning the mapped first type of recommended emoji to the input method program. Optionally, the second type of recommended emoji is obtained by the input method program based on the user's local emoji library. Here, the cloud-based emoji library is an emoji library stored in a cloud database. The local emoji library is an emoji library stored by the user on their terminal device.

[0176] In some embodiments, for the first type of recommended emoji, the input method program obtains voice information, converts the voice information into text content, performs natural language processing on the text content to obtain the sentiment type result, and maps the sentiment type result to the cloud emoji library to obtain the first type of recommended emoji.

[0177] In some embodiments, for the second type of recommended emojis, the input method program obtains the local emoji library (including the user dictionary and the list of commonly used emojis), and uses emojis whose usage frequency in the local emoji library reaches a preset threshold as the second type of recommended emojis.

[0178] Optionally, the input method program merges the first type of recommended emojis and the second type of recommended emojis. That is, the input method program merges the recommended emojis obtained based on the cloud emoji library and the local emoji library, and displays at least one recommended emoji in the candidate bar.

[0179] The following example primarily uses emotional keywords in text content as the factors for selecting recommended emojis. Optionally, during the process of the input method program retrieving the first type of recommended emojis from the cloud emoji library, two different scenarios exist:

[0180] • First scenario: When there are sentiment keywords in the text content, the server directly extracts the sentiment keywords from the text content.

[0181] • The second scenario: When there are no sentiment keywords in the text content, the server analyzes and summarizes the text content to indirectly extract sentiment keywords from the text content.

[0182] The two situations described above will be introduced below.

[0183] In some embodiments, for cases where sentiment keywords exist in the text content, the server first segments the text content into words, directly extracting sentiment keywords, which are associated with the user's emotional state; sentiment detection is performed on the sentiment keywords to obtain the corresponding sentiment type result; the sentiment type result is mapped to emojis in a cloud-based emoji library to obtain the recommended emoji of the first type corresponding to the sentiment type result. Here, sentiment state is used to represent the user's current emotional type, for example, sentiment states include happy, excited, thrilled, humorous, sad, angry, and depressed states. Sentiment keywords are words related to the user's emotions.

[0184] For example, text content is displayed in the text input area, such as "I went hiking with my best friend today, and I was very happy after seeing the beautiful scenery." This text contains sentiment keywords; for example, "happy" can be used as a sentiment keyword, indicating that the user is currently in a happy state. After performing sentiment detection on the sentiment keyword "happy" to obtain the corresponding sentiment type result, it is mapped to emojis in the cloud emoji library, and the mapped recommended emojis are returned to the input method program. For example, the mapped recommended emojis might be emojis of the categories [happy], [excited], and [joyful].

[0185] In some embodiments, when no sentiment keywords exist in the text content, the server performs a holistic understanding of the text content based on natural language processing. Optionally, the server uses semantic analysis technology to identify content related to sentiment keywords, obtain implicit evaluations or opinions related to the sentiment keywords, automatically generate sentiment type results corresponding to the sentiment keywords based on the analysis, and map the sentiment type results to emojis in a cloud-based emoji library to obtain recommended emojis of the first type corresponding to the sentiment type results.

[0186] For example, text content is displayed in the text input area, such as "I went hiking with my best friend today, and the scenery on the mountain was beautiful." This text content does not contain sentiment keywords. The server performs a comprehensive understanding of the text content and extracts content related to sentiment keywords, such as "best friend" and "beautiful scenery," considering these as relevant. After determining the sentiment type as "happy," this is mapped to emojis in a cloud-based emoji library, and the mapped recommended emojis are returned to the input method program. For example, the mapped recommended emojis might be of the "[happy]" category.

[0187] It should be noted that this application primarily uses the example of selecting emotional keywords in text content as the factor for recommending emojis. In other embodiments, the server may also select suitable recommended emojis from cloud-based and local emoji libraries based on other factors, such as the tone of semantic signals or the semantics in the text content. This application does not limit this approach.

[0188] Add or remove the number of emojis in the candidate bar.

[0189] In some embodiments, at least one recommended emoji is displayed in a candidate bar, and the at least one recommended emoji includes a first emoji, which can be considered as a recommended emoji selected by the user in the candidate bar. Optionally, the user performs an add or delete operation on the selected first emoji to increase or decrease the number of the first emoji.

[0190] Figure 12 This is a flowchart of an emoji input method provided in an exemplary embodiment of this application. The method includes step 310.

[0191] Step 310: In response to the addition or deletion operation for the first emoji, increase or decrease the number of the first emoji in the candidate bar.

[0192] The first emoji is the recommended emoji selected by the user in the candidate list.

[0193] In some embodiments, in response to an operation to add or delete a first emoji, the number of candidate displays for the first emoji in the candidate bar is determined. In response to an operation to display the first emoji, the first emoji, representing the number of candidate displays in the candidate bar, is displayed in the text input area.

[0194] Optionally, the addition and deletion operations for the first emoji include adding and deleting the first emoji. The addition operation increases the number of first emojis in the candidate list. The deletion operation decreases the number of first emojis in the candidate list.

[0195] In some embodiments, in response to an add operation for a first emoji, the input method program increases the number of first emojis in the candidate bar; or, in response to a delete operation for a first emoji, the input method program decreases the number of first emojis in the candidate bar.

[0196] In some embodiments, in response to the number of first emojis decreasing to a preset number, the reduction of the number of first emojis in the candidate bar stops. The preset number is the minimum number of first emojis. For example, the preset number is 1; in response to the number of first emojis decreasing to 1, the display of first emojis in the candidate bar stops. By controlling the minimum number of first emojis, it is possible to prevent recommended first emojis from disappearing from the screen due to user error.

[0197] In this embodiment, the number of first emojis in the candidate bar is dynamically adjusted by adding or deleting them. This dynamic adjustment method can quickly adjust the number of first emojis, meet the user's personalized needs, and increase the interactivity between the user and the input method program.

[0198] In some embodiments, the addition or deletion operation for the first emoji includes at least one of the following: swiping, clicking, and long-pressing.

[0199] The different triggering mechanisms for add and delete operations are described below.

[0200] 1. For sliding operations

[0201] like Figure 13 As shown, step 310 above can be replaced by steps 311 and 312.

[0202] Step 311: In response to a swipe operation along a first direction for the first emoji, increase the number of the first emoji in the candidate bar;

[0203] Optionally, the swipe operation includes at least one of the following: swipe up, swipe down, swipe left, swipe right, and custom swipe operation. A custom swipe operation refers to a swipe operation without setting a swipe direction. For example, a custom swipe operation can be a swipe operation with a swipe trajectory of a triangle; or a custom swipe operation can be a swipe operation with a swipe trajectory of a circle; or a custom swipe operation can be a swipe operation with a swipe trajectory of an arc, etc.

[0204] In some embodiments, in response to a swipe operation along a first direction for a first emoji, the number of first emojis in the candidate bar is increased.

[0205] The first direction indicates the sliding direction for increasing the number of first emojis. For example, the first direction may be the sliding direction for a right swipe; or the sliding direction may be the sliding direction for a left swipe; or the sliding direction may be the sliding direction for a custom swipe. This application does not limit the specific direction of the first direction.

[0206] In some embodiments, in response to a sliding operation of a first emoji along a first direction satisfying a first sliding mode, the number of first emojis in the candidate bar is increased. The first sliding mode is used to determine whether a sliding operation of the first emoji along the first direction satisfies the condition for increasing the number.

[0207] Optionally, the first swipe method includes a limitation on the swipe length. The swipe length indicates the distance between the first touch point when the user begins swiping and the second touch point when the user ends swiping. For example, the swipe length is set to a first length threshold, which indicates the minimum threshold for the swipe length. That is, if the swipe length corresponding to a swipe operation of the first emoji along the first direction is greater than or equal to the first length threshold, the number of first emojis in the candidate bar is increased.

[0208] Optionally, the first swiping method includes limiting the swiping area. The swiping area is used to indicate the interactive area where the user performs the swiping operation. For example, the swiping area is set as a first swiping area, which indicates the maximum area of ​​the swiping area. In response to a swiping operation along a first direction within the first swiping area for a first emoji, the number of first emojis is increased in the candidate bar.

[0209] In some embodiments, the number of first emojis is increased in the candidate bar. Optionally, the rate at which the first emojis are increased is positively correlated with the swiping speed of the first emoji. For example, the faster the swiping speed of the first emoji along the first direction, the faster the first emoji is displayed in the candidate bar; the slower the swiping speed of the first emoji along the first direction, the slower the first emoji is displayed in the candidate bar. This application does not limit this.

[0210] Step 312: In response to a swipe operation along the second direction for the first emoji, reduce the number of the first emojis in the candidate bar.

[0211] The second direction is used to indicate the sliding direction that reduces the number of the first emoji.

[0212] Optionally, the second direction is a different sliding direction from the first direction. For example, the first direction is the sliding direction of a right swipe operation, and the second direction is the sliding direction of a left swipe operation; or the first direction is the sliding direction of a left swipe operation, and the second direction is the sliding direction of a right swipe operation; or the first direction is the sliding direction of an up swipe operation, and the second direction is the sliding direction of a down swipe operation; or the first direction is the sliding direction of a first custom sliding operation (such as a triangle sliding operation), and the second direction is the sliding direction of a second custom sliding operation (such as a square sliding operation). This application does not limit the specific direction of the second direction.

[0213] In some embodiments, in response to a swipe operation on a first emoji along a second direction satisfying a second swipe mode, the number of first emojis in the candidate bar is reduced. The second swipe mode is used to determine whether a swipe operation on a first emoji along a second direction satisfies the condition for reducing the number of emojis.

[0214] Optionally, the second swipe method includes a limitation on the swipe length. The swipe length indicates the distance between the third touch point when the user begins swiping and the fourth touch point when the user ends swiping. For example, the swipe length is set to a second length threshold, which indicates the maximum possible swipe length. That is, if the swipe length corresponding to a swipe operation of the first emoji along the second direction is less than or equal to the second length threshold, the number of first emojis in the candidate bar is reduced.

[0215] Optionally, the second swiping method includes limiting the swiping area. The swiping area is used to indicate the interactive area where the user performs a swiping operation. For example, the swiping area is set as a second swiping area, which is different from the first swiping area. In response to a swiping operation along a second direction for a first emoji within the second swiping area, the number of first emojis in the candidate bar is reduced.

[0216] In some embodiments, the number of first emojis in the candidate bar is reduced. Optionally, the rate of reduction of the first emojis is positively correlated with the swiping speed of the first emoji. For example, the faster the swiping speed of the first emoji along the second direction, the faster the first emoji is displayed in the candidate bar; the slower the swiping speed of the first emoji along the second direction, the slower the first emoji is displayed in the candidate bar. This application does not limit this.

[0217] For example, in conjunction with the reference Figure 14In the voice input interface 700, during the speech-to-text process, the input method program obtains the user's voice input information 701, for example, voice information 701 is "jintianjiandaol ehaopengyou", such as... Figure 14 As shown in Figure (a), the text content corresponding to the voice information 701 is displayed in the text input area 702, for example, the text content is displayed as "Today, I met a good friend". Optionally, at least one recommended emoji is displayed in the candidate bar 703, and the at least one recommended emoji includes a first emoji 703a. In response to a swipe operation along a first direction for the first emoji 703a, such as along the swipe direction of a right swipe operation, as shown in Figure (a), the text content corresponding to the voice information 701 is displayed in the text input area 702, for example, the text content is displayed as "Today, I met a good friend". Optionally, at least one recommended emoji is displayed in the candidate bar 703, including a first emoji 703a. Figure 14 As shown in Figure (b), the number of first emojis 703a in the candidate bar is increased, for example, from 1 to 3. In response to a continued swipe operation along a second direction for the first emoji 703a, for example, along the swipe direction of a left swipe operation, the number of first emojis 703a in the candidate bar is decreased, for example, from 3 to 2 (not shown in the figure).

[0218] In this embodiment, users can increase or decrease the number of emojis using an intuitive swipe gesture. This operation is simple, easy to understand, and quick to learn. The swipe operation can quickly respond to user needs, immediately reflecting the increase or decrease in the number of emojis in the candidate bar, thus improving the real-time performance of user operations.

[0219] 2. Regarding click operations

[0220] like Figure 15 As shown, step 310 above can be replaced by step 313.

[0221] Step 313: In response to a click on the first emoji, increase or decrease the number of the first emojis in the candidate bar based on the frequency of the click.

[0222] Click frequency refers to the frequency with which a user clicks on the first emoji, that is, the number of times a user clicks on the first emoji within a certain period of time.

[0223] Optionally, in response to the click operation targeting the first emoji not reaching a click threshold, the number of the first emoji in the candidate column is increased based on the click frequency of the click operation. In response to the click operation targeting the first emoji reaching a click count threshold, the number of the first emoji in the candidate column is decreased based on the click frequency of the click operation.

[0224] Optionally, the click threshold includes a click count threshold. The click count threshold indicates the maximum number of clicks allowed for the first emoji.

[0225] For example, the click threshold is 5 clicks. The correlation between click frequency and the increase in the number of first emojis is pre-set in the input method program. For instance, if the click frequency for the first emoji reaches 2 clicks within 1 second, the number of first emojis displayed in the candidate bar is multiplied by 2; if the click frequency reaches 3 clicks within 1 second, the number of first emojis displayed in the candidate bar is multiplied by 3; and if the click frequency reaches 5 clicks within 1 second, the number of first emojis displayed in the candidate bar is multiplied by 10. When the click frequency reaches 5, clicking the first emoji again reduces the number of first emojis in the candidate bar. For example, if the click frequency reaches 2 clicks within 1 second again, the number of first emojis displayed in the candidate bar is reduced by 2; if the click frequency reaches 3 clicks within 1 second again, the number of first emojis displayed in the candidate bar is reduced by 3; and if the click frequency reaches 5 clicks within 1 second again, the number of first emojis displayed in the candidate bar returns to 1.

[0226] For example, the click threshold includes a quantity threshold. The quantity threshold indicates the maximum number of clicks for the first emoji.

[0227] For example, the quantity threshold is 20, meaning the maximum number of the first emoji in the candidate bar can be increased to 20. The correlation between click frequency and the increase in the number of first emojis is pre-set in the input method program. For instance, if the click frequency for the first emoji reaches 2 times within 1 second, 3 times the number of first emojis displayed in the candidate bar is displayed; if the click frequency reaches 3 times within 1 second, 5 times the number of first emojis displayed is displayed; if the click frequency reaches 4 times within 1 second, 10 times the number of first emojis displayed is displayed; and if the click frequency reaches 5 times within 1 second, 20 times the number of first emojis displayed is displayed. Once the number of first emojis in the candidate bar reaches 20, clicking the first emoji again will reduce the number of first emojis displayed in the candidate bar.

[0228] The above are merely illustrative examples and are not intended to limit the scope of this application. In other embodiments, other settings may exist, such as increasing the number of first emojis in the candidate bar only based on the frequency of the click operation in response to a click operation on the first emoji.

[0229] In this embodiment, the user can increase or decrease the number of first emoticons by clicking. The input method program can dynamically adjust the number of emoticons based on the user's click behavior. This method can enhance the interactivity between the user and the input method program. By quickly increasing or decreasing the number of first emoticons, the time the user spends searching for and selecting emoticons in the candidate bar can be reduced, thereby improving input efficiency.

[0230] 3. For long press operations

[0231] like Figure 16 As shown, step 310 above can be replaced by step 314.

[0232] Step 314: In response to a long press on the first emoji, increase or decrease the number of the first emojis in the candidate bar based on the duration of the long press.

[0233] The long press duration is the time elapsed from when the user triggers the first emoji on the touchscreen until they leave the touchscreen.

[0234] Optionally, in response to a long press operation on the first emoji not reaching a long press threshold, the number of the first emoji in the candidate bar is increased based on the long press duration. In response to a long press operation on the first emoji reaching a long press threshold, the number of the first emoji in the candidate bar is decreased based on the long press duration.

[0235] Optionally, the long-press threshold includes a duration threshold. The duration threshold indicates the maximum long-press duration for the first emoji.

[0236] For example, with a duration threshold of 3 seconds, the correspondence between long press duration and the increase in the number of first emojis is pre-set in the input method program. For instance, a long press of 1 second for the first emoji displays 2 times the number of first emojis in the candidate bar; a long press of 2 seconds displays 3 times the number of first emojis; and a long press of 3 seconds displays 10 times the number of first emojis. When the long press duration reaches 3 seconds, a subsequent long press of the first emoji begins to decrease the number of first emojis in the candidate bar. For example, a subsequent long press of 1 second for the first emoji displays a decrease of 2 times; a subsequent long press of 2 seconds for the first emoji displays a decrease of 3 times; and a subsequent long press of 3 seconds for the first emoji restores the number of first emojis to 1.

[0237] Optionally, the long-press threshold includes a quantity threshold. The quantity threshold indicates the maximum quantity threshold for the first emoji.

[0238] For example, the quantity threshold is 30, meaning the maximum number of the first emoji in the candidate bar can be increased to 30. A pre-defined correspondence between long-press duration and the increase in the number of first emojis is set in the input method program. For instance, a long press of 1 second displays 3 times the number of first emojis in the candidate bar; a long press of 2 seconds displays 5 times; a long press of 3 seconds displays 10 times; a long press of 4 seconds displays 20 times; and a long press of 5 seconds displays 30 times. Once the number of first emojis in the candidate bar reaches 30, a subsequent long press of the first emoji will reduce its quantity in the candidate bar.

[0239] The above are merely illustrative examples and are not intended to limit the scope of this application. In other embodiments, other settings may exist, such as increasing the number of first emojis in the candidate bar only according to the duration of the long press operation in response to a long press operation on the first emoji.

[0240] In this embodiment, the user can increase or decrease the number of first emojis by long-pressing. The input method program can dynamically adjust the number of emojis based on the user's long-press behavior. This method can enhance the interactivity between the user and the input method program. By quickly increasing or decreasing the number of first emojis, the time the user spends searching for and selecting emojis in the candidate bar can be reduced, thereby improving input efficiency.

[0241] Insert recommended emojis at preset positions in the text content of the candidate column.

[0242] • In response to swiping actions on the text content in the candidate bar, insert recommended emojis at the end of the text content or after keywords.

[0243] In some embodiments, at least one recommended emoji is displayed in at least one first candidate in the candidate bar; and text content is displayed in a second candidate. The first candidate is a candidate in the candidate bar for displaying the recommended emoji. The second candidate is a candidate in the candidate bar for displaying the text content.

[0244] In some embodiments, recommended emojis and text content are displayed simultaneously in the candidate bar. Optionally, recommended emojis are inserted at the end of the text content or after keywords by sliding the text content displayed in the candidate bar. Optionally, after determining the number of recommended emojis to be added at the end of the text content or after keywords, the text content with the added recommended emojis is directly displayed in the text input area in response to the on-screen operation for that text content.

[0245] In some embodiments, in response to a swipe operation along a third direction on the text content in the candidate bar, the number of recommended emojis is increased at the end of the text content or after keywords. In response to a swipe operation along a fourth direction on the text content in the candidate bar, the number of recommended emojis is decreased at the end of the text content or after keywords.

[0246] The third direction indicates the sliding direction for increasing the number of recommended emojis inserted into the text content. The fourth direction indicates the sliding direction for decreasing the number of recommended emojis inserted into the text content.

[0247] Optionally, the third direction and the fourth direction are different sliding directions. For example, the third direction is the sliding direction of a right swipe operation, and the fourth direction is the sliding direction of a left swipe operation; or the third direction is the sliding direction of a left swipe operation, and the fourth direction is the sliding direction of a right swipe operation; or the third direction is the sliding direction of an upward swipe operation, and the fourth direction is the sliding direction of a downward swipe operation; or the third direction is the sliding direction of a downward swipe operation, and the fourth direction is the sliding direction of an upward swipe operation; or the third direction is the sliding direction of a first custom sliding operation (such as a triangle sliding operation), and the fourth direction is the sliding direction of a second custom sliding operation (such as a square sliding operation). This application does not limit the specific directions of the third and fourth directions.

[0248] For example, in conjunction with the reference Figure 17 In the voice input interface 800, during the speech-to-text process, the input method program obtains the user's voice input information 801, for example, voice information 801 is "jintianhaokaix inya", such as... Figure 17 As shown in Figure (a), a recommended emoji 803a is displayed in the first candidate option of the candidate bar 803, and text content 803b is displayed in the second candidate option. The text content 803b is the text content corresponding to the voice information 801, for example, the text content 803b is displayed as "I'm so happy today". In response to a swipe operation along a third direction for the text content 803b in the candidate bar 803, for example, the third direction is the swipe direction of a left swipe operation, a recommended emoji is added to the end of the text content 803b. Figure 17As shown in Figure (b), the text content 804 for adding recommended emojis is displayed. For example, if three recommended emojis are added 803a, the text content 804 for adding recommended emojis is displayed as "I'm so happy today [laughing][laughing][laughing]".

[0249] In this embodiment, when recommended emojis and text content are displayed simultaneously in the candidate bar, recommended emojis are inserted at the end of the text content or after keywords by sliding the text content in the candidate bar, reducing the steps for users to manually add emojis and improving input efficiency.

[0250] Furthermore, by swiping along the text content in different directions, the number of recommended emojis inserted into the text can be increased or decreased. Users can directly see the complete text content with recommended emojis, improving the user's preview experience; at the same time, users do not need to add emojis again in the text input area, reducing manual editing.

[0251] On-screen operation for the first emoji

[0252] In some embodiments, the text content corresponding to the voice information is automatically displayed in the text input area, and the recommended emoji corresponding to the text content is displayed in the candidate bar. Optionally, in response to a screen-up operation for a first emoji, the first emoji is screened onto the text input area. Optionally, the screen-up operation includes a drag operation.

[0253] Figure 18 This is a flowchart of an emoji input method provided in an exemplary embodiment of this application. Step 230 above can be replaced by step 231.

[0254] Step 231: In response to a drag operation on the first emoji from the first position to the second position, display the first emoji on the screen at the second position.

[0255] The first position is the position in the candidate bar, and the second position is the position in the text input area. Optionally, in response to dragging the first emoji from its position in the candidate bar to its position in the text input area, the first emoji is displayed on the screen in the text input area.

[0256] In some embodiments, the drag operation is a continuous, uninterrupted operation. Optionally, during the dragging of the first emoji, a second position is changed following the drag operation, and the dragging process indicates the process of dragging the first emoji from the first position. Optionally, when the drag operation ends, the first emoji is displayed on the screen at the second position. The end of the drag operation refers to the user releasing the touch operation on the first emoji to trigger the end of the drag operation.

[0257] In some embodiments, the first position is any position within the candidate bar area, and the second position is any position within the text input area. The text input area is an editable area for displaying text content. Optionally, the first emoticon can be dragged from the candidate bar to any position within the text content. For example, the first emoticon can be dragged from the candidate bar to the end of the text content; or, for example, the first emoticon can be dragged from the candidate bar to a position within the text content adjacent to a keyword. This application does not limit this.

[0258] In this embodiment, the first emoji can be dragged from the candidate bar to the text input area via a direct drag operation, allowing the user to precisely place the first emoji in the appropriate position as needed. The position of the first emoji in the text input area is updated in real time following the drag operation, providing the user with a smoother and more continuous visual experience.

[0259] In some embodiments, in response to a first emoji moving from a first position to a third position, the first emoji is displayed as a cursor. Optionally, in response to a drag operation on the cursor in a second position, the cursor position is changed following the drag operation, and the last position of the cursor in the second position is taken as the on-screen position of the first emoji. The second position is a position belonging to the text input area. The third position is the boundary position between the candidate bar and the text input area.

[0260] In some embodiments, the cursor refers to an interactive marker displayed in the text input area. The cursor indicates the current position of the user's drag operation in the text input area; it can also be understood as the cursor verbally instructing the user to insert a preset number of first emojis into the text input area. Optionally, the cursor is typically represented as a vertical blinking line or graphic.

[0261] In some embodiments, a first emoji is displayed in the candidate bar. The number of first emojis in the candidate bar is increased by sliding the first emojis along a first direction (e.g., sliding to the right), and the number of first emojis in the candidate bar is decreased by sliding the first emojis along a second direction (e.g., sliding to the left). Optionally, after determining the number of first emojis in the candidate bar (at least one), a preset number of first emojis are dragged along a preset direction (e.g., dragged upwards). During the dragging of the first emojis, in response to the first emojis reaching a preset position (e.g., the boundary position between the candidate bar and the text input area), the appearance of the preset number of first emojis changes, converting the preset number of first emojis into a cursor. Optionally, the cursor follows the user's dragging operation, and when the user stops dragging, the preset number of first emojis are inserted into the position of the cursor in the text input area.

[0262] For example, in conjunction with the reference Figure 19 In the voice input interface 900, during the speech-to-text process, the input method program obtains the user's voice input information 901, for example, voice information 901 is "jintianjiandaol ehaopengyou", such as Figure 19 As shown in Figure (a), the text content corresponding to the voice information 901 is displayed in the text input area 902, for example, the text content is displayed as "Today, I met a good friend," and three first emoticons are displayed in the candidate bar 903. Optionally, in response to an upward drag operation on the three first emoticons, during the drag process, in response to the three first emoticons reaching the boundary position of the candidate bar 903 and the text input area 902, such as... Figure 19 As shown in Figure (b), the three first emoticons are converted into cursor 904. Initially, cursor 904 may be located at the end of the text content. In response to a drag operation on cursor 904 within the text input area 902, the display position of cursor 904 changes; for example, cursor 904 may appear between "Today," and "Met a good friend." When the user stops dragging, as shown in Figure (b), the cursor position changes. Figure 19 As shown in Figure (c), three first emoticons are inserted at the position of cursor 904 in the text input area. At this time, the text content is displayed as "Today, [happy][happy][happy] met a good friend".

[0263] In this embodiment, the user drags a preset number of first emoticons along a preset direction. When the preset number of first emoticons reaches a preset position, the preset number of emoticons are converted into a cursor. The cursor can be dragged by the user and moves freely within the text input area according to the user's operation, thereby controlling the insertion position of the preset number of first emoticons in the text. The user does not need to adjust multiple first emoticons one by one; they only need to control the position of the cursor to control the position of multiple first emoticons. This interactive design allows the user to more flexibly control the insertion position of the first emoticons in the text input area, thereby improving the user's input experience.

[0264] In some embodiments, during the dragging of the first emoji, the displayed text input area changes from a first form to a second form. The display size of the second form is larger than that of the first form.

[0265] In some embodiments, the first form is the initial display form of the text input area when the first emoji is not dragged to it. The second form is the display form of the text input area after the first emoji has been dragged to it. Optionally, the display form of the text content displayed in the text input area also changes with the change of the text input area.

[0266] In some embodiments, before the drag operation on the first emoji begins, the text input area is at its normal size, and the font size of the text content within the text input area is displayed in the usual manner, without any enlargement or reduction. Once the first emoji is converted to a cursor, both the text input area and the displayed size of the text content will increase during the dragging process.

[0267] Optionally, the shape changes of the text input area include increasing the height and / or width of the text input area so that users can better drag and place a preset number of first emojis; the shape changes of the text content in the text input area include increasing the font size of the text content so that users can better perform drag operations in the text content.

[0268] For example, in conjunction with the reference Figure 20 In the voice input interface 1000, during the speech-to-text process, the input method program obtains the user's voice input information 1001. For example, if voice information 1001 is "jintianjiandaolehaopengyou", then... Figure 20 As shown in Figure (a), the text content corresponding to the voice information 1001 is displayed in the text input area 1002, for example, the text content is displayed as "Today, I met a good friend". At this time, the first emoticon is not dragged to the text input area 1002, and the text input area 1002 is displayed in the first form. Three first emoticons are displayed in the candidate bar 1003. In response to the upward drag operation of the three first emoticons, when the three first emoticons reach the boundary position of the candidate bar 1003 and the text input area 1002, the three first emoticons are converted into cursor 1004, and cursor 1004 may be located at the end of the text content. Optionally, when controlling the cursor 1004 to move in the text input area 1002, such as Figure 20 As shown in Figure (b), the text input area 1002 is displayed in the second form. When the three first emoticons are inserted into the last position of the cursor 1004, the second form corresponding to the text input area 1002 is larger than the first form.

[0269] In this embodiment of the application, during the dragging of the first emoji, the display size of the text input area increases. This provides the user with clearer visual feedback, indicating that the user is dragging, and the input box size is changed to accommodate more content.

[0270] The embodiments of this application can be applied to any scenario where an input method program is applicable. For example, it can be applied to the chat interface in an instant messaging program; or to the playback bullet screen interface of a video platform; or to the comment interface of an audio platform; or to the comment interface of a live streaming scenario; or to the comment interface of other social media platforms. This application does not limit the application scenario.

[0271] The following is a detailed description of the backend technical implementation of the emoji input method provided in this application.

[0272] Technical Implementation

[0273] 1. Recommended Emojis

[0274] In some embodiments, at least one recommended emoji is displayed in the candidate bar of the input method program. Optionally, the at least one recommended emoji displayed in the candidate bar of the input method program includes a first type of emoji and / or a second type of emoji. The first type of recommended emoji is obtained by the input method program based on a cloud-based emoji library, and the second type of recommended emoji is obtained by the input method program based on the user's local emoji library. The cloud-based emoji library is an emoji library stored in a cloud database. The local emoji library is an emoji library stored by the user on the terminal device. Optionally, the local emoji library includes a user dictionary and a list of frequently used emojis.

[0275] In some embodiments, at least one recommended emoji displayed in the candidate bar of the input method program is obtained by mapping from a cloud emoji library; or by searching a local emoji library; or by a combination of cloud emoji library mapping and local emoji library searching, which is not limited in this application.

[0276] In some embodiments, at least one recommended emoji includes a first type of recommended emoji mapped from a cloud-based emoji library. For the first type of recommended emoji, the input method program acquires voice information, converts the voice information into text content based on speech recognition, performs natural language processing on the text content to obtain a sentiment type result, and maps the sentiment type result to the cloud-based emoji library to obtain the first type of recommended emoji. Optionally, sentiment keywords are extracted from the text content, and these sentiment keywords are associated with the user's emotional state; sentiment detection is performed on the sentiment keywords to obtain the sentiment type result corresponding to the sentiment keyword; the sentiment type result is mapped to emojis in the cloud-based emoji library to obtain the first type of recommended emoji corresponding to the sentiment type result.

[0277] In some embodiments, at least one recommended emoji includes a second type of recommended emoji obtained from a local emoji library. For the second type of recommended emoji, the input method program retrieves the local emoji library (including a user dictionary and a list of frequently used emojis) and selects emojis from the local emoji library whose usage frequency reaches a preset threshold as the second type of recommended emoji.

[0278] Optionally, the input method program merges the first type of recommended emojis and the second type of recommended emojis to obtain at least one recommended emoji. That is, the input method program merges the recommended emojis obtained from the cloud emoji library and the local emoji library, and displays at least one recommended emoji in the candidate bar.

[0279] In some embodiments, at least one recommended emoji displayed in the candidate bar is related to at least one of the following factors: semantics of the voice information, tone of voice information, emotion of voice information, background noise of voice information, semantics of text content, and keywords in text content. That is, the input method program maps recommended emojis in a cloud-based emoji library and / or searches for recommended emojis in a local emoji library based on at least one of the aforementioned factors. Optionally, when at least one recommended emoji is related to at least two factors, the input method program maps recommended emojis in a cloud-based emoji library and / or searches for recommended emojis in a local emoji library based on the weighting of each of the at least two factors.

[0280] Reference Figure 21 This section introduces two types of emojis. For example... Figure 21 As shown, the system architecture 1200 mainly consists of two parts: the server side (also known as the cloud side) and the terminal device side (also known as the end side). The server side includes a speech recognition module 1201 and a post-processing module 1202.

[0281] On the server side, the speech recognition module 1201 converts the user's input speech information into text content based on speech recognition technology. The post-processing module 1202 performs natural language processing on the text content to obtain sentiment type results, and maps these results to the cloud-based emoji library 1203 to obtain recommended emojis of the first type. Specifically, sentiment keywords are directly extracted from the text content; these sentiment keywords are associated with the user's emotional state; sentiment detection is performed on the sentiment keywords to obtain the corresponding sentiment type results.

[0282] On the terminal device side, a local emoji library 1204 is included, which contains a user dictionary and a list of frequently used emojis. Optionally, emojis used in the local emoji library 1204 that reach a preset threshold frequency are selected as the second type of recommended emojis.

[0283] The first type of recommended emojis obtained from the cloud-based emoji library 1203 and the second type of recommended emojis obtained from the local emoji library 1204 are merged to obtain the user's personal emoji library. This merges and expands the user's personal emoji library, ensuring that the user receives relatively accurate emoji recommendations for each voice input. Optionally, the input method program displays recommended emojis in the candidate bar based on the user's voice information. The recommended emojis include the first type of emojis and / or the second type of emojis.

[0284] 2. Adding and deleting emojis

[0285] In this embodiment, users are allowed to quickly copy and insert emojis by dragging, achieving a smooth user experience through touch events.

[0286] In some embodiments, the input method program obtains a first touch position, which is used to characterize the user's initial touch position on the first emoji; obtains a second touch position, which is used to characterize the user's current touch position on the first emoji; optionally, based on the first touch position and the second touch position, calculates the sliding distance of the first emoji in the candidate bar.

[0287] In some embodiments, the input method program determines the sliding direction of the first emoji in the candidate bar based on the relative position of the first touch position and the second touch position, where the relative position is used to characterize the position of the second touch position relative to the first touch position. Optionally, when the relative position indicates a first direction, the number of first emojis in the candidate bar is increased; when the relative position indicates a second direction, the number of first emojis in the candidate bar is decreased.

[0288] For example, the first touch position is at position A on the candidate bar, and the second touch position is at position B on the candidate bar. Taking the first direction as the sliding direction of the right swipe operation as an example, when the relative positions of the first touch position and the second touch position indicate A→B, the number of first emojis is increased on the candidate bar.

[0289] In some embodiments, when the number of first emojis reaches a boundary position, the first emojis are controlled to wrap in the candidate bar, and the boundary position is used to indicate the boundary position of the candidate bar.

[0290] For specific implementation details, please refer to the following steps:

[0291] First, define a clickable emoji container (the emoji container is used to represent the candidate bar and responds to the user's touch events), and add three touch event listeners to the emoji container: touchstart, touchmove, and touchend.

[0292] Optionally, the touchstart event is listened to in real time on the client of the terminal device to obtain the starting touch position, the distance the first emoji moves in the candidate bar is calculated in the touchmove event, and the first emoji is dragged to the right or left based on the relationship between the current touch position and the starting position.

[0293] Specifically, dragging to the right increases the number of the first emoji, and adding `white-space:pre-wrap` allows for automatic line wrapping when the number of the first emoji reaches the container boundary (the right side of the input method panel, i.e., the right side of the candidate bar). Dragging to the left decreases the number of the first emoji, but not less than 1; that is, when the number of the first emoji decreases to 1, the decrease in the number of the first emoji in the candidate bar stops. The sensitivity of the first emoji can be changed by adjusting the divisor in the event.

[0294] Optionally, the touchend event is triggered when the user's finger leaves the screen, and the final number of emojis is determined based on the touch direction and distance. When the user releases their finger, the display area of ​​the first emoji in the candidate bar is cleared, and the final number of the first emojis is displayed in the text input area (text box). After displaying the first emoji in the text input area to remove the used emojis from the candidate bar, the layout of other recommended emojis in the candidate bar is dynamically updated, causing subsequent recommended emojis to move forward to fill the gaps, and adjusting the positions of other recommended emojis in the candidate bar to maintain a compact layout.

[0295] 3. Trigger the cursor

[0296] In some embodiments, during the dragging of the first emoji, the real-time drag position of the first emoji is obtained; if the real-time drag position coincides with a third position, the first emoji is controlled to become the cursor, where the third position is the boundary position between the candidate bar and the text input area. That is, the cursor is triggered when the real-time drag position coincides with the third position.

[0297] In this embodiment, the cursor in the input method program is a user interface element that allows the user to move the cursor in the text input area to select the insertion point or highlight text content within a box. The implementation of triggering the cursor with the first emoji in this embodiment mainly involves the following key aspects:

[0298] 1) The first emoji triggers the cursor.

[0299] First, add a touch event listener to the button corresponding to each recommended emoji in the candidate bar to listen for touch events in real time. When an upward touch action is detected on a recommended emoji, and the vertical drag distance of the button corresponding to the recommended emoji exceeds 20 pixels, return true to activate the cursor. Then, add a gradient color bar to the top of the input method panel to indicate whether the user has entered the control mode corresponding to the cursor.

[0300] 2) Cursor functionality implementation

[0301] After entering cursor control mode, it continues to listen for the user's dragging actions in the text input area and calculates the corresponding drag distance. Optionally, the drag threshold can be limited to a range of -50 to 50 pixels, calculated in real time based on the horizontal drag distance. If the movement is within this drag threshold, the cursor position in the text input area is updated at a constant speed; if the movement exceeds this drag threshold, the cursor is moved faster based on the drag direction and speed.

[0302] Boundary handling and checking: The current text input area value and cursor position are obtained, and a boundary check is performed to ensure the cursor does not exceed the drag threshold. When the user's swipe distance reaches the boundary condition, the cursor's movement step size is increased (initially set to move 1 character at a time) to simulate acceleration. The cursor continues to move in that direction and gradually accelerates until it reaches the boundary of the text input area or the user detects that the finger has stopped dragging (drag operation ends). If no drag operation end or direction change event is detected when the cursor reaches the boundary of the text input area, the cursor position remains at the boundary of the text input area.

[0303] 3) Emoji input and candidate bar update

[0304] Upon entering cursor mode, the system simultaneously listens for user release events targeting the first emoji. If the release occurs in cursor mode, an insertion operation is performed, meaning a preset number of the first emojis are inserted at the last cursor position. The insertion logic is as follows: obtain the current cursor position, update the value of the text input area, split the text content in the text input area at the cursor position, insert the selected first emoji between the two split parts to update the text content in the text input area, and finally update the cursor position so that it is positioned after the inserted first emoji, ensuring that the text input area has focus.

[0305] Candidate bar update: After inserting a preset number of first emojis in the text input area, remove the buttons corresponding to the used first emojis from the candidate bar, dynamically update the layout of the buttons corresponding to other recommended emojis in the candidate bar, so that the buttons corresponding to subsequent recommended emojis move forward to fill the gaps, and adjust the position of other emojis in the candidate bar to keep the candidate bar compact.

[0306] Beneficial effects of this plan

[0307] In conventional voice input methods, the speech-to-text recognition results (i.e., the text content corresponding to the audio signal) often fail to accurately identify and convert the user's emotional expression intentions. Therefore, users have a need to add emoticons to the converted text content to assist in expressing emotions in the text.

[0308] Current voice input methods use a scattered approach to emoji recommendations, emoji quantity adjustments, and emoji position adjustments. This results in a confusing user workflow, a long operation process, and poor convenience, and also fails to meet users' needs for adding different emojis.

[0309] In this embodiment, various interactive operations are defined to chain together a series of emoji operations, allowing users to add, increase or decrease the number of emojis, and adjust their positions using only gestures. This satisfies the preferences of some users for different numbers of emojis when on screen, as well as the scenario where some users add emojis in a single operation after voice recognition. While improving the convenience of emotion processing in voice recognition results, this also meets the different preferences of users for adding emojis and increases the fun and playability of the voice input method.

[0310] The reason why voice input is favored by users is largely due to its convenience and the continuous streaming experience it brings. In this embodiment, by optimizing one of the main operation paths of the voice input process—the addition of emoticons—the fluency of the voice function is improved, which is also conducive to shaping the continuity and integrity of the user's voice mindset.

[0311] In addition, the embodiments of this application can be easily reused in other input scenarios, thereby improving the efficiency of input operations.

[0312] Figure 22 This diagram illustrates a structural block diagram of an emoji input device according to an embodiment of this application. The device has the functionality to implement the aforementioned emoji input method example; this functionality can be implemented in hardware or by hardware executing corresponding software. The device can be the first client described above, or it can be integrated within the first client. Figure 22 As shown, the device may include: an acquisition module 2710, a display module 2720, and a screen display module 2730.

[0313] The acquisition module 2710 is used to acquire voice information during the speech-to-text input process;

[0314] Display module 2720 is used to display at least one recommended emoji on the candidate bar based on the voice information, wherein the at least one recommended emoji includes a first emoji;

[0315] The screen display module 2730 is used to display the first emoticon onto the text input area in response to the screen display operation for the first emoticon.

[0316] In some embodiments, the display module 2720 further includes a display submodule.

[0317] In an optional example, a display submodule is provided for displaying text content in the text input area, the text content being obtained after performing the speech-to-text conversion on the speech information;

[0318] The display submodule is used to display the at least one recommended emoji in at least one first candidate in the candidate bar;

[0319] The at least one recommended emoji is related to at least one of the following factors: the semantics of the voice information, the tone of the voice information, the emotion of the voice information, the background sound of the voice information, the semantics of the text content, and the keywords of the text content.

[0320] In an optional example, a submodule is displayed for displaying the at least one recommended emoji in at least one first candidate in the candidate bar; and displaying text content in a second candidate.

[0321] The text content is obtained by performing speech-to-text conversion on the voice information, and the at least one recommended emoji is related to at least one of the following factors: the semantics of the voice information, the tone of the voice information, the emotion of the voice information, the background sound of the voice information, the semantics of the text content, and the keywords in the text content.

[0322] In an optional example, a submodule is displayed for showing a combination of the text content and recommended emojis in the third candidate option of the candidate bar.

[0323] In some embodiments, the apparatus further includes an add / delete module.

[0324] In an optional example, an add / delete module is used to increase or decrease the number of the first emojis in the candidate bar in response to an add / delete operation for the first emoji.

[0325] In some embodiments, the add / delete module further includes adding sub-modules and removing sub-modules.

[0326] In an optional example, a submodule is added to increase the number of the first emojis in the candidate bar in response to a swipe operation along a first direction for the first emoji;

[0327] A reduction submodule is configured to reduce the number of the first emojis in the candidate bar in response to a swipe operation along a second direction for the first emoji.

[0328] In an optional example, a submodule is added to increase the number of the first emojis in the candidate bar in response to a click operation on the first emoji, based on the frequency of the click operation.

[0329] A reduction submodule is configured to, in response to a click operation on the first emoji, reduce the number of the first emojis in the candidate bar based on the click frequency of the click operation.

[0330] In an optional example, a submodule is added for increasing the number of the first emojis in the candidate bar in response to a long press operation on the first emoji, based on the duration of the long press operation.

[0331] A reduction submodule is configured to, in response to a long press operation on the first emoji, reduce the number of the first emojis in the candidate bar based on the duration of the long press operation.

[0332] In an optional example, a reduction submodule is configured to stop reducing the number of the first emoji in the candidate bar in response to the number of the first emoji decreasing to a preset number.

[0333] In some embodiments, the apparatus further includes an insertion module.

[0334] In an optional example, an insertion module is used to insert the recommended emoji at the end of the text content or after a keyword in response to a swipe operation on the text content in the candidate bar.

[0335] In some embodiments, the insertion module further includes an addition submodule and a reduction submodule.

[0336] In an optional example, a submodule is added to increase the number of recommended emojis at the end of the text content or after keywords in response to a swipe operation along a third direction for the text content in the candidate bar.

[0337] A reduction submodule is used to reduce the number of recommended emojis at the end of the text content or after keywords in response to a swipe operation along a fourth direction on the text content in the candidate bar.

[0338] In some embodiments, the screen module 2730 further includes a screen sub-module.

[0339] In one optional example, the on-screen operation includes a drag operation;

[0340] The on-screen submodule is used to respond to a drag operation on the first emoji from a first position to a second position, and to display the first emoji on the screen at the second position;

[0341] The first position is a position within the candidate column, and the second position is a position within the text input area.

[0342] In some embodiments, the upper screen submodule further includes a follow unit and a display unit.

[0343] In an optional example, a follow unit is configured to follow the drag operation to change the second position during the dragging of the first emoji, the dragging process being used to indicate the process of dragging the first emoji from the first position;

[0344] The display unit is used to display the first emoji being moved to the second position when the dragging operation ends.

[0345] In some embodiments, the apparatus further includes a conversion module.

[0346] In an optional example, the conversion module is configured to display the first emoji as a cursor in response to the first emoji moving from a first position to a third position;

[0347] A following unit is configured to respond to a dragging operation of the cursor in the second position and follow the dragging operation to change the position of the cursor;

[0348] The third position is the boundary between the candidate bar and the text input area.

[0349] In an optional example, a submodule is displayed to show the text input area changing from a first form to a second form during the dragging of the first emoji;

[0350] The display size of the second form is larger than that of the first form.

[0351] In some embodiments, the apparatus further includes a mapping module.

[0352] In an optional example, the at least one recommended emoji includes a first type of recommended emoji obtained from a cloud-based emoji library;

[0353] The acquisition module 2710 is used to acquire input voice information and convert the voice information into text content based on speech recognition;

[0354] The mapping module is used to perform natural language processing on the text content to obtain the sentiment type result, and then map the sentiment type result to the cloud emoji library to obtain the recommended emoji of the first type.

[0355] In some embodiments, the mapping module further includes an extraction submodule, a detection submodule, and a mapping submodule.

[0356] In an optional example, an extraction submodule is used to extract sentiment keywords from the text content, the sentiment keywords being associated with the user's emotional state;

[0357] The detection submodule is used to perform sentiment detection on the sentiment keywords and obtain the sentiment type result corresponding to the sentiment keywords;

[0358] The mapping submodule is used to map the emotion type result to the emojis in the cloud emoji library to obtain the recommended emojis of the first type corresponding to the emotion type result.

[0359] In some embodiments, the device further includes a search module.

[0360] In an optional example, the at least one recommended emoji includes a second type of recommended emoji based on a local emoji library;

[0361] The search module is used to search the local emoji library and select emojis whose usage frequency in the local emoji library reaches a preset threshold as the second type of recommended emojis.

[0362] In some embodiments, the apparatus further includes a computing module.

[0363] In an optional example, the acquisition module 2710 is used to acquire a first touch position, the first touch position being used to characterize the user's initial touch position on the first emoji;

[0364] The acquisition module 2710 is used to acquire a second touch position, which is used to characterize the current touch position of the user on the first emoji;

[0365] The calculation module is used to calculate the sliding distance of the first emoji in the candidate bar based on the first touch position and the second touch position.

[0366] In some embodiments, the apparatus further includes a determination module.

[0367] In an optional example, the judgment module is used to determine the sliding direction of the first emoji in the candidate bar based on the relative positions of the first touch position and the second touch position, wherein the relative position is used to characterize the position of the second touch position relative to the first touch position.

[0368] In some embodiments, the device further includes a control module.

[0369] In an optional example, the control module is configured to control the increase of the number of the first emojis in the candidate bar when the relative position indicates the first direction;

[0370] A control module is configured to control the reduction of the number of the first emojis in the candidate bar when the relative position indicates the second direction.

[0371] In an optional example, the control module is configured to control the first emoji to wrap in the candidate bar when the number of the first emojis reaches a boundary position, the boundary position being used to indicate the boundary position of the candidate bar.

[0372] In an optional example, the acquisition module 2710 is used to acquire the real-time drag position of the first emoji during the dragging process of the first emoji;

[0373] The control module is used to control the first emoji to become a cursor when the real-time drag position coincides with the third position, wherein the third position is the boundary position between the candidate bar and the text input area.

[0374] It should be noted that the specific limitations of the embodiments of the one or more emoji input devices provided above can be found in the limitations of the emoji input methods above, and will not be repeated here. Each module of the above device can be implemented entirely or partially by software, hardware, or a combination thereof. Each module can be embedded in the processor of the computer device in hardware form or independent of the processor, or it can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0375] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0376] This application also provides a computer device, which includes a processor and a memory. The memory stores at least one computer program, which is loaded and executed by the processor to implement the emoji input method provided in the above-described method embodiments.

[0377] Figure 23 This illustration shows a structural block diagram of a computer device 2800 provided in an exemplary embodiment of this application. The computer device 2800 may be a portable mobile terminal, such as a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), or MP4 player (Moving Picture Experts Group Audio Layer IV). The computer device 2800 may also be referred to as a user device, portable terminal, or other names. Typically, the computer device 2800 includes a processor 2801 and a memory 2802.

[0378] Processor 2801 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 2801 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). Processor 2801 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 2801 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 2801 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0379] The memory 2802 may include one or more computer-readable storage media, which may be tangible and non-transitory. The memory 2802 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 2802 is used to store at least one instruction, which is executed by the processor 2801 to implement the emoji input method provided in the embodiments of this application.

[0380] In some embodiments, the computer device 2800 may optionally include a peripheral device interface 2803 and at least one peripheral device. The processor 2801, memory 2802, and peripheral device interface 2803 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 2803 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 2804, a touch display screen 2805, a camera assembly 2806, an audio circuit 2807, and a power supply 2808.

[0381] Peripheral device interface 2803 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 2801 and memory 2802. In some embodiments, processor 2801, memory 2802 and peripheral device interface 2803 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 2801, memory 2802 and peripheral device interface 2803 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0382] The radio frequency (RF) circuit 2804 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 2804 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 2804 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 2804 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 2804 can communicate with other terminals through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or Wi-Fi (Wireless Fidelity) networks. In some embodiments, the RF circuit 2804 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0383] The touch display screen 2805 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. The touch display screen 2805 also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to the processor 2801 for processing. The touch display screen 2805 is used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one touch display screen 2805, located on the front panel of the computer device 2800; in other embodiments, there may be at least two touch display screens, respectively located on different surfaces of the computer device 2800 or in a folded design; in some embodiments, the touch display screen 2805 may be a flexible display screen, located on a curved or folded surface of the computer device 2800. Furthermore, the touch display screen 2805 may be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. The touch display screen 2805 may be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).

[0384] The camera assembly 2806 is used to acquire images or videos. Optionally, the camera assembly 2806 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is used for video calls or selfies, and the rear-facing camera is used for taking photos or videos. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, and a wide-angle camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, and panoramic shooting and VR shooting by fusion of the main camera and the wide-angle camera. In some embodiments, the camera assembly 2806 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash is a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0385] Audio circuitry 2807 provides an audio interface between the user and computer device 2800. Audio circuitry 2807 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to processor 2801 for processing, or input to radio frequency circuitry 2804 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each located at a different location on the computer device 2800. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from processor 2801 or radio frequency circuitry 2804 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, audio circuitry 2807 may also include a headphone jack.

[0386] Power supply 2808 is used to supply power to the various components in computer device 2800. Power supply 2808 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 2808 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, and a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0387] In some embodiments, the computer device 2800 further includes one or more sensors 2809. The one or more sensors 2809 include, but are not limited to, an accelerometer 2810, a gyroscope 2811, a pressure sensor 2812, an optical sensor 2813, and a proximity sensor 2814.

[0388] Accelerometer 2810 can detect the magnitude of acceleration on the three coordinate axes of a coordinate system established by computer device 2800. For example, accelerometer 2810 can be used to detect the components of gravitational acceleration on the three coordinate axes. Processor 2801 can control touch screen 2805 to display the user interface in landscape or portrait view based on the gravitational acceleration signal collected by accelerometer 2810. Accelerometer 2810 can also be used for games or to collect user motion data. Gyroscope 2811 can detect the orientation and rotation angle of computer device 2800. Gyroscope 2811 can work in conjunction with accelerometer 2810 to collect 3D movements of the user on computer device 2800. Based on the data collected by gyroscope 2811, processor 2801 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0389] The pressure sensor 2812 can be disposed on the side bezel of the computer device 2800 and / or on the lower layer of the touch display screen 2805. When the pressure sensor 2812 is disposed on the side bezel of the computer device 2800, it can detect the user's grip signal on the computer device 2800 and perform left / right hand recognition or quick operation based on the grip signal. When the pressure sensor 2812 is disposed on the lower layer of the touch display screen 2805, it can control operable controls on the UI interface based on the user's pressure operation on the touch display screen 2805. Operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0390] Optical sensor 2813 is used to collect ambient light intensity. In one embodiment, processor 2801 can control the display brightness of touch display screen 2805 based on the ambient light intensity collected by optical sensor 2813. Specifically, when the ambient light intensity is high, the display brightness of touch display screen 2805 is increased; when the ambient light intensity is low, the display brightness of touch display screen 2805 is decreased. In another embodiment, processor 2801 can also dynamically adjust the shooting parameters of camera assembly 2806 based on the ambient light intensity collected by optical sensor 2813.

[0391] The proximity sensor 2814, also known as a distance sensor, is typically located on the front of the computer device 2800. The proximity sensor 2814 is used to detect the distance between the user and the front of the computer device 2800. In one embodiment, when the proximity sensor 2814 detects that the distance between the user and the front of the computer device 2800 is gradually decreasing, the processor 2801 controls the touch display screen 2805 to switch from a screen-on state to a screen-off state; when the proximity sensor 2814 detects that the distance between the user and the front of the computer device 2800 is gradually increasing, the processor 2801 controls the touch display screen 2805 to switch from a screen-off state to a screen-on state.

[0392] Those skilled in the art will understand that the above structure does not constitute a limitation on the computer device 2800, and may include more or fewer components than shown, or combine certain components, or employ different component arrangements.

[0393] This application also provides a computer-readable storage medium storing at least one computer program, which is loaded and executed by a processor to implement the emoji input method provided in the above-described method embodiments.

[0394] This application also provides a computer program product, which includes at least one computer program stored in a computer-readable storage medium; the at least one computer program is read from and executed by a processor of a computer device from the computer-readable storage medium, causing the computer device to perform the emoji input method provided in the above-described method embodiments.

[0395] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0396] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0397] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent switching, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for inputting emojis, characterized in that, The method includes: During the speech-to-text input process, voice information is acquired; Based on the voice information, at least one recommended emoji is displayed in the candidate bar, and the at least one recommended emoji includes a first emoji; In response to the screen-up operation of the first emoji, the first emoji is displayed on the text input area.

2. The method according to claim 1, characterized in that, The step of displaying at least one recommended emoji on the candidate bar based on the voice information includes: The text content is displayed in the text input area. The text content is obtained by performing the speech-to-text conversion on the voice information. The at least one recommended emoji is displayed in at least one first candidate in the candidate bar; The at least one recommended emoji is related to at least one of the following factors: the semantics of the voice information, the tone of the voice information, the emotion of the voice information, the background sound of the voice information, the semantics of the text content, and the keywords of the text content.

3. The method according to claim 1, characterized in that, The step of displaying at least one recommended emoji on the candidate bar based on the voice information includes: The at least one recommended emoji is displayed in at least one first candidate in the candidate bar; and text content is displayed in the second candidate. The text content is obtained by performing speech-to-text conversion on the voice information, and the at least one recommended emoji is related to at least one of the following factors: the semantics of the voice information, the tone of the voice information, the emotion of the voice information, the background sound of the voice information, the semantics of the text content, and the keywords in the text content.

4. The method according to claim 2 or 3, characterized in that, The method further includes: The third candidate in the candidate bar displays a combination of the text content and recommended emojis.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: In response to an operation that adds or removes the first emoji, the number of the first emoji is increased or decreased in the candidate bar.

6. The method according to claim 5, characterized in that, The step of increasing or decreasing the number of the first emojis in the candidate bar in response to an operation to add or delete the first emoji includes: In response to a swipe operation along a first direction for the first emoji, the number of the first emojis is increased in the candidate bar; In response to a swipe operation along a second direction for the first emoji, the number of the first emojis in the candidate bar is reduced.

7. The method according to claim 5, characterized in that, The step of increasing or decreasing the number of the first emojis in the candidate bar in response to an operation to add or delete the first emoji includes: In response to a click operation on the first emoji, the number of the first emojis in the candidate bar is increased or decreased according to the frequency of the click operation.

8. The method according to claim 5, characterized in that, The step of increasing or decreasing the number of the first emojis in the candidate bar in response to an operation to add or delete the first emoji includes: In response to a long press operation on the first emoji, the number of the first emojis in the candidate bar is increased or decreased according to the duration of the long press operation.

9. The method according to any one of claims 5 to 8, characterized in that, The method further includes: In response to the number of the first emojis decreasing to a preset number, the reduction of the number of the first emojis in the candidate bar is stopped.

10. The method according to claim 3, characterized in that, The method further includes: In response to a swipe operation on the text content in the candidate bar, the recommended emoji is inserted at the end of the text content or after the keyword.

11. The method according to claim 10, characterized in that, The step of inserting the recommended emoji at the end of the text content or after keywords in response to a swipe operation on the text content in the candidate bar includes: In response to a swipe operation along a third direction on the text content in the candidate bar, the number of recommended emojis is increased at the end of the text content or after the keywords; In response to a swipe operation along a fourth direction on the text content in the candidate bar, the number of recommended emojis is reduced at the end of the text content or after keywords.

12. The method according to any one of claims 1 to 11, characterized in that, The screen operation includes dragging operations; The step of responding to the screen operation of the first emoji and screening the first emoji onto the text input area includes: In response to a drag operation on the first emoji from a first position to a second position, the first emoji is displayed on the screen at the second position; The first position is a position within the candidate column, and the second position is a position within the text input area.

13. The method according to claim 12, characterized in that, The step of displaying the first emoji on the screen at the second position in response to a drag operation on the first emoji from a first position to a second position includes: During the dragging of the first emoji, the second position is changed following the dragging operation, and the dragging process is used to indicate the process of dragging the first emoji from the first position; When the dragging operation ends, the first emoji is displayed on the screen and moved to the second position.

14. The method according to claim 13, characterized in that, The step of changing the second position following the dragging operation during the dragging of the first emoji includes: In response to the first emoji moving from a first position to a third position, the first emoji is displayed as a cursor; In response to a drag operation on the cursor at the second position, the position of the cursor is changed following the drag operation; The third position is the boundary between the candidate bar and the text input area.

15. The method according to any one of claims 12 to 14, characterized in that, The method further includes: During the dragging of the first emoji, the text input area is displayed to change from a first form to a second form; The display size of the second form is larger than that of the first form.

16. An emoji input device, characterized in that, The device includes: The acquisition module is used to acquire voice information during the speech-to-text input process; The display module is used to display at least one recommended emoji on the candidate bar based on the voice information, wherein the at least one recommended emoji includes a first emoji; The screen display module is used to respond to the screen display operation for the first emoji and display the first emoji on the text input area.

17. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to implement the emoji input method as described in any one of claims 1 to 15.

18. A computer storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to implement the emoji input method as described in any one of claims 1 to 15.

19. A computer program product, characterized in that, The computer program product includes a computer program stored in a computer-readable storage medium; the computer program is read from and executed by a processor of a computer device, causing the computer device to perform an input method for emojis as described in any one of claims 1 to 15.

Images