Methods to assist in text input

Abstract

The present disclosure relates to a method for assisting text input, which includes: obtaining the currently running application and the current input scenario corresponding to the current interactive interface displayed on the screen as first information; obtaining the content presented in the input area of ​​the current interactive page as second information; presenting multiple first text creation themes on the auxiliary interactive interface on the screen, wherein the first text creation themes are generated based on at least the first information and the second information using a machine learning model; and in response to selecting a first text creation theme, presenting a first text on the auxiliary interactive interface, wherein the first text is generated around the selected first text creation theme using the machine learning model with at least the second information as auxiliary information. Utilizing the method of the present disclosure, users can be assisted to complete text input in different application scenarios more efficiently, thereby improving user experience.

Description

Technical Field

[0001] The present disclosure relates to a method, a computing device, a storage medium, a program product, etc. for assisting text input. Background Art

[0002] With the popularization of mobile terminal devices and the development of mobile terminal technology, users often need to create texts in various applications on mobile terminals in various aspects of work and life. However, the current input method of mobile terminals cannot well assist users in inputting texts. Summary of the invention

[0003] Embodiments of the present disclosure provide methods for assisting text input and corresponding computing devices, non-transitory machine-readable storage media, and computer program products for executing these methods.

[0004] According to the first aspect of an embodiment of the present disclosure, a method for assisting text input is provided, including: obtaining a currently running application and a current input scenario corresponding to a current interactive interface displayed on a screen as first information; obtaining content presented in an input area in the current interactive page as second information; presenting a plurality of first text creation themes on the auxiliary interactive interface on the screen, wherein the first text creation themes are generated using a machine learning model based at least on the first information and the second information; and in response to selecting a first text creation theme, presenting a first text on the auxiliary interactive interface, wherein the first text is generated using the machine learning model around the selected first text creation theme with at least the second information as auxiliary information.

[0005] Optionally, the content includes text content and / or picture content, wherein the text content and / or picture content are obtained by identifying the input area; when the current input scene is an article creation scene, the input area includes an input picture insertion area, a title input area and / or a text input area; or when the current input scene is a chat scene, the input area only includes a chat record display area.

[0006] Optionally, the current interactive interface is provided by a first window that currently has window focus, and obtaining a currently running application and a current input scene corresponding to the current interactive interface as the first information includes: obtaining information of the first window, wherein the information of the first window includes an application package name and an activity class name corresponding to the first window; and determining the currently running application according to the application package name, and determining the current input scene according to the activity class name.

[0007] Optionally, the auxiliary interaction interface overlays part of the current interaction interface, and the method further includes: while presenting the multiple first text creation themes on the auxiliary interaction interface, at least a first control for setting the length and / or style of the text to be generated is also presented, and the first text is generated using the machine learning model around a selected first text creation theme, and with at least the second information and the length and / or style set by the first control as auxiliary information.

[0008] Optionally, the method also includes: presenting a second control on the auxiliary interaction interface while presenting the first text, the second control being used to return to the auxiliary interaction interface presenting the multiple first text creation themes; in response to triggering the second control, returning to the auxiliary interaction interface presenting the multiple first text creation themes; in response to changing the length and / or style set by the first control, presenting a second text on the auxiliary interaction interface instead of the first text, the second text being generated using the machine learning model around the selected first text creation theme, and with at least the second information and the changed length and / or style as auxiliary information.

[0009] Optionally, the method also includes: presenting a first update button for updating the text creation subject while presenting the multiple first text creation subjects on the auxiliary interaction interface; and in response to triggering the first update button, presenting multiple second text creation subjects on the auxiliary interaction interface to replace the multiple first text creation subjects; wherein the second text creation subject is regenerated using the machine learning model based on at least the first information and the second information.

[0010] Optionally, the method also includes: presenting a third control for communicating with the machine learning model while presenting the multiple first text creation subjects on the auxiliary interaction interface; and in response to inputting a first dialogue via the third control, presenting multiple third text creation subjects on the auxiliary interaction interface instead of the multiple first text creation subjects; wherein the third text creation subjects are generated using the machine learning model based on at least the first information, the second information and the first dialogue.

[0011] Optionally, the method also includes: in response to inputting a second conversation via the third control, presenting a plurality of fourth text creation subjects on the auxiliary interaction interface instead of the plurality of third text creation subjects; wherein the fourth text creation subjects are generated using the machine learning model based at least on the first information, the second information, the first conversation and the second conversation.

[0012] Optionally, the method also includes: presenting a fourth control for communicating with the machine learning model while presenting the first text on the auxiliary interaction interface; and in response to inputting a third dialogue via the fourth control, presenting a third text on the auxiliary interaction interface instead of the first text; wherein the third text is created around a selected first text using the machine learning model, and is generated with at least the second information and the third dialogue as auxiliary information.

[0013] Optionally, the method also includes: in response to inputting a fourth dialogue via the fourth control, presenting a fourth text in the auxiliary interaction interface instead of the third text; wherein the fourth text is created around a selected first text using the machine learning model, and is generated with at least the second information, the third dialogue and the fourth dialogue as auxiliary information.

[0014] Optionally, the method also includes: presenting a second update button for updating the text while presenting the first text on the auxiliary interaction interface; and in response to triggering the second update button, presenting a fifth text on the auxiliary interaction interface to replace the first text; wherein the fifth text is regenerated using the machine learning model around the creative theme of a selected first text and with at least the second information as auxiliary information.

[0015] Optionally, the method further includes: presenting a fifth control for inserting text while presenting the first text on the auxiliary interaction interface; and inserting the first text into an inputtable area of ​​the current interaction interface in response to triggering the fifth control.

[0016] Optionally, the first text creation theme is generated using a machine learning model based at least on the first information and the second information, including: substituting the first information and the second information into a preset first prompt template to obtain a first prompt; and inputting the first prompt into the machine learning model to generate the first text creation theme.

[0017] Optionally, the first text is generated using the machine learning model around a selected first text creation theme with at least the second information as auxiliary information, including: substituting the selected first text creation theme and the second information into a preset second prompt template to obtain a second prompt; and inputting the second prompt into the machine learning model to generate the first text.

[0018] According to a second aspect of an embodiment of the present disclosure, a computing device is provided, including: a processor; and a memory on which executable code is stored, and when the executable code is executed by the processor, the processor executes the method described in any scheme of the first aspect above.

[0019] According to a third aspect of an embodiment of the present disclosure, a non-temporary machine-readable storage medium is provided, on which an executable code is stored. When the executable code is executed by a processor of an electronic device, the processor executes a method as described in any of the schemes in the first aspect above.

[0020] According to a fourth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising an executable code. When the executable code is executed by a processor of an electronic device, the processor is caused to execute a method as described in any one of the schemes in the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components in the exemplary embodiments of the present disclosure.

[0022] Figure 1 The structure diagram of a mobile phone according to at least one embodiment of the present disclosure is exemplarily shown.

[0023] Figure 2 A schematic flow chart of a method for assisting text input according to at least one embodiment of the present disclosure is exemplarily shown.

[0024] Figure 3A and Figure 3B The display interfaces of a mobile phone screen using the method according to at least one embodiment of the present disclosure are exemplarily shown respectively.

[0025] Figure 4 The block diagram of a computing system for performing reasoning according to at least one embodiment of the present disclosure is exemplarily shown.

[0026] Figure 5 and Figure 6 The display effects obtained by using the method according to at least one embodiment of the present disclosure are exemplarily shown respectively.

[0027] Figure 7 A schematic timing diagram of a method for assisting text input according to at least one embodiment of the present disclosure is exemplarily shown.

[0028] Figure 8 The operating interface of the server of the input method according to at least one embodiment of the present disclosure is exemplarily shown.

[0029] Fig. 9 A structural diagram of a computing device according to at least one embodiment of the present disclosure is exemplarily shown. DETAILED DESCRIPTION

[0030] The preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the preferred embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0031] With the popularization of mobile Internet, online social networking scenarios are increasing, covering people's lives and work. Users often need to write articles on social applications to share their dynamics, which brings great creative pressure to users.

[0032] In view of this, the present disclosure proposes a solution for assisting text input, which can intelligently identify the current application, input scenario, and the content presented in the input area and generate relevant text creation themes and text accordingly, thereby assisting users to complete text input in different application scenarios more efficiently and improving user experience. In particular, when users use social applications, the solution of the present disclosure can automatically generate a text creation theme with the characteristics of the social application for users to choose, and generate corresponding text for users to input around the text creation theme selected by the user, thereby better helping users to complete online social interaction and reducing users' social pressure. It should be understood that although some solutions of the present disclosure are discussed below using social applications as an example, the present disclosure does not limit the type of application, and as long as there is a need for text input in the currently running application, the solution of the present disclosure can be used to assist input.

[0033] The method for assisting text input provided by the embodiments of the present disclosure can be applied to various types of handheld devices (such as mobile phones, personal digital assistants (PDAs), etc.), various types of computers (such as tablet computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, laptop computers, etc.), wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices and other terminal devices, and can also be applied to databases, servers and terminal artificial intelligence-based service response systems, etc. The embodiments of the present disclosure do not impose any restrictions on the specific types of application devices such as terminal devices, as long as the application device has a demand for text input.

[0034] As an example but not limitation, when the terminal device is a wearable device, the wearable device can also be a general term for wearable devices that are developed by applying wearable technology to intelligently design daily wearables, such as gloves, watches, AR head-mounted display devices, VR head-mounted display devices or MR (Mixed Reality) head-mounted display devices equipped with far-field communication modules and / or near-field communication modules.

[0035] In some embodiments, the terminal device may be a device having Figure 1 The exemplary hardware structure of the exemplary mobile phone 100 shown in FIG. Figure 1 As shown, the mobile phone 100 may specifically include: a radio frequency (RF) circuit 110, a memory 120, an input unit 130, a display unit 140, a sensor 150, an audio circuit 160, a short-range wireless communication module 170, a processor 180, and a power supply 190. Those skilled in the art will appreciate that Figure 1 The structure of the mobile phone 100 shown in the figure does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0036] Combine the following Figure 1 The components of the mobile phone 100 are described in detail:

[0037] The RF circuit 110 can be used for receiving and sending signals during information transmission or calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 180 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, LNA), a duplexer, etc. In addition, the RF circuit 110 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, and the wireless communication can include Global System For Mobile Communications (Global System For Mobile Communications, GSM), General Packet Radio Service (General Packet Radio Service, GPRS), Code Division Multiple Access (Code Division Multiple Access, CDMA), Wideband Code Division Multiple Access (Wideband Code Division Multiple Access, WCDMA), Time Division Code Division Multiple Access (Time-Division Code Division Multiple Access, TD-SCDMA), Long Term Evolution (Long Term Evolution, LTE), New Radio (New Radio, NR), GNSS, FM, low-orbit satellite connection and / or IR technology, etc. The GNSS may include a Global Positioning System (GPS), a Global Navigation Satellite System (GLONASS), a BeiDou Navigation Satellite System (BDS), a Quasi-Zenith Satellite System (QZSS) and / or a Satellite Based Augmentation System (SBAS), etc.

[0038] The memory 120 can be used to store software programs and modules. The processor 180 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 120. The memory 120 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as pictures, audio data, phone books, etc.), etc. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Specifically, the memory 120 may store an input method program, such as a vocabulary list and / or a vocabulary association table that comes with the program.

[0039] The input unit 130 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone 100. Specifically, the input unit 130 may include a touch panel 131 and other input devices 132. The touch panel 131, also known as a touch screen, can collect user touch operations on or near it (such as operations performed by users using fingers (such as single fingers or multiple fingers), styluses, or any other suitable objects or accessories on or near the touch panel 131), and drive the corresponding connection device according to a pre-set program. For example, a user can input corresponding characters on the touch panel 131 through an input method application, so that the input method application displays associated text and / or words or any combination thereof based on the input characters. Optionally, the touch panel 131 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch direction, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 180, and can receive the command sent by the processor 180 and execute it. In addition, the touch panel 131 can be implemented in various types such as resistive, capacitive, infrared and surface acoustic wave. In addition to the touch panel 131, the input unit 130 can also include other input devices 132. Specifically, other input devices 132 can include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, etc.

[0040] The display unit 140 can be used to display information input by the user or information provided to the user and various menus of the mobile phone. For example, the display unit 140 is configured to display an input method application (such as an input method panel, characters input by the user, and associated vocabulary, etc.). The display unit 140 may include a display panel 141. Optionally, the display panel 141 may be configured in the form of a liquid crystal display (LCD), a light emitting diode (LED), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), etc. Further, the touch panel 131 may cover the display panel 141. When the touch panel 131 detects a touch operation on or near it, it is transmitted to the processor 180 to determine the type of touch event. Subsequently, the processor 180 provides a corresponding visual output on the display panel 141 according to the type of touch event. Although in Figure 1 In the embodiment, the touch panel 131 and the display panel 141 are used as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 131 and the display panel 141 can be integrated to realize the input and output functions of the mobile phone.

[0041] The mobile phone 100 may also include at least one sensor 150, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 141 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 141 and / or the backlight when the mobile phone is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping, etc.); as for other sensors that can be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0042] The audio circuit 160, the speaker 161, and the microphone 162 can provide an audio interface between the user and the mobile phone. The audio circuit 160 can transmit the received audio data to the speaker 161 after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 162 converts the collected sound signal into an electrical signal, which is received by the audio circuit 160 and converted into audio data, and then the audio data is output to the processor 180 for processing, and then sent to another electronic device through the RF circuit 110, or the audio data is output to the memory 120 for further processing.

[0043] Communication technologies such as Wi-Fi, Bluetooth, Near Field Communication (NFC) and Ultra Wide Band (UWB) belong to short-range wireless transmission technologies. The mobile phone can help users send and receive emails, browse web pages and access streaming media through the short-range wireless module 170, which provides users with wireless broadband Internet access. The above-mentioned short-range wireless module 170 may include a Wi-Fi chip, a Bluetooth chip, an NFC chip and a UWB chip. Through the Wi-Fi chip, the mobile phone 100 can achieve the function of Wi-Fi Direct connection with other electronic devices, and can also make the mobile phone 100 work in an AP mode (Access Point mode) that can provide wireless access services and allow other wireless devices to access, or work in a STA mode (Station mode) that can connect to an AP but does not accept wireless devices to access, thereby establishing point-to-point communication between the mobile phone 100 and other Wi-Fi devices.

[0044] The processor 180 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. By running or executing software programs and / or modules stored in the memory 120, and calling data stored in the memory 120, it executes various functions of the mobile phone and processes data, thereby monitoring the mobile phone as a whole. Optionally, the processor 180 may include one or more processing units; optionally, the processor 180 may include, for example, an application processor (Application Processor, AP), a modem processor, a graphics processor (Graphics Processing Unit, GPU), an image signal processor (Image Signal Processor, ISP), a controller, a video codec, a digital signal processor (Digital Signal Processor, DSP), a baseband processor, and / or a neural network processor (Neural-Network Processing Unit, NPU) (for example, for running the language model described in any embodiment of this article), etc. Among them, different processing units can be independent devices or integrated in one or more processors.

[0045] The mobile phone 100 also includes a power source 190 (such as a battery) for supplying power to various components. Preferably, the power source can be logically connected to the processor 180 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption.

[0046] The mobile phone 100 may also include a camera. Optionally, the position of the camera on the mobile phone may be front-mounted or rear-mounted, which is not limited in the embodiments of the present application. The mobile phone may collect a scene image of the current scene through the camera, and determine the scene information and the scene type by analyzing the scene image.

[0047] In some embodiments, the mobile phone 100 can run, for example, social applications, office applications, game applications, or search applications. For example, in a chat interface of a social application, the display panel 141 of the mobile phone 100 can display the input interface of the input method program. The input interface may include, for example, a nine-square grid or a twenty-six-square grid interface, or the input interface may also include special characters, emoticons, or other content (for example, displayed on both sides of the nine-square grid or the twenty-six-square grid), and in response to the user's touch / click input on the input interface, the input interface may also display one or more corresponding predicted / associated words. For example, when the user clicks "TY" on the input interface, the predicted / associated words such as "agree", "same", and "universal" may be displayed on the input interface. In addition, the input method program can also automatically generate text using the method of the present disclosure as will be described in detail later, so as to assist the user in text input.

[0048] In other embodiments, for example, when the mobile phone 100 or other terminal device is connected to an I / O device (such as an external keyboard) and the corresponding input is provided by the I / O device such as the external keyboard, the mobile phone 100 or other terminal device may not display an interface such as a nine-grid or twenty-six-grid interface, but only display the touch / click corresponding text creation theme or text.

[0049] Figure 2 The schematic flow chart of the method for assisting text input according to at least one embodiment of the present disclosure is exemplarily shown. In the present disclosure, "plurality" or similar expressions refer to two or more.

[0050] like Figure 2 As shown, in step S210, the currently running application and the current input scene corresponding to the current interactive interface displayed on the screen are obtained as the first information, and in step S220, the content presented in the input area in the current interactive page is obtained as the second information.

[0051] It should be understood that although Figure 2 It is shown that step S220 is executed after step S210, but the present disclosure does not limit the execution order of step S210 and step S220, and the execution order of these two steps can be arranged arbitrarily as needed.

[0052] In some embodiments, the content obtained in the aforementioned step S220 may include text content and / or image content, wherein the text content and / or the image content is obtained by identifying the input area of ​​the current interactive page. The "input area" in the present disclosure refers to the area in the current interactive page where content already exists, and the content may be the content input by the current user, or the content input historically, such as historical chat records, etc. The scheme of the present disclosure does not require the content to have any meaning. For example, the text content input by the current user may be just a simple word, number or other symbol, etc., and it is not required to be a word or sentence with a certain meaning.

[0053] Figure 3A An exemplary display interface of a mobile phone screen using the method disclosed in the present invention is given. The current interactive interface is Figure 3A The social dynamic publishing page 101 of the social application shown in the figure has a current input scene of a social dynamic article creation scene, which belongs to the article creation scene. In the current interactive interface, there are three areas, namely, the text input area 102, the picture insertion area 103 and the title input area 104, which are used to input corresponding user content, namely, the text, the picture and the title text. Figure 3AIn the example, the user has only inserted a picture in the picture insertion area 103, and has not yet entered any content in the text input area 102 and the title input area 104. Therefore, the content of the second information obtained in step S220 is only the content of the picture inserted in the picture insertion area 103, and the picture insertion area 103 can also be called the input area. Figure 3A In the example of FIG. 1 , the text input area 102 and the title input area 104 in which nothing has been inputted are not inputted areas.

[0054] In other cases, if the current user has entered any text in the text input area 102 or the title input area 104, the text input area 102 or the title input area 104 becomes an input area, and the content of the second information also includes the content in the input area. The text in the input areas 102 and 104 can be recognized by various technologies such as OCR (Optical Character Recognition) to obtain the inserted text content, and the picture in the input area 103 can be recognized by various technologies such as image recognition models to obtain the inserted picture content, for example, the picture can be recognized by the machine learning model of the present disclosure or other suitable large language models, and the description related to the picture can be output.

[0055] As will be described later, there are also auxiliary interactive interfaces on the display screen, such as Figure 3A As shown in 105, it covers part of the current interactive interface. Figure 3A The auxiliary interaction interface shown in is an input method interface, but it is only exemplary and not a limitation of the present disclosure; the auxiliary interaction interface of the present disclosure may also be an interface of other applications or operating systems, that is, the scheme of the present disclosure can be used not only in conjunction with an input method, but also in conjunction with other applications, or can also be implemented separately as an independent application or a functional module of the operating system.

[0056] exist Figure 3A In the auxiliary interaction interface shown, there is also a button 99, which can be used to enable the function of using the auxiliary text input method of the present invention. That is, the auxiliary text input method of the present invention can be started in response to triggering a button 99 in the input method interface. It should be understood that the present invention is not limited to this, but can also automatically start the auxiliary text input method of the present invention according to other methods such as preset rules, currently detected information, etc.

[0057] Figure 3B Another exemplary display interface of a mobile phone screen using the method disclosed in the present invention is given. The current interactive interface is Figure 3BThe interactive page 201 of the instant messaging application shown in the figure, the current input scene belongs to the chat scene, and there are two areas in the current interactive interface, namely the chat record display area 202 and the chat message input area 203, wherein the chat record display area 202 is used to display the content of the instant messaging messages sent by each communication party, and the chat message input area 203 is used to input the message content to be sent by the user, such as text, pictures, etc. Figure 3B In the example, the chat record display area 202 includes 3 historical chat records, and nothing has been entered in the chat message input area 203 at present, so only the chat record display area 202 is the input area, and the content of the second information obtained in step S220 only includes the chat records in the chat record display area 202.

[0058] In some cases, even if the current user has entered any text in the chat message input area 203, the solution of the present disclosure may not consider the content in the area 203, that is, the input area never includes the chat message input area 203, and the content of the second information never includes the content entered by the current user. That is, when the current input scene is a chat scene, the content of the second information obtained in the aforementioned step S220 only includes the chat record, and the input area only includes the chat record display area.

[0059] With the aforementioned Figure 3A Similarly, the text and / or picture in the input area 202 may be recognized by various technologies such as OCR and image recognition models to obtain the text and / or picture content as the second information mentioned above.

[0060] With the aforementioned Figure 3A Similarly, there are auxiliary interactive interfaces on the display screen, such as Figure 3B As shown in 205 in FIG. 1 , it covers part of the current interactive interface. Figure 3A Similar to button 99 in Figure 3B In the auxiliary interactive interface shown, there may also be a button 299 which can be used to enable the function of using the auxiliary text input method disclosed in the present invention.

[0061] In some embodiments, the current interactive interface is provided by the first window that currently has the window focus, so the currently running application and the current input scene corresponding to the current interactive interface can be determined by obtaining information about the first window. For example, the information of the first window can be obtained, wherein the information of the first window includes the application package name and activity class name corresponding to the first window, and the currently running application is determined according to the application package name, and the current input scene is determined according to the activity class name. The currently running application may include a social application, etc., and the current input scene may include a scene in a social application for inputting / editing social dynamic articles, which may also be referred to as a social dynamic article creation scene, which belongs to an article creation scene.

[0062] For example, when the electronic device to which the disclosed method is applied uses an Android operating system, the information of the window currently having the window focus can be obtained through the system interface "dumpsys window|grep mCurrentFocus" as follows:

[0063] mCurrentFocus = Window{f72b3cc u0{application package name} / {activity class name}}

[0064] The "application package name" and "activity class name" are usually expressed in strings that can be understood by the programming language rather than in natural language. Therefore, in some cases, in order to facilitate the understanding of the subsequent machine learning model, a pre-set mapping table can be used to map the application package name and activity class name in the aforementioned information to obtain information expressed in natural language for the currently running application and the current input scenario.

[0065] For example, the following mapping table may be preset, and the corresponding currently running application and current input scenario may be obtained as the first information by searching the following table using the mCurrentFocus information obtained above:

[0066]

[0067] Back to Figure 2 After obtaining the first information and the second information, in step S230, a plurality of first text creation themes are presented on the auxiliary interactive interface on the screen, and the first text creation themes are generated using a machine learning model based at least on the first information and the second information.

[0068] In some embodiments, the auxiliary interaction interface is overlaid on part of the current interaction interface, such as the aforementioned Figure 3A As shown in 105 in the figure. The auxiliary interactive interface can be provided by any application using the method disclosed in the present invention.

[0069] In some embodiments, the machine learning model generates the multiple first text creation themes according to the characteristics of text creation in the current input scenario of the currently running application and using the second information as auxiliary information; the multiple first text creation themes have the characteristics of text creation in the current input scenario of the currently running application.

[0070] The machine learning model can be trained in various ways to learn the characteristics of text creation in the current input scenario of the currently running application, and to generate a text creation theme with the characteristics accordingly. As a result, the solution disclosed in the present invention can more accurately infer the user's current creation intention, thereby better helping the user to complete the creation in the current input scenario.

[0071] The present disclosure does not limit the type, deployment location, etc. of the machine learning model, as long as it can generate the text creation theme and text according to the corresponding information provided. For example, the machine learning model can be a model capable of natural language processing, such as a large language model, and can be deployed on the client, client device (i.e., a device with a client installed), server, server device (i.e., a device with a server installed), or various clouds, etc., of any application using the method of the present disclosure.

[0072] Figure 4 A block diagram of a computing system 400 for performing reasoning according to an example of the present disclosure is depicted. In some cases, the Figure 4 The computing system 400 implements various reasoning and / or training operations of the machine learning model described in the various embodiments of the present disclosure.

[0073] System 400 includes a user computing device 410 , a server computing device 430 , and a training computing device 450 communicatively coupled via a network 470 .

[0074] The user computing device 410 may be any type of computing device, such as the mobile phone or other types of computing devices mentioned above in the present disclosure.

[0075] The user computing device 410 includes one or more processors 412 and memory 414. The one or more processors 412 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or multiple processors operably connected. The memory 414 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 414 can store data 416 and instructions 418 executed by the one or more processors 412 to cause the user computing device 410 to perform operations.

[0076] In some implementations, the user computing device 410 may store or include one or more machine learning models 420. For example, the machine learning model 420 may be or may otherwise include various machine learning models, such as a neural network (e.g., a deep neural network) or other types of machine learning models, including nonlinear models and / or linear models. The neural network may include a feedforward neural network, a recursive neural network (e.g., a long short-term memory recursive neural network), a convolutional neural network, or other forms of neural networks. Some example machine learning models may utilize an attention mechanism such as self-attention. For example, some example machine learning models may include a multi-head self-attention model (e.g., a transformer model).

[0077] In some implementations, one or more machine learning models 420 may be received from a server computing device 430 via a network 470 and stored in a user computing device memory 414 before being used or otherwise implemented by one or more processors 412. In some implementations, a user computing device 410 may implement multiple parallel instances of a single machine learning model 420 (e.g., multiple instances of a machine learning model 420 perform parallel inference).

[0078] More specifically, the machine learning model 420 may be used to generate predicted characters or classification results in response to user input. For example, the user may input content into an operating environment executed by the processor 412 of the user device 410 via a user input component 422 (e.g., keyboard / mic, etc.). It should be noted that the machine learning model 420 may generate predicted characters or classification results in a one-time reasoning time process, so that they may be provided to the user of the user computing device 410 in real time.

[0079] For example, the user input component 422 can be a touch-sensitive component (e.g., a touch-sensitive display screen or a touchpad) that is sensitive to the touch of a user input object (e.g., a finger or a stylus). The touch-sensitive component can be used to implement a virtual keyboard. Other example user input components include a microphone, a traditional keyboard, or other devices that a user can use to provide user input. For another example, the user input component 422 can also include a display component (e.g., an OLED, AMOLED, or Mirco LED display screen), which is configured to provide display functions such as providing an interactive interface and an auxiliary interactive interface, whereby the machine learning model 420 can be based on the inserted content of the interactive interface and the selected auxiliary interactive interface, such as a text creation theme or other suitable selection types for reasoning, such as generating coherent text and being able to display it to the user on an interactive interface such as an auxiliary interactive interface.

[0080] In addition, one or more machine learning models 440 may be included in a server computing device 430 that communicates with the user computing device 410 according to a client-server relationship, and the capabilities provided by the machine learning models 440 are stored and implemented by the server computing device 430. For example, the machine learning model 440 may be implemented by the server computing device 430 as part of a network service (e.g., a content generation service or an inference service or a conversational robot service). Thus, one or more machine learning models 420 may be stored and implemented at the user computing device 410, and / or one or more models 440 may be stored and implemented at the server computing device 430.

[0081] For example, the machine learning model 420 stored and implemented in the user computing device 410 may be configured to provide content mapping capabilities to map the inserted content of the interactive interface and the selected, for example, text creation subject or other suitable selection type of the auxiliary interactive interface into suitable features, and transmit the suitable features to the machine learning model 440 stored and implemented in the server computing device 430 through the network 470. The machine learning model 440 is capable of reasoning based on the suitable features, such as generating coherent text or other suitable content (such as images, videos or audio) and transmitting it to the user computing device 410 for display through the network 470. In other examples, the feature mapping and reasoning generation capabilities are fully implemented by the machine learning model 440, and the user computing device 410 may not include any machine learning model 420.

[0082] The server computing device 430 includes one or more processors 432 and memory 434. The one or more processors 432 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or multiple processors operably connected. The memory 434 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 434 can store data 436 and instructions 438 executed by the processor 432 to cause the server computing system 430 to perform operations.

[0083] In some implementations, server computing device 430 includes or is implemented by one or more server computing devices. Where server computing device 430 includes multiple server computing devices, such server computing devices may operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.

[0084] As described above, the server computing device 430 can store or otherwise include one or more machine learning models 440. For example, the machine learning model 440 can include various machine learning models. Example machine learning models include neural networks or other multi-layer nonlinear models. Example neural networks include feedforward neural networks, deep neural networks, recursive neural networks, and convolutional neural networks. Some example machine learning models can be assisted by attention mechanisms such as self-attention. For example, some example machine learning models can include multi-head self-attention models (e.g., transformer models).

[0085] User computing device 410 and / or server computing device 430 may train machine learning model 420 and / or machine learning model 440 through interaction with training computing device 450 communicatively coupled via network 470. Training computing device 450 may be separate from server computing device 430, or may be part of server computing device 430.

[0086] The training computing device 450 includes one or more processors 452 and a memory 454. The one or more processors 452 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be one processor or a plurality of processors operably connected. The memory 454 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc., and combinations thereof. The memory 454 may store data 456 and instructions 458 executed by the processor 452 to cause the training computing device 450 to perform operations. In some implementations, the training computing device 450 includes or is otherwise implemented by one or more server computing devices.

[0087] The training computing system device may include a model trainer 460 that uses various training or learning techniques, such as error back propagation, to train the machine learning model 420 and / or the machine learning model 440 stored at the user computing device 410 and / or the server computing device 430. For example, a loss function may be back propagated through the (multiple) machine learning models to update one or more parameters of the (multiple) machine learning models (e.g., based on the gradient of the loss function). Various loss functions may be used, such as mean squared error, likelihood loss, cross entropy loss, hinge loss, and / or various other loss functions. Gradient descent techniques may be used to iteratively update parameters over multiple training iterations.

[0088] In some implementations, performing back propagation of errors may include performing truncated back propagation over time. The model trainer 460 may perform a variety of generalization techniques (eg, weight decay, dropout, etc.) to improve the generalization ability of the trained model.

[0089] Specifically, the model trainer 460 can train the machine learning model 420 and / or the machine learning model 440 based on a set of training data 462 to obtain a trained machine learning model. The training data 462 may include, for example, label data indicating input data that needs to be classified and indicating the classification result of the input data. In other examples, the training data 462 may be input data that does not include labels, and the machine learning model 420 and / or the machine learning model 440 can use these input data that do not include labels to learn, for example, the structure, pattern, and usage habits of the input data.

[0090] In some implementations, if the user has provided consent to use private data for training, the training examples may be provided by the user computing device 410. Thus, in such implementations, the machine learning model 420 provided to the user computing device 410 may be trained by the training computing system 450 based on user-specific data received from the user computing device 410. In some cases, this process may be referred to as personalized training of the model.

[0091] Model trainer 460 includes computer logic for providing the desired functionality. Model trainer 460 can be implemented in hardware, firmware, and / or software that controls a general purpose processor. For example, in some implementations, model trainer 460 includes a program file stored on a storage device, loaded into a memory, and executed by one or more processors. In other implementations, model trainer 460 includes one or more sets of computer executable instructions stored in a tangible computer readable storage medium, such as RAM, a hard disk, or an optical or magnetic medium.

[0092] Network 470 can be any type of communication network, such as a local area network (e.g., an intranet), a wide area network (e.g., the Internet), or some combination thereof, and can include any number of wired or wireless links. In general, communications on network 480 can be carried via any type of wired and / or wireless connection using a variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, secure HTTP, SSL).

[0093] Back to Figure 2In some implementations of step S230, a preset prompt template may be used to generate the first text creation theme. For example, the first information and the second information may be substituted into a preset first prompt template to obtain a first prompt, and the first prompt is input into the machine learning model to generate the first text creation theme. The machine learning model may be a large language model, in particular, a large language model fine-tuned for the text creation theme generation task and the text generation task of the present disclosure.

[0094] The preset first prompt template may include a statement declaring the work role, work task and reasoning process of the large language model. The first prompt template may be a part of the total prompt template, which is used to generate both the text creation theme and the subsequent text. For example, the total prompt template may include: "When you receive the content of the first message and the second message, you will give three possible article creation directions based on the user's application and the original text. You will wait for the user's choice first, and then give an application article based on the user's choice." Therefore, the total prompt template is divided into two parts of reasoning: the first part is: "When you receive the content of the first message and the second message, you will give three possible article creation directions based on the user's application and the original text", so that the large language model generates multiple text creation themes; the second part is: "You will wait for the user's choice first, and then give an application article based on the user's choice", so that the large language model generates text around a selected text creation theme as described later; the first prompt template may include the first part of the aforementioned total prompt template.

[0095] The large language model can perform the first part of reasoning based on the first prompt, and based on the first information and the second information obtained previously, obtain multiple text creation themes with the text creation characteristics of the current input scenario of the current application for the user to choose. For example, when the first information indicates that the current running social dynamic article creation scenario is a social application, based on the main characteristics of article creation in this scenario learned by the large language model, the following three text creation themes can be generated: "Generate a travel template", "Generate a grass-planting copy", and "Generate a food review".

[0096] The generated multiple text creation themes can be presented on the auxiliary interactive interface on the current screen, such as Figure 5 As shown in 501 in the above. For example, Figure 3A After the trigger button 99 is pressed, the function of the method disclosed in the present invention is started, the first information and the second information are obtained, and three text creation themes "write a seeding tweet", "recommendation copy for exquisite wallpapers" and "wallpaper creation experience" are generated accordingly, and the user enters Figure 5 The display interface shown in FIG. Figure 5As shown in 501, the three generated text creation themes are respectively presented on the auxiliary interactive interface.

[0097] Back to Figure 2 After presenting multiple first text creation themes, in step S240, in response to selecting a first text creation theme, a first text is presented on the auxiliary interaction interface, wherein the first text is generated using the machine learning model around the selected first text creation theme and using at least the second information as auxiliary information.

[0098] The machine learning model mentioned in this step is the same as the machine learning model in the aforementioned step S230.

[0099] Similarly, in some embodiments, a preset prompt template can also be used to generate the first text. For example, a selected first text creation theme and the second information can be substituted into a preset second prompt template to obtain a second prompt, and the second prompt is input into the machine learning model to generate the first text. The machine learning model can be a large language model, in particular, a large language model fine-tuned for the text creation theme generation task and the text generation task disclosed in the present invention. The second prompt template can include the second part of the aforementioned general prompt template, and the large language model can perform a second part inference based on the second prompt, around a selected first text creation theme and at least with the second information as auxiliary information, to obtain a corresponding first text for user input or modified input.

[0100] The generated text can be presented on the auxiliary interactive interface on the current screen, such as Figure 6 For example, when the user selects the aforementioned Figure 5 After selecting any text creation subject 501 in the text creation subject, a text with the selected content as the text creation subject is generated accordingly, and the text creation subject is entered. Figure 6 The display interface shown in FIG. Figure 6 As shown in 505 , the generated text (ie, the content in the box below “suggested content” in the figure) is presented on the auxiliary interaction interface.

[0101] Then, the user can click the "Insert" button 508 above to insert the text in 505 into the text input area 102 (which is an input area, see Figure 3A ,Right now Figure 6The user can click the "Insert" button 508 to insert the text in 505 into the area 102, or the user can click the "Insert" button 508 to insert the text in 505 into the area 102 and then modify the text in the area 102. The "input area" in the present disclosure refers to an area in the current interactive page that allows the user to enter text, such as the aforementioned Figure 3A The areas 102 and 104 in the figure are input areas, and the aforementioned Figure 3B The area 203 in the figure is an input-enabled area.

[0102] In addition, in response to triggering the "Insert" button 508, the text in 505 can be inserted into any input area where text can be input, not limited to area 102. For example, the input area where the text is to be inserted can be determined based on other default settings or the user's current selection. In addition, in some cases, the input area where the text is to be inserted can be an input area that does not include the content presented in the input area in the aforementioned step S220, that is, it can be an input area where no text content has been inserted.

[0103] That is, in some embodiments, the method of the present disclosure may further include: presenting a fifth control for inserting text while presenting the first text on the auxiliary interaction interface; in response to triggering the fifth control, inserting the first text into an inputtable area of ​​the current interaction interface. The fifth control is, for example, the aforementioned Figure 6 The “Insert” button 508 in the example.

[0104] In addition, in some embodiments, additional settings may be provided for generating text. For example, while presenting multiple text creation themes for the user to choose from, one or more controls may be provided for the user to set other properties of the text that the user desires, such as the length and / or style of the text, and finally the aforementioned machine learning model may generate the corresponding text based on the settings of the controls.

[0105] Therefore, in some embodiments, the method of the present disclosure may also include: while presenting the multiple first text creation themes on the auxiliary interactive interface, at least a first control for setting the length and / or style of the text to be generated is also presented, and the first text is generated using the machine learning model around a selected first text creation theme, and with at least the second information and the length and / or style set by the first control as auxiliary information.

[0106] The first control is, for example, Figure 5 The control 502 shown, such as Figure 5As shown, the default option is "short", which can indicate the length of the text or the style of the text, that is, the text to be generated is short and simple. The user can click on the control 502 and select the length and / or style of the text he or she desires from the drop-down list.

[0107] In addition, after generating the text creation theme, the user may not be satisfied and want to regenerate a different text creation theme. To this end, in some embodiments, the scheme of the present disclosure may also provide one or more corresponding controls for user operation on the auxiliary interactive interface presenting the text creation theme. Using these controls, the user can adjust the generated text creation theme. For example, the user may simply want to change the text creation theme, that is, without providing additional information, just to regenerate a different text creation theme; or, the user can directly input text that expresses his or her needs to adjust the text creation theme generated by the machine learning model, that is, the user can have one or more rounds of dialogue with the machine learning model, so that the machine learning model adjusts the generated text creation theme based on the one or more rounds of dialogue.

[0108] Therefore, in some embodiments, the method of the present disclosure may further include: presenting a first update button for updating the text creation subject while presenting the multiple first text creation subjects on the auxiliary interaction interface; and in response to triggering the first update button, presenting multiple second text creation subjects on the auxiliary interaction interface to replace the multiple first text creation subjects; wherein the second text creation subject is regenerated using the machine learning model based on at least the first information and the second information.

[0109] The first update button is, for example, Figure 5 As shown in the button 503, the user can click the button 503 to refresh the three text creation themes displayed below.

[0110] In addition, in some embodiments, the method of the present disclosure may also include: presenting the multiple first text creation subjects on the auxiliary interaction interface while also presenting a third control for communicating with the machine learning model; and in response to inputting a first dialogue via the third control, presenting multiple third text creation subjects on the auxiliary interaction interface instead of the multiple first text creation subjects; wherein the third text creation subjects are generated using the machine learning model based on at least the first information, the second information and the first dialogue.

[0111] In some cases, after the first conversation, the user may not be satisfied with the newly generated text creation theme, so the user may continue to have a conversation with the machine learning model using a third control. That is, in some embodiments, the method of the present disclosure may also include: in response to inputting a second conversation via the third control, presenting multiple fourth text creation themes in the auxiliary interaction interface to replace the multiple third text creation themes; wherein the fourth text creation theme is generated using the machine learning model based on at least the first information, the second information, the first conversation, and the second conversation. It should be understood that after the second round of conversation, the user can continue one or more rounds of conversation as needed, and the present disclosure does not limit the number of conversations. Each updated text creation theme is generated based on all conversation records from the first round of conversation to the current conversation.

[0112] The third control is, for example, Figure 5 As shown in the control 504, after the user clicks the control 504, a text input box will appear, and the user can enter each conversation in the text input box. After each conversation is entered, the three text creation themes below can be updated based on the conversation record.

[0113] In addition, after generating the text, the user may also be dissatisfied or change his mind, and hope to regenerate a different text or even change the previously selected text creation theme. To this end, in some embodiments, the method of the present disclosure can also provide one or more corresponding controls on the auxiliary interactive interface presenting the text for user operation. Using these controls, the user can adjust the final generated text. For example, the user may simply want to change the text content, that is, not give additional information but just let a different text be regenerated; or, the user can directly enter the text expressing his needs to adjust the text generated by the machine learning model, that is, the user can have one or more rounds of dialogue with the machine learning model, so that the machine learning model can adjust the generated text based on the one or more rounds of dialogue; or, the user can return to the text creation theme interface to change the selected text creation theme and / or other settings (such as length and / or style), and then regenerate the text based on the changed text creation theme and / or other settings.

[0114] Therefore, in some embodiments, the method of the present disclosure may also include: presenting a second control while presenting the first text on the auxiliary interaction interface, the second control being used to return to the auxiliary interaction interface presenting the multiple first text creation themes; in response to triggering the second control, returning to the auxiliary interaction interface presenting the multiple first text creation themes; and then, in response to changing the length and / or style set by the first control, presenting a second text on the auxiliary interaction interface to replace the first text, the second text being generated using the machine learning model around the selected first text creation theme, and with at least the second information and the changed length and / or style as auxiliary information.

[0115] It should be understood that after returning to the auxiliary interactive interface presenting the multiple first text creation themes, it is possible, alternatively or additionally, to change the selected first text creation theme, or to reselect a new text creation theme after updating the text creation theme through one or more of the aforementioned controls, and then regenerate the text around the newly selected text creation theme.

[0116] In addition, in some embodiments, the method of the present disclosure may also include: presenting a second update button for updating the text while presenting the first text on the auxiliary interaction interface; and in response to triggering the second update button, presenting a fifth text on the auxiliary interaction interface to replace the first text; wherein the fifth text is regenerated using the machine learning model around the creative theme of a selected first text and with at least the second information as auxiliary information.

[0117] The second update button is, for example, Figure 6 As shown in the button 507, the user clicks the button 507 to refresh the text in 505 displayed below.

[0118] In addition, in some embodiments, the method of the present disclosure may also include: presenting a fourth control for communicating with the machine learning model while presenting the first text on the auxiliary interaction interface; and in response to inputting a third dialogue via the fourth control, presenting a third text on the auxiliary interaction interface instead of the first text; wherein the third text is created around a selected first text using the machine learning model, and is generated with at least the second information and the third dialogue as auxiliary information.

[0119] In some cases, after the first conversation, the user may not be satisfied with the newly generated text, so the fourth control may be used to continue the conversation with the machine learning model. That is, in some embodiments, the method of the present disclosure may also include: in response to inputting a fourth conversation via the fourth control, presenting a fourth text in the auxiliary interaction interface instead of the third text; wherein the fourth text is generated using the machine learning model around the selected first text creation theme, and at least the second information, the third conversation and the fourth conversation are used as auxiliary information. It should be understood that after the second round of conversation, the user can continue one or more rounds of conversation as needed. The present disclosure does not limit the number of conversations, and each updated text is generated based on all conversation records from the first round of conversation to the current conversation.

[0120] The fourth control is, for example, Figure 6 As shown in the text input box 506, the user can input each conversation in the text input box 506, and after each conversation is input, the text below can be updated based on the conversation record.

[0121] In some embodiments, the aforementioned Figure 2 Each step in the method and other optional steps are executed by the client of the input method, wherein each text creation theme and each text presented by the client can be generated by a machine learning model deployed on the client or by the client calling a machine learning model that is locally deployed (i.e., deployed on the client device) or remotely deployed (e.g., deployed in the cloud). Alternatively, each text creation theme and each text presented by the client can be received from a server that provides background services for the input method, wherein the client sends various information required for generation to the server, and then the machine learning model deployed on the server or the server calls a machine learning model that is locally deployed (i.e., deployed on the server device) or remotely deployed (e.g., deployed in the cloud) to generate the text creation theme and text, and returns the generated results to the client.

[0122] Please refer to the attached Figure 7 As shown, it shows a schematic timing diagram of a method of the present disclosure provided by an example of the present disclosure, in which a client and a server collaborate to implement the method of the present disclosure. It should be understood that the client and the server of the present disclosure are not limited to the client and the server of the input method, but may include the client and the server of any application or operating system functional module that can implement the method of the present disclosure. As mentioned above, the solution of the present disclosure can be embedded not only in the input method, but also in other applications, or can also be implemented separately as an independent application or a functional module of the operating system. Figure 7 For details or variations of each step in the above, please refer to the previous Figure 2 What has been described will not be repeated here.

[0123] like Figure 7 As shown, in step S710, the first information and the second information are obtained at the client.

[0124] Next, in step S711, the client sends the first information and the second information to the server.

[0125] Next, in step S712, the server substitutes the received first information and second information into a preset first prompt template to obtain a first prompt, and inputs the first prompt into the machine learning model to generate a corresponding text creation theme.

[0126] For example, Figure 8 The operation interface of the server is shown as an example, where 109 is a prompt template processed in advance by the server (in actual use, this should be a specific textual expression of a prompt template, and the figure only uses "prompt template" as a representative), the received first information is filled in the input box indicated by 107, and the second information is filled in the input box indicated by 108. Then, the machine learning model is operated to generate multiple text creation themes.

[0127] Next, in step S713, the server sends the generated multiple text creation themes to the client. Then, in step S714, the client completes the presentation of the received text creation themes, for example Figure 5 shown.

[0128] Next, in step S715, the client obtains the text creation subject selected by the user, and in step S716, the client sends the text creation subject selected by the user and, in some cases, the length and / or style setting values ​​(which may be default values ​​if the user has not made a selection) to the server.

[0129] Next, in step S717, the server generates corresponding text using the machine learning model based on the received selected text creation subject, setting value and the second information received previously.

[0130] For example, in Figure 8 In the illustrated server operation interface, the received selected text creation subject is entered into 111, and the length setting value is entered into the "length" input box. Then, the machine learning model is operated to generate the corresponding text.

[0131] Next, in step S718, the server sends the generated text to the client. Then, in step S719, the client completes the presentation of the received text, for example Figure 6 shown.

[0132] Next, in step S720, since the user desires to regenerate the text, the Figure 5 and Figure 6 One or more controls (including buttons) presented in the application are operated on, so the client obtains the user's operation information on the one or more controls, and sends the operation information to the server in step S721.

[0133] Next, in step S722, the server uses the machine learning model to regenerate the corresponding text creation theme or text based on the operation information of the control. For specific details, please refer to the previous description of each control.

[0134] Next, in step S723, the server sends the regenerated text creation subject or text to the client, and then in step S724, the client updates the presented text creation subject or text accordingly.

[0135] Therefore, in some embodiments of the present disclosure as described above, when the user does not input meaningful or semantic information, or only inputs a small amount of information, that is, when the user has not yet figured out how to create the text, the method of the present disclosure can automatically identify the current application and input scenario and generate relevant text creation themes and text for the user to use, thereby saving user time, better helping the user to complete text input in different application scenarios, and improving user experience.

[0136] Fig. 9 A schematic diagram of the structure of a computing device that can be used to implement the above-mentioned method for assisting text input according to at least one embodiment of the present disclosure is shown.

[0137] See also Fig. 9 , the computing device 900 includes a memory 910 and a processor 920 .

[0138] The processor 920 may be a multi-core processor or may include multiple processors. In some embodiments, the processor 920 may include a general-purpose main processor and one or more special coprocessors, such as a graphics processing unit (GPU), a digital signal processor (DSP), etc. In some embodiments, the processor 920 may be implemented using a customized circuit, such as an application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0139] The memory 910 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 920 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 910 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 910 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (e.g., DVD-ROM, double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0140] The memory 910 stores executable codes. When the executable codes are processed by the processor 920 , the processor 920 can execute the above-mentioned method for assisting text input.

[0141] In addition, the method according to the present disclosure may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing the above steps defined in the above method of the present disclosure.

[0142] Alternatively, the present disclosure may also be implemented as a non-temporary machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) on which executable code (or computer program, or computer instruction code) is stored. When the executable code (or computer program, or computer instruction code) is executed by a processor of an electronic device (or computing device, server, etc.), the processor executes the various steps of the above-mentioned method according to the present disclosure.

[0143] Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or a combination of both.

[0144] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the system and method according to multiple embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0145] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for assisting text input, comprising: Acquire a currently running application and a current input scenario corresponding to a current interactive interface displayed on the screen as first information; Acquire the content presented in the input area in the current interactive page as the second information; Presenting a plurality of first text creation themes on the auxiliary interactive interface on the screen, wherein the first text creation themes are generated by using a machine learning model based on at least the first information and the second information; as well as In response to selecting a first text creation theme, a first text is presented on the auxiliary interaction interface, wherein the first text is generated using the machine learning model around the selected first text creation theme with at least the second information as auxiliary information.

2. The method according to claim 1, wherein: The content includes text content and / or picture content, wherein the text content and / or the picture content is obtained by identifying the input area; In the case where the current input scene is an article creation scene, the input area includes an input picture insertion area, a title input area and / or a text input area; or When the current input scene is a chat scene, the input area only includes a chat record display area.

3. The method according to claim 1, wherein: The current interactive interface is provided by the first window currently having the window focus, Acquiring the currently running application and the current input scenario corresponding to the current interactive interface as first information includes: Acquire information of the first window, wherein the information of the first window includes an application package name and an activity class name corresponding to the first window; and The currently running application is determined according to the application package name, and the current input scene is determined according to the activity class name.

4. The method according to claim 1, wherein: The auxiliary interaction interface covers part of the current interaction interface, and the method further includes: The auxiliary interactive interface presents the plurality of first text creation themes and also presents at least a first control for setting the length and / or style of the text to be generated, and The first text is generated using the machine learning model around the creative theme of a selected first text, and with at least the second information and the length and / or style of the first control setting as auxiliary information.

5. The method according to claim 4, further comprising: A second control is also presented on the auxiliary interaction interface while the first text is presented, the second control being used to return to the auxiliary interaction interface presenting the creation themes of the plurality of first texts; In response to triggering the second control, returning to the auxiliary interactive interface presenting the plurality of first text creation themes; In response to changing the length and / or style of the first control setting, a second text is presented in the auxiliary interaction interface instead of the first text, wherein the second text is created around the selected first text using the machine learning model and is generated with at least the second information and the changed length and / or style as auxiliary information.

6. The method according to claim 1, further comprising: Presenting the plurality of first text creation themes on the auxiliary interaction interface and also presenting a first update button for updating the text creation themes; as well as In response to triggering the first update button, presenting a plurality of second text creation subjects in the auxiliary interaction interface to replace the plurality of first text creation subjects; Among them, the second text creation theme is regenerated using the machine learning model based on at least the first information and the second information.

7. The method according to claim 1, further comprising: Presenting the plurality of first text creation themes on the auxiliary interactive interface while also presenting a third control for communicating with the machine learning model; as well as In response to inputting a first dialogue via the third control, presenting a plurality of third text creation subjects in the auxiliary interaction interface instead of the plurality of first text creation subjects; Among them, the third text creation theme is generated using the machine learning model based on at least the first information, the second information and the first conversation.

8. The method according to claim 7, further comprising: In response to inputting a second dialogue via the third control, presenting a plurality of fourth text creation subjects in the auxiliary interaction interface instead of the plurality of third text creation subjects; Among them, the fourth text creation theme is generated using the machine learning model based on at least the first information, the second information, the first conversation and the second conversation.

9. The method according to claim 1, further comprising: Presenting the first text on the auxiliary interaction interface while also presenting a fourth control for communicating with the machine learning model; as well as In response to inputting a third dialogue via the fourth control, presenting a third text in the auxiliary interaction interface instead of the first text; The third text is generated using the machine learning model around the main idea of ​​a selected first text, and with at least the second information and the third dialogue as auxiliary information.

10. The method according to claim 9, further comprising: In response to inputting a fourth dialogue via the fourth control, presenting a fourth text in the auxiliary interaction interface instead of the third text; The fourth text is generated using the machine learning model around the main theme of a selected first text, and with at least the second information, the third conversation and the fourth conversation as auxiliary information.

11. The method according to claim 1, further comprising: Presenting the first text on the auxiliary interactive interface while also presenting a second update button for updating the text; as well as In response to triggering the second update button, presenting a fifth text in the auxiliary interactive interface instead of the first text; The fifth text is regenerated using the machine learning model around the creative theme of a selected first text and with at least the second information as auxiliary information.

12. The method according to claim 1, further comprising: Presenting the first text on the auxiliary interactive interface and also presenting a fifth control for inserting text; In response to triggering the fifth control, the first text is inserted into an inputtable area of ​​the current interactive interface.

13. The method according to any one of claims 1 to 12, wherein: The first text creation subject is generated by using a machine learning model based on at least the first information and the second information, including: Substituting the first information and the second information into a preset first prompt template to obtain a first prompt; and Inputting the first prompt into the machine learning model to generate the first text creation subject; and / or The first text is generated by using the machine learning model around a selected first text creation theme with at least the second information as auxiliary information, including: Substituting the selected first text creation theme and the second information into a preset second prompt template to obtain a second prompt; and The second prompt is input into the machine learning model to generate the first text.

14. A computing device comprising: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to execute the method according to any one of claims 1 to 13.

15. A non-transitory machine-readable storage medium having executable codes stored thereon, which, when executed by a processor of an electronic device, causes the processor to execute the method according to any one of claims 1 to 13.

Images