Handwriting-input-based interface display method and electronic device

Abstract

A handwriting-input-based interface display method and an electronic device, which are used for providing more functions in a handwriting-input-based scenario, so as to shorten the interaction time of handwriting input, and improve the efficiency of handwriting input. In the method, on the basis of handwritten text obtained after a user performs handwriting input, an electronic device may display at least one associational word to the user; and when a confirmation operation of the user on one of the associational words is detected, the associational word that the user confirms may be displayed by referring to the handwriting style of the handwritten text. For example, the trajectory of the associational word may have the style of the trajectory of the handwritten text. With regard to another example, the typesetting of the associational word may be performed with reference to the typesetting style of the handwritten text.

Description

Interface display method based on handwriting input and electronic device

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of the People's Republic of China on December 20, 2023, with application number 202311770192.X and invention name "An interface display method and electronic device based on handwriting input", the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the technical field of electronic devices, and in particular to an interface display method and electronic device based on handwriting input. Background Art

[0004] With the development of electronic devices, the interactive functions between users and electronic devices are becoming increasingly stronger. In some possible scenarios, users want to use a stylus, finger or other handwriting device to implement handwriting input on the electronic device.

[0005] Since handwriting input is generally generated in response to user touch operations, the format and layout of handwritten content are not fixed, and handwriting efficiency is low. Therefore, considering the characteristics of handwriting input scenarios, how to provide more functions for handwriting input scenarios is of great research significance. Summary of the Invention

[0006] The present application provides an interface display method and electronic device based on handwriting input, which are used to provide more functions based on handwriting input scenarios, reduce the interaction time of handwriting input, and improve the efficiency of handwriting input. For example, it is possible to format the content in the handwriting input scenario. For another example, it is also possible to provide associative words based on the user's handwritten text, and to achieve that the trajectory of the associative words has the same or similar style as the trajectory of the handwritten text. Therefore, this method can improve the convenience and user experience of the user's handwriting input.

[0007] In a first aspect, the present application provides an interface display method based on handwriting input, which can be applied to electronic devices. In this method, a first interface is displayed in response to a user's handwriting input operation. The first interface includes handwritten text and at least one associative word, and the handwritten text includes at least one of the following types: Chinese characters and pinyin. In response to a confirmation operation of the first associative word, a second interface is displayed. The first associative word is one of the at least one associative word. The handwriting style of the first associative word is the same as the handwriting style of the user's historical handwriting input of the first associative word. In addition, the user's handwriting style can be reflected by, but is not limited to, the following information: font trajectory, font size, text spacing, typesetting format, etc. Therefore, the same handwriting style can include the same font trajectory, the same font size, the same text spacing, the same typesetting format, etc. Optionally, the handwriting style of the user's historical handwriting input of the first associative word can be: obtained based on the user's historical handwriting input of the first associative word, or generated with reference to the user's historical handwriting input.

[0008] In this method, when a user inputs Chinese characters by handwriting, at least one associated word can be displayed based on the handwritten text. Furthermore, after detecting the user's confirmation operation on the associated word, the trajectory of the associated word can be processed with reference to the user's handwriting style. In this way, the electronic device can not only display associated words in response to the handwritten text, but also display associated words in the user's handwriting style, thereby realizing more functions in the handwriting input scenario.

[0009] In some implementations, the electronic device may be installed with note-taking software or memo software. Models for handwriting style extraction are integrated into these software or the operating system of the electronic device. These models can extract the user's handwriting style based on the user's handwriting input. Thus, when the electronic device displays the associated word selected by the user, the displayed associated word can be consistent with the style of other characters, words, sentences, or paragraphs input by the user's handwriting.

[0010] For those skilled in the art or general consumers, this consistency will generally improve as the amount of handwriting input by the user increases. In other words, the longer the user uses the electronic device to write, the more the associative words provided will look like what the user wrote himself. When the user first uses these software, the handwriting style of the associative words presented may not necessarily be very similar to the user's handwriting style. However, based on the capabilities of the electronic device or the capabilities of the applications installed on the electronic device, after using the electronic device for a period of time, with the user's handwriting input as a reference, the associative words presented by the electronic device can be the same as the user's handwriting style, so as to present a consistent display effect on the display interface.

[0011] In one possible scenario, in response to a user's handwriting input operation, displaying a first interface includes: obtaining handwritten text in response to the handwriting input operation; the handwritten text includes pinyin; and obtaining at least one associated word in response to an operation to convert the pinyin included in the handwritten text into Chinese characters. The at least one associated word is a Chinese character obtained based on the pinyin.

[0012] In this scenario, when the handwriting input of Chinese includes pinyin, the electronic device can also provide the function of converting pinyin to Chinese characters, and the converted Chinese characters can also have the user's handwriting style, thereby realizing more functions in the handwriting input scenario.

[0013] In one possible scenario, the first associated word in the second interface is displayed in a first display position and a second display position; the first display position displays the first at least one Chinese character included in the first associated word, and the second display position displays the remaining at least one Chinese character included in the first associated word. The first display position and the second display position have a positional sequence typesetting relationship. The positional sequence typesetting relationship is derived based on the handwritten text.

[0014] In this scenario, the electronic device can also refer to the layout style of the handwritten text to implement the layout of the associated words, so that the layout of the associated words can have the layout style of the handwritten text. Therefore, more functions in the handwriting input scenario can be realized.

[0015] In a possible scenario, the first at least one Chinese character described above reaches the page boundary, or the first at least one Chinese character reaches the display area of ​​the existing writing track.

[0016] In this scenario, when the electronic device typeset the associated words, it can also consider the interface boundaries and the writing situation on the interface, thereby enabling line break typesetting of the associated words and correct display of the associated words and handwritten text.

[0017] In a possible scenario, the position order typesetting relationship described above may be a downward wrap typesetting relationship.

[0018] In this scenario, the electronic device can detect and respond to the layout of the interface and the layout style of the handwritten text, and implement sequential layout processing such as downward line wrapping of the associated words.

[0019] In a possible scenario, in the second interface, the difference between the character spacing of the handwritten text and the character spacing of the first associative word is less than or equal to a first difference threshold.

[0020] In this scenario, the spacing between the characters of the associated words can refer to the spacing of the handwritten text, so that the text style of the associated words can be similar to the handwritten text. This can provide more functions based on the handwriting input scenario, reduce the interaction time of handwriting input, and improve the efficiency of handwriting input.

[0021] In one possible scenario, the word spacing of the handwritten text in the first interface is the same as or different from the word spacing of the handwritten text in the second interface.

[0022] In this scenario, the electronic device can determine the word spacing of the associative word based on the word spacing of the handwritten text, and can also consider adjusting the word spacing of the handwritten text based on the size of the word spacing. For example, the handwritten text on the first interface is obtained based on the trajectory of the user's handwriting input. In addition, the word spacing of the handwritten text on the second interface can also be adjusted. For example, if the word spacing of the handwritten text is too large, the word spacing of the handwritten text can be reduced. For another example, if the word spacing of the handwritten text is too small or even overlaps, the word spacing of the handwritten text can be increased.

[0023] In one possible scenario, when the character spacing of the handwritten text in the first interface is different from the character spacing of the handwritten text in the second interface; the character spacing of the handwritten text in the first interface is less than the character spacing threshold, and the character spacing of the handwritten text in the second interface is greater than or equal to the character spacing threshold.

[0024] In this scenario, if the electronic device detects that the handwritten text has overlapping characters due to insufficient spacing, it can increase the spacing between the characters, thereby accurately displaying the handwritten text. Furthermore, when the associated words refer to the handwritten text's character spacing, the electronic device can also refer to the adjusted spacing of the handwritten text, thereby accurately displaying the handwritten text and the associated words.

[0025] In a possible scenario, the word spacing is the word spacing in a target direction, where the target direction is determined based on the handwriting direction of the handwritten text.

[0026] In this scenario, the display of associated words can also refer to the writing direction of the handwritten text, for example, horizontal, vertical, or even tilted, which can be determined according to the actual scenario of the user's handwriting input.

[0027] In a possible scenario, the difference between the font size of the handwritten text and the font size of the first associative word is less than or equal to the second difference threshold.

[0028] In this scenario, the size of the associated words can also be displayed based on the size of the handwritten text, so that the associated words can be better displayed with the handwritten style of the handwritten text. Therefore, more functions can be realized in the handwriting input scenario.

[0029] In a second aspect, the present application also provides another interface display method based on handwriting input, which can be applied to electronic devices. The method provided in the second aspect may include: displaying a first interface in response to a user's handwriting input operation. The first interface includes handwritten text and at least one associative word. In response to a confirmation operation on the first associative word, a second interface is displayed. The first associative word is one of at least one associative word. The first associative word in the second interface is displayed in a first display position and a second display position; the first display position displays the first at least one Chinese character included in the first associative word, and the second display position displays the remaining at least one Chinese character included in the first associative word. In addition, the first display position and the second display position have a positional sequence typesetting relationship. The positional sequence typesetting relationship is obtained based on the handwritten text.

[0030] In a possible scenario, the first at least one Chinese character described above reaches the page boundary, or the first at least one Chinese character reaches the display area of ​​the existing writing track.

[0031] In one possible scenario, the position order typesetting relationship described above is a downward wrap typesetting relationship.

[0032] In a possible scenario, in the second interface, the difference between the character spacing of the handwritten text and the character spacing of the first associative word is less than or equal to a first difference threshold.

[0033] In one possible scenario, the word spacing of the handwritten text in the first interface is the same as or different from the word spacing of the handwritten text in the second interface.

[0034] In one possible scenario, when the character spacing of the handwritten text in the first interface is different from the character spacing of the handwritten text in the second interface; the character spacing of the handwritten text in the first interface is less than the character spacing threshold, and the character spacing of the handwritten text in the second interface is greater than or equal to the character spacing threshold.

[0035] In a possible scenario, the word spacing is the word spacing in a target direction, where the target direction is determined based on the handwriting direction of the handwritten text.

[0036] In a possible scenario, the difference between the font size of the handwritten text and the font size of the first associative word is less than or equal to the second difference threshold.

[0037] In a possible scenario, the second interface includes handwritten text and a first associated word, and the handwriting style of the first associated word is the same as the handwriting style of the user's historical handwriting input of the first associated word.

[0038] In a possible scenario, the handwritten text includes at least one of the following types: Chinese characters and pinyin.

[0039] In one possible scenario, in response to a user's handwriting input operation, displaying a first interface includes: obtaining handwritten text in response to the handwriting input operation; the handwritten text includes pinyin; and obtaining at least one associated word in response to an operation to convert the pinyin included in the handwritten text into Chinese characters. The at least one associated word is a Chinese character obtained based on the pinyin.

[0040] In a third aspect, the present application also provides an electronic device, comprising: at least one memory and at least one processor; wherein the at least one memory is used to store computer program code, and the computer program code includes computer instructions; when the computer instructions are executed by at least one processor, the electronic device executes the method in the above-mentioned first aspect and any possible scenario in the first aspect, or executes the method in the above-mentioned second aspect and any possible scenario in the first aspect.

[0041] In a fourth aspect, the present application further provides an interface display device based on handwriting input, the device including modules / units for executing the method described in the first aspect and any possible scenario of the first aspect, or modules / units for executing the method described in the second aspect and any possible scenario of the second aspect. These modules / units may be implemented in hardware, or may be implemented in hardware executing corresponding software.

[0042] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program (also referred to as code, or instructions). When these codes or instructions are run on a computer, the computer executes the method of the above-mentioned first aspect and any possible scenario in the first aspect, or executes the method of the above-mentioned second aspect and any possible scenario in the second aspect.

[0043] In the sixth aspect, a computer program product is provided, which includes: a computer program (also referred to as code, or instructions), which, when run, causes the method of the above-mentioned first aspect and any possible scenario in the first aspect to be executed, or causes the above-mentioned second aspect and any possible scenario in the second aspect to be executed.

[0044] In the seventh aspect, a graphical user interface on an electronic device is also provided, which electronic device has a display screen, one or more memories, and one or more processors, and the one or more processors are used to execute one or more computer programs stored in the one or more memories. The graphical user interface may include the graphical user interface displayed when the electronic device executes the above-mentioned first aspect of the present application and any possible scenario in the first aspect, or may also include the graphical user interface displayed when the electronic device executes the above-mentioned second aspect of the present application and any possible scenario in the second aspect.

[0045] It should be noted that, for the beneficial effects of various scenarios of the electronic device provided in the second to seventh aspects of this application, please refer to the beneficial effects of any possible scenario in the first aspect above, and no further details will be given here. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] FIG1 is a schematic diagram of an interface;

[0047] FIG2 is a schematic diagram of the hardware structure of a possible electronic device provided in an embodiment of the present application;

[0048] FIG3 is a block diagram of a software system architecture of an electronic device provided in an embodiment of the present application;

[0049] FIG4A is a schematic diagram of one of the interfaces of a method for displaying an interface based on handwriting input provided in an embodiment of the present application;

[0050] FIG4B is a second interface diagram of an interface display method based on handwriting input provided in an embodiment of the present application;

[0051] FIG4C is a third interface diagram of an interface display method based on handwriting input provided in an embodiment of the present application;

[0052] FIG5A is a fourth interface diagram of an interface display method based on handwriting input provided in an embodiment of the present application;

[0053] FIG5B is a fifth interface diagram of an interface display method based on handwriting input provided in an embodiment of the present application;

[0054] FIG6A is a sixth interface diagram of an interface display method based on handwriting input provided in an embodiment of the present application;

[0055] FIG6B is a seventh interface diagram of an interface display method based on handwriting input provided in an embodiment of the present application;

[0056] FIG7 is an eighth interface diagram of an interface display method based on handwriting input provided in an embodiment of the present application;

[0057] FIG8 is a ninth interface diagram of a method for displaying an interface based on handwriting input according to an embodiment of the present application;

[0058] FIG9 is a flow chart of a method for displaying an interface based on handwriting input according to an embodiment of the present application;

[0059] FIG10A is a schematic diagram of a handwritten text provided in an embodiment of the present application;

[0060] FIG10B is a schematic diagram of the structure of a method for displaying an interface based on handwriting input according to an embodiment of the present application;

[0061] FIG11 is a schematic diagram of an effect of a segmentation result provided in an embodiment of the present application;

[0062] FIG12 is another schematic diagram of the segmentation result provided in an embodiment of the present application;

[0063] FIG13 is a schematic diagram of the effect of block processing provided in an embodiment of the present application;

[0064] FIG14 is a tenth interface diagram of an interface display method based on handwriting input provided in an embodiment of the present application;

[0065] FIG15 is an eleventh interface diagram of an interface display method based on handwriting input provided in an embodiment of the present application;

[0066] FIG16 is a twelfth interface diagram of an interface display method based on handwriting input provided in an embodiment of the present application;

[0067] FIG17 is another flow chart of a method for displaying an interface based on handwriting input provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0069] The embodiments of the present application can be applied to the field of electronic device technology, and specifically can be applied to handwriting input scenarios based on electronic devices. Exemplarily, the electronic device may include a display screen and a touch sensor. Among them, the touch sensor can be used to detect the user's touch operation on the display screen, and the touch operation may include but is not limited to: the user's handwriting input operation, the user's click operation, long press operation on the control, etc. Optionally, the user can perform handwriting input through a stylus, a finger, or one or a combination of other handwriting tools. Another exemplary embodiment, the electronic device may also include a display screen and other sensors, or a camera, etc. Among them, other sensors or cameras can be used to detect the user's air gesture operations on the display screen, etc. It should be understood that the user operations in the following embodiments may include touch operations, and may also include air gesture operations, etc., and this application does not limit this.

[0070] In one possible scenario, the display interface of an electronic device generally displays handwritten input content in response to a user's touch operation. In other words, the trace of the user's handwriting is the handwritten text displayed on the interface. In this way, handwriting input has the problem of low efficiency. For example, to display the words "human-computer interaction", you can only enter "R, J, J, H, space" on the keyboard to complete the input through keyboard input; while through handwriting input, the user needs to write 18 strokes to complete the input. Therefore, handwriting input is time-consuming in some scenarios, resulting in low efficiency.

[0071] In another possible scenario, in the English handwriting input scenario, taking into account the characteristics of English words, English word completion can be achieved. For example, Figure 1 is a schematic diagram of an interface. As shown in interface 11 in Figure 1, when the user handwrites "I went to the libr", the electronic device can display multiple associative words based on the user's handwriting input of "libr", and the associative words can, for example, at least include "library", "libraries", "librarian", etc. Then, the electronic device can display interface 12 in response to the user's operation on "library" among the multiple associative words. As shown in interface 12 in Figure 1, in the English handwriting input scenario, English word completion can be achieved by displaying associative words to the user, thereby improving the efficiency of handwriting input. However, due to the characteristics of Chinese based on stroke composition, the English word completion method is difficult to apply to the Chinese handwriting input scenario.

[0072] In view of this, embodiments of the present application provide an interface display method and electronic device based on handwriting input. In this method, for handwriting input scenarios, the electronic device can provide a word association function based on the handwritten text input by the user. Furthermore, based on the user's handwriting input style, the electronic device can process one or a combination of the associated word trajectory and the associated word layout, thereby reducing the style difference between the associated word and the handwritten text, and reducing the difference between the associated word layout and the handwritten text layout. Through this method, more functions can be provided for handwriting input scenarios, which can improve the convenience and user experience of handwriting input.

[0073] The technical solutions in the embodiments of the present application can be applied to electronic devices, and the electronic devices can be devices with display screens, and the electronic devices can also have touch sensors or other sensors or cameras, etc. For example, the electronic devices can be electronic devices such as mobile phones, tablet computers, handwriting tablets, wearable devices (e.g., watches, bracelets, etc.), vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptop computers, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), smart home devices (e.g., smart TVs, smart speakers with display screens, etc.), etc. that can receive user operations and display interfaces. It is understandable that the embodiments of the present application do not impose any restrictions on the specific types of electronic devices.

[0074] The electronic devices to which the embodiments of the present application can be applied include, but are not limited to, Or electronic devices with other operating systems. The electronic device may also be other electronic devices, such as a laptop computer with a touch-sensitive surface (eg, a touch panel).

[0075] Figure 2 shows a schematic diagram of the hardware structure of a possible electronic device. As shown in Figure 2, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0076] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors. The controller may serve as the nerve center and command center of the electronic device 100. The controller may generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution. The processor 110 may also include memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a high-speed cache memory. This memory may store instructions or data that have just been used or are being recycled by the processor 110. When the processor 110 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids duplicate accesses, reduces the processor 110's waiting time, and thus improves system efficiency. In an embodiment of the present application, the processor 110 can be used to detect the user's operation instructions, and in response to the user's operation instructions, process the data corresponding to the interface to be displayed, and can also obtain the display data corresponding to the interface and display it with the help of the display screen 194.

[0077] The display screen 194 is used to display the display interface of the application, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1. In an embodiment of the present application, the display screen 194 can be used to display an interface after being processed by the method provided in an embodiment of the present application, such as the interfaces shown in Figures 4A to 8 in the following embodiments.

[0078] The camera 193 is used to capture still images or videos. The camera 193 may include a front camera and a rear camera. In the embodiment of the present application, the camera 193 can be used to detect air gesture operations.

[0079] The internal memory 121 can be used to store computer executable program code, and the executable program code includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, and the software code of at least one application, etc. The data storage area can store data (such as images, videos, etc.) generated during the use of the electronic device 100. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. In an embodiment of the present application, the internal memory 121 may, for example, store program instructions for implementing the interface display method based on handwriting input provided by the present application, the style of the trajectory of the user's historical handwritten text, etc., and related data of preset fonts, etc. The present application does not limit the content stored in the internal memory 121.

[0080] The touch sensor 180K and the display screen 194 form a touch screen, also known as a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor 180K can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194. In an embodiment of the present application, the touch sensor 180K can collect user operations on the handwriting input interface based on the handwriting input interface displayed by the electronic device through the display screen 194. For example, user operations may include but are not limited to: handwriting input, selecting associative words, etc.

[0081] It is understood that the components shown in FIG2 do not constitute a specific limitation on the mobile phone. The mobile phone may include more or fewer components than shown, or combine or separate some components, or arrange the components differently. In the following embodiments, the electronic device 100 shown in FIG2 is used as an example for description.

[0082] The operating system (OS) involved in the embodiments of the present application is the most basic system software running on the electronic device 100. The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiments of the present application use an operating system with a layered architecture as an example to illustrate the software system architecture of the electronic device 100.

[0083] Figure 3 is a block diagram of the software system architecture of an electronic device provided in an embodiment of the present application. As shown in Figure 3, the software system architecture of the electronic device can be a layered architecture. For example, the software can be divided into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the operating system is divided into five layers, from top to bottom, namely, the application layer, the application framework layer (framework, FWK), the runtime and system library, the kernel layer, and the hardware layer.

[0084] The application layer can include a series of application packages. As shown in Figure 3, the application layer can include camera, settings, skin modules, user interface (UI), third-party applications, etc. Among them, third-party applications can include wireless local area network (WLAN), music, calls, Bluetooth, video, memo, notes, etc.

[0085] In one possible implementation, applications can be developed using Java by calling the application programming interface (API) provided by the application framework layer. Developers can use the application framework layer to interact with the underlying operating system layers (e.g., the hardware layer, kernel layer, etc.) to develop their own applications. The application framework layer primarily provides a series of services and management systems for the operating system.

[0086] The application framework layer provides an application programming interface and programming framework for applications in the application layer. The application framework layer includes some predefined functions. As shown in Figure 3, the application framework layer can include an activity manager, a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, and so on.

[0087] The activity manager is used to manage the life cycle of each application and provide common navigation back functions, providing an interactive interface for all program windows.

[0088] The window manager manages windowed applications. It can determine the display size, determine whether a status bar is present, lock the screen, take screenshots, and more. Content providers store and retrieve data and make it accessible to applications. This data can include video, images, audio, incoming and outgoing calls, browsing history and bookmarks, and the phone book.

[0089] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface that includes a text notification icon might include a view for displaying text and a view for displaying images.

[0090] The phone manager is used to provide communication functions for electronic devices, such as call status management (including answering, hanging up, etc.).

[0091] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0092] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0093] The runtime includes the core library and the virtual machine. The runtime is responsible for the scheduling and management of the operating system.

[0094] The core library consists of two parts: one containing the Java language's callable functions and the other containing the operating system's core libraries. The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files from the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0095] The system library can include multiple functional modules, such as surface manager, media framework, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0096] The surface manager is used to manage the display subsystem and provide the fusion of two-dimensional and three-dimensional layers for multiple applications.

[0097] The media framework supports playback and recording of a variety of common audio and video formats, as well as static image files. The media framework can support a variety of audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0098] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0099] A 2D graphics engine is a drawing engine for 2D drawings.

[0100] In some embodiments, a three-dimensional graphics processing library may be used to draw a three-dimensional motion trajectory image, and a two-dimensional graphics engine may be used to draw a two-dimensional motion trajectory image.

[0101] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0102] The hardware layer can include various sensors, such as accelerometers, gravity sensors, touch sensors, etc.

[0103] Typically, the electronic device 100 can run multiple applications simultaneously. In simpler cases, one application corresponds to one process, while in more complex cases, one application corresponds to multiple processes. Each process has a process ID.

[0104] It should be understood that in the embodiments of the present application, "at least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, a and b, a and c, b and c, or a, b and c, where a, b, c can be single or multiple. "Multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship.

[0105] In addition, it should be understood that, in the description of this application, words such as "first" and "second" are only used for the purpose of distinguishing the description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying order.

[0106] It should be understood that the hardware structure of the electronic device can be as shown in Figure 2, and the software system architecture can be as shown in Figure 3, wherein the software programs and / or modules corresponding to the software system architecture in the electronic device can be stored in the internal memory 121, and the processor 110 can run the software programs and applications stored in the internal memory 121 to execute the process of a handwriting input-based interface display method provided in an embodiment of the present application.

[0107] In order to facilitate understanding of the interface display method based on handwriting input provided by the present application, the implementation process of the method provided by the present application is introduced below in combination with the contents shown in Figures 4A to 16.

[0108] The embodiments of the present application are applicable to handwriting input scenarios. For example, handwriting input scenarios include but are not limited to the following applications (application, APP): memo APP, note APP and other note-taking APPs; email APP; instant messaging APP; drawing APP; text processing APP; spreadsheet APP; game APP; video conferencing APP; sports health APP; photo management APP; web APP; etc. When the electronic device executes the above-mentioned APPs, the APP interface can be displayed through the display screen 194, and the handwriting input operation can also be received through the touch sensor 180K. It should be noted that the present application does not limit the handwriting input scenario to APP, but can also be a small program, process or thread, such as a desktop process.

[0109] For ease of understanding, the following explains the technical terms or terminology that may be involved in the embodiments of this application:

[0110] (1) Associative words refer to the association based on the related text, providing possible candidate words, phrases or sentences, etc., which can be the completion of the input text or the conversion of pinyin into Chinese characters.

[0111] (2) Associated texts can be understood as text information related to the associated word, including but not limited to context, language context, pinyin, etc. For example, the associated texts can be selected based on certain rules, such as taking a preset number of input texts forward and / or backward, or taking the previous adjacent punctuation mark, the previous adjacent space, or the previous adjacent paragraph forward or backward, etc. This application does not limit this.

[0112] (3) Handwritten text refers to text information obtained in response to handwritten input. It should be understood that handwritten text has the user's handwriting style. For example, handwritten text may include but is not limited to: Chinese characters, pinyin, English words, punctuation marks, pictures, graphics, tables, etc.

[0113] (4) The user's handwriting style can be reflected in the following information, but is not limited to: font trajectory, font size, text spacing, typesetting format, etc.

[0114] In addition, the same handwriting style can be reflected by, but is not limited to, the following information: the same font trajectory, the same font size, the same text spacing, the same typesetting format, etc.

[0115] Alternatively, the handwriting style can be determined to be identical using a confidence level. When the confidence level of the handwriting styles of two text messages is greater than or equal to a confidence threshold, the two text messages are considered to have the same style. Alternatively, determining the handwriting style to be identical can be achieved using other possible methods, such as a pre-trained judgment model.

[0116] (5) Trajectory refers to the movement trajectory of handwritten text, which can be used to reflect the user's handwriting style. Among them, the forms of handwritten text include image format and vector trajectory format; among them, in the vector trajectory format, the handwritten text is obtained based on vector dot matrix data, which has the characteristic of being easy to edit. For example, vector dot matrix data can be obtained according to a series of operations of "press", "move" and "lift the pen", and a set of "press-move-lift the pen" can correspond to a stroke. It should be understood that vector dot matrix data can be stored at the stroke granularity. On this basis, the editing operation of retracting handwritten text including a stroke can be realized based on the vector dot matrix data.

[0117] In addition, the same trajectory of the fonts can be understood as, for example, the distribution of vector dot matrix data corresponding to the same strokes is the same or similar.

[0118] The following first describes the implementation process of the method provided in the embodiment of the present application through multiple scenarios, so as to facilitate understanding of the interface processing effect that can be achieved based on the method provided in the present application. It can be understood that the embodiment of the present application is not limited to the following application scenarios. In addition, in the embodiment of the present application, the electronic device is a mobile phone as an example, and the following is introduced by taking the interface display on the mobile phone as an example; it should be understood that the interface processing effect of the interface display on other types of electronic devices can be referred to the content of the following embodiments, and this application will not be described in detail.

[0119] Scene A1 and Figure 4A are an interface diagram of an interface display method based on handwriting input provided in an embodiment of the present application. In the interface 400A of Figure 4A, it may include at least: the handwritten text of "Romeo and" input by the user, and a pop-up window 401; wherein, the pop-up window 401 may include at least one associative word, such as "Juliet", "Zhu", "Julie", etc. It should be noted that the embodiment of the present application does not limit the display form of the associative words, for example, it can be in the form of a pop-up window, or in the form of a card, interface, etc.; nor does it limit the specific associative words obtained and the arrangement order of the associative words, and the following embodiments are also not limited, and will not be repeated in the following text. It can be understood that the associative words included in the pop-up window 401 can be obtained by association based on the associated text; that is, they are obtained by association based on the user's handwriting input of "Romeo and".

[0120] Optionally, in interface 400A, the electronic device displays interface 400B in response to an operation on the associated word "Juliet" included in pop-up window 401. Interface 400B includes not only the handwritten text of "Romeo and Juliet" input by the user, but also the displayed text of "Juliet" 402. Furthermore, the trajectory of "Juliet" 402 in interface 400B and the trajectory of the handwritten text have the same or similar handwriting style.

[0121] Based on the content described in Scenario A1 above, in the Chinese handwriting input scenario, it is possible to display associated words based on the associated text. Furthermore, when the user selects an associated word, the associated word trajectory can be displayed in the user's handwriting style. In other words, the associated word trajectory has the same or similar handwriting style as the handwritten text trajectory. This can reduce the difference in trajectory between the handwritten text and the associated word trajectory.

[0122] Scenario A2, Figure 4B, illustrates another interface diagram of a handwriting input-based interface display method provided in an embodiment of the present application. Interface 400C in Figure 4B may include at least the user's handwritten text of "Romeo and Juliet" (lì yè), where "lì yè" is in pinyin.

[0123] Optionally, in interface 400C, "lì yè" may further include a wavy line 403, which can be used to trigger the conversion of pinyin to Chinese. Exemplarily, in response to an operation on the wavy line 403, the electronic device displays interface 400D; wherein, interface 400D may at least include a pop-up window 404, and the pop-up window 404 may include at least one associated word, such as "Lìyè", etc. It should be noted that the embodiments of the present application do not limit the trigger entry for providing pinyin-to-Chinese conversion through the wavy line 403. For example, the trigger entry may also be in the form of a pop-up window, a control, etc. The same is not limited in the following embodiments and will not be repeated hereinafter. It can be understood that the associated words included in the pop-up window 404 can be obtained by association based on the associated text; that is, they can be obtained by association based on the pinyin "lì yè", or can be obtained based on the pinyin "lì yè" and the handwritten text "Romeo and Zhū Lìyè".

[0124] In interface 400D, in response to an operation on the associated word "Lìyè" included in the pop-up window 404, the electronic device displays interface 400E. Among them, in interface 400E, it not only includes the handwritten text of "Romeo and Zhū" input by the user, but also includes the displayed text of "Lìyè" 405. Moreover, the trajectory of "Lìyè" 405 in interface 400E has the same or similar handwriting style as the trajectory of the handwritten text.

[0125] Based on the content introduced in the above scenario A2, in the Chinese handwritten input scenario, it is possible to display the Chinese corresponding to the pinyin based on the associated text and the pinyin included in the associated text; and when the user selects an associated word, it is also possible to display the trajectory of the associated word with the user's handwriting style. In this way, the trajectory difference between the handwritten text and the associated word can be reduced.

[0126] Scenario A3, FIG. 4C is another interface schematic diagram of a method for displaying an interface based on handwritten input provided by an embodiment of the present application. In the interface 400F of FIG. 4C, it may at least include: the handwritten text of "My favorite drama" and "luó mì ōu yǔ zhū lì yè" input by the user; wherein, "luó mì ōu yǔ zhū lì yè" is in pinyin form.

[0127] In some other examples, the electronic device can also recognize the handwritten text of "My favorite drama" and "luo mi ou yu zhu li ye" input by the user; wherein, "luo mi ou yu zhu li ye" can also be pinyin without tones.

[0128] It should be noted that the embodiments of the present application do not limit the form of pinyin included in the handwritten text. Considering the randomness of the user's handwriting input, the handwritten text can be in a standard pinyin form or a form that is highly similar to the standard pinyin. For example, the pinyin included in the handwritten text can be completely without tones, or partially with tones, or all or part of the pinyin can include imprecise tones, etc. The following embodiments are described using pinyin including tones as an example.

[0129] Optionally, in interface 400F, "luó mì ōu yǔ zhū lì yè" may further include a wavy line 406, which may be used to trigger the conversion of pinyin into Chinese. Exemplarily, in response to manipulation of wavy line 406, the electronic device displays interface 400G; interface 400G may include at least a pop-up window 407, which may include at least one associated word, such as "Romeo and Juliet."

[0130] In interface 400G, the electronic device displays interface 400H in response to an operation on the associated word "Romeo and Juliet" included in pop-up window 407. Interface 400H includes not only the handwritten text "My Favorite Drama" input by the user, but also the displayed text "Romeo and Juliet" 408. Furthermore, the trajectory of "Romeo and Juliet" 408 in interface 400H and the trajectory of the handwritten text have the same or similar handwriting style.

[0131] Alternatively, interfaces 400F and 400H may not include the user's handwritten input of "My Favorite Play," that is, the interfaces do not include Chinese handwritten text. In this scenario, the trajectory of "Romeo and Juliet" 408 in interface 400H may have the same or similar handwriting style as the trajectory of the user's historical handwritten text; wherein the handwriting style of the user's historical handwritten text may be obtained based on the user's historical handwritten text, for example, the same or related historical handwritten text may be obtained from space or time. For example, spatially related handwritten text may be the handwritten text of "My Favorite Play" in interface 400F as shown in FIG4C ; temporally identical or related handwritten text may be stored user historical handwritten text, such as the user's historical handwritten input of "Romeo and Juliet."

[0132] Based on the content introduced in the above scenario A3, in the Chinese handwriting input scenario, when there is no associated text or the associated text cannot obtain the user's handwriting style, the handwriting style based on the historical handwriting text can be implemented to display the Chinese corresponding to the pinyin; and the converted Chinese has the user's handwriting style.

[0133] Due to the randomness of handwritten input, associative word typesetting is quite difficult. The following scenarios B1 to B4 introduce several possible typesetting scenarios. It should be understood that the following scenarios can be combined, and can also be combined with the above scenarios A1 to A3. The specific implementation process will not be repeated here.

[0134] Scene B1, Figure 5A is an interface diagram of an interface display method based on handwriting input provided in an embodiment of the present application. In the interface 500A of Figure 5A, the handwritten texts of "I. Typesetting Method 1" and "Handwriting Input Typesetting 1 Hand" input by the user are displayed, and a pop-up window 501a is displayed; wherein, the pop-up window 501a may include at least one associative word, such as "Writing Input Typesetting" shown in the interface 500A, and for example, although not shown in the interface 500A, the associative word can also be "write", "machine", etc. It can be understood that the associative word included in the pop-up window 501a can be obtained by association based on the associated text; that is, it can be obtained by association based on the user's handwriting input of "Handwriting Input Typesetting 1 Hand" or "Hand".

[0135] Optionally, in interface 500A, the electronic device displays interface 500B in response to an operation on the associated word "write input typesetting" included in pop-up window 501a. Similar to scenario A1, the trajectory of the associated word "write input typesetting" has the same or similar handwriting style as the trajectory of the handwritten text.

[0136] In addition, when displaying the associated words, the electronic device can process not only the handwriting style of the trajectory of the associated words, but also the handwriting style of the layout of the associated words.

[0137] For example, in interface 500B, the associated word "write input layout" includes five Chinese characters, and the electronic device detects the first layout scenario. The first layout scenario can be understood as the layout reaching the page boundary after displaying the associated word "write." Then, in response to the first layout scenario, the electronic device wraps the remaining "input layout" 501b in the associated word downward for layout.

[0138] In some examples, in interface 500B, the electronic device may continue to respond to the handwriting input operation, which may be displayed as interface 500C. Compared to interface 500B, interface 500C may also include "2 handwriting input" for further handwriting input by the user, and pop-up window 501c. Among them, pop-up window 501c may include at least one associative word, such as "typesetting 3" in interface 500C. It is understandable that the associative word included in pop-up window 501c may be obtained by association based on the associated text; that is, it may be obtained by association based on the user's handwriting input of "handwriting input typesetting 1 handwriting input typesetting 2 handwriting input" or "handwriting input".

[0139] In interface 500C, in response to an operation on the associated word "typesetting 3" included in the pop-up window 501c, the electronic device displays interface 500D. Interface 500D is similar to interface 500B. The associated word "typesetting 3" consists of three Chinese characters. When the character "pai" in the associated word is displayed, the typesetting has reached the page boundary; then, the remaining "ban 3" 501d in the associated word is typeset with a line break downward.

[0140] As shown by interfaces 500A to 500D, when performing typesetting with a line break downward, it can be typeset in a paragraph format with a first-line indent. It should be understood that the embodiments of the present application do not limit the paragraph format for typesetting with a line break downward. For example, as shown by interfaces 500E, 500F, 500G, and 500H included in FIG. 5B, when performing typesetting with a line break downward, it can also be set to a paragraph format with each line aligned. As shown in interface 500E, text content and a pop-up window are displayed, and the pop-up window includes at least the associated word "writing input typesetting". As shown in interface 500F, the character "xie" in the associated word "writing input typesetting" can be displayed after the last "shou" in interface 500E, and the remaining "input typesetting" 502a in the associated word can be typeset with a line break downward. As shown in interface 500G, text content and a pop-up window are displayed, and the pop-up window includes at least the associated word "input typesetting 3". As shown in interface 500H, the character "input" in the associated word "input typesetting 3" can be displayed after the last "xie" in interface 500G, and the remaining "typesetting 3" 502b in the associated word "input typesetting 3" can be typeset with a line break downward. The specific implementation process can refer to the introduction of interfaces 500A to 500D in the previous text and will not be elaborated here.

[0141] Scenario B2, FIG. 6A is another interface schematic diagram of a method for displaying an interface based on handwritten input provided by an embodiment of the present application. In interface 600A of FIG. 6A, handwritten texts of "III. Handwritten Content 1", "1. Content 1", "2. Content 2", and "3. Content 3" input by the user are displayed. Considering the randomness of the handwritten input scenario, in interface 600A, the electronic device can detect the handwritten input operation of the user at any display position. As shown in interface 600B, compared with interface 600A, the handwritten text of "handwritten input" 601 newly input by the user is displayed on the left side of the interface.

[0142] In interface 600B, in response to the handwritten input operation, the electronic device can display interface 600C; wherein, interface 600C includes a pop-up window 602, and the pop-up window 602 can include at least one associated word, such as the "typesetting" shown in interface 600C. It can be understood that the associated word included in the pop-up window 602 can be obtained by association based on the associated text; that is, it is obtained by association based on the user's handwritten input of "handwritten input" or "input", etc.

[0143] Optionally, in interface 600C, the electronic device displays interface 600D in response to an operation on the associated word "typesetting" included in the pop-up window 602. Similar to scenario A1, the trajectory of the associated word "typesetting" has the same or similar handwriting style as the trajectory of the handwritten text.

[0144] In addition, when displaying the associated word, the electronic device can not only process the handwriting style of the trajectory of the associated word, but also process the handwriting style of the typesetting of the associated word.

[0145] Exemplarily, in interface 600C, the associated word "typesetting" includes two Chinese characters, and the electronic device detects a second typesetting scenario. Among them, the second typesetting scenario can be understood as that after displaying "pai" in the associated word, although the typesetting has not reached the page boundary, there is already a writing trajectory at the planned typesetting position. Then, in response to the second typesetting scenario, the electronic device typesets the remaining "ban" 603 in the associated word to the next line downward and displays it as interface 600D. In this way, when there is already a writing trajectory at the planned typesetting position, by typesetting to the next line downward, the existing writing trajectory can be avoided from being overwritten.

[0146] As shown by interfaces 600A to 600D, when performing typesetting to the next line downward, it can be typeset according to the paragraph format set to be aligned in each line. It should be understood that the embodiment of the present application does not limit the paragraph format of typesetting to the next line downward. For example, as shown by interfaces 600E, 600F, 600G, and 600H included in FIG. 6B, when performing typesetting to the next line downward, the paragraph format can also be set according to the text content. As shown in interface 600F, compared with interface 600E, there is an additional handwritten text of "Content: Handwritten" 604 input by the user. As shown in interface 600G, based on the handwritten text of "Content: Handwritten" 604, the associated word obtained by association is "Input Typesetting" 605. As shown in interface 600H, when the associated word "Input Typesetting" 606 is typeset to the next line downward, it is typeset according to the paragraph format set with a hanging indent, that is, the typesetting to the next line downward can be performed by referring to the display position of ":".

[0147] Scenario B3: In a scenario where an associated word function is provided during handwritten input, although the associated word is displayed after being processed by the electronic device in terms of trajectory, typesetting, etc., there may still be problems such as too large or too small character spacing or even overlap. Therefore, in the embodiment of the present application, the character spacing can also be adjusted.

[0148] For example, FIG. 7 is another interface schematic diagram of a method for displaying an interface based on handwritten input provided by an embodiment of the present application. Interface 701 is included in FIG. 7, and what is shown in interface 701 is the user's handwritten input of "like having a fishbone stuck in one's throat". Among them, "gěng" is in pinyin form.

[0149] Optionally, referring to the descriptions of Scenario A2 and Scenario A3 in the foregoing text, it is possible to convert the pinyin "gěng" into the Chinese character "鲠", which can be displayed on Interface 702. However, in the handwritten input scenario, "gěng" occupies a larger display width than "鲠". Therefore, as shown in Interface 702, there is a problem of excessive character spacing between "鲠" and the previous character "如" and the subsequent character "在".

[0150] Another option is that in response to the operation of converting pinyin to Chinese characters, the electronic device can also be displayed as Interface 703. Among them, compared with Interface 702, Interface 703 can not only convert pinyin to Chinese characters, but also adjust the character spacing. As can be seen from Interface 703, compared with Interface 702, the character spacing between "鲠" and the previous character "如" and the subsequent character "在" is adjusted and the character spacing is reduced.

[0151] Scenario B4, Figure 8 is another interface schematic diagram of a method for displaying an interface based on handwritten input provided by an embodiment of the present application. Figure 8 includes Interface 800A, and the handwritten text of "Before my bed a pool of light" handwritten by the user can be shown in Interface 800A. Based on the content introduced in the foregoing scenarios, the electronic device can display a pop-up window including associated words, and display the associated words in response to the selection operation of the associated words; among them, the associated words can have the same or similar trajectory as the handwritten text, and can also be typeset.

[0152] During the typesetting process, the typesetting characteristics of the handwritten text can also be referred to. For example, based on the handwritten text shown in Interface 800A having an inclined typesetting characteristic, the associated words "I wonder if it's frost aground. Looking up, I find the moon bright; Bowing, in homesickness I'm drowned." shown in Interface 800B can have an inclined typesetting characteristic while being typeset downward to wrap lines.

[0153] Through the content introduced in Scenario B4, when the electronic device typesets the associated words, it can combine the handwritten style typeset by the user to achieve the typesetting of the associated words.

[0154] Based on the content introduced in the foregoing scenarios, the following introduces the implementation process of a method for displaying an interface based on handwritten input provided by the present application, so as to illustrate how to use the method provided by the present application to achieve the interface processing effects shown in Figures 4A to 8 described above, thereby providing a technical solution for how to display associated words in the handwritten input scenario.

[0155] Referring to Figure 9, it is a schematic flowchart of a method for displaying an interface based on handwritten input provided by an embodiment of the present application. This method can be applied to an electronic device and can include the following processes:

[0156] Step 901, in response to the user's handwritten input operation, obtain the handwritten text handwritten by the user.

[0157] It should be noted that, in addition to the user's handwritten input operation, it may also include multiple other operations for obtaining handwritten text. Other operations such as, but not limited to, click operation, double-click operation, long-press operation, etc.; in addition, the operation objects of other operations may include, but not limited to: controls such as undo, interfaces, auxiliary tools included in the interface, etc.

[0158] In the embodiments of the present application, the handwritten text may include, but not limited to: Chinese characters, pinyin, English words, punctuation marks, pictures, graphics, tables, etc.

[0159] Step 902: Based on the handwritten text, display at least one associated word; determine the associated word selected by the user.

[0160] Based on the content introduced in Scenarios A1 to A3, the associated word can be obtained based on the context, context, pinyin, etc. of the handwritten text; or, it can also be obtained based on other methods, for example, it can be based on historical input data, machine learning methods, etc. In the embodiments of the present application, the implementation manner of obtaining at least one associated word is not limited. And, the embodiments of the present application do not limit the display manner of at least one associated word. For example, it can be displayed in the form of a pop-up window, a card, or a two-layer interface, etc. Also, for example, the pop-up window can include a specified number of associated words of at least one associated word, and can be displayed in a horizontal or vertical manner, etc.

[0161] Among them, obtaining an associated word based on the pinyin included in the handwritten text can include multiple scenarios: all-pinyin input scenario, pinyin and Chinese character mixed input scenario, pinyin and English mixed input scenario. The following introduces several possible ways to obtain associated words:

[0162] Method A: Based on the pinyin included in the handwritten text, obtain an associated word, which may include the following processing:

[0163] 1) Determine the object to be recognized.

[0164] Exemplarily, the object to be recognized can be: handwritten text including pinyin, or a block obtained based on the handwritten text including pinyin. For example, FIG. 10A is a schematic diagram of a handwritten text provided by an embodiment of the present application. In the interface 1000A of FIG. 10A, the handwritten text or block can be "yǎn rán a visual and auditory tāo tiè feast"; in the interface 1000B of FIG. 10A, the handwritten text or block can be "women shì woman de fù shù xíng shì". It should be understood that when the object to be recognized corresponds to a block obtained based on the handwritten text including pinyin, the handwritten text may include more text content than the block.

[0165] Another exemplary case is that the object to be recognized can be obtained by extracting based on handwritten text including pinyin, or by extracting based on the chunks obtained from handwritten text including pinyin. For example, the object to be recognized can be obtained by taking pinyin / English as the basis and selecting n Chinese characters forward and / or backward; where n can be 1, 2, 3, etc. Taking n as 3 as an example, in combination with FIG. 10A, for the "yǎn rán a visual and auditory tāo tiè feast" shown in the interface 1000A, including the pinyin "yǎn", the corresponding object to be recognized extracted can be "yǎn rán a"; the interface 1000A also includes "tāo tiè", and the corresponding objects to be recognized extracted can be "auditory tāo tiè", "tāo tiè feast", or "auditory tāo tiè feast".

[0166] In the above example, when reaching the boundary of the handwritten text forward and / or backward based on the handwritten text, and the number of Chinese characters that can be selected is less than n, then all the text forward and / or backward in the handwritten text can be selected. For example, taking n as 3 as an example, in combination with FIG. 10A, for the "women shì woman de fù shù xíng shì" shown in the interface 1000B, the corresponding object to be recognized extracted can be "women shì woman de fù shù xíng shì".

[0167] 2). Based on the object to be recognized, perform pinyin association. Exemplarily, pinyin association can be implemented based on at least one of semantic analysis of context, pinyin-to-Chinese conversion, neural network models, deep learning, etc.

[0168] In an optional embodiment, pinyin association can be performed based on the hidden Markov model (HMM). The following processing can be included:

[0169] 2-1). Perform word segmentation on the object to be recognized. Exemplarily, word segmentation can be performed based on symbols such as spaces, possible combinations of Chinese character pinyins, or possible combinations of English words, etc.

[0170] For example, when a space is recognized in "tāo tiè" from the handwritten text, it can be divided into "tāo" and "tiè" based on the space.

[0171] For another example, when no space is recognized in "tāotiè" in the handwritten text, it can also be divided into "tāo" and "tiè" based on possible combinations of Chinese character pinyins, or divided into "tā", "o" and "tiè".

[0172] Alternatively, for example, for the word "women" in handwritten text, it can be segmented into "women" based on possible English words, or it can be segmented into "wo" and "men" based on possible combinations of Chinese pinyin.

[0173] 2-2), Input the segmented words obtained after the word segmentation operation into the HMM model to obtain the scoring results after converting each segmented word into Chinese. In this way, based on the scoring results, the display order of the associated words can be determined. For example, the associated word with the highest scoring result can be displayed first, and as the scoring results decrease, the associated words can be sorted in sequence.

[0174] Method B, a method based on deep learning, to obtain the associated words corresponding to the pinyin. Among them, the deep learning model is pre-trained based on the training corpus.

[0175] Through pinyin association processing, it can be realized that the handwritten input by the user "yǎn然一场视觉与听觉的tāo tiè盛宴" is converted into "俨然一场视觉与听觉的饕餮盛宴" with the user's handwriting style; it can be realized that the user handwrites "women shì woman de fù shù xíng shì" and is converted into "women是woman的复数形式" with the user's handwriting style. In this way, through pinyin association processing, in the handwritten input scenario, for the scenario of forgetting the word when picking up the pen, it can be realized that the pinyin is converted into Chinese characters.

[0176] Step 903, Generate the trajectory of the associated word.

[0177] In a possible embodiment, when there is associated handwritten text, the trajectory of the associated word can be generated based on the handwriting style of the trajectory of the handwritten text. Among them, the handwriting style of the trajectory of the associated word can include, but is not limited to, the following features:

[0178] (1) Content feature. Exemplarily, based on the associated word selected by the user, search for the content feature of the associated word from the standard library trajectory. Among them, the standard library trajectory can be understood as a dictionary storing {font: font standard library trajectory}, which is used to indicate the mapping relationship between the font and its corresponding font standard library trajectory, and the content feature of the font can be indicated through the font standard library trajectory.

[0179] For example, the handwritten text is "yǎn然", and the associated word corresponding to "yǎn" selected by the user is "俨". The content feature of "俨" can be obtained from the standard library trajectory of "俨" stored in the dictionary.

[0180] (2) Style feature. Exemplarily, the content feature may not have the user's handwriting style. Through the style feature, it can be realized that the trajectory of the associated word has the same or similar style as the trajectory of the handwritten text.

[0181] Optionally, when the electronic device can obtain the user's handwritten text, the style feature can be obtained based on the user's handwritten text. Combining with the example in FIG. 4A, in the semantic association scenario, the style feature of the associated word "Juliet" can be obtained based on the style feature of the handwritten text "Romeo and". Combining with the example in FIG. 4B, in the pinyin association scenario, the style feature of the associated word "Liye" can be obtained based on the style feature of the handwritten text "Romeo and Zhu".

[0182] Another option is that when the electronic device cannot obtain the user's handwritten text, the style feature can be obtained based on the historical handwritten text. The historical handwritten text can include handwritten texts that are spatially related; for example, handwritten texts that are spatially related can be texts that are semantically unrelated but in the same interface. The historical handwritten text can also include historical texts that are temporally related; for example, handwritten texts that are temporally related can be texts that are not in the same interface but belong to the historical handwritten input in the APP. Combining with the example in FIG. 4C, when the handwritten text "My favorite drama" is not recognized as the associated text of the associated word "Romeo and Juliet", it can also be understood that the two do not belong to the same sentence; at this time, the handwritten text "My favorite drama" can be used as the handwritten text that is spatially related, so that the style feature of the associated word "Romeo and Juliet" can be obtained.

[0183] Based on the obtained content feature and style feature, they can be input into the diffusion model to generate the trajectory of the associated word. Through the content feature and style feature, the trajectory of the associated word can be made to have the user's handwriting style.

[0184] (3) Stroke feature. For example, if the associated word is "Yan", the stroke feature of "Yan" can be extracted. Through the stroke feature, the accuracy of generating the trajectory of the associated word can be improved, and the difference degree between the trajectory style of the associated word and the trajectory style of the user's handwritten text can be reduced.

[0185] Based on the obtained content features, style features and stroke features, they can also be input into the diffusion model to generate the trajectory of the associative word. For example, Figure 10B is a structural diagram of an interface display method based on handwriting input provided in an embodiment of the present application. The structure may include a content encoder 1001, a stroke encoder 1002, a style encoder 1003 and a diffusion model 1004. Among them, the content encoder 1001 can be used to extract the content features in the standard library trajectory corresponding to the associative word, the stroke encoder 1002 can be used to extract the stroke features corresponding to the associative word, the style encoder 1003 can be used to extract the style features of the user's handwritten text, and the splicing unit 1004 can be used to receive the splicing features obtained based on the content features, stroke features and style features, and can also be used to output the associative word trajectory with the user's handwriting style; wherein, the content features, stroke features and style features can be spliced ​​in the feature dimension to obtain the splicing features.

[0186] For example, the electronic device generates trajectories of the associative word "Yan" and the associative word "Taotie", which may be shown in the interface 1000C in FIG10A .

[0187] In addition, when the electronic device cannot obtain the user's handwritten text or the user's historical handwritten text, and thus cannot derive the style of the associated word based on the accurate user's handwriting style, the electronic device can select a preset font with the highest confidence level of similarity to the user's handwriting style from the preset fonts to generate the associated word trajectory. Optionally, in this scenario, the user's handwriting style can be derived based on pinyin, and the similarity between the preset font style and the user's handwriting style can be the similarity between the preset font style and the pinyin handwriting style.

[0188] Step 904: typeset and display the associated words.

[0189] Based on the contents introduced in scenes B1 to B4, in order to achieve accurate display of the associated words, it is necessary to perform typesetting processing on the associated words. In an optional embodiment, the typesetting processing may include the following processing steps:

[0190] 1) Segment the handwritten text to obtain a segmentation result. For example, the handwritten text may be in a vector trajectory format and is obtained based on vector dot matrix data. Therefore, the vector dot matrix data corresponding to the handwritten text may be used as input for segmentation processing. Optionally, the vector dot matrix data may be represented by two-dimensional coordinates, such as {x, y}.

[0191] In an optional embodiment, the electronic device can segment the handwritten text into formats such as Chinese characters, pinyin, English words, graphics, and tables. As shown in interface 800A of FIG8 , the handwritten text is "the moonlight shines brightly in front of the bed." After segmentation, the segmentation results of "bed," "front," "bright," "moon," "light," and "," can be obtained.

[0192] Exemplarily, the segmentation results can be used to display associative words on an electronic device, such as determining the character spacing, font size, font trajectory style, typesetting, etc. of the associative words. The specific implementation process is introduced in the following embodiments and will not be described in detail here.

[0193] In another exemplary embodiment, the segmentation result can also be used by the electronic device to adjust the layout of the handwritten text, for example, to adjust the spacing between words in the handwritten text, to align the handwritten text, etc.

[0194] For example, FIG11 is a schematic diagram of the effect of the segmentation result provided by an embodiment of the present application. In FIG11, each Chinese character included in the handwritten text may include at least the bounding box of the Chinese character after the segmentation process. The bounding box may be obtained based on the vector dot matrix data set corresponding to the Chinese character. Optionally, the four borders of the bounding box may be obtained based on the minimum value of X (Xmin), the maximum value of X (Xmax), the minimum value of Y (Ymin) and the maximum value of Y (Ymax) in the vector dot matrix data set corresponding to the Chinese character. For example, the bounding box may be expressed as [Xmin, Ymin, W (Xmax—Xmin) , H (Ymax—Ymin) ]; among them, W (Xmax—Xmin) Indicates the width obtained based on the difference between the maximum value of X and the minimum value of X, H (Ymax—Ymin) Indicates the height obtained based on the difference between the maximum value of Y and the minimum value of Y. For another example, the bounding box can also be expressed as [(Xmin, Ymin), (Xmax, Ymin), (Xmax, Ymax), (Xmax, Ymax)]; where (Xmin, Ymin), (Xmax, Ymin), (Xmax, Ymax), (Xmax, Ymax) can represent the coordinates of the four vertices of the bounding box of the Chinese character.

[0195] For another example, Figure 12 is another schematic diagram of the segmentation results provided by an embodiment of the present application. In Figure 12, each Chinese character included in the handwritten text also includes at least the bounding box of the Chinese character after the segmentation process.

[0196] In another alternative embodiment, the electronic device may also segment the handwritten text into combinations such as words, idioms, sentences, etc. Referring to the interface 701 included in FIG. 7, the handwritten text is "like a lump in the throat". After segmentation, not only can the segmentation results of "like", "lump", "in", and "throat" be obtained, but also the segmentation result of "in the throat" can be obtained, etc.

[0197] In yet another alternative embodiment, the electronic device may also segment the handwritten text at the granularity of lines.

[0198] For example, one segmentation result is still as shown in FIG. 11. FIG. 11 may further include line frames 1101, line frames 1102a, line frames 1102b, line frames 1102c, line frames 1102d, line frames 1103a, and line frames 1103b; among them, a line frame can be understood as a rectangular frame corresponding to the handwritten text divided into one line.

[0199] For another example, another segmentation result is as shown in the interface 1200A included in FIG. 12. The interface 1200A may further include line frames 1201 and line frames 1202. Or for example, the segmentation result may also be as shown in the interface 1200B of FIG. 12. The interface 1200B may further include line frames 1201a, line frames 1201b, line frames 1202a, and line frames 1202b. As can be seen from FIG. 12, compared with the interface 1200A, considering the problem of too large character spacing in the line frame 1201 in the interface 1200A, a further segmentation operation can be performed on the line frame 1201, which can be split into line frames 1201a and line frames 1201b as shown in the interface 1200B; similarly, a further segmentation operation can be performed on the line frame 1202, which can be split into line frames 1202a and line frames 1202b as shown in the interface 1200B. Among them, a line frame can be obtained based on the set of vector dot matrix data corresponding to multiple Chinese characters included in the corresponding line. Optionally, the four borders of the line frame can be obtained based on the minimum value of X, the maximum value of X, the minimum value of Y, and the maximum value of Y in the set of vector dot matrix data corresponding to multiple Chinese characters. Another alternative is that in the foregoing embodiment, the bounding box of a Chinese character can be obtained, and the line frame can also be obtained based on multiple bounding boxes corresponding to multiple Chinese characters one by one.

[0200] 2) Based on the segmentation result, perform block processing.

[0201] Exemplarily, the electronic device may perform block processing based on the segmentation result at the line granularity to obtain one or more text blocks. Through block processing, during the typesetting process, the multi-line text content belonging to the same text block can also be processed by referring to the division situation of the text blocks. For example, the next line can be processed by referring to the character spacing, line spacing, alignment method, etc. of the previous line, so as to reduce the typesetting differences in the interface.

[0202] Optionally, the electronic device may perform block processing on the segmentation result based on preset rules. For example, the preset rules may include but are not limited to the following rules:

[0203] Rule A: Rules based on line box position, length, and width. For example, the spacing between line boxes must be less than or equal to a preset threshold, the average spacing between two line boxes, including the character boxes, must be less than or equal to a preset threshold, and the text alignment between two line boxes must be less than or equal to a preset threshold.

[0204] Rule B: semantic information: For example, the semantic correlation between the text information included in the two line boxes is greater than a preset threshold.

[0205] It should be noted that the preset thresholds corresponding to different rules can be set to the same value or different values.

[0206] Alternatively, the electronic device may also use a machine learning algorithm to cluster the line boxes, and treat the line boxes belonging to the same cluster as a text block. In the clustering process, semantic information may also be combined for analysis.

[0207] Alternatively, the electronic device may use a deep learning method to classify the line boxes and merge the line boxes of the same classification into a text block. In the classification process, semantic information may also be combined for analysis.

[0208] For example, Figure 13 is a schematic diagram of the effect of block processing provided by an embodiment of the present application. As shown in Figure 13, it can include block 1301 and block 1302; wherein, block 1301 is used to indicate that the title of the ancient poem is "Jiang Jin Jiu", block 1302 is the verse content of the ancient poem, and block 1303 is a note on the ancient poem. Combining Figures 11 and 13, it can be seen that block 1302 can be a text block obtained based on line box 1102a, line box 1102b, line box 1102c, and line box 1102d, and block 1303 can be a text block obtained based on line box 1103a and line box 1103b.

[0209] By dividing the handwritten text and associated words displayed on the interface into text blocks, it is possible to uniformly process text content with the same or similar semantics, thereby reducing differences. For example, by treating the verses of ancient poetry as a single text block, the verses can be uniformly formatted, thereby reducing differences between text content belonging to the same verse.

[0210] 3) Layout according to the block to which the associative words belong.

[0211] The typesetting process can include the following processes:

[0212] 3-1). Based on the bounding boxes of the Chinese characters included in the segmentation result, obtain the aspect ratio relationship of the Chinese characters in the handwritten text; based on the aspect ratio relationships corresponding to multiple Chinese characters, search for the target font from the preset fonts. Here, the target font can be understood as the font whose style similarity with the handwritten text meets the preset conditions, and the preset conditions are, for example, the highest similarity or the similarity is greater than the preset threshold.

[0213] Exemplarily, the error between the handwritten text and each preset font can be determined through the following formula 1, and then the preset font whose error meets the preset conditions can be used as the target font, as follows:

[0214] Among them, loss(x,y) represents the error between the preset font and the handwritten font. The smaller the error, the closer the style of the preset font is to the style of the handwritten font. a1, a2, and a3 are coefficients, and the sum of a1, a2, and a3 can be a preset value, for example, the preset value is 1. n is the number of preset fonts. It is used to represent the error between the aspect ratios of the Chinese characters between the preset font and the handwritten font. It is used to represent the error between the widths of the Chinese characters between the preset font and the handwritten font. It is used to represent the error between the heights of the Chinese characters between the preset font and the handwritten font.

[0215] 3-2). According to the aspect ratio relationship of the target font, determine the body box and character spacing information of the Chinese characters in the handwritten text.

[0216] Exemplarily, the body box can be obtained by means such as average or weighted average; similarly, the character spacing information can also be obtained by means such as average or weighted average. For example, FIG. 14 is an interface schematic diagram of a method for interface display based on handwritten input provided by an embodiment of the present application. As shown in the interface 1401 in FIG. 14, the handwritten text can be "human-machine". Obtain the body box 1 corresponding to "human" and the body box 2 corresponding to "machine" in the target font, and then the body box of the Chinese characters in the handwritten text can be obtained according to the average value or weighted average value of the body box 1 and the body box 2; that is, the solid line box shown in the interface 1401 is obtained. Among them, the dashed line box in the interface 1401 is the bounding box of the Chinese character, and the solid line box is the body box of the Chinese character. And, based on the body boxes or bounding boxes of "human" and "machine" in the interface 1401, the character spacing information can be further obtained.

[0217] Another exemplary case is that when the aspect ratio of the character body frame of a Chinese character in the handwritten font is significantly different from that of the character body frame of the same Chinese character in the target font, the character body frame of this Chinese character may not be used to calculate the character body frame of the handwritten text. Taking the character "一" as an example, the aspect ratio corresponding to the handwritten "一" input by the user can be expressed as 20 / 0.01 = 2000, while the aspect ratio corresponding to the "一" in the target font can be expressed as 150 / 17.5 ≈ 8.57. By comparing these two values, it can be seen that the aspect ratio gap between them is relatively large, which may lead to a large deviation in the calculated character body frame. One option is to exclude the character "一" from the calculation of the character body frame of the handwritten text, thereby ensuring the accuracy of the determined character body frame of the handwritten font. Another option is to convert the height of the Chinese character and then calculate the character body frame of the handwritten text. For example, according to the aspect ratio of 8.57 corresponding to "一" in the target font and the length of "一" in the handwritten font, which is 20, the converted height of "一" in the handwritten font can be obtained as 2.33. Another option is to set a minimum threshold for the height of the Chinese character. For example, the minimum threshold can be set to 1 or 2, etc., which can also ensure the accuracy of the determined character body frame.

[0218] 3-3), Based on the character body frame of the handwritten text and the character spacing information, typeset the associative words and display the associative words.

[0219] Exemplarily, based on the processing in step 903 of the foregoing embodiment, the trajectory of the associative words can be obtained.

[0220] Through the obtained character body frame, the size of the associative words can be determined. Optionally, an initial size can be set for the associative words, and the initial size of the associative words can be reduced or enlarged according to the character body frame, so as to ensure that the size of the associative words can match the size of the character body frame.

[0221] Through the obtained character spacing information, the display position of the associative words can be determined. For example, if the character spacing information is a, each Chinese character included in the associative words can be typeset in sequence with a character spacing of a. As shown in the interface 1402 in FIG. 14, the associative words are "Interaction Laboratory Association One Two Three Four a b c d". After typesetting, the character body frame of the associative words is the same as or similar to the character body frame of the handwritten text, and the character spacing of the associative words after typesetting is the same as or close to the character spacing of the handwritten text.

[0222] In addition, optionally, in combination with the content introduced in Scenario B1 and Scenario B2, when the typesetting of the associative words reaches the page boundary or there is already a writing trajectory at the planned typesetting position, the processing of line break and typesetting downwards can be performed; the interface effect diagram after the line break and typesetting downwards can be shown in FIG. 5A or FIG. 5B or FIG. 6A or FIG. 6B.

[0223] Alternatively, in combination with the content introduced in scenario B3, in the scenario of pinyin association, when the spacing between the associated word and the adjacent Chinese character is different from the spacing between other Chinese characters, the spacing between the associated word and / or the Chinese characters in the handwritten text can be adjusted.

[0224] Another option, combined with the content introduced in scenario B4, the character spacing can be horizontal character spacing, vertical character spacing, or a combination of horizontal character spacing and vertical character spacing, so as to achieve various typesetting styles, such as horizontal typesetting style, vertical typesetting style, or inclined typesetting style, etc.

[0225] Alternatively, when the character spacing is less than or equal to a preset threshold, the character spacing of the associative words can be adjusted, and the character spacing between the associative words and the handwritten text can also be adjusted, so as to reduce the overlap of the tracks between the Chinese characters. For example, the Chinese character track, its bounding box, and its character body frame can be taken as a whole, and the character body frame can be moved to adjust the character spacing. For example, Figure 15 is an interface diagram of an interface display method based on handwriting input provided in an embodiment of the present application. Interface 1501 in Figure 15 is an interface display effect without adjusting the character spacing, and there is a certain amount of track overlap between the Chinese characters; interface 1502 in Figure 15 is an interface display effect with adjusted character spacing, and there is basically no track overlap between the Chinese characters.

[0226] Alternatively, based on block processing, the text content included in an associative word usually belongs to the same text block. When the associative word needs to wrap and typeset downward in a scenario, the typeset of the next line can also refer to the typeset of the previous line. For example, Figure 16 is another interface schematic diagram of an interface display method based on handwriting input provided in an embodiment of the present application. In interface 1601 of Figure 16, the spacing of the "Editing and Management Tools" in the next line can be displayed according to the spacing of the "Huawei Notes is a professional note" in the previous line; wherein, the spacing of the "Editing and Management Tools" can be based on the spacing of the "Huawei Notes is a professional note", for example, an average value, a weighted average value, or an average value / weighted average value after removing outliers can be performed.

[0227] The handwriting input-based interface display method provided in the embodiments of the present application can provide more functions for handwriting input scenarios; wherein, the more functions may include, but are not limited to, at least one of the following functions: generating a trajectory of associated words in the user's handwriting style, generating a layout of associated words in the user's handwriting style, adjusting the layout of associated words, balancing the spacing between handwritten text and the Chinese characters included in the associated words, and improving the accuracy of associations in mixed handwriting input including pinyin. Therefore, this method can reduce the interaction time of handwriting input and improve the efficiency of handwriting input, thereby improving the convenience and user experience of handwriting input.

[0228] Figure 17 is a flow chart of a method for displaying an interface based on handwriting input provided by an embodiment of the present application. The method can be applied to electronic devices, and the process may include the following steps:

[0229] Step 1701: Display a first interface in response to a user's handwriting input operation. The first interface includes handwritten text and at least one associative word. The handwritten text includes at least one of the following types: Chinese characters and pinyin. The first interface may, for example, be interface 400A in FIG. 4A , interface 400D in FIG. 4B , interface 400G in FIG. 4C , interface 500A and interface 500C in FIG. 5A , interface 500E and interface 500G in FIG. 5B , interface 600C in FIG. 6A , interface 600G in FIG. 6B , and the like.

[0230] Step 1702: In response to a confirmation operation on the first associative word, a second interface is displayed. The first associative word is one of the at least one associative word. The second interface includes the handwritten text and the first associative word, and the handwriting style of the first associative word is the same as the handwriting style of the user's historical handwriting input of the first associative word. The second interface can be, for example, interface 400B in FIG. 4A , or interface 400E in FIG. 4B , or interface 400H in FIG. 4C , or interface 500B and interface 500D in FIG. 5A , or interface 500F and interface 500H in FIG. 5B , or interface 600D in FIG. 6A , or interface 600H in FIG. 6B , etc.

[0231] The specific implementation of step 1701 and step 1702 can refer to the introduction in the above embodiment and will not be repeated here.

[0232] Based on the above embodiments, the present application further provides an electronic device, which includes multiple functional modules; the multiple functional modules interact with each other to implement the functions performed by the electronic device in each method described in the embodiments of the present application. The multiple functional modules can be implemented based on software, hardware, or a combination of software and hardware, and the multiple functional modules can be arbitrarily combined or divided based on the specific implementation. For example, steps 901 to 904 performed by the electronic device in the embodiment shown in Figure 9 are executed.

[0233] Based on the above embodiments, the present application further provides an electronic device, comprising at least one processor and at least one memory, wherein the at least one memory stores computer program instructions, and when the electronic device is running, the at least one processor executes the functions performed by the electronic device in each method described in the embodiments of the present application, such as executing steps 901 to 904 performed by the electronic device in the embodiment shown in FIG.

[0234] Based on the above embodiments, the present application also provides a computer program product, which includes: a computer program (also referred to as code, or instructions), which enables the various methods described in the embodiments of the present application to be executed when the computer program is run.

[0235] Based on the above embodiments, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program (also referred to as code or instructions), which, when executed by a computer, causes the computer to perform the methods described in the embodiments of the present application.

[0236] Based on the above embodiments, the present application further provides a chip, which is used to read a computer program stored in a memory and implement the methods described in the embodiments of the present application.

[0237] Based on the above embodiments, the present application provides a chip system, which includes a processor for supporting a computer device to implement the various methods described in the embodiments of the present application. In a possible embodiment, the chip system also includes a memory, which is used to store the necessary programs and data for the computer device. The chip system can be composed of a chip, or it can include chips and other discrete devices. It should be understood by those skilled in the art that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0238] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each flow and / or box in the flow chart and / or block diagram, as well as the combination of the flow chart and / or box in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for implementing the functions specified in one or more flow charts and / or one or more boxes in the block diagram.

[0239] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0240] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0241] Obviously, those skilled in the art may make various modifications and variations to this application without departing from the scope of protection of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for displaying an interface based on handwriting input, characterized in that: Used in electronic equipment, including: In response to a handwriting input operation of a user, a first interface is displayed, wherein the first interface includes handwritten text and at least one associated word, wherein the handwritten text includes at least one of the following types: Chinese characters and pinyin; In response to a confirmation operation on a first associative word, a second interface is displayed, wherein the first associative word is one of the at least one associative word; wherein the second interface includes the handwritten text and the first associative word, and the handwriting style of the first associative word is the same as the handwriting style of the user's historical handwriting input of the first associative word.

2. The method according to claim 1, characterized in that: The step of displaying a first interface in response to a handwriting input operation by a user includes: In response to the handwriting input operation, obtaining the handwritten text; the handwritten text includes pinyin; In response to the operation of converting the pinyin included in the handwritten text into Chinese characters, the at least one associated word is obtained, and the at least one associated word is a Chinese character obtained based on the pinyin.

3. The method according to claim 1 or 2, characterized in that: The first associative word in the second interface is displayed at a first display position and a second display position, the first display position displays the first at least one Chinese character included in the first associative word, and the second display position displays the remaining at least one Chinese character included in the first associative word; The first display position and the second display position have a position sequence layout relationship; the position sequence layout relationship is obtained based on the handwritten text.

4. The method according to claim 3, characterized in that The first at least one Chinese character reaches the page boundary, or the first at least one Chinese character reaches the display area of ​​the existing writing track.

5. The method according to claim 3 or 4, characterized in that: The position order typesetting relationship is a downward line wrap typesetting relationship.

6. The method according to any one of claims 1 to 5, characterized in that In the second interface, a difference between the character spacing of the handwritten text and the character spacing of the first associative word is less than or equal to a first difference threshold.

7. The method according to claim 6, characterized in that The character spacing of the handwritten text in the first interface is the same as or different from the character spacing of the handwritten text in the second interface.

8. The method according to claim 7, characterized in that When the word spacing of the handwritten text in the first interface is different from the word spacing of the handwritten text in the second interface; The character spacing of the handwritten text in the first interface is less than the character spacing threshold, and the character spacing of the handwritten text in the second interface is greater than or equal to the character spacing threshold.

9. The method according to any one of claims 6 to 8, characterized in that The character spacing is the character spacing in a target direction; wherein the target direction is determined based on the handwriting direction of the handwritten text.

10. The method according to any one of claims 1 to 9, characterized in that A difference between a font size of the handwritten text and a font size of the first associative word is less than or equal to a second difference threshold.

11. A method for displaying an interface based on handwriting input, characterized in that: Used in electronic equipment, including: In response to a handwriting input operation of a user, displaying a first interface, wherein the first interface includes handwritten text and at least one associated word; In response to a confirmation operation on a first associative word, a second interface is displayed, wherein the first associative word is one of the at least one associative word; wherein the first associative word in the second interface is displayed at a first display position and a second display position, wherein the first display position displays the first at least one Chinese character included in the first associative word, and the second display position displays the remaining at least one Chinese character included in the first associative word; wherein the first display position and the second display position have a positional sequence layout relationship; and the positional sequence layout relationship is obtained based on the handwritten text.

12. The method according to claim 11, characterized in that The first at least one Chinese character reaches the page boundary, or the first at least one Chinese character reaches the display area of ​​the existing writing track.

13. The method according to claim 11 or 12, characterized in that: The position order typesetting relationship is a downward line wrap typesetting relationship.

14. The method according to any one of claims 11 to 13, characterized in that In the second interface, a difference between the character spacing of the handwritten text and the character spacing of the first associative word is less than or equal to a first difference threshold.

15. The method according to claim 14, characterized in that The character spacing of the handwritten text in the first interface is the same as or different from the character spacing of the handwritten text in the second interface.

16. The method according to claim 15, characterized in that When the word spacing of the handwritten text in the first interface is different from the word spacing of the handwritten text in the second interface; The character spacing of the handwritten text in the first interface is less than the character spacing threshold, and the character spacing of the handwritten text in the second interface is greater than or equal to the character spacing threshold.

17. The method according to any one of claims 14 to 16, characterized in that The character spacing is the character spacing in a target direction; wherein the target direction is determined based on the handwriting direction of the handwritten text.

18. The method according to any one of claims 11 to 17, characterized in that A difference between a font size of the handwritten text and a font size of the first associative word is less than or equal to a second difference threshold.

19. The method according to any one of claims 11 to 18, characterized in that The second interface includes the handwritten text and the first associative word, and the handwriting style of the first associative word is the same as the handwriting style of the user's historical handwriting input of the first associative word.

20. The method according to any one of claims 11 to 19, characterized in that The handwritten text includes at least one of the following types: Chinese characters and pinyin.

21. The method according to claim 20, characterized in that The step of displaying a first interface in response to a handwriting input operation by a user includes: In response to the handwriting input operation, obtaining the handwritten text; the handwritten text includes pinyin; In response to the operation of converting the pinyin included in the handwritten text into Chinese characters, the at least one associated word is obtained, and the at least one associated word is a Chinese character obtained based on the pinyin.

22. An electronic device, characterized in that: The method comprises at least one processor coupled to at least one memory, wherein the at least one processor is configured to read and execute a computer program stored in the at least one memory so that the method according to any one of claims 1 to 10 is executed, or the method according to any one of claims 11 to 21 is executed.

23. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 10, or the computer is caused to execute the method according to any one of claims 11 to 21.

24. A computer program product comprising instructions, characterized in that When the computer program product is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 10, or the computer is caused to execute the method according to any one of claims 11 to 21.

Images