Interface display method based on handwriting input and electronic equipment

Abstract

The invention provides an interface display method based on handwriting input and electronic equipment, which are used for providing more functions based on a handwriting input scene, reducing the interaction time of handwriting input and improving the efficiency of handwriting input. According to the method, the electronic equipment can display at least one associational word to a user based on a handwritten text input by the user through handwriting; and when a confirmation operation of the user on one associational word is detected, the associational word confirmed by the user can be displayed by referring to the handwriting style of the handwritten text. For example, the track of the associational word can have the style of the track of the handwritten text. For example, the typesetting of the associational words can be realized by referring to the typesetting style of the handwritten text.

Description

Technical Field

[0001] This application relates to the technical field of electronic devices, and in particular, to an interface display method and an electronic device based on handwriting input. Background Art

[0002] With the development of electronic devices, the interaction function between users and electronic devices has been increasingly enhanced. In some possible scenarios, users hope to use a stylus, finger or other handwriting devices to achieve handwriting input on the electronic device.

[0003] Generally, handwriting input is obtained in response to a user's touch operation. Therefore, the format and layout of the handwriting content are not fixed, and there is a problem of low handwriting efficiency. Therefore, considering the characteristics of the handwriting input scenario, it is of great research significance to provide more functions for the handwriting input scenario. Summary of the Invention

[0004] This application provides an interface display method and an electronic device based on handwriting input, which are used to provide more functions in the handwriting input scenario, reduce the interaction time of handwriting input, and improve the efficiency of handwriting input. For example, it is possible to typeset the content in the handwriting input scenario. For another example, it is also possible to provide associated words based on the user's handwritten text, and to make the trajectory of the associated words the same or similar to the trajectory of the handwritten text. Therefore, this method can improve the convenience and user experience of the user's handwriting input.

[0005] In a first aspect, this application provides an interface display method based on handwriting input, and this method can be applied to an electronic device. In this method, in response to a user's handwriting input operation, a first interface is displayed. The first interface includes handwritten text and at least one associated word, and the handwritten text includes at least one of the following types: Chinese characters, pinyin. In response to a confirmation operation on the first associated word, a second interface is displayed. The first associated word is one of the at least one associated word. Among them, the handwriting style of the first associated word is the same as the handwriting style of the user's historical handwriting input of this first associated word. In addition, the user's handwriting style can be reflected from the following several types of information, but is not limited to: the trajectory of the font, the size of the font, the text spacing, the typesetting format, etc. Therefore, the same handwriting style can be the same trajectory of the font, the same size of the font, the same text spacing, the same typesetting format, etc. Optionally, the handwriting style of the user's historical handwriting input of this first associated word can be: obtained based on the user's historical handwriting input of this first associated word, or generated with reference to the user's historical handwriting input.

[0006] In this method, in the scenario where the user handwrites Chinese, at least one associated word related to the handwritten text can be displayed based on the handwritten text of the handwritten input. Moreover, after detecting the user's confirmation operation on the associated word, the trajectory of the associated word can be processed with reference to the user's handwriting style. In this way, the electronic device can not only display the associated word in response to the handwritten text, but also make the displayed associated word have the user's handwriting style, so that more functions in the handwritten input scenario can be realized.

[0007] In some implementation manners, the electronic device may be installed with note software, memo software, etc. Relevant models for refining the handwriting style are integrated in these software or in the operating system of the electronic device. These models can refine the user's handwriting style based on the user's handwritten input. Thereby, when the electronic device displays the selected associated word, the displayed associated word can be consistent with the styles of other characters, words, sentences or paragraphs of the user's handwritten input.

[0008] For those skilled in the art or ordinary consumers, this consistency generally improves as the number of the user's handwritten inputs increases. Or rather, the longer the user uses the electronic device to write, the more the displayed associated words will resemble the user's own writing. When the user first uses these software, the handwriting styles of the displayed associated words may not necessarily resemble the user's handwriting style very much. However, based on the capabilities of the electronic device or the applications installed on the electronic device, after using the electronic device for a period of time, with the user's handwritten input as a reference, the associated words presented by the electronic device can be the same as the user's handwriting style to present a consistent display effect on the display interface.

[0009] In a possible scenario, in response to the user's handwritten input operation, a first interface is displayed, including: obtaining handwritten text in response to the handwritten input operation; the handwritten text includes pinyin. In response to the operation of converting the pinyin included in the handwritten text into Chinese characters, at least one associated word is obtained. Wherein, at least one associated word is a Chinese character obtained based on the pinyin.

[0010] In this scenario, in the scenario where the handwritten input of Chinese includes pinyin, the electronic device can also provide the function of converting pinyin into Chinese characters, and moreover, the converted Chinese characters can also have the user's handwriting style, so that more functions in the handwritten input scenario can be realized.

[0011] In a possible scenario, the first associated word in the second interface is displayed at a first display position and a second display position; the first display position displays at least the first Chinese characters included in the first associated word, and the second display position displays at least the remaining Chinese characters included in the first associated word. Wherein, there is a position sequence layout relationship between the first display position and the second display position. The position sequence layout relationship is obtained based on the handwritten text.

[0012] In this scenario, the electronic device can also refer to the typesetting style of the handwritten text to typeset the associated words, so that the typesetting of the associated words can have the typesetting style of the handwritten text. Therefore, more functions in the handwritten input scenario can be realized.

[0013] In a possible scenario, at least one of the previously introduced Chinese characters reaches the page boundary or the display area of the existing writing trajectory.

[0014] In this scenario, when the electronic device typesets the associated words, it can also consider the interface boundary and the interface writing situation, so that line break typesetting of the associated words can be realized, and correct display of the associated words and the handwritten text can be realized.

[0015] In a possible scenario, the previously introduced positional order typesetting relationship can be a downward line break typesetting relationship.

[0016] In this scenario, the electronic device can detect and respond to the typesetting of the interface and the typesetting style of the handwritten text to realize sequential typesetting processing such as downward line break typesetting of the associated words.

[0017] In a possible scenario, in the second interface, the difference between the character spacing of the handwritten text and the character spacing of the first associated word is less than or equal to the first difference threshold.

[0018] In this scenario, the character spacing of the associated words can refer to the character spacing of the handwritten text, so that the text style of the associated words can have the text style of the handwritten text. Therefore, more functions based on the handwritten input scenario can be provided, the interaction time of the handwritten input can be reduced, and the efficiency of the handwritten input can be improved.

[0019] In a possible scenario, the character spacing of the handwritten text in the first interface is the same as or different from the character spacing of the handwritten text in the second interface.

[0020] In this scenario, the electronic device can determine the character spacing of the associated words based on the character spacing of the handwritten text, and can also consider adjusting the character spacing of the handwritten text based on the size of the character spacing. For example, the handwritten text in the first interface is obtained based on the trajectory of the user's handwritten input. In addition, the character spacing of the handwritten text in the second interface can also be adjusted. For example, if the character spacing of the handwritten text is too large, the character spacing of the handwritten text can be reduced. Another example is that if the character spacing of the handwritten text is too small or even there is overlap, the character spacing of the handwritten text can be increased, etc.

[0021] In a possible scenario, when the character spacing of the handwritten text in the first interface is different from the character spacing of the handwritten text in the second interface; the character spacing of the handwritten text in the first interface is less than the character spacing threshold, and the character spacing of the handwritten text in the second interface is greater than or equal to the character spacing threshold.

[0022] In this scenario, when the electronic device detects that the kerning of the handwritten text is too small and there is overlap between the characters, the kerning of the handwritten text can be increased, so as to achieve accurate display of the handwritten text. Moreover, when the associated word refers to the kerning of the handwritten text, it can also refer to the adjusted kerning of the handwritten text, so as to achieve accurate display of the handwritten text and the associated word.

[0023] In a possible scenario, the kerning is the kerning in the target direction. Among them, the target direction is determined based on the handwriting direction of the handwritten text.

[0024] In this scenario, the display of the associated word can also refer to the writing direction of the handwritten text. For example, it can be horizontal, vertical, or even inclined, etc., and can be determined according to the actual scenario of the user's handwritten input.

[0025] In a possible scenario, the difference between the font size of the handwritten text and the font size of the first associated word is less than or equal to the second difference threshold.

[0026] In this scenario, the display of the size of the associated word can also refer to the size of the handwritten text, so as to better achieve the handwritten style of the associated word with the handwritten text. Therefore, more functions in the handwritten input scenario can be realized.

[0027] In a second aspect, the present application also provides another interface display method based on handwritten input, and this method can be applied to an electronic device. The method provided in this second aspect may include: in response to a user's handwritten input operation, display a first interface. The first interface includes handwritten text and at least one associated word. In response to a confirmation operation on the first associated word, display a second interface. The first associated word is one of the at least one associated word. Among them, the first associated word in the second interface is displayed at a first display position and a second display position; the first display position displays at least the first Chinese characters included in the first associated word, and the second display position displays at least the remaining Chinese characters included in the first associated word. In addition, the first display position and the second display position have a position sequence layout relationship. The position sequence layout relationship is obtained based on the handwritten text.

[0028] In a possible scenario, at least the first Chinese characters introduced above reach the page boundary, or at least the first Chinese characters reach the display area of the existing writing trajectory.

[0029] In a possible scenario, the position sequence layout relationship introduced above is a line break downward layout relationship.

[0030] In a possible scenario, in the second interface, the difference between the kerning of the handwritten text and the kerning of the first associated word is less than or equal to the first difference threshold.

[0031] In a possible scenario, the kerning of the handwritten text in the first interface is the same as or different from the kerning of the handwritten text in the second interface.

[0032] In a possible scenario, when the kerning of the handwritten text in the first interface is different from the kerning of the handwritten text in the second interface; the kerning of the handwritten text in the first interface is less than the kerning threshold, and the kerning of the handwritten text in the second interface is greater than or equal to the kerning threshold.

[0033] In a possible scenario, the kerning is the kerning in the target direction. Wherein, the target direction is determined based on the handwriting direction of the handwritten text.

[0034] In a possible scenario, the difference between the font size of the handwritten text and the font size of the first associated word is less than or equal to the second difference threshold.

[0035] In a possible scenario, the second interface includes handwritten text and a first associated word, and the handwriting style of the first associated word is the same as the handwriting style of the user's previous handwritten input of the first associated word.

[0036] In a possible scenario, the handwritten text includes at least one of the following types: Chinese characters, pinyin.

[0037] In a possible scenario, in response to the user's handwritten input operation, a first interface is displayed, including: in response to the handwritten input operation, handwritten text is obtained; the handwritten text includes pinyin. In response to the operation of converting the pinyin included in the handwritten text into Chinese characters, at least one associated word is obtained. Wherein, at least one associated word is a Chinese character obtained based on the pinyin.

[0038] In a third aspect, the present application further provides an electronic device, which includes: at least one memory and at least one processor; wherein, at least one memory is used to store computer program code, and the computer program code includes computer instructions; when the computer instructions are executed by at least one processor, the electronic device is caused to execute the method in the above first aspect and any possible scenario in the first aspect, or execute the method in the above second aspect and any possible scenario in the first aspect.

[0039] In a fourth aspect, the present application further provides an interface display device based on handwritten input, which includes a module / unit for executing the method in the above first aspect and any possible scenario in the first aspect, or includes a module / unit for executing the method in the above second aspect and any possible scenario in the second aspect. These modules / units can be implemented by hardware or by hardware executing corresponding software.

[0040] Fifth aspect, a computer-readable storage medium is provided. The computer-readable medium stores a computer program (which may also be referred to as code or instruction). When these codes or instructions are run on a computer, the computer is caused to execute the methods in the above-mentioned first aspect and any possible scenario in the first aspect, or execute the methods in the above-mentioned second aspect and any possible scenario in the second aspect.

[0041] Sixth aspect, a computer program product is provided. The computer program product includes: a computer program (which may also be referred to as code or instruction). When the computer program is run, the methods in the above-mentioned first aspect and any possible scenario in the first aspect are caused to be executed, or the methods in the above-mentioned second aspect and any possible scenario in the second aspect are caused to be executed.

[0042] Seventh aspect, a graphical user interface on an electronic device is further provided. The electronic device has a display screen, one or more memories, and one or more processors. The one or more processors are used to execute one or more computer programs stored in the one or more memories. The graphical user interface may include the graphical user interface displayed when the electronic device executes the first aspect of the present application and any possible scenario in the first aspect, or may further include the graphical user interface displayed when the electronic device executes the second aspect of the present application and any possible scenario in the second aspect.

[0043] It should be noted that for the beneficial effects of each scenario of the electronic device provided in the second to seventh aspects of the present application, please refer to the beneficial effects of any possible scenario in the first aspect above, and will not be elaborated here. Description of the Drawings

[0044] Figure 1 A schematic diagram of an interface;

[0045] Figure 2 A schematic diagram of the hardware structure of a possible electronic device provided by an embodiment of the present application;

[0046] Figure 3 A block diagram of the software system architecture of an electronic device provided by an embodiment of the present application;

[0047] Figure 4A One of the schematic diagrams of the interface of a method for displaying an interface based on handwriting input provided by an embodiment of the present application;

[0048] Figure 4B One of the schematic diagrams of the interface of a method for displaying an interface based on handwriting input provided by an embodiment of the present application;

[0049] Figure 4C One of the schematic diagrams of the interface of a method for displaying an interface based on handwriting input provided by an embodiment of the present application;

[0050] Figure 5A The fourth interface schematic diagram of an interface display method based on handwriting input provided by an embodiment of the present application;

[0051] Figure 5B The fifth interface schematic diagram of an interface display method based on handwriting input provided by an embodiment of the present application;

[0052] Figure 6A The sixth interface schematic diagram of an interface display method based on handwriting input provided by an embodiment of the present application;

[0053] Figure 6B The seventh interface schematic diagram of an interface display method based on handwriting input provided by an embodiment of the present application;

[0054] Figure 7 The eighth interface schematic diagram of an interface display method based on handwriting input provided by an embodiment of the present application;

[0055] Figure 8 The ninth interface schematic diagram of an interface display method based on handwriting input provided by an embodiment of the present application;

[0056] Figure 9 A flowchart of an interface display method based on handwriting input provided by an embodiment of the present application;

[0057] Figure 10A A schematic diagram of a handwritten text provided by an embodiment of the present application;

[0058] Figure 10B A structural schematic diagram of an interface display method based on handwriting input provided by an embodiment of the present application;

[0059] Figure 11 An effect schematic diagram of a segmentation result provided by an embodiment of the present application;

[0060] Figure 12 Another effect schematic diagram of a segmentation result provided by an embodiment of the present application;

[0061] Figure 13 An effect schematic diagram of block processing provided by an embodiment of the present application;

[0062] Figure 14 The tenth interface schematic diagram of an interface display method based on handwriting input provided by an embodiment of the present application;

[0063] Figure 15 The eleventh interface schematic diagram of an interface display method based on handwriting input provided by an embodiment of the present application;

[0064] Figure 16 Figure 12 of the interface schematic diagram of a method for displaying an interface based on handwritten input provided by an embodiment of the present application;

[0065] Figure 17 Another process schematic diagram of a method for displaying an interface based on handwritten input provided by an embodiment of the present application. Detailed implementation manners

[0066] Next, embodiments of the present application will be described in detail with reference to the accompanying drawings.

[0067] Embodiments of the present application can be applied to the technical field of electronic devices, and specifically can be applicable to a handwritten input scenario based on an electronic device. Exemplarily, the electronic device may include a display screen and a touch sensor. Among them, the touch sensor can be used to detect a touch operation of a user on the display screen, and the touch operation may include, but is not limited to: an operation of the user for handwritten input, a click operation of the user on a control, a long-press operation, etc. Optionally, the user can perform handwritten input by using a stylus, a finger, or a combination of other handwritten tools. Another exemplarily, the electronic device may also include a display screen and other sensors, or a camera, etc. Among them, the other sensors or the camera can be used to detect an air gesture operation of the user on the display screen, etc. It should be understood that the user operations in the following embodiments may include touch operations, and may also include air gesture operations, etc., and the present application does not limit this.

[0068] In a possible scenario, on the display interface of an electronic device, generally, in response to a touch operation of a user, handwritten input content is displayed. In other words, the trajectory written by the user is the handwritten text displayed on the interface. In this way, there is a problem of slow efficiency in handwritten input. For example, for displaying the words "human-computer interaction", by means of keyboard input, it is only necessary to input "R, J, J, H, space" on the keyboard to complete the input; while by means of handwritten input, the user needs to write 18 strokes to complete the input. Therefore, handwritten input has the problem of long time consumption and slow efficiency in some scenarios.

[0069] In another possible scenario, in the handwritten input scenario of English, considering the characteristics of English words, completion of English words can be achieved. For example, Figure 1 is a schematic diagram of an interface. As Figure 1 shown in interface 11 in, when the user handwrites and inputs "I went to the libr", the electronic device can display multiple associated words according to the "libr" handwritten and input by the user. The associated words may, for example, at least include "library", "libraries", "librarian", etc. Then, the electronic device can display interface 12 in response to a user operation on "library" among the multiple associated words. As Figure 1As shown in interface 12, in the English handwriting input scenario, the completion of English words can be achieved by displaying associated words to the user, thereby improving the efficiency of handwriting input. However, since Chinese is characterized by strokes, the method of completing English words is difficult to apply to the Chinese handwriting input scenario.

[0070] In view of this, embodiments of the present application provide a method for displaying an interface based on handwriting input and an electronic device. In this method, for the handwriting input scenario, the electronic device can provide an associated word function based on the handwritten text input by the user; and, it can also process one or a combination of the trajectory of the associated word, the layout of the associated word, etc. based on the user's handwriting input style, thereby reducing the style difference between the associated word and the handwritten text, and reducing the difference between the layout of the associated word and the layout of the handwritten text. Through this method, more functions can be provided for the handwriting input scenario, and the convenience and user experience of the user's handwriting input can be improved.

[0071] The technical solutions in the embodiments of the present application can be applied to an electronic device. The electronic device can be a device with a display screen. The electronic device can also have a touch sensor or other sensors or a camera, etc. For example, the electronic device can be an electronic device such as a mobile phone, a tablet computer, a writing tablet, a wearable device (such as a watch, a bracelet, etc.), a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a smart home device (such as a smart TV, a smart speaker with a display screen, etc.) that can receive user operations and display an interface. It can be understood that the embodiments of the present application do not impose any restrictions on the specific type of the electronic device.

[0072] The electronic device to which the embodiments of the present application can be applied. Exemplary embodiments can include, but are not limited to, those equipped with or other operating systems. The above-mentioned electronic device can also be other electronic devices, such as a laptop with a touch-sensitive surface (such as a touch panel), etc.

[0073] Figure 2 shows a schematic diagram of the hardware structure of a possible electronic device. As Figure 2As shown in the figure, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0074] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors. Among them, the controller may be the nerve center and command center of the electronic device 100. The controller may generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions. A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory may save the instructions or data that the processor 110 has just used or recycled. When the processor 110 needs to use the instruction or data again, it can directly call it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system. In the embodiments of the present application, the processor 110 may be used to detect the user's operation instruction, and in response to the user's operation instruction, process the data corresponding to the interface to be displayed, and may also obtain the display data corresponding to the interface, and implement the display by means of the display screen 194.

[0075] The display screen 194 is used to display the display interface of an application, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1. In the embodiments of the present application, the display screen 194 can be used to display the interface after being processed by the method provided in the embodiments of the present application, such as the interface shown in the following embodiments Figures 4A to 8 and so on.

[0076] The camera 193 is used to capture still images or videos. The camera 193 can include a front camera and a rear camera. In the embodiments of the present application, the detection of air gesture operations, etc. can be achieved through the camera 193.

[0077] The internal memory 121 can be used to store computer-executable program code, and the executable program code includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. Among them, the program storage area can store the operating system and software codes of at least one application, etc. The data storage area can store the data generated during the use of the electronic device 100 (such as images, videos, etc.). In addition, the internal memory 121 can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. In the embodiments of the present application, the internal memory 121 can store, for example, program instructions for implementing the interface display method based on handwritten input provided in the present application, the styles of the trajectories of the user's historical handwritten texts, relevant data of the preset fonts, etc. The present application does not limit the content stored in the internal memory 121.

[0078] The touch sensor 180K and the display screen 194 form a touch screen, also known as a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor 180K can transmit the detected touch operations to the application processor to determine the type of touch event. Visual outputs related to the touch operations can be provided through the display screen 194. In some other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100 at a position different from that of the display screen 194. In the embodiments of the present application, the touch sensor 180K can collect user operations of the user on the handwritten input interface based on the handwritten input interface displayed by the electronic device through the display screen 194. For example, the user operations can include, but are not limited to: handwritten input, selecting associated words, etc.

[0079] It can be understood that Figure 2 the components shown do not constitute specific limitations on the mobile phone. The mobile phone can also include more or fewer components than those shown, or combine certain components, or split certain components, or have different component arrangements. In the following embodiments, the Figure 2 shown electronic device 100 is taken as an example for introduction.

[0080] The operating system (OS) involved in the embodiments of the present application is the most basic system software running on the electronic device 100. The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. In the embodiments of the present application, taking the operating system adopting a layered architecture as an example, the software system architecture of the electronic device 100 is exemplarily described.

[0081] Figure 3 This is a block diagram of the software system architecture of an electronic device provided in the embodiments of the present application. As Figure 3 shown, the software system architecture of the electronic device can be a layered architecture. For example, the software can be divided into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the operating system is divided into five layers, from top to bottom, namely the application layer, the application framework layer (FWK), the runtime and system libraries, the kernel layer, and the hardware layer.

[0082] The application layer can include a series of application packages. As Figure 3 shown, the application layer can include a camera, settings, a skin module, a user interface (UI), third-party applications, etc. Among them, the third-party applications can include a wireless local area network (WLAN), music, calls, Bluetooth, videos, memos, sticky notes, etc.

[0083] In a possible implementation, the application can be developed using the Java language and completed by calling the application programming interface (API) provided by the application framework layer. Developers can interact with the underlying layers of the operating system (such as the hardware layer, kernel layer, etc.) through the application framework layer to develop their own applications. The application framework layer is mainly a series of services and management systems of the operating system.

[0084] The application framework layer provides application programming interfaces and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions. For example, Figure 3 as shown, the application framework layer may include an activity manager, a window manager, a content provider, a view system, a telephone manager, a resource manager, a notification manager, etc.

[0085] The activity manager is used to manage the life cycles of various applications and provide common navigation and back functions, and provide an interface for interaction for the windows of all programs.

[0086] The window manager is used to manage window programs. The window manager can obtain the size of the display screen, determine whether there is a status bar, lock the screen, capture the screen, etc. The content provider is used to store and obtain data and make this data accessible to applications. This data can include videos, images, audio, dialed and received calls, browsing history and bookmarks, phone books, etc.

[0087] The view system includes visible controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures.

[0088] The telephone manager is used to provide the communication function of the electronic device. For example, the management of call states (including connection, disconnection, etc.).

[0089] The resource manager provides various resources for applications, such as localized strings, icons, pictures, layout files, video files, etc.

[0090] The notification manager enables an application to display notification information in the status bar. It can be used to convey messages of the notification type, and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to inform that a download is complete, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as a notification of a background-running application, or a notification that appears on the screen in the form of a dialogue window. For example, it can prompt text information in the status bar, emit a prompt sound, vibrate the electronic device, blink the indicator light, etc.

[0091] The runtime includes the core library and the virtual machine. The runtime is responsible for the scheduling and management of the operating system.

[0092] The core library includes two parts: one part is the functional functions that the Java language needs to call, and the other part is the core library of the operating system. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as the management of the object life cycle, stack management, thread management, security and exception management, and garbage collection.

[0093] The system library can include multiple functional modules. For example: the surface manager, the media framework, the 3D graphics processing library (e.g., OpenGLES), the 2D graphics engine (e.g., SGL), etc.

[0094] The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.

[0095] The media framework supports the playback and recording of multiple common audio and video formats, as well as static image files, etc. The media framework can support multiple audio and video coding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0096] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc.

[0097] The 2D graphics engine is the graphics engine for 2D drawing.

[0098] In some embodiments, the 3D graphics processing library can be used to draw 3D motion trajectory images, and the 2D graphics engine can be used to draw 2D motion trajectory images.

[0099] The kernel layer is the layer between the hardware and the software. The kernel layer includes at least a display driver, a camera driver, an audio driver, and a sensor driver.

[0100] The hardware layer can include various sensors, such as an acceleration sensor, a gravity sensor, a touch sensor, etc.

[0101] Generally, the electronic device 100 can run multiple applications simultaneously. More simply, one application can correspond to one process, and more complexly, one application can correspond to multiple processes. Each process has a process number (process ID).

[0102] It should be understood that in the embodiments of the present application, "at least one (item) of the following" or its similar expressions refer to any combination of these items, including any combination of single item (s) or plural item (s). For example, at least one (item) of a, b, or c can represent: a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c can be single or multiple. "Multiple" means two or more. "And / or" describes the association relationship between associated objects and indicates that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, or B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the front and back associated objects.

[0103] In addition, it should be understood that in the description of the present application, terms such as "first" and "second" are only used for the purpose of distinguishing descriptions, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying an order.

[0104] It should be understood that the hardware structure of the electronic device can be as Figure 2 shown, and the software system architecture can be as Figure 3 shown. Among them, the software programs and / or modules corresponding to the software system architecture in the electronic device can be stored in the internal memory 121, and the processor 110 can run the software programs and applications stored in the internal memory 121 to execute the process of a handwriting input-based interface display method provided in the embodiments of the present application.

[0105] To facilitate the understanding of the handwriting input-based interface display method provided in the present application, the implementation process of the method provided in the present application will be introduced below in combination with Figures 4A to 16 the content shown.

[0106] The embodiments of the present application are applicable to the scenario of handwriting input. For example, the handwriting input scenario includes, but is not limited to, the following applications (APPs): note-taking APPs such as memo APPs and sticky note APPs; email APPs; instant messaging APPs; drawing APPs; text processing APPs; spreadsheet APPs; game APPs; video conferencing APPs; sports and health APPs; photo management APPs; web page APPs; etc. When the electronic device executes the above APPs, the APP interface can be displayed through the display screen 194, and in addition, handwriting input operations can be received through the touch sensor 180K. It should be noted that the present application does not limit the handwriting input scenario to APPs, and it can also be a small program, a process, or a thread, such as a desktop process, etc.

[0107] For the convenience of understanding, the following explains the technical terms or terms that may be involved in the embodiments of the present application:

[0108] (1) Association word refers to the association based on the associated text to provide possible candidate words, phrases, or sentences, etc. It can be either the completion of the text that has been input or the conversion of pinyin into Chinese characters.

[0109] (2) Associated text can be understood as the text information related to the association word. For example, it can include, but is not limited to, context, context, pinyin, etc. Exemplarily, the associated text can be selected based on certain rules. For example, a preset number of input texts can be taken forward and / or backward, or it can also be taken forward or backward to the previous adjacent punctuation mark, or the previous adjacent space, or the previous adjacent paragraph, etc. The present application does not limit this.

[0110] (3) Handwritten text refers to the text information obtained in response to handwriting input. It should be understood that the handwritten text has the user's handwriting style. Exemplarily, the handwritten text can include, but is not limited to: Chinese characters, pinyin, English words, punctuation marks, pictures, graphics, tables, etc.

[0111] (4) The user's handwriting style can be reflected from the following information, but is not limited to: the trajectory of the font, the size of the font, the text spacing, the layout format, etc.

[0112] In addition, the same handwriting style can be reflected from the following information, but is not limited to: the same trajectory of the font, the same size of the font, the same text spacing, the same layout format, etc.

[0113] Optionally, whether the handwriting styles are the same can be judged by the confidence level. When the confidence levels of the handwriting styles of two text information are greater than or equal to the confidence level threshold, it can be considered that the styles of the two text information are the same. Or, whether the handwriting styles are the same can also be achieved through other possible judgment methods, such as a pre-trained judgment model, etc.

[0114] (5) A trajectory refers to the movement trajectory of handwritten text, which can be used to reflect the user's handwriting style. Among them, the form of handwritten text includes image format and vector trajectory format; among them, in the vector trajectory format, the handwritten text is obtained based on vector dot matrix data and has the characteristic of being easy to edit. For example, the vector dot matrix data can be obtained according to a series of operations of "press down", "move" and "lift the pen", and a group of "press down - move - lift the pen" can correspond to one stroke. It should be understood that the vector dot matrix data can be stored at the stroke granularity. On this basis, it is possible to implement an editing operation such as withdrawing a handwritten text including one stroke based on the vector dot matrix data.

[0115] In addition, the same trajectory of the font can be understood, for example, as the distribution of the vector dot matrix data corresponding to the same stroke being the same or similar.

[0116] First, the implementation process of the method provided by the embodiments of the present application will be described through multiple scenarios below, so as to facilitate understanding of the interface processing effect that can be achieved based on the method provided by the present application. It can be understood that the embodiments of the present application are not limited to the following application scenarios. In addition, in the embodiments of the present application, a mobile phone is used as an example of an electronic device, and the following will be introduced by taking the interface display on the mobile phone as an example; it should be understood that for the interface processing effect of interface display on other types of electronic devices, reference can be made to the content of the following embodiments, and the present application will not elaborate further.

[0117] Scenario A1 Figure 4A is an interface schematic diagram of a method for interface display based on handwritten input provided by an embodiment of the present application. In Figure 4A interface 400A, it may at least include: handwritten text of "Romeo and" input by the user, and a pop-up window 401; among them, the pop-up window 401 may include at least one associated word, such as "Juliet", "Zhu", "Zhuli", etc. It should be noted that the embodiments of the present application do not limit the display form of the associated words. For example, it can be in the form of a pop-up window, or in the form of a card, an interface, etc.; nor do they limit the specific associated words obtained and the arrangement order of the associated words. The same is not limited in the following embodiments and will not be repeated hereinafter. It can be understood that the associated words included in the pop-up window 401 can be obtained by associating according to the associated text; that is, obtained by associating based on the handwritten input of "Romeo and" by the user.

[0118] Optionally, in interface 400A, the electronic device responds to an operation on the associated word "Juliet" included in the pop-up window 401 and displays interface 400B. Among them, in interface 400B, it not only includes the handwritten text of "Romeo and" input by the user, but also includes the display text of "Juliet" 402. And the trajectory of "Juliet" 402 in interface 400B has the same or similar handwriting style as the trajectory of the handwritten text.

[0119] Based on the content introduced in the above Scenario A1, in the Chinese handwritten input scenario, it is possible to display associated words based on associated text; and when the user selects an associated word, it is also possible to display the trajectory of the associated word with the user's handwriting style, that is, the trajectory of the associated word has the same or similar handwriting style as the trajectory of the handwritten text. In this way, the trajectory difference between the handwritten text and the associated word can be reduced.

[0120] Scenario A2 Figure 4B is another interface schematic diagram of a method for displaying an interface based on handwritten input provided by an embodiment of the present application. In Figure 4B interface 400C, it may at least include: handwritten text of "Romeo and Zhūlìyè" input by the user; where "lìyè" is in pinyin form.

[0121] Optionally, in interface 400C, "lìyè" may further include a wavy line 403, and the wavy line 403 can be used to trigger the conversion of pinyin to Chinese. Exemplarily, the electronic device responds to an operation on the wavy line 403 and displays interface 400D; where interface 400D may at least include a pop-up window 404, and the pop-up window 404 may include at least one associated word, such as "Lìyè", etc. It should be noted that the embodiments of the present application do not limit the trigger entry for providing pinyin-to-Chinese conversion through the wavy line 403. For example, the trigger entry may also be in the form of a pop-up window, a control, etc. The same is not limited in the following embodiments and will not be repeated hereinafter. It can be understood that the associated words included in the pop-up window 404 can be obtained by associating based on the associated text; that is, they can be obtained by associating based on the pinyin "lìyè", or based on the pinyin "lìyè" and the handwritten text "Romeo and Zhūlìyè".

[0122] In interface 400D, the electronic device responds to an operation on the associated word "Lìyè" included in the pop-up window 404 and displays interface 400E. Among them, in interface 400E, it not only includes the handwritten text of "Romeo and Zhū" input by the user, but also includes the display text of "Lìyè" 405. And the trajectory of "Lìyè" 405 in interface 400E has the same or similar handwriting style as the trajectory of the handwritten text.

[0123] Based on the content introduced in the above Scenario A2, in the Chinese handwritten input scenario, it is possible to display the Chinese corresponding to the pinyin based on the associated text and the pinyin included in the associated text; and when the user selects an associated word, it is also possible to display the trajectory of the associated word with the user's handwriting style. In this way, the trajectory difference between the handwritten text and the associated word can be reduced.

[0124] Scenario A3 Figure 4CAnother interface schematic diagram of the interface display method based on handwritten input provided by the embodiments of the present application. In Figure 4C In the interface 400F, it may at least include: the handwritten text of "My favorite drama" and "luómìōu yǔzhūlìyè" input by the user; wherein, "luómìōu yǔzhūlìyè" is in pinyin form.

[0125] In some other examples, the electronic device can also recognize the handwritten text of "My favorite drama" and "luomi ou yu zhu li ye" input by the user; wherein, "luo mi ou yu zhu li ye" can also be pinyin without tones.

[0126] It should be noted that in the embodiments of the present application, the form of pinyin included in the handwritten text is not limited. Considering the randomness of the user's handwritten input, the handwritten text can be in the standard pinyin form, or in a form with a high similarity to the standard pinyin. For example, the pinyin included in the handwritten text can all not include tones, or some pinyin include tones, or all pinyin or some pinyin include inaccurate tones, etc. In the following embodiments, pinyin including tones is used as an example for introduction.

[0127] Optionally, in the interface 400F, "luómìōu yǔzhūlìyè" may further include a wavy line 406, and the wavy line 406 can be used to trigger the conversion of pinyin to Chinese. Exemplarily, the electronic device responds to the operation on the wavy line 406 and displays the interface 400G; wherein, the interface 400G may at least include a pop-up window 407, and the pop-up window 407 may include at least one associated word, such as "Romeo and Juliet", etc.

[0128] In the interface 400G, the electronic device responds to the operation on the associated word "Romeo and Juliet" included in the pop-up window 407 and displays the interface 400H; wherein, the interface 400H not only includes the handwritten text of "My favorite drama" input by the user, but also includes the display text of "Romeo and Juliet" 408. Moreover, the trajectory of "Romeo and Juliet" 408 in the interface 400H has the same or similar handwritten style as the trajectory of the handwritten text.

[0129] Alternatively, the interfaces 400F and 400H may not include the handwritten input "My Favorite Drama" in Chinese, that is, the interface does not include handwritten Chinese text. In this scenario, the trajectory of "Romeo and Juliet" 408 in the interface 400H may have the same or similar handwriting style as the trajectory of the user's historical handwritten text; wherein, the handwriting style of the user's historical handwritten text can be obtained based on the user's historical handwritten text, for example, the same or relevant historical handwritten text can be obtained spatially or temporally. For example, the handwritten text relevant spatially can be the handwritten text of "My Favorite Drama" in the interface 400F as shown in Figure 4C The handwritten text of "My Favorite Drama" in the interface 400F shown; the handwritten text that is the same or relevant temporally can be the stored handwritten text of the user's history, such as the handwritten input of "Romeo and Juliet" by the user in history.

[0130] Based on the content introduced in the above Scenario A3, in the Chinese handwritten input scenario, when there is no associated text or the associated text cannot obtain the user's handwriting style, the Chinese corresponding to the pinyin can be displayed based on the handwriting style of the historical handwritten text; and, the converted Chinese has the user's handwriting style.

[0131] Due to the randomness of handwritten input, it is difficult to typeset the associated words. The following introduces several possible typesetting scenarios through Scenarios B1 to B4. It should be understood that the following scenarios can be combined, and can also be combined with the foregoing Scenarios A1 to A3, and the specific implementation process will not be elaborated.

[0132] Scenario B1, Figure 5A is a schematic diagram of an interface of a method for displaying an interface based on handwritten input provided by an embodiment of the present application. In Figure 5A the interface 500A, the handwritten texts of "I. Typesetting Method 1" and "Handwritten Input Typesetting 1 Hand" input by the user are displayed, and the pop-up window 501a; wherein, the pop-up window 501a may include at least one associated word, such as "Handwritten Input Typesetting" shown in the interface 500A, and for another example, although not shown in the interface 500A, the associated words may also be "Write", "Machine", etc. It can be understood that the associated words included in the pop-up window 501a can be obtained by association according to the associated text; that is, based on the user's handwritten input of "Handwritten Input Typesetting 1 Hand", or "Hand", etc. for association.

[0133] Optionally, in the interface 500A, in response to an operation on the associated word "Handwritten Input Typesetting" included in the pop-up window 501a, the interface 500B is displayed. Similar to Scenario A1, the trajectory of the associated word "Handwritten Input Typesetting" has the same or similar handwriting style as the trajectory of the handwritten text.

[0134] In addition, when the electronic device displays the associated words, it can not only process the handwriting style of the trajectory of the associated words, but also process the handwriting style of the layout of the associated words.

[0135] Exemplarily, in the interface 500B, the associated word "writing input layout" includes 5 Chinese characters, and the electronic device detects the first layout scenario. Among them, the first layout scenario can be understood as that after the word "writing" in the displayed associated words, the layout has reached the page boundary. Then, in response to the first layout scenario, the electronic device wraps the remaining "input layout" 501b in the associated words downward.

[0136] In some examples, in the interface 500B, the electronic device can also continue to respond to the handwriting input operation, and can be displayed as the interface 500C. Compared with the interface 500B, the interface 500C can also include "2 handwriting input", the pop-up window 501c that the user further handwrites. Among them, the pop-up window 501c can include at least one associated word, such as "layout 3" in the interface 500C, etc. It can be understood that the associated words included in the pop-up window 501c can be obtained by associating according to the associated text; that is, based on the user's handwritten input of "handwriting input layout 1 handwriting input layout 2 handwriting input", or "handwriting input", etc.

[0137] In the interface 500C, in response to the operation on the associated word "layout 3" included in the pop-up window 501c, the interface 500D is displayed. The interface 500D is similar to the interface 500B. The associated word "layout 3" includes 3 Chinese characters. When the word "layout" in the displayed associated words, the layout has reached the page boundary; then, the remaining "version 3" 501d in the associated words wraps downward.

[0138] As shown by the interfaces 500A to 500D, when performing downward line wrapping and layout, it can be formatted according to the paragraph format with a first-line indent. It should be understood that the embodiment of the present application does not limit the paragraph format of the downward line wrapping and layout. For example Figure 5BAs shown in the included interfaces 500E, 500F, 500G, and 500H, when performing line breaks for typesetting, it can also be set to a paragraph format where each line is aligned. As shown in interface 500E, there is displayed text content and a pop-up window, and the pop-up window includes at least the associated word "write input typesetting". As shown in interface 500F, the "write" in the associated word "write input typesetting" can be displayed after the last "hand" in interface 500E, and the remaining "input typesetting" 502a in the associated word can be typeset with a line break downwards. As shown in interface 500G, there is displayed text content and a pop-up window, and the pop-up window includes at least the associated word "input typesetting 3". As shown in interface 500H, the "input" in the associated word "input typesetting 3" can be displayed after the last "write" in interface 500G, and the remaining "typesetting 3" 502b in the associated word "input typesetting 3" can be typeset with a line break downwards. For the specific implementation process, refer to the introduction of interfaces 500A to 500D in the previous text, and it will not be elaborated here.

[0139] Scene B2 Figure 6A This is another schematic diagram of an interface for a method of displaying an interface based on handwritten input provided by an embodiment of the present application. In Figure 6A interface 600A, there are displayed handwritten texts of "III. Handwritten Content 1", "1. Content 1", "2. Content 2", and "3. Content 3" handwritten by the user. Considering the randomness of the handwritten input scenario, in interface 600A, the electronic device can detect the handwritten input operation of the user at any display position. As shown in interface 600B, compared with interface 600A, there is a handwritten text of "handwritten input" 601 newly handwritten by the user displayed on the left side of the interface.

[0140] In interface 600B, in response to the handwritten input operation, the electronic device can display interface 600C; among them, interface 600C includes a pop-up window 602, and the pop-up window 602 can include at least one associated word, such as "typesetting" shown in interface 600C. It can be understood that the associated words included in the pop-up window 602 can be obtained by association based on the associated text; that is, by associating based on the user's handwritten input of "handwritten input", or "input", etc.

[0141] Optionally, in interface 600C, in response to the operation on the associated word "typesetting" included in the pop-up window 602, the electronic device displays interface 600D. Similar to scene A1, the trajectory of the associated word "typesetting" has the same or similar handwritten style as the trajectory of the handwritten text.

[0142] In addition, when the electronic device displays the associated words, it can not only process the handwritten style of the trajectory of the associated words, but also process the handwritten style of the typesetting of the associated words.

[0143] Exemplarily, in interface 600C, the associative word "Typesetting" includes 2 Chinese characters, and the electronic device detects the second typesetting scenario. Among them, the second typesetting scenario can be understood as after displaying the "row" in the associative word, although the typesetting has not reached the page boundary, there is already a writing track at the planned typesetting position. Then, in response to the second typesetting scenario, the electronic device wraps the remaining "row" 603 in the associative word downward and displays it as interface 600D. In this way, when there is already a writing track at the planned typesetting position, the existing writing track can be avoided from being covered by the downward typesetting.

[0144] As shown in interfaces 600A to 600D, when performing downward line layout, the layout can be performed according to the paragraph format set to align each line. It should be understood that the embodiment of the present application does not limit the paragraph format of downward line layout, for example Figure 6B As shown in the included interfaces 600E, 600F, 600G and 600H, when performing downward line break typesetting, the paragraph format can also be set according to the text content. As shown in interface 600F, compared with interface 600E, a handwritten text of "Content: Handwriting" 604 input by the user's handwriting is newly added. As shown in interface 600G, according to the handwritten text of "Content: Handwriting" 604, the associated word obtained is "input typesetting" 605. As shown in interface 600H, when the associated word "input typesetting" 605 is typeset in the downward line break typesetting, it is typeset according to the paragraph format with a hanging indent, that is, it can refer to the display position of ":" for downward line break typesetting.

[0145] Scenario B3: In the scenario where the associative word function is provided during handwriting input, although the associative words are displayed after the electronic device performs trace and typesetting, there may still be problems such as too large or too small character spacing or even overlap. Therefore, in the embodiment of the present application, the character spacing can also be adjusted.

[0146] For example, Figure 7 Another interface schematic diagram of a handwriting input-based interface display method provided in an embodiment of the present application. Figure 7 The interface 701 is included, and the interface 701 shows “如gěng在喉” which is input by handwriting of the user. Among them, “gěng” is in pinyin form.

[0147] Optionally, referring to the introduction of scenario A2 and scenario A3 in the previous text, the pinyin "gěng" can be converted into the Chinese character "鲠", which can be displayed as interface 702. However, since "gěng" occupies a larger display width than "鲠" in the handwriting input scenario, as shown in interface 702, there is a problem of too large character spacing between "鲠" and the preceding character "如" and the following character "在".

[0148] Alternatively, the electronic device may also display an interface 703 in response to the operation of converting pinyin to Chinese. Compared with interface 702, interface 703 can not only realize the conversion of pinyin to Chinese, but also realize the adjustment of character spacing. It can be seen from interface 703 that, compared with interface 702, the character spacing between "鲠" and the preceding character "如" and the following character "在" is adjusted, and the character spacing is reduced.

[0149] Scene B4 Figure 8 Another interface schematic diagram of an interface display method based on handwriting input provided in an embodiment of the present application. Figure 8 Including interface 800A, the handwritten text "the moonlight shines brightly before the bed" input by the user can be shown in the interface 800A. Based on the contents introduced in the above-mentioned scenarios, the electronic device can display a pop-up window including an associative word, and in response to the selection operation of the associative word, the associative word can be displayed; wherein the associative word can have the same or similar trajectory as the handwritten text, and can also be typed.

[0150] During the typesetting process, the typesetting characteristics of the handwritten text can also be referred to. For example, based on the fact that the handwritten text shown in the interface 800A has a tilted typesetting characteristic, the associated words "I wonder if it is frost on the ground. I look up at the bright moon and look down at my hometown." can also have a tilted typesetting characteristic while wrapping downward.

[0151] Through the content introduced in scenario B4, when the electronic device typeset the associative words, it can combine the user's handwriting style to achieve the typesetting of the associative words.

[0152] Based on the introduction of the above scenarios, the following describes the implementation process of a handwriting input-based interface display method provided by the present application, in order to illustrate how to use the method provided by the present application to achieve the above-mentioned Figures 4A to 8 The interface processing effect shown provides a technical solution for displaying associated words in a handwriting input scenario.

[0153] See also Figure 9 , is a flow chart of a method for displaying an interface based on handwriting input provided in an embodiment of the present application. The method can be applied to electronic devices and may include the following process:

[0154] Step 901: In response to a handwriting input operation of a user, obtain handwritten text input by the user.

[0155] It should be noted that, in addition to the user's handwriting input operation, multiple other operations for obtaining handwritten text may also be included. Other operations include, but are not limited to, click operations, double-click operations, long press operations, etc. In addition, the operation objects of other operations may include, but are not limited to: undo controls, interfaces, auxiliary tools included in the interfaces, etc.

[0156] In the embodiments of the present application, the handwritten text may include, but is not limited to: Chinese characters, pinyin, English words, punctuation marks, pictures, graphics, tables, etc.

[0157] Step 902: Based on the handwritten text, display at least one associated word; determine the associated word selected by the user.

[0158] Based on the content introduced in Scenarios A1 to A3, the associated word can be obtained based on the context, context, pinyin, etc. of the handwritten text; or, it can also be obtained based on other methods, such as based on historical input data, machine learning methods, etc. In the embodiments of the present application, the implementation manner of obtaining at least one associated word is not limited. Moreover, the embodiments of the present application do not limit the display manner of at least one associated word. For example, it can be displayed in a pop-up window, card, or two-layer interface, etc. Also, for example, the pop-up window can include a specified number of associated words of at least one associated word, and can be displayed horizontally or vertically, etc.

[0159] Among them, obtaining an associated word based on the pinyin included in the handwritten text can include multiple scenarios: all-pinyin input scenario, pinyin and Chinese character mixed input scenario, pinyin and English mixed input scenario. The following introduces several possible ways to obtain associated words:

[0160] Method A: Based on the pinyin included in the handwritten text, obtain an associated word, which may include the following processing:

[0161] 1). Determine the object to be recognized.

[0162] Exemplarily, the object to be recognized can be: the handwritten text including pinyin, or the block obtained based on the handwritten text including pinyin. For example, Figure 10A is a schematic diagram of a handwritten text provided by an embodiment of the present application. In Figure 10A in the interface 1000A, the handwritten text or block can be "yǎn rán a visual and auditory tāo tiè feast"; in Figure 10A in the interface 1000B, the handwritten text or block can be "women shì woman de fùshù xíng shì". It should be understood that when the object to be recognized corresponds to the block obtained based on the handwritten text including pinyin, the handwritten text may include more text content than the block.

[0163] Another exemplarily, the object to be recognized can be: extracted based on the handwritten text including pinyin; or, extracted based on the block obtained from the handwritten text including pinyin. For example, the object to be recognized can be obtained by taking pinyin / English as the basis and selecting n Chinese characters forward and / or backward until n is obtained; where n can be 1, 2, or 3, etc. Taking n as 3 as an example, in combination withFigure 10A For the phrase "a veritable visual and auditory feast" shown in interface 1000A, including the pinyin "yǎn", the corresponding object to be recognized extracted therefrom can be "a veritable"; interface 1000A also includes "tāo tiè", and the corresponding objects to be recognized extracted therefrom can be "auditory feast", "tāo tiè feast", or "auditory tāo tiè feast".

[0164] In the above example, when reaching the boundary of the handwritten text forward and / or backward based on the handwritten text, and the number of Chinese characters that can be selected is less than n, then all the text forward and / or backward in the handwritten text can be selected. For example, taking n as 3 as an example, combined with Figure 10A For the phrase "women shìwoman de fùshùxíng shì" shown in interface 1000B, the corresponding object to be recognized extracted therefrom can be "women shìwoman de fùshùxíng shì".

[0165] 2) Based on the object to be recognized, perform pinyin association. Exemplarily, pinyin association can be implemented in at least one of the following ways: semantic analysis based on context, pinyin to Chinese conversion, neural network model, deep learning, etc.

[0166] In an alternative embodiment, pinyin association can be performed based on a hidden Markov model (HMM). The following processing can be included:

[0167] 2-1) Perform word segmentation on the object to be recognized. Exemplarily, word segmentation can be performed based on symbols such as spaces, possible combinations of Chinese character pinyins, or possible combinations of English words, etc.

[0168] For example, when a space is recognized in "tāo tiè" in the handwritten text, it can be divided into "tāo" and "tiè" based on the space.

[0169] For another example, when no space is recognized in "tāotiè" in the handwritten text, it can also be divided into "tāo" and "tiè" based on possible combinations of Chinese character pinyins, or divided into "tā", "o", and "tiè".

[0170] For yet another example, for "women" in the handwritten text, it can be divided into "women" based on possible English words, or divided into "wo" and "men" based on possible combinations of Chinese character pinyins.

[0171] 2-2) Input the segmented words obtained after the segmentation operation into the HMM model to obtain the scoring results after each segmented word is converted into Chinese. In this way, based on the scoring results, the display order of the associated words can be determined. For example, the associated word with the highest scoring result can be displayed in the first place, and as the scoring result decreases, the associated words can be sorted in sequence.

[0172] Method B, based on the deep learning method, obtains the associated words corresponding to the pinyin, wherein the deep learning model is pre-trained based on the training corpus.

[0173] Through pinyin association processing, the user's handwritten input "yǎn然一个视觉和聽的tāo tiè盛宴" can be converted into " 如然一个视觉和聽的饕餮盛宴" with the user's handwriting style; the user's handwritten "women shìwoman de fùshùxíng shì" can be converted into "women是女人的plural form" with the user's handwriting style. In this way, through pinyin association processing, in the handwriting input scenario, pinyin can be converted into Chinese characters for the scenario where people forget the characters when they pick up the pen.

[0174] Step 903: Generate a trajectory of associated words.

[0175] In a possible embodiment, when there is an associated handwritten text, the trajectory of the associated word can be generated based on the handwriting style of the trajectory of the handwritten text. The handwriting style of the trajectory of the associated word may include but is not limited to the following features:

[0176] (1) Content features. Exemplarily, based on the associative words selected by the user, the content features of the associative words are searched from the standard library track. The standard library track can be understood as a dictionary storing {font: font standard library track}, which is used to indicate the mapping relationship between the font and its corresponding font standard library track. The font standard library track can indicate the content features of the font.

[0177] For example, the handwritten text is "yǎn然", and the associated word corresponding to "yǎn" selected by the user is "嫣", and the content features of "嫣" can be obtained from the standard library track of "嫣" stored in the dictionary.

[0178] (2) Style features. For example, the content features may not have the user's handwriting style. Through the style features, it can be achieved that the trajectory of the associated word has the same or similar style as the trajectory of the handwritten text.

[0179] Optionally, when the electronic device can obtain the user's handwritten text, the style feature can be obtained based on the user's handwritten text. Figure 4AFor example, in the scenario of semantic association, the style features of the associated word "Juliet" can be obtained based on the style features of the handwritten text "Romeo and". Combining Figure 4B For example, in the scenario of pinyin association, the style features of the associated word "Liye" can be obtained based on the style features of the handwritten text "Romeo and Zhu".

[0180] Alternatively, when the electronic device cannot obtain the user's handwritten text, the style features can be obtained based on historical handwritten text. Among them, the historical handwritten text can include handwritten text that is spatially related; among them, the handwritten text that is spatially related can be, for example, text that is semantically unrelated but in the same interface. The historical handwritten text can also include historical text that is temporally related; among them, the handwritten text that is temporally related can be, for example, text that is not in the same interface but belongs to the historical handwritten input in the APP. Combining Figure 4C For example, when the handwritten text "My favorite play" is not recognized as the associated text of the associated word "Romeo and Juliet", it can also be understood that the two do not belong to the same sentence; at this time, the handwritten text "My favorite play" can be used as the handwritten text that is spatially related, so that the style features of the associated word "Romeo and Juliet" can be obtained.

[0181] Based on the obtained content features and style features, they can be input into the diffusion model to generate the trajectory of the associated word. Through the content features and style features, the trajectory of the associated word can be made to have the user's handwritten style.

[0182] (3) Stroke features. For example, if the associated word is "Yan", the stroke features of "Yan" can be extracted. Through the stroke features, the accuracy of generating the trajectory of the associated word can be improved, and the difference degree between the trajectory style of the associated word and the trajectory style of the user's handwritten text can be reduced.

[0183] Based on the obtained content features, style features and stroke features, they can also be input into the diffusion model to generate the trajectory of the associated word. For example, Figure 10BA schematic diagram of the structure of a handwriting input-based interface display method provided in an embodiment of the present application. The structure may include a content encoder 1001, a stroke encoder 1002, a style encoder 1003, and a diffusion model 1004. The content encoder 1001 may be used to extract content features in the standard library track corresponding to the associative word, the stroke encoder 1002 may be used to extract the stroke features corresponding to the associative word, the style encoder 1003 may be used to extract the style features of the user's handwritten text, the splicing unit 1004 may be used to receive the splicing features obtained based on the content features, the stroke features, and the style features, and may also be used to output the associative word track with the user's handwriting style; wherein the content features, the stroke features, and the style features may be spliced ​​in the feature dimension to obtain the splicing features.

[0184] For example, the electronic device generates the trajectory of the associated word "严" and the associated word "饕餮" as follows Figure 10A As shown in interface 1000C.

[0185] In addition, when the electronic device cannot obtain the user's handwritten text or the user's historical handwritten text, and thus cannot obtain the style of the associated word based on the accurate user's handwriting style, the preset font with the highest confidence in the similarity with the user's handwriting style can be selected from the preset fonts to generate the trajectory of the associated word. Optionally, in this scenario, the user's handwriting style can be obtained based on pinyin, and the similarity between the preset font style and the user's handwriting style can be the similarity between the preset font style and the handwriting style of pinyin.

[0186] Step 904: typeset and display the associated words.

[0187] Based on the contents introduced in scenes B1 to B4, in order to realize accurate display of the associated words, it is necessary to perform typesetting processing on the associated words. In an optional embodiment, the typesetting processing may include the following processing process:

[0188] 1) Segment the handwritten text to obtain a segmentation result. For example, the handwritten text may be in a vector trajectory format, which is obtained based on vector dot matrix data, so the vector dot matrix data corresponding to the handwritten text may be used as input for segmentation processing. Optionally, the vector dot matrix data may be represented by two-dimensional coordinates, such as {x, y}.

[0189] In an optional embodiment, the electronic device can segment the handwritten text according to the formats of Chinese characters, pinyin, English words, graphics, tables, etc. Figure 8As shown in the included interface 800A, the handwritten text is "Before my bed a pool of light,". After segmentation, the segmentation results of "bed", "before", "bright", "moon", "light", and "," can be obtained.

[0190] Exemplarily, the segmentation results can be used for an electronic device to display associated words. For example, to determine the character spacing, font size, font trajectory style, typesetting, etc. of the associated words. The specific implementation process will be introduced in the following embodiments and will not be elaborated here for the time being.

[0191] Another exemplarily, the segmentation results can also be used for an electronic device to adjust the typesetting of handwritten text. For example, to adjust the character spacing in the handwritten text, perform alignment processing on the handwritten text, etc.

[0192] For example, Figure 11 is a schematic diagram of an effect of the segmentation results provided by the embodiments of the present application. In Figure 11 , each Chinese character included in the handwritten text can include at least the bounding box of the Chinese character after segmentation processing. Among them, the bounding box can be obtained based on the vector dot matrix data set corresponding to the Chinese character. Optionally, the four borders of the bounding box can be obtained based on the minimum value of X (Xmin), the maximum value of X (Xmax), the minimum value of Y (Ymin), and the maximum value of Y (Ymax) in the vector dot matrix data set corresponding to the Chinese character. For example, the bounding box can be expressed as [Xmin, Ymin, W (Xmax—Xmin) , H (Ymax—Ymin) ; where W (Xmax—Xmin) represents the width obtained based on the difference between the maximum value of X and the minimum value of X, and H (Ymax—Ymin) represents the height obtained based on the difference between the maximum value of Y and the minimum value of Y. Another example, the bounding box can also be expressed as [(Xmin, Ymin), (Xmax, Ymin), (Xmax, Ymax), (Xmax, Ymax)]; where (Xmin, Ymin), (Xmax, Ymin), (Xmax, Ymax), (Xmax, Ymax) can represent the coordinates of the four vertices of the bounding box of the Chinese character.

[0193] Another example, Figure 12 is another schematic diagram of an effect of the segmentation results provided by the embodiments of the present application. In Figure 12 , each Chinese character included in the handwritten text also includes at least the bounding box of the Chinese character after segmentation processing.

[0194] In another alternative embodiment, the electronic device can also segment the handwritten text into combinations such as words, idioms, sentences, etc. Combining Figure 7The included interface 701 has handwritten text "rú gěng zài hóu". After segmentation, not only can the segmentation results of "rú", "gěng", "zài", and "hóu" be obtained, but also the segmentation result of "zài hóu" can be obtained, etc.

[0195] In another alternative embodiment, the electronic device can also segment the handwritten text at the granularity of lines.

[0196] For example, one segmentation result is still as Figure 11 shown, Figure 11 which may further include line box 1101, line box 1102a, line box 1102b, line box 1102c, line box 1102d, line box 1103a, and line box 1103b; wherein, the line box can be understood as a rectangular box corresponding to the handwritten text divided into one line.

[0197] Another example, another segmentation result is as Figure 12 shown in the included interface 1200A. The interface 1200A may further include line box 1201 and line box 1202. Or for example, the segmentation result may also be as Figure 12 shown in the interface 1200B. The interface 1200B may further include line box 1201a, line box 1201b, line box 1202a, and line box 1202b. It can be seen from Figure 12 that compared with the interface 1200A, considering the problem of too large character spacing in the line box 1201 in the interface 1200A, the line box 1201 can be further segmented and can be split into line box 1201a and line box 1201b as shown in the interface 1200B; similarly, the line box 1202 can be further segmented and can be split into line box 1202a and line box 1202b as shown in the interface 1200B. Among them, the line box can be obtained based on the set of vector dot matrix data corresponding to multiple Chinese characters included in the corresponding line. Optionally, the four borders of the line box can be obtained based on the minimum value of X, the maximum value of X, the minimum value of Y, and the maximum value of Y in the set of vector dot matrix data corresponding to multiple Chinese characters. Another option is that in the foregoing embodiment, the bounding box of the Chinese character can be obtained, and the line box can also be obtained based on multiple bounding boxes corresponding to multiple Chinese characters one by one.

[0198] 2) Based on the segmentation result, perform block processing.

[0199] Exemplarily, the electronic device can perform block processing based on the segmentation result at the line granularity to obtain one or more text blocks. Through block processing, during the typesetting process, the typesetting of multiple lines of text belonging to the same text block can also be processed with reference to the division of the text blocks. For example, the next line can be processed with reference to the character spacing, line spacing, alignment method, etc. of the previous line, so as to reduce the typesetting differences in the interface.

[0200] Optionally, the electronic device may perform chunking processing on the segmentation result based on a preset rule. For example, the preset rule may include, but is not limited to, the following rules:

[0201] Rule A: A rule set based on information such as the position of the line box, the length and width of the line box. For example, the spacing between line boxes is less than or equal to a preset threshold, the average spacing between the word boxes included in two line boxes is less than or equal to a preset threshold, the text arrangement direction of two line boxes is less than or equal to a preset threshold, etc.

[0202] Rule B: Semantic information. For example, the semantic association degree of the text information included in two line boxes is greater than a preset threshold, etc.

[0203] It should be noted that the preset thresholds corresponding to different rules can be set to the same value or different values.

[0204] Another option is that the electronic device can also use a machine learning algorithm to cluster the line boxes and regard the line boxes belonging to the same cluster as a text block. Among them, during the clustering process, semantic information can also be combined for analysis.

[0205] Another option is that the electronic device can also use a deep learning method to classify the line boxes and fuse the line boxes belonging to the same classification into a text block. Among them, during the classification process, semantic information can also be combined for analysis.

[0206] For example, Figure 13 is a schematic diagram of the effect of the chunking processing provided by the embodiment of the present application. As Figure 13 shown, it may include chunk 1301 and chunk 1302; among them, chunk 1301 is used to indicate that the title of the ancient poem is "Bringing in the Wine", chunk 1302 is the poem content of the ancient poem, and chunk 1303 is the note for the ancient poem. Combining Figure 11 and Figure 13 it can be obtained that chunk 1302 can be a text block obtained based on line boxes 1102a, 1102b, 1102c, and 1102d, and chunk 1303 can be a text block obtained based on line boxes 1103a and 1103b.

[0207] By performing the processing of dividing text blocks on the handwritten text and associated words displayed on the interface, it is convenient to achieve unified processing of text content with the same or similar semantics, thereby reducing differences. For example, taking the poem content of the ancient poem as a text block can perform unified typesetting processing on the poem content, thereby reducing the differences between the text contents that also belong to the poem content.

[0208] 3) Perform typesetting according to the block to which the associated word belongs.

[0209] The typesetting process may include the following processing procedures:

[0210] 3-1), Based on the bounding boxes of the Chinese characters included in the segmentation result, obtain the aspect ratio relationship of the Chinese characters in the handwritten text; Based on the aspect ratio relationships corresponding to multiple Chinese characters, search for the target font from the preset fonts. Among them, the target font can be understood as the font whose style similarity with the handwritten text meets the preset conditions, and the preset conditions are, for example, the highest similarity or the similarity is greater than the preset threshold.

[0211] Exemplarily, the error between the handwritten text and each preset font can be determined through the following formula 1, and then the preset font whose error meets the preset conditions can be used as the target font, as follows:

[0212]

[0213] Among them, loss(x,y) represents the error between the preset font and the handwritten font. The smaller the error, the closer the style of the preset font is to the style of the handwritten font. a1, a2, and a3 are coefficients, and the sum of a1, a2, and a3 can be a preset value, for example, the preset value is 1. n is the number of preset fonts. Used to represent the error between the aspect ratios of Chinese characters between the preset font and the handwritten font, Used to represent the error between the widths of Chinese characters between the preset font and the handwritten font, Used to represent the error between the heights of Chinese characters between the preset font and the handwritten font.

[0214] 3-2), According to the aspect ratio relationship of the target font, determine the body box and character spacing information of the Chinese characters in the handwritten text.

[0215] Exemplarily, the body box can be obtained by means of average, weighted average, etc.; Similarly, the character spacing information can also be obtained by means of average, weighted average, etc. For example, Figure 14 This is a schematic diagram of the interface of a method for displaying an interface based on handwritten input provided by an embodiment of the present application. As Figure 14 shown in the interface 1401 in, the handwritten text can be "human-machine". Obtain the body box 1 corresponding to the character "human" and the body box 2 corresponding to the character "machine" in the target font, and then the body box of the Chinese characters in the handwritten text can be obtained according to the average value or weighted average value of the body box 1 and the body box 2; That is, obtain the solid line box shown in the interface 1401. Among them, the dotted line box in the interface 1401 is the bounding box of the Chinese character, and the solid line box is the body box of the Chinese character. And, based on the body boxes or bounding boxes of the characters "human" and "machine" in the interface 1401, the character spacing information can be further obtained.

[0216] In another exemplary embodiment, when the length-to-width ratio of the body frame of a Chinese character in a handwritten font is significantly different from the length-to-width ratio of the body frame of the Chinese character in the target font, the body frame of the Chinese character may not be used in the calculation of the body frame of the handwritten text. Taking the character "一" as an example, the length-to-width ratio of the "一" handwritten input by the user can be expressed as 20 / 0.01=2000, while the length-to-width ratio of the "一" in the target font can be expressed as 150 / 17.5≈8.57. From this comparison, it can be found that the length-to-width ratio of the two characters is quite different, which may lead to a large deviation in the calculated body frame. Optionally, the character "一" may be excluded from the calculation of the body frame of the handwritten text, thereby ensuring the accuracy of the body frame of the determined handwritten font. Alternatively, the height of the Chinese character may be converted before calculating the body frame of the handwritten text. For example, the height of the handwritten character "一" may be 2.33 after conversion according to the length-to-width ratio of 8.57 corresponding to the target font and the length of 20 corresponding to the handwritten font. Alternatively, a minimum threshold may be set for the height of the Chinese character, such as 1 or 2, so as to ensure the accuracy of the determined body frame.

[0217] 3-3) According to the character frame and character spacing information of the handwritten text, the associated words are typeset and displayed.

[0218] Exemplarily, based on the processing of step 903 in the above embodiment, the trajectory of the associative word can be obtained.

[0219] The size of the associated word can be determined by the obtained character frame. Optionally, an initial size can be set for the associated word, and the initial size of the associated word can be reduced or enlarged according to the character frame to achieve that the size of the associated word can match the size of the character frame.

[0220] The display position of the associated word can be determined by the obtained character spacing information. For example, if the character spacing information is a, each Chinese character included in the associated word can be arranged in sequence according to the character spacing a. Figure 14 As shown in the interface 1402, the associative words are "interactive laboratory association one two three four abcd", and the body frame of the associative words after typesetting is the same as or similar to the body frame of the handwritten text, and the character spacing of the associative words after typesetting is the same as or similar to the character spacing of the handwritten text.

[0221] In addition, optionally, in combination with the contents introduced in scenes B1 and B2, when the layout of the associated word reaches the page boundary, or there is a writing track at the planned layout position, the downward line break layout process can be performed; the interface effect diagram after the downward line break layout process can be as follows Figure 5A or Figure 5B or Figure 6A or Figure 6B shown.

[0222] Another optional one is that, in combination with the content introduced in Scenario B3, in the pinyin association scenario, when the character spacing between the associated word and the adjacent Chinese character has a certain difference from the character spacing between other Chinese characters, the character spacing between the associated word and / or the Chinese characters of the handwritten text can be adjusted.

[0223] Another optional one is that, in combination with the content introduced in Scenario B4, the character spacing can be the horizontal character spacing, or the vertical character spacing, or a combination of the horizontal character spacing and the vertical character spacing, so as to achieve typesetting in various typesetting styles, such as horizontal typesetting style, vertical typesetting style, or inclined typesetting style, etc.

[0224] Another optional one is that when the character spacing is less than or equal to a preset threshold, the character spacing of the associated word can be adjusted, and the character spacing between the associated word and the handwritten text can also be adjusted, so as to reduce the trajectory overlap between Chinese characters. Exemplarily, the Chinese character trajectory, its bounding box, and its body box can be regarded as a whole, and the body box can be moved to adjust the character spacing. For example, Figure 15 FIG. 10 is an interface schematic diagram of a method for displaying an interface based on handwritten input provided by an embodiment of the present application. Figure 15 In the interface 1501 in FIG. 10, it is the interface display effect without adjusted character spacing, and there is a certain trajectory overlap between Chinese characters; Figure 15 In the interface 1502 in FIG. 10, it is the interface display effect with adjusted character spacing, and there is basically no trajectory overlap between Chinese characters.

[0225] Another optional one is that based on block processing, the text content included in an associated word usually belongs to the same text block. When the associated word needs to be typeset with a line break downward, the typesetting of the next line can also refer to the typesetting of the previous line. For example, Figure 16 FIG. 11 is another interface schematic diagram of a method for displaying an interface based on handwritten input provided by an embodiment of the present application. In Figure 16 in the interface 1601 in FIG. 11, the spacing of "Editing and management tools" in the next line can be displayed according to the spacing of "Huawei Notes is a professional note" in the previous line; among them, the spacing of "Editing and management tools" can be obtained based on the spacing of "Huawei Notes is a professional note", for example, the average value, or the weighted average value, or the average value / weighted average value after removing outliers can be used.

[0226] A method for interface display based on handwritten input provided by an embodiment of the present application can provide more functions for the handwritten input scenario; among them, the more functions can include at least one of the following functions: generating a trajectory of an associated word with the user's handwritten style, generating a layout of an associated word with the user's handwritten style, adjusting the layout of the associated word, balancing the spacing between the handwritten text and the Chinese characters included in the associated word, improving the association accuracy under mixed handwritten input including pinyin, etc. Therefore, this method can reduce the interaction time of handwritten input, improve the efficiency of handwritten input, and thus can improve the convenience and user experience of handwritten input.

[0227] Figure 17 It is a schematic flowchart of a method for interface display based on handwritten input provided by an embodiment of the present application. This method can be applied to an electronic device, and the process can include the following steps:

[0228] Step 1701, in response to the user's handwritten input operation, display a first interface. The first interface includes handwritten text and at least one associated word. The handwritten text includes at least one of the following types: Chinese characters, pinyin. The first interface can be, for example, Figure 4A interface 400A in Figure 4B or can be interface 400D in Figure 4C or can be interface 400G in Figure 5A or can be interface 500A and interface 500C in Figure 5B or can be interface 500E and interface 500G in Figure 6A or can be interface 600C in Figure 6B or can be interface 600G in

[0229] Step 1702, in response to the confirmation operation on the first associated word, display a second interface. The first associated word is one of the at least one associated word. Among them, the second interface includes the handwritten text and the first associated word, and the handwritten style of the first associated word is the same as the handwritten style when the user historically handwrote the first associated word. The second interface can be, for example, Figure 4A interface 400B in Figure 4B or can be interface 400E in Figure 4C or can be interface 400H in Figure 5A or can be interface 500B and interface 500D in Figure 5B or can be interface 500F and interface 500H in Figure 6A or can be interface 600D in Figure 6B or can be interface 600H in

[0230] Among them, the specific implementation of step 1701 and step 1702 can refer to the content described in the foregoing embodiments, and will not be elaborated here.

[0231] Based on the above embodiments, the present application further provides an electronic device, which includes a plurality of functional modules; the plurality of functional modules interact with each other to implement the functions performed by the electronic device in the methods described in the embodiments of the present application. The plurality of functional modules can be implemented based on software, hardware, or a combination of software and hardware, and the plurality of functional modules can be arbitrarily combined or divided based on the specific implementation. For example, execute Figure 9 Steps 901 to 905 executed by the electronic device in the illustrated embodiment.

[0232] Based on the above embodiments, the present application further provides an electronic device, which includes at least one processor and at least one memory. Computer program instructions are stored in the at least one memory. When the electronic device runs, the at least one processor executes the functions performed by the electronic device in the methods described in the embodiments of the present application. For example, execute Figure 9 Steps 901 to 905 executed by the electronic device in the illustrated embodiment.

[0233] Based on the above embodiments, the present application further provides a computer program product, which includes: a computer program (which can also be referred to as code, or instruction). When the computer program is run, the methods described in the embodiments of the present application are executed.

[0234] Based on the above embodiments, the present application further provides a computer-readable storage medium. Computer programs (which can also be referred to as code, or instructions) are stored in the computer-readable storage medium. When these codes or instructions are executed by a computer, the computer executes the methods described in the embodiments of the present application.

[0235] Based on the above embodiments, the present application further provides a chip. The chip is used to read the computer program stored in the memory to implement the methods described in the embodiments of the present application.

[0236] Based on the above embodiments, the present application provides a chip system, which includes a processor for supporting a computer device to implement the various methods described in the embodiments of the present application. In a possible embodiment, the chip system further includes a memory for storing necessary programs and data of the computer device. The chip system may be composed of chips or may include chips and other discrete devices. Those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) that include computer-usable program code.

[0237] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0238] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implements the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0239] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0240] Obviously, those skilled in the art can make various changes and modifications to this application without departing from the scope of protection of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to cover these modifications and variations.

Claims

1. A method for interface display based on handwritten input, characterized in that, Applied to an electronic device, including: In response to a user's handwritten input operation, display a first interface, the first interface including handwritten text and at least one associated word, the handwritten text including at least one of the following types: Chinese characters, pinyin; In response to a confirmation operation on the first associated word, display a second interface, the first associated word being one of the at least one associated word; wherein, the second interface includes the handwritten text and the first associated word, and the handwritten style of the first associated word is the same as the handwritten style when the user historically handwrote the first associated word.

2. The method according to claim 1, characterized in that, The displaying of the first interface in response to the user's handwritten input operation includes: In response to the handwritten input operation, obtain the handwritten text; the handwritten text includes pinyin; In response to an operation of converting the pinyin included in the handwritten text into Chinese characters, obtain the at least one associated word, the at least one associated word being Chinese characters obtained based on the pinyin.

3. The method according to claim 1 or 2, characterized in that, The first associated word in the second interface is displayed at a first display position and a second display position, the first display position displaying at least one of the front Chinese characters included in the first associated word, and the second display position displaying at least one of the remaining Chinese characters included in the first associated word; Wherein, there is a position sequence layout relationship between the first display position and the second display position; the position sequence layout relationship is obtained based on the handwritten text.

4. The method according to claim 3, characterized in that, The at least one of the front Chinese characters reaches the page boundary, or the at least one of the front Chinese characters reaches the display area of the existing writing trajectory.

5. The method according to claim 3 or 4, characterized in that, The position sequence layout relationship is a downward line break layout relationship.

6. The method according to any one of claims 1 to 5, characterized in that, In the second interface, the difference between the character spacing of the handwritten text and the character spacing of the first associated word is less than or equal to a first difference threshold.

7. The method according to claim 6, characterized in that, The character spacing of the handwritten text in the first interface is the same as or different from the character spacing of the handwritten text in the second interface.

8. The method according to claim 7, characterized in that, When the character spacing of the handwritten text in the first interface is different from the character spacing of the handwritten text in the second interface; The character spacing of the handwritten text in the first interface is less than a character spacing threshold, and the character spacing of the handwritten text in the second interface is greater than or equal to the character spacing threshold.

9. The method according to any one of claims 6 to 8, characterized in that, The character spacing is the character spacing in the target direction; wherein, the target direction is determined based on the handwritten direction of the handwritten text.

10. The method according to any one of claims 1 to 9, characterized in that, The difference between the font size of the handwritten text and the font size of the first associated word is less than or equal to a second difference threshold.

11. A method for interface display based on handwritten input, characterized in that, Applied to an electronic device, including: In response to a user's handwritten input operation, display a first interface, the first interface including handwritten text and at least one associated word; In response to a confirmation operation on the first associated word, a second interface is displayed, where the first associated word is one of the at least one associated word; wherein, the first associated word in the second interface is displayed at a first display position and a second display position, the first display position displays at least the first Chinese characters included in the first associated word, and the second display position displays at least the remaining Chinese characters included in the first associated word; wherein, there is a positional order layout relationship between the first display position and the second display position; the positional order layout relationship is obtained based on the handwritten text.

12. The method according to claim 11, characterized in that, The at least the first Chinese characters reach the page boundary, or the at least the first Chinese characters reach the display area of the existing writing trajectory.

13. The method according to claim 11 or 12, characterized in that, The positional order layout relationship is a line break layout relationship downward.

14. The method according to any one of claims 11 to 13, characterized in that, In the second interface, the difference between the character spacing of the handwritten text and the character spacing of the first associated word is less than or equal to a first difference threshold.

15. The method according to claim 14, characterized in that, The character spacing of the handwritten text in the first interface is the same as or different from the character spacing of the handwritten text in the second interface.

16. The method according to claim 15, characterized in that, When the character spacing of the handwritten text in the first interface is different from the character spacing of the handwritten text in the second interface; The character spacing of the handwritten text in the first interface is less than a character spacing threshold, and the character spacing of the handwritten text in the second interface is greater than or equal to the character spacing threshold.

17. The method according to any one of claims 14 to 16, characterized in that, The character spacing is the character spacing in the target direction; wherein, the target direction is determined based on the handwriting direction of the handwritten text.

18. The method according to any one of claims 11 to 17, characterized in that, The difference between the font size of the handwritten text and the font size of the first associated word is less than or equal to a second difference threshold.

19. The method according to any one of claims 11 to 18, characterized in that, The second interface includes the handwritten text and the first associated word, and the handwriting style of the first associated word is the same as the handwriting style when the user historically handwrote the first associated word.

20. The method according to any one of claims 11 to 19, characterized in that, The handwritten text includes at least one of the following types: Chinese characters, pinyin.

21. The method according to claim 20, characterized in that, The displaying of the first interface in response to the user's handwritten input operation includes: In response to the handwritten input operation, the handwritten text is obtained; the handwritten text includes pinyin; In response to the operation of converting the pinyin included in the handwritten text into Chinese characters, the at least one associated word is obtained, and the at least one associated word is Chinese characters obtained based on the pinyin.

22. An electronic device, characterized in that, It includes at least one processor, the at least one processor is coupled with at least one memory, and the at least one processor is configured to read and execute the computer program stored in the at least one memory, so that the method according to any one of claims 1 to 10 is executed, or so that the method according to any one of claims 11 to 21 is executed.

23. A computer-readable storage medium, characterized in that, Instructions are stored in the computer-readable storage medium, and when the instructions run on a computer, the computer is caused to execute the method according to any one of claims 1 to 10, or the computer is caused to execute the method according to any one of claims 11 to 21.

24. A computer program product comprising instructions, characterized in that, When the computer program product runs on a computer, the computer is caused to execute the method according to any one of claims 1 to 10, or the computer is caused to execute the method according to any one of claims 11 to 21.

Images