Method for converting handwritten notes into characters and texts and electronic equipment

Abstract

The invention provides a method for converting handwritten notes into characters and texts and electronic equipment, and relates to the technical field of terminals.The method comprises the steps that in response to operation of a user on a display screen, a selection range is obtained; obtaining a handwriting track in the selection range; identifying a first color of the handwriting track; identifying the handwriting track to obtain text content; determining a text format according to the number and the size of characters in the text content; and displaying the character text on the display screen in response to a confirmation operation of the user. Thus, in the process of converting the handwritten notes into the text, the format and the color corresponding to the handwritten track can be obtained, then the text is generated according to the format and the color corresponding to the handwritten track, format characteristics generated by handwriting are reflected, and the sorting efficiency of the handwritten notes is improved.

Description

Technical Field

[0001] This application relates to the technical field of terminals, and particularly to a method for converting handwritten notes into text and an electronic device. Background Art

[0002] In the trend of paperless office, using a stylus in scenarios such as meetings and classrooms can record content on an electronic device faster. However, when the note content recorded on the electronic device is to be shared with other electronic devices or organized into a word document, it can only be achieved by inserting pictures or manually entering the content according to the recorded content, and the operation is very cumbersome.

[0003] Therefore, in order to improve the efficiency of handwriting-to-text conversion, the electronic device can convert handwritten text into a general format font recognizable by the electronic device through an internal recognition system and display it on the screen. However, in the process of handwriting-to-text conversion, a general format is used for text conversion, and formats such as font size and color during the handwriting process are discarded, affecting the generation of the converted text. Summary of the Invention

[0004] This application provides a method for converting handwritten notes into text and an electronic device to solve the problem that the conversion of handwritten trajectories does not retain formats such as font size and color, thereby affecting the generation of text.

[0005] According to the first aspect in the embodiments of this application, a method for converting handwritten notes into text is provided, which is applied to an electronic device with a display screen. The method includes: in response to a user's operation on a handwriting area, obtaining a selection range in the handwriting area; obtaining the handwritten trajectory within the selection range; identifying the first color of the handwritten trajectory; identifying the handwritten trajectory to obtain text content; determining the text format according to the number of characters and the font size in the text content; in response to the user's confirmation operation, displaying the text on the display screen; the text includes text content with a font format conforming to the text format and a font color of the first color.

[0006] Based on the technical solution provided by this application, it is possible to identify information such as color and font size in the handwritten trajectory during the conversion of the handwritten trajectory, so that the generated text corresponds to the characteristics of the handwritten trajectory, avoiding the loss of characteristics in the handwritten trajectory and improving the efficiency of organizing handwritten notes.

[0007] In a feasible implementation manner, the number of characters includes the number of characters, the number of lines of the text, and the number of characters in each line of the text content; the font size includes the length and width of each character in the text content; the text format includes the font size and line spacing. In this way, by obtaining and determining information such as the number of characters in the text content, it is possible to retain formats such as color and font size in the handwritten trajectory.

[0008] In a feasible implementation manner, according to the number of characters and the character size in the text content, determine the text format, including: calculating the first font size corresponding to each line of text in the text content respectively according to the number of lines of the text content, the number of characters in each line of text, and the length of the characters in each line of text; or calculating the second font size corresponding to each line of text in the text content respectively according to the number of characters in the text content and the length and width of each character; determining the first line spacing according to the line spacing between each line of text in the text content; or determining the second line spacing according to the number of lines of the text content and the length of the characters in each line of text. In this way, the format of the generated text can be restricted by factors such as font size and quantity, so that the format of the generated text is similar to the format of the handwritten trajectory, improving the efficiency of organizing handwritten notes and reducing the loss of handwritten features during the conversion of handwritten notes.

[0009] In a feasible implementation manner, after determining the text format according to the number of characters and the character size in the text content, it further includes: displaying a first interface on the display screen; the first interface includes a text display area and an interface control, the text display area displays the text content, and the interface control displays the text format. In response to the user's confirmation operation, display the text on the display screen, including: in response to the user's confirmation operation on the first interface, display a text box on the display screen; the text box includes the text, the size of the text box is the same as the selected range, and the format of the text is the same as the text format displayed in the interface control. In this way, the user can also preview and confirm the recognition result during the recognition process, improving the user experience.

[0010] In a feasible implementation manner, after recognizing the handwritten trajectory to obtain the text content, it further includes: determining the character type of each character in the text content according to the text content; the character type includes Chinese characters, letters, numbers, and symbols. After calculating the font size corresponding to the text content, it further includes: if the character type of the first character is a letter, number, or symbol, determine the font size of the second character adjacent to the first character as the font size of the first character; the first character is any character in the text content, and the second character includes the characters with the character type of Chinese characters in the text content. In this way, by judging the character type of the characters, a more appropriate font size can be determined for the non-Chinese character type characters, avoiding the problem that the font size of letters, numbers, symbols and other characters is too small due to the influence of their own writing size.

[0011] In a feasible implementation manner, in response to a confirmation operation of a user, literal text is displayed on a display screen, and it further includes: modifying the text content in a text display area in response to a first modification instruction of the user for a first interface; and / or modifying the text format in an interface control in response to a second modification instruction of the user for the first interface. In this way, the user can modify the recognition result during the recognition process, making the subsequently generated literal text more in line with the user's needs and reducing the time required for text arrangement.

[0012] In a feasible implementation manner, if the first color includes at least two colors, after recognizing a handwriting trace to obtain text content, it further includes: obtaining the first color corresponding to each character in the text content according to the correspondence between the first color and the handwriting trace. In this way, when the selection range includes multiple colors, the colors of the characters in the literal text can be determined respectively according to the colors of different traces, thereby retaining the color characteristics in the handwriting trace.

[0013] In a feasible implementation manner, recognizing a handwriting trace to obtain text content includes: determining the trajectory coordinates corresponding to the handwriting trace within a selected range; performing single-character segmentation on the handwriting trace to obtain single-character text; disassembling the single-character text into stroke points according to the trajectory coordinates; and performing stroke recognition on the disassembled stroke points to obtain text content. In this way, the handwriting trace can be recognized to obtain the text content corresponding to the handwriting trace.

[0014] In a feasible implementation manner, performing stroke recognition on the disassembled stroke points to obtain text content includes: determining the feature vector of the handwriting trace in each single-character text according to the stroke points and the trajectory coordinates corresponding to the stroke points; obtaining the trajectory feature according to the feature vector; and determining the text content corresponding to each single-character text according to the trajectory feature of each single-character text. In this way, each single-character text can be recognized through the stroke points, so as to obtain the characters corresponding to each single-character text, and further obtain the text content corresponding to the handwriting trace through the recognition results of all single-character texts.

[0015] In a feasible implementation, the electronic device includes a note display module and a handwriting recognition module. The method includes: the handwriting recognition module obtains a selection range within the display range of the note display module in response to a user's operation on the handwriting area; reads the handwriting trajectory within the selection range through the handwriting recognition module and identifies the first color of the handwriting trajectory; the handwriting recognition module recognizes the handwriting trajectory to obtain text content, and determines the text format according to the number of characters and the character size in the text content; the handwriting recognition module sends the text content and the text format to the note display module; obtains the user's confirmation operation through the note display module, and the handwriting recognition module sends the text content in the text format and the first color to the note display module; the note display module displays the text on the display screen. In this way, it is possible to utilize the interaction between the note display module and the handwriting recognition module to identify information such as the color and font size in the handwriting trajectory, so as to correspond the characteristics of the generated text with the handwriting trajectory, and improve the efficiency of organizing handwritten notes.

[0016] According to a second aspect of the embodiments of the present application, there is provided an electronic device, including: a display screen, a memory, and one or more processors; the display screen and the memory are coupled to the processor; wherein, computer program code is stored in the memory, and the computer program code includes computer instructions. When the computer instructions are executed by the processor, the electronic device executes the method for converting handwritten notes into text as described in any one of the previous embodiments.

[0017] It can be understood that for the beneficial effects that can be achieved by the technical solution provided in the above-mentioned second aspect, reference can be made to the beneficial effects in the first aspect and any feasible implementation thereof, which will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic diagram of the effect of converting handwritten content into text;

[0019] Figure 2 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application;

[0020] Figure 3 It is a schematic diagram of the hierarchical architecture of a software system of an electronic device in an embodiment of the present application;

[0021] Figure 4 It is a schematic diagram of the interaction of a software system of an electronic device in an embodiment of the present application;

[0022] Figure 5 It is a schematic diagram of the flow of a method for converting handwritten notes into text in an embodiment of the present application;

[0023] Figure 6 It is a schematic diagram of responding to a user's operation on the display screen in an embodiment of the present application;

[0024] Figure 7 It is a schematic flow diagram of a handwriting trajectory recognition in an embodiment of the present application;

[0025] Figure 8 It is a schematic diagram of a single-character segmentation in an embodiment of the present application;

[0026] Figure 9 It is a schematic diagram of another single-character segmentation in an embodiment of the present application;

[0027] Figure 10 It is a schematic flow diagram of a process for displaying a text in an embodiment of the present application;

[0028] Figure 11 It is a schematic diagram of a display of a handwritten note converted into a text in an embodiment of the present application;

[0029] Figure 12 It is a schematic diagram of an operation on a first interface in an embodiment of the present application;

[0030] Figure 13 It is a schematic diagram of a system for converting a handwritten note into a text in an embodiment of the present application;

[0031] Figure 14 It is a schematic flow diagram of a conversion system executing a conversion method in an embodiment of the present application. Detailed implementation manners

[0032] Next, the technical solutions in the embodiments of the present application will be clearly described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all embodiments. Based on the embodiments of the present application, other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0033] Hereinafter, terms such as "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first", "second", etc. may explicitly or implicitly include one or more of such features. In the description of the present application, unless otherwise stated, the meaning of "a plurality" is two or more.

[0034] In addition, in the present application, orientation terms such as "upper" and "lower" are defined relative to the orientation of the components shown in the drawings. It should be understood that these directional terms are relative concepts, which are used for relative description and clarification, and they can change accordingly with the change of the orientation of the components placed in the drawings.

[0035] In the trend of paperless office, using electronic devices for content recording has become a common way of meeting minutes. In some scenarios that require text recording, such as meetings and classrooms, using a stylus can record content on electronic devices faster. However, for the note content recorded on an electronic device, if you want to share it to other electronic devices or organize it into a word document, it can only be achieved by inserting pictures or manually entering the content according to the recorded content. The subsequent sorting time is long and the operation is cumbersome.

[0036] In order to improve the efficiency of handwriting-to-text conversion, an electronic device can convert handwritten notes into text in a general format that can be read by the electronic device and display it on the display screen by setting up a handwriting recognition system internally. Since there are situations where the content characteristics are reflected through formats such as font size and font color during the handwriting process of handwritten notes, for example, emphasizing content by changing the text color to other colors, or reflecting characteristics by increasing or decreasing the text size. After converting the handwritten notes into text that can be read by the electronic device, a general format will overwrite the formats such as font size and font color in the handwritten notes, making the generated text unable to reflect the emphasized or marked content during the writing process of the handwritten notes, affecting the generation of the text.

[0037] It should be understood that the general format and other text formats in this application are the specific formats of the text that users can see through electronic devices, such as formats that limit the font shape such as Song typeface and boldface, formats that limit the font size such as No. 4 and small No. 4, and formats that limit the font color such as black and red. This application does not limit the specific types in the text format.

[0038] Figure 1 It is a schematic diagram of the effect of converting handwritten content into text.

[0039] As Figure 1 shown in (a), the user can write on the screen of the electronic device through a stylus or other devices to form a handwriting trajectory. When it is necessary to convert the handwriting trajectory, the electronic device can identify the handwriting trajectory to obtain the corresponding text of the handwriting trajectory.

[0040] After the recognition is completed, as Figure 1 shown in (b), the electronic device will display the recognized text in the general format set by the electronic device on the screen, thus realizing the conversion of the handwriting trajectory into text that the electronic device can recognize. However, during the process of converting the handwriting trajectory by the electronic device, the format of the generated text can only adopt the format preset in the electronic device, or after the text conversion is completed, the user operates on the generated text again, reducing the efficiency of sorting the handwriting trajectory, prolonging the user's sorting time, and affecting the user experience.

[0041] In view of the above problems, an embodiment of the present application provides a method for converting handwritten notes into text, which is applied to an electronic device with a display screen. In this technical solution, before recognizing the handwritten trajectory, the color of the handwritten trajectory can be extracted, and during the recognition process, formats such as the font size and spacing of the handwritten trajectory itself are retained, so as to retain the content characteristics, improve the sorting efficiency of the handwritten trajectory, reduce the subsequent operations of the user, and improve the user experience.

[0042] Next, the implementation manners of the embodiments of the present application will be described in detail with reference to the accompanying drawings.

[0043] In an embodiment of the present application, the user can perform handwritten input on the touch screen of the electronic device through a stylus, or can perform handwritten input by connecting an external device such as a graphics tablet to the electronic device. Therefore, the electronic device in the embodiment of the present application can be any electronic device with a handwriting function or capable of connecting to a handwriting peripheral device. For example, electronic devices such as smart phones, tablet computers, laptop computers, personal computers (PCs), and personal digital assistants (PDAs) are not limited in the embodiments of the present application.

[0044] Exemplarily, taking the electronic device as a tablet computer as an example, Figure 2 FIG. is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

[0045] As Figure 2 shown, the electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a radio frequency circuit 140, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a display screen 193, a subscriber identification module (SIM) card interface 194, and a camera 195, etc. Among them, the sensor module 180 may include a pressure sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.

[0046] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0047] The controller may be the nerve center and command center of the electronic device. The controller may generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0048] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory may save the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0049] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0050] The memory 120 can be used to store computer-executable program code, and the executable program code includes instructions. The memory 120 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.). The data storage area can store data created during the use of the electronic device (such as audio data, phone book, etc.). In addition, the memory 120 can include one or more storage units. For example, it can include volatile memory, such as dynamic random access memory (DRAM), static random access memory (SRAM), etc.; it can also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, etc. The processor 110 executes various functional application programs and data processing of the electronic device by running the instructions stored in the memory 120 and / or the instructions stored in the memory provided in the processor.

[0051] The wireless communication function of the electronic device can be implemented by the radio frequency circuit 140, the mobile communication module 150, the wireless communication module 160, the modem, and the baseband processor, etc.

[0052] The radio frequency circuit 140 can include antennas 141 and 142 for transmitting and receiving electromagnetic wave signals. Each antenna in the electronic device can be used to cover single or multiple communication frequency bands. In some embodiments, the antenna can be used in combination with a tuning switch, and different antennas can also be multiplexed to improve the utilization rate of the antenna.

[0053] The mobile communication module 150 can provide solutions for wireless communications such as 2G / 3G / 4G / 5G applied to the electronic device. The mobile communication module 150 can receive electromagnetic waves by the antenna 141, and perform processing such as filtering and amplifying on the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves through the antenna 141 and radiate it out. In some embodiments, at least some functional modules of the mobile communication module 150 can be provided in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 can be provided in the same device.

[0054] The modulation and demodulation processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Subsequently, the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.), or displays an image or video through the display screen 193. In some embodiments, the modulation and demodulation processor may be an independent device. In other embodiments, the modulation and demodulation processor may be independent of the processor 110 and be provided in the same device as the mobile communication module 150 or other functional modules.

[0055] The wireless communication module 160 may include a wireless fidelity (Wi-Fi) module, a bluetooth (BT) module, a GNSS module, a near field communication (NFC) module, an infrared (IR) module, etc. The wireless communication module 160 may be one or more devices integrating at least one of the above modules. The wireless communication module 160 receives electromagnetic waves via the antenna 141, performs frequency modulation and filtering processing on the electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 may also receive the signal to be transmitted from the processor 110, perform frequency modulation and amplification on it, and convert it into electromagnetic waves through the antenna 141 for radiation.

[0056] The display screen 193 is used to display images, videos, etc. The display screen 193 includes a display panel. The display panel may adopt a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device may include one or N display screens 193, where N is a positive integer greater than 1.

[0057] A touch sensor, also known as a "touch control device". The touch sensor can be coupled with the display screen 193, so that the touch sensor and the display screen 193 form a touch screen, also known as a "touch control screen". The touch sensor is used to monitor touch operations acting on or near it. The touch sensor can transmit the monitored touch operations to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 193. In some other embodiments, the touch sensor can also be disposed on the surface of the electronic device, at a different position from that of the display screen 193.

[0058] The pressure sensor is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor can also be coupled with the display screen 193. There are many types of pressure sensors, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. When a touch operation acts on the display screen 193, the electronic device monitors the intensity of the touch operation according to the pressure sensor. The electronic device can also calculate the position of the touch according to the monitoring signal of the pressure sensor. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions.

[0059] In the embodiments of the present application, the touch sensor and / or the pressure sensor coupled with the display screen 193 can record and respond to operations such as clicking, swiping, and writing of the user on the display screen 193. For example, when the user writes on the display screen 193 with a finger or a stylus, the touch sensor and / or the pressure sensor can record the writing trajectory of the user and display the writing trajectory in real time through the display screen 193, so as to realize the process of stroke drawing through the electronic device.

[0060] It can be understood that the interface connection relationship between the modules illustrated in the embodiments of the present invention is only illustrative and does not constitute a structural limitation on the electronic device. In some other embodiments of the present application, the electronic device can also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods. At the same time, the above Figure 2 is only an exemplary illustration when the form of the electronic device is a tablet computer. When the electronic device is in other device forms such as a mobile phone, a handheld computer, a PC, a PDA, a wearable device (such as a smart watch, a smart bracelet), etc., the structure of the electronic device can include less structure than that shown in Figure 2 or can include more structure than that shown in Figure 2 , which is not limited herein.

[0061] It can be understood that, generally speaking, in addition to the support of hardware, the realization of the functions of an electronic device also requires the cooperation of software. The software system of an electronic device can adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. In the embodiments of the present application, taking the system as an example, the software structure of the electronic device will be exemplarily described.

[0062] Figure 3 FIG. is a schematic diagram of the layered architecture of a software system of an electronic device in the embodiments of the present application. The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. The communication between layers is through software interfaces (such as APIs).

[0063] In some examples, as Figure 3 shown, in the embodiments of the present application, the software of the electronic device is divided into five layers, from top to bottom are the application layer, the framework layer (or called the application framework layer), the system library and Android Runtime, the HAL layer (hardware abstraction layer), and the driver layer (or called the kernel layer). Among them, the system library and Android Runtime can also be called the native framework layer or the native layer.

[0064] Among them, the application layer may include a series of applications. As Figure 3 shown, the application layer may include applications such as camera, gallery, calendar, map, WLAN, Bluetooth, music, video, short message, call, navigation, instant messaging, etc. (application, APP).

[0065] The framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions or services. For example, the application framework layer may include an activity manager, a window manager, a content provider, an audio service, a view system, a telephone manager, a resource manager, a notification manager, a package manager, etc., and the embodiments of the present application do not make any restrictions on this.

[0066] The window manager is used to manage window programs. The window manager can obtain the display screen size, judge whether there is a status bar, lock the screen, capture the screen, etc.

[0067] The content provider is used to store and obtain data, and make these data accessible to applications. These data may include videos, images, audio, dialed and received calls, browsing history and bookmarks, phone books, etc.

[0068] The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. The display interface can be composed of one or more views. For example, a display interface including a text message notification icon can include a view for displaying text and a view for displaying pictures.

[0069] The telephone manager is used to provide the communication function of the electronic device. For example, the telephone manager can manage the call status of the call application (including initiation, connection, hanging up, etc.).

[0070] The resource manager provides various resources for the application, such as localized strings, icons, pictures, layout files, video files, and so on.

[0071] The notification manager enables the application to display notification information in the status bar. It can be used to convey notification-type messages, which can automatically disappear after a short stay without user interaction. For example, the notification manager is used to inform that the download is complete, message reminder, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as the notification of a background running application, and can also be a notification that appears on the screen in the form of a dialogue window. For example, prompt text information in the status bar, emit a prompt sound, the electronic device vibrates, the indicator light flashes, etc.

[0072] The package manager is used to manage application packages in the system. It allows applications to obtain detailed information about installed applications and their services, permissions, etc. The package manager is also used to manage events such as the installation, uninstallation, and upgrade of applications.

[0073] The system library can include multiple functional modules. For example: surface manager, Media Libraries, OpenGL ES, SGL, etc. The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications. The media library supports the playback and recording of multiple common audio and video formats, as well as static image files, etc. The media library can support multiple audio and video coding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. OpenGL ES is used to implement three-dimensional graphic drawing, image rendering, synthesis, and layer processing, etc. SGL is a drawing engine for 2D drawing.

[0074] The Android Runtime includes the core libraries and the ART virtual machine. The Android Runtime is responsible for the scheduling and management of the Android system. The core libraries consist of two parts: one part is the functional functions that the Java language needs to call, and the other part is the core libraries of Android. The application layer and the application framework layer run in the ART virtual machine. The ART virtual machine executes the Java files of the application layer and the application framework layer as binary files. The ART virtual machine is used to perform functions such as the management of object life cycles, stack management, thread management, security and exception management, and garbage collection.

[0075] The HAL layer is an interface layer located between the operating system kernel and the hardware circuit. Its purpose is to abstract the hardware. It hides the hardware interface details of a specific platform and provides a virtual hardware platform for the operating system, making it hardware-independent and portable across multiple platforms. The HAL layer provides a standard interface to display the device hardware functions to a higher-level Java API framework (i.e., the framework layer). The HAL layer contains multiple library modules, and each module implements an interface for a specific type of hardware component. For example: the audio HAL audio module, the bluetooth HAL bluetooth module, the camera HAL camera module (which can also be called the camera HAL or the camera hardware abstraction module), the sensors HAL sensor module (or called the Isensor service, sensor service).

[0076] The kernel layer is the layer between the hardware and the software. The kernel layer includes at least a display driver, a camera driver, an audio driver, a sensor driver, a battery driver, etc., which are not limited in this application. Among them, the sensor driver can specifically include the drivers for each sensor included in the electronic device, such as a pressure sensor driver. Exemplarily, the pressure sensor driver can send the detection data of the pressure sensor to the sensor module 180 in a timely manner in response to an indication or instruction from the sensor module 180 to obtain detection data.

[0077] In the embodiments of this application, the application layer may include a note display module (NotePad), and the framework layer may also include a handwriting recognition module (PenKit).

[0078] Figure 4 This is an interaction schematic diagram of a software system of an electronic device in the embodiments of this application.

[0079] As Figure 4 shown, after receiving a user operation through the note display module, the handwriting recognition module can obtain each handwriting trajectory within the selected range displayed in the note display module.

[0080] Specifically, during the process of drawing the handwriting trajectory, the handwriting recognition module can obtain the drawn handwriting trajectory through the resource manager, and after receiving the user's operation on the display screen 193, read out the handwriting trajectory within the selected range. In some embodiments of the present application, the handwriting trajectory drawn by the note display module can be stored in the content provider, and then after the handwriting recognition module receives the user's operation, it can obtain the handwriting trajectory within the selected range in the content provider according to the user's operation, and then perform subsequent operations on the handwriting trajectory.

[0081] After the handwriting recognition module reads the handwriting trajectory within the selected range, it can perform color recognition on the drawn handwriting trajectory. After the recognition is completed, it can recognize the obtained handwriting trajectory to obtain the content of the text corresponding to the handwriting trajectory, and determine the text format according to the number and size of the characters in the obtained text content.

[0082] After obtaining the text content and text format corresponding to the handwriting trajectory, the handwriting recognition module can send the obtained text content and text format to the note display module, and then display them through the recognition result display window of the note display module. When the note display module receives the user's confirmation operation through the recognition result display window, the handwriting recognition module will generate a text box according to the text content, text format and recognized color and send it to the note display module, and then display the text box on the display screen 193 through the note display module.

[0083] In some embodiments, the handwriting recognition module can also directly perform a read operation on all the drawn handwriting trajectories after a preset time after the handwriting trajectory is drawn, so as to perform subsequent operations.

[0084] Based on the above Figure 2 structure of the electronic device shown in Figure 3 and the software architecture shown in Figure 5 the following will exemplarily illustrate the method for converting handwritten notes into text according to the present application in combination with the content in

[0085] Figure 5 is a schematic flowchart of a method for converting handwritten notes into text according to an embodiment of the present application.

[0086] In the embodiment of the present application, the user can recognize the handwriting trajectory through the electronic device, and then obtain information such as its color and font size, and generate a text corresponding to the handwriting trajectory. As Figure 5 shown, the method includes:

[0087] S100: In response to the user's operation on the handwriting area, obtain a selected range in the handwriting area.

[0088] The electronic device can display the handwritten trajectory through the display screen 193. Among them, the handwritten trajectory can be the trajectory formed by the user writing on the display screen 193, or it can be sent to the electronic device for trajectory recognition and conversion by other electronic devices. In the embodiments of the present application, the source of the handwritten trajectory is not limited.

[0089] Taking the process of the user writing the trajectory and subsequent converted text on the same electronic device as an example, when the user actually writes on the display screen 193, the user will use an external handwriting device or other means to write on the display screen 193 with the handwriting device, so as to improve the writing speed. And connecting an external handwriting device to the electronic device can improve the clarity of the handwritten trajectory and the recognition efficiency.

[0090] In the embodiments of the present application, the handwriting device can be connected to the electronic device by establishing a wireless connection with the wireless communication module 160. Exemplarily, the handwriting device can be provided with a Bluetooth module, so as to cooperate with the Bluetooth function in the wireless communication module 160 to realize the connection between the handwriting device and the electronic device. The handwriting device can also be connected to the electronic device through the universal serial bus interface 130. The present application does not limit the connection method between the handwriting device and the electronic device.

[0091] It should be understood that in order to collect and display the handwritten trajectory, the user can write in the designated area of the display screen 193. Exemplarily, the designated area can be the handwriting area of the display screen 193, and the display area of the handwriting area is smaller than the area of the display screen 193, so as to reserve a position for the controls in the electronic device, and further realize the control of the handwritten trajectory.

[0092] Figure 6 This is a schematic diagram of responding to the user's operation on the display screen in the embodiments of the present application.

[0093] As Figure 6 shown, operation controls can be displayed above the display screen 193. The operation controls can be used to wake up functions such as drawing, erasing, modifying colors, lasso selection, insertion, etc. By clicking on the operation controls, the user can wake up the corresponding functions of the controls, and then the user can implement the above functions through operations.

[0094] Taking lasso selection as an example, the user can activate the function by clicking on the control corresponding to lasso selection, and then slide from one point to another point in the handwriting area to determine a rectangular box with these two points as the diagonals as the selection box 1931.

[0095] It should be noted that the above lasso selection function is only a feasible implementation method in the embodiments of the present application. The controls in the embodiments of the present application can also include other functions, and the present application does not limit the functions that can be woken up by the operation controls in the present application.

[0096] S200: Obtain the handwritten trajectory within the selected range.

[0097] Before the user converts the handwritten note, the user can operate on the display screen 193 to generate an area where the handwritten note needs to be converted. For example, Figure 6 As shown, the user can slide on the display screen 193 to generate a selection box 1931 on the display screen 193. In the embodiment of the present application, the electronic device can generate a rectangular selection box 1931 through the starting point and the ending point of the user's slide, so as to read the handwritten trajectory within the selection box 1931.

[0098] It should be understood that the rectangular selection box 1931 in the embodiment of the present application is only an exemplary illustration. In the actual selection process, other shapes of the selection box 1931 can also be obtained through other methods, and the present application does not limit the specific shape of the selection box 1931.

[0099] For example, Figure 6 As shown, the range included in the selection box 1931 is the selected range where the handwritten trajectory needs to be read in the present application. Therefore, after the user operates on the display screen 193 and the selection box 1931 is displayed on the display screen 193, the handwritten trajectory within the selected range can be obtained. Specifically, all the handwritten trajectories within the selection box 1931 can be extracted to obtain the handwritten trajectory within the selected range.

[0100] S300: Identify the first color of the handwritten trajectory.

[0101] In some embodiments, during the recognition process of the handwritten trajectory, the color on the handwritten trajectory may interfere with the recognized text content. Therefore, during the recognition process, the color of the handwritten trajectory will be removed through binarization processing and other methods. Therefore, in the embodiment of the present application, before recognizing and obtaining the text content, it is necessary to extract the color of the handwritten trajectory to facilitate subsequent color assignment to the converted text.

[0102] Before extracting the color, the background can be filtered first to reduce the influence of the background color on the color of the handwritten trajectory. In some embodiments, the RGB (Red Green Blue) values of the pixel points corresponding to the handwritten trajectory can be read, and the color of all the handwritten trajectories within the selected range can be obtained by taking the average value. It should be understood that the above method is only applicable to the case where the handwritten trajectory within the selected range includes one color. When the handwritten trajectory within the selected range includes two or more colors, calculating the average value of the pixel points will result in inaccurate color reading. Two or more groups of pixel points can be obtained through the difference in RGB values between different pixel points, and the number of groups of pixel points is the same as the number of color types in the handwritten trajectory. Then, the average calculation of the RGB values of each group of pixel points is performed respectively.

[0103] When the number of types of the colors of the handwritten trajectories within the selected range is greater than 1, that is, when the first color recognized according to the handwritten trajectories includes at least two colors, the first color corresponding to each character in the text content can be obtained according to the correspondence between the first color and the handwritten trajectories.

[0104] It should be understood that color extraction through RGB values is only a feasible implementation manner in the embodiments of the present application. Other color recognition methods can also be applied in the present application, and the present application does not limit this.

[0105] After obtaining the colors of the handwritten trajectories, the handwritten trajectories can be recognized by a functional module in the electronic device framework layer. Exemplarily, during the process of recognizing the handwritten trajectories, information such as the shape and size of the trajectories in the handwritten trajectories can be analyzed, and models such as maximum likelihood or Bayesian can be applied for classification; features of the handwritten trajectories can also be extracted, and classification can be performed through the feature vectors generated by the feature extraction; deep learning methods can also be used to extract features and realize the recognition of the trajectories. In the embodiments of the present application, any one of the above methods can be used to recognize the handwritten trajectories, and the present application does not limit the specific recognition method of the handwritten trajectories.

[0106] It should be understood that the methods for recognizing the handwritten trajectories are not limited to the above methods. Other methods for recognizing the handwritten trajectories can also be adopted in the present application, and in some embodiments of the present application, multiple methods can be combined to improve the accuracy of recognizing the handwritten trajectories.

[0107] The following Figure 7 will detail one method for recognizing the handwritten trajectories.

[0108] Figure 7 is a schematic flowchart of a method for recognizing a handwritten trajectory in an embodiment of the present application.

[0109] Taking the extraction of features of the handwritten trajectories as an example, as Figure 7 shown, the process of recognizing the handwritten trajectories may include:

[0110] S410: Determine the trajectory coordinates corresponding to the handwritten trajectories within the selected range.

[0111] In the embodiments of the present application, the trajectory coordinates may be the coordinates of each pixel point in the handwritten trajectory relative to a point within the selected range, so as to obtain the positional relationship of the pixel points in each handwritten trajectory.

[0112] In some embodiments, since the number of pixel points in the handwritten trajectory is large, it will affect the recognition speed of the trajectory coordinates, and thus affect the recognition speed of the handwritten trajectory. Therefore, a sampling method can be adopted to obtain some pixel points as sampling points, and obtain the coordinates of the sampling points, thereby improving the recognition speed of the trajectory coordinates.

[0113] Exemplarily, the handwritten trajectory and the background can be first grayscaled and binarized, and then it is determined whether the pixel points within N×N pixel points are the pixel points corresponding to the handwritten trajectory, and the average value of the trajectory coordinates of the handwritten trajectory pixel points among them is taken as the trajectory coordinates within this range. Where N is an integer greater than 0. When N = 1, no sampling is performed. When N = 2, sampling is performed among the four pixel points of 2×2, and the average value of the trajectory coordinates of the handwritten trajectory pixel points among the four pixel points is calculated. Therefore, the number of pixel points participating in the calculation is less than or equal to N 2 pieces, and the same applies when N is other values.

[0114] In this application, there is no limit on the sampling range, but it should be understood that when the sampling range is too large, pixel points in different handwritten trajectories will be repeatedly sampled, affecting the accuracy of subsequent recognition.

[0115] In some embodiments of this application, if the handwritten trajectory is drawn through the display screen 193 of the electronic device, the electronic device can also generate and record the drawing time of each trajectory through the pressure sensor when the user draws the handwritten trajectory, so as to distinguish the handwritten trajectories, thereby facilitating the judgment of the trajectory coordinates in the mutually intersecting handwritten trajectories, improving the sampling efficiency of the pixel points, and being able to improve the subsequent stroke decomposition and character recognition efficiency.

[0116] S420: Segment the handwritten trajectory into single characters to obtain single-character texts.

[0117] After obtaining the trajectory coordinates of each handwritten trajectory, it can be determined whether any two handwritten trajectories belong to the same character according to the trajectory position relationship and the trajectory overlap degree in each handwritten trajectory. The trajectory position relationship is used to describe the distance between trajectories, and the trajectory overlap degree is used to represent the overlap degree of the projections of different trajectories in a certain direction.

[0118] It should be understood that in actual handwritten trajectory recognition, the handwritten trajectory may include trajectories corresponding to characters such as Chinese characters, letters, numbers, symbols, etc. Compared with Chinese characters, the structures of characters such as letters, numbers, and symbols are relatively simple. In the recognition process, the handwritten trajectories corresponding to letters, numbers, and symbols can be segmented through the trajectory position relationship and the overlap degree to obtain single-character texts.

[0119] Figure 8Schematic diagram of single-character segmentation in an embodiment of this application; Figure 9 Schematic diagram of another single-character segmentation in an embodiment of this application.

[0120] In written languages such as Chinese characters, since a single character has multiple strokes, there are many handwriting trajectories in the single-character text of Chinese characters. Therefore, there will be problems such as missed judgment and misjudgment only by using the relationship between the trajectory positions and the degree of trajectory overlap. Therefore, in some embodiments of this application, as Figure 8 shown, during the process of single-character segmentation of the handwriting trajectory, the horizontal direction of the handwriting trajectory can be statistically analyzed to obtain a horizontal projection, so as to obtain the upper and lower boundaries of each line of text. Then, by statistically analyzing the vertical direction of the handwriting trajectory to obtain a vertical projection, and combining the horizontal projection, the handwriting trajectory can be segmented into single characters.

[0121] It should be understood that during the above-mentioned single-character segmentation process, the drawing direction of the handwriting trajectory is from left to right, that is, the text corresponding to the handwriting trajectory is arranged horizontally. And since the text corresponding to the handwriting trajectory is also arranged horizontally in the handwriting trajectory with the drawing direction from right to left. Therefore, the above single-character segmentation method is also applicable to the handwriting trajectory with the drawing direction from right to left.

[0122] However, when the drawing direction of the handwriting trajectory is from bottom to top or from top to bottom, the text corresponding to the handwriting trajectory is arranged vertically, and the text corresponding to the handwriting trajectories in different columns cannot be grouped into the same row. Therefore, when performing single-character segmentation, the above single-character segmentation method will have a situation where it cannot be segmented. As Figure 9 shown, in some embodiments of this application, if the text corresponding to the handwriting trajectory is arranged vertically, the vertical direction of the handwriting trajectory can be statistically analyzed to obtain a vertical projection, so as to obtain the left and right boundaries of each column of text. Then, by statistically analyzing the horizontal direction of each column of handwriting trajectories to obtain a horizontal projection, and combining the horizontal projection with the vertical projection, the vertically arranged handwriting trajectory can be segmented into single characters.

[0123] In some embodiments, since there may be connectivity between Chinese characters and the trajectory formed by Chinese characters may have a complex internal structure, which affects subsequent trajectory recognition. Therefore, before performing the vertical direction statistics, the handwriting trajectory can be subjected to a closing operation to expand the gap between the handwriting trajectories representing Chinese characters. Among them, the closing operation includes a dilation operation and an erosion operation, and the dilation operation is performed before the erosion operation. Through the closing operation, the loss of features of the handwriting trajectory during the single-character segmentation process can be avoided, and the accuracy of subsequent recognition can be improved.

[0124] In an embodiment of the present application, after the single-word segmentation is completed, the distance between the upper and lower limits of each line of handwriting trajectories obtained by horizontal statistics of the handwriting trajectories can be used as the line height of each line of text, and the distance between the left and right limits of each single-word text obtained by vertical statistics of each line of handwriting trajectories can be used as the width of each single-word text, which is convenient for subsequent calculation of the font size.

[0125] S430: Decomposing the stroke points of the single-word text according to the trajectory coordinates.

[0126] After obtaining the single-word text, the stroke points in the single-word text can be disassembled through the trajectory coordinates to obtain the stroke direction and stroke length in the single-word text, as well as the relationship between different strokes in the single-word text.

[0127] Since the speed of generating the feature vector corresponding to the trajectory is related to the number of collected feature coordinates, the more feature coordinates there are, the slower the decomposition speed is. In addition, in the same trajectory, most of the trajectory coordinates are redundant data, which affects the recognition speed of the handwritten trajectory by the electronic device.

[0128] Exemplarily, the stroke positions of a single-word text can be obtained by identifying the feature points in the handwriting trajectory. For example, the starting point, the end point, and the inflection point of each trajectory in the single-word text can be identified and used as feature points. The stroke positions in the single-word text can be disassembled through the trajectory coordinates of the feature points. During the disassembly process, the stroke positions can be represented and stored, so that in subsequent steps, the stroke recognition can be performed through the stroke positions obtained by disassembly to obtain text.

[0129] In the embodiment of the present application, any handwriting trajectory includes at least one starting point and one ending point, and the number and position of the inflection points are related to the strokes corresponding to the trajectory, so a handwriting trajectory may include M inflection points, where M is an integer greater than or equal to 0. For example, the horizontal stroke "一" includes 1 stroke starting point, 1 stroke ending point, and 0 stroke inflection points; the stroke hook "亅" includes 1 stroke starting point, 1 stroke ending point, and 1 stroke inflection point; the horizontal curved hook "乙" includes 1 stroke starting point, 1 stroke ending point, and 3 stroke inflection points.

[0130] It should be understood that if Figure 8 and Figure 9 As shown, in addition to the starting point, end point and turning point, the strokes in Chinese characters may also include intersection points. Since the intersection points can represent the internal structure of the single-word text, the intersection points can also be used as feature points for identifying the text. Therefore, when decomposing the strokes, the intersection points in the handwriting trajectory can also be collected and decomposed.

[0131] Furthermore, there are also strokes with a certain arc but without bends in some Chinese characters. During the process of disassembling the positions of these strokes, the trajectory coordinates at the positions with a larger arc in the trajectory can also be used as feature points, and combined with feature points such as the starting point, ending point, and inflection point, the stroke position disassembling is carried out.

[0132] At the same time, since the trajectories in the handwritten trajectory have a certain width and length, if the trajectory coordinates obtained in the previous steps are all coordinates representing a single pixel point, in some embodiments, in order to accurately disassemble the stroke positions, multiple trajectory coordinates can be selected as feature points at the starting point and / or ending point of the trajectory, so as to more precisely represent the stroke through the disassembled stroke positions.

[0133] It should be noted that during the process of stroke disassembling, the electronic device can store the disassembled stroke positions, which is convenient for subsequent stroke recognition. For example, the electronic device can represent and store the disassembled stroke positions in the following format.

[0134]

[0135] It should be understood that the foregoing disassembling and representation of stroke positions are only one feasible implementation manner in this application, and the application does not limit the ways of disassembling and representing stroke positions.

[0136] In another part of the embodiments of this application, if the handwritten trajectory is drawn through the display screen 193 of the electronic device, the electronic device can also obtain and record the drawing time of each trajectory by generating the recorded time through the pressure sensor when the user draws the handwritten trajectory, so as to distinguish the handwritten trajectory. In this embodiment, after disassembling the handwritten trajectory into single-character texts, the starting point, ending point, and inflection point and other information in the handwritten trajectory can be obtained according to the drawing order of the handwritten trajectory in the single-character text, and further obtain their trajectory coordinates, so as to obtain the corresponding stroke positions.

[0137] S440: Perform stroke recognition on the disassembled stroke positions to obtain the text content.

[0138] After disassembling the stroke positions, the feature vector can be obtained by connecting the stroke positions. Then, through the angle and direction of the feature vector, the corresponding stroke can be determined, and further, through the stroke information in each single-character text, the corresponding text content can be obtained.

[0139] Exemplarily, after obtaining the stroke points of the single-character text, the stroke points and the corresponding trajectory coordinates are connected to obtain feature vectors, and then the feature vectors of each handwritten trajectory in each single-character text are determined. It should be understood that the feature vectors have information features such as direction and length. Therefore, through each feature vector in the single-character text, the trajectory features in the single-character text can be obtained, where the trajectory features include the length of the feature vector and the angular relationship between different feature vectors. After obtaining the trajectory features of each single-character text, the text content corresponding to each single-character text can be determined according to the trajectory features.

[0140] As Figure 5 shown, after obtaining the text content by recognizing the handwritten trajectory, it is also necessary to determine the text format, so that the generated text is closer to the features in the handwritten trajectory, improve the feature retention of the generated text, and then reflect the features in the handwritten trajectory through the generated text, improving the sorting efficiency of the handwritten trajectory. Therefore, the method includes:

[0141] S500: Determine the text format according to the number of characters and the size of the characters in the text content.

[0142] In the embodiments of the present application, in order to determine the text format, it is necessary to calculate the text format through information such as the number of characters and the size of the characters in the obtained text content. Exemplarily, the number of texts may include the total number of characters in the text content. In the above-mentioned process of recognizing the characters of the handwritten trajectory, single-character segmentation is performed to obtain single-character texts. Therefore, after recognizing and obtaining the text content, the number of characters in the text content can be determined according to the number of single-character texts obtained by segmentation.

[0143] The character size may include the length and width of each character in the text content, and the text format may include the font size of the characters in the text content and the line spacing between different lines. Among them, since the handwritten trajectory and its corresponding text content displayed on the display screen 193 are both two-dimensional structures, the length of each character can be obtained through the vertical scale of the character, and the width of each character can be obtained through the horizontal scale of the character.

[0144] Further, in the embodiments of the present application, the character size can be determined through the length and width of the single-character text, and then the text format of the text content can be determined. Exemplarily, as Figure 8 shown, if the trajectory arrangement direction in the handwritten trajectory is horizontal, then by calculating the distance between the upper boundary and the lower boundary of each line of characters, the line height of each line of characters in the text content can be obtained, and by calculating the distance between the left boundary and the right boundary of each line of characters, the width of each character in the text content can be obtained.

[0145] Among them, the line height of each line of text can be used as the length of each character in that line of text, and the upper and lower boundaries are obtained by statistically analyzing the horizontal direction of each line of the handwritten trajectory, while the left and right boundaries are obtained by statistically analyzing the vertical direction of each line of the handwritten trajectory.

[0146] Exemplarily, the size of a single-character text can also be obtained by recording the specific positions of the upper, lower, left, and right boundaries of each single-character text. As Figure 8 shown, each single-character text has its corresponding upper, lower, left, and right boundaries. Therefore, after the single-character text is recognized, the electronic device can be expressed in the following format.

[0147]

[0148] Among them, left represents the left boundary of the corresponding single-character text, top represents the upper boundary of the corresponding single-character text, right represents the right boundary of the corresponding single-character text, and bottom represents the lower boundary of the corresponding single-character text. Based on the above method, after the text is recognized by strokes, the size corresponding to the text can be recorded together to facilitate the subsequent determination of the text format.

[0149] It should be understood that the foregoing recording of the single-character text and the boundaries of the single-character text is only a feasible implementation manner in this application, and the application does not limit the recording method.

[0150] In another part of the embodiments of this application, as Figure 9 shown, if the trajectory arrangement direction in the handwritten trajectory is vertical, then by calculating the distance between the left and right boundaries of each column of text, the width of each character in the text content can be obtained, and by calculating the distance between the upper and lower boundaries of each column of text, the length of each character in the text content can be obtained.

[0151] Among them, the left and right boundaries are obtained by statistically analyzing the vertical direction of each column of the handwritten trajectory, and the upper and lower boundaries are obtained by statistically analyzing the horizontal direction of each row of the handwritten trajectory.

[0152] It should be understood that in some embodiments of this application, the number of rows of the handwritten trajectory can also be obtained by statistically analyzing the horizontal direction of the handwritten trajectory, so as to determine the number of rows of the text content. Or, the number of columns of the handwritten trajectory can be obtained by statistically analyzing the vertical direction of the handwritten trajectory, so as to determine the number of columns of the text content.

[0153] After obtaining the number of characters in the text content, the length and width of each character through the text content, the text format can be calculated using the above parameters.

[0154] In some embodiments of the present application, taking the horizontal direction of the trajectory drawing in the handwritten trajectory as an example, the text format determined by calculation includes font size and line spacing. Since the font sizes between different lines may be different, during the calculation of the font size, each line of text can be used as a unit to calculate the font size of each line of text separately.

[0155] Exemplarily, the first font size corresponding to each line of text in the text content can be calculated through the number of lines of the text content, the number of characters in each line of text, and the length of the characters in each line of text. The length of the characters in each line of text can be the length value of any character in that line of text.

[0156] It should be understood that since there will be characters such as letters, numbers, or symbols in the text content, the length values of different characters in the same line of text vary greatly. Therefore, in some embodiments, in order to avoid selecting the length value of a character with a relatively small length such as a symbol or a letter as the length of the characters in that line of text, the length of the characters in each line of text can be the maximum length value in that line of text.

[0157] Furthermore, the second font size corresponding to each character can also be calculated through the number of characters in the text content, the length and width of each character. The second font size can correspond to multiple font sizes, but each character has at most one second font size corresponding to its font size.

[0158] For calculating the line spacing between different lines of text, the distance of the text-free area between two adjacent lines of text can be obtained through the distance between the lower and upper boundaries of the adjacent lines of text, so as to determine the distance between two adjacent lines of text as the first line spacing.

[0159] Alternatively, the second line spacing between different lines of text can also be obtained through the number of lines in the text content, the length of the characters in each line of text, and in combination with the length of the selection range. The length of the selection range can be obtained through the extension distance of the selection box 1931 in the vertical direction, so that the finally obtained text content based on this text format can fill the selection range, so that the generated text retains more characteristics in the handwritten trajectory.

[0160] Such as Figure 8As shown, during the process of calculating the text format, the first rectangle corresponding to each line of text and the second rectangle corresponding to each character in each line of text can be constructed by respectively performing horizontal projection on each line of text and vertical projection on each character in each line of text. Each second rectangle is set within the first rectangle, and only one second rectangle is set within one first rectangle. Through the length and width of the first rectangle and the number of second rectangles in the first rectangle, the font size of the characters in the first rectangle can be calculated. By calculating the font size of the characters in each first rectangle respectively, the font size corresponding to the characters in each line of text within the selected range can be obtained.

[0161] In another part of the embodiments of the present application, taking the vertical direction of the trajectory drawing in the handwritten trajectory as an example, the text format determined by calculation includes the font size and the column spacing. Since the font sizes between different columns may be different, during the process of calculating the font size, the font size of the characters in each column can be calculated separately with each column of characters as a unit.

[0162] Exemplarily, the third font size corresponding to each column of text in the text content can be calculated through the number of columns of the text content, the number of characters in each column of text, and the width of the characters in each column of text. Among them, the width of the characters in each column of text can be the width value of any character in the column of text.

[0163] It should be understood that since there are characters such as letters, numbers, or symbols in the text content, the width values of different characters in the same column of characters vary greatly. Therefore, in some embodiments, in order to avoid selecting the width value of a character with a smaller width such as a symbol or a letter as the width of the characters in the column of text, the width of the characters in each column of text can be the maximum width value in the column of text.

[0164] Furthermore, the fourth font size corresponding to each character can be calculated respectively through the number of characters in the text content and the length and width of each character. Among them, the fourth font size can also correspond to multiple font sizes, and each character has at most one fourth font size corresponding to its font size.

[0165] For calculating the column spacing between different columns of text, the distance of the text-free area between two adjacent columns can be obtained through the distance between the right boundary and the left boundary of the adjacent columns of text, so as to determine the distance between the adjacent columns of text as the first column spacing.

[0166] Alternatively, the second column spacing between different columns of text can also be obtained through the number of columns in the text content, the width of the characters in each column of text, and in combination with the width of the selected range. The width of the selected range can be obtained through the horizontal extension distance of the selection box 1931, so that the finally obtained text content based on this text format can fill the selected range, so that the generated text retains more characteristics in the handwritten trajectory.

[0167] As Figure 9 shown, during the process of performing text format calculation, the third rectangle corresponding to each column of text and the fourth rectangle corresponding to each character in each column of text can also be constructed by respectively performing vertical projection on each column of text and horizontal projection on each character in each column of text. Each fourth rectangle is set within the third rectangle, and one fourth rectangle is only set within one third rectangle. By the length and width of the third rectangle and the number of fourth rectangles in the third rectangle, the font size of the characters in the third rectangle can be calculated. By respectively calculating the font size of the characters in each third rectangle, the font size corresponding to the characters in each column of text within the selected range can be obtained.

[0168] Since for characters of different types, such as letters and Chinese characters, the length and width of the characters are different under the same font size. Therefore, in some embodiments of the present application, before determining the text format, the character type of the recognized text content can be determined.

[0169] Exemplarily, according to the text content, the character type of each character in the text content can be determined. Among them, the character type can include Chinese characters, letters, numbers, and symbols. It should be understood that the text content recognized through the handwriting trajectory may include one or more types of characters in the character type, and the specific character type of the characters in the text content in the present application is not limited.

[0170] After identifying and obtaining the character type corresponding to each character, the font size of different characters can be determined according to the character type corresponding to the characters. For example, if the character type of the first character is a letter, number, or symbol, the font size of the second character adjacent to the first character is determined as the font size of the first character. Wherein, the first character is any character in the text content, and the second character includes the characters in the text content whose character type is Chinese characters. Through the above method, the font size of the characters such as letters, numbers, and symbols in the text content can be determined as the font size of the adjacent Chinese characters, avoiding the problem of incorrect font size recognition caused by the small size of the characters such as letters, numbers, and symbols.

[0171] In a feasible implementation manner, the characters of the type of Chinese characters and the characters of other character types can also be respectively determined for the font size, so that each character has a font size more in line with its corresponding handwriting trajectory. The process of determining the font size of the characters of different character types in the present application is not limited.

[0172] After determining the text format through any of the above methods, the recognized text and the obtained text format can be displayed for the user to view. At the same time, by receiving user instructions, modification of the text content and / or the text format can be realized.

[0173] Figure 10 This is a schematic diagram of a process for displaying text in an embodiment of the present application.

[0174] like Figure 10 As shown, in some embodiments of the present application, the process of displaying text on the display screen 193 may include:

[0175] S610: Displaying a first interface on the display screen.

[0176] Figure 11 This is a schematic diagram of displaying the conversion of handwritten notes into text in an embodiment of the present application.

[0177] like Figure 11 As shown in (b), the first interface 1932 includes a text display area 1933 and an interface control 1934. The text display area 1933 can display text content, and the interface control 1934 can display text format. Specifically, Figure 11 The interface control 1934 exemplarily shown in the figure includes font size and color. Since information such as line spacing is also obtained when the text format is recognized as mentioned above, the interface control 1934 may also include a control for displaying line spacing.

[0178] It should be understood that the first interface 1932 in the present application may not be provided with the interface control 1934, that is, the recognized text format is not displayed, so as to retain the format characteristics of the handwriting trace. In addition to displaying the recognized text content, the first interface 1932 is not limited in the present application to the display content of the first interface.

[0179] S620: In response to the user's confirmation operation, display the text on the display screen.

[0180] The text includes text content whose font format conforms to the text format and whose font color is the first color. It should be understood that if the user modifies the text content, text format and font color through the modification instruction, the generated text is the user modified version.

[0181] like Figure 11 As shown in (b) and (c), in order to display the text content on the display screen 193, a confirmation control 1935 is further provided on the first interface 1932, namely, "Confirm Replacement" in the figure, and in response to the user's confirmation operation on the first interface 1932, the text can be displayed on the display screen 193. Among them, in response to the user's confirmation operation on the first interface 1932, it is a response to the user's click operation on the confirmation control 1935.

[0182] Specifically, Figure 11As shown in (d), after the electronic device receives a confirmation operation on the first interface 1932, it can generate a text box 1936 with the same size as the selected range. The text is set within the text box 1936, and the format of the text is the same as the text format displayed in the interface control 1934. In this way, by converting it into text that can be recognized by the electronic device and replacing the original selected range at the same time, the organization of the handwriting trajectory can be achieved.

[0183] Through Figure 5 It can be seen that the method for converting handwritten notes into text in the embodiments of the present application further includes:

[0184] S400: Recognize the handwriting trajectory to obtain the text content.

[0185] It should be understood that step S400 can be implemented through the above steps S410 to S440, so it will not be elaborated in the embodiments of the present application.

[0186] S600: In response to the user's confirmation operation, display the text on the display screen.

[0187] It should be understood that step S600 can be implemented through the above steps S610 to S620, so it will not be elaborated in the embodiments of the present application.

[0188] Based on the implementation manner of the above-mentioned method for converting handwritten notes into text, an interaction method for an electronic device is also provided in the present application. As Figure 11 shown, taking a tablet computer as an example of the electronic device, the display screen 193 is coupled with at least one of a touch sensor and a pressure sensor, so the electronic device can receive user instructions through the display screen 193.

[0189] As described above Figure 6 As shown in the above content, in order to be able to input and convert handwritten notes, operation controls can be displayed above the display screen 193. The operation controls can be used to wake up functions such as drawing, erasing, modifying colors, lasso selection, and insertion. By clicking on the operation controls, the user can wake up the corresponding functions of the controls, and then the user can implement the conversion of handwritten notes through the operations of the above functions.

[0190] As Figure 11 shown in (a), during the conversion of handwritten notes, the user can first activate the lasso selection function by clicking on the control corresponding to the lasso selection function, and then slide from one point to another point in the handwriting area to determine a rectangular box with these two points as the diagonals as the selection box 1931. At the same time, a temporary control 1937 is displayed on one side of the selection box 1931.

[0191] The temporary control 1937 is used to operate on the handwritten trajectory selected by the selection box 1931. Exemplarily, the temporary control 1937 may include operations such as cut, copy, adjust color, convert handwriting to text, delete, etc. It should be understood that Figure 11 Only one feasible implementation manner in this application is shown in (a). The temporary control 1937 may include other functions, and the content included in the temporary control 1937 is not limited in this application.

[0192] After the temporary control 1937 is displayed, the user can click on the convert handwriting to text in the temporary control 1937 to convert the handwritten trajectory selected by the selection box 1931. After the electronic device receives the user's operation, it can convert the handwritten trajectory according to the method of converting handwritten notes to text as described above, which is not elaborated in this application.

[0193] When the electronic device obtains the text content through step S440 and obtains the text format through step S500, it is necessary to display the recognized text content and text format through the first interface 1932. As Figure 11 shown in (b), after the recognition is completed, the first interface 1932 can be displayed on the display screen 193. The content displayed on the first interface 1932 is the same as that in the foregoing embodiments, which is not elaborated in this application.

[0194] Figure 12 This is a schematic diagram of operating on the first interface in the embodiments of this application.

[0195] In order to achieve the modifiability of the text content, the text display area 1933 and the interface control 1934 are editable areas, so as to be able to receive the user's modification instructions. As Figure 12 shown in (a) and (c), the text content obtained by recognizing the handwritten trajectory can be displayed on the display screen 193 through the text display area 1933. During the display process, the user can click on the text display area 1933 and, as Figure 11 shown in (c), adopt an input method such as a virtual keyboard to input a first modification instruction to the text display area 1933. After the text display area 1933 in the first interface 1932 of the electronic device receives the first modification instruction, it can modify the text content displayed in the text display area according to the content input by the user.

[0196] Among them, the first modification instruction includes adding and / or deleting the text content. Specifically, as Figure 12 shown in (c), the electronic device can receive the first modification instruction, obtain the text content input by the user, and display it together with the recognized text content in the text display area 1933, so as to achieve the effect of adding to the text content.

[0197] Further, the electronic device can also receive a first modification instruction to obtain an instruction to delete some characters in the text content, so as to delete the corresponding characters from the text display area 1933, thereby achieving the effect of deleting the text content.

[0198] As Figure 12 shown in (a) and (d) of [the figure], the interface control 1934 can include a plurality of drop-down boxes. By clicking on the drop-down box, the user can bring up a drop-down menu with other font sizes or colors, and the user can make a selection in the drop-down menu to modify the text format.

[0199] The interface control 1934 can display the text format. The electronic device can receive and respond to a second modification instruction for the first interface 1932 through the interface control 1934, thereby modifying the text format identified through the foregoing steps to obtain a text format that better meets the user's needs and improving the efficiency of text arrangement. For example, changing the originally identified font size of No. 4 to No. 3.

[0200] In addition to displaying the identified text format, the interface control 1934 can also display the first color in the identified handwriting trajectory. On this basis, at the same time, the interface control 1934 can also receive a third modification instruction to modify the first color in the interface control 1934. For example, changing the originally identified purple to red.

[0201] Exemplarily, as Figure 12 shown in (a) and (b) of [the figure], the text content displayed in the text display area 1933 can also receive a long-press instruction. When the display screen 193 detects that the user long-presses on the text display area 1933 through the sensor, it can select the characters pressed in the text display area 1933 according to the pressing position and sliding method, and a control will appear after the user finishes pressing, so as to facilitate the user to perform operations such as copying, cutting, translating, sharing, and selecting all on the selected characters. It should be understood that Figure 12 the control shown in (b) of [the figure] is only an example, and the position where the control appears and the content it includes are not limited in this application.

[0202] In some embodiments, such as in some embodiments of this application, the first interface 1932 further includes a copy control. The user can control the copy control to copy the content in the text display area 1933 to the system clipboard, so that the copied content can be directly pasted to the position where the user needs to input in the text input scenario.

[0203] As Figure 12As shown in Figures (e) and (f), if the electronic device fails to recognize the text in the handwritten trajectory or the selected range does not include the handwritten trajectory, the display screen 193 can still display the first interface 1932. However, at this time, the text display area 1933 and / or the display control 1934 are empty, and a prompt message "No content recognized" can be displayed in the text display area 1933, thereby reminding the user that the recognition is completed but no valid content is recognized within the selected range. At this time, the user can click on the text display area 1933 to eliminate the prompt message. At this time, the text display area 1933 can receive a first modification instruction to add and modify text.

[0204] In another part of the embodiments, if the electronic device fails to recognize the text in the handwritten trajectory or the selected range does not include the handwritten trajectory, the user can click on the exit control in the first interface 1932 to abandon the currently selected content. As Figure 12 As shown in Figures (e) and (g), an exit control can be provided in the upper left corner of the first interface 1932. By clicking on the exit control, the user can switch to the interface as shown in Figure 12 Figure (g). If the user clicks to abandon, the first interface 1932 of the electronic device will be closed and the handwritten trajectory before the drawing selection box 1931 will be displayed. If the user clicks to cancel, the first interface 1932 will continue to be displayed.

[0205] It should be understood that the user can close the first interface 1932 at any time by clicking on the exit control of the first interface 1932, and the timing of clicking on the exit control is not restricted in this application. At the same time Figure 12 the page as shown in Figure (g) can also directly appear in the case of recognition errors, and after the user clicks to abandon, the display screen 193 before the drawing selection box 1931 will be displayed to facilitate the user to select the range again.

[0206] Through the above implementation manners and Figure 11 as shown in Figure (c), the user can click on the text display area 1933 and the interface control 1934 to call out the corresponding function and modify the displayed text content and text format through a modification instruction.

[0207] As Figure 11 shown in Figure (c), after the user finishes the modification, the user can click on the confirmation control 1935 to confirm the current text content and text format. As Figure 11 shown in Figure (d), the electronic device generates a text to replace the handwritten trajectory based on the current text content and text format, thereby realizing the function of converting handwritten notes into text.

[0208] It should be noted that, in order to implement the above functions, the above electronic device includes the corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should easily realize that, combining the units and algorithm steps of each example described in the embodiments disclosed herein, the embodiments of the present invention can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the way of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of this application.

[0209] The embodiments of this application can divide the functional modules of the above electronic device according to the above method examples. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of a software functional module. It should be noted that the division of modules in the embodiments of the present invention is illustrative, only a logical function division, and there can be other division methods in actual implementation.

[0210] Figure 13 It is a schematic diagram of a handwritten note conversion to text system in the embodiments of this application; Figure 14 It is a schematic flowchart of the conversion system in the embodiments of this application executing the conversion method.

[0211] In the case of dividing each functional module corresponding to each function, as Figure 13 and Figure 14 shown, the embodiments of this application also provide a handwritten note to text system applied in an electronic device. The system can include: a note display module 1310 and a handwriting recognition module 1320.

[0212] It should be understood that the electronic device in this embodiment still takes the electronic device shown in Figure 2 as an example for illustration.

[0213] The note display module 1310 can be provided with a canvas 1311 and a window display unit 1312. The note display module 1310 can display the content on the display screen 193 of the electronic device through the canvas 1311. And the handwriting recognition module 1320 can be provided with a trajectory drawing unit 1321, an instruction response unit 1322, a trajectory recognition unit 1323, and a text generation unit 1324.

[0214] Such as Figure 14As shown, the user can draw handwritten notes in the handwriting area of the display screen 193, where the handwriting area of the display screen 193 can be the canvas 1311. During the process of the user's handwritten drawing, the trajectory drawing unit 1321 can receive pressure signals and / or tactile signals through the pressure sensor and / or touch sensor coupled to the display screen 193, so as to realize the drawing of the handwritten trajectory. Then, the trajectory drawing unit 1321 can send the completed handwritten trajectory image to the canvas 1311 in the note display module 1310, so as to display the corresponding handwritten trajectory on the display screen 193.

[0215] When the conversion of handwritten notes is required, the user can select the content in the canvas 1311 and frame a selection range on the canvas 1311. After receiving the selection range, the trajectory drawing unit 1321 can send the selection range and the handwritten trajectory to the instruction response unit 1322 together, so that the instruction response unit 1322 can send the handwritten trajectory image with the selection range to the canvas 1311, making the selection range clearer.

[0216] When sending the handwritten trajectory image with the selection range to the canvas 1311, the instruction response unit 1322 needs to send the handwritten trajectory within the selection range to the trajectory recognition unit 1323, so that the trajectory recognition unit 1323 can recognize the handwritten trajectory to obtain the first color, text content and text format. It should be understood that the process of the instruction response unit 1322 sending the handwritten trajectory to the trajectory recognition unit 1323 can be carried out simultaneously with the process of the instruction response unit 1322 sending the image to the canvas 1311, and this application does not limit the order of these two steps.

[0217] At the same time, the functional implementation process of the trajectory recognition unit 1323 refers to the above steps S200 to S440, and this application will not elaborate here.

[0218] After the trajectory recognition unit 1323 recognizes the text content, text format and the first color, it can send the recognized content to the window display unit 1312, so as to display the recognized text content, text format, first color and other information, facilitating the user to modify.

[0219] If the user needs to modify, the user can operate on the window display unit 1312 through the display screen 193. The specific operation process refers to the above-mentioned interaction method for the first interface 1932, and this application will not elaborate here. By operating on the window display unit 1312, the recognized text content, text format and / or the first color can be modified, so that the content and format of the text better meet the user's needs and improve the sorting efficiency of handwritten notes.

[0220] If the user does not need to modify or has already completed the modification, the result can be confirmed by clicking on the display screen 193. Specifically, when the window display unit 1312 receives the user's confirmation operation, it can send information such as the text content, text format, and first color in the display to the text generation unit 1324, and the text generation unit 1324 generates the corresponding text and sends it to the canvas 1311 to replace the selected range in the canvas 1311.

[0221] It should be understood that the above structural implementation is only an example in the embodiments of the present application. In this system, the connection in the handwriting recognition module 1320 can also be reduced and the recognition efficiency can be improved by setting a memory in the handwriting recognition module 1320 and storing the sent and received information in the memory.

[0222] Regarding the electronic device in the above embodiments, the specific ways in which each module performs operations have been described in detail in the embodiments of the method for converting handwritten notes into text in the foregoing embodiments, and will not be specifically elaborated here. The related beneficial effects can also be referred to the related beneficial effects of the foregoing method for converting handwritten notes into text, and will not be repeated here.

[0223] The embodiments of the present application also provide an electronic device, which includes: a display screen, a memory, and one or more processors; the display screen, the memory are coupled to the processor; wherein, the memory stores computer program code, and the computer program code includes computer instructions. When the computer instructions are executed by the processor, the electronic device executes the method for converting handwritten notes into text provided in the foregoing embodiments. The specific structure of the electronic device can be as Figure 2 shown in the structure of the electronic device.

[0224] Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and simplicity of description, only the above division of each functional module is used as an example. In actual applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0225] In several embodiments provided by this application, it should be understood that the disclosed system / devices and methods can be implemented in other ways. For example, the system / device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections between each other can be through some interfaces. The indirect couplings or communication connections of the systems or units can be in electrical, mechanical or other forms.

[0226] The units described as separate components may or may not be physically separated. The components displayed as units can be one physical unit or multiple physical units, that is, they can be located in one place, or they can be distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0227] In addition, in each embodiment of this application, each functional unit can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0228] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on such an understanding, the technical solution of the embodiments of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to enable a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of this application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read only memory (ROM), random access memory (RAM), magnetic disks or optical discs that can store program codes.

[0229] The above content is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A method for converting handwritten notes into text, characterized in that, Applied to an electronic device with a display screen, the method includes: In response to a user's operation on a handwriting area, obtaining a selection range in the handwriting area; the handwriting area is an area on the display screen that can receive the user's drawn trajectory; Obtaining the handwriting trajectory within the selection range; Identifying the first color of the handwriting trajectory; Identifying the handwriting trajectory to obtain text content; Determining a text format according to the number of characters and the character size in the text content; In response to a user's confirmation operation, displaying the text in text form on the display screen; the text in text form includes the text content with a font format conforming to the text format and a font color being the first color.

2. The method for converting a handwritten note into text in text form according to claim 1, characterized in that The number of characters includes the number of characters, the number of lines of text, and the number of characters in each line of text in the text content; The character size includes the length and width of each character in the text content; The text format includes font size and line spacing.

3. The method for converting a handwritten note into text in text form according to claim 2, characterized in that The determining the text format according to the number of characters and the character size in the text content includes: Calculating, according to the number of lines of the text content, the number of characters in each line of text, and the length of each character in each line of text, the first font size corresponding to each line of the text content respectively; or, Calculating, according to the number of characters in the text content and the length and width of each character, the second font size corresponding to each line of the text content respectively; Determining a first line spacing according to the spacing between each line of text in the text content; or, Determining a second line spacing according to the number of lines of the text content and the length of each character in each line of text.

4. The method for converting a handwritten note into text in text form according to any one of claims 1 to 3, characterized in that After the determining the text format according to the number of characters and the character size in the text content, it further includes: Displaying a first interface on the display screen; the first interface includes a text display area and an interface control, the text display area displays the text content, and the interface control displays the text format; The displaying the text in text form on the display screen in response to a user's confirmation operation includes: In response to a user's confirmation operation on the first interface, displaying a text box on the display screen; the text box includes the text in text form, the size of the text box is the same as the selection range, and the format of the text in text form is the same as the text format displayed in the interface control.

5. The method for converting a handwritten note into text in text form according to claim 4, characterized in that After the identifying the handwriting trajectory to obtain text content, it further includes: Determining the character type of each character in the text content according to the text content; the character type includes Chinese characters, letters, numbers, and symbols; After the calculating the font size corresponding to the text content, it further includes: If the character type of the first character is a letter, a digit, or a symbol, determine the font size of the second character adjacent to the first character as the font size of the first character; the first character is any character in the text content, and the second character includes characters in the text content whose character type is a Chinese character.

6. The method for converting a handwritten note into a text as claimed in claim 4, wherein in response to the user's confirmation operation, displaying the text on the display screen further includes: in response to a first modification instruction of the user for the first interface, modifying the text content in the text display area; and / or in response to a second modification instruction of the user for the first interface, modifying the text format in the interface control.

7. The method for converting a handwritten note into a text as claimed in claim 1, wherein if the first color includes at least two colors, after recognizing the handwritten trajectory to obtain the text content, further includes: obtaining the first color corresponding to each character in the text content according to the correspondence between the first color and the handwritten trajectory.

8. The method for converting a handwritten note into a text as claimed in claim 1, wherein recognizing the handwritten trajectory to obtain the text content includes: determining the trajectory coordinates corresponding to the handwritten trajectory within the selected range; segmenting the handwritten trajectory into single characters to obtain single-character texts; decomposing the single-character texts according to the trajectory coordinates into stroke point positions; recognizing the stroke point positions obtained by decomposition to obtain the text content.

9. The method for converting a handwritten note into a text as claimed in claim 8, wherein recognizing the stroke point positions obtained by decomposition to obtain the text content includes: determining the feature vector of the handwritten trajectory in each single-character text according to the stroke point positions and the trajectory coordinates corresponding to the stroke point positions; obtaining the trajectory feature according to the feature vector; determining the text content corresponding to each single-character text according to the trajectory feature of each single-character text.

10. The method for converting handwritten notes into text according to claim 1, characterized in that, The electronic device includes a note display module and a handwriting recognition module, and the method includes: the handwriting recognition module, in response to the user's operation on the handwriting area, obtains the selected range within the display range of the note display module; reads the handwritten trajectory within the selected range through the handwriting recognition module and recognizes the first color of the handwritten trajectory; the handwriting recognition module recognizes the handwritten trajectory to obtain the text content, and determines the text format according to the number of characters and the character size in the text content; the handwriting recognition module sends the text content and the text format to the note display module; obtaining the user's confirmation operation through the note display module, and the handwriting recognition module sends the text content in the text format and the first color to the note display module; The text is displayed on the display screen through the note display module.

11. An electronic device, characterized in that, Comprising: A display screen, a memory, and one or more processors; the display screen, the memory are coupled to the processor; wherein, computer program code is stored in the memory, the computer program code includes computer instructions, when the computer instructions are executed by the processor, the electronic device is caused to execute the method for converting handwritten notes into text as described in any one of claims 1-10.

Images