Handwriting conversion method and electronic equipment

Abstract

The invention provides a handwriting conversion method, the method can be applied to electronic equipment, and the method comprises the steps that a target picture is acquired, and the target picture comprises target identification information; determining the contour of the target identification information; determining data information of the filling contour; and filling the contour of the target identification information with the target color according to the data information. Therefore, the target identification information in the target picture can form the graphic entity in the manner of filling the outline and is displayed in the electronic equipment, so that the target identification information can be completely displayed in the electronic equipment, and the phenomenon of wrong or incomplete identification is avoided.

Description

Technical Field

[0001] The present application relates to the field of electronic technology, and more specifically, to a method for converting handwriting and an electronic device. Background Art

[0002] When users are working or studying, they often need to record some text information on paper notebooks. With the popularization of electronic devices, users hope to convert paper text information into notes in electronic devices, so that users can share their offline or online learning notes and improve work and study efficiency.

[0003] Currently, electronic devices may have artificial intelligence (AI) capabilities. For example, electronic devices may use text recognition algorithms to extract text information from images. However, this method has a high error rate for text recognition and cannot recognize some graphic marks. Summary of the invention

[0004] The present application provides a method and electronic device for converting handwriting. The technical solution can display the text information recorded by the user on paper in the form of handwriting on the electronic device, so that the text or other graphic marks or symbols in the text information can be efficiently and accurately converted into handwriting displayed by the electronic device.

[0005] In a first aspect, a method for converting handwriting is provided, which is applied to an electronic device, and the method comprises: acquiring a target image, wherein the target image comprises target identification information; determining an outline of the target identification information; determining data information for filling the outline; and filling the outline of the target identification information with a target color according to the data information.

[0006] For example, the target recognition information may include the user's handwriting, such as text, graphic logos or symbols, etc. The target recognition information may also include a certain type of font, such as Kaiti, Songti and other non-handwriting fonts.

[0007] Based on the embodiment of the present application, the electronic device can obtain a picture including target identification information, determine the outline of the target identification information, further determine data information that can fill the outline, and fill the outline of the target identification information with the target color according to the data information. In this way, the target identification information in the target picture can be formed into a graphic entity in the form of a filled outline and displayed in the electronic device, so that the target identification information can be fully displayed in the electronic device, avoiding the phenomenon of recognition errors or incompleteness.

[0008] In combination with the first aspect, in an implementation method of the first aspect, determining the data information for filling the contour includes: filling the contour with multiple circles; determining the center coordinates and radii of the multiple circles, wherein the data information is a set of the center coordinates and radii of the multiple circles.

[0009] Based on the embodiment of the present application, the electronic device can use multiple circles to fill the outline of the target recognition information, and determine the center coordinates and radius of the multiple circles. In this way, the electronic device can store the set of the center coordinates and radius of the multiple circles converted from the outline, so that the outline of the target recognition information can be filled according to the center coordinates and radius of the multiple circles to restore the target recognition information.

[0010] In other examples, the electronic device may also use other images to fill in the outline of the target recognition information. Alternatively, the electronic device may also directly fill in the outline of the target recognition information, which is not limited in the embodiments of the present application.

[0011] In combination with the first aspect, in an implementation manner of the first aspect, filling the outline of the target identification information with the target color according to the data information includes: filling the multiple circles with the target color according to the data information.

[0012] Based on the embodiments of the present application, the electronic device can use the target color to fill the above-mentioned multiple circles, thereby filling the entire outline, so that the electronic device can form a graphic entity of the target identification information to avoid identification errors.

[0013] In combination with the first aspect, in an implementation manner of the first aspect, before determining the outline of the target identification information, the method further includes: performing pixel compression processing on the target image.

[0014] Based on the embodiments of the present application, the electronic device may also perform pixel compression processing on the target image before determining the outline of the target identification information, thereby reducing the amount of data processed by the electronic device and speeding up the processing speed of the electronic device.

[0015] In combination with the first aspect, in an implementation method of the first aspect, the pixel compression processing of the target image includes: when the color gamut of an area in the target image that is greater than or equal to a preset proportional area is in the same color gamut, pixel compression processing is performed on the target image.

[0016] Exemplarily, the preset proportional area may be 90% or 80%, etc. The embodiment of the present application does not limit the specific value of the preset proportional area.

[0017] Based on the embodiments of the present application, when the electronic device determines that the color gamuts of most areas in the target image are in the same color gamut, it can be considered that most areas in the target image belong to the same color, and pixel compression processing can be performed on the target image.

[0018] In combination with the first aspect, in an implementation manner of the first aspect, before determining the outline of the target recognition information, the method further includes: determining a noise area in the target image; and deleting data in the noise area.

[0019] Based on the embodiment of the present application, the electronic device can also determine the noise area in the target image and delete the data in the noise area before determining the outline of the target recognition information. This technical solution can enable the electronic device to identify some useless noise information and reduce the amount of data processed.

[0020] In combination with the first aspect, in an implementation method of the first aspect, determining the noise area in the target image includes: when the number of pixels in the target area that are different from the background color of the target image is less than a preset number, determining that the target area is the noise area.

[0021] For example, the size of the target area may be a circular area with a diameter of 10 or 15 pixels. It should be understood that the target area may also be other shapes, which are not limited in the embodiments of the present application.

[0022] Based on the embodiment of the present application, when the number of pixels in the target area that are different from the background color of the target image is less than a preset number, it means that the number of pixels of other colors in the area is small and it is unlikely to be the handwriting of the user, so the target area can be determined to be a noise area. This technical solution is conducive to the electronic device to determine the noise area in the target image.

[0023] In combination with the first aspect, in an implementation of the first aspect, the target recognition information includes handwriting of the user, and determining the outline of the target recognition information includes: determining the overall outline of the handwriting, or determining the outline of continuous strokes of the handwriting.

[0024] Based on the embodiment of the present application, the target recognition information may be the user's handwriting, and the electronic device may directly determine the overall outline of the handwriting or may also determine the outline of the continuous strokes of the handwriting. The technical solution may enable the electronic device to determine the outline of the handwriting.

[0025] Optionally, the target identification information may also include a certain type of printed font or symbol mark other than the user's handwriting, which is not limited in the embodiment of the present application.

[0026] In combination with the first aspect, in an implementation manner of the first aspect, the handwriting includes text information and graphic symbols.

[0027] It should be understood that the graphic symbol may also be referred to as a graphic logo, mark, etc.

[0028] In combination with the first aspect, in an implementation manner of the first aspect, the method further includes: editing the handwriting in response to a user operation.

[0029] Based on the embodiment of the present application, the electronic device can also edit the recognized handwriting according to the user's operation, such as erasing or continuing writing, adding marks, etc. This technical solution allows the user to perform secondary creation on the handwriting recognized by the electronic device, for example, correcting the errors in the original handwriting, etc., thereby improving the user experience.

[0030] In combination with the first aspect, in an implementation manner of the first aspect, the target color is black, or the target color is the same color as the color of the target identification information.

[0031] Based on the embodiments of the present application, the target color can be black by default, or it can be the same color as the target identification information. This technical solution can make the handwriting displayed on the electronic device the same color as the handwriting on the original paper, giving users a consistent color experience.

[0032] In a second aspect, an electronic device is provided, comprising: one or more processors; one or more memories; the one or more memories storing one or more programs, and when the one or more programs are executed by one or more processors, the method for converting handwriting as described in the first aspect and any possible implementation thereof is executed.

[0033] In a third aspect, a handwriting conversion device is provided, comprising a module for implementing the handwriting conversion method as described in the first aspect and any possible implementation manner thereof.

[0034] In a fourth aspect, a chip is provided, comprising a processor and a communication interface, wherein the communication interface is used to receive a signal and transmit the signal to the processor, and the processor processes the signal so that the method for converting handwriting as described in the first aspect and any possible implementation thereof is executed.

[0035] In a fifth aspect, a readable storage medium is provided, wherein instructions are stored in the readable storage medium. When the instructions are executed on an electronic device, the method for converting handwriting as described in the first aspect and any possible implementation thereof is executed.

[0036] In a sixth aspect, a program product is provided, the program product comprising a program code, and when the program code is run on an electronic device, the method for converting handwriting as described in the first aspect and any possible implementation thereof is executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application.

[0038] Figure 2 It is a schematic diagram of the software structure of the electronic device provided in the embodiment of the present application.

[0039] Figure 3 It is a module diagram of a handwriting application program provided in an embodiment of the present application.

[0040] Figure 4 It is a schematic diagram of a group of graphical user interfaces GUI provided in an embodiment of the present application.

[0041] Figure 5 It is a schematic flow chart of a method for converting handwriting provided in an embodiment of the present application.

[0042] Figure 6 It is a schematic diagram of a filled graphic outline provided in an embodiment of the present application.

[0043] Figure 7 It is a schematic flow chart of a method for converting handwriting provided in an embodiment of the present application.

[0044] Figure 8 It is a schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0046] The method for converting handwriting in the embodiment of the present application can be applied to smart phones, smart TVs, tablet computers, laptops, personal computers (PCs), ultra-mobile personal computers (UMPCs), netbooks, vehicle-mounted devices, car computers, wearable devices, foldable devices, Internet of Things (IOT) devices and other devices.

[0047] Figure 11 shows a schematic diagram of the structure of the device 100. The device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0048] It is understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the device 100. In other embodiments of the present application, the device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0049] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0050] In the present application, the processor 110 may be used to encode and decode an image. For example, the processor 110 in one device may be used to change the brightness of a display screen according to binary coded information, where the binary coded information includes device connection information of the device. The processor in another device may be used to identify the device connection information from an acquired image.

[0051] The controller may be the nerve center and command center of the device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0052] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0053] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus interface, etc.

[0054] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL).

[0055] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170.

[0056] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface.

[0057] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160.

[0058] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 .

[0059] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc.

[0060] The USB interface 130 is an interface that complies with USB standard specifications, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 may be used to connect a charger to charge the device 100, and may also be used to transmit data between the device 100 and a peripheral device.

[0061] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the device 100. In other embodiments of the present application, the device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0062] The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input through the wireless charging coil of the device 100. While the charging management module 140 is charging the battery 142, it can also power the device through the power management module 141.

[0063] The power management module 141 is used to connect the battery 142 , the charging management module 140 and the processor 110 .

[0064] The wireless communication function of the device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0065] The mobile communication module 150 can provide wireless communication solutions including 2G / 3G / 4G / 5G etc. applied on the device 100 .

[0066] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0067] The wireless communication module 160 can provide wireless communication solutions applied to the device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), Bluetooth low energy (BLE), global navigation satellite system (GNSS), frequency modulation (FM), nearfield communication technology (NFC), infrared technology (IR), etc.

[0068] In some embodiments, antenna 1 of device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that device 100 can communicate with the network and other devices through wireless communication technology.

[0069] The device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0070] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), or a display panel made of one of the materials such as an organic light-emitting diode (OLED), an active matrix organic light-emitting diode or an active matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed or a quantum dot light emitting diode (QLED). In some embodiments, the device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0071] In the embodiment of the present application, the device 100 can display an image through the display screen 194, and display an image superimposed with digital watermark information.

[0072] The device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0073] The ISP is used to process the data fed back by the camera 193. The camera 193 is used to capture a static image or a video. In the present application, the camera 193 can be used to obtain an image displayed on a display screen of another device.

[0074] Digital signal processors are used to process digital signals. In addition to processing digital image signals, they can also process other digital signals.

[0075] The video codec is used to compress or decompress digital video. The device 100 may support one or more video codecs.

[0076] The external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the device 100 .

[0077] The internal memory 121 may be used to store computer executable program codes, wherein the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the device 100 by running the instructions stored in the internal memory 121 .

[0078] The device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0079] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals.

[0080] The speaker 170A, also called a "horn", is used to convert audio electrical signals into sound signals.

[0081] The receiver 170B, also called a "handset", is used to convert audio electrical signals into sound signals.

[0082] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals.

[0083] The earphone jack 170D is used to connect a wired earphone.

[0084] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194 .

[0085] The gyro sensor 180B may be used to determine the motion posture of the device 100 .

[0086] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

[0087] The acceleration sensor 180E can detect the magnitude of the acceleration of the device 100 in various directions (generally three axes).

[0088] The distance sensor 180F is used to measure the distance.

[0089] The fingerprint sensor 180H is used to collect fingerprints.

[0090] The touch sensor 180K is also called a “touch panel.” The touch sensor 180K may be disposed on the display screen 194 , and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen.”

[0091] The bone conduction sensor 180M can obtain vibration signals. In some embodiments, the bone conduction sensor 180M can obtain vibration signals of vibrating bones of the human body. The bone conduction sensor 180M can also contact the human body's pulse to receive blood pressure beating signals.

[0092] Buttons 190 include a power button, a volume button, and the like.

[0093] Motor 191 can generate vibration prompts.

[0094] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.

[0095] The SIM card interface 195 is used to connect a SIM card.

[0096] The software system of the device 100 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes an operating system of a layered architecture as an example to exemplify the software structure of the device 100.

[0097] Figure 2 1 is a software structure diagram of the electronic device 100 of the embodiment of the present application. The layered architecture divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the operating system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the system library, and the kernel layer. The application layer can include a series of application packages.

[0098] like Figure 2 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, notes, etc.

[0099] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0100] like Figure 2 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0101] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0102] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0103] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0104] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0105] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0106] The notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages and disappear automatically after a short stay without user interaction.

[0107] The core library consists of two parts: one part is the function that the Java language needs to call, and the other part is the core library.

[0108] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0109] The system library may include multiple functional modules, such as surface manager, media libraries, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0110] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0111] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0112] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0113] A 2D graphics engine is a drawing engine for 2D drawings.

[0114] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0115] Before introducing the technical solution of the present application, some technical terms that may be involved in the present application are briefly introduced as follows.

[0116] Handwriting: In the embodiment of the present application, handwriting can be understood as text information, symbol information, etc. written by the user on paper. For example, it can be the notes recorded by the user when studying. It should be understood that handwriting can also be called font, handwriting, etc.

[0117] Color gamut: refers to the range of colors that can be expressed by a certain color representation mode.

[0118] When users are working or studying, they often need to record some text information on paper notebooks. With the popularization of electronic devices, users hope to convert paper text information into notes in electronic devices, so that users can share their offline or online learning notes and improve work and study efficiency.

[0119] Currently, electronic devices may have artificial intelligence (AI) capabilities. For example, electronic devices may use text recognition algorithms to extract text information from images. However, this method has a high error rate for text recognition and cannot recognize some graphic marks.

[0120] In view of this, the embodiments of the present application provide a method and electronic device for converting handwriting. The technical solution can display the text information recorded by the user on paper in the form of handwriting in the electronic device, so that the text or other graphic marks in the text information can be efficiently and accurately converted into handwriting displayed by the electronic device. In addition, the user can also edit the handwriting converted by the electronic device to create the handwriting again.

[0121] In the embodiment of the present application, the electronic device may have an application (application, App) that implements the above functions. For example, the application may be a note application, and the electronic device may convert the handwriting (text or other graphic marks) recorded by the user on paper into handwriting displayed in the electronic device through the note application. In addition, the note application may also provide an editing function, for example, the editing function may include functions such as erasing, writing, and brushing, and the user may realize secondary creation of handwriting through the editing function.

[0122] For example, Figure 3 is a schematic module diagram of a note-taking application provided in an embodiment of the present application. Figure 3 As shown, the note-taking App may at least include an interactive display interface, an image acquisition module, an image processing module, a data storage module, an editing function module and an engine, etc.

[0123] The interactive display interface may provide a function of identifying the user's related operations so as to interact with the electronic device. For example, the electronic device may identify the user's operations in the interactive display interface, take a picture including the user's handwriting, or select a picture including the user's handwriting from a gallery.

[0124] In other examples, the image may include not only the user's handwriting but also printed text. For example, the image may be a screenshot of a display interface of an electronic device, and the screenshot may include text content in one format or multiple formats.

[0125] The picture acquisition module can be used to read the picture selected by the user from the memory into the system, or read the picture taken by the camera into the system.

[0126] The image processing module can be used to perform corresponding data processing on the acquired image. For example, the image processing module can be used to compress the pixels in the image to reduce the amount of data to be processed; the image processing module can also be used to perform noise processing on the image to remove some noise points in the image that are not related to the handwriting; the image processing module can also be used to identify the contour information of the handwriting in the image; the image processing module can also be used to fill the contour of the handwriting according to the contour information to generate the handwriting displayed in the electronic device.

[0127] It should be understood that the image processing module can also perform the same processing as above on text content in other formats besides handwriting, and the embodiments of the present application are not limited to this.

[0128] The data storage module can be used to store the data information of the handwriting displayed in the electronic device. For example, the data information of the handwriting can be the data information of filling the outline of the handwriting, such as the electronic device can fill the outline of the handwriting with a certain number of circles with a radius, and the data information is the center coordinates and radius of the circle, and the electronic device can subsequently generate the handwriting displayed in the electronic device according to the center coordinates and radius of these circles.

[0129] The editing function module can be used to provide the user with a function of secondary editing of the displayed handwriting. For example, after the electronic device displays the handwriting in the recognized picture in the display interface of the note application, the user can perform operations such as erasing and continuing to write the handwriting generated by the electronic device according to the editing function module.

[0130] The engine is the core component of the note-taking application. Using the engine, developers of note-taking applications can quickly build and lay out the required functions.

[0131] The following will combine Figure 4The graphical user interface (GUI) in the embodiment of the present application introduces the process of an electronic device using a note-taking App to convert handwriting on paper included in a picture into handwriting displayed in the electronic device.

[0132] For example, Figure 4 is a schematic diagram of a set of GUIs provided in an embodiment of the present application. Figure 4 (a) to (h) in the figure show the process of converting handwriting on paper into handwriting generated by the electronic device.

[0133] See also Figure 4 In (a), the GUI is a display interface 210 of the electronic device 200. The display interface 210 may be a display desktop of the electronic device 200, and may include multiple applications in the electronic device 200. For example, the application may include sports health, gallery, application store, camera, smart life, weather, clock, email, memo, handwriting, etc.

[0134] For example, when the electronic device 200 detects that the user clicks the icon 211 of the note application, the following information may be displayed: Figure 4 GUI shown in (b) in .

[0135] See also Figure 4 In (b), the GUI may be a display interface 220 of a note application. The display interface 220 may include a toolbar and a display area 214. The toolbar may include a select picture function button 211, an eraser function button 212, a write function button 213, a zoom function button, a brush function button, and the like.

[0136] For example, when the electronic device 200 detects that the user clicks the select picture function button 211, the following information may be displayed: Figure 4 GUI shown in (c) in FIG.

[0137] See also Figure 4 In (c), a function area 231 for selecting a picture and a function card 232 may be displayed in the display area 214. The function card 232 may include a photo taking option and a selection option from a gallery.

[0138] In other examples, the electronic device 200 may also first display the function area 231 for selecting a picture in the display area 214 , and then display the function card 232 when detecting that the user clicks on the function area 231 .

[0139] For example, when the electronic device 200 detects that the user clicks on the operation of selecting an option from the gallery, the following information may be displayed: Figure 4GUI shown in (d).

[0140] In other examples, after the electronic device 200 detects that the user clicks on a photo option, the camera may be called to take a picture.

[0141] See also Figure 4 In (d), the GUI may be a display interface 240 for the user to select a picture. The display interface 240 may be used to display pictures in the gallery of the electronic device 200. The display interface 240 may also include a display area 242, which may be used to display pictures selected by the user. The display area 242 may also include prompt information for reminding the user to select a picture, information on the number of selected pictures, and a completion function button 244.

[0142] Exemplarily, the picture 411 may be a picture including the user's handwriting that is pre-photographed or stored by the electronic device. For example, the picture 241 may include the text content "code" handwritten by the user and a rectangular graphic mark.

[0143] For example, the prompt information may be a text content such as "Please select the material you want" or "Please select a picture", etc. The embodiment of the present application does not limit the specific content of the prompt information.

[0144] For example, when the electronic device 200 detects that the user clicks on the picture 241, the following information may be displayed: Figure 4 GUI shown in (e) in .

[0145] See also Figure 4 In (e), the picture 241 selected by the user is in a selected state, and the picture 241 selected by the user is also displayed in the display area 242.

[0146] For example, when the electronic device 200 detects that the user clicks the completion function button 244, the following information may be displayed: Figure 4 GUI shown in (f) in .

[0147] See also Figure 4 In (f), the GUI may be a display interface 250 of the electronic device 200. The display interface 250 may include a picture 241 selected by a user, a function bar 252, and a function button 253 for rotating the picture 241.

[0148] The function bar 252 may include a plurality of function buttons, such as a crop function button, a copy function button, a delete function button, a handwriting function button 2521, and the like.

[0149] It should be understood that "handwriting" can be understood as a data set that converts the handwriting in a picture into points and radii. That is, subsequently, the electronic device can fill the outline of the handwriting based on the coordinates and radii of a certain number of points, so that the handwriting displayed on the electronic device is consistent with the handwritten handwriting in the picture.

[0150] Exemplarily, after the electronic device 200 detects the operation of the user clicking the handwriting function button 2521, it can display a GUI as shown in (g) of Figure 4 the figure.

[0151] Referring to Figure 4 (g) of the figure, the electronic device 200 can display the user's handwriting 261 in the picture 241 on the display interface 250. The handwriting 261 can include the text "code" and rectangular graphic marks or symbols.

[0152] It can be understood that after the electronic device 200 detects the operation of the user clicking the handwriting function button 2521, it can perform contour recognition on the handwritten handwriting in the picture 241. After the contour recognition, corresponding contour information can be obtained, and the contour information can include a rectangular contour and the font contour of the text "code".

[0153] The font contour of the text "code" can be the font contour of each character, or the contour of the continuous strokes included in each character. Exemplarily, for the character "代", the font contour can be the overall contour of "代", or the contour obtained by dividing "代" into three continuous strokes: the left radical, the horizontal plus oblique hook, and the dot.

[0154] After the contour recognition, the electronic device 200 can directly fill the font contour with a relevant color (such as black). Alternatively, the electronic device 200 can also use multiple circles with a relevant color to fill the font contour. For example, the electronic device first fills the font contour with multiple circles. The multiple circles can be non-overlapping with each other. When the number of circles is sufficient, the gap area between the circles exceeds the recognition range of the human eye, and it can be considered that the font contour is completely filled. Then, the electronic device renders and colors the multiple circles (such as black), and the handwriting consistent with the text content in the picture can be displayed. The electronic device can also use other shapes to fill the font contour.

[0155] In some other examples, the multiple circles can also overlap partially, and the embodiments of the present application do not limit this. In this way, since the multiple circles can overlap, the number of circles covering the font contour can be reduced, thereby improving the processing speed of the electronic device.

[0156] In this way, the electronic device 200 displays handwriting consistent with the text content in the picture in the display interface of the note app, efficiently moving the content recorded in the user's paper to the electronic device, and improving the user's experience.

[0157] It is understandable that the electronic device 200 can store the data of the center and radius of each circle, so that the electronic device 200 can call the data at any time to generate corresponding handwriting later.

[0158] In the present application, the color of the font outline filled by the electronic device can be consistent with the color of the text content in the image, or the electronic device can also use black to fill the font outline by default.

[0159] For example, when the electronic device 200 detects that the user clicks the eraser function button 212, the eraser can be displayed. When the electronic device 200 detects that the user uses the eraser to erase the displayed handwriting, the following information can be displayed: Figure 4 GUI shown in (h).

[0160] See also Figure 4 In (h), the handwriting displayed in the electronic device 200 is partially erased.

[0161] In other exemplary embodiments, the electronic device may also restore the erased handwriting according to other operations of the user. Alternatively, the electronic device may also continue to write other content in the displayed handwriting according to other operations of the user, which is not limited in the embodiments of the present application.

[0162] Based on the embodiment of the present application, the electronic device can convert the handwriting in the captured picture or the handwriting in the stored picture through the note application to generate handwriting that is consistent with the handwriting in the picture and displayed in the electronic device. In addition, the electronic device can also edit the converted handwriting according to the user's operation, so that the corresponding handwriting can be modified and continued to be created.

[0163] For example, Figure 5 FIG. 1 is a schematic flow chart of a method for converting handwriting provided in an embodiment of the present application. Figure 5 As shown, the method 300 may be applied in an electronic device, and the method 300 may include steps 310-380.

[0164] 310, the electronic device obtains a target image, where the target image includes the handwriting of the user.

[0165] In some examples, the electronic device may include a note application for converting handwriting, in which a target image may be captured by a camera. Alternatively, the electronic device may capture a target image in advance and store the target image in a memory of the electronic device or in a cloud server.

[0166] It should be understood that, in the case where the target image is stored in a memory of the electronic device, the electronic device may first read the target image from the memory into the internal memory for subsequent processing.

[0167] 320, the electronic device determines whether to perform pixel compression processing on the target image.

[0168] In a possible implementation manner, the electronic device may determine whether to perform pixel compression processing on the target image by determining the color gamut of the target image.

[0169] For example, when the electronic device determines that the color gamuts of the regions in the target image that are greater than or equal to the preset proportional area are in the same color gamut, it may be determined to perform pixel compression processing on the target image, and the electronic device may execute step 330 .

[0170] It should be understood that the electronic device can count the color values ​​of the pixels in the target image and determine its color range to determine the corresponding color gamut. For example, the color gamuts of pixels whose color value differences are within a preset difference (such as 100 or 50, etc.) can be considered to be in the same color gamut.

[0171] The embodiment of the present application does not limit the specific value of the preset ratio. For example, the preset ratio may be 90% or 80%.

[0172] Correspondingly, when the electronic device determines that the color gamuts of the areas in the target image that are smaller than the preset proportion are in the same color gamut, it can be determined not to perform pixel compression processing on the target image, and the electronic device can execute step 340.

[0173] In a possible implementation, the electronic device may also determine whether to perform pixel compression processing on the target image based on a user operation.

[0174] For example, when the electronic device detects that the user performs a grayscale operation on a picture, or when the electronic device detects that the user performs a pixel compression operation on a picture, it can be determined to perform pixel compression processing on the target picture, and the electronic device can execute step 330.

[0175] Correspondingly, when the electronic device does not detect the user's operation of grayscale on the image, or when the electronic device does not detect the user's operation of pixel compression on the image, it can be determined that pixel compression processing is not performed on the target image, and the electronic device can execute step 340.

[0176] It is understandable that the electronic device may also determine whether to perform pixel compression processing on the target image in other ways, and the embodiments of the present application do not limit this specific implementation method.

[0177] It should be understood that step 320 is an optional step. In other examples, step 320 may not be performed, and the embodiments of the present application are not limited thereto.

[0178] 330, the electronic device performs pixel compression processing on the target image.

[0179] Exemplarily, the pixel compression processing performed by the electronic device on the target image can be understood as storing each pixel point included in the target image with 1 bit, thereby reducing the amount of data processed by the electronic device on the target image and improving the speed of converting handwriting.

[0180] In some examples, the electronic device may also perform pixel compression processing on the target image by default.

[0181] It should be understood that step 330 may also be an optional step. In other examples, step 330 may not be performed, and the embodiments of the present application are not limited thereto.

[0182] At 340, the electronic device performs noise processing on the target image.

[0183] Exemplarily, the background color of the target image is white, and a pixel with a color (other than white) in the target image is not connected to other pixels with colors. The pixel can be considered as a noise point, and the electronic device can remove the pixel point.

[0184] It should be understood that the pixel point here may be a pixel point with a different color from the background color of the target image, and the present application does not limit the specific color of the pixel point.

[0185] Exemplarily, if the number of pixels of a color different from the background color included in a certain area in the target image is less than a preset number, the area can be considered as a noise area, and the electronic device can remove the pixels in the area.

[0186] The size of the certain area may be a preset size, such as a circle with a diameter of 10 or 20 pixels. The preset number is 10 or 5, and the specific value of the preset number is not limited in the embodiment of the present application.

[0187] It should be understood that the electronic device performs noise processing on the target image, which can be understood as the electronic device identifying the noise area in the target image and then deleting the data in the noise area.

[0188] It can be understood that this step 340 can also be an optional step. In other examples, this step 340 may not be executed, and the embodiments of the present application do not limit this.

[0189] 350, the electronic device obtains the contour information of the handwritten handwriting in the target picture.

[0190] The electronic device can use a graphic contour recognition algorithm to obtain the contour information of the handwritten handwriting in the target picture. For example, this graphic contour recognition algorithm can be the graphic contour recognition algorithm in the open source computer vision library (OPENCV).

[0191] For example, when the handwritten handwriting includes graphics and text, the contour information may include the graphic contour in the handwritten handwriting and the font contour of the text.

[0192] Exemplarily, referring to Figure 4 (g) in, this graphic mark can be a black rectangle, and the contour information of this black rectangle can be a rectangular contour. Specifically, the contour information of the rectangle can be the coordinate information of the four vertices of the rectangle.

[0193] The font contour of the text "code" can be the font contour of each character, or the contour of the continuous strokes included in each character. Exemplarily, for the character "代", the font contour can be the overall contour of "代", or can be the contour divided into three parts: the single-person radical, the horizontal and the oblique hook, and the dot according to the continuous strokes of the character "代". Specifically, the contour information can be the coordinate information of each point included in the contour.

[0194] It should be understood that for a rectangle with white in the middle, its contour information can be the coordinate information of the eight vertices of a rectangular ring with a certain thickness.

[0195] 360, the electronic device obtains the data information for filling the contour of the handwritten handwriting.

[0196] In some examples, the electronic device can use a shape to fill the contour of the handwritten handwriting. For example, the electronic device can use a circle to fill the contour of the handwritten handwriting.

[0197] Referring to Figure 6 , Figure 6 is a schematic diagram of filling a graphic contour provided by the embodiments of the present application. It should be understood that Figure 6 taking the shape used by the electronic device as a circle to fill the rectangular contour as an example for description.

[0198] As Figure 6 (a) in shows, the contour recognized by the electronic device is a rectangle.

[0199] When the electronic device uses a circle to fill the rectangular outline, see Figure 6 In (b), the electronic device can first draw an inscribed circle 1 of a rectangle; then, continue to draw inscribed circle 2, inscribed circle 3, inscribed circle 4 and inscribed circle 5 between inscribed circle 1 and the rectangle; then, continue to draw inscribed circle 5 and inscribed circle 6 between inscribed circle 1, inscribed circle 2 and the rectangle, continue to draw inscribed circle 7 and inscribed circle 8 between inscribed circle 1, inscribed circle 3 and the rectangle, continue to draw inscribed circle 9 and inscribed circle 10 between inscribed circle 1, inscribed circle 4 and the rectangle, and continue to draw inscribed circle 11 and inscribed circle 12 between inscribed circle 1, inscribed circle 5 and the rectangle. In this way, a large number of circles can be drawn to fill the rectangle. When the number of circles is large enough, it can be understood that these circles have filled the entire rectangular outline. The electronic device can record the center coordinates and radius of these circles, that is, the data information of the outline of the filled handwriting.

[0200] In some other possible implementations, the electronic device may also use multiple circles to fill or cover the rectangular outline. There may be partial overlap between the multiple circles, as long as the multiple circles can fill the entire rectangular outline. The embodiments of the present application are not limited to this.

[0201] For follow-up, see Figure 6 In (c), the electronic device fills all circles with color (such as black) according to the above data information, and a rectangle consistent with the handwriting in the target image can be obtained.

[0202] Optionally, the electronic device may also store data information of the outline of the filled handwriting in a memory of the electronic device or in a cloud server.

[0203] It is understandable that the electronic device can also use other shapes to fill the outline of the handwriting, such as equilateral triangles, polygons, etc.

[0204] In other exemplary embodiments, when the outline of the handwritten trace is a closed figure (such as a circle), the electronic device may also directly fill the closed figure with a related color.

[0205] It should be understood that for text in handwriting, when the electronic device recognizes its contour information, it can also first split the text into continuous strokes, determine the contour of the continuous strokes, and use the above-mentioned filling method to fill each continuous stroke, so as to obtain text consistent with the handwriting in the target image.

[0206] At 370 , the electronic device fills the outline of the handwriting with a target color according to the data information.

[0207] For example, when the data information is the center coordinates and radius of all circles used to fill the outline, the electronic device can fill all the circles with the target color according to the data information. Since these circles basically fill the outline of the handwriting, see Figure 6 (c) in FIG. 5 can be understood as filling the entire outline of the handwriting with the target color.

[0208] Exemplarily, the target color may be the default black, or may be the same color as the handwriting in the image, which is not limited in the embodiments of the present application.

[0209] 380. The electronic device edits the handwriting according to the user's operation.

[0210] For example, see Figure 4 In (h), the electronic device can erase part of the handwriting according to the user's erasing operation. Alternatively, the electronic device can also recognize the content added by the user according to the user's writing operation. In this way, when the user makes a partial mistake in handwriting, it can be accurately modified in the electronic device, further improving the user experience.

[0211] It is understandable that step 380 may also be an optional step. In other examples, step 380 may not be performed, and the embodiment of the present application is not limited thereto.

[0212] Based on the embodiments of the present application, the electronic device can recognize the outline of handwriting and form a handwriting entity by filling the outline, so that the electronic device can efficiently display the text or special graphic symbols recorded by the user on paper in the form of handwriting in the electronic device without any recognition errors, which will greatly improve the user experience.

[0213] Figure 7 FIG. 1 is a schematic flow chart of a method for converting handwriting provided in an embodiment of the present application. Figure 7 As shown, the method 600 may be applied to an electronic device, and the method 600 may include steps 610 to 640.

[0214] 610, the electronic device obtains a target image, where the target image includes target identification information.

[0215] In some examples, the electronic device can use a camera to take a target picture including target identification information. For example, the electronic device can use a camera in a note application to take a target picture.

[0216] In other examples, the electronic device may also pre-take a target picture and store it in a memory of the electronic device or a cloud server. When needed, the electronic device calls out the target picture from the gallery.

[0217] The target recognition information may include the user's handwriting. Alternatively, the target recognition information may also include the user's handwriting and a certain type of printed font (such as Kaiti, Songti, etc., non-handwritten fonts) or symbols.

[0218] The handwriting may include text information and graphic symbols.

[0219] It should be understood that the graphic symbol may also be referred to as a graphic logo, a pictorial mark, etc.

[0220] At 620 , the electronic device determines an outline of the target identification information.

[0221] In some implementations, the electronic device may determine the contour of the target identification information by using a contour recognition algorithm. The embodiment of the present application does not limit the specific contour recognition algorithm.

[0222] For example, when the target recognition information includes handwriting, and the handwriting includes graphics and text, the outline may include the outline of the graphics in the handwriting and the outline of the font of the text.

[0223] Alternatively, the outline may also be an outline of a graphic in handwriting or an outline of continuous strokes of a text.

[0224] 630 , the electronic device determines data information for filling the outline.

[0225] In some examples, the electronic device may fill the outline with multiple circles, and the data information may be a set of center coordinates and radii of the multiple circles.

[0226] In other examples, the electronic device may also use multiple other shapes to fill the outline, and the data information may be related information of the other shapes.

[0227] At 640 , the electronic device fills the outline of the target identification information with the target color according to the data information.

[0228] Exemplarily, the target color may be black by default, or the target color may be the same color as the target identification information.

[0229] The electronic device can fill the outline of the target identification information with the target color according to the data information, thereby forming a graphic entity of the target identification information and displaying it in the electronic device, and the target identification information displayed in the electronic device is consistent with the target identification information in the target picture.

[0230] Based on the embodiment of the present application, the electronic device can obtain a picture including target identification information, determine the outline of the target identification information, further determine data information that can fill the outline, and fill the outline of the target identification information with the target color according to the data information. In this way, the target identification information in the target picture can be formed into a graphic entity in the form of a filled outline and displayed in the electronic device, so that the target identification information can be fully displayed in the electronic device, avoiding the phenomenon of recognition errors or incompleteness.

[0231] In some embodiments, the electronic device determines data information for filling a contour, including:

[0232] Electronic devices use multiple circular filled outlines;

[0233] The electronic device determines the center coordinates and radii of multiple circles, wherein the data information is a collection of the center coordinates and radii of the multiple circles.

[0234] For example, see Figure 6 In (b) and (c), the electronic device can use multiple non-overlapping circles to fill the rectangular outline. When the number of circles is large enough and the gaps between the circles are small enough to exceed the recognition range of the human eye, it can be considered that the multiple circles have filled the rectangular outline. Then, the electronic device can record the center coordinates and radius of the multiple circles.

[0235] In other examples, there may be partial overlap between the multiple circles, as long as the multiple circles can fill the entire outline.

[0236] Based on the embodiment of the present application, the electronic device can use multiple circles to fill the outline of the target recognition information, and determine the center coordinates and radius of the multiple circles. In this way, the electronic device can store the set of the center coordinates and radius of the multiple circles converted from the outline, so that the outline of the target recognition information can be filled according to the center coordinates and radius of the multiple circles to restore the target recognition information.

Claims

1. A method for converting handwriting, characterized in that: The method is applied to an electronic device, and the method comprises: Acquire a target image, wherein the target image includes target identification information; determining an outline of the target identification information; Determine data information for filling the outline; The outline of the target identification information is filled with the target color according to the data information.

2. The method according to claim 1, characterized in that The determining of data information for filling the outline includes: filling the outline with a plurality of circles; Determine the center coordinates and radii of the multiple circles, wherein the data information is a set of the center coordinates and radii of the multiple circles.

3. The method according to claim 2, characterized in that Filling the outline of the target identification information with the target color according to the data information includes: The plurality of circles are filled with a target color according to the data information.

4. The method according to any one of claims 1 to 3, characterized in that Before determining the outline of the target identification information, the method further includes: Perform pixel compression on the target image.

5. The method according to claim 4, characterized in that The pixel compression processing of the target image includes: When the color gamuts of the regions in the target image that are greater than or equal to the preset proportion area are in the same color gamut, pixel compression processing is performed on the target image.

6. The method according to any one of claims 1 to 5, characterized in that Before determining the outline of the target identification information, the method further includes: Determining a noise area in the target image; The data in the noise area is deleted.

7. The method according to any one of claims 1 to 6, characterized in that The determining the noise area in the target image includes: When the number of pixel points in the target area that are different in color from the background color of the target image is less than a preset number, the target area is determined to be the noise area.

8. The method according to any one of claims 1 to 7, characterized in that The target recognition information includes handwriting of a user, and determining the outline of the target recognition information includes: Determine the overall outline of the handwriting, or determine the outlines of continuous strokes of the handwriting.

9. The method according to claim 8, characterized in that The handwriting includes text information and graphic symbols.

10. The method according to claim 8 or 9, characterized in that: The method further comprises: In response to the user's operation, the handwritten trace is edited.

11. The method according to any one of claims 1 to 10, characterized in that The target color is black, or the target color is the same color as the target identification information.

12. An electronic device, characterized in that: include: one or more processors; One or more memories; the one or more memories store one or more programs, and when the one or more programs are executed by one or more processors, the method for converting handwriting as described in any one of claims 1-11 is executed.

13. A handwriting conversion device, characterized in that: The method comprises a module for implementing the method for converting handwriting as described in any one of claims 1-11.

14. A chip, characterized in that: The chip includes a processor and a communication interface, wherein the communication interface is used to receive a signal and transmit the signal to the processor, and the processor processes the signal so that the method for converting handwriting as described in any one of claims 1 to 11 is executed.

15. A readable storage medium, characterized in that: The readable storage medium stores instructions, and when the instructions are executed on an electronic device, the method for converting handwriting according to any one of claims 1 to 11 is executed.

Images