Method for beautifying handwriting and electronic equipment

Abstract

The invention provides a handwritten character beautifying method and electronic equipment, on one hand, information of a first handwritten character can be obtained, and on the other hand, a fusion style can be determined through information of a first gear, a handwritten character library and a target font. The fusion style can be used for representing a style after the target font and the handwritten font corresponding to the character in the handwritten font library are fused, and finally generating a first beautified character corresponding to the first handwritten character based on the information of the first handwritten character and the fusion style. According to the method and the device, the handwritten characters of the user can be beautified based on the first gear selected by the user, and the styles of the handwritten characters of the user can be reserved in the finally presented handwritten character beautification effect.

Description

Technical Field

[0001] The present application relates to the field of electronic whiteboard handwriting, and more specifically, to a method and electronic device for beautifying handwriting. Background Art

[0002] With the rapid development of society, traditional teaching and conference systems have been gradually replaced by emerging electronic products. Cloud teaching and cloud conferencing have become more and more well-known and accepted by the public, and touch display devices such as electronic whiteboards have emerged. They have replaced traditional non-electronic devices such as blackboards and chalks, greatly saving costs and being environmentally friendly. They can also better connect to the cloud to meet a variety of teaching or conference needs. However, most of the whiteboard software or screens on the market are often not beautiful when writing due to screen slippage, software delays, etc. Therefore, the fonts displayed on the electronic whiteboard lack beauty.

[0003] The existing solutions mainly beautify the handwriting written by users on the electronic whiteboard through technologies such as font topology structure and handwriting recognition. However, in the current solutions, the fonts of handwriting written by different users on the electronic whiteboard after beautification are relatively consistent, and the style of the user's personal handwriting cannot be preserved. In other words, the beautification effect achieved by the existing solutions has a single style and cannot provide personalized handwriting beautification effects for different users. Summary of the invention

[0004] The present application provides a method for beautifying handwriting, which can not only beautify the handwriting of the user on the screen, but also retain the style of the user's handwriting while beautifying the handwriting, and provide different beautification effects for different users.

[0005] In a first aspect, a method for beautifying handwriting is provided, which can be executed by a terminal device (eg, a mobile phone, a laptop computer, etc.), or can also be executed by a component of the terminal device (eg, a chip or a circuit), without limitation.

[0006] The method includes: obtaining information of a first handwritten character, the first handwritten character is a character written by a user on a handwriting area of ​​an electronic device; detecting a first input operation of the user, and obtaining information of a first gear in response to the first input operation; determining a fusion style according to the information of the first gear, a handwritten character library, and a target font, wherein the handwritten character library is a character written by a user on a handwriting area of ​​the electronic device and stored in the electronic device, the target font is a font preset in the target font library of the electronic device, the fusion style is used to characterize the style of the target font and the handwritten font corresponding to the character included in the handwritten character library after fusion, and the first gear is used to determine the ratio between the target font style corresponding to the target font and the handwritten font style corresponding to the handwritten font in the fusion style; generating a first beautified character, the first beautified character being the first handwritten character beautified according to the information of the first handwritten character and the fusion style.

[0007] In the present application, "first handwriting" refers to the words written by the user on the electronic whiteboard, "second handwriting" refers to the words in the user's handwriting library stored in the terminal device, and "standard words" refer to the words preset in the standard font library of the terminal device. "Target word" refers to the word corresponding to at least one font preset in the target font library of the terminal device. In the present application, in one possible implementation, it can be said that the user's handwriting library is obtained by collecting the user's historical handwriting. In another possible implementation, the terminal device can obtain the user's handwriting library by collecting the user's current handwriting.

[0008] In the present application, "the information of the first gear position" may indicate the "first gear position", or it may be understood that the first gear position may be determined according to the information of the first gear position. The "first gear position" may be specified by the user. Exemplarily, the first gear position may be set to three levels, namely "low", "medium", "high", or "weak", "medium", "strong", etc. The user may input the required first gear position on the interface of the terminal device.

[0009] In the present application, the first gear is used to determine the ratio between the target font style corresponding to the target font in the fusion style and the handwriting font style corresponding to the handwriting font. It can also be understood that the first gear is used to determine the weight between the target font style corresponding to the target font in the fusion style and the handwriting font style corresponding to the handwriting font. In the present application, "ratio" can also be replaced by "weight".

[0010] In the present application, different first gears can represent the ratio between the handwriting font style (i.e., the handwriting font style corresponding to the handwritten characters) and the target font style (i.e., the target font style corresponding to the target characters) in the fusion style. The higher the first gear, the higher the proportion of the target font style in the fusion style of the generated beautified characters, and the lower the first gear, the higher the proportion of the handwriting font style in the fusion style of the generated beautified characters. Exemplarily, if the first gear selected by the user is "low", it indicates that the generated beautified characters are more like the user's own handwriting; if the first gear selected by the user is "high", it indicates that the generated beautified characters are more like the target font selected by the user. In terms of specific implementation, in one possible implementation, the first gear affects the number of handwritten characters and the number of target characters. In another possible implementation, the first gear affects the ratio between the first coefficient corresponding to the handwriting font style and the second coefficient corresponding to the target font style.

[0011] In the present application, the information of the first handwriting includes an image corresponding to the first handwriting and / or a trajectory corresponding to the first handwriting.

[0012] In the present application, the first input operation, for example, can be understood as a first instruction input by a user, and the first instruction is used to indicate information of a first gear position.

[0013] Based on the above technical solution, in this application, on the one hand, the information of the first handwritten word can be obtained, and on the other hand, the fusion style can be determined through the first gear, the handwritten font library and the target font. The fusion style can be used to characterize the style of the handwritten font corresponding to the user's handwritten word in the handwritten font library after fusion, and finally generate the first beautified word corresponding to the first handwritten word based on the information of the first handwritten word and the fusion style. Not only can the user's handwritten word be beautified based on the first gear selected by the user, but the user's personal handwritten style can also be retained in the beautification effect of the handwritten word finally presented.

[0014] In combination with the first aspect, in a possible implementation, the fonts pre-set in the target font library of the electronic device include at least two fonts, and the method also includes: receiving a second instruction, the second instruction is used to indicate a target font, wherein the target font is one of the at least two fonts; and determining the target font according to the second instruction.

[0015] In the present application, the target font library of the electronic device may include multiple fonts, and the user may select one of the target fonts according to the needs. In the present application, multiple different fonts may be pre-set inside the electronic device. For example, the multiple different fonts may include: Kaiti, Lishu, Songti, etc. In some scenarios, if the user does not select the target font, the electronic device may default to a certain font in the target font library as the target font. The present application does not limit the method for determining the target font.

[0016] In combination with the first aspect, in a possible implementation, the method also includes: receiving a third instruction, the third instruction is used to indicate a second handwritten character, the second handwritten character is a character written by a user in a handwriting area of ​​an electronic device, and the second handwritten character is used to generate the handwritten character library; according to the third instruction, a handwritten character library is generated, and the handwritten character library includes one or more second handwritten characters.

[0017] Exemplarily, the "second handwriting" is some words circled by the user in a paragraph of text written by the user before, and these circled words constitute the handwriting library; Exemplarily, the "second handwriting" can be based on some words circled by the user in the first handwriting currently written, so as to constitute the handwriting library. For example, the first handwriting may be more, so the user can circle some words in the first handwriting as the second handwriting, so as to form the handwriting library; Exemplarily, as long as the user writes words in the handwriting area of ​​the electronic device, the handwriting can be automatically added to the handwriting library. It can also be understood that the second handwriting in the handwriting library in this application can come from the first handwriting or from the words written by the user before, etc., without limitation.

[0018] In combination with the first aspect, in a possible implementation, the method further includes: receiving a fourth instruction, the fourth instruction being used to instruct updating a second handwritten character in the handwritten character library; generating an updated handwritten character library according to the fourth instruction, the updated handwritten character library including one or more updated second handwritten characters.

[0019] In this application, the handwritten characters in the handwritten character library can also be updated. Here, "update" can be understood as replacing the handwritten characters in the handwritten character library, or adding handwritten characters to the handwritten character library. For example, the user can subsequently rewrite a paragraph of text in the handwriting area of ​​the electronic device and circle some handwritten characters to replace the handwritten characters in the handwriting library.

[0020] In combination with the first aspect, in a possible implementation method, a fusion style is determined based on information of the first gear, a handwritten character library, and a target font, including: determining target characters corresponding to a first number of second handwritten characters and a second number of target fonts based on information of the first gear, wherein the first number and the second number are determined based on information of the first gear; determining a fusion style based on target characters corresponding to the first number of second handwritten characters and the second number of target fonts.

[0021] In the present application, the fusion style can be determined by extracting a first number of second handwritten characters and a second number of target characters. Specifically, the fusion style can be determined by the following method:

[0022] Method 1:

[0023] In a possible implementation, the fusion style is determined by simultaneously extracting the font styles of the first number of second handwritten characters and the second number of target characters. This implementation can also be understood as extracting the font styles of the first number of second handwritten characters and the second number of target characters (i.e., all characters input) simultaneously, thereby obtaining the fusion style.

[0024] Method 2:

[0025] In another possible implementation, the fusion style is determined based on the first number of second handwritten characters and the target characters corresponding to the second number of target fonts, including: extracting the handwritten font style corresponding to each second handwritten character word by word, and extracting the target font style corresponding to each target character word by word; determining the fusion style based on the handwritten font style corresponding to each second handwritten character and the target font style corresponding to each target character. This implementation can also be understood as extracting the font style of each input handwritten character or each target character word by word, and then reshaping and splicing the font style corresponding to each extracted character to obtain the fusion style.

[0026] Method 3:

[0027] In another possible implementation, the fusion style is determined according to the information of the first gear, the handwriting font library, and the target font, including: determining any number of second handwritings and any number of target fonts corresponding to the target fonts; determining the style of the handwriting font corresponding to the second handwriting according to any number of second handwritings, and determining the style of the target font corresponding to the target font according to any number of target fonts; determining the second ratio according to the information of the first gear, wherein the second ratio is the ratio between the second coefficient and the first coefficient, the second coefficient is used to characterize the ratio of the style of the target font in the fusion style, and the first coefficient is used to characterize the ratio of the style of the handwriting font in the fusion style; determining the fusion style according to the first coefficient, the style of the handwriting font, the second coefficient, and the style of the target font. In this implementation, the number of input handwritings and the number of target characters in the target character set are not limited, but the handwriting font style corresponding to the extracted second handwriting and the target font style corresponding to the target character can be weighted according to their respective corresponding weighting coefficients to obtain the fusion style.

[0028] In combination with the first aspect, in a possible implementation, the method further includes: determining a first ratio based on information of the first gear, the first ratio being the ratio between the second quantity and the first quantity; and determining the first quantity and the second quantity based on the first ratio.

[0029] In terms of specific implementation, for example, the first ratio refers to the ratio of the second quantity to the first quantity, and for another example, the first ratio refers to the ratio of the first quantity to the second quantity, without limitation.

[0030] Exemplarily, in the present application, the beautification module 130 can directly determine the number of handwritten characters and the number of target characters according to the different first gears determined by the user. Exemplarily, if the first gear selected by the user is "low", the beautification module 130 can directly determine the number of handwritten characters to be 7 and the number of target characters to be 3. Exemplarily, if the first gear selected by the user is "medium", the beautification module 130 can directly determine the number of handwritten characters to be 5 and the number of target characters to be 5. Exemplarily, if the first gear selected by the user is "high", the beautification module 130 can directly determine the number of handwritten characters to be 3 and the number of target characters to be 7.

[0031] Exemplarily, the beautification module 130 can determine the first ratio according to the first gear determined by the user. For example, the first ratio is the ratio between the number of handwritten characters and the number of target characters, and the number of handwritten characters and the number of target characters are determined according to the first ratio. Exemplarily, if the first gear selected by the user is "low", the beautification module 130 can determine that the ratio between the number of target characters and the number of handwritten characters is 3:7 (number of handwritten characters: number of target characters), and further, the number of handwritten characters is determined to be 7 and the number of target characters is 3. Exemplarily, if the first gear selected by the user is "medium", the beautification module 130 can determine that the ratio between the number of target characters and the number of handwritten characters is 5:5 (number of handwritten characters: number of target characters), and further, the number of handwritten characters is determined to be 5 and the number of target characters is 5. Exemplarily, if the first gear selected by the user is "high", the beautification module 130 can determine that the ratio between the number of target characters and the number of handwritten characters is 7:3 (number of handwritten characters: number of target characters), and further determine that the number of handwritten characters is 3 and the number of target characters is 7.

[0032] Based on the above technical solution, in the present application, it is possible to achieve multi-level handwriting beautification effects based on the user selecting the first level, and the beautification effect on handwriting in the present application can retain the user's personal handwriting style. The user can choose different first levels to achieve whether the final beautified words are biased towards the user's handwriting style or the target font style.

[0033] In combination with the first aspect, in a possible implementation, generating a first beautified character includes: determining a standard character corresponding to the first handwritten character based on information of the first handwritten character, the standard character being a character preset in a standard character library of the terminal device; generating the first beautified character based on the standard character and a fusion style.

[0034] In this application, "standard characters" can be understood as characters with correct stroke order and squareness. The standard characters can be characters without any fonts, or characters with fonts, without limitation.

[0035] In combination with the first aspect, in a possible implementation, generating a first beautified character according to the standard character and the fusion style includes: extracting content features corresponding to the standard character, the content features are used to determine the standard character; generating the first beautified character according to the content features and the fusion style.

[0036] The "content feature" in this application can be used to characterize the stroke structure information of the standard character and / or the semantic information of the standard character. For example, the specific character of the standard character can be determined based on the content feature.

[0037] In combination with the first aspect, in a possible implementation, the method also includes: determining the stroke information of the standard character, wherein the stroke information includes the stroke order of the standard character and / or the stroke category of the standard character; generating a first beautified character based on the content characteristics and the fusion style, including: generating the first beautified character based on the content characteristics and the fusion style, and the stroke information of the standard character.

[0038] Based on the above technical solution, the problem of font collapse in the trajectory of generating beautified characters can be effectively alleviated, the error rate of generated beautified characters can be reduced, and the generation quality of beautified characters can be improved.

[0039] In combination with the first aspect, in a possible implementation, generating a first beautified character based on a standard character and a fusion style includes: generating a first beautified character based on a standard character and a fusion style, and a first model, wherein a training data set of the first model includes at least one of the following: a training data set of handwritten characters, a training data set of target characters, and a training data set of beautified characters, and the training data set of beautified characters includes training beautified characters of different first gears.

[0040] In combination with the first aspect, in a possible implementation method, the training beautified characters are obtained based on a first training standard character, a first training handwritten character set, a first training target character set, and a second model, the first training handwritten character set includes a third number of training handwritten characters, the first training target character set includes a fourth number of training target characters, the third number and the fourth number are determined based on the information of the first gear, wherein the training data set of the second model is a training data set of handwritten characters, when training the second model, the input of the second model is the second training standard characters and the second training handwritten character set, and the training label is the second training handwritten character set, or; the training data set of the second model is a training data set of target characters, when training the second model, the input of the second model is the second training standard characters and the second training target character set, and the training label is the second training target character set.

[0041] It should be understood that the data set in the present application may be a public data set, which is used for model training of beautified characters. For example, the data set may further include a training data set and an inference data set. Among them, the training data set is used for model training, and the inference data set is used for model inference. For example, the training data set may include a training data set of standard characters, a training data set of handwritten characters, and a training data set of target characters. For example, the inference data set may include an inference data set of standard characters, an inference data set of handwritten characters, and an inference data set of target characters. In the present application, the training data set and the inference data set are no longer distinguished, and are collectively referred to as "training data sets".

[0042] In the present application, it is considered that when the first gear is higher, the target font style corresponding to the target font gradually increases, and the error rate of the generated beautified characters will increase. This is because, under normal circumstances, the training data set is mostly handwritten words written by users on electronic whiteboards, which are usually sloppy and not neat enough, resulting in a high proportion of target words under the first gear. The "decoder" in the model is prone to errors in reconstructing the words, and re-collecting neat handwritten words requires a certain amount of manpower costs. In view of this, in order to expand the training data set of the model and make the model perform better, the present application proposes that since the output of the second model is beautified words of different gears, the output result of the second model can be used as the input of the first model, so as to further train the first model and optimize the performance of the first model. The beautified words output by the model are more accurate, and the generation quality of the beautified words is higher.

[0043] It should be understood that the first model and the second model in the present application can be the same model. At this time, it is equivalent to the output result of the first model being used as its own input again, and the first model is trained again to optimize the performance of the first model.

[0044] In combination with the first aspect, in a possible implementation, the method also includes: determining the pen stroke width of the first beautified character, and / or smoothing the pen stroke width of the first beautified character; rendering the first beautified character according to the pen stroke width and / or the smoothed pen stroke width to generate a rendered first beautified character.

[0045] Based on the method of rendering beautified characters proposed above, the thickness of the pen strokes of the generated beautified characters can be changed, which is closer to the characteristics of users actually writing on the electronic whiteboard, thereby improving the user experience.

[0046] In combination with the first aspect, in a possible implementation, the method also includes: determining format information of the first handwritten character, the format information of the first handwritten character includes at least one of the following: the position of the first handwritten character, the size of the first handwritten character, the length-to-width ratio of the first handwritten character, and the direction of the first handwritten character; determining format information of the first beautified character based on the format information of the first handwritten character, wherein the format information of the first beautified character includes at least one of the following: the position of the first beautified character, the size of the first beautified character, the length-to-width ratio of the first beautified character, and the direction of the first beautified character; and performing single-word adjustments on the first beautified character based on the format information of the first beautified character.

[0047] Based on the above technical solution, in this application, the size, position and other information of the handwritten characters can be retained so that the handwritten characters will not be misplaced after beautification, or it can also be understood that the beautified characters and handwritten characters generated in this application can be well fitted.

[0048] In combination with the first aspect, in a possible implementation, the method also includes: performing multi-word adjustment on multiple first beautified characters, and the multi-word adjustment includes at least one of the following: adjusting the character spacing between adjacent first beautified characters; and / or, aligning multiple first beautified characters in rows; and / or, adjusting the size of all first beautified characters in one or more rows.

[0049] Based on the above technical solution, in this application, in order to make the display effect of multiple beautified characters more neat and beautiful and improve the user experience, this application also proposes to optimize the character spacing, line spacing, etc. of multiple beautified characters.

[0050] In a second aspect, an electronic device is provided, which is used to execute the method in any possible implementation of the first aspect. Specifically, the electronic device may include a unit and / or module, such as a transceiver unit and / or a processing unit, for executing the method in any possible implementation of the first aspect.

[0051] In one implementation, the electronic device is a terminal device, the communication unit may be a transceiver, or an input / output interface; the processing unit may be at least one processor. Optionally, the transceiver may be a transceiver circuit. Optionally, the input / output interface may be an input / output circuit.

[0052] In another implementation, the electronic device is a chip, a chip system or a circuit for a terminal device. When the device is a chip, a chip system or a circuit for an electronic device, the communication unit may be an input / output interface, an interface circuit, an output circuit, an input circuit, a pin or a related circuit on the chip, the chip system or the circuit; the processing unit may be at least one processor, a processing circuit or a logic circuit.

[0053] In a third aspect, an electronic device is provided, the electronic device comprising: at least one processor, configured to execute a computer program or instruction stored in a memory, so as to execute the method in any possible implementation of the first aspect. Optionally, the device further comprises a memory, configured to store a computer program or instruction. Optionally, the device further comprises a communication interface, and the processor reads the computer program or instruction stored in the memory through the communication interface.

[0054] In one implementation, the electronic device is a terminal device.

[0055] In another implementation, the electronic device is a chip, a chip system or a circuit for a terminal device.

[0056] In a fourth aspect, the present application provides a processor, comprising: an input circuit, an output circuit, and a processing circuit. The processing circuit is used to receive a signal through the input circuit and transmit a signal through the output circuit, so that the processor executes the method in any possible implementation of the first aspect.

[0057] In the specific implementation process, the processor can be one or more chips, the input circuit can be an input pin, the output circuit can be an output pin, and the processing circuit can be a transistor, a gate circuit, a trigger, and various logic circuits. The input signal received by the input circuit can be received and input by, for example, but not limited to, a transceiver, and the signal output by the output circuit can be, for example, but not limited to, output to a transmitter and transmitted by the transmitter, and the input circuit and the output circuit can be the same circuit, which is used as an input circuit and an output circuit at different times. The embodiments of the present application do not limit the specific implementation methods of the processor and various circuits.

[0058] For the operations such as sending and acquiring / receiving involved in the processor, unless otherwise specified, or unless they conflict with their actual function or internal logic in the relevant description, they can be understood as operations such as processor output, reception, input, etc., or as sending and receiving operations performed by the radio frequency circuit and antenna, and this application does not limit this.

[0059] In a fifth aspect, a processing device is provided, comprising a processor and a memory. The processor is used to read instructions stored in the memory, and can receive signals through a transceiver and transmit signals through a transmitter to execute the method in any possible implementation of the first aspect.

[0060] Optionally, the number of the processors is one or more, and the number of the memories is one or more.

[0061] Optionally, the memory may be integrated with the processor, or the memory may be provided separately from the processor.

[0062] In the specific implementation process, the memory can be a non-transitory memory, such as a read-only memory (ROM), which can be integrated with the processor on the same chip or can be set on different chips respectively. The embodiments of the present application do not limit the type of memory and the setting method of the memory and the processor.

[0063] It should be understood that the related data interaction process, such as sending indication information, can be a process of outputting indication information from the processor, and receiving capability information can be a process of receiving input capability information by the processor. Specifically, the data output by the processor can be output to the transmitter, and the input data received by the processor can come from the transceiver. Among them, the transmitter and the transceiver can be collectively referred to as a transceiver.

[0064] The processing device in the fifth aspect may be one or more chips. The processor in the processing device may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc.; when implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory, which may be integrated in the processor or located outside the processor and exist independently.

[0065] In a sixth aspect, a computer-readable storage medium is provided, which stores a program code for execution by a device, wherein the program code includes a method for executing any possible implementation of the first aspect described above.

[0066] In a seventh aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute a method in any possible implementation of the first aspect.

[0067] In an eighth aspect, a chip system is provided, comprising a processor for calling and running a computer program from a memory, so that a device equipped with the chip system executes the methods in each implementation manner in any one of the first to middle aspects above. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 It is a schematic structural block diagram of an electronic device 100 provided in an embodiment of the present application.

[0069] Figure 2 It is a schematic diagram of the software architecture of the electronic device 100 applicable to the embodiment of the present application.

[0070] Figure 3 It is a schematic structural block diagram of a handwriting beautification system 300 provided in an embodiment of the present application.

[0071] Figure 4 It is a schematic diagram of handwritten segmentation provided in an embodiment of the present application.

[0072] Figure 5 It is a schematic flow chart of a method 500 for beautifying handwriting provided in an embodiment of the present application.

[0073] Figure 6 It is a schematic block diagram of a method for beautifying handwriting provided in an embodiment of the present application.

[0074] Figure 7 It is a schematic diagram of the beautification effect of beautifying characters under different first gears provided in an embodiment of the present application.

[0075] Figure 8 It is a schematic block diagram of another method for beautifying handwriting provided in an embodiment of the present application.

[0076] Fig. 9 It is a schematic diagram of another beautification effect of beautifying characters at different first gears provided in an embodiment of the present application.

[0077] Fig.10 This is a schematic block diagram of another method for beautifying handwriting provided in an embodiment of the present application.

[0078] Fig.11 It is a schematic diagram of another beautification effect of beautifying characters at different first gears provided in an embodiment of the present application.

[0079] Fig.12 It is a schematic diagram of the collapse of a generated beautified character provided in an embodiment of the present application.

[0080] Fig.13 This is a schematic block diagram of another method for beautifying handwriting provided in an embodiment of the present application.

[0081] Fig.14 It is a schematic diagram of the beautification effect of beautifying characters after introducing the stroke information of standard characters provided in an embodiment of the present application.

[0082] Fig.15 It is a schematic block diagram of a model for training beautified characters provided in an embodiment of the present application.

[0083] Fig.16 It is another schematic block diagram of a model for training beautified characters provided in an embodiment of the present application.

[0084] Fig.17 It is a schematic flow chart of calculating the pen stroke width of beautified characters and smoothing the pen stroke width provided in an embodiment of the present application.

[0085] Fig.18It is a schematic diagram of the S curve provided in the embodiment of the present application.

[0086] Fig.19 It is a schematic diagram of interpolating trajectory points of beautified characters provided in an embodiment of the present application.

[0087] Fig. 20 It is a schematic diagram of the beautification effect after rendering of beautified characters provided in an embodiment of the present application.

[0088] Fig.21 It is a schematic diagram of determining the center of gravity of handwritten characters provided in an embodiment of the present application.

[0089] Fig. 22 It is a schematic diagram of determining the distance from the key point of handwritten characters to the surrounding areas provided in an embodiment of the present application.

[0090] Fig.23 It is a schematic diagram of a single beautified word and a corresponding handwritten word provided in an embodiment of the present application.

[0091] Fig.24 It is a schematic diagram of the bonding of multiple beautified characters and corresponding multiple handwritten characters provided in an embodiment of the present application.

[0092] Fig.25 It is a schematic diagram of aligning multiple beautified characters in rows provided in an embodiment of the present application.

[0093] Fig.26 It is a schematic diagram of adjusting the line spacing of multiple beautified characters provided in an embodiment of the present application.

[0094] Fig. 27 It is a schematic diagram of aligning multiple lines of beautified characters provided in an embodiment of the present application.

[0095] Fig.28 It is a schematic diagram of aligning a title of beautified words provided in an embodiment of the present application.

[0096] Fig.29 It is a schematic diagram of character segmentation and segmentation of beautified characters provided in an embodiment of the present application.

[0097] Fig.30 It is a character frame schematic diagram of a target font provided in an embodiment of the present application.

[0098] Fig.31 It is a schematic diagram of various parameters in a character box of a target font provided in an embodiment of the present application.

[0099] Fig.32 It is a schematic diagram of the beautification effect of beautifying characters provided in an embodiment of the present application.

[0100] Fig.33This is a schematic diagram of beautifying handwritten characters on a user interface provided by an embodiment of the present application.

[0101] Fig.34 It is a schematic block diagram of an electronic device 1000 provided in an embodiment of the present application.

[0102] Fig.35 It is a schematic block diagram of an electronic device 2000 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0103] The technical solution in this application will be described below in conjunction with the accompanying drawings.

[0104] The technical solution provided in the present application can also be applied to electronic devices, which are included in terminal devices that support user handwriting, including but not limited to user equipment (UE), tablets, personal computers (PC), office large screens and other screen devices that support writing with fingers or stylus pens, and also include virtual reality (VR), augmented reality (AR) and other virtual projection devices that support writing with hands or handles. The terminal device may also be a wireless terminal in industrial control, a machine type communication (MTC) terminal, a customer premise equipment (CPE), a wireless terminal in self-driving, a wireless terminal in remote medical, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a smart home, a cellular phone, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), a handheld device with wireless communication function, a computing device or other processing device connected to a wireless modem, a vehicle-mounted device, a wearable device, a terminal device in a 5G network, or a terminal device in a future evolved public land mobile network (PLMN), etc.

[0105] For example, Figure 11 shows a schematic diagram of the structure of the electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0106] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0107] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0108] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0109] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0110] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0111] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0112] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0113] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0114] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0115] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0116] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0117] The mobile communication module 150 may provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc.

[0118] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0119] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0120] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1. Generally, the display screen 194 serves as an important entrance for the user to interact with the electronic device. For example, in this application, the user can write handwritten words on the display panel; for example, the user can transmit the user's instructions to the electronic device 100 through the display screen.

[0121] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0122] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0123] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0124] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0125] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0126] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.

[0127] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0128] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0129] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0130] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0131] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0132] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0133] Figure 2 1 is a software structure diagram of the electronic device 100 of the embodiment of the present application. The layered architecture divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer. The application layer can include a series of application packages.

[0134] like Figure 2 As shown, the application package may include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and other applications.

[0135] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0136] like Figure 2 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0137] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0138] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0139] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0140] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including connecting, hanging up, etc.).

[0141] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0142] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0143] Android runtime includes core libraries and virtual machines. Android runtime is responsible for scheduling and management of the Android system.

[0144] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0145] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0146] The system library may include multiple functional modules, such as surface manager, media libraries, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0147] The surface manager is used to manage the display subsystem and provide the fusion of 2D and 3D layers for multiple applications.

[0148] The media library supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0149] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0150] A 2D graphics engine is a drawing engine for 2D drawings.

[0151] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0152] It should be understood that the technical solutions in the embodiments of the present application can be used in Android, IOS, Hongmeng and other systems.

[0153] Figure 3 3 is a schematic structural block diagram of a handwriting beautification system 300 provided in an embodiment of the present application. For example, the system can be applied in electronic devices. Figure 3 As shown, the system includes a user input module 310, a handwriting recognition and segmentation module 320, a beautification module 130, a brush rendering module 340, and a typesetting module 350. The main purpose of each module is first introduced below.

[0154] 1. User Input Module 310

[0155] In the present application, the user input module 110 (for example, it can be a handwriting area) generally includes the following uses: (1) for the user to write handwritten words; (2) for the user to select the target font and the first gear; (3) for the user to start ("start" can also be understood as "turn on") the function of beautifying handwritten words. Among them, the "target font style" can also be understood as the direction in which the user expects the handwritten words to be beautified. The user can choose different target fonts according to personal preferences and application scenarios. It should be understood that in the present application, a variety of fonts can be pre-set in the terminal device for user selection. The "first gear" can be understood as the degree to which the user expects the handwritten words to be beautified. If the user expects to retain more of the personal handwritten style, a lower first gear can be selected; on the contrary, if the user expects his handwritten words to be as close to the target font as possible, a higher first gear can be selected.

[0156] 2. Handwritten Character Recognition and Segmentation Module 320

[0157] When the user activates the handwriting beautification function, the handwriting recognition and segmentation module 320 will recognize and segment the text part that needs to be beautified. Figure 4 As shown, the handwriting recognition and segmentation module 320 will recognize the user's handwriting and segment the user's handwriting. Specifically, the handwriting recognition and segmentation module 320 will recognize the character information corresponding to the user's handwriting, and establish a rectangular segmentation frame for the handwriting in units of words. For example, for English, the handwriting recognition and segmentation module 320 will also establish a segmentation frame in units of words. In addition, it will also establish a line segmentation frame for a single line of text.

[0158] 3. Beautification module 330

[0159] The beautification module 330 can be used to construct a handwritten character set according to the first gear input by the user, and the handwritten character set includes one or more handwritten characters in the handwritten character library, and then construct a target character set according to the target font selected by the user, and the target character set includes one or more target characters corresponding to the target font. The beautification module 330 will adjust the ratio between the number of handwritten characters and the number of target characters according to the size of the first gear selected by the user, so as to obtain the handwritten character set and the target character set corresponding to the current first gear. When the first gear tends to the user's handwritten font style, the proportion of the number of handwritten characters in the handwritten character set is higher; when the first gear tends to the target font style, the proportion of the number of target characters in the target character set is higher. The beautification module 330 is also used to extract fusion features corresponding to the handwritten character set and the target character set. At the same time, the beautification module 330 is used to extract the standard characters corresponding to the handwriting image or the handwriting trajectory from the standard character library according to the result of handwriting recognition, and then weight the content features and fusion styles corresponding to the standard characters with different weighting coefficients, and generate the trajectory points of the beautified characters after decoding and reconstruction.

[0160] 4. Rendering module 340

[0161] The rendering module 340 is used to render preset brush information such as pen tip thickness, beautified character color, beautified character texture, beautified character opacity, pen tip, etc. on the trajectory points of the beautified character, thereby generating rendered beautified characters.

[0162] 5. Typesetting module 350

[0163] After the rendering module 340 generates the beautified characters, the handwritten characters need to be replaced with the beautified characters, and the size and position of the beautified characters correspond to the handwritten characters. The typesetting module 350 will calculate the size, aspect ratio, position, orientation and other information of each handwritten character, and then set the corresponding beautified characters to the same parameters. In addition, the typesetting module 350 will further optimize the spacing, size, character spacing and line spacing between two adjacent characters. For example, each line of characters will be centered and the character spacing will be consistent, etc.

[0164] Figure 5 This is a schematic flow chart of a method 500 for beautifying handwriting provided by the present application. The method can be executed by a terminal device, or by a chip or chip system for the terminal device, without limitation. Figure 5 As shown, the method includes:

[0165] 510, obtaining first handwritten information.

[0166] In the present application, "first handwriting" refers to words written by a user on an electronic whiteboard. In the present application, information about the first handwriting includes an image corresponding to the first handwriting and / or a trajectory corresponding to the first handwriting. For example, the handwriting recognition and segmentation module 320 can obtain an image or trajectory corresponding to the first handwriting.

[0167] 520 , detecting a first input operation of the user, and acquiring information of a first gear position in response to the first input operation.

[0168] In the present application, the first gear is determined based on the first instruction input by the user, and the target font is a font preset in a target font library of the electronic device.

[0169] In the present application, the "first gear information" can be used to indicate the first gear, or it can also be understood that the first gear can be determined based on the first gear information. The "first gear" in the present application can be specified by the user. Exemplarily, the first gear can be set to three levels, namely "low", "medium", "high" or "weak", "medium", "strong", etc. The user can enter the required first gear on the interface of the terminal device.

[0170] In the present application, different first gears can be used to determine the ratio between the handwritten font style and the target font style in the fusion style. The higher the first gear, the higher the proportion of the target font style in the fusion style of the generated beautified characters, and the lower the first gear, the higher the proportion of the handwritten font style in the fusion style of the generated beautified characters. Exemplarily, if the first gear selected by the user is "low", it means that the generated beautified characters are more like the user's own handwriting; if the first gear selected by the user is "high", it means that the generated beautified characters are more like the target font selected by the user. In terms of specific implementation, in one possible implementation, the first gear affects the number of handwritten characters and the number of target characters. In another possible implementation, the first gear affects the ratio between the first coefficient corresponding to the handwritten font style and the second coefficient corresponding to the target font style.

[0171] For example, the user input module 310 may receive a first input operation from a user. In the present application, the first input operation may be understood as a first instruction input by a user, and the first instruction is used to indicate information of a first gear position.

[0172] 530, determining a fusion style according to the information of the first gear, the handwriting font library, and the target font.

[0173] In the present application, the handwriting character library refers to characters written by a user in a handwriting area of ​​the electronic device and stored in the electronic device.

[0174] In the present application, the target font library of the electronic device may include multiple fonts, and the user may select one of the target fonts according to the needs. In the present application, multiple different fonts may be pre-set inside the electronic device. For example, the multiple different fonts may include: Kaiti, Lishu, Songti, etc. In some scenarios, if the user does not select the target font, the electronic device may default to a certain font in the target font library as the target font. The present application does not limit the method for determining the target font.

[0175] Optionally, the user can also determine the target font first. Specifically, in one possible implementation, the character library of the terminal device includes at least two different fonts, and the method also includes: receiving a second instruction from the user, the second instruction is used to indicate the target font, and the target font is one of the at least two different fonts; determining the target font according to the second instruction. Exemplarily, it is assumed that the present application includes three fonts, namely font #1, font #2, and font #3, and different fonts represent different target font styles. Alternatively, it can also be understood that the user input module 310 can determine the target font according to the user's second instruction.

[0176] Optionally, if the user has not selected a target font in advance, the terminal device may use a default font as the target font. For example, the terminal device may set font #A as the default target font.

[0177] In the present application, the fusion style is used to characterize the style of the fusion of the target font and the handwriting font corresponding to the characters included in the handwriting font library.

[0178] In a possible implementation, a third instruction is received, the third instruction is used to indicate a second handwritten word, the second handwritten word is a word written by a user in a handwriting area of ​​an electronic device, and the second handwritten word is used to generate a handwritten word library; according to the third instruction, a handwritten word library is generated, the handwritten word library includes one or more second handwritten words.

[0179] In the present application, the electronic device can obtain the user's handwriting library by collecting the handwriting written by the user before. Alternatively, the electronic device can also obtain the user's handwriting library by collecting the handwriting currently written by the user. Exemplarily, the "second handwriting" is a paragraph of text written by the user before, in which some words are circled by the user, and these circled words constitute the handwriting library; Exemplarily, the "second handwriting" can be based on the user's circled words in the first handwriting currently written, thereby constituting a handwriting library. For example, the first handwriting may be more, so the user can circle some words in the first handwriting as the second handwriting, thereby constituting a handwriting library; Exemplarily, as long as the user writes words in the handwriting area of ​​the electronic device, the handwriting can be automatically added to the handwriting library. It can also be understood that the second handwriting in the handwriting library in the present application can come from the first handwriting or from the words written by the user before, and so on, without limitation.

[0180] In a possible implementation, the method further includes: receiving a fourth instruction, the fourth instruction being used to instruct updating of a second handwritten character in the handwritten character library; and generating an updated handwritten character library according to the fourth instruction, the updated handwritten character library including one or more updated second handwritten characters.

[0181] In this application, the handwritten characters in the handwritten character library can also be updated. Here, "update" can be understood as replacing the handwritten characters in the handwritten character library, or adding handwritten characters to the handwritten character library. For example, the user can subsequently rewrite a paragraph of text in the handwriting area of ​​the electronic device and circle some handwritten characters to replace the handwritten characters in the handwriting library.

[0182] In a possible implementation, a fusion style is determined based on information from the first gear, a handwritten character library, and a target font, including: determining a first number of second handwritten characters (also understood as a "handwritten character set") and a second number of target characters corresponding to the target font (also understood as a "target character set") based on the information from the first gear, wherein the first number and the second number are determined based on information from the first gear; determining a fusion style based on the first number of second handwritten characters and the second number of target characters corresponding to the target font.

[0183] In other words, in the present application, the first number is the number of handwritten characters, and the second number is the number of target characters.

[0184] In a possible implementation, in the present application, the beautification module 330 can directly determine the number of handwritten characters and the number of target characters according to the different gears determined by the user. Exemplarily, if the first gear selected by the user is "low", the beautification module 330 can directly determine that the number of handwritten characters is 7 and the number of target characters is 3. For example, 7 handwritten characters can be randomly found from the handwritten character library, and 3 target characters corresponding to the target font can be randomly found from the target font library. Exemplarily, if the first gear selected by the user is "medium", the beautification module 330 can directly determine that the number of handwritten characters is 5 and the number of target characters is 5. Exemplarily, if the first gear selected by the user is "high", the beautification module 330 can directly determine that the number of handwritten characters is 3 and the number of target characters is 7.

[0185] In another possible implementation, the beautification module 130 may determine a first ratio according to a first gear position determined by the user, wherein the first ratio is a ratio between the second number and the first number, and the number of handwritten characters and the number of target characters are determined according to the first ratio. Exemplarily, if the first gear position selected by the user is "low", the beautification module 330 may determine that the ratio between the number of target characters and the number of handwritten characters is 3:7 (number of handwritten characters: number of target characters), and further, the number of handwritten characters is determined to be 7 and the number of target characters is 3. Exemplarily, if the first gear position selected by the user is "medium", the beautification module 330 may determine that the ratio between the number of target characters and the number of handwritten characters is 5:5 (number of handwritten characters: number of target characters), and further, the number of handwritten characters is determined to be 5 and the number of target characters is 5. Exemplarily, if the first gear selected by the user is "high", the beautification module 130 can determine that the ratio between the number of target characters and the number of handwritten characters is 7:3 (number of handwritten characters: number of target characters), and further determine that the number of handwritten characters is 3 and the number of target characters is 7.

[0186] As an example, after determining the target font, the beautification module 330 can determine the number of target characters according to the first gear and randomly determine the second number of target characters corresponding to the target font from the target font library; the beautification module 330 can determine the number of handwritten characters according to the first gear and randomly determine the first number of handwritten characters from the handwritten character library. Exemplarily, when the first gear selected by the user is "low", the beautification module 330 can determine to include 7 handwritten characters and 3 target characters. Exemplarily, if the first gear selected by the user is "medium", the beautification module 330 can determine to include 5 handwritten characters and 5 target characters. Exemplarily, if the first gear selected by the user is "high", the beautification module 330 can determine to include 3 handwritten characters and 7 target characters.

[0187] 540, generate the first beautified character.

[0188] In this application, the first beautified character is the first handwritten character beautified according to the information of the first handwritten character and the fusion style.

[0189] In a possible implementation manner, according to the information of the first handwritten character, determine the standard character corresponding to the first handwritten character, where the standard character is a character pre-set in the standard character library of the terminal device; generate the first beautified character according to the standard character and the fusion style.

[0190] In this application, the "standard character" is a character pre-set in the standard character library of the terminal device. In this application, the "standard character" can be understood as a character with correct stroke order and square shape. The standard character can be a character without any font, or a character with a font, which is not limited.

[0191] For example, the handwritten character recognition and segmentation module 320 can recognize which character the user writes on the tablet. For example, assume that the user writes a character "目" on the electronic whiteboard, then the handwritten character recognition and segmentation module 320 can recognize that this character is "目" instead of "日". The beautification module 330 can extract the standard character from the character library of the terminal device according to the recognition result.

[0192] In a possible implementation manner, extract the content feature corresponding to the standard character, where the content feature is used to determine the standard character; generate the first beautified character according to the content feature and the fusion style.

[0193] The "content feature" in this application can be used to characterize, for example, the information of the stroke structure of the standard character and / or the semantic information of the standard character. For example, according to the content feature, it can be determined which specific character the standard character is. Among them, the "information of the stroke structure of the standard character" can include, for example, at least one of the following: the glyph structure of the standard character, the stroke layout. The "semantic information of the standard character" can be understood as being able to determine which character the standard character is.

[0194] In the present application, generating a first beautified character may also be understood as generating a trajectory point corresponding to the first beautified character.

[0195] Several methods of generating beautified characters proposed in the present application are specifically introduced below. For the convenience of description, in the following introduction, "handwritten character set" is used to represent the first number of second handwritten characters, and "target character set" is used to represent the second number of target characters.

[0196] Method 1

[0197] Figure 6 A schematic block diagram of a method for generating beautified characters provided in this application, such as Figure 6 As shown in the figure, "ContentEncoder" is a content extractor, which is used to extract the content features of standard characters; "Style Encoder" is a style extractor, which is used to extract the fusion style of the handwritten character set and the target character set; "Attention" is an attention mechanism, which is used for feature weighting. It can also be understood that the content features and fusion styles are weighted with different weighting coefficients respectively; "Decoder" is a decoder, which is used for feature fusion and content reconstruction to generate beautified characters.

[0198] It can also be understood that the previous "content extractor", "style extractor" and "attention mechanism" can obtain some digital information for characterizing beautified characters, and the "decoder" can be used to decode the digital information to obtain beautified characters that can be used for display. For example, the "decoder" can fuse the final fusion style with the content features of the standard characters and their corresponding weighting coefficients through the decoder and reconstruct the font to obtain the beautified characters in the current first gear. The solution of method one can be understood as extracting the font style of all characters included in the input handwritten character set and the target character set at the same time, so as to obtain the fusion style corresponding to the handwritten character set and the target character set.

[0199] Exemplarily, assuming that the first gear level of the input is "high", the beautification module 330 determines that the number of target characters in the handwritten character set and the target character set is 6, and the number of handwritten characters in the handwritten character set is 4. Figure 7 This is a schematic diagram of beautified characters corresponding to different first gears obtained by beautifying handwritten characters using a method provided by the present application, such as Figure 7 As shown in the figure, when the first gear is low, the fusion style of the beautified characters is closer to the handwritten characters. As the first gear increases, the fusion style of the beautified characters gradually approaches the target font style. Figure 7It can be seen that as the first gear increases, the connected strokes of the handwritten characters become weaker, and the final beautified characters are more neat; and it can be seen that under the same first gear, the curvature of the strokes of font #1 and font #2 are different; different fonts ultimately correspond to different beautified characters. If the selected target font is font #2, the final beautified characters are similar to the style of font #2.

[0200] Exemplarily, in the specific implementation, it can be implemented through "matrix" operation. For example, the size of the matrix corresponding to each character in the handwritten character set and the target character set input by the beautification module 130 is (1, 110, 3), where "1" refers to the current number of the character (it can also be understood as the first dimension of the matrix), "110" refers to the sequence length of the character, and "3" refers to the current dimension of the character (for example, including the X-axis dimension, Y-axis dimension, and point attributes of the character). Then, all the characters in the handwritten character set and the target character set are spliced ​​according to the first dimension. Assuming that there are 10 characters in the handwritten character set and the target character set, including 4 handwritten characters and 6 beautified characters, after splicing, the size of the matrix corresponding to the handwritten character set and the target character set is (10, 110, 3), where "10" represents that the handwritten character set and the target character set include 10 characters. Then, the matrix can be multiplied. For example, a dimension can be added to the matrix corresponding to the handwritten character set and the target character set. If a word needs to be inferred, the matrix is ​​increased in dimension to (1, 10, 110, 3), where "1" means that one word needs to be inferred at present, "10" refers to the number of words in the current handwritten character set and the target character set (it can also be understood as the second dimension of the matrix), "110" refers to the sequence length of each word in the handwritten character set and the target character set, and "3" refers to the dimension of each word in the handwritten character set and the target character set. Finally, the handwritten character set and the target character set after the increased dimension can be reshaped. For example, the first dimension and the increased second dimension can be merged to obtain a reshaped matrix of size (1×10, 110, 3).

[0201] Method 2

[0202] Figure 8 A schematic block diagram of another method for generating beautified characters provided by the present application, Figure 8 The functions of each module can be referred to in Method 1. Figure 6In the technical solution of method one, the overall style of the input handwritten character set and the target character set can be extracted to obtain a fusion style. However, in method two, the font style of each handwritten character or target character in the input handwritten character set and the target character set can be extracted word by word, and then the font style corresponding to each extracted character is reshaped and spliced ​​to obtain a fusion style. After that, the content features and the fusion style are weighted with different weighting coefficients through the "attention mechanism", and then the decoder is used to perform feature fusion and content reconstruction to generate beautified characters.

[0203] Fig. 9 is a schematic diagram of beautified characters corresponding to different first gears obtained by beautifying handwritten characters using the second method provided by the present application, such as Fig. 9 As shown, when the first gear is low, the fusion style of the beautified characters is closer to the handwritten font style. As the first gear increases, the fusion style of the beautified characters gradually approaches the target font style.

[0204] Exemplarily, in the specific implementation, it can be implemented through "matrix" operation. For example, the size of the matrix corresponding to each character in the handwritten character set and the target character set in the beautification module 330 is (1, 110, 3), where "1" refers to the current number of the character (it can also be understood as the first dimension of the matrix), "110" refers to the sequence length of the character, and "3" refers to the current dimension of the character (for example, including the X-axis dimension, Y-axis dimension, and point attributes of the character). It should be noted that in method 2, each character in the handwritten character set and the target character set needs to be fed into the model one by one. Assuming that the number of characters to be inferred is 1, the matrix size of each character is (1×1, 110, 3), where "1×1" refers to "the number of inferred characters × the number of beautified characters in the handwritten character set and the target character set", "110" refers to the current sequence length of the character, and "3" refers to the dimension of the character. The feature matrix is ​​obtained with a size of (1×1, 110, dim), where "dim" refers to the feature layer dimension (the matrix is ​​usually multiplied, and the value of "dim" will be greater than 3). Then, the matrix is ​​reshaped. The matrix can be split according to the first dimension. The size of the matrix after splitting is (1, 1, 110, dim); preferably, the current matrix is ​​spliced ​​according to the second dimension. If the handwritten character set and the target character set include a total of 10 characters, the size of the matrix corresponding to the handwritten character set and the target character set after splicing is (1, 10, 110, dim).

[0205] Method 3

[0206] Fig.10 A schematic block diagram of another method for generating beautified characters provided by the present application, Fig.10The functions of each module can be referred to in Method 1. Figure 6 The introduction can be understood without further elaboration. In the technical solutions of Method 1 and Method 2, the beautification module 330 can determine the number of handwritten characters in the handwritten character set and the number of target characters in the target character set according to the first gear. However, in Method 3, the number of handwritten characters in the input handwritten character set and the number of target characters in the target character set are not limited, but the handwritten font style corresponding to the extracted handwritten characters and the target font style corresponding to the target characters need to be weighted according to a certain weighting coefficient to obtain a fusion style. In other words, in Method 3, the number of handwritten characters in the input handwritten character set and the number of target characters in the target character set can be unrestricted, the beautification module 330 can extract the handwritten font style of the handwritten characters and the target font style of the target characters, and then the beautification module 330 can determine the first coefficient corresponding to the handwritten font style of the handwritten characters and the second coefficient corresponding to the target font style according to the first gear, and then determine the fusion style according to their respective weighting coefficients and the handwritten font style and the target font style. Among them, the second coefficient is used to characterize the proportion of the target font style in the fusion style, and the first coefficient is used to characterize the proportion of the handwritten font style in the fusion style.

[0207] In a possible implementation, the beautification module 330 may determine the second ratio according to the first gear, where the second ratio is a ratio between the second coefficient and the first coefficient.

[0208] Exemplarily, if the first gear selected by the user is "low", the beautification module 330 can determine that the weighted coefficient of the handwriting font style of the handwritten character is 0.7 (an example of the first coefficient), and the weighted coefficient of the target font style of the target character is 0.3 (an example of the second coefficient). According to the respective weighted coefficients, the handwriting font style of the handwritten character is merged with the target font style of the target character to obtain a fusion style. Exemplarily, if the first gear selected by the user is "medium", the beautification module 330 can determine that the weighted coefficient of the handwriting font style of the handwritten character is 0.5, and the weighted coefficient of the target font style of the target character is 0.5. According to the respective weighted coefficients, the handwriting font style of the handwritten character is merged with the target font style of the target character to obtain a fusion style. Exemplarily, if the first gear selected by the user is "high", the beautification module 330 can determine that the weighting coefficient of the handwriting font style of the handwritten character is 0.3, and the weighting coefficient of the target font style of the target character is 0.7, and according to their respective weighting coefficients, the handwriting font style of the handwritten character is merged with the target font style of the target character to obtain a fusion style.

[0209] Fig.11 is a schematic diagram of beautified characters corresponding to different first gears obtained by beautifying handwritten characters using method three provided by the present application, such as Fig.11As shown, when the first gear is low, the fusion style of the beautified characters is closer to the handwritten font style. As the first gear increases, the fusion style of the beautified characters gradually approaches the target font style.

[0210] It should be understood that the technical solutions provided by the first to third methods can also be understood as a model for beautifying handwriting, such as Figure 6 , Figure 8 , Fig.10 As shown, the input of the model is standard characters, handwritten character sets and target character sets, and the output of the model is beautified characters.

[0211] In the present application, the above-mentioned methods 1 to 3 can be used to achieve multi-level handwriting beautification effects based on the user selecting the first level, and the beautification effect on handwriting in the present application can retain the user's personal handwriting style. The user can choose different first levels to achieve whether the final beautified words are biased towards the user's handwriting style or the target font style.

[0212] Method 4

[0213] Furthermore, in some scenarios, the structure of the generated beautified characters does not match their corresponding standard structures. It can also be understood that the final generated beautified characters may be inconsistent with the original handwritten characters, that is, the generated beautified characters are not the user's original handwritten characters on the tablet. In some scenarios, the generated beautified characters appear to be incorrect. Fig.12 A generated beautified word collapse effect diagram is shown, such as Fig.12 As shown, the beautified characters generated in the example on the left are not standard and lack some strokes, and the beautified characters generated in the example on the right are no longer recognizable. In view of this, the present application proposes that when generating beautified characters, it is possible to consider adding the stroke information of standard characters, so as to help the generated beautified characters be accurate and consistent with the characters handwritten by the user on the electronic whiteboard, thereby reducing the error rate of the generated beautified characters. The following introduces a technical solution for beautifying handwritten characters after adding the stroke information of standard characters.

[0214] In this technical solution, it is necessary to determine the structural information of the standard character so that the generated beautified character can match the standard structure of the character. For example, the stroke information of the standard character can be obtained, and the stroke information of the standard character can include the stroke order and stroke category of the standard character, and then the beautified character is generated according to the standard character, the handwritten character set and the target character set, and the stroke information of the standard character. In a possible implementation method, the first beautified character can be generated according to the content features and fusion style corresponding to the standard character, and the stroke information of the standard character.

[0215] As an example, the strokes of a standard character can be encoded by category. For example, the strokes of common Chinese characters can be divided into 32 types, and these 32 types of strokes can be numbered respectively, with each number uniquely representing a type of stroke. Table 1 below shows a possible stroke encoding. For example, when recognizing an image of handwritten characters, the handwritten characters can be determined to correspond to standard characters, and the structural information of the standard characters can be determined according to Table 1. Suppose the handwritten character written by the user on the electronic whiteboard is "人 (rén)". Then, according to Table 1, the stroke information of the corresponding standard character of this character can be obtained as (04, 05). This encoding method can also reflect the stroke order. For example, the stroke information (04, 05) can also reflect that when forming this character, the "left-falling stroke (丿)" is written first and then the "right-falling stroke (乀)".

[0216] Table 1

[0217]

[0218]

[0219] As another example, another encoding method can also be adopted. Suppose there are 32 stroke types in each row. The "0" and "1" can be used to indicate which stroke type each stroke of the character is. The number of strokes that the character is composed of can be represented by the number of rows. For example, for the character "人 (rén)", which is composed of two strokes, it can be represented by two rows. Its stroke information can be: in the first row, only the stroke type of the "left-falling stroke (丿)" is indicated as "1", and the other 31 stroke types are all indicated as "0"; in the second row, only the stroke category of the "right-falling stroke (乀)" is indicated as "1", and the other 31 stroke types are all indicated as "0". Another example is the character "目 (mù)". Since this character is composed of 5 strokes, it can be represented by 5 rows. Its stroke information can be: in the first row, only the stroke type of the "vertical stroke (丨)" is indicated as "1", and the other 31 stroke types are all indicated as "0". In the second row, only the stroke category of the "horizontal fold " is indicated as "1", and the other 31 stroke types are all indicated as "0"; in the third to fifth rows, only the stroke category of the "horizontal stroke (一)" is indicated as "1", and the other 31 stroke types are all indicated as "0".

[0220] Fig.13 is a schematic block diagram of another way to generate beautified characters provided by this application, Fig.13 in which the "StrokesEncoder" is a stroke encoding module. By introducing the stroke information corresponding to the standard character into the "decoder", the accuracy of the generated beautified character can be improved, and the problem of font collapse can be avoided. Fig.13 The functions of the other modules in Figure 6 can be understood by referring to the introduction of Fig.14 This is the effect diagram of the beautified characters generated by the technical solution of method 4 of this application. Fig.14 The above is the final effect of the beautified characters after the stroke order is introduced. Fig.14 The following is the final beautified effect image generated after the stroke categories are introduced.

[0221] It should be understood that the technical solution provided by the fourth method can also be understood as a model for beautifying handwriting, such as Fig.11 As shown, the input of the model is standard characters, handwritten character sets and target character sets, as well as stroke information of standard characters, and the output of the model is beautified characters.

[0222] In the present application, the solution provided by the fourth method can effectively alleviate the problem of font collapse in the trajectory of generating beautified characters, reduce the error rate of the generated beautified characters, and improve the generation quality of beautified characters.

[0223] It should be noted that in the present application, when the above-mentioned methods 1 to 4 are used to beautify handwritten characters, the standard characters input into the model can be a single standard character. In other words, in the process of beautifying handwritten characters using the model, each handwritten character can be beautified separately.

[0224] In addition, considering that when the first gear is higher, the target font style corresponding to the target font gradually increases, the error rate of the generated beautified characters will increase. This is because, usually, the training data set consists of handwritten characters written by many users on electronic whiteboards, which are usually sloppy and not neat enough, resulting in a high proportion of target characters under the first gear. The "decoder" in the model is prone to errors in reconstructing the characters, and re-collecting neat handwritten characters requires a certain amount of manpower costs. In view of this, in order to expand the training data set of the model and make the model perform better, the application proposes that since the original output is beautified characters of different gears, the output results of the original model can be used as its own input, so as to further train the original model and optimize the performance of the original model. The beautified characters output by the original model are more accurate, and the generation quality of the beautified characters is higher.

[0225] It should be understood that the data set in the present application may be a public data set, which is used for model training of beautified characters. For example, the data set may further include a training data set and an inference data set. Among them, the training data set is used for model training, and the inference data set is used for model inference. For example, the training data set may include a training data set of standard characters, a training data set of handwritten characters, and a training data set of target characters. For example, the inference data set may include an inference data set of standard characters, an inference data set of handwritten characters, and an inference data set of target characters. In the present application, the training data set and the inference data set are no longer distinguished, and are collectively referred to as training data sets.

[0226] Specifically, Fig.15 As shown, the original model can be divided into two training stages and two reasoning stages. The following introduces the training stages and reasoning stages of the original model respectively.

[0227] Step 1: Training phase#A

[0228] Training phase #A can be understood as the initial training phase. During the initial training, the training data set of the original model can be obtained from a public training data set. Among them, the training data set of standard characters includes training standard characters, the training data set of handwritten characters includes training handwritten characters, and the training data set of target characters includes training target characters. Exemplarily, during the initial training, the training data set of handwritten characters can be used for training. For example, training standard characters #1 and training handwritten characters #1 can be input, and the training label is also training handwritten characters #1. Exemplarily, during the initial training, the training data set of target characters can also be used for training, for example, training standard characters #2 and training target characters #2 can be input, and the training label is also training target characters #2. In other words, the training goal of the model is to keep the generated characters expected to be output consistent with the input training handwritten characters (that is, the style of the generated characters is consistent with the style of the training handwritten characters), that is, what is expected to be input can be output. For example, multiple iterations of training can be performed to optimize the performance of the model. After the initial training, when the performance of the original model reaches the expected level (or when the performance of the original model no longer changes), the inference phase #A can be performed.

[0229] Step 2: Reasoning Phase #A

[0230] Exemplarily, in the inference stage #A, the first gear is determined to be 40%, and the training standard word #11 is input. The input training handwritten word set includes: training handwritten word #11 to training handwritten word #16, and the input target handwritten word set includes: training target word #11 to training target word #14. At this time, the model can output the beautified word #1 corresponding to the training standard word #11. For example, after multiple inferences, a collection of many beautified words can be obtained. In this application, the training data set of beautified words can be obtained through the inference stage #A.

[0231] Step 3: Training phase#B

[0232] After obtaining the training data set of beautified characters, the model can be further trained based on the training data set of beautified characters to optimize the performance of the model. The training process can also be understood as the process of fine-tuning the model. For the specific training process, please refer to Fig.16, assuming that the inference result of the original model in the inference stage #A is the beautified word #1, at this time, the beautified word #1 can be used as input to further train the original model. For example, the training standard word #21 and the beautified word #1 can be input to train the original model, and the training label is still the beautified word #1. The original model can be iteratively trained based on the multiple beautified words obtained in the inference stage #A, and the parameters of the original model can be adjusted to optimize the performance of the model. Exemplarily, in the training stage #B, the model parameters of the "decoder" can be mainly adjusted, and the model parameters in the previous "content extractor" and "style extractor" do not need to be adjusted. For example, during the training process, the model parameters in the decoder can be optimized by loss calculation until the desired model performance is achieved (or, the model performance no longer changes), and the training can be ended, and the original model training is considered to be completed.

[0233] It should be noted that, in this embodiment, through multiple training stages #A and training stages #B, the parameters of the original model can be continuously optimized, so that the beautified characters output by the original model are more accurate and the generation quality of the beautified characters is higher.

[0234] Step 4: Reasoning Phase#B

[0235] The reasoning stage #B can be understood as the process of the user using the model. For example, based on the recognized image corresponding to the user's first handwriting, the standard word corresponding to the user's first handwriting is determined, and then based on the first gear determined by the user, the handwritten word set and the target word set are determined. The model outputs the first beautified word corresponding to the first handwriting based on the standard word, handwritten word set, and target word set corresponding to the input first handwriting.

[0236] The various technical solutions introduced above can be understood as specific implementation methods of the beautification module 330 beautifying handwritten characters. The following will introduce the rendering method of beautified characters.

[0237] Normally, the pen stroke effect of handwritten characters is calculated based on parameters such as the pressure and speed of the pressure bar of the stylus used by the user, but the beautified characters generated by the model do not have this information. Therefore, this embodiment proposes a method for calculating the pen stroke width based on the dispersion degree of the beautified character trajectory points.

[0238] According to the rules of writing habits, in general, the writing speed of the stylus is faster when writing the straight part of the stroke, and the track width of the beautified character is thinner; the speed slows down when writing the bending part of the stroke, and the track width of the beautified character is thicker. Therefore, the width of the pen tip of the beautified character can be inferred by the sparseness of the track points of the beautified character. For example, when the sampling points of the track of the beautified character are sparser, or the distance between the connected points is longer, it means that the writing speed of the pen is faster, that is, the brush width is thinner; on the contrary, when the sampling points of the track of the beautified character are denser, or the distance between the connected points is longer, it means that the writing speed of the pen is slower, that is, the brush width is thicker. The main process is introduced below:

[0239] Step 1: Calculate the width of the pen tip for beautifying characters

[0240] (1) The trajectory points of the beautified characters are simplified by the Ramer-Douglas-Peucker (RDP) algorithm. For example, the sampling points are denser at the bends of the strokes and sparser at the straight parts of the strokes.

[0241] (2) Mapping the trajectory points of the simplified beautified characters to the writing rules, that is, the densely populated areas indicate a slower writing speed, and the trajectory point width is larger; the sparsely populated areas indicate a faster writing speed, and the trajectory point width is smaller. Based on the dispersion of the trajectory points, the density distribution of the trajectory points can be obtained, thereby realizing the calculation of the pen tip width of the beautified characters.

[0242] Step 2: Smooth the width of the pen tip of the beautified character

[0243] (3) In order to make the trajectory points of the simplified beautified characters smoothly connected, the trajectory points can be interpolated using a curve smoothing method, and the width of the interpolation points can be calculated accordingly according to the width of the pen tip of the trajectory points;

[0244] (4) Perform sliding average on the width value to achieve a smooth transition of the pen tip width between each stroke.

[0245] It should be understood that in a specific implementation, step 1 and step 2 are in an and / or relationship. For example, only step 1 or only step 2 may be executed; for another example, step 2 may be executed after step 1 is executed.

[0246] Fig.17A flow chart of a method for calculating the stroke width of a beautified character and smoothing the stroke width is shown, the method comprising: 1401, generating trajectory points of the beautified character; 1402, determining the stroke and sampling the stroke; 1403, when the sampling point is greater than 2, using the RDP algorithm to remove redundant points; 1404, after removing the redundant points, if the sampling point is still greater than 4, calculating the stroke width based on the dispersion degree of the sampling point; 1405, after removing the redundant points, if the sampling point is less than 4, continue to sample the original sampling points corresponding to the stroke, and then calculate the stroke width based on the dispersion degree of the sampling point; 1406, when the sampling point is less than 2, calculate the stroke width based on the dispersion degree of the sampling point. The above steps 1401 to 1406 can be understood as the process of calculating the stroke width of the beautified character, and the following steps 1407 to 1412 can be understood as the process of smoothing the stroke width of the beautified character. Step 1407, if the sampling points are greater than 4, Bezier interpolation can be used for interpolation; Step 1408, if the sampling points are less than 4, linear interpolation can be used for interpolation; Step 1409, calculate the pen tip width of the interpolation point; Step 1410, smooth the pen tip width; Step 1411, when it is determined to be the last stroke, the final pen tip width can be generated; Step 1412, when it is determined not to be the last stroke, the above steps 1402 to 1411 can be repeated.

[0247] The following introduces a method A for calculating the pen tip width of beautified characters and a method B for smoothing the pen tip width.

[0248] Method A

[0249] When the number of stroke sampling points is greater than 2, the sum of the distances between the current sampling point and the two previous and next sampling points is calculated, that is, the distances between the i-th sampling point P and the i-th sampling point P are calculated respectively. i With the previous sampling point P i-1 The distance between i and the next sampling point P i+1 In order to solve the influence of the size of the word on the distance, the distance sum is normalized and divided by the width of the word (width) to obtain the normalized distance sum D i When D i When D is larger, it means that the current sampling point is in a sparse area. i When it is smaller, it means that the current sampling point is in a denser area.

[0250] in,

[0251] Based on the normalized distance of the current sampling point and D i Calculate the current pen width W i, where base_W is the base pen width, Scale i is the stroke weighting coefficient, which can be realized based on the S-curve. The function graph is as follows Fig.18 As shown, the coefficient value Scale i With distance and D i The coefficient is negatively correlated and converges to 1. When the distance and the value increase, the coefficient value decreases and the pen tip width value decreases. When the distance and the value are too large or too small, the coefficient converges, which is conducive to a smooth transition of the pen tip width. i =1 / (1+exp(D i )),W i =Scale i *base_W. For the starting point 0 and the end point N of the stroke trajectory, the pen tip width values ​​are: W 0 =W 1 ,W N =W N-1 , when the stroke sampling point number is 2, point P 0 ,P 1 The pen tip width is calculated as follows: W 1 =1 / (1+exp(D 1 ))*base_W,W 0 =W 1 , where W 0 ,W 1 Represent the starting point P 0 and the end point P 1 Width of pen tip.

[0252] Method B

[0253] In order to smooth the font trajectory, the trajectory points can be interpolated, such as Fig.19 As shown in the figure, the linear weighted summation is used to calculate the width of the interpolation point. Assume that the pen tip width values ​​of the i-1th trajectory point and the i-th trajectory point are W respectively. i-1 and W i , the number of interpolation points between two points is M, and the jth interpolation point is I j Width value The calculation formula is: i,j =(1-k)*W i-1 +k*W i , where k and I j The location is related, that is, the interpolation point I j The ratio of the trajectory distance from the starting point to the trajectory distance between the i-1th trajectory point and the i-th trajectory point is calculated as: Fig. 20 The diagram shows the effect achieved after rendering the beautified characters based on the method of this embodiment.

[0254] The above technical solution can also be understood as that the rendering module 340 can determine the pen stroke width of the beautified character and / or smooth the pen stroke width of the beautified character; the rendering module 340 can render the beautified character according to the pen stroke width and / or the smoothed pen stroke width to generate the rendered beautified character.

[0255] The method for rendering beautified characters proposed in the present application can achieve changes in the thickness of the pen strokes of the generated beautified characters, which is closer to the characteristics of users actually writing on an electronic whiteboard, thereby improving the user experience.

[0256] Considering the user experience, the size, aspect ratio, position, and orientation of the beautified characters should be basically consistent with the handwritten characters on the electronic whiteboard. Therefore, this embodiment mainly introduces a method for typesetting beautified characters. This embodiment mainly includes the following four steps:

[0257] Step 1: respectively obtain the vector dot matrix data of each handwritten character and the vector dot matrix data of each beautified character;

[0258] Step 2: Determine the format information of each handwritten word.

[0259] Step 3: Determine the format information of the beautified characters according to the format information of each handwritten character, so that the beautified characters are basically consistent with the corresponding handwritten characters.

[0260] Step 4: Optimize the character spacing, line spacing, etc. of multiple beautified characters.

[0261] It should be noted that the vector dot matrix data of the handwritten characters in step 1 and the vector dot matrix data of each beautified character can be found in Fig. 20 The beautified words before rendering in each group of pictures in the image. The following is a detailed description of how to determine the format information of each handwritten word in step 2. The handwritten word format information in this application may include at least one of the following: the position of the handwritten word, the size of the handwritten word, the aspect ratio of the handwritten word, and the direction of the handwritten word. In order to determine the format information of the handwritten word, the position of the key point of a single handwritten word can be determined first. Similarly,

[0262] In this application, "key point" can refer to the center point of the character's "center of gravity", "center", etc., which is used to quantify the center coordinate position of the character. This application introduces a method for calculating the center based on the convex hull of a handwritten character, that is, generating the convex hull of the character based on the vector dot matrix of the character strokes. Taking the lowest point of the convex hull as the base point, divide it into several triangles, and then calculate the center of gravity of the convex hull. The calculated center of gravity is the coordinate position of the key point of the character. Among them, the specific calculation formula is as follows:

[0263] The abscissa of the centroid of the polygon = ∑ the abscissa of the centroid of each triangle of the polygon × (the area of ​​the triangle / the total area of ​​the polygon);

[0264] The ordinate of the centroid of the polygon = ∑(the ordinate of the centroid of each triangle in the polygon triangulation) × (the area of the triangle / the total area of the polygon)0

[0265] Exemplarily, assume that a convex hull is decomposed into four triangles, namely triangle #1, triangle #2, triangle #3, and triangle #4. Then the abscissa Mx of the centroid of the polygon can be specifically expressed as: Mx = x1*f1 + x2*f2 + x3*f3 + x4*f4, where the coefficient f1 = s1 / Ms, s1 is the area of triangle #1, and Ms is the total area of the polygon; the coefficient f2 = s2 / Ms, s2 is the area of triangle #2, and Ms is the total area of the polygon; the coefficient f13 = s3 / Ms, s3 is the area of triangle #3, and Ms is the total area of the polygon; the coefficient f4 = s4 / Ms, s4 is the area of triangle #4, and Ms is the total area of the polygon. Similarly, for the ordinate of the centroid of the polygon, it can be understood by referring to the above example.

[0266] Fig.21 A schematic diagram showing the calculation of the centroid of the convex hull based on the above scheme is shown. Fig.21 In (a), it is a schematic diagram of generating a convex hull based on a character dot matrix. Fig.21 In (b), it is a schematic diagram of dividing the convex hull into multiple triangles. Fig.21 In (c), it is a schematic diagram of calculating the centroid of the convex hull.

[0267] Since at least three coordinate points are required to generate a convex hull, for a straight-line character like the Chinese character "一" (which can be understood as having only two coordinate points), the convex hull does not need to be generated, and the center point of the line connecting the two points can be directly used as the key point of the character.

[0268] After determining the key points of the character, the size of each character and the aspect ratio of each character can be calculated. For example, the size and width-to-height ratio information of handwritten characters can be calculated based on the distance from the key points to the rectangular bounding box, as Fig. 22 shown, d l 、d r 、d t 、d b These four parameters are the handwritten character size parameters.

[0269] The following introduces a method for implementing how to set the size and aspect ratio of the beautified characters according to the size and aspect ratio of the handwritten characters in step three. This method mainly includes the following four steps:

[0270] (1) Align the key points of the beautified characters with the key points of the original handwritten characters.

[0271] (2) Calculate the original handwritten character parameters: dl ,d r ,d t ,d b And the size parameter of the beautified word: d' l , d' r , d' t , d' b It should be noted that the size parameter of the beautified word may be a default initial size preset when the beautified word is generated.

[0272] (3) Calculate the ratio between the handwritten font size parameter and the default initial size parameter of the beautified font: S l =d l / d' l , S r =d r / d' r , S t =d t / d' t , S b =d b / d' b .

[0273] (4) Calculate the minimum value among the four proportional parameters. If the minimum value exists in S l and S r In the horizontal direction, the enlargement ratio of the beautified word is min{S l ,S r}, the vertical magnification ratio is based on the dtw algorithm, that is, in (S t ,S b ) to find a value (usually using linear interpolation) in the range of S to maximize the similarity between the beautified font and the handwritten font. Similarly, if the minimum value exists in S t and S b In the vertical direction, the beautified characters are enlarged by min{S t ,S b}, the horizontal magnification ratio is (S l ,S r ) range.

[0274] Fig.23 : This is the effect of fitting a single beautified word with the corresponding handwritten word after adopting the above solution. Similarly, for multiple beautified words, the effect of fitting the key points with the key points of the handwritten word after adjusting the size and proportion of the multiple beautified words is as follows: Fig.24 shown.

[0275] Based on the above technical solution, in this application, the size, position and other information of the handwritten characters can be retained so that the handwritten characters will not be misplaced after beautification, or it can also be understood that the beautified characters and handwritten characters generated in this application can be well fitted.

[0276] In order to make the display effect of multiple beautified characters more neat and beautiful and improve the user experience, this application also proposes to optimize the character spacing, line spacing, etc. of multiple beautified characters. The following mainly introduces the typesetting method mentioned in step four.

[0277] For example, multiple beautified words in the same line can be aligned, and the alignment direction is to align the key points of the beautified words along the beautified line direction. In addition to the "center of gravity" or "center" mentioned above, the key point of the beautified word can also be the "highest point" or "lowest point" of the beautified word, to respectively achieve the center alignment, top alignment, and bottom alignment of a single line of text. Taking the horizontal center of gravity alignment of the beautified words as an example, the effect of the beautified words in the same line after line alignment is as follows: Fig.25 shown.

[0278] For example, multiple beautified characters can be evenly distributed in a single line, that is, the spacing between adjacent beautified characters in the same line is equal. This method optimizes the typesetting effect of a single line of beautified characters, that is, the fonts in the same line of beautified characters will not be overcrowded or scattered. In this example, the total spacing D1 between the first and last characters in the line is calculated, and the sum of the widths of other characters except the first and last characters is subtracted from it to obtain the sum of the spacing D2, and then D2 is divided by the total number of characters in the single line N-1 (for example, Fig.26 N is 13), calculate the character spacing D3 that each character should have after uniform distribution, and readjust the positions of other characters under the premise that the positions of the first and last characters remain unchanged, so that the distance between each character is D3. The effect of typesetting the beautified characters in the same line by this method is as follows Fig.26 shown.

[0279] For example, in addition to unifying the character spacing, the size of the beautified characters in a single line or multiple lines can be normalized according to the user's handwriting, so that the size of the beautified characters in a line or all lines is consistent. In addition, the minimum rectangular bounding box of each beautified character in a single line and the minimum rectangular bounding box of each line of handwriting between multiple lines can be unified, and each line of beautified characters can be aligned, such as left alignment, center alignment, right alignment, etc. The effect of left alignment and unified size of all characters and line spacing is as follows: Fig. 27 shown.

[0280] Exemplarily, for multiple lines of handwritten text containing subheadings, the same type of subheadings can also be individually aligned based on the recognition results of the images corresponding to the handwritten text. Among them, the specific judgment method for subheadings of the same type can be, for example, judging whether the first or multiple texts starting from the first of each line of text in the recognition results contain special text of the same type of title type, such as numbers 1, 2, 3; letters a, b, c; special symbols "·", etc., and aligning the lines of text with the same type of titles, generally on the side where the first word is located (for example, the left or right side). The basis for alignment can be the first title of the same type, or the average of multiple titles can be taken. The effect of aligning multiple lines of titles is as follows. Fig.28 shown.

[0281] For example, some punctuation marks cannot be aligned in the same way as the text in the same line. For example, when other text is aligned with the center of gravity, the quotation marks cannot be aligned according to the center of gravity, but aligned to the highest point of the adjacent text. Similarly, the alignment of the comma "," can be aligned according to the lowest point of the adjacent text.

[0282] The above embodiment mainly introduces a method for typeset beautifying characters. The following embodiment specifically introduces a typeset algorithm for typeset beautifying characters. The typeset algorithm specifically includes the following steps:

[0283] Step 1: Obtain the character segmentation frame and line segmentation frame of the handwritten characters, and segment each line of handwritten characters to obtain segment frames.

[0284] Step 2: Calculate the size and key point position of each beautified character based on the paragraph frame to perform typesetting.

[0285] First, a method for implementing step 1 is introduced below. In this application, the word segmentation frame and line segmentation frame of the handwritten word can be obtained first. For example, the handwritten word recognition and segmentation module 320 can segment each handwritten word to obtain the word segmentation frame and line segmentation frame of the handwritten word. After obtaining the word segmentation frame and line segmentation frame of the handwritten word, the average width w( Fig.29 The width w of the character box of one handwritten character is shown in 1 ) and the average spacing d between the character box ( Fig.28 The width d of the spacing between the character boxes of two handwritten characters is shown in 1 ), if the distance between two character boxes is d i If it exceeds the average value of the spacing between the first n word frames and is greater than s1*w and s2*d (where n, s1 and s2 are all preset coefficients), it can be determined that a line of handwriting is divided into two sections at this point (i.e., between the two word frames), such as Fig.29 shown.

[0286] For step 2, the present application proposes a method for calculating the size and magnification ratio of the beautified characters based on the segment frame. Since all characters of the same target font have the same font size, and each target character has a corresponding character frame. Fig.30 Schematic diagram of a character frame of a target font is shown. Specifically, in each segment of handwriting, the width W of the segment frame can be obtained first. p With segment frame height H p , then the handwriting recognition and segmentation module 320 recognizes the image corresponding to the handwriting, extracts the corresponding standard word based on the recognition result, and generates the beautified word based on the first gear. Then, the sum of the widths w_s of the word frames of all the beautified words and the maximum word frame height h_max are calculated. Then, the distance d from the left edge of the word frame of the first word of each paragraph to the word is calculated. 1 And the distance d from the right edge of the body frame of each last character to the changed character 2 ,like Fig.31 As shown, the magnification ratio s of the beautified word is min(W p / (w_s-d1-d2)), Hp / h_max). After determining the magnification ratio of the beautified word, the beautified word can be multiplied by s along with the word frame, and then arranged in the corresponding segment frame in sequence, wherein the leftmost point of the first beautified word is aligned with the left edge of the segment frame, and the height center of all the word body frames is aligned with the height of the segment frame. Based on the above scheme, the effect of the beautified word is as follows: Fig.32 shown.

[0287] It can be understood that the term "and / or" in this article is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0288] Those skilled in the art should be aware that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is performed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0289] The embodiment of the present application can divide the functional modules of the electronic device according to the above method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. The following is an example of dividing each functional module corresponding to each function.

[0290] Fig.33 This is a schematic diagram of an embodiment of the present application providing a method for beautifying handwritten characters on a user interface, such as Fig.33 As shown, for example, when beautifying handwritten words, the user can adjust the format of the beautified words. For example, the user can choose to adjust multiple beautified words, or choose to adjust multiple beautified words. For another example, the user can choose the target font to be Songti. For another example, the user can choose the first gear. In the center of the interface is a preview area, which is the beautified words generated based on the user's settings.

[0291] Fig.34 1 is a schematic block diagram of an electronic device 1000 provided in an embodiment of the present application. Fig.34 As shown, the electronic device 1000 may include: a processing module 1010 and an acquisition module 1020 .

[0292] The above modules are respectively used to execute the steps in the above method 500 for beautifying handwriting, which will not be described in detail here.

[0293] It should also be understood that the electronic device 1000 herein is embodied in the form of a functional unit. The term "unit" herein may refer to an application specific integrated circuit (ASIC), an electronic circuit, a processor (e.g., a shared processor, a dedicated processor, or a group processor, etc.) and a memory for executing one or more software or firmware programs, a combined logic circuit, and / or other suitable components that support the described functions.

[0294] The electronic device 1000 of each of the above schemes has the function of implementing the corresponding steps of the above method 500. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions; for example, the determination unit and the like can be replaced by a processor to respectively perform the transceiver operations and related processing operations in each method embodiment. In addition, the determination unit can be a processing circuit.

[0295] It should be pointed out that Fig.34The electronic device in the embodiment may be the electronic device in the aforementioned method, or may be a chip or chip system of the electronic device, such as a system on chip (SoC). The processing unit is a processor or microprocessor or integrated circuit integrated on the chip. This is not limited here.

[0296] Fig.35 2000 is a schematic block diagram of another electronic device 2000 provided in an embodiment of the present application. As shown in the figure, the device 2000 includes: at least one processor 2010. The processor 2010 is coupled to the memory and is used to execute instructions stored in the memory to send signals and / or receive signals. Optionally, the device 2000 also includes a memory 2030 for storing instructions. Optionally, the device 2000 also includes a transceiver 2020, and the processor 2010 controls the transceiver 2020 to send signals and / or receive signals.

[0297] It should be understood that the processor 2010 and the memory 2030 can be combined into one processing device, and the processor 2010 is used to execute the program code stored in the memory 2030 to implement the above functions. In specific implementation, the memory 2030 can also be integrated into the processor 2010, or independent of the processor 2010.

[0298] It should also be understood that the transceiver 2020 may include a transceiver (or receiver) and a transmitter (or transmitter). The transceiver may further include an antenna, and the number of antennas may be one or more. The transceiver 2020 may also be a communication interface or an interface circuit.

[0299] Specifically, the processor 2010 in the device 2000 may correspond to the processing unit 1010 and the acquisition module 1020 in the electronic device 1000 .

[0300] As a solution, the electronic device 2000 is used to implement each step in the above method 500 embodiment.

[0301] For example, the processor 2010 is used to execute the computer program or instructions stored in the memory 2030 to implement each step in the above method 500.

[0302] According to the method provided in the embodiment of the present application, the present application also provides a computer program product, on which a computer program code is stored. When the computer program code is run on a computer, the computer executes each step in the embodiment of method 500.

[0303] According to the method provided in the embodiment of the present application, the present application also provides a computer-readable medium, which stores a program code. When the program code is executed on a computer, the computer executes each step in the above method 500.

[0304] The explanation of the relevant contents and beneficial effects of any of the above-mentioned devices can be referred to the corresponding method embodiments provided above, which will not be repeated here.

[0305] A computer-readable medium stores a program code, and when the program code is executed on a computer, the computer executes each step in the method 500.

[0306] The explanation of the relevant contents and beneficial effects of any of the above-mentioned devices can be referred to the corresponding method embodiments provided above, which will not be repeated here.

[0307] It should be understood that the specific process of each transceiver and processor executing the above corresponding steps has been described in detail in the above method embodiment, and for the sake of brevity, it will not be repeated here.

[0308] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0309] It should be noted that the processor in the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (digitalsignal processor, DSP), an application-specific integrated circuit (application-specific integrated circuit, ASIC), a field-programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor can be combined and performed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0310] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disc (SSD)).

[0311] In the above-mentioned various device embodiments, the corresponding modules or units perform the corresponding steps. For example, the transceiver unit (transceiver) performs the steps of receiving or sending in the method embodiment, and other steps except sending and receiving can be performed by the processing unit (processor). The functions of the specific units can refer to the corresponding method embodiments. Among them, there can be one or more processors.

[0312] The terms "component", "module", "system", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process, a processor, an object, an executable file, an execution thread, a program and / or a computer running on a processor. By way of illustration, both applications and electronic devices running on electronic devices can be components. One or more components may reside in a process and / or an execution thread, and a component may be located on a computer and / or distributed between two or more computers. In addition, these components may be executed from various computer-readable media having various data structures stored thereon. Components may, for example, communicate through local and / or remote processes according to signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system and / or a network, such as the Internet interacting with other systems through signals).

[0313] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0314] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0315] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0316] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0317] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0318] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0319] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for beautifying handwriting, characterized in that: include: Acquire information of a first handwritten word, where the first handwritten word is a word written by a user in a handwriting area of ​​an electronic device; detecting a first input operation of the user, and acquiring information of a first gear position in response to the first input operation; Determine a fusion style according to the information of the first gear, a handwriting font library, and a target font, wherein the handwriting font library includes characters written by the user on a handwriting area of ​​the electronic device and stored by the electronic device, the target font is a font preset in the target font library of the electronic device, the fusion style is used to characterize a style after the target font and a handwriting font corresponding to the characters included in the handwriting font library are fused, and the first gear is used to determine a ratio between a target font style corresponding to the target font and a handwriting font style corresponding to the handwriting font in the fusion style; A first beautified character is generated, where the first beautified character is the first handwritten character beautified according to the information of the first handwritten character and the fusion style.

2. The method according to claim 1, characterized in that The target font library of the electronic device includes at least two fonts, and the method further includes: receiving a second instruction, where the second instruction is used to indicate the target font, wherein the target font is one of the at least two fonts; According to the second instruction, the target font is determined.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: receiving a third instruction, the third instruction being used to indicate a second handwritten word, the second handwritten word being a word written by the user in the handwriting area, and the second handwritten word being used to generate the handwritten word library; According to the third instruction, the handwritten character library is generated, and the handwritten character library includes one or more second handwritten characters.

4. The method according to claim 3, characterized in that The method further comprises: receiving a fourth instruction, wherein the fourth instruction is used to instruct to update the second handwritten character in the handwritten character library; According to the fourth instruction, the updated handwritten character library is generated, and the updated handwritten character library includes one or more updated second handwritten characters.

5. The method according to claim 3 or 4, characterized in that: The determining of the fusion style according to the information of the first gear, the handwriting font library, and the target font includes: Determine, according to the information of the first gear, a first number of the second handwritten characters and a second number of target characters corresponding to the target font, wherein the first number and the second number are determined according to the information of the first gear; The fusion style is determined according to the first number of the second handwritten characters and the second number of the target characters.

6. The method according to claim 5, characterized in that The method further comprises: Determining a first ratio according to information of the first gear, where the first ratio is a ratio between the second quantity and the first quantity; The first quantity and the second quantity are determined according to the first ratio.

7. The method according to any one of claims 1 to 6, characterized in that The generating of the first beautified word comprises: Determining a standard character corresponding to the first handwritten character according to information of the first handwritten character, wherein the standard character is a character preset in a standard character library of the terminal device; The first beautified characters are generated according to the standard characters and the fusion style.

8. The method according to claim 7, characterized in that The step of generating the first beautified character according to the standard character and the fusion style includes: Extracting content features corresponding to the standard word, wherein the content features are used to determine the standard word; The first beautified characters are generated according to the content features and the fusion style.

9. The method according to claim 8, characterized in that The method further comprises: Determining stroke information of the standard character, wherein the stroke information includes a stroke order of the standard character and / or a stroke category of the standard character; The step of generating the first beautified characters according to the content feature and the fusion style includes: The first beautified character is generated according to the content feature, the fusion style, and the stroke information of the standard character.

10. The method according to any one of claims 7 to 9, characterized in that The step of generating the first beautified character according to the standard character and the fusion style includes: The first beautified characters are generated according to the standard characters, the fusion style, and the first model, wherein the training data set of the first model includes at least one of the following: a training data set of handwritten characters, a training data set of target characters, and a training data set of beautified characters, and the training data set of beautified characters includes training beautified characters of different first gears.

11. The method according to any one of claims 1 to 10, characterized in that The method further comprises: Determining the stroke width of the first beautified character, and / or smoothing the stroke width of the first beautified character; The first beautified character is rendered according to the pen stroke width and / or the smoothed pen stroke width to generate the rendered first beautified character.

12. The method according to any one of claims 1 to 11, characterized in that The method further comprises: Determine format information of the first handwritten character, where the format information of the first handwritten character includes at least one of the following: a position of the first handwritten character, a size of the first handwritten character, a length-to-width ratio of the first handwritten character, and a direction of the first handwritten character; Determining format information of the first beautified character according to the format information of the first handwritten character, wherein the format information of the first beautified character includes at least one of the following: a position of the first beautified character, a size of the first beautified character, a length-to-width ratio of the first beautified character, and a direction of the first beautified character; According to the format information of the first beautified word, single-word adjustment is performed on the first beautified word.

13. The method according to claim 12, characterized in that The method further comprises: Perform multi-word adjustment on the plurality of the first beautified characters, wherein the multi-word adjustment includes at least one of the following: adjusting the character spacing between adjacent first beautified characters; and / or, aligning the plurality of the first beautified characters in a row; and / or, adjusting the size of all the first beautified characters in one or more rows.

14. The method according to any one of claims 1 to 13, characterized in that The information of the first handwritten character includes an image corresponding to the first handwritten character and / or a trajectory corresponding to the first handwritten character.

15. An electronic device, characterized in that: include: Acquisition module, processing module, The acquisition module is used to acquire information of a first handwritten word, where the first handwritten word is a word written by a user in a handwriting area of ​​an electronic device; The acquisition module is used to detect a first input operation of the user, and acquire information of the first gear position in response to the first input operation; The processing module is used to determine a fusion style according to the information of the first gear, a handwriting font library, and a target font, wherein the handwriting font library includes characters written by the user in a handwriting area of ​​the electronic device and stored by the electronic device, the target font is a font preset in the target font library of the electronic device, the fusion style is used to characterize a style after the target font and a handwriting font corresponding to the characters included in the handwriting font library are fused, and the first gear is used to determine a ratio between a target font style corresponding to the target font and a handwriting font style corresponding to the handwriting font in the fusion style; The processing module is used to generate a first beautified character, where the first beautified character is the first handwritten character beautified according to the information of the first handwritten character and the fusion style.

16. The electronic device according to claim 15, characterized in that: The fonts pre-set in the target font library of the electronic device include at least two fonts, The acquisition module is used to receive a second instruction, where the second instruction is used to indicate the target font, wherein the target font is one of the at least two fonts; The processing module is used for determining the target font according to the second instruction.

17. The electronic device according to claim 15 or 16, characterized in that: The acquisition module is used to receive a third instruction, the third instruction is used to indicate a second handwritten word, the second handwritten word is a word written by the user in the handwriting area, and the second handwritten word is used to generate the handwritten word library; The processing module is used to generate the handwritten character library according to the third instruction, and the handwritten character library includes one or more second handwritten characters.

18. The electronic device according to claim 17, characterized in that: The acquisition module is used to receive a fourth instruction, where the fourth instruction is used to instruct to update the second handwritten character in the handwritten character library; The processing module is used for generating the updated handwritten character library according to the fourth instruction, wherein the updated handwritten character library includes one or more updated second handwritten characters.

19. The electronic device according to claim 17 or 18, characterized in that: The processing module is used to determine the fusion style according to the information of the first gear, the handwriting font library, and the target font, including: The processing module is used to determine, according to the information of the first gear, a first number of the second handwritten characters and a second number of target characters corresponding to the target font, wherein the first number and the second number are determined according to the information of the first gear; The processing module is used to determine the fusion style based on a first number of the second handwritten characters and a second number of target characters corresponding to the target font.

20. The electronic device according to claim 19, characterized in that The processing module is used to determine a first ratio according to the information of the first gear position, where the first ratio is a ratio between the second number and the first number; The processing module is used to determine the first quantity and the second quantity according to the first ratio.

21. The electronic device according to any one of claims 15 to 20, characterized in that: The processing module is used to generate a first beautified word, including: The processing module is used to determine, based on the information of the first handwritten character, a standard character corresponding to the first handwritten character, wherein the standard character is a character preset in a standard character library of the terminal device; The processing module is used to generate the first beautified character according to the standard character and the fusion style.

22. The electronic device according to any one of claims 21, characterized in that: The processing module is used to generate the first beautified character according to the standard character and the fusion style, including: The processing module is used to extract content features corresponding to the standard word, and the content features are used to determine the standard word; The processing module is used to generate the first beautified word according to the content feature and the fusion style.

23. The electronic device according to claim 22, characterized in that: The processing module is further used to determine the stroke information of the standard character, wherein the stroke information includes the stroke order of the standard character and / or the stroke category of the standard character; The processing module is used to generate the first beautified word according to the content feature and the fusion style, including: The processing module is used to generate the first beautified character according to the content feature and the fusion style, and the stroke information of the standard character.

24. The electronic device according to any one of claims 21 to 23, characterized in that: The processing unit is used to generate the first beautified character according to the standard character and the fusion style, including: The processing unit is used to generate the first beautified characters according to the standard characters, the fusion style, and the first model, wherein the training data set of the first model includes at least one of the following: a training data set of handwritten characters, a training data set of target characters, and a training data set of beautified characters, and the training data set of beautified characters includes training beautified characters of different first gears.

25. The electronic device according to any one of claims 15 to 24, characterized in that: The processing module is used to determine the stroke width of the first beautified character, and / or to perform smoothing processing on the stroke width of the first beautified character; The processing module is used for rendering the first beautified character according to the pen stroke width and / or the smoothed pen stroke width to generate the rendered first beautified character.

26. The electronic device according to any one of claims 15 to 25, characterized in that: The processing module is used to determine the format information of the first handwritten character, wherein the format information of the first handwritten character includes at least one of the following: the position of the first handwritten character, the size of the first handwritten character, the aspect ratio of the first handwritten character, and the direction of the first handwritten character; The processing module is used to determine the format information of the first beautified character according to the format information of the first handwritten character, wherein the format information of the first beautified character includes at least one of the following: the position of the first beautified character, the size of the first beautified character, the aspect ratio of the first beautified character, and the direction of the first beautified character; The processing module is used for performing single-word adjustment on the first beautified word according to the format information of the first beautified word.

27. The electronic device according to claim 26, characterized in that: The processing module is also used to perform multi-word adjustment on multiple first beautified words, and the multi-word adjustment includes at least one of the following: adjusting the word spacing between adjacent first beautified words; and / or aligning multiple first beautified words in rows; and / or adjusting the size of all the beautified words in one or more rows.

28. The electronic device according to any one of claims 15 to 27, characterized in that: The information of the first handwritten character includes an image corresponding to the first handwritten character and / or a trajectory corresponding to the first handwritten character.

29. An electronic device, characterized in that: The terminal device comprises a processor and a memory, wherein the memory is used to store computer programs or instructions, and the processor is used to execute the computer programs or instructions in the memory, so that the method according to any one of claims 1 to 14 is executed.

30. A chip system, characterized in that: It includes a processor, which is used to call and run a computer program from a memory, so that a terminal device equipped with the chip system executes the method as claimed in any one of claims 1 to 14.

31. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 14.

32. A computer program product, characterized in that The computer program product comprises means for executing the method according to any one of claims 1 to 14.

Images