Method for beautifying handwritten character and electronic device

Abstract

A method for beautifying a handwritten character. On the one hand, information of a first handwritten character can be acquired; on the other hand, a fusion style can be determined on the basis of information of a first level, a handwritten character library, and a target font, and the fusion style can be used for representing a style obtained by fusing the target font with a handwritten font corresponding to a character in the handwritten character library; and finally a first beautified character corresponding to the first handwritten character is generated on the basis of information of the first handwritten character and the fusion style. According to the method, the first level can be selected by a user, and a handwritten character of the user can be beautified, and the style of the handwritten character of the user can be reserved in the beautification effect of the finally presented handwritten character. Also disclosed is an electronic device.

Description

Method for beautifying handwriting and electronic device

[0001] This application claims priority to the Chinese patent application with application number 202311167833.2 filed with the State Intellectual Property Office of China on September 8, 2023, and priority to the Chinese patent application with the invention name “A method and electronic device for beautifying handwriting”, all contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of electronic whiteboard handwriting, and more specifically, to a method and electronic device for beautifying handwriting. Background Art

[0003] With the rapid development of society, traditional teaching and conferencing systems have been gradually replaced by emerging electronic products. Cloud-based teaching and conferencing methods have become increasingly familiar and accepted by the public. Touch-screen display devices, such as electronic whiteboards, have emerged as a replacement for traditional non-electronic blackboards and chalk-based devices, significantly reducing costs and being environmentally friendly. They can also better integrate with the cloud to meet diverse teaching and conferencing needs. However, most whiteboard software or screens currently available on the market often suffer from unsightly handwriting due to screen slippage, software latency, and other issues. Consequently, the fonts displayed on electronic whiteboards lack aesthetic appeal.

[0004] Existing solutions primarily use technologies like glyph topology and handwriting recognition to beautify handwriting on electronic whiteboards. However, these solutions produce uniform fonts across different users, failing to preserve the user's individual handwriting style. In other words, the beautification effects achieved by these solutions are monotonous and fail to provide personalized handwriting enhancements for different users.

[0005] Summary of the Invention

[0006] The present application provides a method for beautifying handwriting, which can not only beautify the user's handwriting on the screen, but also retain the user's handwriting style while beautifying the handwriting, providing different beautification effects for different users.

[0007] In a first aspect, a method for beautifying handwriting is provided. The method can be executed by a terminal device (eg, a mobile phone, a laptop computer, etc.), or can be executed by a component of the terminal device (eg, a chip or a circuit), without limitation.

[0008] The method includes: obtaining information of a first handwritten character, where the first handwritten character is a character written by a user on a handwriting area of ​​an electronic device; detecting a first input operation of the user, and obtaining information of a first gear in response to the first input operation; determining a fusion style based on the information of the first gear, a handwritten character library, and a target font, wherein the handwritten character library is a character written by a user on a handwriting area of ​​the electronic device and stored by the electronic device, the target font is a font preset in a target font library of the electronic device, the fusion style is used to characterize a fusion style of the target font and a handwritten font corresponding to a character included in the handwritten character library, and the first gear is used to determine a ratio between a target font style corresponding to the target font and a handwritten font style corresponding to the handwritten font in the fusion style; and generating a first beautified character, where the first beautified character is the first handwritten character beautified based on the information of the first handwritten character and the fusion style.

[0009] In this application, "first handwriting" refers to the words written by the user on the electronic whiteboard, "second handwriting" refers to the words in the user's handwriting font library stored in the terminal device, and "standard words" are words pre-set in the standard font library of the terminal device. "Target words" are words corresponding to at least one font pre-set in the target font library of the terminal device. In this application, in one possible implementation, it can be said that the user's handwriting font library is obtained by collecting the user's historical handwriting. In another possible implementation, the terminal device can obtain the user's handwriting font library by collecting the user's current handwriting.

[0010] In this application, "first gear information" may indicate "first gear", or may be understood as determining the first gear based on the first gear information. The "first gear" may be user-specified. For example, the first gear may be set to three levels: "low", "medium", and "high", or "weak", "medium", and "strong", etc. The user may enter the desired first gear on the terminal device interface.

[0011] In this application, the first gear is used to determine the ratio between the target font style corresponding to the target font in the fusion style and the handwriting font style corresponding to the handwriting font. It can also be understood that the first gear is used to determine the weight between the target font style corresponding to the target font in the fusion style and the handwriting font style corresponding to the handwriting font. In this application, the term "ratio" can also be replaced by "weight."

[0012] In the present application, different first gears can represent the ratio between the handwriting font style (i.e., the handwriting font style corresponding to the handwritten characters) and the target font style (i.e., the target font style corresponding to the target characters) in the fusion style. The higher the first gear, the higher the proportion of the target font style in the fusion style of the generated beautified characters, and the lower the first gear, the higher the proportion of the handwriting font style in the fusion style of the generated beautified characters. For example, if the first gear selected by the user is "low", it indicates that the generated beautified characters are more like the user's own handwriting; if the first gear selected by the user is "high", it indicates that the generated beautified characters are more like the target font selected by the user. In terms of specific implementation, in one possible implementation, the first gear affects the number of handwritten characters and the number of target characters. In another possible implementation, the first gear affects the ratio between the first coefficient corresponding to the handwriting font style and the second coefficient corresponding to the target font style.

[0013] In the present application, the information of the first handwriting includes an image corresponding to the first handwriting and / or a trajectory corresponding to the first handwriting.

[0014] In the present application, the first input operation, for example, can be understood as a first instruction input by a user, and the first instruction is used to indicate information of a first gear.

[0015] Based on the above technical solution, in this application, on the one hand, information about the first handwritten character can be obtained, and on the other hand, a fusion style can be determined using the first gear, the handwritten font library, and the target font. This fusion style can be used to characterize the style of the target font and the handwritten font corresponding to the user's handwritten character in the handwritten font library after fusion. Ultimately, a first beautified character corresponding to the first handwritten character is generated based on the information about the first handwritten character and the fusion style. Not only can the user's handwritten character be beautified based on the first gear selected by the user, but the user's personal handwritten style can also be retained in the beautified effect of the final rendered handwritten character.

[0016] In combination with the first aspect, in one possible implementation, the fonts pre-installed in the target font library of the electronic device include at least two fonts, and the method further includes: receiving a second instruction, the second instruction being used to indicate a target font, wherein the target font is one of the at least two fonts; and determining the target font according to the second instruction.

[0017] In this application, the target font library of the electronic device may include multiple fonts, and the user may select one of the target fonts according to needs. In this application, multiple different fonts may be pre-installed in the electronic device. For example, the multiple different fonts may include: Kaiti, Lishu, Songti, etc. In some scenarios, if the user does not select a target font, the electronic device may default to a font in the target font library as the target font. This application does not limit the method for determining the target font.

[0018] In combination with the first aspect, in a possible implementation, the method also includes: receiving a third instruction, the third instruction is used to indicate a second handwritten word, the second handwritten word is a word written by a user on a handwriting area of ​​an electronic device, and the second handwritten word is used to generate the handwritten word library; according to the third instruction, a handwritten word library is generated, and the handwritten word library includes one or more second handwritten words.

[0019] Exemplarily, the "second handwriting" is a text previously written by the user, in which the user has circled some words, and these circled words constitute the handwriting library; exemplary, the "second handwriting" can be based on some words circled in the first handwriting currently written by the user, thereby constituting a handwriting library. For example, the first handwriting may be more, so the user can circle some words in the first handwriting as the second handwriting, thereby constituting a handwriting library; exemplary, as long as the user writes words in the handwriting area of ​​the electronic device, the handwriting can be automatically added to the handwriting library. It can also be understood that the second handwriting in the handwriting library in this application can come from the first handwriting or from the words previously written by the user, and so on, without limitation.

[0020] In combination with the first aspect, in a possible implementation, the method further includes: receiving a fourth instruction, the fourth instruction being used to instruct updating the second handwritten character in the handwritten character library; generating an updated handwritten character library according to the fourth instruction, the updated handwritten character library including one or more updated second handwritten characters.

[0021] In this application, the handwritten characters in the handwriting character library can also be updated. Here, "updating" can be understood as replacing the handwritten characters in the handwriting character library or adding handwritten characters to the handwriting character library. For example, the user can then rewrite a paragraph of text in the handwriting area of ​​the electronic device and circle some handwritten characters to replace the handwritten characters in the handwriting character library.

[0022] In combination with the first aspect, in a possible implementation method, a fusion style is determined based on the information of the first gear, the handwritten font library, and the target font, including: determining the target characters corresponding to the first number of second handwritten characters and the second number of target fonts based on the information of the first gear, wherein the first number and the second number are determined based on the information of the first gear; determining the fusion style based on the target characters corresponding to the first number of second handwritten characters and the second number of target fonts.

[0023] In this application, the fusion style can be determined by extracting a first number of second handwritten characters and a second number of target characters. Specifically, the fusion style can be determined by the following method:

[0024] Method 1:

[0025] In one possible implementation, the fused style is determined by simultaneously extracting the font styles of the first number of second handwritten characters and the second number of target characters. This implementation can also be understood as simultaneously extracting the font styles of the first number of second handwritten characters and the second number of target characters (i.e., all input characters) to obtain the fused style.

[0026] Method 2:

[0027] In another possible implementation, determining a fusion style based on the first number of second handwritten characters and the target characters corresponding to the second number of target fonts includes: extracting the handwritten font style corresponding to each second handwritten character, and extracting the target font style corresponding to each target character; and determining the fusion style based on the handwritten font style corresponding to each second handwritten character and the target font style corresponding to each target character. This implementation can also be understood as extracting the font style of each input handwritten character or each target character, and then reshaping and concatenating the extracted font styles corresponding to each character to obtain the fusion style.

[0028] Method 3:

[0029] In another possible implementation, a fusion style is determined based on the information of the first level, a handwritten font library, and a target font, including: determining an arbitrary number of second handwritten characters and an arbitrary number of target characters corresponding to the target font; determining the styles of the handwritten fonts corresponding to the second handwritten characters based on the arbitrary number of second handwritten characters, and determining the styles of the target fonts corresponding to the target characters based on the arbitrary number of target characters corresponding to the target font; determining a second ratio based on the information of the first level, wherein the second ratio is the ratio between the second coefficient and the first coefficient, the second coefficient being used to characterize the proportion of the target font style in the fusion style, and the first coefficient being used to characterize the proportion of the handwritten font style in the fusion style; and determining the fusion style based on the first coefficient, the style of the handwritten font, the second coefficient, and the style of the target font. In this implementation, the number of input handwritten characters and the number of target characters in the target character set are not limited, but the handwritten font styles corresponding to the extracted second handwritten characters and the target font styles corresponding to the target characters can be weighted according to their respective corresponding weighting coefficients to obtain a fusion style.

[0030] In combination with the first aspect, in a possible implementation, the method further includes: determining a first ratio based on information of the first gear, the first ratio being the ratio between the second quantity and the first quantity; and determining the first quantity and the second quantity based on the first ratio.

[0031] In terms of specific implementation, for example, the first ratio refers to the ratio of the second quantity to the first quantity, and for another example, the first ratio refers to the ratio of the first quantity to the second quantity, without limitation.

[0032] Exemplarily, in the present application, the beautification module 130 can directly determine the number of handwritten characters and the number of target characters according to the different first gears determined by the user. Exemplarily, if the first gear selected by the user is "low", the beautification module 130 can directly determine the number of handwritten characters to be 7 and the number of target characters to be 3. Exemplarily, if the first gear selected by the user is "medium", the beautification module 130 can directly determine the number of handwritten characters to be 5 and the number of target characters to be 5. Exemplarily, if the first gear selected by the user is "high", the beautification module 130 can directly determine the number of handwritten characters to be 3 and the number of target characters to be 7.

[0033] Exemplarily, the beautification module 130 can determine a first ratio based on the first gear determined by the user. For example, the first ratio is the ratio between the number of handwritten characters and the number of target characters, and the number of handwritten characters and the number of target characters are determined based on the first ratio. Exemplarily, if the first gear selected by the user is "low", the beautification module 130 can determine that the ratio between the number of target characters and the number of handwritten characters is 3:7 (number of handwritten characters: number of target characters), and further, determine that the number of handwritten characters is 7 and the number of target characters is 3. Exemplarily, if the first gear selected by the user is "medium", the beautification module 130 can determine that the ratio between the number of target characters and the number of handwritten characters is 5:5 (number of handwritten characters: number of target characters), and further, determine that the number of handwritten characters is 5 and the number of target characters is 5. For example, if the first gear selected by the user is "high", the beautification module 130 can determine that the ratio between the number of target characters and the number of handwritten characters is 7:3 (number of handwritten characters: number of target characters), and further determine that the number of handwritten characters is 3 and the number of target characters is 7.

[0034] Based on the above technical solution, in this application, it is possible to achieve multi-level handwriting beautification effects based on the user selecting the first level, and the beautification effect on handwriting in this application can retain the user's personal handwriting style. The user can choose different first levels to achieve whether the final beautified words are biased towards the user's handwriting style or the target font style.

[0035] In combination with the first aspect, in a possible implementation method, generating a first beautified character includes: determining a standard character corresponding to the first handwritten character based on information of the first handwritten character, where the standard character is a character preset in a standard character library of the electronic device; and generating the first beautified character based on the standard character and a fusion style.

[0036] In this application, "standard characters" can be understood as characters with correct stroke order and squareness. The standard characters can be characters without any fonts or characters with fonts, without limitation.

[0037] In combination with the first aspect, in a possible implementation, generating a first beautified character based on the standard character and the fusion style includes: extracting content features corresponding to the standard character, the content features are used to determine the standard character; and generating the first beautified character based on the content features and the fusion style.

[0038] The "content feature" in this application can be used to characterize the stroke structure information of the standard character and / or the semantic information of the standard character. For example, the specific character of the standard character can be determined based on the content feature.

[0039] In combination with the first aspect, in a possible implementation, the method also includes: determining the stroke information of the standard character, wherein the stroke information includes the stroke order of the standard character and / or the stroke category of the standard character; generating a first beautified character based on the content characteristics and the fusion style, including: generating the first beautified character based on the content characteristics and the fusion style, and the stroke information of the standard character.

[0040] Based on the above technical solution, the problem of font collapse in the trajectory of generating beautified characters can be effectively alleviated, the error rate of generated beautified characters can be reduced, and the generation quality of beautified characters can be improved.

[0041] In combination with the first aspect, in a possible implementation method, generating a first beautified character based on the standard character and the fusion style includes: generating the first beautified character based on the standard character and the fusion style, and a first model, wherein the training data set of the first model includes at least one of the following: a training data set of handwritten characters, a training data set of target characters, and a training data set of beautified characters, and the training data set of beautified characters includes training beautified characters of different first gears.

[0042] In combination with the first aspect, in a possible implementation method, the training beautified characters are obtained based on the first training standard characters, the first training handwritten character set, the first training target character set, and the second model, the first training handwritten character set includes a third number of training handwritten characters, the first training target character set includes a fourth number of training target characters, the third number and the fourth number are determined based on the information of the first gear, wherein the training data set of the second model is a training data set of handwritten characters, when training the second model, the input of the second model is the second training standard characters and the second training handwritten character set, and the training label is the second training handwritten character set, or; the training data set of the second model is a training data set of target characters, when training the second model, the input of the second model is the second training standard characters and the second training target character set, and the training label is the second training target character set.

[0043] It should be understood that the data set in this application may be a public data set, which is used for model training of beautified characters. For example, the data set may further include a training data set and an inference data set. Among them, the training data set is used for model training, and the inference data set is used for model inference. For example, the training data set may include a training data set of standard characters, a training data set of handwritten characters, and a training data set of target characters. For example, the inference data set may include an inference data set of standard characters, an inference data set of handwritten characters, and an inference data set of target characters. In this application, the training data set and the inference data set are no longer distinguished, and are collectively referred to as "training data sets".

[0044] In this application, considering that when the first gear is high, the target font style corresponding to the target font gradually increases, the error rate of the generated beautified characters will become larger. This is because, under normal circumstances, the training data set consists of handwritten characters written by many users on electronic whiteboards, which are usually sloppy and not neat enough, resulting in a high proportion of target characters under the first gear. The "decoder" in the model is prone to errors in reconstructing the characters, and re-collecting neat handwritten characters requires a certain amount of manpower cost. In view of this, in order to expand the training data set of the model and make the model perform better, this application proposes that since the output of the second model is beautified characters of different gears, the output result of the second model can be used as the input of the first model, so as to further train the first model and optimize the performance of the first model. The beautified characters output by the model are more accurate, and the generation quality of the beautified characters is higher.

[0045] It should be understood that the first model and the second model in this application can be the same model. In this case, the output result of the first model is used as its own input again to train the first model again to optimize the performance of the first model.

[0046] In combination with the first aspect, in a possible implementation, the method also includes: determining the stroke width of the first beautified character, and / or smoothing the stroke width of the first beautified character; rendering the first beautified character according to the stroke width and / or the smoothed stroke width to generate a rendered first beautified character.

[0047] Based on the above-proposed method for rendering beautified characters, the thickness of the pen strokes of the generated beautified characters can be changed, which is closer to the characteristics of users actually writing on the electronic whiteboard, thereby improving the user experience.

[0048] In combination with the first aspect, in a possible implementation, the method further includes: determining the format information of the first handwritten character, the format information of the first handwritten character includes at least one of the following: the position of the first handwritten character, the size of the first handwritten character, the aspect ratio of the first handwritten character, and the direction of the first handwritten character; determining the format information of the first beautified character based on the format information of the first handwritten character, wherein the format information of the first beautified character includes at least one of the following: the position of the first beautified character, the size of the first beautified character, the aspect ratio of the first beautified character, and the direction of the first beautified character; and performing single-character adjustments on the first beautified character based on the format information of the first beautified character.

[0049] Based on the above technical solution, in this application, the size, position and other information of the handwritten characters can be retained, so that the handwritten characters will not be misplaced after beautification, or it can also be understood that the beautified characters and handwritten characters generated in this application can be well fitted.

[0050] In combination with the first aspect, in a possible implementation, the method further includes: performing multi-word adjustment on multiple first beautified characters, where the multi-word adjustment includes at least one of the following: adjusting the character spacing between adjacent first beautified characters; and / or, aligning multiple first beautified characters in rows; and / or, adjusting the size of all first beautified characters in one or more rows.

[0051] Based on the above technical solution, in this application, in order to make the display effect of multiple beautified characters more neat and beautiful and improve the user experience, this application also proposes to optimize the character spacing, line spacing, etc. of multiple beautified characters.

[0052] In a second aspect, an electronic device is provided, which is configured to perform the method of any possible implementation of the first aspect. Specifically, the electronic device may include units and / or modules, such as a transceiver unit and / or a processing unit, configured to perform the method of any possible implementation of the first aspect.

[0053] In one implementation, the electronic device is a terminal device, the communication unit may be a transceiver or an input / output interface, and the processing unit may be at least one processor. Optionally, the transceiver may be a transceiver circuit. Optionally, the input / output interface may be an input / output circuit.

[0054] In another implementation, the electronic device is a chip, chip system, or circuit for a terminal device. When the device is a chip, chip system, or circuit for an electronic device, the communication unit may be an input / output interface, interface circuit, output circuit, input circuit, pin, or related circuit on the chip, chip system, or circuit; and the processing unit may be at least one processor, processing circuit, or logic circuit.

[0055] In a third aspect, an electronic device is provided, comprising: at least one processor configured to execute a computer program or instructions stored in a memory to perform the method of any possible implementation of the first aspect. Optionally, the device further comprises a memory configured to store the computer program or instructions. Optionally, the device further comprises a communication interface, through which the processor reads the computer program or instructions stored in the memory.

[0056] In one implementation, the electronic device is a terminal device.

[0057] In another implementation, the electronic device is a chip, a chip system, or a circuit for a terminal device.

[0058] In a fourth aspect, the present application provides a processor, comprising: an input circuit, an output circuit, and a processing circuit. The processing circuit is configured to receive a signal through the input circuit and transmit a signal through the output circuit, so that the processor executes the method in any possible implementation of the first aspect.

[0059] In a specific implementation, the processor may be one or more chips, the input circuit may be an input pin, the output circuit may be an output pin, and the processing circuit may be a transistor, a gate circuit, a trigger, or various logic circuits. The input signal received by the input circuit may be, for example, but not limited to, received and input by a transceiver, and the signal output by the output circuit may be, for example, but not limited to, output to and transmitted by a transmitter. The input circuit and the output circuit may be the same circuit, which functions as an input circuit and an output circuit at different times. The embodiments of the present application do not limit the specific implementation of the processor and various circuits.

[0060] For the operations such as sending and acquiring / receiving involved in the processor, unless otherwise specified, or if they do not conflict with their actual functions or internal logic in the relevant descriptions, they can be understood as processor output, reception, input and other operations, and can also be understood as sending and receiving operations performed by the radio frequency circuit and antenna. This application does not limit this.

[0061] In a fifth aspect, a processing device is provided, comprising a processor and a memory. The processor is configured to read instructions stored in the memory and receive signals via a transceiver and transmit signals via a transmitter to execute the method of any possible implementation of the first aspect.

[0062] Optionally, there are one or more processors and one or more memories.

[0063] Optionally, the memory may be integrated with the processor, or the memory may be provided separately from the processor.

[0064] In the specific implementation process, the memory can be a non-transitory memory, such as a read-only memory (ROM), which can be integrated with the processor on the same chip or can be set on different chips. The embodiments of the present application do not limit the type of memory and the setting method of the memory and the processor.

[0065] It should be understood that related data interaction processes, such as sending indication information, can be the process of outputting indication information from the processor, and receiving capability information can be the process of receiving input capability information from the processor. Specifically, data output by the processor can be output to the transmitter, and input data received by the processor can be received from the transceiver. The transmitter and transceiver can be collectively referred to as a transceiver.

[0066] The processing device in the fifth aspect may be one or more chips. The processor in the processing device may be implemented in hardware or software. When implemented in hardware, the processor may be a logic circuit, an integrated circuit, or the like; when implemented in software, the processor may be a general-purpose processor implemented by reading software code stored in a memory, which may be integrated into the processor or located independently of the processor.

[0067] In a sixth aspect, a computer-readable storage medium is provided, which stores a program code for execution by a device, wherein the program code includes a method for executing any possible implementation of the first aspect.

[0068] In a seventh aspect, a computer program product comprising instructions is provided, which, when run on a computer, enables the computer to execute the method in any possible implementation of the first aspect.

[0069] In an eighth aspect, a chip system is provided, comprising a processor for calling and running a computer program from a memory, so that a device equipped with the chip system executes the methods in each implementation of any one of the first to middle aspects above. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] FIG1 is a schematic structural block diagram of an electronic device 100 provided in an embodiment of the present application.

[0071] FIG2 is a schematic diagram of a software architecture of an electronic device 100 applicable to an embodiment of the present application.

[0072] FIG3 is a schematic structural block diagram of a handwriting beautification system 300 provided in an embodiment of the present application.

[0073] FIG4 is a schematic diagram of handwritten character segmentation provided in an embodiment of the present application.

[0074] FIG5 is a schematic flowchart of a method 500 for beautifying handwriting provided in an embodiment of the present application.

[0075] FIG6 is a schematic block diagram of a method for beautifying handwriting provided in an embodiment of the present application.

[0076] FIG7 is a schematic diagram of the beautification effects of beautified characters at different first gears provided by an embodiment of the present application.

[0077] FIG8 is a schematic block diagram of another method for beautifying handwriting provided in an embodiment of the present application.

[0078] FIG9 is a schematic diagram of another embodiment of the present application showing the beautification effect of beautified characters at different first gears.

[0079] FIG10 is a schematic block diagram of another method for beautifying handwriting provided in an embodiment of the present application.

[0080] FIG11 is a schematic diagram of another beautification effect of beautifying characters at different first gears provided in an embodiment of the present application.

[0081] FIG12 is a schematic diagram of the collapse of a generated beautified character provided in an embodiment of the present application.

[0082] FIG13 is a schematic block diagram of another method for beautifying handwriting provided in an embodiment of the present application.

[0083] FIG14 is a schematic diagram of the beautification effect of a beautified character after introducing the stroke information of a standard character provided in an embodiment of the present application.

[0084] FIG15 is a schematic block diagram of a model for training beautified characters provided in an embodiment of the present application.

[0085] FIG16 is another schematic block diagram of a model for training beautified characters provided in an embodiment of the present application.

[0086] FIG17 is a schematic flowchart of calculating the stroke width of a beautified character and smoothing the stroke width provided in an embodiment of the present application.

[0087] FIG18 is a schematic diagram of an S-curve provided in an embodiment of the present application.

[0088] FIG19 is a schematic diagram of interpolating trajectory points of beautified characters provided in an embodiment of the present application.

[0089] FIG20 is a schematic diagram of the beautification effect of rendering beautified characters provided in an embodiment of the present application.

[0090] FIG21 is a schematic diagram of determining the center of gravity of handwritten characters provided in an embodiment of the present application.

[0091] FIG22 is a schematic diagram of determining the distance from a key point of a handwritten character to its surroundings provided in an embodiment of the present application.

[0092] FIG23 is a schematic diagram of a single beautified character and a corresponding handwritten character provided in an embodiment of the present application.

[0093] FIG24 is a schematic diagram of the bonding of a plurality of beautified characters and a corresponding plurality of handwritten characters provided in an embodiment of the present application.

[0094] FIG25 is a schematic diagram of row alignment of multiple beautified characters provided in an embodiment of the present application.

[0095] FIG26 is a schematic diagram of adjusting the line spacing of multiple beautified characters provided in an embodiment of the present application.

[0096] Figure 27 is a schematic diagram of aligning multiple lines of beautified characters provided in an embodiment of the present application.

[0097] FIG28 is a schematic diagram of aligning titles of beautified characters provided in an embodiment of the present application.

[0098] FIG29 is a schematic diagram of character segmentation and segmentation of beautified characters provided in an embodiment of the present application.

[0099] FIG30 is a schematic diagram of a character frame of a target font provided in an embodiment of the present application.

[0100] FIG31 is a schematic diagram of various parameters in a character box of a target font provided in an embodiment of the present application.

[0101] FIG32 is a schematic diagram of the beautification effect of a beautified character provided in an embodiment of the present application.

[0102] FIG33 is a schematic diagram of a method for beautifying handwritten characters on a user interface according to an embodiment of the present application.

[0103] FIG34 is a schematic block diagram of an electronic device 1000 provided in an embodiment of the present application.

[0104] FIG35 is a schematic block diagram of an electronic device 2000 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0105] The technical solution in this application will be described below with reference to the accompanying drawings.

[0106] The technical solution provided in this application can also be applied to electronic devices, which are included in terminal devices that support user handwriting, including but not limited to user equipment (UE), tablets, personal computers (PC), office large screens and other screen devices that support writing with fingers or stylus, and also include virtual reality (VR), augmented reality (AR) and other virtual projection devices that support writing with hands or handles. The terminal device may also be a wireless terminal in industrial control, a machine type communication (MTC) terminal, a customer premise equipment (CPE), a wireless terminal in self-driving, a wireless terminal in remote medical, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a smart home, a cellular phone, a cordless phone, a session initiation protocol (SIP) phone, a wireless local loop (WLL) station, a personal digital assistant (PDA), a handheld device with wireless communication capabilities, a computing device or other processing device connected to a wireless modem, an in-vehicle device, a wearable device, a terminal device in a 5G network, or a terminal device in a future evolved public land mobile network (PLMN), etc.

[0107] 1 shows a schematic structural diagram of an electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0108] It should be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown, or may combine or separate certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0109] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0110] The controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.

[0111] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0112] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface.

[0113] The USB interface 130 is an interface that complies with USB standards and may be a Mini USB interface, a Micro USB interface, a USB Type-C interface, or the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100, or to transfer data between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio. This interface can also be used to connect other electronic devices, such as augmented reality devices.

[0114] It is understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is merely an illustrative illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods from the above embodiments, or a combination of multiple interface connection methods.

[0115] The charging management module 140 is configured to receive charging input from a charger. The charger can be either a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also provide power to the electronic device via the power management module 141.

[0116] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the external memory, the display 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0117] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0118] Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In other embodiments, the antennas can be used in conjunction with a tuning switch.

[0119] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 can include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc.

[0120] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0121] Electronic device 100 implements display functionality through a GPU, display screen 194, and an application processor. A GPU is a microprocessor for image processing that connects display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0122] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1. Typically, the display screen 194 serves as an important entry point for a user to interact with the electronic device. For example, in this application, the user can write handwritten words on the display panel; for example, the user can transmit user instructions to the electronic device 100 through the display screen.

[0123] The electronic device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0124] The ISP processes data fed back by camera 193. For example, when taking a photo, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, which is then passed to the ISP for processing and converted into a visible image. The ISP can also perform algorithmic optimization on image noise, brightness, and skin tone. It can also optimize parameters such as exposure and color temperature of the captured scene. In some embodiments, the ISP can be located within camera 193.

[0125] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0126] The digital signal processor is used to process digital signals. In addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0127] Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. This allows electronic device 100 to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0128] The NPU is a neural network (NN) computing processor. Drawing on the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it rapidly processes input information and can continuously self-learn. The NPU can enable intelligent cognitive applications in electronic device 100, such as image recognition, face recognition, speech recognition, and text comprehension.

[0129] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0130] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0131] The electronic device 100 can implement audio functions such as music playback and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0132] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be provided in the processor 110, or some functional modules of the audio module 170 can be provided in the processor 110.

[0133] The buttons 190 include a power button, a volume button, and the like. The buttons 190 may be mechanical buttons or touch buttons. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100.

[0134] The software system of the electronic device 100 can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. In the embodiment of the present application, the Android system with a layered architecture is used as an example to illustrate the software structure of the electronic device 100.

[0135] Figure 2 is a software structure diagram of the electronic device 100 according to an embodiment of the present application. The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, namely, the application layer, the application framework layer, the Android runtime (Android runtime) and the system library, and the kernel layer, from top to bottom. The application layer can include a series of application packages.

[0136] As shown in FIG2 , the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.

[0137] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0138] As shown in FIG2 , the application framework layer may include a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, and the like.

[0139] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0140] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0141] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0142] The phone manager is used to provide communication functions of the electronic device 100, such as management of call status (including answering, hanging up, etc.).

[0143] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0144] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically without user interaction. For example, the Notification Manager is used to notify users of completed downloads and message reminders. The Notification Manager can also display notifications in the top status bar of the system as icons or scrolling text, such as notifications from background applications, or as dialog windows on the screen. Examples include text messages in the status bar, beeps, vibrations on electronic devices, and flashing indicator lights.

[0145] The Android runtime includes the core library and the virtual machine. The Android runtime is responsible for scheduling and management of the Android system.

[0146] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0147] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0148] The system library can include multiple functional modules, such as a surface manager, media libraries, a 3D graphics processing library (such as OpenGL ES), and a 2D graphics engine (such as SGL).

[0149] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0150] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0151] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0152] A 2D graphics engine is a drawing engine for 2D drawings.

[0153] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0154] It should be understood that the technical solutions in the embodiments of the present application can be used in Android, IOS, Hongmeng and other systems.

[0155] Figure 3 is a schematic block diagram of a handwriting beautification system 300 provided in an embodiment of the present application. This system can be used, for example, in electronic devices. As shown in Figure 3 , the system includes a user input module 310, a handwriting recognition and segmentation module 320, a beautification module 130, a brush rendering module 340, and a typesetting module 350. The following describes the primary purpose of each module.

[0156] 1. User Input Module 310

[0157] In this application, the user input module 110 (for example, it can be a handwriting area) generally includes the following uses: (1) for the user to write handwritten words; (2) for the user to select the target font and the first gear; (3) for the user to start ("start" can also be understood as "turn on") the function of beautifying handwritten words. Among them, "target font style" can also be understood as the direction in which the user expects the handwritten words to be beautified. The user can choose different target fonts according to personal preferences and application scenarios. It should be understood that in this application, a variety of fonts can be pre-installed in the terminal device for the user to choose. "First gear" can be understood as the degree to which the user expects the handwritten words to be beautified. If the user expects to retain more of the personal handwriting style, a lower first gear can be selected; on the contrary, if the user expects his handwriting to be as close to the target font as possible, a higher first gear can be selected.

[0158] 2. Handwritten Character Recognition and Segmentation Module 320

[0159] When the user activates the handwriting beautification function, the handwriting recognition and segmentation module 320 will identify and segment the text that needs to be beautified. As shown in Figure 4, the handwriting recognition and segmentation module 320 will recognize the user's handwriting and segment it. Specifically, the handwriting recognition and segmentation module 320 will identify the character information corresponding to the user's handwriting and create rectangular segmentation frames for the handwriting on a character basis. For example, for English, the handwriting recognition and segmentation module 320 will also create segmentation frames on a word basis. In addition, it will also create line segmentation frames for single-line text.

[0160] 3. Beautification Module 330

[0161] The beautification module 330 can be used to construct a handwritten character set based on the first gear input by the user, and the handwritten character set includes one or more handwritten characters in the handwritten character library, and then construct a target character set based on the target font selected by the user, and the target character set includes one or more target characters corresponding to the target font. The beautification module 330 will adjust the ratio between the number of handwritten characters and the number of target characters according to the size of the first gear selected by the user, so as to obtain the handwritten character set and the target character set corresponding to the current first gear. When the first gear tends to the user's handwritten font style, the proportion of the number of handwritten characters in the handwritten character set is higher; when the first gear tends to the target font style, the proportion of the number of target characters in the target character set is higher. The beautification module 330 is also used to extract the fusion features corresponding to the handwritten character set and the target character set. At the same time, the beautification module 330 is used to extract the standard characters corresponding to the handwriting image or handwriting trajectory from the standard character library based on the results of handwriting recognition, and then weight the content features and fusion styles corresponding to the standard characters with different weighting coefficients, and generate the trajectory points of the beautified characters after decoding and reconstruction.

[0162] 4. Rendering Module 340

[0163] The rendering module 340 is used to render the preset brush information such as pen tip thickness, beautified character color, beautified character texture, beautified character opacity, pen tip, etc. on the trajectory points of the beautified character, thereby generating the rendered beautified character.

[0164] 5. Typesetting Module 350

[0165] After the rendering module 340 generates the beautified characters, it needs to replace the handwritten characters with the beautified characters, and the size and position of the beautified characters correspond to the handwritten characters. The typesetting module 350 will calculate the size, aspect ratio, position, orientation and other information of each handwritten character, and then set the corresponding beautified characters to the same parameters. In addition, the typesetting module 350 will further optimize the spacing, size, character spacing, and line spacing between adjacent characters. For example, each line of characters will be centered and the character spacing will be consistent.

[0166] FIG5 is a schematic flow chart of a method 500 for beautifying handwriting provided by the present application. The method may be executed by a terminal device, or by a chip or chip system for the terminal device, without limitation. As shown in FIG5 , the method includes:

[0167] 510, obtaining first handwritten information.

[0168] In this application, "first handwriting" refers to words written by a user on an electronic whiteboard. In this application, information about the first handwriting includes an image corresponding to the first handwriting and / or a trajectory corresponding to the first handwriting. For example, the handwriting recognition and segmentation module 320 can obtain the image or trajectory corresponding to the first handwriting.

[0169] 520 , detecting a first input operation of the user, and acquiring information of the first gear position in response to the first input operation.

[0170] In the present application, the first gear is determined based on the first instruction input by the user, and the target font is a font preset in a target font library of the electronic device.

[0171] In this application, the "first gear information" can be used to indicate the first gear, or it can be understood that the first gear can be determined based on the first gear information. The "first gear" in this application can be specified by the user. For example, the first gear can be set to three levels: "low", "medium", "high" or "weak", "medium", "strong", etc. The user can enter the required first gear on the interface of the terminal device.

[0172] In the present application, different first gears can be used to determine the ratio between the handwritten font style and the target font style in the fusion style. The higher the first gear, the higher the proportion of the target font style in the fusion style of the generated beautified characters, and the lower the first gear, the higher the proportion of the handwritten font style in the fusion style of the generated beautified characters. For example, if the first gear selected by the user is "low", it means that the generated beautified characters are more like the user's own handwriting; if the first gear selected by the user is "high", it means that the generated beautified characters are more like the target font selected by the user. In terms of specific implementation, in one possible implementation, the first gear affects the number of handwritten characters and the number of target characters. In another possible implementation, the first gear affects the ratio between the first coefficient corresponding to the handwritten font style and the second coefficient corresponding to the target font style.

[0173] For example, the user input module 310 may receive a first input operation from the user. In this application, the first input operation may be understood as a first instruction input by the user, where the first instruction is used to indicate information about the first gear position.

[0174] 530, determine the fusion style based on the information of the first gear, the handwriting font library, and the target font.

[0175] In this application, the handwriting font library refers to the characters written by the user in the handwriting area of ​​the electronic device and stored in the electronic device.

[0176] In this application, the target font library of the electronic device may include multiple fonts, and the user may select one of the target fonts according to needs. In this application, multiple different fonts may be pre-installed in the electronic device. For example, the multiple different fonts may include: Kaiti, Lishu, Songti, etc. In some scenarios, if the user does not select a target font, the electronic device may default to a font in the target font library as the target font. This application does not limit the method for determining the target font.

[0177] Optionally, the user can also determine the target font first. Specifically, in one possible implementation, the font library of the terminal device includes at least two different fonts, and the method further includes: receiving a second instruction from the user, the second instruction is used to indicate the target font, and the target font is one of at least two different fonts; determining the target font according to the second instruction. Exemplarily, it is assumed that the present application includes three fonts, namely font #1, font #2, and font #3, and different fonts represent different target font styles. Alternatively, it can also be understood that the user input module 310 can determine the target font according to the user's second instruction.

[0178] Optionally, if the user has not selected a target font in advance, the terminal device may use a default font as the target font. For example, the terminal device may set font #A as the default target font.

[0179] In this application, the fusion style is used to represent the style of the fusion of the target font and the handwriting font corresponding to the characters included in the handwriting font library.

[0180] In one possible implementation, a third instruction is received, where the third instruction is used to indicate a second handwritten character, where the second handwritten character is a character written by a user in a handwriting area of ​​an electronic device, and the second handwritten character is used to generate a handwriting character library; according to the third instruction, a handwriting character library is generated, where the handwriting character library includes one or more second handwritten characters.

[0181] In the present application, the electronic device can obtain the user's handwriting library by collecting the handwriting written by the user before. Alternatively, the electronic device can also obtain the user's handwriting library by collecting the handwriting currently written by the user. Exemplarily, the "second handwriting" is some words circled by the user in a paragraph of text written by the user before, and these circled words constitute the handwriting library; Exemplarily, the "second handwriting" can be based on some words circled in the first handwriting currently written by the user, thereby constituting a handwriting library. For example, the first handwriting may be more, so the user can circle some words in the first handwriting as the second handwriting, thereby constituting a handwriting library; Exemplarily, as long as the user writes words on the handwriting area of ​​the electronic device, the handwriting can be automatically added to the handwriting library. It can also be understood that the second handwriting in the handwriting library in the present application can come from the first handwriting or from the words written by the user before, and so on, without limitation.

[0182] In a possible implementation, the method further includes: receiving a fourth instruction, the fourth instruction being used to instruct updating of a second handwritten character in the handwritten character library; and generating an updated handwritten character library according to the fourth instruction, the updated handwritten character library including one or more updated second handwritten characters.

[0183] In this application, the handwritten characters in the handwriting character library can also be updated. Here, "updating" can be understood as replacing the handwritten characters in the handwriting character library or adding handwritten characters to the handwriting character library. For example, the user can then rewrite a paragraph of text in the handwriting area of ​​the electronic device and circle some handwritten characters to replace the handwritten characters in the handwriting character library.

[0184] In one possible implementation, a fusion style is determined based on information from the first gear, a handwritten character library, and a target font, including: determining a first number of second handwritten characters (also understood as a "handwritten character set") and a second number of target characters corresponding to the target font (also understood as a "target character set") based on the information from the first gear, wherein the first number and the second number are determined based on the information from the first gear; determining a fusion style based on the first number of second handwritten characters and the second number of target characters corresponding to the target font.

[0185] In other words, in this application, the first number is the number of handwritten characters, and the second number is the number of target characters.

[0186] In a possible implementation, in the present application, the beautification module 330 can directly determine the number of handwritten characters and the number of target characters according to the different gears determined by the user. Exemplarily, if the first gear selected by the user is "low", the beautification module 330 can directly determine that the number of handwritten characters is 7, and the number of target characters is 3. For example, 7 handwritten characters can be randomly found from the handwritten character library, and 3 target characters corresponding to the target font can be randomly found from the target font library. Exemplarily, if the first gear selected by the user is "medium", the beautification module 330 can directly determine that the number of handwritten characters is 5, and the number of target characters is 5. Exemplarily, if the first gear selected by the user is "high", the beautification module 330 can directly determine that the number of handwritten characters is 3, and the number of target characters is 7.

[0187] In another possible implementation, the beautification module 130 may determine a first ratio based on a first gear position determined by the user, wherein the first ratio is the ratio between the second number and the first number, and the number of handwritten characters and the number of target characters are determined based on the first ratio. Exemplarily, if the first gear position selected by the user is "low", the beautification module 330 may determine that the ratio between the number of target characters and the number of handwritten characters is 3:7 (number of handwritten characters: number of target characters), and further, determine that the number of handwritten characters is 7 and the number of target characters is 3. Exemplarily, if the first gear position selected by the user is "medium", the beautification module 330 may determine that the ratio between the number of target characters and the number of handwritten characters is 5:5 (number of handwritten characters: number of target characters), and further, determine that the number of handwritten characters is 5 and the number of target characters is 5. For example, if the first gear selected by the user is "high", the beautification module 130 can determine that the ratio between the number of target characters and the number of handwritten characters is 7:3 (number of handwritten characters: number of target characters), and further determine that the number of handwritten characters is 3 and the number of target characters is 7.

[0188] As an example, after determining the target font, the beautification module 330 can determine the number of target characters according to the first gear and randomly determine the second number of target characters corresponding to the target font from the target font library; the beautification module 330 can determine the number of handwritten characters according to the first gear and randomly determine the first number of handwritten characters from the handwritten character library. Exemplarily, when the first gear selected by the user is "low", the beautification module 330 can determine that it includes 7 handwritten characters and 3 target characters. Exemplarily, if the first gear selected by the user is "medium", the beautification module 330 can determine that it includes 5 handwritten characters and 5 target characters. Exemplarily, if the first gear selected by the user is "high", the beautification module 330 can determine that it includes 3 handwritten characters and 7 target characters.

[0189] 540, generate the first beautified character.

[0190] In this application, the first beautified character is the first handwritten character beautified according to the information of the first handwritten character and the fusion style.

[0191] In a possible implementation manner, according to the information of the first handwritten character, determine the standard character corresponding to the first handwritten character, and the standard character is the character preset in the standard character library of the terminal device; generate the first beautified character according to the standard character and the fusion style.

[0192] In this application, the "standard character" is the character preset in the standard character library of the terminal device. In this application, the "standard character" can be understood as a character with correct stroke order and square shape. The standard character can be a character without any font, or a character with a font, which is not limited.

[0193] For example, the handwritten character recognition and segmentation module 320 can recognize which character the user writes on the tablet. For example, assume that the user writes a character "目" on the electronic whiteboard, then the handwritten character recognition and segmentation module 320 can recognize that this character is "目" rather than "日". The beautification module 330 can extract the standard character from the character library of the terminal device according to the recognition result.

[0194] In a possible implementation manner, extract the content feature corresponding to the standard character, and the content feature is used to determine the standard character; generate the first beautified character according to the content feature and the fusion style.

[0195] The "content feature" in this application can be used to characterize the information of the stroke structure of the standard character and / or the semantic information of the standard character. For example, according to the content feature, it can be determined which specific character the standard character is. Among them, the "information of the stroke structure of the standard character" can include, for example, at least one of the following: the glyph structure of the standard character, the stroke layout. The "semantic information of the standard character" can be understood as being able to determine which character the standard character is.

[0196] In the present application, generating a first beautified character may also be understood as generating a trajectory point corresponding to the first beautified character.

[0197] The following specifically introduces several methods of generating beautified characters proposed in this application. For the convenience of description, in the following introduction, "handwritten character set" is used to represent the first number of second handwritten characters, and "target character set" is used to represent the second number of target characters.

[0198] Method 1

[0199] Figure 6 is a schematic block diagram of a method for generating beautified characters provided by this application. As shown in Figure 6, the "Content Encoder" is a content extractor used to extract the content features of standard characters; the "Style Encoder" is a style extractor used to extract the fusion style of the handwritten character set and the target character set; the "Attention" is an attention mechanism used for feature weighting. It can also be understood that the content features and the fusion style are weighted with different weighting coefficients respectively; the "Decoder" is a decoder used for feature fusion and content reconstruction to generate beautified characters.

[0200] It can also be understood that the previous "content extractor", "style extractor" and "attention mechanism" can obtain some digital information used to represent beautified characters, and the "decoder" can be used to decode the digital information to obtain beautified characters that can be used for display. For example, the "decoder" can fuse the final fusion style with the content features of the standard characters and their corresponding weighting coefficients through the decoder and reconstruct the font to obtain the beautified characters in the current first gear. The solution of method one can be understood as extracting the font style of all characters included in the input handwritten character set and the target character set at the same time, so as to obtain the fusion style corresponding to the handwritten character set and the target character set.

[0201] For example, assuming that the first gear of the input is "high", the beautification module 330 determines that the number of target characters in the handwritten character set and the target character set is 6, and the number of handwritten characters in the handwritten character set is 4. Figure 7 is a schematic diagram of beautified characters corresponding to different first gears obtained by beautifying handwritten characters using a method provided by the present application. As shown in Figure 7, when the first gear is low, the fusion style of the beautified characters is closer to the handwritten characters. As the first gear increases, the fusion style of the beautified characters gradually approaches the target font style. It can be seen from Figure 7 that as the first gear increases, the connected strokes of the handwritten characters become weaker, and the beautified characters finally presented are more neat; and it can be seen that under the same first gear, the stroke curvature of font #1 and font #2 is different; different fonts ultimately correspond to different beautified characters. If the target font selected is font #2, the beautified characters finally generated are similar in style to font #2.

[0202] Exemplarily, in a specific implementation, it can be implemented through "matrix" operations. For example, the size of the matrix corresponding to each character in the handwritten character set and the target character set input by the beautification module 130 is (1, 110, 3), where "1" refers to the current number of the character (can also be understood as the first dimension of the matrix), "110" refers to the sequence length of the character, and "3" refers to the current dimension of the character (for example, including the X-axis dimension, Y-axis dimension, and point attributes of the character). Then, all the characters in the handwritten character set and the target character set are spliced ​​according to the first dimension. Assuming that there are 10 characters in the handwritten character set and the target character set, including 4 handwritten characters and 6 beautified characters respectively, after splicing, the size of the matrix corresponding to the handwritten character set and the target character set is (10, 110, 3), where "10" represents that the handwritten character set and the target character set include 10 characters. Then, the matrix can be multiplied. For example, one dimension can be added to the matrix corresponding to the handwritten character set and the target character set. If a single character needs to be inferred, the matrix is ​​increased in dimension to (1, 10, 110, 3), where "1" indicates that a single character needs to be inferred, "10" refers to the number of characters in the current handwritten character set and the target character set (which can also be understood as the second dimension of the matrix), "110" refers to the sequence length of each character in the handwritten character set and the target character set, and "3" refers to the dimension of each character in the handwritten character set and the target character set. Finally, the increased-dimensional handwritten character set and the target character set can be reshaped. For example, the first dimension and the increased-dimensional second dimension can be merged to obtain a reshaped matrix of size (1×10, 110, 3).

[0203] Method 2

[0204] Figure 8 is a schematic block diagram of another method for generating beautified characters provided by the present application. The functions of the various modules in Figure 8 can be understood by referring to the introduction of Figure 6 in Method 1 and will not be repeated here. In the technical solution of Method 1, the style of the input handwritten character set and the target character set can be extracted as a whole to obtain a fusion style. However, in Method 2, the font style of each handwritten character or target character in the input handwritten character set and the target character set can be extracted word by word, and then the font style corresponding to each extracted character is reshaped and spliced ​​to obtain a fusion style. After that, the content features and the fusion style are weighted with different weighting coefficients through the "attention mechanism", and then the feature fusion and content reconstruction are performed through the decoder to generate beautified characters.

[0205] Figure 9 is a schematic diagram of beautified characters corresponding to different first gears obtained by beautifying handwritten characters using the second method provided in this application. As shown in Figure 9, when the first gear is low, the fusion style of the beautified characters is closer to the handwritten font style. As the first gear increases, the fusion style of the beautified characters gradually approaches the target font style.

[0206] Exemplarily, in a specific implementation, it can be implemented through "matrix" operations. For example, the size of the matrix corresponding to each character in the handwritten character set and the target character set in the beautification module 330 is (1, 110, 3), where "1" refers to the number of the current character (which can also be understood as the first dimension of the matrix), "110" refers to the sequence length of the character, and "3" refers to the dimension of the current character (for example, including the X-axis dimension, Y-axis dimension, and point attributes of the character). It should be noted that in method 2, each character in the handwritten character set and the target character set needs to be fed into the model one by one. Assuming that the number of characters to be inferred is 1, the matrix size of each character is (1×1, 110, 3), where "1×1" refers to "the number of inferred characters × the number of beautified characters in the handwritten character set and the target character set", "110" refers to the current sequence length of the character, and "3" refers to the dimension of the character. The resulting feature matrix has a size of (1×1, 110, dim), where "dim" refers to the feature layer dimension (the matrix is ​​usually multiplied, and the value of "dim" will be greater than 3). Then, the matrix is ​​reshaped. The matrix can be split according to the first dimension. The size of the matrix after splitting is (1, 1, 110, dim); preferably, the current matrix is ​​spliced ​​according to the second dimension. If the handwritten character set and the target character set include a total of 10 characters, the size of the matrix corresponding to the handwritten character set and the target character set after splicing is (1, 10, 110, dim).

[0207] Method 3

[0208] Figure 10 is a schematic block diagram of another method for generating beautified characters provided by the present application. The functions of each module in Figure 10 can be understood by referring to the introduction of Figure 6 in Method 1 and will not be repeated here. In the technical solutions of Methods 1 and 2, the beautification module 330 can determine the number of handwritten characters in the handwritten character set and the number of target characters in the target character set according to the first gear. However, in Method 3, the number of handwritten characters in the input handwritten character set and the number of target characters in the target character set are not limited, but the handwritten font styles corresponding to the extracted handwritten characters and the target font styles corresponding to the target characters need to be weighted according to certain weighting coefficients to obtain a fusion style. In other words, in Method 3, the number of handwritten characters in the input handwritten character set and the number of target characters in the target character set can be unrestricted, the beautification module 330 can extract the handwritten font styles of the handwritten characters and the target font styles of the target characters, and then the beautification module 330 can determine the first coefficient corresponding to the handwritten font style of the handwritten characters and the second coefficient corresponding to the target font style according to the first gear, and then determine the fusion style according to their respective weighting coefficients and the handwritten font style and the target font style. The second coefficient is used to represent the proportion of the target font style in the fusion style, and the first coefficient is used to represent the proportion of the handwritten font style in the fusion style.

[0209] In a possible implementation, the beautification module 330 may determine a second ratio according to the first gear, where the second ratio is a ratio between the second coefficient and the first coefficient.

[0210] For example, if the first gear selected by the user is "low", the beautification module 330 can determine that the weighting coefficient of the handwriting font style of the handwritten character is 0.7 (an example of the first coefficient), and the weighting coefficient of the target font style of the target character is 0.3 (an example of the second coefficient). According to their respective weighting coefficients, the handwriting font style of the handwritten character is merged with the target font style of the target character to obtain a fusion style. For example, if the first gear selected by the user is "medium", the beautification module 330 can determine that the weighting coefficient of the handwriting font set style of the handwritten character is 0.5, and the weighting coefficient of the target font style of the target character is 0.5. According to their respective weighting coefficients, the handwriting font style of the handwritten character is merged with the target font style of the target character to obtain a fusion style. For example, if the first gear selected by the user is "high", the beautification module 330 can determine that the weighting coefficient of the handwriting font style of the handwritten word is 0.3, and the weighting coefficient of the target font style of the target word is 0.7. According to their respective weighting coefficients, the handwriting font style of the handwritten word and the target font style of the target word are merged to obtain a fusion style.

[0211] Figure 11 is a schematic diagram of beautified characters corresponding to different first gears obtained by beautifying handwritten characters using method three provided in this application. As shown in Figure 11, when the first gear is low, the fusion style of the beautified characters is closer to the handwritten font style. As the first gear increases, the fusion style of the beautified characters gradually approaches the target font style.

[0212] It should be understood that the technical solutions provided by Methods 1 to 3 can also be understood as a model for beautifying handwritten characters, as shown in Figures 6, 8, and 10. The input of the model is standard characters, handwritten character sets, and target character sets, and the output of the model is beautified characters.

[0213] In the present application, the above-mentioned methods 1 to 3 can realize the beautification effect of multiple levels of handwriting based on the user's selection of the first level, and the beautification effect of handwriting in the present application can retain the user's personal handwriting style. The user can choose different first levels to realize whether the beautified words finally presented are biased towards the user's handwriting style or the target font style.

[0214] Method 4

[0215] Furthermore, in some scenarios, the structure of the generated beautified characters does not match their corresponding standard structures. It can also be understood that the finally generated beautified characters may not be the same as the original handwritten characters, that is, the generated beautified characters are not the user's original handwritten characters on the tablet, and there are scenarios where errors occur in the generated beautified characters. FIG. 12 shows an effect diagram of the collapse of a generated beautified character. As shown in FIG. 12, in the left example, the generated beautified character is not standard and lacks some strokes, and in the right example, the generated beautified character is no longer recognizable as a character. In view of this, the present application proposes that when generating beautified characters, the stroke information of the standard characters can be considered, so as to help the generated beautified characters to be accurate and consistent with the characters written by the user on the electronic whiteboard, and reduce the error rate of the generated beautified characters. The technical solution for beautifying handwritten characters after adding the stroke information of the standard characters is introduced below.

[0216] In this technical solution, it is necessary to determine the structural information of the standard characters so that the generated beautified characters can match the standard structure of the characters. For example, the stroke information of the standard characters can be obtained. The stroke information of the standard characters can include the stroke order and stroke category of the standard characters, and then, based on the standard characters, the handwritten character set, the target character set, and the stroke information of the standard characters, the beautified characters are generated. In a possible implementation manner, the first beautified character can be generated according to the content features and fusion styles corresponding to the standard characters, and the stroke information of the standard characters.

[0217] As an example, the strokes of the standard characters can be encoded according to categories. For example, the strokes of Chinese characters can usually be divided into 32 types, and these 32 types of strokes can be numbered respectively, and each number can uniquely represent a type of stroke. Table 1 below shows a possible stroke encoding. For example, when recognizing the image of a handwritten character, the handwritten character corresponding to the standard character can be determined, and the structural information of the standard character can be determined according to Table 1. Suppose the handwritten character written by the user on the electronic whiteboard is "人", then according to Table 1, the stroke information of the standard character corresponding to this character can be obtained as (04, 05). This encoding method can also reflect the stroke order. For example, the stroke information (04, 05) can also reflect that when forming this character, the "left-falling stroke (丿)" is written first and then the "right-falling stroke (乀)".

[0218] Table 1

[0219] As another example, another encoding method can also be adopted. Suppose there are 32 stroke types in each line. It can be indicated by "0" and "1" which stroke type each stroke of the character is, and how many strokes the character is composed of can be represented by as many lines. For example, for the character "人" which is composed of two strokes, it can be represented by two lines. Its stroke information can be: in the first line, only the stroke type of "撇(丿)" is indicated as "1", and the other 31 stroke types are all indicated as "0"; in the second line, only the stroke category of "捺(乀)" is indicated as "1", and the other 31 stroke types are all indicated as "0". Another example is the character "目". Since this character is composed of 5 strokes, it can be represented by 5 lines. Its stroke information can be: in the first line, only the stroke type of "竖(丨)" is indicated as "1", and the other 31 stroke types are all indicated as "0", and in the second line, only the stroke category of "横折 ” is indicated as "1", and the other 31 stroke types are all indicated as "0"; in the third to fifth lines, only the stroke category of "横(一)" is indicated as "1", and the other 31 stroke types are all indicated as "0".

[0220] FIG. 13 is a schematic block diagram of another way to generate beautified characters provided by the present application. The "Strokes Encoder" in FIG. 13 is a stroke encoding module. By introducing the stroke information corresponding to the standard character into the "decoder", the accuracy of the generated beautified character can be improved, and the problem of font collapse can be avoided. The functions of the other modules in FIG. 13 can be understood by referring to the introduction of FIG. 6 in Method 1 and will not be elaborated here. FIG. 14 is an effect diagram of the beautified character generated by the technical solution of Method 4 of the present application. The upper part of FIG. 14 is the effect diagram of the finally generated beautified character after introducing the stroke order, and the lower part of FIG. 14 is the effect diagram of the finally generated beautified character after introducing the stroke category.

[0221] It should be understood that the technical solution provided by Method 4 can also be understood as a model for beautifying handwritten characters. As shown in FIG. 11, the input of this model is the standard character, the handwritten character set and the target character set, as well as the stroke information of the standard character, and the output of the model is the beautified character.

[0222] In the present application, the solution provided by Method 4 can effectively alleviate the problem of font collapse in the trajectory of generating beautified characters, reduce the error rate of the generated beautified characters, and improve the generation quality of beautified characters.

[0223] It should be noted that in the present application, when beautifying handwritten characters by using the above-mentioned Method 1 to Method 4, the standard character input into the model can be a single standard character. In other words, in the process of beautifying handwritten characters by using the model, each handwritten character can be beautified separately.

[0224] In addition, considering that when the first gear is high, the proportion of target font styles corresponding to the target font gradually increases, the error rate of the generated beautified characters will become larger. This is because, under normal circumstances, the training data set consists of handwritten words written by many users on electronic whiteboards, which are usually sloppy and not neat enough, resulting in a high proportion of target words under the first gear. The "decoder" in the model is prone to errors in reconstructing the words, and re-collecting neat handwritten words requires a certain amount of manpower cost. In view of this, in order to expand the training data set of the model and make the model perform better, the application proposes that since the original output is beautified words of different gears, the output results of the original model can be used as its own input, so as to further train the original model and optimize the performance of the original model. The beautified words output by the original model are more accurate, and the generation quality of the beautified words is higher.

[0225] It should be understood that the data set in this application may be a public data set, which is used for model training of beautified characters. For example, the data set may further include a training data set and an inference data set. Among them, the training data set is used for model training, and the inference data set is used for model inference. For example, the training data set may include a training data set of standard characters, a training data set of handwritten characters, and a training data set of target characters. For example, the inference data set may include an inference data set of standard characters, an inference data set of handwritten characters, and an inference data set of target characters. In this application, the training data set and the inference data set are no longer distinguished, and are collectively referred to as training data sets.

[0226] Specifically, as shown in FIG15 , the original model can be divided into two training stages and two inference stages. The following introduces each training stage and inference stage of the original model respectively.

[0227] Step 1: Training Phase #A

[0228] Training phase #A can be understood as the initial training phase. During the initial training, the training data set of the original model can be obtained from a public training data set. Among them, the training data set of standard characters includes training standard characters, the training data set of handwritten characters includes training handwritten characters, and the training data set of target characters includes training target characters. Exemplarily, during the initial training, the training data set of handwritten characters can be used for training. For example, training standard characters #1 and training handwritten characters #1 can be input, and the training label is also training handwritten characters #1. Exemplarily, during the initial training, the training data set of target characters can also be used for training. For example, training standard characters #2 and training target characters #2 can be input, and the training label is also training target characters #2. In other words, the training goal of the model is to ensure that the generated characters expected to be output are consistent with the training handwritten characters input (that is, the style of the generated characters is consistent with the style of the training handwritten characters), that is, what is expected to be input can be output. For example, multiple iterative training can be performed to optimize the performance of the model. After the initial training, when the performance of the original model reaches the expected level (or when the performance of the original model no longer changes), the inference phase #A can be performed.

[0229] Step 2: Reasoning Phase #A

[0230] For example, in inference stage #A, the first gear is determined to be 40%. Training standard character #11 is input, the training handwritten character set includes training handwritten characters #11 to training handwritten characters #16, and the target handwritten character set includes training target characters #11 to training target characters #14. At this point, the model can output beautified character #1 corresponding to training standard character #11. For example, after multiple inferences, many sets of beautified characters can be obtained. In this application, inference stage #A can obtain a training dataset of beautified characters.

[0231] Step 3: Training Phase #B

[0232] After obtaining a training dataset of beautified characters, the model can be further trained based on this dataset to optimize model performance. This training process can also be considered a fine-tuning process for the model. The specific training process can be seen in Figure 16. Assume that in inference stage #A, the original model's inference result is beautified character #1. In this case, beautified character #1 can be used as input to further train the original model. For example, the original model can be trained using training standard character #21 and beautified character #1, with the trained label still being beautified character #1. The original model can be iteratively trained based on the multiple beautified characters obtained in inference stage #A, adjusting the original model's parameters to optimize model performance. For example, in training stage #B, the model parameters of the "decoder" can be primarily adjusted, while the model parameters of the previous "content extractor" and "style extractor" do not need to be adjusted. For example, during the training process, the model parameters of the decoder can be optimized through loss calculation until the desired model performance is achieved (or when the model performance no longer changes). Training can then be terminated, and the original model training is considered complete.

[0233] It should be noted that, in this embodiment, through multiple training stages #A and training stages #B, the parameters of the original model can be continuously optimized, so that the beautified characters output by the original model are more accurate and the quality of the generated beautified characters is higher.

[0234] Step 4: Reasoning Phase #B

[0235] The reasoning stage #B can be understood as the process of the user using the model. For example, based on the recognized image corresponding to the user's first handwriting, the standard character corresponding to the user's first handwriting is determined, and then based on the first gear determined by the user, the handwritten character set and the target character set are determined. The model outputs the first beautified character corresponding to the first handwriting based on the standard character, handwritten character set, and target character set corresponding to the input first handwriting.

[0236] The various technical solutions introduced above can be understood as specific implementation methods for the beautification module 330 to beautify handwritten characters. The following will introduce the rendering method of the beautified characters.

[0237] Normally, the pen stroke effect of handwritten characters is calculated based on parameters such as the pressure and speed of the stylus's pressure bar used by the user, but the beautified characters generated by the model do not have this information. Therefore, this embodiment proposes a method for calculating the pen stroke width based on the dispersion degree of the beautified character trajectory points.

[0238] According to the rules of writing habits, generally speaking, the writing speed of the stylus is faster when writing the straight part of the stroke, and the width of the trajectory of the beautified character is thinner; the speed slows down when writing the bend of the stroke, and the width of the trajectory of the beautified character is thicker. Therefore, the width of the pen tip of the beautified character can be inferred by the sparseness of the trajectory points of the beautified character. For example, when the trajectory sampling points of the beautified character are sparser, or the distance between the connected points is longer, it means that the writing speed of the pen is faster, that is, the width of the brush is thinner; on the contrary, when the trajectory sampling points of the beautified character are denser, or the distance between the connected points is longer, it means that the writing speed of the pen is slower, that is, the width of the brush is thicker. The main process is introduced below:

[0239] Step 1: Calculate the width of the pen tip for beautifying characters

[0240] (1) The trajectory points of the beautified characters are simplified by the Ramer-Douglas-Peucker (RDP) algorithm. For example, the sampling points are denser at the bends of the strokes and sparser at the straight parts of the strokes.

[0241] (2) Map the trajectory points of the simplified beautified characters to the writing pattern. That is, densely populated areas indicate slower writing speeds, and thus the trajectory point width is larger; sparsely populated areas indicate faster writing speeds, and thus the trajectory point width is smaller. Based on the dispersion of the trajectory points, the density distribution of the trajectory points can be obtained, thereby realizing the calculation of the stroke width of the beautified characters.

[0242] Step 2: Smooth the width of the pen strokes of the beautified characters

[0243] (3) In order to make the trajectory points of the simplified beautified characters smoothly connected, the trajectory points can be interpolated using a curve smoothing method, and the width of the interpolation points can be calculated accordingly according to the width of the stroke of the trajectory points;

[0244] (4) Perform sliding average on the width value to achieve a smooth transition of the pen tip width between each stroke.

[0245] It should be understood that, in a specific implementation, step 1 and step 2 are in an and / or relationship. For example, only step 1 or only step 2 may be executed; for another example, step 2 may be executed after step 1 is completed.

[0246] Figure 17 shows a flow chart of a method for calculating the stroke width of a beautified character and smoothing the stroke width, the method comprising: 1401, generating trajectory points of the beautified character; 1402, determining the strokes and sampling the strokes; 1403, when the number of sampling points is greater than 2, removing redundant points using the RDP algorithm; 1404, after removing the redundant points, if the number of sampling points is still greater than 4, calculating the stroke width based on the dispersion of the sampling points; 1405, after removing the redundant points, if the number of sampling points is less than 4, continuing to sample the original sampling points corresponding to the stroke, and then calculating the stroke width based on the dispersion of the sampling points; 1406, when the number of sampling points is less than 2, calculating the stroke width based on the dispersion of the sampling points. The above steps 1401 to 1406 can be understood as the process of calculating the stroke width of the beautified character, and the following steps 1407 to 1412 can be understood as the process of smoothing the stroke width of the beautified character. Step 1407, if the sampling points are greater than 4, Bezier interpolation can be used for interpolation; Step 1408, if the sampling points are less than 4, linear interpolation can be used for interpolation; Step 1409, calculate the pen tip width of the interpolation point; Step 1410, smooth the pen tip width; Step 1411, when it is determined to be the last stroke, the final pen tip width can be generated; Step 1412, when it is determined not to be the last stroke, the above steps 1402 to 1411 can be repeated.

[0247] The following introduces a method A for calculating the width of the pen tip for beautifying characters and a method B for smoothing the pen tip width.

[0248] Method A

[0249] When the number of stroke sampling points is greater than 2, calculate the sum of the distances between the current sampling point and the two previous and next sampling points, that is, calculate the distance between the i-th sampling point P i With the previous sampling point P i-1 The distance between i and the next sampling point P i+1 The distance between them is summed up. In order to solve the influence of the size of the word on the distance, the distance sum is normalized and divided by the width of the word to obtain the normalized distance sum D i When D i When it is larger, it means that the current sampling point is in a sparse area. i When it is smaller, it means that the current sampling point is in a denser area.

[0250] in,

[0251] Based on the normalized distance of the current sampling point and D i Calculate the current pen width W i, where base_W is the base pen width, Scale i is the stroke weighting coefficient, which can be realized based on the S-shaped curve. The function diagram is shown in Figure 18. The coefficient value Scale i With distance and D i The coefficient is negatively correlated and converges to 1 at both ends. When the distance and the sum increase, the coefficient value decreases and the pen tip width value becomes smaller. When the distance and the sum are too large or too small, the coefficient converges, which is conducive to a smooth transition of the pen tip width. i =1 / (1+exp(D i )),W i =Scale i *base_W. For the starting point 0 and the end point N of the stroke trajectory, the pen tip width values ​​are: W0=W1, W N =W N-1 ,When the number of stroke sampling points is 2, the stroke width of points P0 and P1 is calculated as follows: W1=1 / (1+exp(D1))*base_W, W0=W1, where W0 and W1 represent the pen tip widths at the starting point P0 and the end point P1 respectively.

[0252] Method B

[0253] In order to smooth the font trajectory, the trajectory points can be interpolated. As shown in Figure 19, the width of the interpolation point is calculated using the idea of ​​linear weighted summation. Assume that the pen tip width values ​​of the i-1th trajectory point and the i-th trajectory point are W respectively. i-1 and W i , the number of interpolation points between two points is M, the jth interpolation point I j Width value The calculation formula is: i,j =(1-k)*W i-1 +k*W i , where k and I j The location is related, that is, the interpolation point I j The ratio of the trajectory distance from the starting point to the trajectory distance between the i-1th trajectory point and the i-th trajectory point is calculated as: FIG. 20 shows the effect achieved after rendering the beautified characters based on the method of this embodiment.

[0254] The above technical solution can also be understood as that the rendering module 340 can determine the stroke width of the beautified character and / or smooth the stroke width of the beautified character; the rendering module 340 can render the beautified character according to the stroke width and / or the smoothed stroke width to generate the rendered beautified character.

[0255] The method for rendering beautified characters proposed in this application can achieve changes in the thickness of the pen strokes of the generated beautified characters, which is closer to the characteristics of users actually writing on an electronic whiteboard, thereby improving the user experience.

[0256] Considering the user experience, the size, aspect ratio, position, and orientation of the beautified characters should be basically consistent with the handwritten characters on the electronic whiteboard. Therefore, this embodiment mainly introduces a method for typesetting beautified characters. This embodiment mainly includes the following four steps:

[0257] Step 1: respectively obtain the vector dot matrix data of each handwritten character and the vector dot matrix data of each beautified character;

[0258] Step 2: Determine the format information of each handwritten word.

[0259] Step 3: Determine the format information of the beautified characters according to the format information of each handwritten character, so that the beautified characters are substantially consistent with the corresponding handwritten characters.

[0260] Step 4: Optimize the character spacing, line spacing, etc. of multiple beautified characters.

[0261] It should be noted that the vector dot matrix data of the handwritten characters in step one and the vector dot matrix data of the characters of each beautified character can be found in the beautified characters before rendering in each group of pictures in Figure 20. The following details how to determine the format information of each handwritten character in step two. The handwritten character format information in this application may include, for example, at least one of the following: the position of the handwritten character, the size of the handwritten character, the aspect ratio of the handwritten character, and the direction of the handwritten character. In order to determine the format information of the handwritten character, the position of the key points of a single handwritten character can be determined first. Similarly,

[0262] In this application, "key point" can refer to the center point of the character's "center of gravity", "center", etc., which is used to quantify the center coordinate position of the character. This application introduces a method for calculating the center based on the convex hull of a handwritten character, that is, generating the convex hull of the character based on the vector dot matrix of the character strokes. Taking the lowest point of the convex hull as the base point, divide it into several triangles, and then calculate the center of gravity of the convex hull. The calculated center of gravity is the coordinate position of the key point of the character. The specific calculation formula is as follows:

[0263] The abscissa of the centroid of the polygon = the abscissa of the centroid of each triangle of the Σ polygon decomposition × (area of ​​the triangle / total area of ​​the polygon);

[0264] The vertical coordinate of the centroid of the polygon = ∑ the vertical coordinate of the centroid of each triangle of the polygon decomposition × (area of ​​the triangle / total area of ​​the polygon) 0.

[0265] Exemplarily, assume that a convex hull is decomposed into four triangles, namely triangle #1, triangle #2, triangle #3, and triangle #4. Then, the abscissa Mx of the centroid of the polygon can be specifically expressed as: Mx = x1 * f1 + x2 * f2 + x3 * f3 + x4 * f4, where the coefficient f1 = s1 / Ms, s1 is the area of triangle #1, and Ms is the total area of the polygon; the coefficient f2 = s2 / Ms, s2 is the area of triangle #2, and Ms is the total area of the polygon; the coefficient f13 = s3 / Ms, s3 is the area of triangle #3, and Ms is the total area of the polygon; the coefficient f4 = s4 / Ms, s4 is the area of triangle #4, and Ms is the total area of the polygon. Similarly, for the ordinate of the centroid of the polygon, it can be understood by referring to the above example.

[0266] FIG. 21 shows a schematic diagram of calculating the centroid of a convex hull based on the above solution. (a) in FIG. 21 is a schematic diagram of generating a convex hull based on a character dot matrix, (b) in FIG. 21 is a schematic diagram of dividing the convex hull into multiple triangles, and (c) in FIG. 21 is a schematic diagram of calculating the centroid of the convex hull.

[0267] Since at least three coordinate points are required to generate a convex hull, for a straight-line character like the Chinese character "一" (which can be understood as having only two coordinate points), the convex hull may not need to be generated, and the center point of the line connecting the two points can be directly used as the key point of the character.

[0268] After determining the key point of the character, the size of each character and the aspect ratio of each character can be calculated. For example, the size and aspect ratio information of handwritten characters can be calculated based on the distance from the key point to the rectangular bounding box. As shown in FIG. 22, d l 、d r 、d t 、d b These four parameters are the size parameters of handwritten characters.

[0269] The following introduces a method for implementing step three to set the size and aspect ratio of the corresponding beautified characters according to the size and aspect ratio of handwritten characters. This method mainly includes the following four steps:

[0270] (1) Align the key points of the beautified characters and the key points of the original handwritten characters.

[0271] (2) Calculate the original handwritten character parameters: d l 、d r 、d t 、d b and the size parameters of the beautified characters: d' l 、d' r 、d' t 、d' bIt should be noted that the size parameter of the beautified word may be a default initial size preset when the beautified word is generated.

[0272] (3) Calculate the proportional relationship between the handwritten font size parameter and the default initial size parameter of the beautified font: S l =d l / d' l 、S r =d r / d' r 、S t =d t / d' t 、S b =d b / d' b .

[0273] (4) Calculate the minimum value among the four proportional parameters. If the minimum value exists in S l and S r In the horizontal direction, the beautified word is enlarged to min{S l ,S r}, the vertical magnification ratio is based on the dtw algorithm, that is, in (S t ,S b ) to find a value (usually using linear interpolation) in the range of , so that the beautified font has the highest similarity with the handwritten font. Similarly, if the minimum value exists in S t and S b In the vertical direction, the beautified word is enlarged to min{S t ,S b}, the horizontal magnification ratio is (S l ,S r ) range.

[0274] Figure 23 shows the effect of combining a single beautified character with the corresponding handwritten character using the above solution. Similarly, for multiple beautified characters, the effect of combining the key points with the key points of the handwritten character after adjusting the size and proportion of the multiple beautified characters is shown in Figure 24.

[0275] Based on the above technical solution, in this application, the size, position and other information of the handwritten characters can be retained, so that the handwritten characters will not be misplaced after beautification, or it can also be understood that the beautified characters and handwritten characters generated in this application can be well fitted.

[0276] In order to make the display effect of multiple beautified characters more neat and beautiful and improve the user experience, this application also proposes to optimize the character spacing, line spacing, etc. of multiple beautified characters. The following mainly introduces the typesetting method mentioned in step 4.

[0277] For example, multiple beautified characters in the same line can be aligned, with the alignment direction being to align the key points of the beautified characters along the beautified line. In addition to the "center of gravity" or "center" mentioned above, the key points of the beautified characters can also be the "highest point" or "lowest point" of the beautified characters, to respectively achieve the center alignment, top alignment, and bottom alignment effects of a single line of text. Taking the horizontal center of gravity alignment of the beautified characters as an example, the effect of the aligned beautified characters in the same line is shown in Figure 25.

[0278] Exemplarily, multiple beautified characters can also be evenly distributed in a single line, that is, the character spacing of adjacent beautified characters in the same line is equal. This method optimizes the typesetting effect of a single line of beautified characters, that is, the fonts in the same line of beautified characters will not be overly crowded or scattered. This example is to calculate the total spacing D1 between the first and last characters in the line, deduct the sum of the widths of other characters except the first and last characters, obtain the sum of the character spacing D2, and then divide D2 by the total number of characters in the single line N-1 (for example, N is 13 in Figure 26), calculate the character spacing D3 that each character should have after uniform distribution, and under the premise that the positions of the first and last characters remain unchanged, readjust the positions of other characters so that the distance between each character is D3. The effect of typeset the beautified characters in the same line by this method is shown in Figure 26.

[0279] For example, in addition to unifying the character spacing, the size of the beautified characters in a single line or multiple lines can be normalized based on the user's handwriting, so that the size of the beautified characters in a line or all lines is consistent. In addition, the minimum rectangular bounding box of each beautified character in a single line and the minimum rectangular bounding box of each line of handwriting across multiple lines can be unified, and each line of beautified characters can be aligned, such as left-aligned, centered, right-aligned, etc. The effect of left-aligned and unified character size and line spacing is shown in Figure 27.

[0280] For example, for multiple lines of handwritten characters containing subheadings, the subheadings of the same type can also be individually aligned based on the recognition results of the pictures corresponding to the handwritten characters. Among them, the specific judgment method for subheadings of the same type can be, for example, to judge whether the first or multiple texts starting from the first of each line of text in the recognition results contain special texts of the same type of title type, such as numbers 1, 2, 3; letters a, b, c; special symbols "·", etc., and align the lines of text with the same type of title, generally according to the side where the first word is located (for example, the left or right side). The basis for alignment can be the first title of the same type, or the average value of multiple titles can be taken. The effect of multi-line title alignment is shown in Figure 28.

[0281] For example, some punctuation marks cannot be aligned with the same text as the surrounding text. For example, when other text is aligned with the center of gravity, the quotation marks cannot be aligned with the center of gravity, but with the highest point of the adjacent text. Similarly, the alignment of the comma "," can be aligned with the lowest point of the adjacent text.

[0282] The above embodiment mainly introduces a method for typeset beautifying characters. The following embodiment specifically introduces a typeset algorithm for typeset beautifying characters. The typeset algorithm specifically includes the following steps:

[0283] Step 1: Obtain the character segmentation frame and line segmentation frame of the handwritten characters, and segment each line of handwritten characters to obtain segment frames.

[0284] Step 2: Calculate the size and key point position of each beautified character based on the paragraph box to perform typesetting.

[0285] The following first introduces a method for implementing step one. In this application, the word segmentation frame and line segmentation frame of the handwritten word can be obtained first. For example, the handwritten word recognition and segmentation module 320 can segment each handwritten word to obtain the word segmentation frame and line segmentation frame of the handwritten word. After obtaining the word segmentation frame and line segmentation frame of the handwritten word, the average width w of the word frame of a single line of words (the width w1 of the word frame of one handwritten word is shown in Figure 29) and the average spacing d between the word frames (the spacing width d1 between the word frames of two handwritten words is shown in Figure 28) can be calculated. If the distance di between two word frames exceeds the average value of the spacing between the first n word frames and is greater than s1*w and s2*d, (where n, s1 and s2 are all preset coefficients), it can be determined that a line of handwriting is divided into two sections of handwriting here (i.e., the two word frames), as shown in Figure 29.

[0286] For step two, the present application proposes a method for calculating the size and magnification ratio of beautified characters based on the segment frame. Due to the same type of target font, the font size of all characters is consistent, and each target character has a corresponding character frame. Figure 30 shows a schematic diagram of the character frame of a target font. Specifically, in each paragraph of handwriting, the width Wp and height Hp of the segment frame can be obtained first, and then the handwritten character recognition and segmentation module 320 can be used to identify the picture corresponding to the handwritten character, and the corresponding standard character can be extracted based on the recognition result, and the beautified character can be generated based on the first gear. Then, the sum of the widths w_s of the character frames of all beautified characters and the maximum character body frame height h_max are calculated. Then, the distance d1 from the left edge of the character body frame of the first character of each paragraph to the character and the distance d2 from the right edge of the character body frame of the last character of each paragraph to the changed character are calculated, as shown in Figure 31, and the magnification ratio s of the beautified character is min(Wp / (w_s-d1-d2)), Hp / h_max). After determining the magnification ratio for the beautified characters, the characters can be multiplied by s along with their character frames, and then sequentially arranged into their corresponding segment frames. The leftmost point of the first beautified character is aligned with the left edge of the segment frame, and the height centers of all character frames are aligned with the segment frame height. Based on this scheme, the beautified characters are shown in Figure 32.

[0287] It should be understood that the term "and / or" in this document simply describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0288] Those skilled in the art should be aware that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is performed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0289] In the embodiment of the present application, the electronic device can be divided into functional modules according to the above method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. The following is an example of dividing each functional module corresponding to each function.

[0290] FIG33 is a schematic diagram of a user interface for beautifying handwritten characters according to an embodiment of the present application. As shown in FIG33 , when beautifying handwritten characters, the user can adjust the format of the beautified characters. For example, the user can select to adjust multiple beautified characters, or select to adjust multiple beautified characters. For another example, the user can select Songti as the target font. For another example, the user can select the first gear. In the center of the interface is a preview area, which displays the beautified characters generated based on the user's settings.

[0291] FIG34 is a schematic block diagram of an electronic device 1000 according to an embodiment of the present application. As shown in FIG34 , the electronic device 1000 may include: a processing module 1010 and an acquisition module 1020 .

[0292] The above modules are used to execute the steps in the above-mentioned method 500 for beautifying handwriting, which will not be described in detail here.

[0293] It should also be understood that the electronic device 1000 herein is embodied in the form of functional units. The term "unit" herein may refer to an application specific integrated circuit (ASIC), an electronic circuit, a processor (e.g., a shared processor, a dedicated processor, or a group processor, etc.) and memory for executing one or more software or firmware programs, combined logic circuits, and / or other suitable components that support the described functionality.

[0294] The electronic device 1000 of each of the above-described embodiments has the function of implementing the corresponding steps of the above-described method 500. The functions can be implemented by hardware, or by hardware executing corresponding software implementations. The hardware or software includes one or more modules corresponding to the above-described functions; for example, the determination unit and the like can be replaced by a processor to respectively perform the transceiver operations and related processing operations in each method embodiment. In addition, the determination unit can be a processing circuit.

[0295] It should be noted that the electronic device in FIG34 can be the electronic device in the aforementioned method embodiment, or it can be a chip or chip system of the electronic device, such as a system on chip (SoC). The processing unit is a processor, microprocessor, or integrated circuit integrated on the chip. This is not limited here.

[0296] Figure 35 is a schematic block diagram of another electronic device 2000 provided in an embodiment of the present application. As shown in the figure, the device 2000 includes: at least one processor 2010. The processor 2010 is coupled to a memory and is configured to execute instructions stored in the memory to send and / or receive signals. Optionally, the device 2000 also includes a memory 2030 for storing instructions. Optionally, the device 2000 also includes a transceiver 2020, and the processor 2010 controls the transceiver 2020 to send and / or receive signals.

[0297] It should be understood that the processor 2010 and memory 2030 may be combined into one processing device, and the processor 2010 is configured to execute program codes stored in the memory 2030 to implement the above functions. In specific implementations, the memory 2030 may also be integrated into the processor 2010 or independent of the processor 2010.

[0298] It should also be understood that the transceiver 2020 may include a transceiver (or receiver) and a transmitter (or transmitter). The transceiver may further include an antenna, and the number of antennas may be one or more. The transceiver 2020 may also be a communication interface or interface circuit.

[0299] Specifically, the processor 2010 in the device 2000 may correspond to the processing unit 1010 and the acquisition module 1020 in the electronic device 1000 .

[0300] As a solution, the electronic device 2000 is used to implement each step in the above method 500 embodiment.

[0301] For example, the processor 2010 is configured to execute computer programs or instructions stored in the memory 2030 to implement the various steps in the above method 500 .

[0302] According to the method provided in the embodiment of the present application, the present application also provides a computer program product, which stores computer program code. When the computer program code runs on a computer, the computer executes each step in the embodiment of method 500.

[0303] According to the method provided in the embodiment of the present application, the present application also provides a computer-readable medium, which stores program code. When the program code is run on a computer, the computer executes each step in the above method 500.

[0304] The explanation of the relevant contents and beneficial effects of any of the above-mentioned devices can be referred to the corresponding method embodiments provided above, and will not be repeated here.

[0305] A computer-readable medium stores a program code, and when the program code is executed on a computer, the computer is caused to execute each step in the above method 500.

[0306] The explanation of the relevant contents and beneficial effects of any of the above-mentioned devices can be referred to the corresponding method embodiments provided above, and will not be repeated here.

[0307] It should be understood that the specific process of each transceiver and processor executing the above corresponding steps has been described in detail in the above method embodiment, and for the sake of brevity, it will not be repeated here.

[0308] During implementation, each step of the above method can be completed by an integrated logic circuit of the hardware in the processor or by instructions in the form of software. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it will not be described in detail here.

[0309] It should be noted that the processor in the embodiments of the present application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above-mentioned method embodiment can be completed by hardware integrated logic circuits in the processor or by software instructions. The above-mentioned processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above-mentioned method.

[0310] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disc (SSD)).

[0311] In each of the above-mentioned device embodiments, the corresponding modules or units perform the corresponding steps. For example, the transceiver unit (transceiver) performs the receiving or sending steps in the method embodiments, and other steps except sending and receiving can be performed by the processing unit (processor). The functions of the specific units can be referred to in the corresponding method embodiments. There can be one or more processors.

[0312] The terms "component", "module", "system", etc. used in this specification are used to represent computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program and / or a computer. By way of illustration, both applications running on an electronic device and an electronic device can be components. One or more components can reside in a process and / or an execution thread, and a component can be located on a computer and / or distributed between two or more computers. In addition, these components can be executed from various computer-readable media having various data structures stored thereon. Components can communicate, for example, through local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system and / or a network, such as the Internet interacting with other systems via signals).

[0313] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0314] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0315] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0316] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0317] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0318] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0319] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for beautifying handwriting, characterized in that: include: Acquire information of a first handwritten word, where the first handwritten word is a word written by a user in a handwriting area of ​​an electronic device; detecting a first input operation of the user, and acquiring information of a first gear position in response to the first input operation; Determine a fusion style according to the information of the first gear, a handwriting font library, and a target font, wherein the handwriting font library includes characters written by the user on a handwriting area of ​​the electronic device and stored by the electronic device, the target font is a font preset in the target font library of the electronic device, the fusion style is used to characterize a style after the target font and a handwriting font corresponding to the characters included in the handwriting font library are fused, and the first gear is used to determine a ratio between a target font style corresponding to the target font and a handwriting font style corresponding to the handwriting font in the fusion style; A first beautified character is generated, where the first beautified character is the first handwritten character beautified according to the information of the first handwritten character and the fusion style.

2. The method according to claim 1, characterized in that The target font library of the electronic device includes at least two fonts, and the method further includes: receiving a second instruction, where the second instruction is used to indicate the target font, wherein the target font is one of the at least two fonts; According to the second instruction, the target font is determined.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: receiving a third instruction, the third instruction being used to indicate a second handwritten word, the second handwritten word being a word written by the user in the handwriting area, and the second handwritten word being used to generate the handwritten word library; According to the third instruction, the handwritten character library is generated, and the handwritten character library includes one or more second handwritten characters.

4. The method according to claim 3, characterized in that The method further comprises: receiving a fourth instruction, wherein the fourth instruction is used to instruct to update the second handwritten character in the handwritten character library; According to the fourth instruction, the updated handwritten character library is generated, and the updated handwritten character library includes one or more updated second handwritten characters.

5. The method according to claim 3 or 4, characterized in that: The determining of the fusion style according to the information of the first gear, the handwriting font library, and the target font includes: Determine, according to the information of the first gear, a first number of the second handwritten characters and a second number of target characters corresponding to the target font, wherein the first number and the second number are determined according to the information of the first gear; The fusion style is determined according to the first number of the second handwritten characters and the second number of the target characters.

6. The method according to claim 5, characterized in that The method further comprises: Determining a first ratio according to information of the first gear, where the first ratio is a ratio between the second quantity and the first quantity; The first quantity and the second quantity are determined according to the first ratio.

7. The method according to any one of claims 1 to 6, characterized in that The generating of the first beautified word comprises: Determining a standard character corresponding to the first handwritten character according to information of the first handwritten character, wherein the standard character is a character preset in a standard character library of the electronic device; The first beautified characters are generated according to the standard characters and the fusion style.

8. The method according to claim 7, characterized in that The step of generating the first beautified character according to the standard character and the fusion style includes: Extracting content features corresponding to the standard word, wherein the content features are used to determine the standard word; The first beautified characters are generated according to the content features and the fusion style.

9. The method according to claim 8, characterized in that The method further comprises: Determining stroke information of the standard character, wherein the stroke information includes a stroke order of the standard character and / or a stroke category of the standard character; The step of generating the first beautified characters according to the content feature and the fusion style includes: The first beautified character is generated according to the content feature, the fusion style, and the stroke information of the standard character.

10. The method according to any one of claims 7 to 9, characterized in that The step of generating the first beautified character according to the standard character and the fusion style includes: The first beautified characters are generated according to the standard characters, the fusion style, and the first model, wherein the training data set of the first model includes at least one of the following: a training data set of handwritten characters, a training data set of target characters, and a training data set of beautified characters, and the training data set of beautified characters includes training beautified characters of different first gears.

11. The method according to any one of claims 1 to 10, characterized in that The method further comprises: Determining the stroke width of the first beautified character, and / or smoothing the stroke width of the first beautified character; The first beautified character is rendered according to the pen stroke width and / or the smoothed pen stroke width to generate the rendered first beautified character.

12. The method according to any one of claims 1 to 11, characterized in that The method further comprises: Determine format information of the first handwritten character, where the format information of the first handwritten character includes at least one of the following: a position of the first handwritten character, a size of the first handwritten character, a length-to-width ratio of the first handwritten character, and a direction of the first handwritten character; Determining format information of the first beautified character according to the format information of the first handwritten character, wherein the format information of the first beautified character includes at least one of the following: a position of the first beautified character, a size of the first beautified character, a length-to-width ratio of the first beautified character, and a direction of the first beautified character; According to the format information of the first beautified word, single-word adjustment is performed on the first beautified word.

13. The method according to claim 12, characterized in that The method further comprises: Perform multi-word adjustment on the plurality of the first beautified characters, wherein the multi-word adjustment includes at least one of the following: adjusting the character spacing between adjacent first beautified characters; and / or, aligning the plurality of the first beautified characters in a row; and / or, adjusting the size of all the first beautified characters in one or more rows.

14. The method according to any one of claims 1 to 13, characterized in that The information of the first handwritten character includes an image corresponding to the first handwritten character and / or a trajectory corresponding to the first handwritten character.

15. An electronic device, characterized in that: include: Acquisition module, processing module, The acquisition module is used to acquire information of a first handwritten word, where the first handwritten word is a word written by a user in a handwriting area of ​​an electronic device; The acquisition module is used to detect a first input operation of the user, and acquire information of the first gear position in response to the first input operation; The processing module is used to determine a fusion style according to the information of the first gear, a handwriting font library, and a target font, wherein the handwriting font library includes characters written by the user in a handwriting area of ​​the electronic device and stored by the electronic device, the target font is a font preset in the target font library of the electronic device, the fusion style is used to characterize a style after the target font and a handwriting font corresponding to the characters included in the handwriting font library are fused, and the first gear is used to determine a ratio between a target font style corresponding to the target font and a handwriting font style corresponding to the handwriting font in the fusion style; The processing module is used to generate a first beautified character, where the first beautified character is the first handwritten character beautified according to the information of the first handwritten character and the fusion style.

16. The electronic device according to claim 15, characterized in that: The fonts pre-set in the target font library of the electronic device include at least two fonts, The acquisition module is used to receive a second instruction, where the second instruction is used to indicate the target font, wherein the target font is one of the at least two fonts; The processing module is used for determining the target font according to the second instruction.

17. The electronic device according to claim 15 or 16, characterized in that: The acquisition module is used to receive a third instruction, the third instruction is used to indicate a second handwritten word, the second handwritten word is a word written by the user in the handwriting area, and the second handwritten word is used to generate the handwritten word library; The processing module is used to generate the handwritten character library according to the third instruction, and the handwritten character library includes one or more second handwritten characters.

18. The electronic device according to claim 17, characterized in that: The acquisition module is used to receive a fourth instruction, where the fourth instruction is used to instruct to update the second handwritten character in the handwritten character library; The processing module is used for generating the updated handwritten character library according to the fourth instruction, wherein the updated handwritten character library includes one or more updated second handwritten characters.

19. The electronic device according to claim 17 or 18, characterized in that: The processing module is used to determine the fusion style according to the information of the first gear, the handwriting font library, and the target font, including: The processing module is used to determine, according to the information of the first gear, a first number of the second handwritten characters and a second number of target characters corresponding to the target font, wherein the first number and the second number are determined according to the information of the first gear; The processing module is used to determine the fusion style based on a first number of the second handwritten characters and a second number of target characters corresponding to the target font.

20. The electronic device according to claim 19, characterized in that: The processing module is used to determine a first ratio according to the information of the first gear position, where the first ratio is a ratio between the second number and the first number; The processing module is used to determine the first quantity and the second quantity according to the first ratio.

21. The electronic device according to any one of claims 15 to 20, characterized in that: The processing module is used to generate a first beautified word, including: The processing module is used to determine, based on the information of the first handwritten character, a standard character corresponding to the first handwritten character, wherein the standard character is a character preset in a standard character library of the electronic device; The processing module is used to generate the first beautified character according to the standard character and the fusion style.

22. The electronic device according to claim 21, characterized in that: The processing module is used to generate the first beautified character according to the standard character and the fusion style, including: The processing module is used to extract content features corresponding to the standard word, and the content features are used to determine the standard word; The processing module is used to generate the first beautified word according to the content feature and the fusion style.

23. The electronic device according to claim 22, characterized in that: The processing module is further used to determine the stroke information of the standard character, wherein the stroke information includes the stroke order of the standard character and / or the stroke category of the standard character; The processing module is used to generate the first beautified word according to the content feature and the fusion style, including: The processing module is used to generate the first beautified character according to the content feature and the fusion style, and the stroke information of the standard character.

24. The electronic device according to any one of claims 21 to 23, characterized in that: The processing module is used to generate the first beautified character according to the standard character and the fusion style, including: The processing unit is used to generate the first beautified characters according to the standard characters, the fusion style, and the first model, wherein the training data set of the first model includes at least one of the following: a training data set of handwritten characters, a training data set of target characters, and a training data set of beautified characters, and the training data set of beautified characters includes training beautified characters of different first gears.

25. The electronic device according to any one of claims 15 to 24, characterized in that: The processing module is used to determine the stroke width of the first beautified character, and / or to perform smoothing processing on the stroke width of the first beautified character; The processing module is used for rendering the first beautified character according to the pen stroke width and / or the smoothed pen stroke width to generate the rendered first beautified character.

26. The electronic device according to any one of claims 15 to 25, characterized in that: The processing module is used to determine the format information of the first handwritten character, wherein the format information of the first handwritten character includes at least one of the following: the position of the first handwritten character, the size of the first handwritten character, the aspect ratio of the first handwritten character, and the direction of the first handwritten character; The processing module is used to determine the format information of the first beautified character according to the format information of the first handwritten character, wherein the format information of the first beautified character includes at least one of the following: the position of the first beautified character, the size of the first beautified character, the aspect ratio of the first beautified character, and the direction of the first beautified character; The processing module is used for performing single-word adjustment on the first beautified word according to the format information of the first beautified word.

27. The electronic device according to claim 26, characterized in that: The processing module is also used to perform multi-word adjustment on multiple first beautified words, and the multi-word adjustment includes at least one of the following: adjusting the word spacing between adjacent first beautified words; and / or aligning multiple first beautified words in rows; and / or adjusting the size of all the beautified words in one or more rows.

28. The electronic device according to any one of claims 15 to 27, characterized in that: The information of the first handwritten character includes an image corresponding to the first handwritten character and / or a trajectory corresponding to the first handwritten character.

29. An electronic device, characterized in that: The electronic device comprises a processor and a memory, wherein the memory is used to store a computer program or instructions, and the processor is used to execute the computer program or instructions in the memory, so that the method according to any one of claims 1 to 14 is executed.

30. A chip system, characterized in that: It includes a processor, which is used to call and run a computer program from a memory, so that a terminal device equipped with the chip system executes the method as claimed in any one of claims 1 to 14.

31. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 14.

32. A computer program product, characterized in that The computer program product comprises means for executing the method according to any one of claims 1 to 14.

Images