Electronic calligraphy style editing method based on action information, electronic equipment and storage medium

Abstract

The invention discloses an electronic calligraphy style editing method based on action information, electronic equipment and a storage medium, the action information of the electronic calligraphy is defined, the sampled action information of the electronic calligraphy is obtained from hardware input equipment or a data source of the electronic calligraphy, a stroke model is input, a calligraphy character image is incrementally drawn, and the calligraphy character is edited. And modifying and adjusting the action information sequence for forming the calligraphy character image to form a new style calligraphy character image. According to the invention, the calligraphy character style is edited based on the writing action information, so that after the style is edited, the writing continuity is still kept, the traction belt between the strokes can be automatically adjusted, and the later image processing is not needed for the cursive script. The calligraphy character style is edited based on the writing action information, so that the writing dynamic state is still kept after the calligraphy character style is changed. By adjusting the style editing parameters, the effects of interpolation transition and dynamic adjustment between calligraphy styles can be achieved.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision and computer graphics, and particularly relates to a method for editing the style of electronic calligraphy based on action information, an electronic device, and a storage medium. Background Art

[0002] In the field of computer calligraphy, calligraphy style adjustment and dynamic calligraphy writing generation are important technical directions and have great application value.

[0003] Existing methods for calligraphy style adjustment focus on the image data processing of calligraphy characters. The mainstream is to use deep generation models to generate calligraphy character images of specific styles. For example, a method for generating calligraphy font images based on a diffusion model disclosed in a patent application with the publication number CN118691460A proposes a method for generating high-quality calligraphy font images of a specific calligrapher style through style feature fusion and classifier-free guidance training. The calligraphy style adjustment method based on calligraphy images is limited by the image format and cannot further represent the writing dynamics of a specific style.

[0004] Since calligraphy images do not contain dynamic writing information, some researchers have tried to extract the dynamic writing process from calligraphy character images. In the paper "Model-Based Analysis of Chinese Calligraphy Images" [S.T.S.Wong, H.Leung and H.H.S.Ip, "Model-based analysis of Chinese calligraphy images," Ninth International Conference on Information Visualisation (IV'05), London, UK, 2005, pp.221-226, doi:10.1109 / IV.2005.81.], the authors proposed a method for animating the writing process based on the analysis of calligraphy character images. This technology has certain limitations. For example, the reproduced writing process is a uniform process and cannot represent the rhythm of the calligraphy creation process; the stroke segmentation based on calligraphy character images requires manual processing, which is relatively cumbersome.

[0005] There is no calligraphy style editing method based on calligraphy action information that can maintain the writing dynamics while realizing calligraphy style editing. Summary of the Invention

[0006] In view of the above, the present invention provides a method for editing the style of electronic calligraphy based on action information, an electronic device, and a storage medium, which define the action information of electronic calligraphy, obtain the sampled action information of electronic calligraphy from the hardware input device or data source of electronic calligraphy, input it into a stroke model, incrementally draw a calligraphy character image, and modify and adjust the action information sequence forming the calligraphy character image to form a calligraphy character image with a new style.

[0007] To achieve the above object of the invention, the present invention provides a method for editing the style of electronic calligraphy based on action information, including the following steps:

[0008] Obtain an input event from the hardware input device of electronic calligraphy, and then store the input event in the form of action information of electronic calligraphy to obtain the sampled action information of electronic calligraphy; or, obtain the sampled action information of electronic calligraphy from a data source;

[0009] Preferably, storing the input event in the form of action information of electronic calligraphy includes:

[0010] Define the form of the action information of the electronic calligraphy as operation{x,y,operation_type i ,operation_params i}, where x and y are used to indicate the coordinates of the action, operation_type i indicates whether the action type is in progress, and operation_params i are the parameters of the action type, and the action types include pressing down and lifting.

[0011] Input the sampled action information of electronic calligraphy into a stroke model for drawing to obtain the drawing parameters and drawing identifier for the current step;

[0012] Preferably, inputting the sampled action information of electronic calligraphy into a stroke model for drawing includes:

[0013] The stroke model can be a physics-based, experience-based, or data-driven model. The stroke model receives the input of action information, establishes a stroke state transition function based on the action information, and realizes the temporal evolution of the stroke state. Among them, the stroke state is a series of parameters used to describe the stroke change stage. According to the stroke state, the drawing parameters and drawing identifier are obtained. The drawing identifier is used to indicate whether to perform the drawing of the stroke in the current step, and the drawing parameters are used to obtain the data for drawing the stroke image. The stroke state transition function can be expressed as f(operation i ,state i-1 ) = state i . state i-1is the initial state or the previous state of the stroke, operation i is the action information of the current step, state i is the stroke state obtained from the input of the current step.

[0014] Judge whether to perform drawing according to the drawing identifier of the current step. If drawing is performed, input the drawing parameters of the current step into the shader to draw the stroke image of the current step;

[0015] Obtain a new input event from the hardware input device of electronic calligraphy, store the input event in the form of action information of electronic calligraphy to obtain new action information, input it into the stroke model for drawing to obtain new drawing parameters and a new drawing identifier, and judge whether to perform incremental drawing of the stroke image according to the drawing identifier, and form a calligraphy character image img through incremental drawing;

[0016] Or, continue to sample from the data source to obtain new action information, repeat to obtain new action information, input it into the stroke model for drawing, obtain new drawing parameters and a new drawing identifier, and judge whether to perform incremental drawing of the stroke image according to the drawing identifier, and repeat the incremental drawing of the stroke image to obtain a calligraphy character image img;

[0017] For the action information sequence forming the calligraphy character image img, after modifying the action information in the action information sequence, form an edited action information sequence, and form a calligraphy character image img in a new style according to the edited action information sequence edited Or form a calligraphy character image img in a new style according to the action information sequence that has been smoothed and transitioned after editing edited ;

[0018] Preferably, the action information sequence forming the calligraphy character image img includes: the data representation of the entire process of electronic calligraphy creation, and the action information sequence is represented as operations[operation 0 , operation 1 , …, operation n-1 . operation 0 represents the first action information of the action information sequence, operation 1 represents the second action information of the action information sequence, operation n-1 represents the last action information of the action information sequence, where n is the length of the action information sequence.

[0019] The modification of the action information in the action information sequence includes: replacement and / or deletion and / or insertion of the action information in the action information sequence.

[0020] Preferably, a calligraphy character image of a new style is formed according to the edited action information sequence or a calligraphy character image of a new style is formed according to the edited action information sequence with smooth transition, including:

[0021] The edited action information sequence is operations edited , for the action information before and after the edited part in the sequence operations edited , specify the forward transition window length len 1 and the backward transition window length len 2 , respectively perform smooth transition correction on the action information of the two parts where the subscript of operations edited ranges from i - len 1 to i - 1 and the subscript ranges from i + k + 1 to i + k + len 2 , where the range of the subscript of the edited part of the action information sequence before smooth transition correction is from i to i + k. By completing the action information sequence with smooth transition after editing, a calligraphy character image of a new style is formed;

[0022] The smooth transition method includes using or combining methods such as linear interpolation, quadratic interpolation, cubic Bezier interpolation, high - order interpolation, spline interpolation, Lagrange interpolation, and Kalman filtering to smooth the coordinates of the action information or the parameter values of the action types.

[0023] Preferably, forming a calligraphy character image of a new style according to the edited action information sequence with smooth transition specifically includes:

[0024] The edited and smoothly - transitioned action information sequence forms a calligraphy character image of a new style through calligraphy rewriting. First, set the stroke model to the starting state and use the immediate rewriting or demonstration rewriting method for calligraphy redrawing. For demonstration rewriting, read the action information in the action information sequence in order at the specified frequency to obtain new action information, input it into the stroke model for drawing, obtain new drawing parameters and new drawing identifiers, and determine whether to perform incremental drawing of the stroke image according to the drawing identifier, and repeat the incremental drawing of the stroke image to obtain the calligraphy character image. The difference between immediate rewriting and demonstration rewriting is that immediate rewriting does not follow the specified frequency but processes all the action information in the action information sequence at once and immediately draws all the strokes that make up the stroke.

[0025] The stroke model used for calligraphy rewriting is not limited, that is, the stroke model can be replaced. Replacing the stroke model means changing the numerical value of the stroke state or changing the stroke state transition function f(operation i , state i-1 ) = state i, such that there exists an action information sequence. After inputting the same action information, the drawing parameters and / or drawing identifiers obtained after the update of the stroke state are different from those before the stroke replacement, where state i-1 is the stroke state of the previous step, operation i is the action information of the current step, state i is the stroke state of the current step.

[0026] The present invention proposes to store and edit electronic calligraphy in a new form of action information. The calligraphy image or calligraphy writing video can be re-obtained from the action information. The calligraphy style editing based on the action information enables the preservation of the coherence and rhythm of writing after the calligraphy style editing, solves the problem that the image-based calligraphy style adjustment method does not support writing reproduction, and the connected strokes of running script and cursive script can be automatically adjusted after style editing without additional image processing; the interpolation transition of the calligraphy style can also be achieved by adjusting the editing parameters. The present invention has application value in calligraphy video content creation and calligraphy education, and can promote the digital dissemination and development of calligraphy.

[0027] An electronic device, comprising:

[0028] One or more processors;

[0029] A memory for storing one or more programs;

[0030] When the one or more programs are executed by the one or more processors, the one or more processors implement the style editing method of the electronic calligraphy based on action information.

[0031] A computer-readable storage medium, on which a program is stored. When the program is executed by a processor, the style editing method of the electronic calligraphy based on action information is implemented.

[0032] Compared with the prior art, the present invention has the following advantages:

[0033] The present invention edits the calligraphy character style based on the writing action information, so that after the style editing, the coherence of writing is still maintained, and the connected strokes between the strokes can be automatically adjusted. For running script and cursive script, no post-processing of images is required.

[0034] The present invention edits the calligraphy character style based on the writing action information, so that after the change of the calligraphy character style, the dynamics of writing are still retained.

[0035] The present invention can achieve the effects of interpolation transition and dynamic adjustment between calligraphy styles by adjusting the style editing parameters. Description of the Drawings

[0036] Figure 1Schematic diagram for the creation of the method for editing the style of electronic calligraphy based on motion information according to the present invention;

[0037] Figure 2 Schematic diagram for the editing and rewriting of the method for editing the style of electronic calligraphy based on motion information according to the present invention;

[0038] Figure 3 Calligraphy character image before style editing of the method for editing the style of electronic calligraphy based on motion information according to the present invention;

[0039] Figure 4 Calligraphy character image after style editing of the method for editing the style of electronic calligraphy based on motion information according to the present invention;

[0040] Figure 5 Screenshot of the dynamic video of calligraphy character writing before style editing of the method for editing the style of electronic calligraphy based on motion information according to the present invention;

[0041] Figure 6 Screenshot of the dynamic video of calligraphy character writing after style editing of the method for editing the style of electronic calligraphy based on motion information according to the present invention;

[0042] Figure 7 Interpolation transition diagram of calligraphy style for the method for editing the style of electronic calligraphy based on motion information according to the present invention. Detailed implementation manners

[0043] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific implementation manners described herein are only used to explain the present invention and do not limit the protection scope of the present invention.

[0044] As Figure 1 shown, in the present invention, the motion information is first represented as a structure struct Operation{float x,float y,bool is_pressed,float press_intensity,bool is_lifted,float lift_intensity} in C++, where x and y represent the coordinate positions of the motion, is_pressed represents whether the writing brush is pressed, press_intensity represents the degree of pressing, is_lifted represents whether the writing brush is lifted, and lift_intensity represents the degree of lifting.

[0045] The present invention uses a mouse as an input device, and represents the input signal of the device as a structure struct Input {float pos_x, float pos_y, bool is_mouse_left_pressed, bool is_mouse_right_pressed}.

[0046] The input signal is acquired at a frequency of 144 hz. In each signal acquisition, the screen coordinates float pos_x and float pos_y of the mouse, whether the left mouse button is pressed bool is_mouse_left_pressed, and whether the right mouse button is pressed bool is_mouse_right_pressed are acquired; according to the obtained input signal structure, an action encoding rule is set, and the input signal information is encoded into action information. First, the coordinates are transformed in the coordinate system, and the input screen coordinates are transformed into canvas coordinates to obtain x and y in Operation. If is_mouse_left_pressed is true, set is_pressed in the action information to true and set press_intensity to 0.1, otherwise set is_pressed to false and set press_intensity to 0; if is_mouse_right_pressed is true, set is_lifted to true and set lift_intensity to 0.1, otherwise set is_lifted to false and set lift_intensity to 0. Finally, the action information Operation for this acquisition is obtained new 。

[0047] In an embodiment, the present invention designs and implements an experience-based stroke model. The state of the stroke model is represented by a structure, which is represented as struct StrokeUnitStatus {float control_pos_x, float control_pos_y, float pos_x, float pos_y, bool is_touching_paper, float radius}, where control_pos_x and control_pos_y are the canvas coordinates of the control unit, pos_x and pos_y are the canvas coordinates of the stroke unit, is_touching_paper indicates whether the stroke unit is in contact with the paper surface, and radius is the shape size of the stroke model. Among them, pos_x, pos_y, and radius are drawing parameters, and is_touching_paper is a drawing identifier.

[0048] When a new input information is entered, if it is the first input of action information, then this input information is stored as the previous input information when updating the pen stroke state in the next frame, and the update ends. According to the new input Operation new action information and the Operation previous information, the coordinate movement amount [dx, dy] is obtained from the difference in the coordinate positions.

[0049] In this embodiment, the rule for the pen stroke model to receive action information and perform state update is designed as follows:

[0050] The pen stroke movement update behavior updates the canvas coordinates of the pen stroke unit to [pos_x + a * dx, pos_y + a * dy], where a is the pen stroke unit movement speed factor, set to 0.4;

[0051] The control unit movement behavior updates the canvas coordinates of the control unit to [control_pos_x + c * dx, control_pos_y + c * dy], where c is the control unit movement speed factor, set to 1.0;

[0052] The pen stroke unit follows the control unit to update the canvas coordinates of the pen stroke unit to [pos_x + b * (control_pos_x – pos_x), pos_y + b * (control_pos_y – pos_x)], where b is the speed factor for the pen stroke unit to follow the control unit, set to 0.7;

[0053] The pen stroke expansion behavior updates radius = radius * (1 + press_intensity);

[0054] The pen stroke shrinkage behavior updates radius = radius * (1 - lift_intensity);

[0055] If dx and dy are not both equal to 0, execute the control unit movement behavior;

[0056] Execute the control unit movement behavior, the pen stroke unit movement behavior, and the pen stroke unit following the control unit behavior in sequence; if is_pressed in the action information is true and the radius radius of the pen stroke is less than the maximum scaling degree 1.5, execute the pen stroke expansion behavior; if is_lifted in the action information is true and the pen stroke size radius is greater than the minimum size 0.5, execute the pen stroke shrinkage behavior.

[0057] After completing all the stroke state update operations for the current step, if the current step does not perform the stroke expansion operation and the distance between the coordinates of the stroke unit and the control unit is less than the pen-lifting limit of 4.0, set the paper-touching state is_touching_paper of the stroke unit to false;

[0058] If the current step performs the stroke expansion operation, then set the paper-touching state is_touching_paper of the stroke to true;

[0059] Through the above steps, obtain the structure StrokeUnitStatus of the new stroke state new 。

[0060] Obtain is_touching_paper from StrokeUnitStatus new as the drawing flag. If is_touching_paper is true, construct an array of vertex coordinates for a circle formed by triangular fans as the basic primitive, bind the array of vertex coordinates to the vertex buffer object (VBO), configure the vertex attribute pointer, obtain the stroke model position and radius as the drawing parameters from StrokeUnitStatus new as uniform variables and input them into the shader. Use the glDrawArrays method to perform incremental rendering in the GL_TRIANGLE_FAN mode, update the vertex buffer data during each drawing, capture the rendering result through the frame buffer object (FBO), finally generate the calligraphy character image img, and obtain all the captured action information sequences std::vector <operation>OperationSequence, save it in the format of a text document and name it OperationSequence.txt.

[0061] As Figure 2 shown, read the saved text document file OperationSequence.txt to obtain the content of the action information sequence OperationSequence. Edit the j-th to j+k-th items of OperationSequence, where j and k are set by the user through the system interface to specify the subscript range of the action information sequence to be edited. The editing method is as follows:

[0062] The user sets editing parameters through the system interface, including the adjustment intensity Intensity and the frequency parameter f. The value range of Intensity is from 0 to 0.1, and the value range of f is from 0.02 to 0.2. The data types of Intensity and f are both float.

[0063] Set the float type variable t to 0 to control the phase. Traverse the action information of the action information sequence OperationSequence in order. For each traversed action information, calculate the float type variables dx and dy. dx is obtained by calculating Intensity*(cos(f*t)*sin(f*t)), and dy is obtained by calculating Intensity*(cos(f*t))*0.7. Update the abscissa information x of the traversed action information to x+dx*300, and update the ordinate information y of the action information to y+dy*300. For each updated action information, t is increased by 0.02. Complete the update of the action information from the j-th item to the j+k-th item, and end the editing process.

[0064] After editing, set both the forward transition window length and the backward transition window length to n, and use the method of linear interpolation to smoothly transition the coordinates of the action information for the items from j-n to j-1 in OperationSequence:

[0065] Take the canvas coordinate information x j and y j of the j-th action information Operation j in the edited OperationSequence. For each action information from j-n to j-1, traverse in ascending order of the sequence subscript. For the k-th action information Operation k traversed, for its canvas coordinate information x k and y k , update the ordinate y k to y k +(y j-1 -y j )*(k - 1) / n, update the abscissa x k is x k +(x j-1 -x j )*(k - 1) / n.

[0066] Use the same linear interpolation method to perform smooth transition processing on the coordinate values of the action information for the action information from the (j + k + 1)-th to the (j + k + n)-th items in OperationSequence. In this embodiment, finally obtain the edited and smoothly transition-adjusted action information sequence std::vector <operation>OperationSequenceEdited is saved in the format of a text document and named OperationSequenceEdited.txt.

[0067] As Figure 3 shown, the content of the action information sequence OperationSequence is read from the text document file OperationSequence.txt, and a demonstration rewrite is performed at a frequency of 144 hz. The action information is read and input to update the state of the stroke model, and incremental rendering of the stroke is performed. Finally, the calligraphy character image img is obtained; As Figure 4 shown, the content of the action information sequence OperationSequenceEdited is read from the text document file OperationSequenceEdited.txt, and the process of updating the stroke state and incremental rendering is repeated. Finally, the calligraphy character image img after style editing is obtained edited .

[0068] As Figure 5 and Figure 6 shown, during the process of obtaining the calligraphy image again by reading the action information sequence from the text document file, the incremental rendering results after each update of the stroke model state are saved in the png format, and FFmpeg is used to synthesize the saved image sequence into a video, and the calligraphy writing dynamic video is output. Figure 5 is a screenshot of the calligraphy writing dynamic video obtained from OperationSequence.txt, Figure 6 is a screenshot of the calligraphy writing dynamic video obtained from OperationSequenceEdited.txt.

[0069] As Figure 7 shown, by adjusting the adjustment intensity Intensity in the editing parameters from 0 to 0.1, the action information sequence OperationSequence is edited and smoothly transitioned respectively, and calligraphy character images with gradually transitioning styles are obtained, realizing the interpolation transition of the calligraphy style.< / operation> < / operation>

Claims

1. A method for editing the style of electronic calligraphy based on motion information, characterized in that: The following steps are involved: 1) Obtaining input events from the hardware input device of the electronic calligraphy, and then storing the input events in the form of action information of the electronic calligraphy to obtain sampled action information of the electronic calligraphy; Alternatively, the motion information of the sampled electronic calligraphy is obtained from a data source; 2) inputting the action information of the electronic calligraphy sampled in step 1) into the brush stroke model used for drawing, and obtaining the drawing parameters and drawing identifiers of the current step; 3) Determine whether to draw according to the drawing identifier of the current step. If drawing is to be done, input the drawing parameters of the current step into the shader to draw the stroke image of the current step; 4) obtaining a new input event from the hardware input device of the electronic calligraphy, repeating 1) to obtain new action information, repeating 2) to obtain new drawing parameters and new drawing identifiers, and repeating 3) to perform incremental drawing of the stroke image to form a calligraphy character image img; Alternatively, continue sampling from the data source to obtain new action information, repeat 2) to obtain new drawing parameters and new drawing identifiers, and repeat 3) to perform incremental drawing of the stroke image to form the calligraphy character image img; 5) forming an action information sequence of a calligraphy character image img, and forming an edited action information sequence after modifying the action information in the action information sequence, and forming a new style of calligraphy character image img according to the edited action information sequence edited Or a new style of calligraphy image img is formed according to the edited and smoothly transitioned action information sequence edited .

2. The method for editing the style of electronic calligraphy based on motion information according to claim 1, characterized in that: In step 1), the action information form of the electronic calligraphy is defined as operation{x,y,operation_type i ,operation_params i }, where x and y are used to indicate the coordinates of the action, operation_type i Indicates whether the action type is in progress, operation_params i Parameters for the action type.

3. The method for editing the style of electronic calligraphy based on motion information according to claim 2, characterized in that: In step 1), the action types include pressing and lifting.

4. The method for editing the style of electronic calligraphy based on motion information according to claim 1, characterized in that: In step 5), the action information sequence for forming the calligraphy character image img includes: as the data representation of the whole process of electronic calligraphy creation, the action information sequence is represented as operations[operation0, operation1, ..., operation n-1 ].

5. The method for editing the style of electronic calligraphy based on motion information according to claim 4, characterized in that: In step 5), the modification of the action information in the action information sequence includes: replacement and / or deletion and / or insertion of the action information in the action information sequence.

6. The method for editing the style of electronic calligraphy based on motion information according to claim 5, characterized in that: In step 5), a new style of calligraphy image img is formed according to the edited and smoothly transitioned action information sequence edited , specifically including: The edited action information sequence is operations edited , for the action information sequence operations edited The action information before and after the edited part is specified, and the forward transition window length len1 and the backward transition window length len2 are respectively for operations edited The action information of the two parts with subscripts from i-len1 to i-1 and from i+k+1 to i+k+len2 are smoothly transitioned, where i to i+k is the subscript range of the edited part of the action information sequence before smooth transition correction. After the editing, the action information sequence with smooth transition is completed to form a new style of calligraphy image img edited .

7. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method for editing the style of electronic calligraphy based on motion information as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the method for editing the style of electronic calligraphy based on motion information as described in any one of claims 1 to 6 is implemented.

Images