Robot writing brush writing control method and device, medium and equipment
By establishing a pen control model during the brush writing process and combining the CCD-BSM brush stroke model, the problem of robots being difficult to simulate traditional calligraphy brush strokes is solved, and high-quality brush writing effect is achieved.
Patent Information
- Application Number
- CN202510120371.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-25
- Publication Date
- 2025-05-27
AI Technical Summary
When writing brushes with robots in the prior art, it is difficult to effectively simulate the brush strokes and styles of traditional calligraphy, resulting in a gap between the strokes and standard brush strokes of the written brush strokes.
By establishing a pen control model for each pen movement during the writing process, and combining it with the CCD-BSM brushstroke model, a parametric model that meets the requirements of calligraphy is established, and a robot is driven to control the brush pen shaft to move to realize brush writing.
It has achieved that robots can better imitate the brushstrokes and styles of traditional calligraphy, reduce the gap between the written brushstrokes and standard brushstrokes, and improve the quality and accuracy of writing.
Smart Images
Figure CN120038742A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of robot brush writing control, and in particular to a robot brush writing control method, device and equipment. Background Art
[0002] The writing control model is a crucial part of intelligent robot calligraphy and plays an important role in the presentation of robot calligraphy works. Unlike some intelligent writing robots that use hard pens, writing with a brush requires considering the brush stroke model, that is, the special pattern formed on the paper by the brush at a certain descending height and tilt angle. The establishment of a writing control model based on the brush stroke model needs to be solved urgently.
[0003] Recently, deep reinforcement learning and generative adversarial networks have been widely used to train robots to perform actual writing. GAN-AC combines deep reinforcement learning and generative adversarial networks, and uses the action-evaluation AC algorithm multiple times during the model training process to control the robot to write repeatedly and generate calligraphy images.
[0004] However, the above method requires the robot to repeatedly write certain strokes or glyphs to gradually improve its behavior, thereby outputting "similar" brush calligraphy. A large amount of repetitive writing will not only cause the service life of the robot to decrease rapidly, but this method also does not conform to the writing rules of calligraphy, resulting in a gap between the strokes of the written brush calligraphy and the standard brush calligraphy. Summary of the invention
[0005] Based on this, in order to solve the technical problems in the prior art, the present invention provides a robot brush writing control method, device, medium and equipment.
[0006] The present invention provides a robot brush writing control method, comprising:
[0007] Establishing a brush control model for each writing point where a brush movement occurs during the brush writing process, wherein the brush control model describes the brush movement by controlling the brush movement height, the lag of the brush tip, and the deviation of the brush tip direction angle;
[0008] The brush control model is combined with the CCD-BSM brush stroke model used to describe the trajectory sequence of brush writing, so as to add brush movement to the brush writing process and obtain the brush stroke model;
[0009] According to the writing rules of the brush, the starting stage, the moving stage and the ending stage are modeled respectively to obtain the corresponding writing control model;
[0010] The robot is driven by the brush stroke model and the writing control model to control the movement of the brush holder to achieve brush writing.
[0011] Furthermore, the pen control model includes a pen control model for describing the pen movement action, specifically including:
[0012] T 1 T 1 ′=T 2 T 2 ′=d
[0013]
[0014] w(H,α)=w p
[0015] Among them, H is the height of the pen shaft; α is the tilt angle of the pen shaft; the pen tip moves at a uniform speed to produce a speed of T 1 Starting from T 2 The pen shaft moves at a constant speed to produce a line segment with T as the end point. 1 ' as the starting point, with T 2 ' is the end point of the line segment, d is the position offset between the pen shaft and the pen tip; P is the position of the pen tip on the line segment T 1 T 2 P' is the pen position corresponding to P; w p is the stroke width corresponding to the pen tip at point P; w(·) is a function used to describe the stroke width.
[0016] Furthermore, the pen movement control model includes a pen turning and adjusting control model for describing the pen turning and adjusting action, specifically including:
[0017] w(H,α)=w o
[0018]
[0019] ∠AOT'=∠AOT+θ
[0020] θ r =∠AOT
[0021] Wherein, A is the starting point of the pen tip rotating and adjusting the peak; T is the end point of the pen tip rotating and adjusting the peak; O is the center of the concentric arc formed after the pen tip rotating and adjusting the peak, OT=r, r is the inner radius of the concentric arc; T' is the intersection of the tangent of point T and the outer arc of the concentric arc, TT'=d, d is the position offset of the pen shaft and the pen tip; θ=α, is the inclination angle of the pen shaft; θ r is the angle of the pen's rotation, w O is the stroke width of the concentric arcs; H is the height of the pen shaft descending.
[0022] Furthermore, the pen movement control model includes a pen folding control model for describing a pen folding action, specifically including:
[0023]
[0024] Among them, T 1 ~T 4 are the four trajectory points that the pen tip passes through during the folding action, among which T 2 It is the starting point for the pen tip to fold. 1 '~T 4 ' is with T 1 ~T 4 Corresponding pen position; T 1 T 2 and T 3 T 4 are all straight line segments; d is the position offset between the pen shaft and the pen tip; θ r It is the angle of the pen's rotation.
[0025] Furthermore, the brushstroke model specifically includes:
[0026]
[0027] Among them, P i-1 , P i and P i+1 are the three trajectory points that the pen tip passes through in sequence during the writing process; P' i-1 , P' i and P' i+1 is with P i-1 , P i and P i+1 Corresponding to the trajectory points of the pen; AB and CD are P i and P i+1 The corresponding stroke width, P i and P i+1 are the midpoints of line segments AB and CD respectively; w -1 (AB) and w -1 (CD) are P i and P i+1 Corresponding to the descending height of the pen holder; is the rotation angle of the pen holder, where
[0028] Furthermore, the writing control model includes a writing control model at the start-of-writing stage, specifically including:
[0029]
[0030]
[0031] Among them, F is the starting point of the reverse tip stroke; A is the end point of the reverse tip stroke and the starting point of the forward tip stroke, and H is the end point of the forward tip stroke; the reverse tip stroke is regarded as the superposition of the two line segments FA and AH; A' is the pen position corresponding to A, which is located on the extension line of line segment FA; σσ FA The vector description of the reverse stroke; M FA’ Indicates the action of writing with the tip of the brush facing backwards; A The vector description of the pen handle from point A' back to point A corresponding to the second stage of the reverse pen stroke process; M A’A The second process of writing with the tip of the pen in reverse direction corresponds to the description of the pen movement from point A' back to point A; Represents the vector corresponding to the movement from point A to point A'; Represents the vector corresponding to the movement from point F to point A', w min is the minimum drop height of the pen; Δh FA It is the height of the brush descending from point A to point A'.
[0032] Furthermore, the writing control model includes a writing control model in the stroke phase, specifically including:
[0033]
[0034] ΔH i =w -1 (CD)-w -1 (AB)
[0035] Among them, M i is the action sequence of writing control in the writing stage; i P is the writing control vector description of the writing stage; i-1 , P i and P i+1 are the three trajectory points in the pen trajectory of the pen tip folding; P i and P i+1 are the midpoints of line segments AB and CD, P' i-1 , P' i and P' i+1 is with P i-1 , P i and P i+1 Corresponding to the trajectory point of the pen; w i and w i+1 Respectively represent the width of the stroke corresponding to the i-th track point and the i+1-th track point; ΔH i is the height change of the brush, Δθ i Indicates the rotation angle of the pen; θ r is the angle of the pen's rotation; w -1 (·) is the height of the brush descent analyzed based on the stroke width.
[0036] Furthermore, the writing control model includes a writing control model in the finishing phase, specifically including:
[0037]
[0038] In the formula, σ AF M represents the stroke model vector corresponding to the end of the stroke; AF’ Indicates the corresponding pen movement of hiding the tip of the brush and ending the stroke; Indicates the vector corresponding to the pen moving from point A' to F'; Represents the vector corresponding to the movement from point A to point F'; Represents the vector corresponding to the movement from point F to point F'.
[0039] The present invention provides a robot brush writing control device, comprising:
[0040] A pen control model building module is used to build a pen control model for each writing point where a pen movement occurs during the writing process of a brush, wherein the pen control model describes the pen movement by controlling the pen height, the pen tip lag and the pen tip direction angle offset;
[0041] A brush stroke model building module is used to combine the brush control model with the CCD-BSM brush stroke model used to describe the trajectory sequence of brush writing, so as to add brush movement in the brush writing process and obtain the brush stroke model;
[0042] The writing control model building module is used to model the starting stage, the moving stage and the ending stage according to the writing rules of the brush, and obtain the corresponding writing control model;
[0043] The brush writing control module is used to drive the robot to control the movement of the brush holder through the brush stroke model and the writing control model to achieve brush writing.
[0044] The present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned robot brush writing control method when executing the program.
[0045] At least one of the above technical solutions adopted by the present invention can achieve the following beneficial effects:
[0046] In the robot brush writing control method provided by the present invention, the brush writing process is decomposed and refined, the brush movements in the writing process are analyzed and modeled, and then a parameterized model description that meets the requirements of calligraphy is established by combining the writing control model used to describe the starting, moving and ending of the brush. The parameterized model description provides a set of clear operating instruction sets for the robot. Without performing a large number of repetitive writing operations, the robot can be finely controlled to write according to the requirements of calligraphy at each stage, so that the robot can better imitate the brush strokes and style of traditional calligraphy, thereby writing brush characters that are closer to standard brush characters. BRIEF DESCRIPTION OF THE DRAWINGS
[0047] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:
[0048] Figure 1 A schematic diagram of the brush writing control process and robot writing effect provided by the present invention;
[0049] Figure 2 A schematic diagram of the writing trajectory of the pen tip and the pen shaft during the writing process provided by the present invention;
[0050] Figure 3 A schematic diagram of an arc trajectory formed during the pen turning and sharpening process provided by the present invention;
[0051] Figure 4 A schematic diagram of the trajectory of the pen during folding provided by the present invention, Figure 4 (a) in the figure is a schematic diagram showing an arc as the writing trajectory. Figure 4 (b) is a schematic diagram of a writing trajectory using a broken line;
[0052] Figure 5 A schematic diagram of writing control of the stroke model provided by the present invention;
[0053] Figure 6 A schematic diagram of writing control of the brush stroke model in the writing stage provided by the present invention;
[0054] Figure 7 This is a schematic diagram of the pen movement in the starting stage provided by the present invention. Figure 7 (a) is a schematic diagram of the reverse tip of the brush. Figure 7 (b) is a schematic diagram of starting with the tip of the brush;
[0055] Figure 8 A schematic diagram of the pen movement in the closing phase provided by the present invention; Figure 8 (a) is a schematic diagram of hiding the tip of the brush and finishing the stroke. Figure 8 (b) is a schematic diagram of finishing the stroke with the tip of the brush exposed;
[0056] Fig. 9 Schematic diagram of the robotic writing brush calligraphy hardware system provided by the present invention Fig. 9 In (a) is a schematic diagram of the robot and the writing platform Fig. 9 In (b) is a schematic diagram of the writing brush
[0057] Fig.10 Schematic diagram of the writing result of the stroke "horizontal" provided by the present invention Fig.10 In (a) is the reference stroke Fig.10 In (b) is a schematic diagram of the writing result where the direction of the brush tip is inconsistent with the direction of the pen movement Fig.10 In (c) is a schematic diagram of the writing result where the direction of the brush tip is consistent with the direction of the pen movement
[0058] Fig.11 Schematic diagram of the writing result of the stroke "vertical" provided by the present invention Fig.11 In (a) is the reference stroke Fig.11 In (b) is a schematic diagram of the writing result with the pen tip retracted at H = 15 mm Fig.11 In (c) is a schematic diagram of the writing result with the pen tip retracted at H = 0 mm
[0059] Fig.12 Schematic diagram of the writing result of the stroke "horizontal and left-falling stroke" provided by the present invention Fig.12 In (a) is the reference stroke Fig.12 In (b) is a schematic diagram of the writing result with the pen moving in the center stroke Fig.12 In (c) is a schematic diagram of the writing result with the pen moving in the side stroke
[0060] Fig.13 Schematic diagram of the writing result of the stroke "vertical and left-falling stroke" provided by the present invention Fig.13 In (a) is the reference stroke Fig.13 In (b) is a schematic diagram of the writing result with the pen moving in the center stroke Fig.13 In (c) is a schematic diagram of the writing result with the pen moving in the side stroke
[0061] Fig.14 Schematic diagram of the writing result of the stroke "right-falling stroke" provided by the present invention Fig.14 In (a) is the reference stroke Fig.14 In (b) is a schematic diagram of the writing result without turning the pen tip at the end of the stroke Fig.14 In (c) is a schematic diagram of the writing result with the pen tip turned at the end of the stroke
[0062] Fig.15 Schematic diagram of the writing result of the Chinese character "bu" provided by the present invention Fig.15 In (a) is the reference Chinese character "bu" Fig.15In it, (b) is a schematic diagram of the writing result without applying the stroke method, Fig.15 In it, (c) is a schematic diagram of the writing result with the stroke method applied;
[0063] Fig.16 It is a schematic diagram of the writing result of the Chinese character "qu" provided by the present invention; Fig.16 In it, (a) is the reference Chinese character "qu", Fig.16 In it, (b) is a schematic diagram of the writing result without the stroke method, Fig.16 In it, (c) is a schematic diagram of the writing result with the stroke method applied;
[0064] Fig.17 It is a schematic diagram of the writing result of the Chinese character "zi" provided by the present invention; Fig.16 In it, (a) is the reference Chinese character "zi", Fig.17 In it, (b) is a schematic diagram of the writing result before the stroke method is added, Fig.17 In it, (c) is a schematic diagram of the writing result after the stroke method is added;
[0065] Fig.18 Qualitative comparison between the present invention and other model algorithms provided by the present invention: Fig.18 In it, (a1) to Fig.18 In it, (a5) are reference images; Fig.18 In it, (b1) to Fig.18 In it, (b5) are schematic diagrams of the writing results of the present invention; Fig.18 In it, (c1) to Fig.18 In it, (c5) are schematic diagrams of the writing results of the DDPG algorithm; Fig.18 In it, (d1) to Fig.18 In it, (d5) are schematic diagrams of the writing results of the GAN-AC model; Fig.18 In it, (e1) to Fig.18 In it, (e5) are schematic diagrams of the writing results of the GANCC model;
[0066] Fig.18 In it, (f1) to Fig.18 In it, (f5) are schematic diagrams of the writing results of the GAN-LSTM model. Specific embodiments
[0067] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the specific embodiments of the present invention and the corresponding drawings. Apparently, the described embodiments are only a part of the embodiments of the present invention, rather than all of them. Based on the embodiments in the specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present invention.
[0068] Most of the existing control models use traditional B-spline for trajectory interpolation. Wen Y et al. proposed a robot trajectory tracking control scheme, using cubic spline interpolation to generate a closed geometric description of the three-dimensional path. A three-dimensional kinematic model of the end effector is established to control the input speed and path tangent vector. Zhenyu X et al. studied the method of brush character modeling by converting the writing problem into an optimal function approximation problem to minimize the input energy of the writing trajectory. The B-Spline method of control theory is used, with normalized uniform B-spline as the basis function, and the character skeleton image is generated from the trajectory curve. Based on the deep learning method pix2pix image conversion framework, the single-pixel character is converted into the corresponding brush writing character. However, the use of B-spline interpolation to control writing cannot fully reflect the local writing features of strokes or even complete Chinese characters, and there is a large error between the generated model and the real Chinese character strokes. Existing brush control methods also include: ① Based on Bezier curves and B-Spline curves: The traditional cubic B-spline algorithm is widely used to plan Cartesian space paths, optimize the trajectory of brush strokes, and control robot writing strokes. Mueller S et al. established a six-degree-of-freedom robot writing system based on KUKA. Based on the B-Spline curve, basic strokes were trained, and the B-spline of the reference example strokes and the stroke image written by the robot were obtained, and the image contour was processed. The learning process uses visual feedback to calculate the error between the reference character and the drawn character, and updates the trajectory information to generate the training function. However, this writing system can only reproduce simple strokes in the database. Lin HI et al. proposed a brush motion trajectory model to provide three-dimensional coordinates for robot writing control. Berio D et al. used a spline-based method to control polygon editing and learn to synthesize a continuous virtual trajectory. ② Based on learning control and optimal control: Berio D et al. proposed a method for generating interactive curves and motion paths. The random formula of optimal control theory was used to calculate the changes in the dynamic system. Huebel N et al. calculated the error between the reference character and the drawn character based on the projection position to form the paper coordinate position of the robot writing trajectory. The learning node uses the P-type iterative learning controller (ILC) to obtain the z-axis coordinate position, and finally forms the three-dimensional coordinates of the robot trajectory point. Inspired by pseudo-spectral optimal control, Wang S et al. parameterized the execution trajectory of each stroke into Chebyshev polynomials and applied the pseudo-spectral optimal control (PSOC) method to optimize the open-loop control trajectory of the robot end effector. Mueller S et al. proposed an iterative optimization writing method based on visual feedback, but the iterative optimization process is relatively complex and time-consuming. The writing result is compared with the coverage of the reference stroke, and the writing trajectory is updated as the evaluation result.Based on the error data points and the previous spline, the control point position of the next iteration is obtained, and the stroke contour is updated. Ma Z et al. established a closed-loop calligraphy system to reduce the impact of modeling errors and perform aesthetic optimization. The position of the n+1th stroke is predicted by the position of the first n strokes to control the writing strategy of the calligraphy robot. A constrained optimization problem is defined to optimize the aesthetic evaluation effect and control the writing trajectory of each stroke of the robot. ③ Based on deep learning: LSTM-GAN enables the robot to learn and generate Chinese character stroke sequences, i.e., writing trajectories, and optimize the writing trajectories based on the writing results. In the absence of a robot's motion trajectory dataset, the network structure combined with LSTM-GAN can convert pixel stroke images into vector trajectory sequences controlled by the robot. Zhou P et al. extracted the character image description parameters with the target style, converted the parameters into motion control parameters of the robot brush to guide the writing process, so as to accurately control the robot's writing trajectory.
[0069] In order to solve the technical problems of the prior art, the present invention studies the writing control of calligraphy robots on the basis of the existing CCD-BSM brush stroke model, and proposes a fine-grained modeling method to reflect local writing characteristics. By decomposing and refining each process of brush writing, the pen control model, pen turning and pen adjustment control model, and pen folding control model commonly used in the process of writing are analyzed and modeled. Based on the writing models of different pen control methods, the writing control based on the brush stroke model CCD-BSM is modeled, and then the writing control model of the strokes is established. According to the writing rules of the brush, the corresponding model descriptions and representations are given for the three steps of starting, moving, and ending the strokes. Experimental results show that the proposed fine-grained writing control model shows excellent performance in both basic stroke writing and Chinese character writing. The brush writing control process and the robot writing effect are shown in Figure 2. Figure 1 shown.
[0070] Example 1
[0071] The following is a detailed description of the process of the robot brush writing control method of this embodiment, which specifically includes the following steps:
[0072] S1: Establishing a brush control model for each writing point where a brush movement occurs during brush writing. The brush control model describes the brush movement by controlling the brush height, the lag of the brush tip, and the offset of the brush tip direction angle.
[0073] Since the writing control method based on the stroke model needs to first construct a control model for the key pen-moving methods in the writing process and give a corresponding model representation; then, on this basis, the trajectory sequence of the strokes is obtained, and the writing control of brush calligraphy is given in combination with different pen-moving methods.
[0074] During the writing process, the factors affecting the pen control method mainly include pen height, pen tip lag and pen tip angular deviation. The pen height depends on the vertical operations of lifting and pressing; the pen tip lag is mainly determined by the bending of the brush hair; the pen twist formed by the rotation of the pen shaft around itself and the pen shaft on the paper plane jointly determine the angular deviation of the pen tip. The writing control of the stroke model is composed of different strokes. First, the control models of pen movement, pen rotation and pen folding are studied, and then the control methods of the stroke model in the process of stroke generation and Chinese character writing are obtained.
[0075] S101: pen stroke control model.
[0076] During the writing process, the height of the brush is set to H (the descent here refers to the vertical descent from the center point of the brush handle to the height H, that is, the height of the brush handle descent, the point where the tip of the brush just touches the paper surface is set to H = 0; the tip of the brush handle is tilted at a certain angle based on the height H of the brush handle), and the tilt angle is α. The tip of the brush moves at a uniform speed on the paper plane along the stroke trajectory path, and the writing trajectory can be approximated as a line from T 1 Click to T 2 At this time, the position of the pen shaft lags behind the pen tip, and the trajectory on the paper plane is from T 1 'To T 2 ', the direction of the pen tip is opposite to the direction of the pen movement, along T 2 T 1 Exercise, such as Figure 2 shown.
[0077] Let the position P of the pen tip be T 1 T 2 A point on the straight line has a corresponding pen position of P′ and a stroke width of w. p The depth and height of the brush are H and α respectively. The tip of the brush moves at a constant speed to produce a speed of T. 1 Starting from T 2 The pen shaft moves at a constant speed to produce a line segment with T as the end point. 1 ' as the starting point, with T 2 ' is the end point of the line segment. It satisfies the control model shown in equations (1)-(3):
[0078] T 1 T 1 ′=T 2 T 2 ′=d(1)
[0079]
[0080] w(H,α)=w p (3)
[0081] Where d is the position offset between the pen shaft and the pen tip; P is the position of the pen tip on line segment T 1 T 2 P' is the pen position corresponding to P; w p is the stroke width corresponding to the pen tip at point P; w(·) is a function used to describe the stroke width.
[0082] S102: Control model for pen spinning and pen tip adjustment.
[0083] The pen spinning and peak adjustment action in brush calligraphy can be seen as a circular motion on the paper plane, which is equivalent to the brush being pressed on a disc at a certain descending height H and inclination angle α. The disc rotates counterclockwise while the brush remains stationary, and a stable state will be obtained after a time interval T. The brush shaft rotates around itself, and the brush rotates with the central axis of the disc as the reference, performing a circular motion, and its rotation speed is equal to the rotation speed of the pen shaft. Since the pen tip lags behind the pen shaft during the pen spinning process, the brush bundle of the brush writes on the horizontal disc surface to generate a trajectory of concentric arcs. Based on the above analysis, the brush generates the following during the pen spinning and peak adjustment process: Figure 3 The writing trajectory shown takes the center of the concentric arcs as the origin of the reference coordinate system, and sets the radius distribution of the inner arc and the outer arc to r and R.
[0084] In the process of pen turning and adjusting the peak, when the brush tip moves to position A, the position of the pen shaft (the projection of the center point of the brush shaft on the horizontal plane) is located at point A', and the distance between the two is d. In order to obtain an arc with a radius of r, the pen shaft must be located on the extension line of the brush tip trajectory and the position offset from the brush tip must always be maintained at d. According to geometric calculations, when the brush tip moves to the trajectory point T with parameters H and α, since the pen shaft is always in the tangent direction of the concentric arc, the relationship is as shown in formula (4):
[0085] OT⊥TT'(4)
[0086] In the formula, OT = r, TT' = d. At this time, the brush rod takes A' as the starting point of the writing trajectory and moves in an arc with a radius of OT'. Compared with the tip of the brush, its starting point direction angle should be ahead of the tip of the brush by α. The solutions of OT' and α are shown in formulas (5) and (6):
[0087]
[0088] α=tan -1 (d / r)(6)
[0089] From the above analysis, we can see that setting the width of the concentric arc to w o , the angle of the pen's rotation is θ rAny point on the concentric arc AB is T. When the pen tip rotates from point A to point T along the writing trajectory, the pen shaft rotates from point A' to point T', and the control model shown in equations (7)-(10) can be obtained:
[0090] w(H,α)=w o (7)
[0091]
[0092] ∠AOT'=∠AOT+θ(9)
[0093] θ r =∠AOT(10)
[0094] Wherein, A is the starting point of the pen tip turning and adjusting the peak; T is the end point of the pen tip turning and adjusting the peak; O is the center of the concentric arc formed after the pen tip turning and adjusting the peak, OT = r, r is the inner radius of the concentric arc; T' is the intersection of the tangent of point T and the outer arc of the concentric arc, TT' = d, d is the position offset of the pen shaft and the pen tip; θ = α, is the inclination angle of the pen shaft, w O is the stroke width of the concentric arcs; H is the height of the pen shaft descending.
[0095] S103: Control model of folding pen.
[0096] Different from writing and turning the brush, folding the brush is a process in which the tip of the brush is pressed on the paper surface at a certain descending height and inclination angle, and after it moves to a certain position along a straight line, the writing direction is changed to move to the next point along a straight line or circular trajectory.
[0097] Assume that the trajectory of the pen is a curve segment T consisting of four trajectory points 1 T 2 T 3 T 4 , where T 1 T 2 and T 3 T 4 is a straight line segment, arc T 2 T 3 The radius is r and the central angle is ∠T 2 OT 3 , T 1 T 2 and T 3 T 4 With arc T 2 T 3 Tangent to T 2 Dot and T 3 Points, such as Figure 4 As shown in (a) in the figure. From the way the pen is folded, it can be seen that when the pen tip moves from T 1Start from point T 2 At T 2 There is no point Figure 4 (a) is not the original arc trajectory, but moves to the upper right along T 2 T 4 Writing, the writing trajectory is a broken line T 1 T 2 T 4 ,like Figure 4 As shown in (b) in the figure. At this time, T 2 T 3 The arc segment becomes a straight line T with radius r = 0 2 T 3 '. The pen path of the pen holder is the curve segment T 1 'T 2 'T 3 'T 4 ', by two straight lines T 1 'T 2 '、T 3 'T 4 ' and arc segment T 2 'T 3 'Composition. 2 'T 3 'T 2 T 2 ' is the radius, T 2 is the center of the circle, and there exists a relationship as shown in formula (11):
[0098] T 2 T 2 '=T 2 T 3 '=T 4 T 4 ' = d(11)
[0099] Among them, T 1 ~T 4 are the four trajectory points that the pen tip passes through during the folding action, among which T 2 It is the starting point for the pen tip to fold. 1 '~T 4 ' is with T 1 ~T 4 The corresponding pen position.
[0100] In summary, when the pen tip is along T 1 T 2 Arrival point T 2 After that, the pen holder is bent to the upper right, along the arc segment T 2 'T 3 'Writing. Assuming that the height and tilt angle of the brush remain unchanged during this process, 3 'After that, follow T3 'T 4 'Continue writing. The pen tip is at point T 2 The control model for the broken pen is shown in equations (12) and (13):
[0101]
[0102] In the formula, θ r Indicates the rotation angle of the pen shaft around the rotation axis.
[0103] S2: The pen control model is combined with the CCD-BSM stroke model used to describe the trajectory sequence of stroke writing, so as to add pen movement to the brush writing process and obtain the stroke model.
[0104] The writing models of different pen control methods for given points have been derived above. The writing control based on the pen stroke model is needed to generate the writing control model of the strokes. Assume that the trajectory points of the writing are P i-1 P i P i+1 The control diagram of the brush stroke model CCD-BSM is as follows: Figure 5 , the vector description is as shown in formula (14):
[0105]
[0106] w i =AB (15)
[0107] Δw i =CD-AB (16)
[0108]
[0109] Where P i , P i+1 Respectively represent the midpoints of AB and CD; w i and Δw i Respectively represent the width of the stroke and the change in width; and The input angle and rotation angle of the pen tip are respectively θ i and Δθ are represented.
[0110] like Figure 5 As shown, along P i P i+1 The mark left by the writing trajectory on the paper is the ABCD part of the stroke model (the area surrounded by the red straight line and curve), P i , P i+1 Represent the midpoints of AB and CD, P' i and P” i+1 Located in P i-1 P i and Pi P i+1 The writing process of this stroke model is regarded as the pen tip at point P i Fold the pen downward to the right, so that P i P i+1 is the new direction of the pen, along P i P i+1 Continue to P i+1 Then, fold the pen downward to the right so that the tip of the pen is P i+1 P' i+1 Write in the direction of .
[0111] When the pen tip follows the trajectory P i-1 P i Go to P i When the pen is at point P' i , along P' i P” i Move the pen, the pen tip rotates θ' r As shown in formula (19), the pen moves to P" with a constant descending height and tilt angle. i point, along P" i P” i+1 Move the pen to P" i+1 , satisfying the relationship shown in equation (20). As the pen is folded, the width of the stroke changes from AB to CD, and the pen continues to move in P” i+1 P' i+1 Move the pen upward until point P' i+1 , the rotation of the pen shaft θ" r As shown in formula (21):
[0112]
[0113] In the formula, θ′ r and θ″ r They represent the rotation angle of the pen tip and the rotation angle of the pen shaft respectively.
[0114] The complete writing process is the superposition of the above three sections. i P i+1 In the case of extremely small size, the three segments are treated as one segment, and the corresponding pen control can be simplified as the pen in P' i Directly along P' i P' i+1 Move the pen to P' i+1 Where Figure 5 By calculating the vector, the control relationship can be derived as shown in formula (22):
[0115]
[0116] ΔH=w -1 (CD)-w-1 (AB) (24)
[0117] Where θ r It represents the rotation angle of the pen holder, and ΔH is the change in the height of the brush when it is pressed down.
[0118] The writing control vector based on the stroke model is defined as in formula (25). Figure 5 The corresponding pen movement is as shown in formula (26):
[0119]
[0120]
[0121] In the formula, represents the vector of the brush holder on the paper plane based on CCD-BSM, ΔH is the change in the height of the brush when it is pressed down, and Δθ represents the rotation angle of the brush holder.
[0122] S3: According to the writing rules of the brush, the starting stage, the stroke stage and the ending stage are modeled respectively to obtain the corresponding writing control model.
[0123] Based on the stroke model, the writing trajectory of the stroke is controlled. The stroke width in the actual writing process is determined by the CCD-BSM model parameters and the writing control model.
[0124] S301: Writing control during the writing phase.
[0125] The writing process is smooth and fluent. The writing control diagram of the pen stroke model is as follows Figure 6 , the action sequence is discrete, which divides the entire writing process into several action steps. Each step can contain one or more control vectors. The vector description is as shown in formula (27):
[0126]
[0127] Substituting equations (29) to (31) into the formula, we can get the writing control parameters, from which we can deduce the writing control action sequence in the writing stage as shown in equation (28):
[0128]
[0129] ΔH i =w -1 (CD)-w -1 (AB) (31)
[0130] Where ΔH i Δθ is the height change of the brush. i Indicates the rotation angle of the pen, M i is the action sequence of writing control in the writing stage; iIt is the writing control vector description of the pen-stroke stage.
[0131] S302: Writing control at the start of writing.
[0132] The starting stage of the pen is usually composed of multiple stroke models, corresponding to multiple pen movement units. FA , σ A , σ AH and σ HI The pen movement along the tip of the pen is composed of σ A , σ AH and σ HI Combination, such as Figure 7 As shown in (a) and (b) in 7. Take points A' and H' on the extension line of FA and AH respectively, so that AA'=HH'=d; IJ⊥CD, and take point I' on IJ, so that II'=d.
[0133] (1) Start writing with the tip of the brush facing backwards
[0134] The reverse stroke can be regarded as the superposition of two straight line FA and AH strokes. In the first stroke, the brush moves from the initial position F along the straight line FA to point A'. At the same time, the height of the brush gradually increases. The corresponding vector and stroke description are shown in equations (32) and (33), respectively:
[0135]
[0136] in, Represents the vector corresponding to the movement from point F to point A', w min is the minimum drop height of the pen; Δh FA It is the height of the brush descending from point A to point A'.
[0137] Due to the characteristic of reverse tip, the second section of the pen shaft returns from point A' to A, and the height of the brush descent also decreases from w min becomes 0, and the vector and pen movement description of this segment are as follows:
[0138]
[0139] In the formula represents the direction vector of IH, and σ A The vector description of the pen shaft returning from point A' to A corresponding to the second process of writing with the tip of the pen reversed.
[0140] (2) Start writing with the tip of the brush in the correct direction.
[0141] The pen tip of the forward-stroke writing starts directly from A, skipping the first two stages of the reverse-stroke writing.
[0142] S303: Writing control in the pen-lifting stage.
[0143] The pen-lifting stage usually consists of multiple stroke models, corresponding to multiple pen movement units. According to the pen movement analysis in the pen-lifting stage, the pen movement of the pen shaft is composed of σ IH 、σ HA 、σ AF combined. Figure 8 Figure 12 is a schematic diagram of the pen-lifting action in the pen-lifting stage. Take points H', A' and F' on the extension lines of IH, HA and AF respectively, such that HH' = AA' = FF' = d; II' connects the pen movement in the stroke stage, and II' = d.
[0144] Similar to the pen-down stage, the pen movement control of the pen shaft in the pen-lifting stage mainly includes two different methods: hidden-tip pen-lifting and exposed-tip pen-lifting.
[0145] (1) Hidden-tip pen-lifting.
[0146] When performing hidden-tip pen-lifting, the pen tip moves from point A to point F, and the pen shaft moves from point A' to point F'. At the same time, the writing brush is continuously lifted, and at point F', the pen tip just leaves the paper plane. The corresponding stroke model vector σ AF and pen movement M AF' are described as shown in equations (38) and (39):
[0147]
[0148] In the formula, w min can be solved from the descent height h min of the writing brush.
[0149] (2) Exposed-tip pen-lifting.
[0150] For exposed-tip pen-lifting, the pen tip directly leaves the paper surface from A, omitting the last two stages of hidden-tip pen-lifting.
[0151] Furthermore, this embodiment also provides robot writing tests and performance evaluations. The experimental environment is as Fig. 9 shown.
[0152] The basic stroke writing test is as Figure 10 to Figure 14 shown. Analyze the partial experimental results of each stroke respectively. The partial writing results of the stroke "horizontal" are as Fig.10 shown. From left to right are the reference stroke, the stroke written when the pen tip direction is inconsistent with the pen movement direction, and the writing result when the pen tip direction is consistent with the pen movement direction. The Chinese character writing test results are as Figure 15 to Figure 17 shown.
[0153] When the robot performs writing tasks, the writing speed variation range can reflect the stability of the writing process. The smaller the speed variation range, the more stable the writing process. Otherwise, the robot will shake. In order to further prove the stability of the writing process, this section evaluates the writing speed variation of the five basic strokes "horizontal", "vertical", "horizontal left stroke", "vertical left stroke" and "right stroke". The speed value of the end effector is collected by the teach pendant, and the speed variation of the five basic strokes during the robot's pen movement is analyzed to examine the impact of speed on writing performance. The writing postures generated by the application plan were used to conduct 100 experimental tests on the five strokes, and the results are shown in Table 1. The speed variation ranges of the five strokes are all small, and the speed variation range of the stroke "vertical" is only 0.47mm / s, indicating that the robot can move at a steady speed during the writing process and will not cause impact due to sudden changes in speed.
[0154] Table 1 Changes in writing speed of basic strokes (mm / s)
[0155] Strokes Top speed Minimum speed Range of variation "horizontal" 22.87 21.32 1.55 "Horizontal stroke" 20.15 19.54 0.61 "vertical" 20.28 19.81 0.47 "Vertical stroke" 21.66 20.23 1.43 "N" 22.72 20.95 1.77
[0156] The intelligent calligraphy robot system of the present invention is compared and analyzed with four advanced robot calligraphy systems and algorithms, namely, the GANCC Chinese calligraphy robot, the GAN-LSTM based calligraphy robot learning system, the GAN-AC fusion network model, and the DDPG algorithm training generation model. The results are as follows: Fig.18 , as shown in Tables 2 and 3.
[0157] Table 2 Comparison of CSIM with other model algorithms (%)
[0158]
[0159] Table 3 SSIM comparison with other model algorithms (%)
[0160]
[0161] By comparing with the current optimal models and algorithms, the superiority of the proposed intelligent robot writing control model in stroke generation is demonstrated.
[0162] The robot end uses a brush to complete the writing experiment of each basic stroke. The results of 100 experiments are quantitatively evaluated based on CSIM and SSIM, as shown in Table 4, which records the best writing similarity, worst similarity and average similarity of each stroke. The average CSIM of the five strokes is more than 98%, indicating that different brushes have strong writing ability for basic strokes. As can be seen from Table 4, the basic stroke "vertical" has the highest CSIM, with a maximum similarity of 99.88%.
[0163] Table 4 CSIM and SSIM (%) of basic stroke writing experiment
[0164]
[0165] According to the CSIM and SSIM in Table 4, among the five strokes, the robot has better writing effects on the strokes of "horizontal", "vertical" and "vertical left-falling", the writing effect on the stroke of "horizontal left-falling" is second, and the writing effect on the stroke of "right-falling" is slightly weaker.
[0166] In order to verify the generalization and robustness of the calligraphy robot under the condition of Chinese character changes, the robot was controlled to conduct 100 writing experiments on Chinese characters with different structures and styles, and the writing results were quantitatively evaluated based on the CSIM and SSIM similarity evaluation indicators, as shown in Table 5. Table 5 records the similarity of each Chinese character written by the robot in the best and worst cases, as well as the average similarity of all Chinese character images. It can be seen from Table 5 that the average CSIM of the Chinese characters written by the robot exceeds 85%. According to the average values of CSIM and SSIM, it shows that the robot has a certain writing ability for Chinese characters with different structures.
[0167] Table 5 CSIM and SSIM (%) of Chinese character writing experiment
[0168]
[0169]
[0170] The writing control model plays a vital role in the presentation effect of intelligent robot calligraphy works. The existing writing control model mainly models the writing trajectory, and no fine-grained writing control model based on brush strokes has been studied. How to establish writing control based on the brush stroke model needs to be solved urgently. Based on the proposed brush stroke model CCD-BSM, the writing control of the intelligent calligraphy robot is studied, and a fine-grained model modeling method that conforms to the rules of brush calligraphy is proposed to reflect the local writing characteristics. By decomposing and refining each process of brush writing, the commonly used control models in the brush movement process are analyzed and modeled. Based on different brush movement control writing models, the writing control based on the brush stroke model CCD-BSM is modeled; according to the brush writing rules, parameterized models are established to describe the stroke start, stroke, and end. Experimental results show that the proposed writing control model shows excellent performance in basic stroke writing, Chinese character writing, and writing performance.
[0171] This paper mainly studies the fine-grained control of robot brush calligraphy, and proposes a robot writing control model based on the brush stroke model and in accordance with the brush calligraphy rules. By modeling different brush control methods, a control method of the brush stroke model and a three-stage writing control model of the stroke are established. Writing tests and performance evaluations are performed. Compared with most existing models, it has major advantages. The main contributions are summarized in three aspects: 1) By decomposing and refining each process of brush writing, the commonly used brush control model, pen turning and pen adjustment control model, and pen folding control model in the brush writing process are analyzed and modeled. 2) Based on the writing model of different brush control methods, the writing control based on the brush stroke model CCD-BSM is modeled; according to the writing rules of the brush, the corresponding model descriptions and representations are given for the three steps of starting, moving, and ending the stroke. 3) Based on the existing CCD-BSM brush stroke model, the writing control of the calligraphy robot is studied, and a fine-grained model modeling method that conforms to the rules of brush calligraphy is proposed. The experimental results show that the proposed writing control model shows excellent performance in both basic stroke writing and Chinese character writing.
[0172] The above is a robot brush writing control method provided by one or more embodiments of the present invention. Based on the same idea, the present invention also provides a corresponding robot brush writing control device, including:
[0173] The pen control model building module is used to establish a pen control model for each writing point where a pen movement occurs during the brush writing process. The pen control model describes the pen movement by controlling the pen height, pen tip lag and pen tip direction angle offset.
[0174] The brush stroke model building module is used to combine the brush control model with the CCD-BSM brush stroke model used to describe the trajectory sequence of brush writing, so as to add brush movement in the brush writing process and obtain the brush stroke model.
[0175] The writing control model building module is used to model the starting stage, the stroke stage and the ending stage respectively according to the writing rules of the brush to obtain the corresponding writing control model.
[0176] The brush writing control module is used to drive the robot to control the movement of the brush holder through the brush stroke model and the writing control model to achieve brush writing.
[0177] For the specific definition of the robot brush writing control device, please refer to the definition of the robot brush writing control method above, which will not be repeated here. Each module in the above-mentioned robot brush writing control device can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.
[0178] The present invention also provides a structure of a computer device. At the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the robot brush writing control method provided above.
[0179] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).
[0180] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present invention.
Claims
1. A robot brush writing control method, characterized in that: include: Establishing a brush control model for each writing point where a brush movement occurs during the brush writing process, wherein the brush control model describes the brush movement by controlling the brush movement height, the lag of the brush tip, and the deviation of the brush tip direction angle; The brush control model is combined with the CCD-BSM brush stroke model used to describe the trajectory sequence of brush writing, so as to add brush movement to the brush writing process and obtain the brush stroke model; According to the writing rules of the brush, the starting stage, the moving stage and the ending stage are modeled respectively to obtain the corresponding writing control model; The robot is driven by the brush stroke model and the writing control model to control the movement of the brush holder to achieve brush writing.
2. The robot brush writing control method according to claim 1, characterized in that: The pen control model includes a pen control model for describing the pen movement action, specifically including: T1T1′=T2T2′=d w(H,α)=w p Among them, H is the height of the pen shaft; α is the tilt angle of the pen shaft; the pen tip moves at a uniform speed to generate a line segment with T1 as the starting point and T2 as the end point; the pen shaft moves at a uniform speed to generate a line segment with T1' as the starting point and T2' as the end point, d is the position offset between the pen shaft and the pen tip; P is a point on the line segment T1T2, P' is the pen shaft position corresponding to P; w p is the stroke width corresponding to the pen tip at point P; w(·) is a function used to describe the stroke width.
3. The robot brush writing control method according to claim 1, characterized in that: The pen movement control model includes a pen turning and adjusting control model for describing the pen turning and adjusting action, specifically including: w(H,α)=w o ∠AOT'=∠AOT+θ i r =∠AOT Wherein, A is the starting point of the pen tip rotating and adjusting the peak; T is the end point of the pen tip rotating and adjusting the peak; O is the center of the concentric arc formed after the pen tip rotating and adjusting the peak, OT=r, r is the inner radius of the concentric arc; T' is the intersection of the tangent of point T and the outer arc of the concentric arc, TT'=d, d is the position offset of the pen shaft and the pen tip; θ=α, is the inclination angle of the pen shaft; θ r is the angle of the pen's rotation, w O is the stroke width of the concentric arcs; H is the height of the pen shaft descending.
4. The robot brush writing control method according to claim 1, characterized in that: The pen movement control model includes a pen folding control model for describing the pen folding action, specifically including: Among them, T1~T4 are the four trajectory points of the pen tip during the folding action, among which T2 is the starting point of the pen tip during the folding action; T1'~T4' are the pen shaft positions corresponding to T1~T4; T1T2 and T3T4 are both straight line segments; d is the position offset between the pen shaft and the pen tip; θ r It is the angle of the pen's rotation.
5. The robot brush writing control method according to claim 1, characterized in that: The stroke model specifically includes: Among them, P i-1 , P i and P i+1 are the three trajectory points that the pen tip passes through in sequence during the writing process; P' i-1 , P' i and P' i+1 is with P i-1 , P i and P i+1 Corresponding to the trajectory points of the pen; AB and CD are P i and P i+1 The corresponding stroke width, P i and P i+1 are the midpoints of line segments AB and CD respectively; w -1 (AB) and w -1 (CD) are P i and P i+1 Corresponding to the descending height of the pen holder; is the rotation angle of the pen holder, where 6. The robot brush writing control method according to claim 1, characterized in that: The writing control model includes a writing control model at the start of writing, specifically including: Among them, F is the starting point of the reverse tip stroke; A is the end point of the reverse tip stroke and the starting point of the forward tip stroke, and H is the end point of the forward tip stroke; the reverse tip stroke is regarded as the superposition of the two line segments FA and AH; A' is the pen position corresponding to A, which is located on the extension line of line segment FA; σ FA The vector description of the reverse stroke; M FA’ Indicates the action of writing with the tip of the brush facing backwards; A The vector description of the pen handle from point A' back to point A corresponding to the second stage of the reverse pen stroke process; M A’A The second process of writing with the tip of the pen in reverse direction corresponds to the description of the pen movement from point A' back to point A; Represents the vector corresponding to the movement from point A to point A'; Represents the vector corresponding to the movement from point F to point A', w min is the minimum drop height of the pen; Δh FA It is the height of the brush descending from point A to point A'.
7. The robot brush writing control method according to claim 1, characterized in that: The writing control model includes a writing control model in the stroke phase, specifically including: ΔH i =w -1 (CD)-w -1 (AB) Among them, M i is the action sequence of writing control in the writing stage; i is the writing control vector description of the writing stage; P i-1 , P i and P i+1 are the three trajectory points in the pen trajectory of the pen tip folding; P i and P i+1 are the midpoints of line segments AB and CD, P' i-1 , P' i and P' i+1 is with P i-1 , P i and P i+1 Corresponding to the trajectory point of the pen; w i and w i+1 Respectively represent the width of the stroke corresponding to the i-th track point and the i+1-th track point; ΔH i is the height change of the brush, Δθ i Indicates the rotation angle of the pen; θ r is the angle of the pen's rotation; w -1 (·) is the height of the brush descent analyzed based on the stroke width.
8. The robot brush writing control method according to claim 1, characterized in that: The writing control model includes a writing control model at the end of writing, which specifically includes: In the formula, σ AF M represents the stroke model vector corresponding to the end of the stroke; AF’ Indicates the corresponding pen movement of hiding the tip of the brush and ending the stroke; Indicates the vector corresponding to the pen moving from point A' to F'; Represents the vector corresponding to the movement from point A to point F'; Represents the vector corresponding to the movement from point F to point F'.
9. A robot brush writing control device, characterized in that: include: A pen control model building module is used to build a pen control model for each writing point where a pen movement occurs during the writing process of a brush, wherein the pen control model describes the pen movement by controlling the pen height, the pen tip lag and the pen tip direction angle offset; A brush stroke model building module is used to combine the brush control model with the CCD-BSM brush stroke model used to describe the trajectory sequence of brush writing, so as to add brush movement in the brush writing process and obtain the brush stroke model; The writing control model building module is used to model the starting stage, the moving stage and the ending stage according to the writing rules of the brush, and obtain the corresponding writing control model; The brush writing control module is used to drive the robot to control the movement of the brush holder through the brush stroke model and the writing control model to achieve brush writing.
10. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any one of claims 1 to 8 is implemented.