Force feedback virtual writing brush simulation method, system and device and storage medium

Abstract

The invention relates to the technical field of three-dimensional virtual drawing, in particular to a force feedback virtual writing brush simulation method, system and device and a storage medium. The force feedback virtual writing brush simulation method comprises the following steps: acquiring space state information of touch equipment corresponding to a virtual writing brush in real time; adjusting the pose of the virtual writing brush according to the space state information to determine the pressure depth, the inclination angle, the tangential speed and the vertical speed of the nib of the virtual writing brush; determining an ellipse parameter of the virtual writing brush handwriting according to the position and the pressure depth of the pen point under the world coordinate system, and generating an initial handwriting texture according to the ellipse parameter of the handwriting; generating a flying white transparency shade according to the inclination angle, the tangential speed and the vertical speed; and processing the initial handwriting texture and the fly-white transparency mask by using a preset rule to obtain the virtual writing brush handwriting with a fly-white transparency mask effect. According to the method, the diversification and controllability of the white flying phenomenon in visual representation can be realized, so that the generated virtual writing brush handwriting is more real.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional virtual drawing technology, and particularly to a force feedback virtual brush simulation method, system, device and storage medium. Background Art

[0002] In the field of virtual brush writing simulation, the existing technologies mainly focus on tactile feedback modeling, handwriting geometry calculation and simulation of the flying white effect, but there is still a significant gap from the actual calligraphy experience. The following are representative technical solutions and their limitations:

[0003] The first one is the real-time tactile brush and ink simulation system proposed by Yeh in 2002. This solution uses a Phantom six-degree-of-freedom force feedback device to collect the pen posture and pressure information of the writer in real time, uses eight linear elastic springs to simulate the mechanical behavior of the brush bundle of the brush, and simulates the diffusion effect of the ink through a capillary diffusion model on the CPU side. The entire system finally uses OpenGL for image rendering and forms a closed-loop tactile feedback mechanism. Although this technology initially realizes the combination of tactile and ink visual effects, since the springs used are linear models, it is impossible to achieve the tactile realism and stability of the pen tip under large indentation and fast pen sweeping conditions. In addition, this system completely lacks the simulation of the unique flying white phenomenon in calligraphy and cannot show the side-edge ink-lacking texture common in real brush writing.

[0004] The second one is a brush modeling method based on force feedback technology proposed in Patent CN103345773A. This solution collects pressure signals through a force feedback device, uses a single-degree-of-freedom linear spring oscillator model to convert the pressure into the displacement of the pen hair node, and realizes the deformation of the brush hair shape of the brush based on a geometric model of a central broken line plus a circular cross-section, and finally calculates the pen stroke width of the elliptical cross-section based on the principle of area conservation. However, this solution is too simple in force feedback modeling, with only a normal linear spring, lacking the comprehensive effects of exponential non-linear elasticity, tangential viscous resistance and friction in the real writing process; at the same time, the major axis of the ellipse is fixed at 1.5 times the maximum circle radius and lacks the ability to be flexibly adjusted according to the tilt posture. In addition, this technical solution does not involve the realization of the flying white effect in calligraphy, making the output handwriting lack the delicate texture changes in real writing.

[0005] The third is the "Kasure" energy model proposed by Takeda in 2005. This model analyzes the input data of a two-dimensional digitizing tablet (including writing speed and pen pressure), and judges the occurrence of the scribbling phenomenon by calculating the difference in ink energy. When the energy difference exceeds a preset threshold, it is judged that the dry pen effect occurs, and then a prefabricated texture template is used for the synthesis of the handwriting. However, this technical solution is completely a two-dimensional pure visual simulation, lacking the support of tactile feedback and a three-dimensional brush model. At the same time, its simulation of the scribbling effect only considers a single ink energy threshold, and does not consider the rich texture changes caused by complex factors such as speed, pen-lifting rate, tilt, and side bias in actual writing; moreover, its calculation process completely relies on the CPU side and cannot achieve real-time and high-frame-rate writing feedback.

[0006] Although the above three existing technical solutions each have certain research value, there are still obvious deficiencies and limitations in virtual brush writing simulation, which are specifically reflected in the following aspects:

[0007] First of all, in terms of tactile feedback, the existing technologies mainly use a single linear spring or a single-degree-of-freedom oscillator model, which cannot represent the real non-linear elastic hardening effect during the brush writing process, and also lack the combined effect of tangential viscous resistance and friction that is closely related to speed. This deficiency directly leads to unstable or even oscillating tactile feedback during rapid writing or deep pressing, reducing the authenticity of the writing simulation.

[0008] Secondly, in terms of the simulation of the scribbling effect, the existing technologies either completely ignore the scribbling phenomenon or simply judge the scribbling effect based on a single energy threshold, and fail to accurately and flexibly represent the delicate scribbling transparency mask unique to calligraphy. The scribbling phenomenon in real writing is closely related to multiple factors such as pen speed, pen-lifting speed, tilt angle, and side bias. A single factor cannot reflect the complexity of the real writing process, resulting in an obvious gap between the output handwriting and the actual effect.

[0009] In addition, in terms of handwriting calculation and real-time performance, the existing technologies often need to adopt the method of high-degree-of-freedom spring brush bundle projection per hair or CPU-side texture splicing. These methods have a large amount of calculation and poor real-time performance, and it is difficult to meet the requirements of 1kHz high bandwidth for tactile feedback and high frame rate for visual rendering, and it is impossible to achieve a truly high-real-time and high-fidelity virtual writing experience.

[0010] Finally, the existing technical solutions generally lack a complete integrated design from force feedback to visual output. The tactile module, geometric module, and visual output module are independent of each other, lacking the coordinated cooperation of unified parameters and data streams, resulting in the inability to unify and coordinate the tactile feedback, shape, and visual effects during the writing process, reducing the realism and consistency of the overall simulation experience of the system. Summary of the Invention

[0011] This application aims to at least solve the technical problems existing in the prior art, and provides a force feedback virtual brush simulation method, system, device and storage medium.

[0012] In a first aspect, a force feedback virtual brush simulation method provided by the present invention includes:

[0013] Obtain the spatial state information of the tactile device corresponding to the virtual brush in real time; the spatial state information includes the position, direction and movement speed of the pen tip of the tactile device in the world coordinate system; the tactile device is a hardware device for providing force feedback when simulating brush writing.

[0014] Adjust the pose of the virtual brush according to the spatial state information to determine the penetration depth, tilt angle, tangential speed and vertical speed of the virtual brush pen tip.

[0015] Determine the elliptical parameters of the virtual brush handwriting according to the position and penetration depth of the pen tip in the world coordinate system, and generate an initial handwriting texture according to the elliptical parameters of the handwriting.

[0016] Generate a flying white transparency mask according to the tilt angle, tangential speed and vertical speed.

[0017] Process the initial handwriting texture and the flying white transparency mask using a preset rule to obtain a virtual brush handwriting with the effect of the flying white transparency mask.

[0018] In a second aspect, the present invention provides a force feedback virtual brush simulation system, and the system includes:

[0019] The mechanics layer is used to receive the spatial state information of the tactile device corresponding to the virtual brush in real time, and analyze the tactile device posture and movement state according to the spatial state information; the spatial state information includes the position, direction and movement speed of the pen tip of the tactile device in the world coordinate system; the tactile device is a hardware device for providing force feedback when simulating brush writing.

[0020] The tactile layer is used to calculate non-linear force feedback information according to the spatial state information and send the non-linear force feedback information to the tactile device.

[0021] The handwriting generation layer is used to determine the elliptical parameters of the virtual brush handwriting according to the position and penetration depth of the pen tip in the world coordinate system, and generate an initial handwriting texture according to the elliptical parameters of the handwriting; and generate a flying white transparency mask according to the tilt angle, tangential speed and vertical speed.

[0022] The rendering and interaction layer is used to process the initial handwriting texture and the flying white transparency mask using a preset rule to obtain a virtual brush handwriting with the effect of the flying white transparency mask.

[0023] In a third aspect, the present invention provides an electronic device, and the electronic device includes:

[0024] At least one processor; and,

[0025] A memory communicatively connected to the at least one processor; wherein,

[0026] The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the at least one processor is enabled to execute the force feedback virtual writing brush simulation method described above.

[0027] In a fourth aspect, the present invention further provides a computer-readable storage medium, in which at least one computer program is stored, and the at least one computer program is executed by a processor in an electronic device to implement the force feedback virtual writing brush simulation method described above.

[0028] In summary, the present application includes the following beneficial technical effects:

[0029] By analyzing the spatial attitude information obtained by the tactile device, the present application infers the pen force and movement changes applied by the user during writing. According to this, not only the thickness and shape of the stroke are changed, but also the ink color density and the texture of the flying white are synchronously affected. The present application determines the user's writing action based on the pressing depth, tilt angle, tangential speed, and vertical speed of the virtual writing brush tip, and adjusts the texture of the virtual writing brush handwriting in real time according to the micro-changes of the user's writing action, realizing the diversification and controllability of the visual performance of the flying white phenomenon, and making the generated virtual writing brush handwriting more realistic. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a schematic flowchart of a force feedback virtual writing brush simulation method provided by an embodiment of the present invention;

[0031] Figure 2 is a schematic structural diagram of a tactile device provided by an embodiment of the present invention;

[0032] Figure 3 is a flowchart of virtual writing brush handwriting texture generation provided by an embodiment of the present invention;

[0033] Figure 4 are some virtual writing brush handwritings generated by using the force feedback virtual writing brush simulation method of the present application

[0034] Figure 5 is a functional module diagram of a force feedback virtual writing brush simulation system provided by an embodiment of the present invention;

[0035] Figure 6 is a schematic structural diagram of an electronic device for implementing the force feedback virtual writing brush simulation method provided by an embodiment of the present invention.

[0036] Reference numerals: 10, processor; 11, memory; 12, communication bus; 13, communication interface.

[0037] The realization, functional features and advantages of the present invention will be further described in conjunction with embodiments with reference to the accompanying drawings. Detailed implementation manners

[0038] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described by referring to the accompanying drawings are exemplary and are only used to explain the present invention and should not be construed as a limitation to the present invention.

[0039] In the description of the present invention, it should be understood that the orientation or positional relationships indicated by the terms "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. are based on the orientation or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation to the present invention.

[0040] In the description of the present invention, unless otherwise specified and limited, it should be noted that the terms "installation", "connection", and "connection" should be understood in a broad sense. For example, it may be a mechanical connection or an electrical connection, or it may be the communication inside two elements. It may be directly connected or indirectly connected through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific situations.

[0041] Refer to Figure 1 As shown, it is a schematic flowchart of a force feedback virtual brush simulation method provided by an embodiment of the present invention. In this embodiment, the force feedback virtual brush simulation method includes:

[0042] S1. Real-time obtain the spatial state information of the tactile device corresponding to the virtual brush.

[0043] The spatial state information includes the position, direction, and movement speed of the pen tip of the tactile device in the world coordinate system. The tactile device is a hardware device for providing force feedback when simulating brush writing. In brush simulation, the tactile device (HapticDevice) is a hardware device that can provide force feedback or tactile perception and is used to simulate the physical interaction experience when writing with a real brush. The tactile device in this embodiment is as Figure 2As shown, through means such as mechanical force, vibration, or motion constraints, it enables users to feel tactile information such as the resistance of the pen tip contacting the paper, the elastic deformation of the pen hairs, and the friction sensation in the virtual environment, thereby enhancing the authenticity and immersion of the simulation.

[0044] The spatial state information includes information such as the position, direction, and movement speed of the virtual writing brush tip in the world coordinate system; in this embodiment, the spatial state information is represented in the form of a pose matrix; through the pose matrix of the tactile device, the position, direction, and movement speed of the virtual writing brush tip in the world coordinate are deduced.

[0045] S2. Adjust the pose of the virtual writing brush according to the spatial state information to determine the penetration depth, tilt angle, tangential speed, and vertical speed of the virtual writing brush tip.

[0046] Specifically, the penetration depth is the distance that the pen tip presses down vertically onto the paper surface, the tilt angle is the angle between the axis of the pen hairs and the normal vector of the paper surface, the tangential speed represents the component of the movement speed of the pen tip in the direction parallel to the paper surface, and the vertical speed represents the component of the movement speed of the pen tip in the direction perpendicular to the paper surface.

[0047] The virtual writing brush is represented by a rigid body writing brush model. The rigid body writing brush model includes a pen rod rigid body (cylinder) and a pen tip rigid body (cone). In this embodiment, the spatial state information is a 4×4 pose transformation matrix, which maps the device pose to the poses of the cylinder (pen rod) and the cone (pen hairs), and updates the Kinematic state of the corresponding rigid bodies in the Bullet system; Kinematic Rigid Body (interpreted in Chinese as kinematic rigid body) is the bridge connecting the "physical simulation world" and the "external control device", ensuring that the Bullet system can perceive the real device position without redundant simulation.

[0048] Specifically, adjusting the pose of the rigid body writing brush model according to the spatial state information to determine the current penetration depth, tilt angle, tangential speed, and vertical speed of the virtual writing brush tip includes:

[0049] S21. Deduce the local transformation information of the rigid body writing brush model according to the spatial state information. The local transformation information includes the offset paths of the pen rod rigid body and the pen tip rigid body in the local coordinate system.

[0050] In this embodiment, the spatial state information is a 4×4 pose transformation matrix.

[0051] S22. Adjust the pose of the rigid body writing brush model according to the local transformation information to determine the positions and orientations of the pen rod rigid body and the pen tip rigid body in the world coordinate system after adjustment, and obtain the adjustment information.

[0052] S23. Calculate the current penetration depth, tilt angle, tangential speed, and vertical speed of the virtual writing brush tip according to the adjustment information.

[0053] In this embodiment, the calculation formula of the pressing depth is:

[0054] d=max(0,-y apex );

[0055] Among them, d represents the pressure depth, y apex Indicates the Y coordinate of the pen tip in the world coordinate system. Since the paper is at y=0, when y apex When <0, it indicates that the pen hair is pressed into the paper surface; in this embodiment, the unit of the pressure depth is meter (m), which indicates the depth that the tip of the cone is pressed downward into the plane of the paper surface.

[0056] The calculation formula of vertical velocity is:

[0057]

[0058] Among them, v ⊥ represents the vertical velocity, Indicates the Y coordinate of the vertex apex of the brush cone in the current frame. Indicates the Y coordinate of the vertex apex of the brush hair cone in the previous frame, Δt is the time step; the vertical velocity is in meters per second (m / s) and is directly used to calculate the normal damping force; if the vertical velocity result is a negative value, it means that the pen tip is moving toward the paper.

[0059] Tangential velocity v tan The calculation process is as follows:

[0060] First calculate the total displacement of the vertex apex world position of the brush hair cone in three-dimensional space: in, Indicates the three-dimensional coordinates of the vertex apex of the brush cone in the current frame in three-dimensional space. Represents the three-dimensional coordinates of the vertex apex of the brush hair cone in the previous frame in the three-dimensional space, and Δt represents the time step;

[0061] Then extract its component in the tangential plane (i.e. parallel to the paper): Among them, v ⊥ represents the vertical velocity, is the normal vector of the paper;

[0062] The modulus of the tangential velocity is: The unit of tangential velocity is meters per second (m / s), which is used for tangential friction and subsequent fly white factor calculations.

[0063] S3. Determine the ellipse parameters of the handwriting according to the position and pressure depth of the pen tip in the world coordinate system, and generate an initial handwriting texture according to the ellipse parameters of the handwriting.

[0064] Specifically, the elliptical parameters of the virtual brushstroke are determined based on the position and pressing depth of the pen tip in the world coordinate system, and an initial brushstroke texture is generated according to the elliptical parameters of the brushstroke, including:

[0065] S31. Determine the elliptical parameters of the brushstroke according to the position and pressing depth of the cone corresponding to the pen tip in the world coordinate system.

[0066] First, based on the attitude and pressing depth information of the cone corresponding to the pen tip, the system uses the analytic geometry relationship between the cone and the plane intersection line to determine that the brushstroke of each pen tip on the paper is an elliptical area. The elliptical parameters of the brushstroke include the minor axis length and the major axis length. By calculating the projections of the two endpoints of the cone in the world coordinates, the following parameters can be obtained:

[0067] 1. Center coordinates (U, V): The intersection point of the cone axis corresponding to the pen tip and the paper surface;

[0068] 2. The lengths of the major axis and the minor axis (a, b): jointly determined by the pressing depth d and the tilt angle θ;

[0069] 3. Rotation angle φ: determined by the projection direction of the cone axis corresponding to the pen tip in the XZ plane.

[0070] Based on this, the calculation formula for the minor axis length is:

[0071] r minor =[r0+(r max -r0)(1 - e -αd )]×(1 + γ1sin p θ)

[0072] where r minor represents the minor axis length, r0 represents the minimum brushstroke radius (lower limit of the width when lightly touching), r max represents the maximum brushstroke radius (upper limit of the width when deeply pressing), α represents the exponential hardening coefficient, d represents the depth of the pen tip pressed into the paper surface, γ1 represents the magnification coefficient of the tilt on the minor axis, p represents the power of sinθ, used to adjust the minor axis tilt response curve, and θ represents the angle (rad) between the pen axis and the paper normal.

[0073] The calculation formula for the major axis length after tilt correction is:

[0074] r major =r minor (1 + γ2sin q θ)

[0075] where r major represents the major axis length, γ2 represents the magnification coefficient of the tilt on the major axis, and q represents the power of sinθ, used to adjust the major axis tilt response curve.

[0076] This application uses cone-plane intersection theory to analytically construct elliptical handwriting, controlling the minor and major axes via depth and tilt angle, respectively, to create a dynamic ellipse adjustment mechanism based on physical input. Compared to methods like CN103345773A, which fix the major axis of the ellipse at 1.5 times the circle radius, this invention extends the major axis to three times the minor axis when the tilt angle θ reaches 45°, demonstrating the stretching and deformation effect of realistic side-edge writing.

[0077] This algorithm has a fixed computational overhead per frame and no iteration process, and can be directly used for GPU elliptical decals. While maintaining a high frame rate, it ensures that the deformation of each stroke conforms to mechanics and writing logic, showing the natural transitions of the brushstrokes.

[0078] S32. Determine a contact area between the pen tip and the paper surface according to the ellipse parameters of the handwriting, where the contact area includes at least one contact point.

[0079] S33. Map the contact points to texture coordinates to generate an initial handwriting texture.

[0080] S4. Determine and generate a Feibai transparency mask according to the tilt angle, tangential velocity and vertical velocity.

[0081] The steps of generating a Feibai transparency mask according to the tilt angle, tangential velocity and vertical velocity include:

[0082] S41. Use random noise and directional stripes to construct a flying white mixed layer.

[0083] Reference Figure 3 In this embodiment, the Feibai mixed layer includes Feibai dot texture and band-like directional stripes; in this embodiment, Perlin noise texture is used to generate a highly random, spatially non-periodic Feibai dot texture to simulate the randomly distributed or naturally split fiber edges of ink; a sinusoidal stripe function is used to generate band-like directional stripes; in the elliptical local coordinate system, a periodic strip structure is generated along the main axis direction to express the "fibrous ribbon" Feibai that is commonly seen in high-speed movements such as left-falling, right-falling, and right-falling.

[0084] These "two" are presented in the fragment shader as image sampling and analysis functions, and then the blending factor α is used to stripes Do weighted averaging to form the final Feibai texture channel used for alpha mask judgment.

[0085] The overall effect is to retain the directionality of Feibai while having local irregularities, which is a fitting expression of the real Feibai texture in the visual space.

[0086] S42, calculating a Feibai factor according to the tilt angle, the tangential velocity, and the vertical velocity, where the Feibai factor represents the Feibai intensity of the current stroke;

[0087] The calculation formula of the stippling factor is as follows:

[0088]

[0089] Among them, F fly represents the stippling factor, W v represents the speed weight, v represents the numerical value of the instantaneous speed modulus of the pen tip, and the numerical value of the instantaneous speed modulus of the pen tip is a value obtained by non-dimensionalizing the instantaneous speed modulus of the pen tip; v0 represents the speed threshold, and W l represents the pen-lifting weight, represents the numerical value of the pressure depth change rate. The pressure depth change rate is negative when lifting the pen; W θ represents the tilt weight, θ represents the numerical value of the angle between the pen axis and the normal of the paper surface. In this embodiment, the angle uses the standard unit, and the numerical value of the angle is a value obtained by non-dimensionalizing the angle; θ0 represents the radian tilt threshold, and W side represents the deviation weight, s is the numerical value of the side deviation degree, s is used to represent the position of the major axis of the ellipse, and the side deviation degree value is a value obtained by non-dimensionalizing the side deviation degree. max(.) represents selecting the maximum value from the parentheses;

[0090] S43. Generate a control instruction based on the stippling factor to control the fragment shader to color the stippling mixing layer in response to the control instruction, and obtain the final stippling transparency mask. [[ID=2I4]]

[0091] Specifically, generating a control instruction based on the stippling factor to control the fragment shader to color the stippling mixing layer in response to the control instruction, and obtaining the final stippling transparency mask includes:

[0092] Calculate the dynamic stippling threshold according to the stippling factor;

[0093] Set a dynamic stippling threshold T fly , and its calculation method is:

[0094] T fly = T0 + T1·F fly

[0095] Among them: T fly represents the current stippling threshold, T0 is the reference threshold of the stippling mask, T1 is the adjustment amplitude coefficient; F fly is the stippling factor of the current frame.

[0096] The mask logic is: if the mixing value at a certain fragment position is lower than the threshold, the handwriting at that place becomes transparent, that is, stippling is formed.

[0097] The specific judgment rule is:

[0098]

[0099] Among them, αfinal It is the alpha channel value (transparency) finally applied to the fragment, which controls whether the pixel is displayed as solid ink or blank. The alpha channel is the fourth component in the image pixel, which is used to indicate the degree of transparency of the current pixel, and its value range is [0,1].

[0100] In this application, the alpha channel is used to:

[0101] Controls whether the handwriting appears in the paper image: alpha 0 means the pixel is completely transparent (i.e. "flying white"), and 1 means it is completely opaque (solid ink);

[0102] For each fragment of the Feibai blending layer, compare the fragment's noise stripe blending value with the dynamic Feibai threshold:

[0103] If the noise stripe mixed value of the fragment is less than or equal to the dynamic Feibai threshold, the transparency of the current fragment is set to 0, indicating Feibai; otherwise, the transparency of the current fragment is set to 1, indicating that there is an ink mark, and the control information is obtained;

[0104] A control instruction is generated according to the control information to control the fragment shader to adjust the transparency of the Feibai mask texture to obtain a final Feibai transparency mask.

[0105] Specifically, in the Feibai transparency mask synthesis process, the fragment color values are superimposed through alpha blending, presenting the effect of "transparent Feibai + background showing through" in real writing.

[0106] The "Kasure Energy Model" proposed by Takeda (2005) uses only a single energy difference threshold to determine Feibai (flying white) and employs a static template for image splicing. This model is unable to adjust handwriting texture in real time based on subtle changes in writing movements. This application, by implementing a dynamic Feibai threshold adjustment, achieves diverse and controllable visual representations of Feibai (flying white) phenomena.

[0107] Figure 4 In order to generate some virtual brush handwritings by using the force feedback virtual brush simulation method of this application, Figure 4 The effectiveness of the above mechanism can be verified: in the quick sweeping action of the left-stroke, the flying white stripes are born with directionality and continuity; and in the stage of lifting the pen at the end of the right-stroke, the flying white particles and the broken ink texture that appear are obviously dominated by the pen lifting speed.

[0108] In addition, when writing the horizontal stroke slowly and the entire character "永", the Feibai texture is consistent and naturally distributed, without any breaks or abrupt changes, proving that the Feibai mask of this system has stable performance in terms of inter-frame coherence.

[0109] S5. Process the initial brushstroke texture and the dry-brush transparency mask using a preset rule to obtain a virtual brushstroke with a dry-brush transparency mask effect.

[0110] Perform alpha channel composition on the dry-brush transparency mask and the initial brushstroke texture. The composition uses the pre-multiplied alpha blending mode to ensure that the ink color does not turn gray and there is no white border at the overlapping boundary when continuous brushstrokes are overlaid.

[0111] The formula for pre-multiplied transparency alpha blending is as follows:

[0112] C final = C ink ·α ink + C bg ·(1 - α ink )

[0113] where α ink is the final alpha channel value of the current fragment, which is controlled by the dry-brush transparency mask;

[0114] C final is the color output actually rendered to the frame buffer.

[0115] This blending method ensures the following rendering effects:

[0116] At the dry-brush position (α ink = 0), only the background is shown and the ink is not shown;

[0117] At the solid-ink position (α ink = 1), the background is completely obscured;

[0118] At the semi-transparent position (0 < α ink < 1), an intermediate transition zone between wet ink and dry ink is achieved.

[0119] In the preferred implementation of this embodiment, after obtaining the virtual brushstroke, perform alpha channel (alpha) composition on the background calligraphy copy texture (i.e., the paper) and the virtual brushstroke generated in the current frame; in this embodiment, the overall rendering uses the modern OpenGL pipeline and enables the Framebuffer Object (FBO) and texture rendering technology, supporting transparent backgrounds, dynamic blending, and layer reuse, allowing users to switch between multiple calligraphy copy background textures using keyboard shortcuts, and the system automatically falls back to a white paper when the calligraphy copy fails to load, ensuring the accuracy and detailed level performance of ink composition even at high frame rates.

[0120] In the preferred implementation of this embodiment, the force-feedback virtual brush simulation method further includes:

[0121] Calculate the non - linear force feedback information based on the current depression depth, tilt angle, tangential velocity, and vertical velocity of the writing brush tip, and send the non - linear force feedback information to the haptic device.

[0122] In this embodiment, a non - linear mechanical model is used to solve the feedback force vector to obtain the non - linear force feedback information. Specifically, the feedback force vector is calculated according to the following steps:

[0123] The first step is to determine whether it is in contact with the paper surface (depression depth d > 0).

[0124] If there is no contact (d = 0), then all force terms are set to zero, and the mechanical calculation is skipped;

[0125] If in contact, then perform the second to sixth steps in sequence and enter the calculation process.

[0126] The second step is to calculate the normal elastic force.

[0127] The normal elastic force reflects the exponential hardening elastic reaction force growth trend. Use the formula F spring = k n (e αd - 1) to calculate the normal elastic force, where F spring represents the exponential elastic force (N), the normal elastic response generated by the depression depth; k n represents the spring stiffness coefficient (N / m), α represents the exponential hardening coefficient (1 / m); d represents the depression depth (m), the depth at which the pen tip vertically penetrates the paper surface.

[0128] The third step is to calculate the normal damping force.

[0129] The normal damping force is used to suppress the jerks or oscillations when quickly pressing into the paper surface. According to the formula F damp = - b n ·v ⊥ calculate the normal damping force, where F damp represents the velocity damping force (N), that is, the damping force generated by the vertical velocity; b n represents the normal damping coefficient (Ns / m); v ⊥ represents the vertical velocity (m / s), that is, the movement velocity of the pen tip in the normal direction.

[0130] The fourth step is to synthesize the total normal force and limit the amplitude.

[0131] The total normal force combines the normal elastic force and the normal damping force. The expression of the total normal force is

[0132] F n = max(0, min(F spring + F damp , F max ))

[0133] Among them, F n represents the synthetic normal force (N), the sum of the elastic and damping forces, limited to the maximum output allowed by the device, F max represents the maximum force output allowed by the device (N), and min(.,.) represents taking the minimum value in the parentheses.

[0134] The purpose of synthesizing the total normal force and limiting it is to restrict the maximum output from exceeding the upper limit that the device can withstand.

[0135] Step 5: Calculate the tangential frictional force.

[0136] When calculating the tangential frictional force, first calculate the magnitude of the tangential force, then calculate the unit vector in the tangential direction, and finally obtain the total tangential force:

[0137] The formula for the tangential force is: F t = μF n + b t ·‖v tan ‖

[0138] The unit vector in the tangential direction is:

[0139] The total tangential force is:

[0140] Among them, F t represents the magnitude of the tangential frictional force (N), μ represents the friction coefficient, b t represents the tangential damping coefficient (Ns / m), is the tangential velocity vector (m / s), used to represent the velocity component of the pen tip in the plane of the paper; represents the magnitude of the tangential velocity vector; represents the unit vector in the tangential direction, opposite to the sliding direction; represents the total tangential force.

[0141] Step 6: Synthesize the final feedback force vector according to the total normal force and the total tangential force.

[0142] The final feedback force vector is:

[0143] Among them, represents the total feedback force vector (N), used to feedback to the haptic device; : the unit normal vector of the paper plane (usually (0, 1, 0)).

[0144] Step 7: Send the non - linear force feedback information to the haptic device.

[0145] Compared with the traditional linear spring model, the exponential hardening model (nonlinear mechanical model) adopted in the present invention can quickly generate a strong resistance within a small indentation range, simulating the stiffening process of a real pen tip under a force state; combined with velocity damping and tangential friction, it can effectively suppress virtual contact and oscillation, improving the stability and realism of haptic feedback.

[0146] For the convenience of those skilled in the art, the process of calculating the feedback force vector is described below in conjunction with specific examples.

[0147] Example 1: If the indentation depth d of the virtual writing brush in the current frame is 1.2 mm, the vertical velocity v ⊥ = -0.01 m / s, and the magnitude of the tangential velocity ||v tan || = 0.04 m / s, using the default parameters:

[0148] k n = 40.0 N / m, α = 3.0 m -1

[0149] b n = 0.005 Ns / m, b t = 0.005 Ns / m, μ = 0.005

[0150] F max = 3.0 N

[0151] Then it is calculated that:

[0152] F spring ≈ 0.53 N

[0153] F damp = 5.0 × 10 -5 N

[0154] F n ≈ 0.53005 N

[0155] F t ≈ 0.00615 N

[0156] The final feedback force vector is synthesized according to the total normal force and the total tangential force, and this three-dimensional force vector is input into the OpenHaptics interface to complete a haptic loop closed-loop at the 1 kHz level; at this time, the end of the haptic device will obtain a three-dimensional force vector feedback with a sense of elasticity and friction, and the user can clearly perceive the paper surface shape and action effect in terms of touch feeling.

[0157] Through multiple technological improvements in haptic feedback modeling, scribble texture generation, elliptical stroke control, and graphics rendering efficiency, this application is superior to the prior art in multiple dimensions such as realism, controllability, response rate, and immersion consistency, and has clear and verifiable technical effects, making it suitable for multiple scenarios such as high-fidelity virtual calligraphy, interactive art education, and immersive cultural and creative products.

[0158] Referring to Figure 5 , based on the same inventive concept, an embodiment of the present invention provides a force feedback virtual brush simulation system.

[0159] The force feedback virtual brush simulation system described in the present invention can be loaded into an electronic device. According to the functions achieved, the force feedback virtual brush simulation system includes a mechanics layer, a haptic layer, a stroke generation layer, and a rendering interaction layer, where:

[0160] The mechanics layer can receive the spatial state information of the haptic device corresponding to the virtual brush in real time, and analyze the attitude and motion state of the haptic device according to the spatial state information; the spatial state information includes the position, direction, and motion speed of the tip of the haptic device in the world coordinate system; the haptic device is a hardware device for providing force feedback when simulating brush writing.

[0161] The haptic layer can calculate non-linear force feedback information according to the spatial state information, and send the non-linear force feedback information to the haptic device.

[0162] The stroke generation layer can determine the elliptical parameters of the virtual brush stroke according to the position and pressure depth of the tip in the world coordinate system, and generate an initial stroke texture according to the elliptical parameters of the stroke; and generate a scribble transparency mask according to the tilt angle, tangential speed, and vertical speed.

[0163] The stroke generation layer includes an elliptical stroke calculation module, a scribble mask rendering module, and a dual-frame buffer (Ping-Pong FBO) texture cache module. The stroke generation layer is responsible for mapping the tip geometric state provided by the mechanics and haptic layers into the image representation of each frame of the stroke, and fusing the scribble texture effect for real-time calculation in the GPU.

[0164] The elliptical stroke calculation module derives the contact area shape of the current tip on the paper according to the geometric relationship between the spatial attitude of the pen hair cone and the intersection of the paper surface. Its output is the elliptical parameters to be drawn in the current frame, including the center position, the lengths of the major and minor axes, and the rotation angle.

[0165] The scribble mask rendering module dynamically adjusts the scribble transparency mask in the fragment shader according to real-time physical quantities (such as speed, pen lifting, tilt, etc.), combines random noise and directional stripes to generate an "ink-deficient" area, thereby synthesizing a realistic dry brush effect in each frame of the stroke. Its control logic is reflected by the scribble factor and the dynamic scribble threshold.

[0166] The dual-frame buffer (Ping-Pong FBO) texture cache module maintains two frame buffer textures on the GPU side. One frame serves as the current canvas (source), and the other frame is used to write the current frame's handwriting (destination), and the roles are swapped for each frame. Its function is to achieve frame-by-frame accumulation of handwriting without clearing the screen, while ensuring the coherence of the trailing effect and the stability of handwriting accumulation.

[0167] The above elliptical handwriting calculation module, trailing mask rendering module, and dual-frame buffer (Ping-Pong FBO) texture cache module are connected in series to complete the complete handwriting generation pipeline of "geometric form definition → visual feature modulation → cumulative rendering output".

[0168] The rendering interaction layer can process the initial handwriting texture and the trailing transparency mask using preset rules to obtain a virtual brush handwriting with a trailing transparency mask effect.

[0169] All trailing determination, elliptical projection, and handwriting update in the system are completed on the GPU. The dual-FBO mechanism is used to achieve seamless accumulation of handwriting textures between frames. The rendering time for each frame is less than 0.3ms, and the real-time rendering ability of 300FPS can be maintained, and no additional delay is generated due to the trailing mask or stroke deformation.

[0170] At the same time, the tactile thread and the graphics thread are decoupled through data buffering and pose, maintaining 1kHz and a high frame rendering frequency respectively, fully avoiding the "frame rate mutual jamming" phenomenon in traditional systems, and ensuring tactile response and visual feedback. All handwriting rendering parameters in the present invention are uniformly driven by tactile inputs (pressure depth, speed, tilt), maintaining the "homologous mapping" logic in the encoding and rendering pipelines. The pen force and movement changes applied by the user during writing not only change the stroke thickness and shape but also synchronously affect the ink color shade and trailing texture, truly realizing the integrated dynamic response of force perception, geometry, and visual texture.

[0171] The module described in the present invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can complete fixed functions, and are stored in the memory of the electronic device.

[0172] The various change methods and specific examples in the force feedback virtual brush simulation method provided in the above embodiments are equally applicable to the force feedback virtual brush simulation system in this embodiment. Through the detailed description of the force feedback virtual brush simulation method above, those skilled in the art can clearly know the implementation method of the force feedback virtual brush simulation system in this embodiment. For the sake of simplicity of the specification, it will not be elaborated here.

[0173] This application also discloses an electronic device, such as Figure 6As shown, it is a schematic structural diagram of an electronic device for a force feedback virtual writing brush simulation method provided by an embodiment of the present invention. The electronic device may include at least one processor 10, a memory 11 communicatively connected to the at least one processor, a communication bus 12, and a communication interface 13. It may also include a computer program stored in the memory 11 and executable on the processor 10, such as a method program for force feedback virtual writing brush simulation.

[0174] Among them, the processor 10 may be composed of integrated circuits in some embodiments. For example, it may be composed of a single packaged integrated circuit, or may be composed of multiple integrated circuits with the same or different functions, including a combination of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control core (Control Unit) of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. By running or executing programs or modules stored in the memory 11 (such as executing the method of force feedback virtual writing brush simulation, etc.), and calling data stored in the memory 11, it performs various functions of the electronic device and processes data.

[0175] The memory 11 includes at least one type of readable storage medium. The readable storage medium includes flash memory, mobile hard disks, multimedia cards, card-type memories (such as SD or DX memories, etc.), magnetic memories, magnetic disks, optical discs, etc. The memory 11 may be an internal storage unit of the electronic device in some embodiments, such as the mobile hard disk of the electronic device. The memory 11 may also be an external storage device of the electronic device in other embodiments, such as a plug-in mobile hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device. Further, the memory 11 may include both the internal storage unit and the external storage device of the electronic device. The memory 11 can not only be used to store application software installed on the electronic device and various types of data, such as the code of the method program for force feedback virtual writing brush simulation, but also be used to temporarily store data that has been output or will be output.

[0176] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable connection communication between the memory 11 and at least one processor 10, etc.

[0177] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface can include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between this electronic device and other electronic devices. The user interface can be a display, an input unit (such as a keyboard), and optionally, the user interface can also be a standard wired interface or a wireless interface. Optionally, in some embodiments, the display can be an LED display, a liquid crystal display, a touch liquid crystal display, and an OLED (Organic Light-Emitting Diode) toucher, etc. Among them, the display can also be appropriately referred to as a display screen or a display unit, which is used to display the information processed in the electronic device and to display a visual user interface.

[0178] Figure 6 Only the electronic device with components is shown. Those skilled in the art can understand that Figure 6 the shown structure does not constitute a limitation on the electronic device, and it can include fewer or more components than shown, or combine certain components, or have a different component arrangement. For example, although not shown, the electronic device can also include a power source (such as a battery) for powering each component. Preferably, the power source can be logically connected to at least one processor 10 through a power management device, so as to implement functions such as charge management, discharge management, and power consumption management through the power management device. The power source can also include any components such as one or more DC or AC power sources, a recharge device, a power failure detection circuit, a power converter or an inverter, and a power status indicator. The electronic device can also include various sensors, a Bluetooth module, a Wi-Fi module, etc., which will not be elaborated here.

[0179] It should be understood that the embodiments are only for illustration purposes and are not limited by this structure in the scope of the patent application.

[0180] Furthermore, if the modules / units integrated in the electronic device are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile.

[0181] An embodiment of the present application provides a computer-readable storage medium, for example, including: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disc, a computer memory, a read-only memory (ROM, Read-Only Memory). The computer-readable storage medium stores a computer program that can be loaded and executed by a processor to perform the force feedback virtual brush simulation method in the above embodiment.

[0182] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", "one implementation manner", "one preferred implementation manner" or "some examples", etc. means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0183] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the claims and their equivalents.

Claims

1. A force feedback virtual writing brush simulation method, characterized in that, The method includes: Obtaining the spatial state information of the haptic device corresponding to the virtual writing brush in real time; the spatial state information includes the position, direction, and movement speed of the pen tip of the haptic device in the world coordinate system; the haptic device is a hardware device for providing force feedback data when simulating writing with a writing brush. Adjusting the pose of the virtual writing brush according to the spatial state information to determine the pressing depth, tilt angle, tangential speed, and vertical speed of the pen tip of the virtual writing brush. Determining the elliptical parameters of the virtual writing brush stroke according to the position and pressing depth of the pen tip in the world coordinate system, and generating an initial stroke texture according to the elliptical parameters of the stroke. Generating a flying white transparency mask according to the tilt angle, tangential speed, and vertical speed. Processing the initial stroke texture and the flying white transparency mask using a preset rule to obtain a virtual writing brush stroke with a flying white transparency mask effect.

2. The force feedback virtual writing brush simulation method according to claim 1, wherein The virtual writing brush is represented by a rigid body writing brush model, and the rigid body writing brush model includes a cylindrical pen rod rigid body and a conical pen tip rigid body.

3. The force feedback virtual writing brush simulation method according to claim 2, wherein The adjusting the pose of the rigid body writing brush model according to the spatial state information to determine the current pressing depth, tilt angle, tangential speed, and vertical speed of the pen tip of the virtual writing brush includes: Deriving the local transformation information of the rigid body writing brush model according to the spatial state information, and the local transformation information includes the offset paths of the pen rod rigid body and the pen tip rigid body in the local coordinate system. Adjusting the pose of the rigid body writing brush model according to the local transformation information to determine the positions and orientations of the pen rod rigid body and the pen tip rigid body in the world coordinate system after adjustment, and obtaining adjustment information. Calculating the pressing depth, tilt angle, tangential speed, and vertical speed of the pen tip according to the adjustment information.

4. The force feedback virtual brush simulation method according to claim 1, wherein The determining the elliptical parameters of the virtual writing brush stroke according to the position and pressing depth of the pen tip in the world coordinate system, and generating an initial stroke texture according to the elliptical parameters of the stroke includes: Determining the elliptical parameters of the stroke according to the position and pressing depth of the cone corresponding to the pen tip in the world coordinate system. Determining the contact area between the pen tip and the paper surface according to the elliptical parameters of the stroke; the contact area includes at least one contact point. Mapping the contact points to texture coordinates to generate an initial stroke texture.

5. The force feedback virtual brush simulation method according to any one of claims 1 to 4, characterized in that, The flying white transparency mask represents the texture of the ink-deficient area of the virtual writing brush, and the generating a flying white transparency mask according to the tilt angle, tangential speed, and vertical speed includes: Constructing a flying white blending layer using random noise and directional stripes. Calculating a flying white factor according to the tilt angle, tangential speed, and vertical speed, and the flying white factor represents the flying white intensity of the current stroke. Generating a control instruction based on the flying white factor to control the fragment shader to color the flying white blending layer in response to the control instruction to obtain the final flying white transparency mask.

6. The force feedback virtual brush simulation method according to claim 5, characterized in that The calculation formula of the flying white factor is: Among them, F fly represents the stippling factor, W v represents the speed weight, v represents the instantaneous speed modulus of the pen tip, v0 represents the speed threshold, W l represents the pen-lifting weight, represents the pressure depth change rate, and the pressure depth change rate is negative when lifting the pen, W θ represents the tilt weight, θ represents the angle between the pen axis and the normal of the paper surface, θ0 represents the radian tilt threshold, W side represents the deviation weight, s is the deviation degree on one side, s is used to indicate which side the major axis of the ellipse is located on, and max(.) represents selecting the maximum value from the parentheses; The generating a control instruction based on the flying white factor to control the fragment shader to color the flying white blending layer in response to the control instruction to obtain the final flying white transparency mask includes: Calculating a dynamic flying white threshold according to the flying white factor. For each fragment of the flying white blending layer, comparing the noise stripe blending value of the fragment with the dynamic flying white threshold: If the noise stripe blending value of the fragment is less than or equal to the dynamic flying white threshold, setting the transparency degree of the current fragment to 0, indicating flying white; otherwise, setting the transparency degree of the current fragment to 1, indicating an ink mark, to obtain control information. Generate a control instruction according to the control information to control the fragment shader to adjust the transparency of the scribble mask texture, and obtain the final scribble transparency mask.

7. The force feedback virtual brush simulation method according to claim 1 or 2 or 3 or 4 or 6, characterized in that, The spatial state information of the virtual writing brush is collected by a haptic device, and the method further includes: Calculating non-linear force feedback information based on the current pressing depth, tilt angle, tangential velocity, and vertical velocity of the writing brush tip, and sending the non-linear force feedback information to the haptic device.

8. A force feedback virtual writing brush simulation system for implementing the force feedback virtual writing brush simulation method according to any one of claims 1 to 7, characterized in that, Including: The mechanical layer is used to receive the spatial state information of the haptic device corresponding to the virtual writing brush in real time, and analyze the posture and motion state of the haptic device according to the spatial state information; The spatial state information includes the position, direction, and motion speed of the writing brush tip of the haptic device in the world coordinate system; The haptic device is a hardware device for providing force feedback when simulating writing with a writing brush; The haptic layer is used to calculate non-linear force feedback information according to the spatial state information, and send the non-linear force feedback information to the haptic device; The handwriting generation layer is used to determine the elliptical parameters of the virtual writing brush handwriting according to the position and pressing depth of the writing brush tip in the world coordinate system, and generate an initial handwriting texture according to the elliptical parameters of the handwriting; And generate a scribble transparency mask according to the tilt angle, tangential velocity, and vertical velocity; The rendering interaction layer is used to process the initial handwriting texture and the scribble transparency mask using a preset rule to obtain a virtual writing brush handwriting with a scribble transparency mask effect.

9. An electronic device, characterized in that, The electronic device includes: At least one processor (10); and, A memory (11) communicatively connected to the at least one processor (10); Wherein, the memory (11) stores a computer program executable by the at least one processor (10), and the computer program is executed by the at least one processor (10) so that the at least one processor (10) can execute the force feedback virtual writing brush simulation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program; when the computer program is executed by a processor, it implements the force feedback virtual writing brush simulation method according to any one of claims 1 to 7.

Images