Stylus, stylus writing trajectory generation method, device and computer equipment

Abstract

This application relates to a stylus, a method and apparatus for generating a stylus writing trajectory, a computer device, a computer-readable storage medium, and a computer program product. The method includes: acquiring multiple frames of pen stroke images captured during stylus writing, and the writing pressure corresponding to each frame of pen stroke images; extracting texture information of the writing medium from each frame of pen stroke images; determining the movement path of the stylus on the writing medium based on the correspondence between the texture information of each frame of pen stroke images; for each frame of pen stroke images, determining the line width of the corresponding segment of the movement path based on the writing pressure corresponding to the current pen stroke image; wherein the line width of the line width is positively correlated with the writing pressure; and generating the writing trajectory of the stylus on the writing medium based on the line width of each segment of the movement path. This method can improve the flexibility of interaction.

Description

Technical Field

[0001] This application relates to the field of intelligent interactive technology, and in particular to a stylus, a method and apparatus for generating stylus writing trajectories, a computer device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] With the rapid development of smart interactive devices and mobile office scenarios, users are increasingly demanding convenience, flexibility, and low cost of stylus writing.

[0003] However, existing mainstream positioning technologies have significant limitations: electromagnetic induction technology relies on a specially designed writing board, using an array of electromagnetic coils to sense the pen tip's position. This hardware coupling method not only increases the device's size and cost but also restricts the writing area to a fixed surface. More importantly, these solutions all depend on external dedicated equipment for positioning, preventing the stylus from being used independently of the accompanying hardware. This severely restricts the real-time writing experience in mobile scenarios and results in poor interactive flexibility. Summary of the Invention

[0004] Therefore, it is necessary to provide a stylus, a stylus writing trajectory generation method, a device, a computer device, a computer-readable storage medium, and a computer program product that can improve the flexibility of interaction in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a method for generating a stylus writing trajectory, including:

[0006] Acquire multiple frames of pen stroke images captured during the writing process of the stylus, as well as the writing pressure corresponding to each frame of the pen stroke image;

[0007] The texture information of the writing medium is extracted from each frame of the brushstroke image;

[0008] Based on the correspondence of texture information between the pen stroke images in each frame, the movement path of the stylus on the writing medium is determined;

[0009] For each frame of the pen stroke image, based on the writing pressure corresponding to the current pen stroke image, the line width of the corresponding segment in the movement path is determined; wherein, the line width of the segment is positively correlated with the writing pressure; and

[0010] Based on the line width of each segment in the movement path, the writing trajectory of the stylus on the writing medium is generated.

[0011] In one embodiment, determining the movement path of the stylus on the writing medium based on the texture information correspondence between the stroke images of each frame includes:

[0012] For each frame of the pen stroke image, obtain the texture information of the current pen stroke image and the texture information of the previous frame of the pen stroke image; match the texture information of the current pen stroke image with the texture information of the previous frame of the pen stroke image to determine the position of the same texture feature point in the writing medium in the current pen stroke image and the previous frame of the pen stroke image respectively.

[0013] Based on the determined positions, the movement segment of the stylus between the current stroke image and the previous stroke image is determined; and

[0014] By piecing together the various moving segments, the movement path of the stylus on the writing medium is obtained.

[0015] In one embodiment, the method further includes:

[0016] Acquire multiple frames of infrared light spot images captured during the writing process of the stylus; wherein, each of the multiple frames of infrared light spot images corresponds one-to-one with the multiple frames of pen stroke images;

[0017] For each frame of the infrared spot image, the tilt angle of the stylus in the current infrared spot image is determined based on the degree of ellipse fitting of the current infrared spot image.

[0018] The movement path is corrected based on each of the aforementioned tilt angles to obtain an updated movement path; and

[0019] Based on the line width of each segment in the updated movement path, the writing trajectory of the stylus on the writing medium is generated.

[0020] In one embodiment, determining the tilt angle of the stylus in the current infrared spot image based on the ellipse fitting degree of the current infrared spot image for each frame of the infrared spot image includes:

[0021] For each frame of the infrared spot image, the current infrared spot image is binarized to extract the edge contour of the infrared spot in the current infrared spot image; and

[0022] Based on the degree of ellipse fitting represented by the edge contour, the tilt angle of the stylus in the current infrared spot image is determined.

[0023] In one embodiment, the method further includes:

[0024] Obtain the ambient light intensity of the environment in which the stylus is located;

[0025] Perform quality analysis on each frame of the brushstroke image to determine the quality score corresponding to the current brushstroke image;

[0026] Based on the ambient light intensity and the quality score, determine the fill light information for the current brushstroke image; and

[0027] The writing medium is illuminated according to the illumination information.

[0028] In one embodiment, extracting the texture information of the writing medium from each frame of the pen stroke image includes:

[0029] Obtain the image extraction parameters used to extract texture information;

[0030] For each frame of the pen stroke image, if the writing pressure corresponding to the current pen stroke image does not meet the distortion condition, the texture information of the writing medium is extracted from the current pen stroke image based on the image processing parameters;

[0031] If the writing pressure corresponding to the current pen stroke image satisfies the distortion condition, the image processing parameters are adjusted according to the writing pressure to obtain updated processing parameters; and

[0032] Based on the updated processing parameters, the texture information of the writing medium is extracted from the current pen stroke image.

[0033] Secondly, this application also provides a stylus, which includes an image sensing device, a pressure sensing device, and a control device; the control device is connected to the image sensing device and the pressure sensing device.

[0034] The image sensing device is used to acquire multiple frames of pen stroke images during the writing process of the stylus;

[0035] The pressure sensing device is used to collect the writing pressure corresponding to each frame of the pen stroke image;

[0036] The control device is used to implement the stylus writing trajectory generation method described in any of the above embodiments.

[0037] In one embodiment, the stylus further includes a fill light device connected to the control device;

[0038] The supplemental lighting device is used to provide supplemental lighting for the writing medium;

[0039] The control device is also used to acquire the ambient light intensity of the environment in which the stylus is located;

[0040] Perform quality analysis on each frame of the brushstroke image to determine the quality score corresponding to the current brushstroke image;

[0041] Based on the ambient light intensity and the quality score, determine the fill light information for the current brushstroke image; and

[0042] The supplementary lighting device is controlled to provide supplementary lighting to the writing medium according to the supplementary lighting information.

[0043] Thirdly, this application also provides a stylus writing trajectory generation device, comprising:

[0044] The image acquisition module is used to acquire multiple frames of pen stroke images captured during the writing process of the stylus, as well as the writing pressure corresponding to each frame of the pen stroke image;

[0045] The texture information extraction module is used to extract the texture information of the writing medium from each frame of the pen stroke image;

[0046] The movement path determination module is used to determine the movement path of the stylus on the writing medium based on the texture information correspondence between the pen stroke images in each frame;

[0047] The road segment width determination module is used to determine the road segment width corresponding to the current stroke image in the movement path based on the writing pressure corresponding to the current stroke image for each frame of the stroke image; wherein, the road segment width is positively correlated with the writing pressure; and

[0048] The writing trajectory generation module is used to generate the writing trajectory of the stylus on the writing medium based on the line width of each segment in the movement path.

[0049] Fourthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described above.

[0050] Fifthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the method described above.

[0051] Sixthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described above.

[0052] The aforementioned stylus, stylus writing trajectory generation method, device, computer equipment, computer-readable storage medium, and computer program product determine the microscopic texture features of the writing medium by dynamically acquiring continuous multi-frame stylus stroke images and their corresponding pressure data during stylus writing, providing a precise medium adaptation basis for subsequent trajectory analysis. Secondly, based on the spatiotemporal correlation of texture between each frame of stylus stroke images, the movement path of the stylus on the writing medium is determined, effectively overcoming the trajectory breakpoint problem that easily occurs in traditional single-frame independent processing mode in fast writing or continuous writing scenarios, ensuring the continuous restoration of complex handwriting. Furthermore, by constructing a pressure-sensing-based dynamic line width adjustment mechanism, the real-time acquired pressure data is mapped to the gradual line width parameters of the trajectory segments, enabling the generated digital handwriting to accurately reproduce the complete thickness change process from light touch to heavy pressure during handwriting. Finally, based on the line width of each segment in the movement path, the writing trajectory of the stylus on the writing medium is generated, achieving a natural and realistic writing effect on different material surfaces such as paper, screen, and leather, significantly improving the flexibility of interaction. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is an application environment diagram of a stylus writing trajectory generation method in one embodiment;

[0055] Figure 2 This is a flowchart illustrating a method for generating a stylus writing trajectory in one embodiment;

[0056] Figure 3 This is a diagram illustrating the writing process of a stylus in one embodiment;

[0057] Figure 4 This is a schematic diagram of the previous frame of a stylus writing process in one embodiment;

[0058] Figure 5 This is a schematic diagram of the next frame of a stylus writing process in one embodiment;

[0059] Figure 6 This is a schematic diagram of an infrared light spot in one embodiment;

[0060] Figure 7 This is a schematic diagram of the infrared spot in another embodiment;

[0061] Figure 8This is a schematic diagram of an infrared spot image in one embodiment;

[0062] Figure 9 This is a schematic diagram of the stylus structure in one embodiment;

[0063] Figure 10 This is a flowchart illustrating the method for generating stylus writing trajectories in another embodiment;

[0064] Figure 11 This is a structural block diagram of a stylus writing trajectory generation device in one embodiment;

[0065] Figure 12 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0067] The stylus writing trajectory generation method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, the control device 102 is connected to the image sensing device 104 and the pressure sensing device 106 via a network. The image sensing device 104 is a miniature optical acquisition device integrated into the tip or front of the stylus, typically consisting of a high-resolution camera, lens assembly, and image processing chip, used to capture microscopic images of the area where the pen tip contacts the writing medium in real time. The pressure sensing device 106 is a mechanical sensing device embedded in the stylus tip, typically employing a piezoresistive, capacitive, or piezoelectric sensor, used to detect pressure changes when the pen tip contacts the writing medium. The control device 102 is the core processing unit of the stylus, typically composed of a low-power microcontroller or application-specific integrated circuit, responsible for coordinating the acquisition, processing, and transmission of image and pressure data. Specifically, during the generation of the stylus writing trajectory, the control device 102 acquires multiple frames of pen stroke images captured during the stylus writing process from the image sensor 104 and the pressure sensor 106, as well as the writing pressure corresponding to each frame of pen stroke images. The pen stroke images are acquired when the stylus tip is in contact with the writing medium. Texture information of the writing medium is extracted from each frame of pen stroke images. Based on the correspondence between the texture information of each frame of pen stroke images, the movement path of the stylus on the writing medium is determined. For each frame of pen stroke images, based on the writing pressure corresponding to the current pen stroke image, the line width of the corresponding segment in the movement path is determined. The line width of the segment is positively correlated with the writing pressure. Based on the line width of each segment in the movement path, the writing trajectory of the stylus on the writing medium is generated.

[0068] In one exemplary embodiment, such as Figure 2 As shown, a method for generating stylus writing trajectories is provided, which can be applied to... Figure 1 Taking the control device 102 as an example, the explanation includes the following steps S202 to S210. Wherein:

[0069] Step S202: Obtain multiple frames of pen stroke images captured during the writing process, as well as the writing pressure corresponding to each frame of pen stroke images.

[0070] The pen stroke image is captured when the tip of the stylus is in contact with the writing medium. A stylus is an input tool used for writing or drawing on a writing medium, such as... Figure 3 The diagram illustrates the writing process of a stylus on a writing medium. The writing process refers to the complete action of a user holding the stylus on the writing medium, from the initial touch to the final lift. Each frame of the pen stroke image refers to a series of static images continuously captured or photographed by an image sensor or dedicated sensor array on the device during the writing process. Each frame represents a snapshot of the pen tip's contact area at a very short point in time. The higher the frame rate, the finer the temporal resolution. For example, the image sensor could be a miniature camera module integrated at the tip of the stylus, facing the writing surface, capable of continuously capturing images of the area where the pen tip contacts the writing surface. When the user writes on any surface, the camera continuously captures the pen tip's movement trajectory and the texture features of the surrounding paper surface at a high frame rate (e.g., 120fps or higher).

[0071] A pen stroke image specifically refers to the image captured when the tip of a stylus is in direct contact with the writing medium. This image does not depict the pen itself, but rather the small area surrounding the point of contact. Writing pressure refers to the magnitude of the instantaneous vertical force exerted by the stylus tip on the surface of the writing medium. It is typically measured by a pressure sensor within the pen. The writing medium is the surface that the stylus directly contacts and writes on, such as paper, screen protectors, and electronic paper displays.

[0072] Specifically, when a user writes or draws on a writing surface using a stylus, the control device initiates a data acquisition process in real time. Within a very short time window, the control device simultaneously records two types of key data: visual data, namely "frames of pen stroke images" acquired through image sensing devices. This series of images precisely captures every instantaneous state left by the pen tip as it glides across the surface of the medium, provided that the pen tip remains in contact with the medium; and physical data, namely the "writing pressure" value precisely corresponding to each frame of the image, provided by the pressure sensing device inside the stylus. The control device can timestamp each frame of the image and bind it to the pressure value at the same moment, forming a synchronized data sequence to provide raw material for subsequent analysis.

[0073] Step S204: Extract the texture information of the writing medium from each frame of pen stroke images.

[0074] The texture information of a writing medium refers to the inherent microstructural features of its surface. No two sheets of paper are exactly alike; their fiber arrangement and uneven microstructure constitute a unique "texture." Under pen pressure, the surface of the medium undergoes minute deformations. These deformations manifest as specific light and shadow patterns or pixel distribution patterns in the pen stroke image—this is the texture information that needs to be extracted, similar to the medium's "fingerprint." For example, extracting the texture information of a writing medium can be achieved through computer vision or image processing algorithms, separating, identifying, and quantifying key features representing the medium's texture from the original image data (e.g., by analyzing local brightness variations, gradient directions, and signals of specific frequencies in the image).

[0075] Specifically, after acquiring each frame of pen stroke images, the data analysis stage begins. The goal of this step is to extract irrelevant information from each image, retaining only the core features that represent the writing medium itself. In practice, the control device scans and analyzes each frame of pen stroke images. The pressure generated when the pen tip contacts the writing medium causes changes in the microstructure of the medium's surface, resulting in unique grayscale or texture patterns in the image. The control device captures these patterns and converts them into a set of quantifiable mathematical descriptors—a process similar to fingerprint extraction. Regardless of where the pen tip moves, as long as it's on the same medium, the extracted texture features exhibit inherent consistency, providing reliable tracking markers for the next step of tracking pen movement.

[0076] In some specific embodiments, the control device can perform frequency domain transformation on each frame of pen stroke images, converting the image from the spatial domain to the frequency domain. In the frequency spectrum, the texture information of the writing medium is concentrated in specific frequency regions. By designing bandpass filters to extract these frequency components, and then reconstructing a clean texture image through inverse transformation, the texture information of the writing medium can be obtained.

[0077] In other specific embodiments, the control device can also acquire image extraction parameters for extracting texture information. For each frame of pen stroke image, if the writing pressure corresponding to the current pen stroke image does not meet the distortion condition, the texture information of the writing medium is extracted from the current pen stroke image based on the image processing parameters. If the writing pressure corresponding to the pen stroke image meets the distortion condition, the image processing parameters are adjusted according to the writing pressure to obtain updated processing parameters. Based on the updated processing parameters, the texture information of the writing medium is extracted from the current pen stroke image.

[0078] Step S206: Determine the movement path of the stylus on the writing medium based on the correspondence of texture information between each frame of pen stroke images.

[0079] In this context, texture information correspondence refers to identifying the common, identical medium texture regions covered by images acquired at different time points. For example, if a unique texture A is found in the upper left corner of the image at time point T1, and the exact same texture A is also found in the center of the image at time point T2, this establishes a correspondence. For instance, as shown... Figure 4 The image shown is a schematic diagram of the previous frame's brushstroke image. Figure 5 This is a schematic diagram of the pen stroke image for the current frame. By analyzing the positional changes of this shared texture in the image, the direction and distance of the pen tip's movement can be calculated. The movement path refers to the trajectory line formed by countless consecutive points as the pen tip moves across the surface of the writing medium. It describes the writing sequence and spatial orientation.

[0080] Specifically, the control device can compare and analyze two or more consecutive frames of pen stroke images. Because the texture of the medium surface is constant, while the pen position is moving, the same textured area will appear at different pixel positions in images at different times. By finding and matching these "texture information correspondences," it's like piecing together a picture. Figure 1Similarly, the displacement vector of the texture block between two frames is calculated. By concatenating the displacements calculated between each pair of frames, the position points traversed by the stylus on the writing medium throughout the entire process from pen placement to pen release can be constructed, ultimately forming a complete and continuous "movement path." This is essentially an optical navigation technology based on the microscopic characteristics of the medium itself. For example, the control device can determine the stylus's movement path on the writing medium based on the correspondence of texture information between adjacent pen stroke images, or it can determine the stylus's movement path on the writing medium based on the correspondence of texture information between non-adjacent pen stroke images. For example, non-adjacent pen stroke images can be separated by one pen stroke image, two pen stroke images, etc., and the specific number of interval images can be determined based on the actual situation.

[0081] Step S208: For each frame of pen stroke image, based on the writing pressure corresponding to the current pen stroke image, determine the line width of the road segment corresponding to the current pen stroke image in the movement path.

[0082] The line width of a writing segment is positively correlated with writing pressure; that is, as writing pressure increases, the calculated line width increases, and as pressure decreases, the line width decreases. A segment refers to the result of dividing a continuous movement path into segments. For example, each frame of an image can be considered a key node of a segment, and the path between that point and the next point is a "segment." A segment represents the range of pen movement within a very short time. The line width of a segment is the visual thickness value assigned to a specific segment on the movement path. It determines the width of the final displayed line at that local location.

[0083] Specifically, the control device traverses every point or segment along the movement path and retrieves the writing pressure at the corresponding time points. Based on the principle that "segment width is positively correlated with writing pressure," a mapping calculation is performed: the greater the pressure, the larger the line width allocated to that segment; the smaller the pressure, the smaller the line width. This mapping can be a simple linear proportional relationship or a more complex non-linear curve to simulate the ink diffusion effect caused by different pen pressures during actual writing. Ultimately, the movement path is no longer a geometric line with only positional information, but a segmented line with different thickness attributes.

[0084] Step S210: Based on the line width of each segment in the movement path, generate the writing trajectory of the stylus on the writing medium.

[0085] The writing trajectory is the final, visual result presented to the user. It is a line that reflects changes in pressure, speed, and stroke order during the writing process.

[0086] Specifically, using the movement path determined in step S206 as the skeleton and the "segment width of each road segment" calculated in step S208 as the stroke attribute, the rendering engine can generate a smooth and continuous visual line on the display screen or in the data file based on this information. This line not only accurately reproduces the trajectory of the pen tip on the medium (stroke order and direction), but also vividly restores the pressure at every moment during the writing process through the variation in the thickness of the line itself, thus realistically reproducing the user's handwriting on the electronic device.

[0087] The aforementioned method dynamically acquires multiple consecutive frames of pen stroke images and their corresponding pressure data during stylus writing to determine the microscopic texture features of the writing medium, providing a precise medium adaptation basis for subsequent trajectory analysis. Secondly, based on the spatiotemporal correlation of textures between each frame of pen stroke images, the movement path of the stylus on the writing medium is determined, effectively overcoming the trajectory breakpoint problem that easily occurs in traditional single-frame independent processing mode in fast writing or continuous writing scenarios, ensuring the continuous restoration of complex handwriting. Furthermore, by constructing a pressure-sensing-based dynamic line width adjustment mechanism, the real-time acquired pressure data is mapped to the gradual line width parameters of the trajectory segments, enabling the generated digital handwriting to accurately reproduce the complete thickness change process from light touch to heavy pressure during handwriting. Finally, based on the line width of each segment in the movement path, the writing trajectory of the stylus on the writing medium is generated, achieving a natural and realistic writing effect on different material surfaces such as paper, screen, and leather, significantly improving the flexibility of interaction.

[0088] In an exemplary embodiment, determining the movement path of the stylus on the writing medium based on the correspondence of texture information between each frame of pen stroke images includes: for each frame of pen stroke images, obtaining the texture information of the current pen stroke image and the texture information of the previous frame of pen stroke image; matching the texture information of the current pen stroke image with the texture information of the previous frame of pen stroke image to determine the positions of the same texture feature points in the writing medium in the current pen stroke image and the previous frame of pen stroke image respectively; based on the determined positions, determining the movement segment of the stylus between the current pen stroke image and the previous frame of pen stroke image; and splicing the movement segments to obtain the movement path of the stylus on the writing medium.

[0089] The previous frame of the pen stroke image is the image captured chronologically before the current pen stroke image. The texture information of the previous frame is the texture features extracted from the previous frame. Texture feature points are distinctive local feature points in the texture information, such as fiber intersections. Position refers to the coordinates of the texture feature points in the image coordinate system. The movement segment is the distance and direction of the pen tip's movement between two adjacent frames. The movement path is the complete trajectory of the pen tip on the writing medium.

[0090] Specifically, in this embodiment, the control device calculates the pen tip movement path using adjacent frame matching. After acquiring multiple frames of pen stroke images and extracting texture information from them, starting from the second frame, the texture information of the current pen stroke image and the texture information of the previous frame are acquired, and then compared and matched. Specifically, the region in the previous frame that is most similar to the texture features of the current pen stroke image is searched, thus determining the position of the same texture feature point in both the previous and current frames. The coordinate difference between these two positions is the displacement vector of the pen tip between these two frames, i.e., the movement segment. By calculating the movement segments between all adjacent frames, the control device connects these segments sequentially to obtain the complete movement path from pen placement to pen lifting. This method is similar to the principle of visual odometry, which calculates the trajectory by continuously tracking fixed feature points on the surface of the medium.

[0091] In this embodiment, the movement path is determined by matching the textures of adjacent frames, achieving high-precision pen tip trajectory tracking without relying on additional positioning equipment and with strong adaptability to writing media.

[0092] In an exemplary embodiment, the method further includes: acquiring multiple frames of infrared spot images captured by the stylus during the writing process; wherein the multiple frames of infrared spot images correspond one-to-one with multiple frames of pen stroke images; for each frame of infrared spot image, determining the tilt angle of the stylus in the current infrared spot image based on the degree of ellipse fitting of the current infrared spot image; correcting the movement path based on each tilt angle to obtain an updated movement path; and generating the writing trajectory of the stylus on the writing medium based on the line width of each segment in the updated movement path.

[0093] The infrared spot image is an image captured by an infrared sensor, showing the reflected light spot formed near the pen tip by an infrared light source. Ellipse fitting degree is the degree of match between the shape of the infrared spot and a standard ellipse model. Tilt angle is the degree of tilt of the stylus relative to the normal direction of the writing medium. Updated movement path is the new path obtained after tilt angle correction.

[0094] Specifically, while capturing the pen stroke image, an infrared light spot image is also captured simultaneously. The infrared light spot is formed by the infrared light source on the stylus illuminating the writing medium, such as... Figure 6 and Figure 7As shown, when the pen is vertical, the light spot is a perfect circle; when the pen is tilted, the light spot becomes elliptical. Here, b is the length of the major axis and a is the length of the minor axis. For each frame of the infrared light spot image, the edge contour of the light spot region is first extracted, and then this contour is fitted to a standard ellipse. By analyzing parameters such as the direction of the major axis and the ratio of the major and minor axes of the ellipse, the tilt angle of the stylus at that moment can be deduced. After obtaining the tilt angle, the previously determined movement path is corrected: since there is an offset between the actual contact point of the pen tip and the center point of the image, tilting will cause this offset to change. Therefore, it is necessary to adjust the position coordinates of each path point according to the tilt angle to obtain an updated movement path. Finally, a writing trajectory is generated based on this corrected path and the line width of each segment.

[0095] This embodiment calculates the pen's tilt angle by fitting the ellipse of the infrared light spot and corrects the movement path, effectively eliminating the positional deviation caused by the pen's tilt and making the generated writing trajectory closer to the real pen tip movement.

[0096] In an exemplary embodiment, for each frame of infrared spot image, the tilt angle of the stylus in the current infrared spot image is determined based on the degree of ellipse fitting of the current infrared spot image, including: for each frame of infrared spot image, performing binarization processing on the current infrared spot image to extract the edge contour of the infrared spot in the current infrared spot image; and determining the tilt angle of the stylus in the current infrared spot image based on the degree of ellipse fitting represented by the edge contour.

[0097] Binarization is the process of converting a grayscale image into an image with only black and white colors. For example... Figure 8 As shown, the bright-colored aperture is the infrared spot in the acquired infrared spot image.

[0098] Specifically, the current infrared spot image is first binarized. A suitable grayscale threshold is set, and pixels with brightness above the threshold are set to white (representing the spot area), while those below the threshold are set to black (representing the background area), thus separating the infrared spot from the background. Next, the control device performs edge detection on the binarized image to extract the edge contour of the spot area. Then, the control device performs ellipse fitting on this contour, calculating an ellipse model that best matches the contour and evaluating the degree of fit. A higher degree of fit indicates that the spot is closer to a standard ellipse. By analyzing the geometric parameters of the fitted ellipse, such as the ratio of the major axis to the minor axis and the direction angle of the major axis, the control device can determine the tilt angle of the stylus at that moment. For example, a longer major axis indicates a larger tilt angle, and the direction of the major axis indicates the tilt direction of the pen.

[0099] In this embodiment, the mathematical methods of binarization and ellipse fitting are used to achieve accurate quantitative calculation of the stylus tilt angle, providing a reliable data foundation for path correction.

[0100] In an exemplary embodiment, the method further includes: acquiring the ambient light intensity of the environment in which the stylus is located; performing quality analysis on each frame of the stylus image to determine the quality score corresponding to the current stylus image; determining the supplementary lighting information of the current stylus image based on the ambient light intensity and the quality score; and supplementing the writing medium with supplementary lighting according to the supplementary lighting information.

[0101] Ambient light intensity refers to the strength of the ambient light during writing. Quality rating is a quantitative evaluation of quality indicators such as the sharpness and contrast of the pen stroke image. Supplemental lighting information refers to the control parameters that determine whether the supplemental light is turned on and its intensity. Supplemental lighting is the operation of increasing the brightness of the writing interface through artificial light sources.

[0102] Specifically, during stylus writing, the ambient light sensor first acquires the light intensity of the current environment. Simultaneously, real-time quality analysis is performed on each captured frame of pen stroke images, calculating an image quality score. The score indicators include image sharpness, texture contrast, and noise level. The ambient light intensity and quality score are combined for judgment: if the ambient light intensity is low, resulting in a dark image, or if the quality score is below a preset threshold indicating unclear image texture, a supplementary lighting mechanism is triggered. Based on these two input values, the control device calculates appropriate supplementary lighting information, including whether supplementary lighting is needed, the brightness level of the supplementary light, and the duration of supplementary lighting. Then, the control device controls the supplementary light to illuminate the writing medium according to this supplementary lighting information, improving the quality of subsequently acquired images and ensuring accurate extraction of texture information. For example, brightness can be dynamically adjusted using pulse width modulation (PWM) technology. The adjustment range is from 10% to 100%, the resolution is 1%, and the adjustment frequency is greater than 1kHz to avoid flicker. The adjustment strategies include: reducing or turning off the fill light to save power when the ambient light is sufficient and the image quality is good; gradually increasing the fill light brightness until the image quality meets the standard when the ambient light is insufficient and the image is too dark; reducing the fill light brightness and adjusting the camera exposure parameters when the writing surface is too reflective and causes overexposure; and operating the fill light at maximum brightness in extremely dark environments to ensure that the image is usable.

[0103] This embodiment achieves adaptive supplementary lighting through ambient light intensity monitoring and image quality assessment, which can ensure the acquisition quality of brushstroke images under different lighting conditions and improve the stability and accuracy of texture extraction.

[0104] In an exemplary embodiment, the texture information of the writing medium is extracted from each frame of pen stroke images, including: obtaining image extraction parameters for extracting texture information; for each frame of pen stroke images, if the writing pressure corresponding to the current pen stroke image does not meet the distortion condition, extracting the texture information of the writing medium from the current pen stroke image based on the image processing parameters; if the writing pressure corresponding to the current pen stroke image meets the distortion condition, adjusting the image processing parameters according to the writing pressure to obtain updated processing parameters; and extracting the texture information of the writing medium from the current pen stroke image based on the updated processing parameters.

[0105] Image extraction parameters are algorithmic parameters used to extract texture information from images, such as the threshold for feature point extraction and the tolerance of the matching algorithm. Distortion conditions are critical conditions that cause excessive deformation of the medium and distortion of texture information when writing pressure is too great.

[0106] Specifically, a set of standard image extraction parameters can be pre-set for texture extraction under normal conditions. When processing each frame of pen stroke images, the writing pressure value corresponding to the current pen stroke image is first checked. If the current pressure value is within the normal range and does not meet the distortion condition, it means that the medium texture has not been severely deformed, and the preset image extraction parameters are directly used to extract texture information from the current pen stroke image. If the current pressure value exceeds the threshold and meets the distortion condition, it means that the pen tip pressure is too high, causing the medium surface to be over-compressed and the texture shape to change. In this case, using the standard parameters for extraction will introduce errors. Therefore, the control device will dynamically adjust the image extraction parameters according to the pressure value, such as increasing the filter scale and adjusting the binarization threshold, to obtain updated processing parameters. Then, these adjusted parameters are used to extract texture information from the current pen stroke image, so that relatively accurate texture features can still be obtained under high pressure.

[0107] This embodiment effectively reduces the impact of texture distortion caused by excessive pressure on the extraction results by dynamically adjusting the image extraction parameters according to the writing pressure, thereby improving the robustness and accuracy of texture information extraction.

[0108] In one exemplary embodiment, such as Figure 1 As shown, a stylus is also provided, including an image sensing device, a pressure sensing device, and a control device; the control device is connected to the image sensing device and the pressure sensing device; the image sensing device is used to acquire multiple frames of pen stroke images during the writing process of the stylus; the pressure sensing device is used to acquire the writing pressure corresponding to each frame of pen stroke images; the control device is used to implement the stylus writing trajectory generation method of any one of the above embodiments.

[0109] The image sensing device 104 is a miniature optical acquisition device integrated into the tip or front end of the stylus. It typically consists of a high-resolution camera, a lens assembly, and an image processing chip, used to capture microscopic images of the area where the pen tip contacts the writing medium in real time. The pressure sensing device 106 is a mechanical sensing device embedded in the stylus tip, typically employing a piezoresistive, capacitive, or piezoelectric sensor, used to detect pressure changes when the pen tip contacts the writing medium. The control device 102 is the core processing unit of the stylus, typically composed of a low-power microcontroller or application-specific integrated circuit, responsible for coordinating the acquisition, processing, and transmission of image and pressure data.

[0110] Specifically, the stylus integrates three core components: an image sensor, a pressure sensor, and a control device. The image sensor, installed near the pen tip, continuously captures multiple frames of pen stroke images at a high frame rate during the writing process, as the pen tip contacts the writing medium. The pressure sensor, located at the rear of the pen tip, senses the pressure applied to the pen tip in real time and provides corresponding pressure values ​​for each frame of pen stroke images. The control device, acting as the core processor, is connected to the image and pressure sensors, receiving the image and pressure data they acquire. The control device contains pre-installed program code for the stylus writing trajectory generation method described in the aforementioned embodiments. It is responsible for running these algorithms, processing the input image and pressure data in real time, and ultimately generating a visualized writing trajectory.

[0111] In an exemplary embodiment, the stylus further includes a fill light device connected to a control device; the fill light device is used to provide fill light to the writing medium; the control device is also used to acquire the ambient light intensity of the environment in which the stylus is located; perform quality analysis on each frame of pen stroke image to determine the quality score corresponding to the current pen stroke image; determine the fill light information of the current pen stroke image based on the ambient light intensity and the quality score; and control the fill light device to provide fill light to the writing medium according to the fill light information.

[0112] The pen tip integrates a micro LED (Light Emitting Diode) supplementary lighting device, using white or warm white LEDs with a color temperature range of 5000K-6500K and a color rendering index ≥80 to ensure accurate color reproduction of images. The LED beads are 0.5mm in diameter, with power consumption controlled below 50mW. Uniform illumination is achieved through lenses or light guide structures, and the illumination angle matches the camera's field of view. The supplementary lighting uses a ring layout, with 3-4 LEDs evenly distributed around the camera lens to avoid noticeable shadows or reflections. The distance between the LEDs and the writing surface is approximately 2-5mm, providing an illuminance of 500-2000 lux, ensuring stable supplementary lighting under various lighting conditions.

[0113] Specifically, in addition to image sensing, pressure sensing, and control devices, the stylus also includes a supplementary lighting device connected to the control device. During writing, the control device continuously acquires ambient light intensity data and performs real-time quality analysis on each frame of pen stroke images captured by the image sensing device, calculating a quality score. Based on the ambient light intensity and the quality score, the control device makes a comprehensive judgment: if the ambient light intensity is too low or the quality score is substandard, it indicates that the current imaging conditions are poor and manual intervention is required. The control device then calculates appropriate supplementary lighting information, including the on / off state of the supplementary lighting, brightness level, and supplementary lighting mode, and sends control commands to the supplementary lighting device according to this information. Upon receiving the command, the supplementary lighting device illuminates the writing medium according to the set parameters, creating favorable lighting conditions for the acquisition of the next frame image.

[0114] This embodiment significantly improves the image acquisition effect in low-light environments by integrating a fill light device into the stylus and realizing intelligent fill light control, ensuring that the writing trajectory generation system can work stably under various lighting conditions.

[0115] In one specific embodiment, a stylus is also provided, in which an ultra-small CMOS (Complementary Metal-Oxide-Semiconductor Image Sensor) image sensing device is integrated at the stylus tip. The sensor has a resolution of 640×480 pixels and is equipped with a wide-angle lens with an f / 2.0 aperture, providing a 120-degree field of view to cover a sufficiently large writing area around the pen tip. The entire camera module has a diameter of less than 1.8mm and a length of no more than 5mm to ensure that it does not affect the pen's writing feel and balance. The image sensing device can operate at a maximum frame rate of 120fps to 240fps, capable of capturing subtle movements during rapid writing. To reduce power consumption, the image sensing device employs dynamic frame rate adjustment technology: it drops to 20fps when stationary or moving slowly, and automatically increases to the maximum frame rate when rapid movement is detected.

[0116] Specifically, the CMOS image sensor continuously captures images of the writing surface. Each frame undergoes the following processing steps: Image preprocessing: denoising, contrast enhancement, and edge sharpening are performed on the original image to improve the recognizability of texture features. Adaptive histogram equalization (CLAHE) algorithm is used to optimize image quality. Feature point extraction: Key feature points in the image are extracted using the ORB (Oriented FAST and RotatedBRIEF) algorithm or the SIFT (Scale-Invariant Feature Transform) algorithm. These feature points typically correspond to microstructures such as paper fibers and surface textures. 200-500 texture feature points can be extracted from each frame. The texture feature points of the current frame are matched with those of the previous frame, and the FLANN (Fast Library for Approximate Nearest NeighborsMatcher) matcher is used to accelerate the matching process. False matches are eliminated using the RANSAC (Random Sample Consensus) algorithm, preserving reliable correspondences. Based on the matched feature point pairs, the movement segment of the current frame relative to the previous frame is calculated. Optical flow or homography matrix estimation methods are used to calculate the displacement in the X and Y axes with an accuracy of 0.01 mm. The movement segments between consecutive frames are accumulated sequentially over time to reconstruct the stylus's movement path on the writing medium. Simultaneously, the timestamp of each trajectory point is recorded for subsequent stroke segmentation and speed analysis. The processed trajectory data is transmitted to the host computer via the CMOS image sensor, which transmits the raw image data to the control device within the pen through the MIPICSI-2 interface. This interface supports data transmission rates up to 1 Gbps, meeting the real-time transmission requirements of high frame rate video streams. For example, the control device also includes a Main Processing Dedicated Image Signal Processor (ISP) responsible for executing the aforementioned image processing and trajectory reconstruction algorithms. The processed data is converted into a sequence of trajectory coordinate points, with each point containing information such as (x, y, timestamp, pressure), resulting in significant data compression. The processed trajectory data is transmitted in real time to the paired host device (phone, tablet, or computer) via Bluetooth 5.0 or higher (supporting BLE high-speed mode). The Bluetooth module uses a low-latency transmission protocol, with end-to-end latency controlled within 10ms, ensuring a smooth writing experience.For higher bandwidth requirements, a Wi-Fi 6 module can be selected, supporting faster data transmission and longer connection distances. Transmitted data uses a custom binary protocol or the standard InkML format, containing information such as stroke trajectory, timestamps, and pressure values. Data packets are compressed and encrypted to ensure transmission efficiency and security. In online mode, real-time image transmission is supported. The Wi-Fi module can send photos to the host for handwriting processing; the photos have local timestamps, and real-time pressure data is also uploaded to the host.

[0117] In addition, a miniature LED fill light is integrated at the pen tip, using white or warm white LEDs with a color temperature range of 5000K-6500K and a color rendering index (CRI) ≥80 to ensure accurate color reproduction of the image. The LED beads are 0.5mm in diameter, with power consumption controlled below 50mW. Uniform illumination is achieved through a lens or light guide structure, and the illumination angle matches the camera's field of view. The fill light uses a ring layout, with 3-4 LEDs evenly distributed around the camera lens to avoid obvious shadows or reflections. The distance between the LEDs and the writing surface is approximately 2-5mm, providing an illuminance of 500-2000 lux, offering stable fill light under various lighting conditions. An integrated ambient light sensor (ALS) can be integrated if needed to monitor the ambient light intensity in real time. The sensor has a sampling frequency of 10Hz, enabling rapid response to changes in light. The control device analyzes the pen stroke image captured by the CMOS sensor, calculating quality indicators such as average brightness, contrast, and sharpness. A no-reference image quality assessment algorithm is used to quantify and score image quality. Based on ambient light intensity and image quality score, the brightness of the LED is dynamically adjusted using PWM (Pulse Width Modulation) technology. The adjustment range is from 10% to 100%, with a resolution of 1%, and an adjustment frequency greater than 1kHz to avoid flicker. The adjustment strategies include: reducing or turning off the fill light to save power when ambient light is sufficient and image quality is good; gradually increasing the fill light brightness until the image quality meets standards when insufficient ambient light causes the image to be too dark; reducing the fill light brightness and adjusting the camera exposure parameters when the writing surface is too reflective and causes overexposure; and operating the fill light at maximum brightness in extremely dark environments to ensure usable images. The camera's white balance parameters are automatically adjusted based on the fill light's color temperature and ambient light characteristics to ensure color consistency under different light sources.

[0118] For example, a high-precision pressure sensor is integrated at the pen tip, employing capacitive or piezoresistive pressure sensing technology. The sensor has a sensing range of 0-500 gf (grams of force), a resolution of 1 gf, and a sampling frequency supporting up to 200 Hz, enabling it to accurately capture pressure changes during writing. The pressure sensor is in direct contact with the pen tip through an elastic structure; when the pen tip presses against the writing surface, the elastic deformation is proportional to the pressure. The sensor converts the pressure signal into an electrical signal, which is then digitized by an ADC (Analog-to-Digital Converter) and transmitted to the control device.

[0119] Furthermore, the pressure sensing device is time-synchronized with the CMOS camera to ensure that each image frame corresponds to an accurate pressure value. The synchronization mechanism includes: the control device generating a unified time base, and all sensor data being precisely timestamped; the pressure sampling trigger signal being synchronized with the camera exposure signal to ensure a one-to-one correspondence between pressure data and image frames; and the use of hardware interrupts or methods to acquire pressure data, reducing latency and jitter.

[0120] Understandably, to ensure the accuracy of trajectory generation, the line width of the generated ink needs to be dynamically adjusted according to the pressure applied. A pressure-line width mapping curve is established, typically using a non-linear function (such as a logarithmic or power function) to simulate the characteristics of real writing. For example: pressure 0-100gf corresponds to a line width of 0.3-0.8mm (fine strokes); pressure 100-300gf corresponds to a line width of 0.8-1.5mm (medium strokes); and pressure 300-500gf corresponds to a line width of 1.5-2.5mm (thick strokes). The mapping curve can be personalized and learned according to user habits.

[0121] For example, the start and end of a stroke can be determined by detecting a pressure threshold. When the pressure exceeds a set threshold (e.g., 10 gf), it is determined that a new stroke has begun; when the pressure drops below the threshold, it is determined that the current stroke has ended. This is more reliable and faster than simply relying on image changes. When the writing pressure is high, the contact area between the pen tip and the paper increases, which may cause distortion or blurring in the image captured by the camera. The system predicts the degree of this distortion through the pressure signal and applies the corresponding image correction algorithm: establishing a pressure-distortion model, obtained through experimental data or machine learning training; dynamically adjusting image processing parameters based on the current pressure value, such as the threshold for feature point extraction and the tolerance of the matching algorithm; correcting the calculated displacement vector to compensate for the positional deviation caused by changes in the contact area. By combining pressure changes and movement speed, the system simulates the pen stroke effects of different writing tools such as brushes and pens, making the digital ink more natural and vivid.

[0122] Pressure data, as part of the trajectory data, is stored and transmitted along with the coordinate points. The data structure for each trajectory point is: (timestamp, x, y, pressure), where pressure is a normalized pressure value (0.0-1.0). After receiving the pressure data, the host software can render pressure-sensitive handwriting in real time, and can also adjust the pressure-line width mapping during later editing to achieve changes in handwriting style.

[0123] Optionally, such as Figure 9 As shown, an infrared LED projection device 108 is integrated at the pen tip. In this embodiment, the supplementary lighting device is also located at the infrared LED projection device 108. It emits near-infrared light with a wavelength of 850nm or 940nm, and the power is controlled between 10-20mW. The infrared LED is equipped with a collimating lens, projecting a circular light spot onto the writing surface in a vertical position. The light spot has a diameter of approximately 1mm and clear edges. The projection direction of the infrared light spot is parallel to the pen's axis, ensuring that the light spot presents a standard circle when the pen is perpendicular to the writing surface. The infrared LED operates using pulse modulation to avoid interference with ambient infrared light sources (such as sunlight or incandescent lamps) and reduce power consumption. When the stylus is tilted while writing, the projection of the infrared light spot on the writing surface changes from a circle to an ellipse. The ratio of the major and minor axes and the direction of the ellipse reflect the pen's tilt angle and azimuth. The system detects the angle through the following steps: the CMOS camera captures the image of the infrared light spot while simultaneously photographing the texture of the writing surface. Because the CMOS sensor is sensitive to near-infrared light, the light spot forms a bright area in the image. To improve recognition accuracy, an infrared filter can be added in front of the camera, or an image processing algorithm can be used to separate the infrared spot from the visible light information. The control device binarizes the captured spot image and extracts the edge contour of the spot. An ellipse fitting algorithm (such as least squares ellipse fitting) is used to calculate the major and minor axis lengths and the major axis orientation angle of the spot. The pen's tilt angle is calculated based on the ellipse parameters of the spot. The relationship between the tilt angle θ and the ratio of the major and minor axes of the ellipse is: cos(θ) = a / b, where b is the major axis length and a is the minor axis length. The azimuth angle φ is the orientation angle of the major axis of the ellipse. This method can accurately measure the pen's three-dimensional posture.

[0124] Specifically, during initial use or periodic calibration, users are required to write vertically and at different tilt angles to establish a mapping relationship between the light spot shape and the actual angle. Considering the differences in reflectivity of different writing surfaces, the angle calculation parameters will be dynamically adjusted to improve measurement accuracy.

[0125] Furthermore, it can simulate the changes in pen strokes when writing with tools such as fountain pens and brushes at an angle, adjusting the shape and thickness of the handwriting based on the tilt angle and direction to make digital ink more realistic. In some drawing applications, it automatically generates shadow effects for the pen strokes based on the pen's tilt angle, enhancing the sense of depth. It also combines angle information to correct the optical tracking trajectory, compensating for changes in camera perspective and positional deviations caused by pen tilt.

[0126] Optionally, to avoid infrared light spots interfering with texture recognition, the following techniques are used to separate infrared and visible light information:

[0127] Time-division multiplexing: The infrared LED operates in a pulsed manner, switching on and off alternately within a frame period. The camera captures a spot image when the infrared LED is on and a texture image when it is off, extracting pure spot information through frame difference. Spectral separation: A dual-channel image sensor or switchable filters are used to acquire visible light and infrared light images separately, achieving physical-level spectral separation. Algorithm separation: Image processing algorithms identify and remove infrared spot regions, preserving texture information for trajectory tracking.

[0128] For example, a high-capacity flash memory chip with a storage capacity of 512MB to 4GB is integrated inside the stylus body, using NAND Flash or eMMC technology. The storage chip connects to the control device via SPI (Serial Peripheral Interface) or SDIO (Secure Digital Input / Output Interface), achieving read / write speeds of 20-100MB / s to meet real-time data storage requirements. When the stylus is not connected to the host device, the system enters offline mode, and the written data is stored directly inside the pen. The control device executes complete image processing and trajectory reconstruction algorithms, converting the images captured by the camera into a sequence of coordinate points in real time. Each coordinate point contains information such as (timestamp, x, y, pressure, angle). To save storage space, the system compresses the trajectory data. Incremental encoding is used, storing only the displacement difference between adjacent points instead of absolute coordinates; duplicate or redundant data points are extracted; feature points in the handwriting are extracted and stored, further compressing the data. The compressed data volume can be reduced by 50-70%. Each stroke (the continuous trajectory from pen touch to pen lift) is stored as an independent data segment, containing the stroke's metadata (start and end times, number of points, bounding box, etc.) and an array of coordinate points. Stroke data is organized chronologically to form the writing trajectory. The system monitors storage space usage in real time; when storage space is insufficient, it prompts the user to synchronize data or delete old data. Based on a 512MB storage capacity, it is estimated to store approximately 50-100 hours of continuous writing content (depending on writing speed and complexity). The processed trajectory coordinates are transmitted to the host software in real time via Bluetooth or Wi-Fi, with latency controlled within 10-20ms. The host software renders and displays the data immediately upon receiving it, achieving a "write-and-display" effect. Simultaneously, data is backed up and stored in the pen to prevent data loss due to transmission interruptions. After the connection is established, the system automatically detects whether there is any unsynchronized offline data in the pen. If so, a synchronization process is initiated, quickly transmitting the stored trajectory data to the host. The synchronization process uses batch transmission, fully utilizing the bandwidth of Bluetooth or Wi-Fi, achieving transmission speeds of 500KB / s to 5MB / s. Depending on the data volume, complete synchronization typically takes several seconds to several minutes. To improve efficiency, the system supports incremental synchronization. A synchronization status flag is maintained between the host and the pen, transmitting only data added since the last synchronization to avoid duplicate transmissions. CRC or MD5 checksums are used during transmission to ensure data integrity. If data corruption is detected, the corresponding data segment is automatically retransmitted. Upon receiving the data, the host software decompresses and converts the format to restore the complete handwriting content. The pen's internal storage uses a compact binary format to optimize storage efficiency and read / write speed.The data structure includes a file header (version information, device ID, etc.), a session list, and stroke data, forming a hierarchical structure. After synchronization to the host, the data can be converted to various standard formats: InkML (Ink Markup Language, W3C standard), SVG (Scalable Vector Graphics), JSON (JavaScriptObject Notation for easy program processing), PDF (Portable Document Format for document archiving), and image formats (PNG, JPEG, etc.). Users can choose the export format as needed. In addition to trajectory coordinates, the system retains complete metadata information, including writing time, pressure value, tilt angle, device information, and writing surface type, providing a basis for subsequent analysis and processing. Data stored in the pen is encrypted, with the key stored in a secure chip or derived from the user's password. Unauthorized access prevents reading of the pen's data. An encrypted Bluetooth pairing connection is established between the pen and the host, using SecureSimple Pairing to prevent man-in-the-middle attacks and data eavesdropping. Users can view, export, or delete the data stored in the pen at any time in the host software, maintaining complete control over their writing. The flash memory chip employs a low-power design, with standby power consumption less than 10μA and write power consumption controlled within 10-30mA. When not writing, the storage module and control device enter a deep sleep mode, reducing power consumption to the microampere level. It wakes up quickly upon detecting pen movement or key presses, with a response time of less than 100ms. A caching mechanism reduces the number of flash memory writes. Data is first stored in a RAM cache, and then written to flash memory in batches only when a certain amount is accumulated or a pen lift is detected, extending flash memory lifespan and reducing power consumption.

[0129] Through the above technical solutions, the stylus achieves complete offline storage and online transmission functions, allowing users to write anytime, anywhere without worrying about data loss, truly achieving "data arrives with the pen".

[0130] In a specific embodiment, such as Figure 10 As shown, a method for generating stylus writing trajectories is also provided, including:

[0131] Step S1001: Acquire multiple frames of pen stroke images captured during the stylus writing process, as well as the writing pressure corresponding to each frame of pen stroke images;

[0132] Among them, the pen stroke image is acquired when the tip of the stylus is in contact with the writing medium;

[0133] Step S1002: Obtain the ambient light intensity of the environment where the stylus is located;

[0134] Step S1003: Perform quality analysis on each frame of brush stroke image to determine the quality score corresponding to the current brush stroke image;

[0135] Step S1004: Determine the fill light information of the current brush stroke image based on ambient light intensity and quality score;

[0136] Step S1005: Apply supplemental lighting to the writing medium according to the supplemental lighting information;

[0137] Step S1006: Obtain image extraction parameters for extracting texture information;

[0138] Step S1007: For each frame of pen stroke image, if the writing pressure corresponding to the current pen stroke image does not meet the distortion condition, extract the texture information of the writing medium from the current pen stroke image based on the image processing parameters.

[0139] Step S1008: If the writing pressure corresponding to the current pen stroke image meets the distortion condition, adjust the image processing parameters according to the writing pressure to obtain updated processing parameters;

[0140] Step S1009: Based on the updated processing parameters, extract the texture information of the writing medium from the current pen stroke image;

[0141] Step S1010: For each frame of brush stroke image, obtain the texture information of the current brush stroke image and the texture information of the previous frame of brush stroke image.

[0142] Step S1011: Match the texture information of the current pen stroke image with the texture information of the previous frame pen stroke image to determine the positions of the same texture feature points in the writing medium in the current pen stroke image and the previous frame pen stroke image, respectively.

[0143] Step S1012: Based on the determined positions, determine the movement segment of the stylus between the current stroke image and the previous stroke image.

[0144] Step S1013: Piece together the moving segments to obtain the movement path of the stylus on the writing medium;

[0145] Step S1014: Acquire multiple frames of infrared light spot images captured by the stylus during the writing process;

[0146] Among them, multiple frames of infrared spot images correspond one-to-one with multiple frames of brush stroke images;

[0147] Step S1015: For each frame of infrared spot image, perform binarization processing on the current infrared spot image and extract the edge contour of the infrared spot in the current infrared spot image.

[0148] Step S1016: Determine the tilt angle of the stylus in the current infrared spot image based on the degree of ellipse fitting represented by the edge contour.

[0149] Step S1017: Correct the movement path based on each tilt angle to obtain an updated movement path;

[0150] Step S1018: For each frame of pen stroke image, based on the writing pressure corresponding to the current pen stroke image, determine the line width of the road segment corresponding to the current pen stroke image in the movement path;

[0151] Among them, the width of the road section is positively correlated with the writing pressure;

[0152] Step S1019: Based on the updated road segment widths of each segment in the moving path, generate the writing trajectory of the stylus on the writing medium.

[0153] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0154] Based on the same inventive concept, this application also provides a stylus writing trajectory generation device for implementing the stylus writing trajectory generation method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations of one or more stylus writing trajectory generation device embodiments provided below can be found in the limitations of the stylus writing trajectory generation method described above, and will not be repeated here.

[0155] In one exemplary embodiment, such as Figure 11 As shown, a stylus writing trajectory generation device 1100 is provided, including: an image acquisition module 1102, a texture information extraction module 1104, a movement path determination module 1106, a road segment width determination module 1108, and a writing trajectory generation module 1110, wherein:

[0156] The image acquisition module 1102 is used to acquire multiple frames of pen stroke images captured during the writing process of the stylus, as well as the writing pressure corresponding to each frame of pen stroke images; wherein, the pen stroke images are acquired when the tip of the stylus is in contact with the writing medium.

[0157] The texture information extraction module 1104 is used to extract the texture information of the writing medium from each frame of pen stroke image;

[0158] The movement path determination module 1106 is used to determine the movement path of the stylus on the writing medium based on the correspondence of texture information between each frame of pen stroke images;

[0159] The road segment width determination module 1108 is used to determine the road segment width corresponding to the current stroke image in the movement path based on the writing pressure corresponding to the current stroke image for each frame of stroke image; wherein, the road segment width is positively correlated with the writing pressure; and

[0160] The writing trajectory generation module 1110 is used to generate the writing trajectory of the stylus on the writing medium based on the line width of each segment in the movement path.

[0161] In an exemplary embodiment, the movement path determination module 1106 is specifically used for:

[0162] For each frame of brush stroke image, obtain the texture information of the current brush stroke image and the texture information of the previous frame of brush stroke image;

[0163] The texture information of the current brush stroke image is matched with the texture information of the previous brush stroke image to determine the position of the same texture feature point in the writing medium in the current brush stroke image and the previous brush stroke image, respectively.

[0164] Based on the determined positions, the movement segment of the stylus between the current stroke image and the previous stroke image is determined; and

[0165] By piecing together the various moving segments, the movement path of the stylus on the writing medium is obtained.

[0166] In one exemplary embodiment, the stylus writing trajectory generation device 1100 further includes an updated movement path determination module, comprising:

[0167] The infrared spot image acquisition unit is used to acquire multiple frames of infrared spot images collected by the stylus during the writing process; wherein, the multiple frames of infrared spot images correspond one-to-one with the multiple frames of pen stroke images;

[0168] The tilt angle determination unit is used to determine the tilt angle of the stylus in the current infrared spot image based on the degree of ellipse fitting of the current infrared spot image for each frame of infrared spot image.

[0169] The path correction unit is used to correct the movement path based on each tilt angle to obtain an updated movement path;

[0170] The writing trajectory generation unit is used to generate the writing trajectory of the stylus on the writing medium based on the updated line width of each segment in the movement path.

[0171] In one exemplary embodiment, the tilt angle determination unit is specifically used for:

[0172] For each frame of infrared spot image, the current infrared spot image is binarized to extract the edge contour of the infrared spot in the current infrared spot image; and

[0173] The tilt angle of the stylus in the current infrared spot image is determined based on the degree of ellipse fitting represented by the edge contour.

[0174] In one exemplary embodiment, the stylus writing trajectory generation device 1100 further includes a supplementary light module, specifically used for:

[0175] Obtain the ambient light intensity of the environment where the stylus is located;

[0176] Perform quality analysis on each frame of the brush stroke image to determine the quality score corresponding to the current brush stroke image;

[0177] Based on ambient light intensity and quality score, determine the fill light information for the current brushstroke image; and

[0178] Apply supplemental lighting to the writing medium according to the supplemental lighting information.

[0179] In an exemplary embodiment, the texture information extraction module 1104 is specifically used for:

[0180] Obtain the image extraction parameters used to extract texture information;

[0181] For each frame of pen stroke image, if the writing pressure corresponding to the current pen stroke image does not meet the distortion condition, the texture information of the writing medium is extracted from the current pen stroke image based on the image processing parameters.

[0182] If the writing pressure corresponding to the current pen stroke image meets the distortion condition, the image processing parameters are adjusted according to the writing pressure to obtain updated processing parameters; and

[0183] Based on updated processing parameters, texture information of the writing medium is extracted from the current pen stroke image.

[0184] Each module in the aforementioned stylus writing trajectory generation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0185] In one exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 12 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When executed by the processor, the computer program implements a method for generating stylus writing trajectories. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0186] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0187] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0188] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0189] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described above.

[0190] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0191] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0192] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0193] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for generating a writing trajectory with a stylus, characterized in that, The method includes: Acquire multiple frames of pen stroke images captured during the writing process of the stylus, as well as the writing pressure corresponding to each frame of the pen stroke image; The texture information of the writing medium is extracted from each frame of the brushstroke image; Based on the correspondence of texture information between the pen stroke images in each frame, the movement path of the stylus on the writing medium is determined; For each frame of the pen stroke image, based on the writing pressure corresponding to the current pen stroke image, the line width of the corresponding segment in the movement path is determined; wherein, the line width of the segment is positively correlated with the writing pressure; and Based on the line width of each segment in the movement path, the writing trajectory of the stylus on the writing medium is generated.

2. The method according to claim 1, characterized in that, Determining the movement path of the stylus on the writing medium based on the texture information correspondence between the stroke images in each frame includes: For each frame of the brushstroke image, obtain the texture information of the current brushstroke image and the texture information of the brushstroke image of the previous frame; The texture information of the current pen stroke image is matched with the texture information of the previous pen stroke image to determine the position of the same texture feature point in the writing medium in the current pen stroke image and the previous pen stroke image, respectively. Based on the determined positions, the movement segment of the stylus between the current stroke image and the previous stroke image is determined; and By piecing together the various moving segments, the movement path of the stylus on the writing medium is obtained.

3. The method according to claim 1, characterized in that, The method further includes: Acquire multiple frames of infrared light spot images captured during the writing process of the stylus; wherein, each of the multiple frames of infrared light spot images corresponds one-to-one with the multiple frames of pen stroke images; For each frame of the infrared spot image, the tilt angle of the stylus in the current infrared spot image is determined based on the degree of ellipse fitting of the current infrared spot image. The movement path is corrected based on each of the aforementioned tilt angles to obtain an updated movement path; and Based on the line width of each segment in the updated movement path, the writing trajectory of the stylus on the writing medium is generated.

4. The method according to claim 3, characterized in that, For each frame of the infrared spot image, determining the tilt angle of the stylus in the current infrared spot image based on the ellipse fitting degree of the current infrared spot image includes: For each frame of the infrared spot image, the current infrared spot image is binarized to extract the edge contour of the infrared spot in the current infrared spot image; and Based on the degree of ellipse fitting represented by the edge contour, the tilt angle of the stylus in the current infrared spot image is determined.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Obtain the ambient light intensity of the environment in which the stylus is located; Perform quality analysis on each frame of the brushstroke image to determine the quality score corresponding to the current brushstroke image; Based on the ambient light intensity and the quality score, determine the fill light information for the current brushstroke image; and The writing medium is illuminated according to the illumination information.

6. The method according to any one of claims 1 to 4, characterized in that, The step of extracting the texture information of the writing medium from each frame of the brushstroke image includes: Obtain the image extraction parameters used to extract texture information; For each frame of the pen stroke image, if the writing pressure corresponding to the current pen stroke image does not meet the distortion condition, the texture information of the writing medium is extracted from the current pen stroke image based on the image processing parameters; If the writing pressure corresponding to the current pen stroke image satisfies the distortion condition, the image processing parameters are adjusted according to the writing pressure to obtain updated processing parameters; and Based on the updated processing parameters, the texture information of the writing medium is extracted from the current pen stroke image.

7. A stylus, characterized in that, The stylus includes an image sensing device, a pressure sensing device, and a control device; the control device is connected to the image sensing device and the pressure sensing device. The image sensing device is used to acquire multiple frames of pen stroke images during the writing process of the stylus; The pressure sensing device is used to collect the writing pressure corresponding to each frame of the pen stroke image; The control device is used to implement the stylus writing trajectory generation method as described in any one of claims 1 to 6.

8. The stylus according to claim 7, characterized in that, The stylus also includes a fill light device connected to the control device; The supplemental lighting device is used to provide supplemental lighting for the writing medium; The control device is also used to acquire the ambient light intensity of the environment in which the stylus is located; Perform quality analysis on each frame of the brushstroke image to determine the quality score corresponding to the current brushstroke image; Based on the ambient light intensity and the quality score, determine the fill light information of the current brushstroke image; as well as The supplementary lighting device is controlled to provide supplementary lighting to the writing medium according to the supplementary lighting information.

9. A stylus writing trajectory generation device, characterized in that, The device includes: The image acquisition module is used to acquire multiple frames of pen stroke images captured during the writing process of the stylus, as well as the writing pressure corresponding to each frame of the pen stroke image; The texture information extraction module is used to extract the texture information of the writing medium from each frame of the pen stroke image; The movement path determination module is used to determine the movement path of the stylus on the writing medium based on the texture information correspondence between the pen stroke images in each frame; The road segment width determination module is used to determine the road segment width corresponding to the current stroke image in the movement path based on the writing pressure corresponding to the current stroke image for each frame of the stroke image; wherein, the road segment width is positively correlated with the writing pressure; and The writing trajectory generation module is used to generate the writing trajectory of the stylus on the writing medium based on the line width of each segment in the movement path.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

Images