Digital calligraphy system based on block touch

A digital calligraphy system with a high-density touch module, capacitive brush, and high-definition screen accurately captures and renders traditional calligraphy strokes, addressing the limitations of existing systems by replicating the feel and look of traditional calligraphy.

CN119065518APending Publication Date: 2024-12-03HANGZHOU YIMO DIGITAL TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310629179.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-31
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Existing digital calligraphy systems fail to accurately replicate the nuances of traditional calligraphy due to the use of hard-tipped capacitive styluses and LCD screens unsuitable for calligraphy, lacking information on stroke thickness and contact shape, and providing a subpar experience compared to traditional rice paper.

Method used

A digital calligraphy system utilizing a high-density sensing unit touch module, a capacitive brush with flexible material, and a high-definition ink screen display to capture and render brush strokes with block touch data, converting two-dimensional voltage differences into vector graphics to restore the feel of traditional handwriting on a touch-sensitive device.

Benefits of technology

The efficacy of the system is achieved by accurately capturing and rendering traditional calligraphy strokes with high fidelity, providing a realistic writing experience akin to rice paper.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119065518A_ABST
    Figure CN119065518A_ABST
Patent Text Reader

Abstract

The invention discloses a digital calligraphy system which is characterized in that a high-precision special touch control module is adopted, block touch original data is collected, an optimized block touch conversion module is applied, and highly-restored writing brush calligraphy handwriting is generated. The digital calligraphy system comprises a special touch module, an ink display screen, a block touch conversion module, a capacitance writing brush, calligraphy software and the like. The special touch module is characterized in that fine touch event data can be obtained, and the block data conversion module uses the data to calculate and restore the writing brush handwriting of a user and render the writing brush handwriting on an ink screen. The block touch conversion module carries out training in a machine learning mode to obtain a conversion algorithm which comprises simple stroke training and complex stroke training. In addition, the invention further discloses a calligraphy shell and an auxiliary calligraphy tablet, which can be applied to a common touch screen, so that more flexible and convenient digital calligraphy experience is brought to the user by utilizing user stock equipment. The method has the advantages and effects that the characteristic of'lifting, pressing, pausing and rotating 'in the writing process of the writing brush is truly obtained, the real writing experience of a traditional soft pen is fully restored, digitization of the traditional soft pen in the fields of calligraphy, painting and the like is achieved, and colorful painting and calligraphy art works can be created easily.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of touch screen interaction technology, and specifically refers to a digital calligraphy system implemented based on block touch data. Background Art

[0002] At present, there are many digital calligraphy software in the industry that realize the function of calligraphy creation on tablets and mobile phones. Among them, personal consumer-level devices such as Apple Pencil can realize the effect of writing brush characters on specific software; there are also some larger devices such as digital calligraphy tables, which are widely used in calligraphy education and training, cultural centers and libraries to show the charm of calligraphy art. However, the digital realization of traditional calligraphy by these products is relatively rudimentary and crude.

[0003] The reason is: First, the tip of a capacitive stylus like the Apple Pencil is a hard tip, and its feel is far from the soft tip of a traditional animal hair stylus.

[0004] Secondly, whether you use a capacitive pen or a capacitive brush to write on an ordinary touch screen, the system only provides the application software with the coordinates and pressure of one or more points. If you use an active capacitive pen like Apple Pencil, the software can send back more information via Bluetooth. However, the software can still use limited information, such as point coordinates (×1, Y1) and (x2, Y2), pressure, gyroscope information, movement speed, etc., and then select the built-in brush and use the algorithm to present the brush handwriting effect on the screen, which essentially belongs to the category of hard pen calligraphy. In traditional soft pen calligraphy, "lifting, pressing, stopping, and turning" is reflected by the strength, thickness of the strokes, and contact shape area. From the implementation principle, the software receives the point information of the pen, without information such as stroke thickness and contact shape, so the final effect is distorted.

[0005] Third, the current commercial LCD screens are designed for general tasks, and all aspects of the requirements must be taken into account, such as color display effects and smooth screens. Traditional calligraphy has its own characteristics, such as only two colors - black and white. The experience and feel of watching and moving the pen on an ordinary LCD screen are far different from the experience of rice paper without light. Therefore, ordinary LCD screens are not the most suitable devices for calligraphy presentation.

[0006] In response to the above defects, this application proposes a solution that is completely different from the traditional calligraphy technology route to achieve a high-fidelity digital calligraphy system. Summary of the invention

[0007]

OS

[0008]

Calligraphy Program

[0009] [Sensing unit] The touch module senses the user's touch position and pressure through the electrode unit array of the ITO (Indium Tin Oxide) layer. These arrays are composed of mutually perpendicular horizontal and vertical channels, called drive channels and sensing channels respectively. Each intersection of a sensing channel and a drive channel forms an electrode unit, called a sensing unit, such as Figure 1 As shown in the figure, the distance between the sensing units is recorded as L0\L1. In existing commercial touch screens, such as the 4.3-inch touch module of Nexus 5, the number of channels is usually (TX, RX) = 14 X 25, that is, the sensing unit spacing L0\L1 = 4.1mm. When the number of touch channels is small, the touch accuracy is low and the response speed is faster; when the number of touch channels is large, the position accuracy that can be sensed is finer, which also means that a higher frequency scanning speed is required to maintain a faster response speed.

[0010]

Touch IC

[0011]

Point Touch

[0012] [Block touch] When the user's entire thumb presses on the screen, the shape of the contact between the finger and the touch screen is a plane. At this time, the touch IC can obtain the difference in the capacitance change of all sensing units, usually two-dimensional data, called block touch data. The above data is returned to the OS and application layer as input data for downstream programs. The coordinates of the aforementioned point touch (x0, y0) are also calculated from the block touch data.

[0013] The solution of this application consists of several sub-solutions, which are organically combined and applied in the field of digitization of traditional calligraphy.

[0014] The first part is the block touch module. This module has a high density of sensing units and can sense the details of brush writing. The touch IC provides communication interfaces such as SPI, USB, I2C, etc. The external host obtains the raw data of the touch data through these interfaces, usually a set of two-dimensional arrays. They represent the difference between the voltage of all units in the entire electrode unit array when the touch occurs and when there is no touch, which is used to indicate the contact area when the touch event occurs. The above data is passed to the downstream block touch conversion module to generate the strokes when writing with a brush.

[0015] The accuracy of the above-mentioned block touch data varies in different models of touch products, and the accuracy of the device depends on the number and density of its sensing units. For example, a highly accurate fingerprint recognition module contains up to 208 x 288 sensing units in a sensing area of ​​10.4mm x 14.4mm; while the screen size of Nexus 5 is 69.2mm x 137.8mm, but it only has 15 x 27 sensing units. The more sensing units and the higher the density, the more accurate the data and the higher the degree of stroke restoration. Of course, the corresponding cost will be higher, but the writing effect will be better. Therefore, this application prefers a customized touch screen with a sensing unit spacing of 0.4mm and an efficient signal scanning algorithm.

[0016] The second part is the block touch conversion module. The capacitive brush starts writing on the flat panel display, which is part 100 in Figure 2. The calligraphy software obtains the block touch data from the touch IC, which is the voltage difference of all corresponding sensing units, which is part 110. Through the corresponding algorithm, these two-dimensional voltage difference arrays are converted into vector diagrams, which is part 120, so as to restore the touch data to the brush strokes, just like the handwriting left by the real brush on the rice paper.

[0017] The third part is a capacitive brush, which is a brush with conductive function. The brush head is made of flexible material, preferably wolf, sheep, weasel and other animal hair commonly used in making traditional Chinese brushes. The simplest way to implement it is to dip an ordinary brush in water instead of ink, and touch the brush head with your hand. In order to cooperate with the best implementation effect of the present invention, a patent is applied for to detail the design of a special capacitive brush. In this application, in order to ensure the integrity of this solution, the following description uses an ordinary brush dipped in water to represent the capacitive pen for the purpose, which does not affect the implementation effect of this application. Preferably, an active capacitive pen is used, so that data such as the inclination and force of the pen can be obtained at the same time, and the final writing effect will be more outstanding.

[0018] The fourth part, the high-definition ink screen display module, is bonded to the block touch module of the first part to form the calligraphy device screen. The block touch conversion module obtains real-time touch data, processes it into vector graphics, calls the rendering interface of the ink screen, and presents it on the screen in a partially refreshed manner. Users can write on the touch screen while seeing their own strokes, just like writing on real rice paper, thereby achieving the effect of calligraphy practice and creation. In order to make the display screen have the roughness and feel of rice paper, the surface of the touch module of the calligraphy system of this application adopts a touch module glass protective layer made of frosted technology, which is bonded to the display module. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Schematic diagram of the sensing unit array of a capacitive touch module. L0, L1: spacing between sensing units, TX: driving channel, RX: sensing channel.

[0020] Figure 2 Block touch conversion module technical principle. 100: Flat panel display, 110: Block touch raw data, two-dimensional array, 120: Converted vector diagram.

[0021] Figure 3 The structure of the touch screen of the calligraphy tablet. From top to bottom, the top layer is 210: customized touch module, 220: touch IC, 270: ink screen display module.

[0022] Figure 4 Display the digital calligraphy system structure. 10: OS system software, 20: calligraphy software, 25: block touch conversion module, 210: touch module, 270: ink screen display module.

[0023] Figure 5 Steps to generate a simple stroke training set. S1: Reset the test device, S2: Select a simple stroke, S3: Get raw data, S4: Collect handwriting images, S5: Add data to the training set.

[0024] Figure 6 Steps for generating a complex stroke training set. S1: Reset the test device, S2: Select complex strokes, S3: Get several sets of raw data, S4: Collect handwriting images, S5: Add data to the training set.

[0025] Figure 7 Schematic diagram of calligraphy shell structure. 210: Customized touch module, 220: Touch IC, 230: Bluetooth module, 240: USB class interface, 250: Power / switch unit, 260: Buckle. DETAILED DESCRIPTION

[0026] The first implementation case of the present application is a calligraphy tablet. The design and manufacturing process of the calligraphy tablet adopt mature solutions in the industry, and the difference lies in the display component using a customized touch screen, pre-installed OS, and pre-installed calligraphy software.

[0027] From the hardware structure of the customized touch screen, the touch module 210 and the display module 270 are bonded to form a flat display screen. Preferably, the surface of the protective layer of the touch module 210 uses a frosted process. The touch module uses a support block touch touch module. Preferably, the sensing unit spacing is 0.4mm and a customized touch screen with an efficient signal scanning algorithm is used. The touch IC 220 firmware is specially optimized for the calligraphy system, and can also be used as an ordinary touch IC to provide normal touch event processing. The display module 270 uses a high-definition ink screen, and the flat display screen is connected to other modules of the system through an interface.

[0028] The notable feature of this implementation case is that the customized touch module and the high-definition ink screen are bonded together and connected to other modules of the tablet. The tablet's pre-installed operating system has built-in calligraphy software, and the hardware and software have been deeply adapted and can be used out of the box. Different grades of tablet devices will have structural adjustments in specific structures based on industrial design considerations, but they do not deviate from the spirit of the present invention.

[0029] From the perspective of calligraphy system workflow, the process from user writing on the touch screen to displaying brush handwriting is shown in Figure 4.

[0030] STEP 1 Start the calligraphy software after turning on the tablet.

[0031] STEP 2 The calligraphy software calls the system interface and sets the touch module to enter the calligraphy recognition mode.

[0032] STEP 3 The user starts writing on the touch screen with a capacitive brush, and the calligraphy software calls the OS interface to obtain the original touch data.

[0033] STEP 4 The touch conversion module obtains the raw data, selects the best conversion algorithm for calculation according to the device parameters of the software, and outputs the vector diagram after processing. In order to restore the real handwriting as faithfully as possible, this application prefers a customized touch screen with a sensing unit spacing of 0.4mm and an efficient signal scanning algorithm. For reference, the sensing unit spacing of Nexus 5 is 4.1mm; the sensing unit spacing of commonly used fingerprint recognition is 0.05mm.

[0034] STEP 5 The upper layer interface calls the ink screen driver interface to render the vector graphics generated in STEP 4 on the ink screen.

[0035] The above are the core steps of digital calligraphy software. Other functions such as copybook, handwriting undo, background, zooming in and out, and printing are all functions of calligraphy software and are not described in this manual.

[0036] In the workflow steps described above, the fidelity of stroke restoration is determined by the block touch data conversion algorithm. The above algorithms include empirical formulas and publicly released machine learning algorithms. The machine learning algorithms based on the same principle have been widely used in the high-definition of blurred images and the compression and reproduction of galaxy photos. This algorithm is applied to the field of brush strokes. After the capacitive brush is moved on the screen, the two-dimensional array generated by the touch module and its actual brush strokes are used as training sets. The training is iterated to obtain the final model parameters and deployed in the client conversion software module or the cloud. Repeat the above training for touch modules with different perception unit densities to obtain the best algorithm suitable for the device.

[0037] The training content includes two categories. One is simple strokes, which is the correspondence between the original touch data and the handwriting formed when any stroke is put down and does not move; the second is complex strokes, a complete stroke, such as "horizontal stroke and hook", which is the correspondence between all the data obtained in the completion process and the complete stroke.

[0038] The following are the steps for practicing simple strokes. Do not move the pen after it is put to paper, as shown in Figure 5.

[0039] STEP 1 Reset the screen, wipe it clean, dip the capacitive brush into ink, and write different strokes on the screen. The strokes to be trained include the common strokes in calligraphy, and only one stroke is written at a time. At this time, the calligraphy software is in training mode, only collecting data without rendering and analyzing the handwriting obtained.

[0040] STEP 2 Select a stroke, such as “、”, and write without moving.

[0041] STEP 3 Get the original data of the touch screen at this time, that is, a two-dimensional array.

[0042] STEP 4 After picking up the pen, collect the ink marks left on the screen and save them as a picture.

[0043] STEP 5: Add the above two parts as a set of experimental data to the training set.

[0044] This process is repeated to traverse all common strokes. The above process can also be repeated for different touch modules to obtain the optimal model for this device.

[0045] Complex stroke training is the stroke training of the writing process of the brush after it is put down, as shown in Figure 6. It is necessary to record the correspondence between all the original touch data and the handwriting during the writing process. The steps are as follows.

[0046] STEP 1 Reset the test environment and write different strokes on the screen, including the common strokes in calligraphy. Set the calligraphy software to training mode.

[0047] STEP 2 Choose a certain stroke, such as the "horizontal stroke, curved hook", and the writing process after putting pen to paper will reflect the techniques of "lifting, pressing, pausing, and turning".

[0048] STEP 3 During the writing process, the original data of the touch screen is obtained at a certain frequency, that is, a two-dimensional array. There will be many sets of data in a writing process, which are collected according to the time axis.

[0049] STEP 4 After STEP 2, collect the handwriting left on the screen and save it as a picture. In theory, the multiple sets of data collected in STEP 3 correspond to the final picture after superposition.

[0050] STEP 5: Add the above two parts as a set of experimental data to the training set.

[0051] This process is repeated to traverse all commonly used strokes and form a training set for the target device.

[0052] The algorithm model obtained by the training set can be deployed to the local device or in the cloud, and an API is provided for local calls. The algorithm can be built into the calligraphy program, with the ability to hot update, or it can be deployed to the cloud for clients to call through APIs.

[0053] The second implementation case of this application is a calligraphy case, which is suitable for use with existing ordinary tablets and mobile phones. This implementation case combines the calligraphy case with the user's existing tablet to form a digital calligraphy system. The calligraphy case is a transparent tablet with Bluetooth and wired communication capabilities, which can be fixed on the screen of an ordinary tablet.

[0054] Its physical structure is shown in Figure 7. The top layer is 210, which is a custom touch module with a frosted surface, which has good light transmittance and a certain damping feeling; the algorithm in the firmware of touch IC 220 is specially optimized for the calligraphy system, with high response and high precision characteristics; 230 is a Bluetooth module, which provides two-way communication after pairing with an ordinary tablet; 240 is a wired connection that can transmit data and power supply, and 250 is a power switch; 260 is a buckle that can slide left and right to fix the calligraphy shell on the surface of the ordinary tablet to prevent shaking during writing.

[0055] Preferably, the calligraphy shell uses intelligent sleep to replace the manual power switch.

[0056] When in use, the user places the peripheral on other tablets and mobile phones and fixes it. The upper calligraphy shell and the lower tablet are fixed with the active buckle 260, and paired with Bluetooth 240 or cable connection. After calibration through the calligraphy software, it can be used normally. Compared with the first case, because the display screen part is an ordinary screen, it is not optimized for digital calligraphy, and the restoration effect is second to none.

[0057] STEP 1 Open the calligraphy software on the tablet, set the Bluetooth pairing mode, and pair it with the calligraphy case.

[0058] STEP 2 After pairing successfully, place the calligraphy case on the tablet surface, fit it tightly, and fix the case with 260 parts. For the first use, you need to follow the instructions to adapt the tablet size and the brush head.

[0059] STEP 3 Set the calligraphy software to calligraphy mode, and automatically, the calligraphy shell also works in calligraphy mode.

[0060] STEP 4 Use the capacitive brush to create calligraphy in the calligraphy shell.

[0061] STEP 5 Operations in calligraphy software include copybook switching, printing, sharing, zooming, and signing.

[0062] STEP 6 Release the 260 component, remove the calligraphy cover, use the 250 button to shut down the device, exit the calligraphy software, and disconnect the Bluetooth connection.

[0063] This implementation case can also be implemented as an auxiliary calligraphy touch panel. Using opaque materials such as silicon as the base of the touch module has good firmness and does not need to be placed on top of an ordinary tablet. When in use, write on the auxiliary calligraphy touch panel and view the handwriting effect on the ordinary tablet screen. The product structure is shown in Figure 7, remove component 260, and use steps.

[0064] STEP 1 Open the calligraphy software on the tablet, set the Bluetooth pairing mode, and pair it with the auxiliary calligraphy touchpad.

[0065] STEP 2 After pairing is successful, set the calligraphy software to calligraphy mode. At the same time, the auxiliary calligraphy touchpad will automatically work in calligraphy mode.

[0066] STEP 3 Use the capacitive brush to create calligraphy on the auxiliary calligraphy touchpad.

[0067] STEP 4 Observe the writing effect in real time in the calligraphy software.

[0068] STEP 5 Operations in calligraphy software include copybook switching, printing, sharing, zooming, and signing.

[0069] STEP 6 Use the 250 button to shut down the device, disconnect the Bluetooth connection, and exit the calligraphy software.

[0070] The third implementation case of this application is a digital calligraphy app. This implementation case can be implemented using only existing mobile phones and tablets on the market. By installing the digital calligraphy app on the above devices, you can experience the digital calligraphy system described in the present invention and the implementation steps.

[0071] STEP 1 Modify the Android kernel driver source code, the file name is touch_synaptics_ds5.c, etc.

[0072] STEP 2 Recompile the Android operating system source code.

[0073] STEP 3 Use the adb tool to install the compiled operating system to the device.

[0074] STEP 4 Install the calligraphy software and start using it.

[0075] For the iOS platform, Apple's system software is also required to support reading the original two-dimensional data of the touch screen.

[0076] In this implementation case, because the touch module and display module are not optimized for calligraphy, the experience is worse than the previous two categories, but it is the most convenient way to experience digital calligraphy.

[0077] Based on the technology known in the art, the measurement of the voltage value in this specification is equivalent to the measurement of the capacitance value, and is also equivalent to the time and current consumed by measuring the change of the capacitance.

[0078] It is necessary to clarify the novelty of the invention of this application again. Compared with the technical solutions of traditional intelligent calligraphy and digital calligraphy solutions that reconstruct calligraphy handwriting by adding various sensors to the pen and obtaining parameters in the writing process of the intelligent brush (these solutions focus on actions and processes), the solution of this application focuses on the handwriting left by the brush on the screen at the end (starting from the result). Regardless of the technique used to write calligraphy strokes, this solution only cares about the shape of the handwriting left on the touch screen, and finally uses block touch data to restore the brush handwriting.

[0079] It should be understood that the implementation cases described in this application are not limitative descriptions of the usage scenarios of this application, but are for exemplary descriptions.

[0080] It is obvious to those skilled in the art that the present application is not limited to the details of the exemplary embodiments described above, and that the present application can be implemented in other specific forms without departing from the spirit or essential features of the present application. Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims be included in the present application. Any reference numeral in a claim should not be regarded as limiting the claim to which it relates.

Claims

1. A digital calligraphy system, characterized in that: A touch screen comprising a customized touch module (210) and an ink display screen (270) is laminated to form a system. The system runs a block touch conversion module (25) in calligraphy software (20) to receive writing actions of a capacitive brush on the touch module (210), convert the writing actions and display them on the ink display screen (270).

2. The digital calligraphy system according to claim 1, characterized in that The density of the sensing unit of the customized touch module (210) reaches 0.4mm, which can obtain high-precision touch raw data and provide it to the upper-level application. However, it should be understood that 0.4mm is only an order of magnitude display and is not a restrictive description. The surface of the module is frosted and has the touch of rice paper.

3. The ink display screen according to claim 1, characterized in that High definition, working in local, high-frequency refresh mode.

4. The digital calligraphy system according to claim 1, characterized in that Use machine learning to train calligraphy software to obtain the best block touch conversion algorithm.

5. The block touch conversion algorithm according to claim 4, characterized in that The algorithm model parameters are obtained using simple stroke model training and auxiliary stroke model training.

6. A calligraphy case, characterized in that: The device comprises a customized touch module (210) and an ink screen (270), and is paired with an ordinary tablet using Bluetooth. The calligraphy shell is placed on the ordinary tablet screen for calligraphy creation.

7. The calligraphy case according to claim 6, characterized in that The calligraphy shell uses a transparent material as the base of the touch module, wherein the (260) component is a snap-on fixture adapted to a variety of different screen sizes.

8. An auxiliary calligraphy tablet, comprising a touch module (210) and an ink display screen (270), characterized in that The touch module is made of an opaque substrate, and the auxiliary calligraphy tablet and the ordinary tablet are placed separately. Calligraphy is created on the auxiliary calligraphy tablet, and handwriting is viewed on the ordinary tablet.

9. A digital calligraphy app that runs on an OS that provides block touch data. Its features are Based on low-pixel touch data, a block touch conversion algorithm is used to draw brush strokes.

10. According to the digital calligraphy system of claim 1, the ink screen can be replaced by other ordinary screens. Such modification is regarded as a direct extension of the present invention and is also regarded as the protection scope of the present invention.