Information processing apparatus and control method
The device addresses the challenge of maintaining handwriting quality by identifying and adjusting continuous strokes in complex characters, ensuring accurate rendering and thickness adjustments, thus displaying beautiful characters with preserved handwriting.
Patent Information
- Application Number
- JP2023075085
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-04-28
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2043-04-28
AI Technical Summary
Conventional information processing devices struggle to maintain the quality of handwriting, particularly for complex characters like kanji, when continuous calligraphy is mixed in, and they fail to accurately handle delicate ink thickness adjustments and continuous writing styles.
The device includes a continuous calligraphy determining unit to identify continuous strokes, a classification processing unit to divide strokes into character elements, and a rendering processing unit to adjust stroke shape based on pen pressure and brush information, ensuring accurate rendering of handwritten characters.
The solution enables the display of beautiful characters, especially kanji, while preserving the quality of handwriting, even when continuous calligraphy is involved, by adjusting stroke thickness and shape based on pen pressure and brush characteristics.
Smart Images

Figure 2024159232000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing device and a control method. [Background technology]
[0002] In recent years, information processing devices such as tablet terminals that allow handwritten character input have become widespread (see, for example, Patent Document 1). In such conventional information processing devices, a technique is known that recognizes handwritten input characters and converts them into a specified font. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] Patent Publication No. 2021-43851 Summary of the Invention [Problem to be solved by the invention]
[0004] In the above-mentioned conventional information processing device, it is possible to display beautiful characters by converting handwritten input characters into a predetermined font. However, in the conventional information processing device, by replacing characters with a predetermined font, the user's handwriting is lost, and the freedom of handwriting, such as the aspect ratio of the characters and the size of the characters, is lost, and the merits of handwriting are lost. On the other hand, in the conventional information processing device, it is also possible to adjust the rendering thickness of the drawing ink based on the pen pressure and display characters drawn with a pen. However, in such cases, drawing that requires delicate adjustment of the ink thickness, such as the tome, hane, and harai, which is often necessary especially for complex characters such as kanji, is very difficult on a tablet terminal, which has a hard and slippery surface compared to paper.
[0005] In addition, in the conventional information processing device, it is difficult to appropriately handle cases where continuous writing styles such as cursive writing and script writing are mixed in part of a character. In other words, in the conventional information processing device, it is difficult to display beautiful characters while maintaining the good qualities of handwriting when continuous writing styles are mixed in.
[0006] The present invention has been made to solve the above problems, and its object is to provide an information processing device and control method that can display beautiful characters, particularly kanji, while maintaining the quality of handwriting, even when continuous writing styles are mixed in handwritten input. [Means for solving the problem]
[0007] In order to solve the above problem, one aspect of the present invention is an information processing device comprising: a continuous writing style determination unit that determines whether an input handwritten stroke is continuous writing style, which is a style of writing in which strokes are continuous; a division processing unit that divides the handwritten stroke at corners and generates divided partial strokes; a classification processing unit that classifies the partial strokes divided by the division processing unit into any of a plurality of character elements that are basic patterns that constitute Chinese character characters; and a rendering processing unit that, when the continuous writing style determination unit determines that the handwritten stroke is not of the continuous writing style, adjusts a shape of the partial stroke based on brush information and writing pressure characteristics corresponding to the character element classified by the classification processing unit, and displays the handwritten stroke with the adjusted shape on a display unit; and, when the continuous writing style determination unit determines that the handwritten stroke is of the continuous writing style, adjusts a shape of the handwritten stroke based on the writing pressure at the time the handwritten stroke was input, and displays the handwritten stroke on the display unit.
[0008] In one aspect of the present invention, in the information processing device described above, the continuous writing style determination unit may determine that the handwritten stroke is in the continuous writing style when the length of the handwritten stroke is equal to or greater than a threshold length.
[0009] In one aspect of the present invention, in the information processing device described above, the continuous writing style determination unit may be a brush that draws the handwritten strokes, and may change the threshold length depending on a thickness of a selected brush.
[0010] In addition, in one aspect of the present invention, in the above-mentioned information processing device, the continuous writing style determination unit may determine that the handwritten stroke is in continuous writing style when the number of corner portions contained in the handwritten stroke is equal to or greater than a threshold number.
[0011] In addition, in one aspect of the present invention, in the above-mentioned information processing device, the continuous writing determination unit may determine that the handwritten stroke is continuous writing when the sum of the angle changes of the drawing lines in the handwritten stroke is greater than or equal to a threshold angle.
[0012] In one aspect of the present invention, in the information processing device, the continuous writing style determination unit may determine that the handwritten stroke is in the continuous writing style when the handwritten stroke has a self-intersection.
[0013] In addition, one aspect of the present invention is that in the above-mentioned information processing device, when the continuous writing style determination unit determines that the handwritten stroke is the continuous writing style, the rendering processing unit may adjust the thickness of the handwritten stroke and display it on the display unit so that the higher the writing pressure, the thicker the handwritten stroke, and the faster the drawing speed of the handwritten stroke, the thinner the handwritten stroke.
[0014] In addition, one aspect of the present invention is that in the above-mentioned information processing device, when the continuous writing style determination unit determines that the handwritten stroke is in the continuous writing style, the rendering processing unit may adjust the maximum thickness of the handwritten stroke by normalizing it based on a standard thickness based on a standard writing pressure.
[0015] In addition, one aspect of the present invention is an information processing device comprising: a brush information storage unit that stores, in correspondence with the character elements, brush information including the size, aspect ratio, and angle of the brush used; and a pen pressure characteristics storage unit that stores, in correspondence with the character elements, a pressure curve indicating the relationship between the stroke position and pen pressure in the character elements; and when the continuous writing style determination unit determines that the handwritten stroke is not of the continuous writing style, the rendering processing unit may obtain the brush information corresponding to the classified character element from the brush information storage unit and obtain the pressure curve corresponding to the classified character element from the pen pressure characteristics storage unit, and adjust the shape of the partial stroke based on the obtained brush information and pressure curve.
[0016] In addition, one aspect of the present invention is an information processing device as described above, which includes the display unit, a touch sensor unit that detects the contact and contact position of an operating medium on a detection surface on a screen of the display unit, and an input processing unit that detects the input of the handwritten stroke based on the contact position of the operating medium detected by the touch sensor unit, wherein the input processing unit detects the input of the handwritten stroke and displays a movement trajectory of the contact position of the operating medium on the display unit as the input handwritten stroke, and the rendering processing unit replaces the input handwritten stroke with the handwritten stroke whose shape has been adjusted and displays it on the display unit.
[0017] Another aspect of the present invention is a control method for an information processing device, the control method including the steps of: a continuous writing style determination unit determining whether an input handwritten stroke is continuous writing style, which is a style of writing in which strokes are continuous; a segmentation processing unit dividing the handwritten stroke at corners to generate divided partial strokes; a classification processing unit classifying the partial stroke divided by the segmentation processing unit into any of a plurality of character elements which are basic patterns that constitute Chinese character characters; and a rendering processing unit, if the continuous writing style determination unit determines that the handwritten stroke is not of the continuous writing style, adjusting a shape of the partial stroke based on brush information and writing pressure characteristics corresponding to the character element classified by the classification processing unit, and displaying the handwritten stroke with the adjusted shape on a display unit; and, if the continuous writing style determination unit determines that the handwritten stroke is of the continuous writing style, adjusting a shape of the handwritten stroke based on the writing pressure at the time the handwritten stroke was input, and displaying the handwritten stroke on the display unit. Effect of the Invention
[0018] According to the above-described aspects of the present invention, even when consecutive calligraphy styles are mixed in handwritten input, beautiful characters (especially kanji) can be displayed while maintaining the quality of the handwriting. [Brief description of the drawings]
[0019] [Figure 1] 1 is an external view showing an example of a tablet terminal and an information processing system according to an embodiment of the present invention. [Diagram 2] 1 is a diagram illustrating an example of a main hardware configuration of a tablet terminal and an information processing system according to an embodiment of the present invention. [Diagram 3] 1 is a block diagram showing an example of a functional configuration of a tablet terminal and an information processing system according to an embodiment of the present invention. [Figure 4] 5A and 5B are diagrams illustrating an example of data in a brush information storage unit according to the embodiment. [Diagram 5] 3A and 3B are diagrams illustrating examples of character elements according to the present embodiment. [Figure 6] 5A and 5B are diagrams illustrating an example of data in a writing pressure characteristics storage unit according to the embodiment. [Figure 7] 11A and 11B are diagrams illustrating the determination conditions for continuous writing on a tablet terminal according to the present embodiment. [Figure 8] 1 is a diagram showing an overview of a handwriting input process of continuous calligraphy on a tablet terminal according to an embodiment of the present invention. [Figure 9] 1 is a diagram showing an overview of handwriting input processing using character elements on a tablet terminal according to an embodiment of the present invention. [Figure 10] 11A to 11C are diagrams illustrating an example of correction and normalization processing of pen pressure of a tablet terminal according to the present embodiment. [Figure 11] 10 is a flowchart illustrating an example of a handwriting input process of the tablet terminal according to the embodiment. [Figure 12] 11A to 11C are diagrams illustrating an example of a display of a handwriting input process for characters that do not include continuous calligraphy on a tablet terminal according to the embodiment. [Figure 13] 11A to 11C are diagrams illustrating an example of a display of a handwritten input process for characters including continuous calligraphy on a tablet terminal according to the embodiment. [Figure 14] 10 is a flowchart illustrating an example of a learning process of the tablet terminal according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0020] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An information processing apparatus and a control method according to an embodiment of the present invention will be described below with reference to the drawings.
[0021] FIG. 1 is an external view showing an example of a tablet terminal 1 and an information processing system 100 according to the present embodiment. 1, the information processing system 100 includes a tablet terminal 1 and a pen 30. In this embodiment, the tablet terminal 1 will be described as an example of an information processing device.
[0022] The tablet terminal 1 has a touch screen 20 installed on one main surface of a housing CS1, and an application program such as a memo pad or drawing program is executed using a pen 30. The tablet terminal 1 is capable of handwritten input using the pen 30. Note that the pen 30 can be set to various types of brushes when inputting handwritten text.
[0023] The touch screen 20 includes a display unit 21 and a touch sensor unit 22, and the display unit 21 displays various types of information on a display screen DF. The touch sensor unit 22 is disposed over the display unit 21, and detects when the pen 30 touches the display screen DF of the display unit 21, and detects the touch position of the pen 30. The touch screen 20, the display unit 21, and the touch sensor unit 22 will be described in detail later.
[0024] Next, the main hardware configuration of the tablet terminal 1 and the information processing system 100 will be described with reference to FIG. FIG. 2 is a diagram showing an example of the main hardware configuration of the tablet terminal 1 and the information processing system 100 according to this embodiment.
[0025] 2, the information processing system 100 includes a tablet terminal 1 and a pen 30. The tablet terminal 1 also includes a processor 11, a main memory 12, a flash memory 13, a touch screen 20, a peripheral device 23, an audio system 24, a microphone 25, a speaker 26, a baseband chip 27, a wireless unit 28, and a wireless communication unit 29.
[0026] The processor 11 is, for example, an application processor including a CPU (Central Processing Unit). The processor 11 controls the entire tablet terminal 1.
[0027] The main memory 12 is a writable memory used as a read area for the execution program of the processor 11 or as a work area for writing processing data for the execution program. The main memory 12 is composed of, for example, a plurality of DRAM (Dynamic Random Access Memory) chips. The execution program includes an OS (Operating System), various device drivers for operating the hardware of peripheral devices, various services / utilities, application programs (application software), and the like.
[0028] The flash memory 13 is, for example, a flash EEPROM (Electrically Erasable Programmable Read Only Memory), and stores the OS, various drivers, various services / utilities, application programs (hereinafter sometimes referred to as applications), and various data.
[0029] The display unit 21 is, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display, and displays a display screen based on drawing data (display data) output from the processor 11. The display unit 21 is capable of displaying, for example, the movement trajectory (drawing trajectory) of the pen 30.
[0030] The touch sensor unit 22 detects the position of an operation medium such as the pen 30 on the screen of the display unit 21 (on the display screen DF) and the contact of the pen 30 with the screen. The touch sensor unit 22 detects the contact position of the pen 30 on the screen of the display unit 21.
[0031] The peripheral device 23 is, for example, a Wireless Local Area Network (WLAN) module, a Global Positioning System (GPS) module, and sensors such as an acceleration sensor.
[0032] The audio system 24 is, for example, an audio IC (Integrated Circuit), and inputs, records, plays back, and outputs sound data. For example, a microphone 25 and a speaker 26 are connected to the audio system 24. For example, the audio system 24 outputs sound data collected by the microphone 25 to the processor 11 or the baseband chip 27. In addition, the audio system 24 converts sound data acquired from the processor 11 or the baseband chip 27 into a sound signal, and outputs the sound signal to the speaker 26.
[0033] The microphone 25 picks up sounds around the tablet terminal 1. For example, the microphone 25 picks up sounds such as the user's voice when audio integration with another terminal is performed. The speaker 26 outputs various sounds to the outside of the tablet terminal 1. For example, the speaker 26 outputs (emits) sounds received from other terminals when performing audio integration with other terminals.
[0034] The baseband chip 27 is a dedicated IC that controls wireless communication such as 4G (fourth generation mobile communication system) and 5G (fourth generation mobile communication system). The baseband chip 27 outputs, for example, audio data received using the wireless unit 28 to the speaker 26 via the audio system 24. The baseband chip 27 also acquires, for example, audio data collected from the microphone 25 via the audio system 24, and outputs the data by the mobile communication system using the wireless unit 28. The baseband chip 27 also communicates input / output data of data communication by the mobile communication system with the processor 11.
[0035] The radio unit 28 is a wireless communication device including an antenna for performing wireless communication in a mobile communication system. The wireless communication unit 29 is, for example, a Bluetooth (registered trademark) module, and performs wireless communication with the pen 30. The wireless communication unit 29 receives, for example, sensor information detected by the pen 30 (contact pressure, angle of the pen 30, etc.) from the pen 30.
[0036] The pen 30 is a pen-shaped operating medium, such as a touch pen, a stylus pen, etc. The pen 30 is capable of communicating with the tablet terminal 1 by wireless communication such as Bluetooth (registered trademark). The pen 30 includes a wireless communication unit 31, a sensor unit 32, a memory 33, and an MCU 34.
[0037] The wireless communication unit 31 is, for example, a Bluetooth (registered trademark) module, and performs wireless communication between the tablet terminal 1 and the pen 30. The wireless communication unit 31 transmits, for example, sensor information detected by the sensor unit 32 to the tablet terminal 1. The sensor unit 32 detects sensor information such as the contact pressure (writing pressure) of the pen 30, the angle of the pen 30, and the like.
[0038] The memory 33 is composed of, for example, a RAM, and stores part of the control program for the pen 30 and processing data for the MCU 34. Note that the memory 33 may include a flash memory, a ROM, and the like in addition to the RAM.
[0039] The MCU (Micro Controller Unit) 34 includes a CPU, memories such as ROM and RAM, and I / O related components, and generally controls the pen 30. The MCU 34 controls data communication with the tablet terminal 1 via the wireless communication unit 31. The MCU 34 transmits, for example, sensor information detected by the sensor unit 32 to the tablet terminal 1 via the wireless communication unit 31.
[0040] Next, the functional configuration of the information processing system 100 according to the present embodiment will be described with reference to FIG. FIG. 3 is a block diagram showing an example of the functional configuration of the tablet terminal 1 and the information processing system 100 according to this embodiment.
[0041] As shown in FIG. 3, the information processing system 100 includes a tablet terminal 1 and a pen 30. The tablet terminal 1 includes a control unit 10, a touch screen 20, and a storage unit 40.
[0042] The touch screen 20 includes a display unit 21 and a touch sensor unit 22 . The touch sensor unit 22 detects the contact and contact position of an operating medium (pen 30) on a detection surface on the screen of the display unit 21. The touch sensor unit 22 detects, for example, the contact position of the pen 30 on the screen of the display unit 21 as a pen input. The touch sensor unit 22 detects, for example, contact position information indicating two-dimensional coordinates on the display screen DF of the display unit 21 as the contact position of the pen 30. The touch sensor unit 22 outputs the contact position information to the control unit 10.
[0043] The memory unit 40 is a memory unit realized, for example, by the main memory 12 or the flash memory 13, and includes a brush information memory unit 41, a pen pressure characteristic memory unit 42, a CE classification information memory unit 43, a handwritten stroke memory unit 44, a partial stroke memory unit 45, and a learning data memory unit 46.
[0044] The brush information storage unit 41 is, for example, a storage unit realized by the main memory 12, and stores brush information corresponding to character elements (CE: Character Elements) which are basic parts (basic patterns constituting a kanji character) obtained by dividing a kanji character. Here, an example of data in the brush information storage unit 41 will be described with reference to FIG. 4.
[0045] FIG. 4 is a diagram showing an example of data stored in the brush information storage unit 41 in this embodiment. As shown in FIG. 4, the brush information storage unit 41 stores a CEID (character element ID), a brush size, an aspect ratio, a standard brush angle, and a standard stroke angle in association with each other.
[0046] CEID is character element identification information that identifies a character element. Brush size indicates the size of the brush. Aspect ratio indicates the aspect ratio of the brush, that is, the ratio of width to height. Standard brush angle indicates the standard (reference) brush angle of the character element, and standard stroke angle indicates the standard (reference) angle of the stroke of the character element.
[0047] 4 shows that the brush size of the character element with CEID "CE001" is "R1" and the aspect ratio is "XX1:YY1". It also shows that the standard brush angle of the character element is "α1" and the standard stroke angle is "β1".
[0048] Now, with reference to FIG. 5, the character elements will be described. Fig. 5 is a diagram showing an example of character elements in this embodiment, in which character elements constituting the Chinese character "eternal" will be described.
[0049] As shown in Fig. 5, the Chinese character "eternity" is decomposed into character elements CE1 to CE8. Character elements CE1 to CE8 include strokes such as a stop, a swing, and a line, and corresponding brush information is defined for each of character elements CE1 to CE8 as shown in Fig. 4 above.
[0050] In this embodiment, a plurality of character elements including the above-mentioned character elements CE1 to CE8, which are determined from various kanji characters, are defined, and rendering (drawing) is performed for each defined character element, thereby achieving beautiful drawing of characters when handwriting inputting kanji characters. It should be noted that "beautiful characters" here refers to characters that are rendered similarly to a preferred font, for example.
[0051] Returning to the explanation of Fig. 3, the writing pressure characteristic storage unit 42 is, for example, a storage unit realized by the main memory 12, and stores a pressure curve (writing pressure characteristic) indicating the relationship between the stroke position and writing pressure in a character element. The writing pressure characteristic storage unit 42 stores, for example, identification information of a character element and the writing pressure characteristic in association with each other. Here, an example of data in the writing pressure characteristic storage unit 42 will be explained with reference to Fig. 6.
[0052] FIG. 6 is a diagram showing an example of data in the writing pressure characteristics storage unit 42 in this embodiment. As shown in FIG. 6, the writing pressure characteristic storage unit 42 stores the CEID and the writing pressure characteristic in association with each other.
[0053] For example, in the example shown in Fig. 6, the pen pressure characteristic corresponding to CEID "CE001" is a pressure curve like waveform W1. Note that the horizontal axis of the graph showing waveform W1 indicates stroke position, and the vertical axis indicates pen pressure (pressure). The pressure curve indicates that the higher the pen pressure, the thicker the stroke line will be when rendered, and the lower the pen pressure, the thinner the stroke line will be when rendered.
[0054] The pen pressure characteristics (waveform W1) corresponding to CEID "CE001" indicates a character element (character element CE1 shown in FIG. 5) in which the stroke line becomes thicker as the stroke position advances and then suddenly thins at the end.
[0055] For example, the pen pressure characteristics corresponding to CEID "CE002" indicate a pressure curve like waveform W2. The pen pressure characteristics (waveform W2) corresponding to CEID "CE002" indicate a character element (character element CE2 shown in FIG. 5) in which the stroke line thins once as the stroke position advances and then thickens again.
[0056] Returning to the description of Fig. 3, the CE classification information storage unit 43 is, for example, a storage unit realized by the main memory 12, and stores information for classifying character elements. The CE classification information storage unit 43 is, for example, a trained model generated by machine learning, and stores a classification model for classifying a partial stroke, which will be described later, into one of a plurality of character elements. The CE classification information storage unit 43 is an example of a model storage unit.
[0057] The handwritten stroke storage unit 44 is, for example, a storage unit realized by the main memory 12, and stores handwritten stroke data detected by the touch sensor unit 22 using the pen 30. The handwritten stroke storage unit 44 stores, as handwritten strokes, collection data of time-series detection points detected by the touch sensor unit 22 when handwriting inputting a Chinese character, for example. Note that the handwritten stroke storage unit 44 stores, for example, position information of the time-series detection points in association with contact pressure (pen pressure).
[0058] The partial stroke storage unit 45 is a storage unit realized by, for example, the main memory 12, and stores data of partial strokes generated from handwritten strokes. The partial strokes will be described in detail later.
[0059] The learning data storage unit 46 is a storage unit realized by, for example, the main memory 12, and stores learning data for generating the above-mentioned classification model. The learning data storage unit 46 stores learning data in which partial strokes of a pre-specified character handwritten by a user are associated with character elements.
[0060] The control unit 10 is a functional unit realized, for example, by the processor 11 executing a program stored in the main memory 12 or the flash memory 13, and executes various processes based on an OS (for example, Android (registered trademark)). The control unit 10 includes an input processing unit 101, a continuous writing style determination unit 102, a division processing unit 103, a classification processing unit 104, a rendering processing unit 105, a display processing unit 106, and a learning processing unit 107.
[0061] The input processing unit 101 is a functional unit that is realized by the processor 11 executing a program stored in the main memory 12 or the flash memory 13. The input processing unit 101 is, for example, a device driver that controls an input by the touch sensor unit 22, and detects the position and contact of the pen 30 on the screen of the display unit 21 (on the display screen DF) by the input from the touch sensor unit 22.
[0062] Furthermore, the input processing unit 101 may acquire, for example, contact pressure (writing pressure) and angle information of the pen 30 from the pen 30. In this embodiment, the input processing unit 101 detects position information of detection points in time series and the corresponding contact pressure (writing pressure).
[0063] The input processing unit 101 detects the input of a handwritten stroke based on the contact position of an operation medium (for example, the pen 30) detected by the touch sensor unit 22. That is, the input processing unit 101 associates time-series position information of a detection point, which is contact position information (position coordinates) of the detected pen 30, with a contact pressure (writing pressure) as a handwritten stroke, and stores the information in the handwritten stroke storage unit 44.
[0064] In addition, a handwritten stroke, or simply a stroke, refers to a collection of data of detection points of the trajectory of the pen tip from the time the pen 30 makes contact (touch) to the time the pen 30 releases contact (up), and may include a display of the trajectory of the pen tip on the screen of the display unit 21 (on the display screen DF).
[0065] Furthermore, the input processing unit 101 detects the input of handwritten strokes, and displays the movement trajectory of the contact position of the operation medium as the input handwritten strokes on the display unit 21. The input processing unit temporarily displays the input handwritten strokes on the display unit 21 via the display processing unit 106, which will be described later.
[0066] The continuous writing style determination unit 102 is a functional unit that is realized when the processor 11 executes a program stored in the main memory 12 or the flash memory 13. The continuous writing style determination unit 102 determines whether or not the input handwritten strokes are continuous writing styles, which are writing styles in which strokes are continuous.
[0067] Here, continuous writing is writing in which a plurality of character elements, such as running script, cursive script, and cursive script, are continuously drawn in one stroke. If an input handwritten stroke is a continuous writing, it is difficult to classify the stroke into the character elements described above, and it is difficult to realize a beautiful character by rendering using the character elements. Therefore, in this embodiment, in order to perform a rendering process for continuous writing that is different from that for character elements, the continuous writing determination unit 102 determines whether or not the input handwritten stroke is a continuous writing.
[0068] The continuous writing style determination unit 102 determines whether or not a handwritten stroke is continuous writing style based on the determination conditions (a) to (d) shown in Fig. 7. The continuous writing style determination unit 102 determines whether or not a handwritten stroke stored in the handwritten stroke storage unit 44 by the input processing unit 101 is continuous writing style.
[0069] FIG. 7 is a diagram for explaining the conditions for determining continuous writing in the tablet terminal 1 according to this embodiment. As shown in FIG. 7, the continuous writing style determination unit 102 determines whether or not a handwritten stroke is in continuous writing style based on the determination conditions (a) to (d).
[0070] (a) The continuous writing style determination unit 102 determines that a handwritten stroke is continuous writing style when the length of the handwritten stroke is equal to or greater than a threshold length. The continuous writing style determination unit 102 determines that a handwritten stroke is continuous writing style when the length of the handwritten stroke is equal to or greater than a threshold length, for example, as in the case of handwritten stroke CW1 in FIG. 7. The threshold length is changed and set, for example, according to the thickness of the selected brush. For example, the threshold length is set to be longer as the brush thickness increases and shorter as the brush thickness decreases.
[0071] (b) The continuous writing style determination unit 102 determines that a handwritten stroke is in continuous writing style when the number of corners included in the handwritten stroke is equal to or greater than a threshold number. For example, the continuous writing style determination unit 102 determines that a handwritten stroke is in continuous writing style when the number of corners included in the handwritten stroke is equal to or greater than a threshold number, as in the case of the handwritten stroke CW2 and the handwritten stroke CW2 in FIG. 7.
[0072] (c) The continuous writing style determination unit 102 determines that a handwritten stroke is continuous writing style when the sum of the angle changes of the drawing lines in the handwritten stroke is equal to or greater than a threshold angle. For example, the continuous writing style determination unit 102 determines that a handwritten stroke is continuous writing style when the sum of the angle changes of the handwritten stroke is equal to or greater than a threshold angle, as in the case of the handwritten stroke CW4 in FIG. 7.
[0073] (d) The continuous writing style determination unit 102 determines that a handwritten stroke is of continuous writing style when the handwritten stroke has a self-intersection. For example, the continuous writing style determination unit 102 determines that a handwritten stroke is of continuous writing style when the handwritten stroke has a self-intersection, such as the handwritten stroke CW5 in FIG. 7.
[0074] In this manner, the continuous writing determination unit 102 determines that a handwritten stroke is in continuous writing when any one of the above determination conditions (a) to (d) is satisfied. For example, the continuous writing determination unit 102 determines that a handwritten stroke ST1 as shown in step S11 of FIG. 8 (to be described later) is continuous writing based on the above-mentioned determination conditions (a) to (d).
[0075] As shown in Fig. 8, when the handwritten stroke is continuous writing, the tablet terminal 1 performs rendering processing on the handwritten stroke ST1 which is continuous writing, as shown in step S12. The rendering processing on the handwritten stroke which is continuous writing will be described in detail later. When the handwritten stroke is not continuous writing, the tablet terminal 1 performs processing on step S21 shown in Fig. 9 which will be described later.
[0076] The division processing unit 103 is a functional unit that is realized when the processor 11 executes a program stored in the main memory 12 or the flash memory 13. The division processing unit 103 divides handwritten strokes input by a user at corners to generate divided partial strokes. The division processing unit 103 divides handwritten strokes stored in the handwritten stroke storage unit 44 by the input processing unit 101 into partial strokes.
[0077] Specifically, as shown in step S21 in FIG. 9, the division processing unit 103 divides a handwritten stroke ST2 having two corners at the corners to generate partial strokes SS1, SS2, and SS3.
[0078] The division processing unit 103 stores the divided partial strokes in the partial stroke storage unit 45. Note that, when the continuous writing style determination unit 102 determines that the handwritten stroke is not of continuous writing style, the division processing unit 103 may execute a division process for dividing the handwritten stroke into partial strokes.
[0079] The classification processing unit 104 classifies the partial strokes divided by the division processing unit 103 into character elements, for example, based on a classification model that classifies character elements from partial strokes. Here, the classification model is a classification model generated by executing a learning process on learning data that associates partial strokes of a character handwritten by a user of a pre-specified character with character elements. The classification processing unit 104 uses the classification model stored in the CE classification information storage unit 43 to classify the partial strokes into character elements.
[0080] 9, the classification processor 104 classifies, for example, the partial stroke SS1 into a character element CE2 and the partial stroke SS2 into a character element CE3 using the classification model. Also, the classification processor 104 classifies, for example, the partial stroke SS3 into a character element CE4 using the classification model. If the continuous writing style determination unit 102 determines that the handwritten stroke is not in continuous writing style, the classification processing unit 104 may classify the partial stroke into a character element.
[0081] The rendering processing unit 105 is a functional unit that is realized when the processor 11 executes a program stored in the main memory 12 or the flash memory 13. When the continuous writing style determination unit 102 determines that a handwritten stroke is in continuous writing style, the rendering processing unit 105 adjusts the shape of the handwritten stroke based on the writing pressure when the handwritten stroke was input, and displays it on the display unit 21. For example, the rendering processing unit 105 adjusts the shape of the handwritten stroke like the stroke CST1 shown in step S12 in FIG. 8, and displays it on the display unit 21.
[0082] When the handwritten stroke is a continuous calligraphy stroke, the rendering processor 105 adjusts the shape of the handwritten stroke by, for example, adopting a brush that is most commonly used for Chinese characters. The rendering processor 105 also adjusts the thickness of the handwritten stroke based on the writing pressure and the drawing speed, which is the moving speed of the pen 30. The rendering processor 105 corrects the writing pressure using the drawing speed according to the following formula (1).
[0083]
number
[0084] Here, a, b, c, d, and e are constants (parameters) for correction. L is the drawing distance of the handwritten stroke, and is the stroke position from the starting point of the handwritten stroke. P(L) is the writing pressure at stroke position L, and is the contact pressure of the pen 30 obtained from the pen 30. v(L) is the drawing speed at stroke position L. Ptmp(L) is the corrected writing pressure at stroke position L.
[0085] The rendering processor 105 uses the formula (1) to correct the writing pressure applied when drawing a handwritten stroke, and generates the corrected writing pressure Ptmp(L) as shown by the waveform W3 in FIG. As shown in formula (1), the corrected pen pressure Ptmp(L) is corrected to be lower as the pen pressure is higher and the drawing speed is faster.
[0086] Moreover, the rendering processor 105 further normalizes the corrected pen pressure Ptmp(L) according to the following formula (2).
[0087]
number
[0088] Here, Pmin is the minimum value of the pen pressure Ptmp(L), and Pmax is the maximum value of the pen pressure Ptmp(L). Furthermore, Nmax is, for example, the maximum value of the pen pressure when a character element is used, and is the maximum value of the standard pen pressure. The rendering processing unit 105 normalizes the corrected pen pressure Ptmp(L) according to equation (2) so that the maximum value of the pen pressure becomes Nmax.
[0089] FIG. 10 is a diagram showing an example of the correction process and normalization process of the pen pressure of the tablet terminal 1 in this embodiment. 10, the waveform W3 shows the change (pressure curve) in the pen pressure Ptmp(L) after correction using the above-mentioned formula (1), while the waveform W4 shows the pen pressure characteristic (pressure curve) after normalization of the pen pressure Ptmp(L) after correction using the above-mentioned formula (2).
[0090] The rendering processing unit 105 generates the corrected pen pressure Ptmp(L) of the waveform W3 using the above-mentioned equation (1), and further generates a normalized pressure curve as shown in the waveform W4 using the above-mentioned equation (2).
[0091] When the handwritten stroke is in continuous writing, the rendering processing unit 105 adjusts the shape of the handwritten stroke using a normalized pressure curve so that the higher the writing pressure of the normalized pressure curve, the thicker the handwritten stroke is, and the lower the writing pressure of the normalized pressure curve, the thinner the handwritten stroke is, and displays the handwritten stroke with the adjusted shape on the display unit 21.
[0092] That is, when the continuous writing style determination unit 102 determines that a handwritten stroke is continuous writing style, the rendering processing unit 105 uses the above-mentioned formula (1) to adjust the thickness of the handwritten stroke so that the higher the writing pressure, the thicker the handwritten stroke, and the faster the drawing speed of the handwritten stroke, the thinner the handwritten stroke, and displays the adjusted thickness on the display unit 21. Furthermore, when the continuous writing style determination unit 102 determines that a handwritten stroke is continuous writing style, the rendering processing unit 105 uses the above-mentioned formula (2) to adjust the maximum thickness of the handwritten stroke by normalizing it with a standard thickness based on a standard writing pressure.
[0093] In addition, when the continuous writing style determination unit 102 determines that the handwritten stroke is not continuous writing style, the rendering processing unit 105 adjusts the shape of the partial stroke based on the brush information and writing pressure characteristics corresponding to the character element classified by the classification processing unit 104, and displays the handwritten stroke with the adjusted shape on the display unit 21.
[0094] When the handwritten stroke is not in continuous calligraphy, the rendering processing unit 105, for example, acquires brush information corresponding to the character element from the brush information storage unit 41, and acquires pen pressure characteristics (pressure curve) corresponding to the character element from the pen pressure characteristics storage unit 42. The rendering processing unit 105 adjusts the shape of the partial stroke based on the brush information and the pen pressure characteristics (pressure curve) and renders it. The rendering processing unit 105 sets the brush size, aspect ratio, and brush angle according to the brush information, adjusts the line thickness according to the pen pressure characteristics (pressure curve), and displays the partial stroke on the display unit 21.
[0095] The rendering processor 105 recombines the partial strokes with the adjusted brush angle and line thickness to generate an adjusted handwritten stroke. Specifically, as shown in step S23 of FIG. 9, the rendering processor 105 adjusts and recombines the partial strokes SS1, SS2, and SS3 to generate the rendered handwritten stroke CST2.
[0096] The rendering processing unit 105 replaces the input handwritten strokes and displays, as continuous calligraphy, on the display unit 21, the handwritten strokes whose shape has been adjusted or the handwritten strokes based on the partial strokes whose shape has been adjusted. The rendering processing unit 105 causes the display unit 21 to display the adjusted handwritten strokes via the display processing unit 106, which will be described later.
[0097] The display processing unit 106 is a functional unit realized by the processor 11 executing a program stored in the main memory 12 or the flash memory 13. The display processing unit 106 causes the display unit 21 to display a movement trajectory on the screen where the pen 30 has come into contact with the screen and moved, based on the contact position of the pen 30 detected by the touch sensor unit 22.
[0098] The display processing unit 106 displays the input handwritten strokes on the display unit 21 once, and then replaces the input handwritten strokes with handwritten strokes whose shape has been adjusted (handwritten strokes whose shape has been adjusted as continuous calligraphy, or handwritten strokes based on partial strokes whose shape has been adjusted) and displays them on the display unit 21.
[0099] The learning processing unit 107 is a functional unit that is realized by the processor 11 executing a program stored in the main memory 12 or the flash memory 13. The learning processing unit 107 executes a learning process on the learning data stored in the learning data storage unit 46 to generate a classification model.
[0100] Here, the learning data is a correspondence between partial strokes of a character handwritten by a user of a pre-specified character and character elements, and is generated based on handwritten strokes input by handwriting multiple times for multiple characters including character elements. The learning processing unit 107 stores the generated classification model (learning result) in the CE classification information storage unit 43.
[0101] Next, the operation of the tablet terminal 1 according to this embodiment will be described with reference to the drawings. FIG. 8 is a diagram showing an overview of a handwriting input process in continuous calligraphy on the tablet terminal 1 according to this embodiment.
[0102] 8, the tablet terminal 1 first determines whether or not an input handwritten stroke is continuous calligraphy (step S11). The continuous calligraphy determination unit 102 of the tablet terminal 1 determines that the handwritten stroke is continuous calligraphy if any of the above-mentioned determination conditions (a) to (d) is satisfied.
[0103] When the continuous writing style determination unit 102 determines that the handwritten stroke is continuous writing style, the tablet terminal 1 employs the brush that is most commonly used for kanji, and adjusts the shape of the handwritten stroke that is continuous writing style using the pressure curve generated using the above-mentioned equations (1) and (2), and displays it on the display unit 21 (step S12). When the continuous writing style determination unit 102 determines that the handwritten stroke is not in continuous writing style, the tablet terminal 1 advances the process to step S21 shown in FIG.
[0104] FIG. 9 is a diagram showing an overview of handwriting input processing using character elements on the tablet terminal 1 in this embodiment. 9, when the continuous writing style determination unit 102 determines that the handwritten stroke is not of continuous writing style, the tablet terminal 1 divides the handwritten stroke at its corners to generate partial strokes (step S21). The division processing unit 103 of the tablet terminal 1 divides the handwritten stroke to generate partial strokes.
[0105] Next, the tablet terminal 1 classifies the divided partial strokes into simple character elements using the classification model (step S22). The classification processing unit 104 of the tablet terminal 1 executes a classification process to classify the partial strokes into character elements using the classification model stored in the CE classification information storage unit 43.
[0106] When the tablet terminal 1 classifies the partial stroke into a character element, it adjusts the shape of the handwritten stroke based on the classified character element and displays it on the display unit 21 (step S23).
[0107] Next, with reference to FIG. 11, a detailed description will be given of the handwriting input process of the tablet terminal 1 shown in FIG. 8 and FIG. FIG. 11 is a flowchart showing an example of handwriting input processing of the tablet terminal 1 in this embodiment.
[0108] As shown in FIG. 11, the tablet terminal 1 first determines whether or not a handwritten stroke has been detected (step S101). The input processing unit 101 of the tablet terminal 1 determines whether or not a handwritten stroke has been detected by the touch sensor unit 22. If the input processing unit 101 detects a handwritten stroke (step S101: YES), the process proceeds to step S102. If the input processing unit 101 does not detect a handwritten stroke (step S101: NO), the process returns to step S101.
[0109] In step S102, the input processing unit 101 temporarily displays the handwritten strokes on the display unit 21. The rendering used at this time is performed with a thickness according to the conventional writing pressure, or with a uniform thickness regardless of the writing pressure. That is, the input processing unit 101 displays the detected handwritten strokes on the display unit 21 via the display processing unit 106. The input processing unit 101 acquires the writing pressure corresponding to the detection point of each position from the pen 30, and stores the writing pressure and the handwritten stroke in association with each other in the handwritten stroke storage unit 44.
[0110] Next, the input processing unit 101 determines whether the pen 30 has been removed from the screen of the display unit 21 (step S103). That is, the input processing unit 101 determines whether the input of the single handwritten stroke has been completed. If the pen 30 has been removed from the screen of the display unit 21 (the input of the handwritten stroke has been completed) (step S103: YES), the input processing unit 101 advances the process to step S104. If the pen 30 has not been removed from the screen of the display unit 21 (the input of the handwritten stroke has not been completed) (step S103: NO), the input processing unit 101 returns the process to step S102.
[0111] In step S104, the continuous writing style determination unit 102 of the tablet terminal 1 next determines whether or not the length of the handwritten stroke is equal to or greater than a threshold length. If the length of the handwritten stroke is equal to or greater than the threshold length (step S104: YES), for example, as in the case of the handwritten stroke CW1 shown in FIG. 7, the continuous writing style determination unit 102 determines that the handwritten stroke is of continuous writing style, and proceeds to step S112. If the length of the handwritten stroke is less than the threshold length (step S104: NO), the continuous writing style determination unit 102 proceeds to step S105.
[0112] In step S105, the continuous writing style determination unit 102 determines whether the number of corners of the handwritten stroke is equal to or greater than the threshold number. If the number of corners of the handwritten stroke is equal to or greater than the threshold number (step S105: YES), for example, as in the case of the handwritten stroke CW2 and the handwritten stroke CW3 shown in Fig. 7, the continuous writing style determination unit 102 determines that the handwritten stroke is of continuous writing style and proceeds to step S112. If the number of corners of the handwritten stroke is less than the threshold number (step S105: NO), the continuous writing style determination unit 102 proceeds to step S106.
[0113] In step S106, the continuous writing style determination unit 102 determines whether the sum of the angle changes of the handwritten strokes is equal to or greater than the threshold angle. If the sum of the angle changes of the handwritten strokes is equal to or greater than the threshold angle (step S106: YES), for example, as in the case of the handwritten stroke CW4 shown in Fig. 7, the continuous writing style determination unit 102 determines that the handwritten stroke is continuous writing and proceeds to step S112. If the sum of the angle changes of the handwritten strokes is less than the threshold angle (step S106: NO), the continuous writing style determination unit 102 proceeds to step S107.
[0114] In step S107, the continuous writing style determination unit 102 determines whether the handwritten stroke has a self-intersection. If the handwritten stroke has a self-intersection, such as the handwritten stroke CW5 shown in Fig. 7 (step S107: YES), the continuous writing style determination unit 102 determines that the handwritten stroke is continuous writing and proceeds to step S112. If the handwritten stroke does not have a self-intersection (step S107: NO), the continuous writing style determination unit 102 proceeds to step S108.
[0115] In step S108, the division processing unit 103 of the tablet terminal 1 divides the handwritten stroke at the corners to generate partial strokes. The division processing unit 103 divides the handwritten stroke to generate partial strokes, for example, as in step S21 shown in FIG. 9. The division processing unit 103 stores the generated partial strokes in the partial stroke storage unit 45.
[0116] Next, the classification processing unit 104 of the tablet terminal 1 classifies the partial strokes into character elements (step S109). The classification processing unit 104 classifies the partial strokes stored in the partial stroke storage unit 45 into character elements using the classification model stored in the CE classification information storage unit 43 (see step S22 in FIG. 9). Note that when there are no excessive parts in the handwritten strokes, the classification processing unit 104 classifies the handwritten strokes into character elements using the classification model.
[0117] Next, the classification processing unit 104 determines whether the confidence level of the classification process is equal to or higher than a threshold (step S110). If the confidence level of the classification process is equal to or higher than the threshold (step S110: YES), the classification processing unit 104 advances the process to step S111. If the confidence level of the classification process is lower than the threshold (step S110: NO), the classification processing unit 104 advances the process to step S112.
[0118] In step S111, the rendering processing unit 105 of the tablet terminal 1 adjusts the shape of the partial stroke based on the brush information and pen pressure characteristics (pressure curve) corresponding to the classified character element, and displays the handwritten stroke whose shape has been adjusted on the display unit 21. The rendering processing unit 105 causes the handwritten stroke whose shape has been adjusted to be displayed on the display unit 21 via the display processing unit 106 (see step S23 in FIG. 9). For example, in the case of handwritten input of the character "autumn" as in image G1 shown in FIG. 12, the rendering processing unit 105 displays on the display unit 21 a display as shown in image G2. After the process of step S111, the rendering processing unit 105 returns the process to step S101.
[0119] In step S112, the rendering processor 105 corrects the writing pressure of the handwritten stroke based on the drawing speed. The rendering processor 105 generates the corrected writing pressure Ptmp(L) using the above-mentioned formula (1).
[0120] Next, the rendering processing unit 105 normalizes the corrected pen pressure (step S113). The rendering processing unit 105 normalizes the corrected pen pressure Ptmp(L) by the above-mentioned formula (2) to generate the normalized corrected pen pressure Pnml(L).
[0121] Next, the rendering processing unit 105 adjusts the shape of the handwritten stroke using the most frequently used brush in Chinese characters and the normalized pen pressure, and displays it on the display unit 21 (step S114). The rendering processing unit 105 adjusts the thickness of the handwritten stroke that is continuous handwriting or a partial stroke that could not be classified into character elements using, for example, the pen pressure characteristics (pressure curve) after the normalized correction h of the waveform W4 in FIG. 10, and causes the handwritten stroke with the adjusted shape to be displayed on the display unit 21 via the display processing unit 106. For example, in the case of handwritten input including the cursive continuous writing of the character "autumn" like the image G3 shown in FIG. 12, the rendering processing unit 105 displays the display as shown in the image G4 on the display unit 21. After the processing in step S114, the rendering processing unit 105 returns the processing to step S101.
[0122] Next, referring to FIG. 14, the learning process of the tablet terminal 1 will be described. FIG. 14 is a flowchart showing an example of the learning process of the tablet terminal 1 in the present embodiment.
[0123] As shown in FIG. 14, the learning processing unit 107 of the tablet terminal 1 acquires handwritten strokes of a specific character a plurality of times for a plurality of users (step S201). The learning processing unit 107 acquires handwritten strokes of a specific character a plurality of times for a plurality of users via the input processing unit 101. Note that the specific character is preferably a plurality of characters that can learn a plurality of predetermined character elements.
[0124] Next, the learning processing unit 107 divides each handwritten stroke to generate partial strokes, labels the corresponding character elements as learning data (step S202). The learning processing unit 107 stores the learning data in which the partial strokes are associated with the labeling of the character elements in the learning data storage unit 46.
[0125] Next, the learning processing unit 107 performs machine learning using the learning data to generate a classification model that classifies character elements from the partial strokes (step S203). The learning processing unit 107 executes machine learning processing using the learning data stored in the learning data storage unit 46, and generates a classification model as a learning result.
[0126] Next, the learning processing unit 107 stores the classification model, which is a result of the learning, in the CE classification information storage unit 43 (step S204). After the process of step S204, the learning processing unit 107 ends the learning process.
[0127] As described above, the tablet terminal 1 (information processing device) according to this embodiment includes a continuous writing style determination unit 102, a division processing unit 103, a classification processing unit 104, and a rendering processing unit 105. The continuous writing style determination unit 102 determines whether or not an input handwritten stroke is continuous writing style, which is a style of writing in which strokes are continuous. The division processing unit 103 divides the handwritten stroke at its corners to generate divided partial strokes. The classification processing unit 104 classifies the partial stroke divided by the division processing unit 103 into one of a plurality of character elements that are basic patterns constituting a kanji character. When the continuous writing style determination unit 102 determines that the handwritten stroke is not continuous writing style, the rendering processing unit 105 adjusts the shape of the partial stroke based on the brush information and writing pressure characteristics corresponding to the character element classified by the classification processing unit 104, and displays the handwritten stroke with the adjusted shape on the display unit 21. When the continuous writing style determination unit 102 determines that the handwritten stroke is continuous writing style, the rendering processing unit 105 adjusts the shape of the handwritten stroke based on the writing pressure when the handwritten stroke was input, and displays it on the display unit 21.
[0128] As a result, the tablet terminal 1 according to this embodiment classifies partial strokes obtained by dividing a handwritten stroke into character elements, and adjusts the shape of each partial stroke based on the brush information and pen pressure characteristics corresponding to the character element, so that when inputting handwritten characters, it is possible to display beautiful characters (especially kanji) while maintaining the quality of the handwriting.
[0129] Furthermore, the tablet terminal 1 according to this embodiment determines whether a handwritten stroke is continuous calligraphy or not, and if the handwritten stroke is continuous calligraphy, adjusts the shape of the handwritten stroke based on the writing pressure when the handwritten stroke was input and displays it on the display unit 21, so that it can handle cases where continuous calligraphy is mixed, for example. Therefore, the tablet terminal 1 according to this embodiment can display beautiful characters while maintaining the quality of handwriting, even when continuous calligraphy is mixed.
[0130] In addition, the tablet terminal 1 according to the present embodiment adjusts the shape of partial strokes in accordance with character elements based on handwritten strokes, so that it is possible to display beautiful characters while maintaining the size and position of handwritten strokes. Unlike conventional techniques that simply replace a given font, the tablet terminal 1 according to the present embodiment does not erase the user's handwriting, and does not lose the freedom of handwriting, such as the aspect ratio of free handwritten characters and the size.
[0131] Furthermore, in this embodiment, the continuous writing style determination unit 102 determines that a handwritten stroke is in continuous writing style when the length of the handwritten stroke is equal to or greater than a threshold length. This allows the tablet terminal 1 according to the present embodiment to easily and appropriately determine whether or not a handwritten stroke is in continuous writing based on the length of the handwritten stroke.
[0132] Furthermore, in this embodiment, the continuous writing style determination unit 102 changes the threshold length in accordance with the thickness of the selected brush, which is a brush that renders handwritten strokes. This allows the tablet terminal 1 according to the present embodiment to easily and appropriately determine whether or not a handwritten stroke is in continuous writing based on the length of the handwritten stroke.
[0133] Furthermore, in this embodiment, the continuous writing style determination unit 102 determines that a handwritten stroke is in continuous writing style when the number of corners included in the handwritten stroke is equal to or greater than a threshold number. As a result, the tablet terminal 1 according to the present embodiment can easily and appropriately determine whether or not a handwritten stroke is in continuous writing based on the number of corners of the handwritten stroke.
[0134] Furthermore, in this embodiment, the continuous writing style determination unit 102 determines that a handwritten stroke is in continuous writing style when the sum of the angle changes of the drawing lines in the handwritten stroke is equal to or greater than a threshold angle.
[0135] As a result, the tablet terminal 1 according to the present embodiment can easily and appropriately determine whether or not a handwritten stroke is in continuous writing based on the total value of the angle change amount of the handwritten stroke.
[0136] Furthermore, in this embodiment, the continuous writing style determination unit 102 determines that a handwritten stroke is in continuous writing style if the handwritten stroke has a self-intersection. This allows the tablet terminal 1 according to the present embodiment to easily and appropriately determine whether or not a handwritten stroke is in continuous calligraphy based on the self-intersection of the handwritten stroke.
[0137] Furthermore, in this embodiment, when the continuous writing style determination unit 102 determines that a handwritten stroke is continuous writing style, the rendering processing unit 105 adjusts the thickness of the handwritten stroke so that the higher the writing pressure, the thicker the stroke, and the faster the drawing speed of the handwritten stroke, the thinner the stroke, and displays the handwritten stroke on the display unit 21.
[0138] As a result, the tablet terminal 1 according to the present embodiment can display (draw) beautiful strokes while maintaining the quality of handwriting, even for connected calligraphy that cannot be classified into character elements.
[0139] In addition, in this embodiment, when the continuous writing style determination unit 102 determines that a handwritten stroke is in continuous writing style, the rendering processing unit 105 normalizes and adjusts the maximum thickness of the handwritten stroke using a standard thickness based on a standard writing pressure.
[0140] As a result, the tablet terminal 1 according to the present embodiment can suppress drawing of extremely thin lines for continuous writing, for example, and can display character elements with a natural thickness similar to that of the rendering part. For example, when a part in which the ink thickness is forcibly adjusted by recognizing a character element is mixed with a part in which the thickness is determined based on the writing pressure or the speed of the pen 30, a significant sense of discomfort is generated in the conventional technology. In contrast, the tablet terminal 1 according to the present embodiment can reduce the sense of discomfort in such a case.
[0141] Moreover, the tablet terminal 1 according to this embodiment includes a brush information storage unit 41 that stores character elements in association with brush information including the size, aspect ratio, and angle of the brush used, and a pen pressure characteristics storage unit 42 that stores character elements in association with a pressure curve indicating the relationship between the stroke position and pen pressure in the character element. When the continuous calligraphy determination unit 102 determines that a handwritten stroke is not of continuous calligraphy, the rendering processing unit 105 acquires brush information corresponding to the classified character element from the brush information storage unit 41 and acquires a pressure curve corresponding to the classified character element from the pen pressure characteristics storage unit 42, and adjusts the shape of the partial stroke based on the acquired brush information and pressure curve.
[0142] As a result, the tablet terminal 1 according to this embodiment can change the brush according to the character element using the brush information corresponding to the character element to appropriately adjust the shape of the partial stroke (for example, the shape of the corners at the beginning and end of writing, the tame, the flick, etc.). Also, the pressure curve corresponding to the character element can appropriately adjust the thickness of the partial stroke (for example, the change in thickness in the middle of the partial stroke) according to the stroke position.
[0143] Moreover, the tablet terminal 1 according to this embodiment includes a display unit 21, a touch sensor unit 22 that detects the contact and contact position of an operation medium on a detection surface on the screen of the display unit 21, and an input processing unit 101 that detects the input of a handwritten stroke based on the contact position of the operation medium detected by the touch sensor unit 22. The input processing unit 101 detects the input of a handwritten stroke, and displays a movement trajectory of the contact position of the operation medium as an input handwritten stroke on the display unit 21. The rendering processing unit 105 replaces the input handwritten stroke with a handwritten stroke whose shape has been adjusted, and displays the handwritten stroke on the display unit 21.
[0144] As a result, the tablet terminal 1 according to this embodiment first displays the handwritten strokes that are input, making it possible to input handwritten characters while checking the display, making handwritten input easier, and by replacing the handwritten strokes with handwritten strokes whose shape has been adjusted, it is possible to display neat characters.
[0145] In this embodiment, the classification processing unit 104 classifies the partial strokes divided by the division processing unit 103 into character elements based on a classification model that is generated by executing a learning process on the learning data and that classifies character elements from partial strokes. Note that the learning data is data that associates partial strokes of a character that is previously designated and handwritten by a user with character elements.
[0146] As a result, the tablet terminal 1 according to the present embodiment can classify partial strokes into character elements with high accuracy using the classification model, and can display characters more neatly using the character elements that are accurately classified.
[0147] Moreover, the tablet terminal 1 according to this embodiment includes a learning processing unit 107 and a CE classification information storage unit 43 (model storage unit). The learning processing unit 107 executes a learning process on the above-mentioned learning data to generate a classification model. The CE classification information storage unit 43 stores the classification model generated by the learning processing unit 107. The classification processing unit 104 classifies the partial strokes divided by the division processing unit 103 into character elements based on the classification model stored in the CE classification information storage unit 43. As a result, the tablet terminal 1 according to this embodiment can accurately classify partial strokes into character elements using the classification model.
[0148] Furthermore, the control method according to this embodiment is a control method for the tablet terminal 1, and includes a continuous writing style determination step, a division processing step, a classification processing step, and a rendering processing step. In the continuous writing style determination step, the continuous writing style determination unit 102 determines whether the input handwritten stroke is a continuous writing style in which strokes are continuous. In the division processing step, the division processing unit 103 divides the handwritten stroke at its corners to generate divided partial strokes. In the classification processing step, the classification processing unit 104 classifies the partial strokes divided by the division processing unit 103 into one of a plurality of character elements that are basic patterns constituting kanji characters. In the rendering processing step, if the continuous writing style determination unit 102 determines that the handwritten stroke is not continuous writing style, the rendering processing unit 105 adjusts the shape of the partial stroke based on the brush information and writing pressure characteristics corresponding to the character element classified by the classification processing unit 104, and displays the handwritten stroke with the adjusted shape on the display unit 21; if the continuous writing style determination unit 102 determines that the handwritten stroke is continuous writing style, the rendering processing unit 105 adjusts the shape of the handwritten stroke based on the writing pressure when the handwritten stroke was input, and displays it on the display unit 21.
[0149] As a result, the control method according to this embodiment has the same effect as the tablet terminal 1 described above, and can display beautiful characters (especially kanji) while maintaining the quality of handwriting, even when continuous writing is mixed in.
[0150] The present invention is not limited to the above-described embodiment, and can be modified without departing from the spirit of the present invention. For example, in the above embodiment, the information processing device is a tablet terminal 1, but the present invention is not limited to this. The information processing device may be, for example, a smartphone or a notebook personal computer equipped with a tablet mode.
[0151] In addition, in the above embodiment, an example of the OS used is described using Android (registered trademark), but this is not limited to this and may be applied to other OS such as iOS (registered trademark).
[0152] In addition, in the above embodiment, the processing of the continuous writing style determination unit 102, the division processing unit 103, the classification processing unit 104, and the rendering processing unit 105 may be realized by an application program, or part of it may be realized by a device driver, etc.
[0153] Furthermore, when the information processing device is a notebook personal computer, part of the processing of the continuous writing style determination unit 102, the division processing unit 103, the classification processing unit 104, and the rendering processing unit 105 may be performed by an embedded controller.
[0154] In the above embodiment, an example has been described in which the tablet terminal 1 includes the learning processing unit 107, but the present invention is not limited to this, and for example, the tablet terminal 1 may not include the learning processing unit 107. In this case, for example, an external device such as a server device connectable to the tablet terminal 1 may include the learning processing unit 107.
[0155] In addition, in the above embodiment, an example has been described in which the continuous writing style determination unit 102 determines continuous writing style based on all of the above-mentioned determination conditions (a) to (d), but this is not limited to this, and the continuous writing style may be determined by executing at least one of the determination conditions (a) to (d).
[0156] In addition, in the above embodiment, an example has been described in which the rendering processing unit 105 executes a pen pressure characteristic correction process using equation (1) and a pen pressure characteristic normalization process using equation (2). However, this is not limited to this, and only one of the pen pressure characteristic correction process and the pen pressure characteristic normalization process may be executed.
[0157] Each of the components of the tablet terminal 1 described above has an internal computer system. A program for implementing the functions of each of the components of the tablet terminal 1 described above may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed to perform processing in each of the components of the tablet terminal 1 described above. Here, "reading a program recorded on a recording medium into a computer system and executing it" includes installing the program into a computer system. The "computer system" referred to here includes hardware such as an OS and peripheral devices.
[0158] Furthermore, a "computer system" may include multiple computer devices connected via a network including the Internet, WAN, LAN, dedicated lines, and other communication lines. Furthermore, a "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into a computer system. In this way, the recording medium storing the program may be a non-transitory recording medium such as a CD-ROM.
[0159] The recording medium also includes a recording medium installed inside or outside and accessible from a distribution server to distribute the program. The program may be divided into a plurality of parts, downloaded at different times, and then combined with each component of the tablet terminal 1, or each divided program may be distributed by a different distribution server. Furthermore, the "computer-readable recording medium" includes a recording medium that holds a program for a certain period of time, such as a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network. The above program may be a recording medium for realizing part of the above-mentioned functions. Furthermore, the recording medium may be a so-called difference file (difference program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.
[0160] In addition, some or all of the above-mentioned functions may be realized as an integrated circuit such as LSI (Large Scale Integration). Each of the above-mentioned functions may be individually processed, or some or all of the functions may be integrated into a processor. The integrated circuit method is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. In addition, if an integrated circuit technology that can replace LSI appears due to advances in semiconductor technology, an integrated circuit based on that technology may be used. [Explanation of symbols]
[0161] 1. Tablet device 10 Control section 11 Processors 12. Main memory 13. Flash Memory 20. Touchscreen 21 Display section 22 Touch sensor section 23 Peripheral Devices 24 Audio System 25. Mike 26 Speaker 27 Baseband chip 28 Radio Section 29, 31 Wireless Communication Department 30 pens 32 Sensor section 33 Memory 34 MCU 40 Storage section 41 Brush information storage unit 42 Pen pressure characteristics memory section 43 CE classification information storage unit 44 Handwritten stroke memory 45 Partial stroke memory 46 Learning data storage unit 100 Information Processing Systems 101 Input processing unit 102 Continuous writing judgment section 103 Division Processing Unit 104 Classification Processing Unit 105 Rendering Processing Unit 106 Display processing unit 107 Learning processing unit
Claims
1. a continuous writing style determination unit that determines whether or not the input handwritten strokes are continuous writing styles, which are writing styles in which strokes are continuously formed; a division processing unit that divides the handwritten stroke at corners and generates divided partial strokes; a classification processing unit that classifies the partial strokes divided by the division processing unit into any one of a plurality of character elements that are basic patterns constituting a Chinese character; a rendering processing unit that adjusts a shape of the partial stroke based on brush information and writing pressure characteristics corresponding to the character element classified by the classification processing unit when the continuous writing style determination unit determines that the handwritten stroke is not of the continuous writing style, and displays the handwritten stroke with the adjusted shape on a display unit; and a rendering processing unit that adjusts a shape of the handwritten stroke based on writing pressure when the handwritten stroke was input when the continuous writing style determination unit determines that the handwritten stroke is of the continuous writing style, and displays the handwritten stroke on the display unit. An information processing device comprising:
2. The continuous writing style determination unit determines that the handwritten stroke is in the continuous writing style when the length of the handwritten stroke is equal to or greater than a threshold length. The information processing device according to claim 1 .
3. The continuous writing style determination unit changes the threshold length in accordance with a thickness of a selected brush that draws the handwritten stroke. The information processing device according to claim 2 .
4. The continuous writing style determination unit determines that the handwritten stroke is in the continuous writing style when the number of the corners included in the handwritten stroke is equal to or greater than a threshold number. The information processing device according to claim 1 .
5. The continuous writing style determination unit determines that the handwritten stroke is in the continuous writing style when a sum of angle changes of drawing lines in the handwritten stroke is equal to or greater than a threshold angle. The information processing device according to claim 1 .
6. The continuous writing style determination unit determines that the handwritten stroke is in the continuous writing style when the handwritten stroke has a self-intersection. The information processing device according to claim 1 .
7. When the continuous writing style determination unit determines that the handwritten stroke is the continuous writing style, the rendering processing unit adjusts the thickness of the handwritten stroke so that the higher the writing pressure, the thicker the handwritten stroke is, and the faster the drawing speed of the handwritten stroke, the thinner the handwritten stroke is, and displays the handwritten stroke on the display unit. The information processing device according to claim 1 .
8. When the continuous writing style determination unit determines that the handwritten stroke is the continuous writing style, the rendering processing unit normalizes and adjusts a maximum thickness of the handwritten stroke with a standard thickness based on a standard writing pressure. The information processing device according to claim 7.
9. a brush information storage unit that stores the character elements in association with the brush information including a size, an aspect ratio, and an angle of a brush to be used; a writing pressure characteristic storage unit that stores the character elements in association with a pressure curve that indicates a relationship between a stroke position in the character element and writing pressure; Equipped with When the continuous writing style determination unit determines that the handwritten stroke is not in the continuous writing style, the rendering processing unit obtains the brush information corresponding to the classified character element from the brush information storage unit, obtains the pressure curve corresponding to the classified character element from the writing pressure characteristic storage unit, and adjusts a shape of the partial stroke based on the obtained brush information and pressure curve. The information processing device according to claim 8.
10. The display unit; a touch sensor unit that detects a contact and a contact position of an operation medium on a detection surface on a screen of the display unit; an input processing unit that detects an input of the handwritten stroke based on a contact position of the operation medium detected by the touch sensor unit; Equipped with the input processing unit detects the input of the handwritten stroke, and displays a movement trajectory of a contact position of the operation medium on the display unit as the input handwritten stroke; The rendering processing unit replaces the input handwritten stroke with the handwritten stroke whose shape has been adjusted, and displays the handwritten stroke on a display unit. The information processing device according to claim 9.
11. A method for controlling an information processing device, comprising: A step of a continuous writing style determination unit determining whether or not the input handwritten strokes are continuous writing styles, which are writing styles in which strokes are continuous; a segmentation processing unit segmenting the handwritten stroke at a corner portion to generate segmented partial strokes; a classification processing unit classifying the partial stroke divided by the division processing unit into any one of a plurality of character elements which are basic patterns constituting a Chinese character; a rendering processing unit, when the continuous writing style determination unit has determined that the handwritten stroke is not of the continuous writing style, adjusting a shape of the partial stroke based on brush information and writing pressure characteristics corresponding to the character element classified by the classification processing unit, and displaying the handwritten stroke with the adjusted shape on a display unit; and, when the continuous writing style determination unit has determined that the handwritten stroke is of the continuous writing style, adjusting a shape of the handwritten stroke based on the writing pressure at the time of inputting the handwritten stroke, and displaying the handwritten stroke on the display unit; A control method comprising:
Citation Information
Patent Citations
Information processing device, information processing system, information processing method and information processing program
JP2021043851A
