Information processing apparatus and control method

The information processing device uses machine learning to classify and adjust handwritten strokes into composite and simple character elements, addressing the challenge of displaying clear characters while preserving handwriting quality, particularly for kanji, by accurately rendering strokes based on pen pressure and brush information.

JP2024159230AActive Publication Date: 2024-11-08LENOVO IRELAND INTERNATIONAL LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023075081
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-04-28
Publication Date
2024-11-08
Estimated Expiration
2043-04-28

AI Technical Summary

Technical Problem

Conventional information processing devices struggle to display clear characters while maintaining the quality and freedom of handwriting, particularly for complex characters like kanji, due to difficulties in adjusting ink thickness and preserving the aspect ratio and size of handwritten strokes on hard and slippery surfaces.

Method used

An information processing device employs machine learning to classify and adjust handwritten strokes into composite and simple character elements, using classification models to identify and divide strokes at corners, and adjust shape and thickness based on pen pressure and brush information, allowing for accurate rendering on a display.

Benefits of technology

The device effectively displays beautiful characters, especially kanji, while maintaining the quality of handwriting by accurately classifying and adjusting stroke shapes and thickness, preserving the freedom of handwriting input.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024159230000001_ABST
    Figure 2024159230000001_ABST
Patent Text Reader

Abstract

To display beautiful characters while maintaining the taste of handwriting in handwriting input.SOLUTION: An information processing apparatus comprises: a first classification processing unit that uses a first machine learning result to classify a composite character element corresponding to an input handwriting stroke from a predetermined number of composite character elements selected in advance; a division processing unit that creates partial strokes obtained by dividing the handwriting stroke at a corner portion; a second classification processing unit that uses a second machine learning result to classify a character element equivalent to each of the partial strokes from character elements corresponding to the plurality of partial strokes; a classification control unit that causes the first classification processing unit to execute first classification processing and the second classification processing unit to execute second classification processing in a switchable manner according to whether the handwriting stroke includes the corner portion; and a rendering processing unit that adjusts the shape of the handwriting stroke on the basis of brush information and handwriting pressure characteristics corresponding to the composite character elements or the character element and displays the handwriting stroke on a display.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an information processing device and a control method. [Background technology]

[0002] In recent years, information processing devices such as tablet terminals that allow handwritten character input have become widespread (see, for example, Patent Document 1). In such conventional information processing devices, a technique is known that recognizes handwritten input characters and converts them into a specified font. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-43851 Summary of the Invention [Problem to be solved by the invention]

[0004] In the above-mentioned conventional information processing device, it is possible to display beautiful characters by converting handwritten input characters into a predetermined font. However, in the conventional information processing device, by replacing characters with a predetermined font, the user's handwriting is lost, and the freedom of handwriting, such as the aspect ratio of the characters and the size of the characters, is lost, and the goodness of handwriting is lost. On the other hand, in the conventional information processing device, it is also possible to adjust the rendering thickness of the drawing ink based on the pen pressure and display characters drawn with a pen. However, in such cases, drawings that require delicate adjustment of the ink thickness, such as the tome, hane, and harai, which are often necessary especially for complex characters such as kanji, are very difficult on tablet terminals, which have a hard and slippery surface compared to paper. In other words, in the conventional information processing device, it was difficult to display beautiful characters while maintaining the goodness of handwriting.

[0005] The present invention has been made to solve the above problems, and its object is to provide an information processing device and control method that can display beautiful characters, particularly kanji, while maintaining the quality of handwriting when inputting handwritten characters. [Means for solving the problem]

[0006] In order to solve the above problem, one aspect of the present invention is an information processing device including: a first classification processing unit that uses a first machine learning result to identify and classify a composite character element that corresponds to an input handwritten stroke from a preselected predetermined number of composite character elements, the composite character element being a composite character element formed by combining character elements that are basic patterns that constitute a Chinese character character; a segmentation processing unit that divides the handwritten stroke at corners and generates divided partial strokes; a second classification processing unit that uses a second machine learning result different from the first machine learning result to identify and classify the character element that corresponds to the partial stroke from a plurality of the character elements that is more than the predetermined number; a classification control unit that switches between the first classification process by the first classification processing unit and the second classification process by the second classification processing unit depending on whether the handwritten stroke includes the corners; and a rendering processing unit that adjusts a shape of the handwritten stroke based on brush information and pen pressure characteristics corresponding to the classified composite character element or the character element, and displays the handwritten stroke with the shape adjusted on a display unit.

[0007] In addition, in one aspect of the present invention, in the above-mentioned information processing device, the classification control unit may cause the first classification processing unit to execute the first classification process using the first machine learning result when the handwritten stroke includes the corner portion, and may cause the second classification processing unit to execute the second classification process using the second machine learning result when the handwritten stroke does not include the corner portion.

[0008] In another aspect of the present invention, in the above-mentioned information processing device, the classification control unit may, when a confidence level of the first classification process is less than a threshold as a result of causing the first classification processing unit to execute the first classification process using the first machine learning result, cause the second classification processing unit to execute the second classification process using the second machine learning result, and when the rendering processing unit has executed the second classification process, adjusts a shape of the handwritten stroke based on the brush information and the pen pressure characteristics corresponding to the character element classified by the second classification process, and displays the handwritten stroke with the adjusted shape on the display unit.

[0009] In addition, in one aspect of the present invention, in the above-mentioned information processing device, when a result of executing the first classification process shows that the number of corner parts contained in the handwritten stroke does not match the number of corner parts contained in the classified compound character element, the classification control unit may cause the second classification processing unit to execute the second classification process using the second machine learning result.

[0010] In addition, one aspect of the present invention, in the above-mentioned information processing device, the first machine learning result may be a classification model generated by executing a learning process on learning data in which a predetermined number of the compound character elements selected in advance are associated with the corresponding handwritten strokes, and the second machine learning result may be a classification model generated by executing a learning process on learning data in which preselected character elements for the partial strokes are associated with the corresponding partial strokes.

[0011] In addition, one aspect of the present invention, in the above-mentioned information processing device, includes a pen pressure characteristics storage unit that stores the character element or the composite character element in correspondence with a pressure curve that indicates the relationship between stroke position and pen pressure in the character element or the composite character element, and the rendering processing unit may obtain from the pen pressure characteristics storage unit the pressure curve corresponding to the classified character element or the composite character element, and adjust a thickness of the handwritten stroke based on the obtained pressure curve.

[0012] In addition, one aspect of the present invention is a method for processing a character string using a brush information storage unit that stores brush information including a size, aspect ratio, and angle of a brush to be used in association with the character element or the composite character element, and the rendering processing unit may obtain the brush information corresponding to the classified character element or the composite character element from the brush information storage unit, and adjust a shape of the handwritten stroke based on the obtained brush information.

[0013] In addition, one aspect of the present invention is the above-mentioned information processing device, the display unit, a touch sensor unit that detects the contact and the contact position of an operation medium on a detection surface on a screen of the display unit, and an input processing unit that detects the input of the handwritten stroke based on the contact position of the operation medium detected by the touch sensor unit, wherein the input processing unit detects the input of the handwritten stroke and displays a movement trajectory of the contact position of the operation medium on the display unit as the input handwritten stroke, and the rendering processing unit may replace the input handwritten stroke with the handwritten stroke whose shape has been adjusted and display it on the display unit.

[0014] Moreover, one aspect of the present invention is a control method for an information processing device including: a first classification processing unit that uses a first machine learning result to identify and classify a composite character element that corresponds to an input handwritten stroke from a predetermined number of composite character elements, the composite character element being a compound character element formed by combining character elements that are basic patterns that constitute a Chinese character character; a segmentation processing unit that divides the handwritten stroke at corner portions and generates divided partial strokes; and a second classification processing unit that uses a second machine learning result different from the first machine learning result to identify and classify the character element that corresponds to the partial stroke from a plurality of the character elements that is more than the predetermined number, the control method including: a step of the classification control unit switching between executing the first classification process by the first classification processing unit and the second classification process by the second classification processing unit depending on whether the handwritten stroke includes the corner portion; and a step of the rendering processing unit adjusting a shape of the handwritten stroke based on brush information and pen pressure characteristics corresponding to the composite character element or the character element classified by the first classification process or the second classification process, and displaying the handwritten stroke with the shape adjusted on a display unit. Effect of the Invention

[0015] According to the above-described aspects of the present invention, in handwritten input, beautiful characters (especially kanji) can be displayed while maintaining the quality of the handwritten text. [Brief description of the drawings]

[0016] [Figure 1] 1 is an external view showing an example of a tablet terminal and an information processing system according to an embodiment of the present invention. [Diagram 2] 1 is a diagram illustrating an example of a main hardware configuration of a tablet terminal and an information processing system according to an embodiment of the present invention. [Diagram 3] 1 is a block diagram showing an example of a functional configuration of a tablet terminal and an information processing system according to an embodiment of the present invention. [Figure 4] 5A and 5B are diagrams illustrating an example of data in a brush information storage unit according to the embodiment. [Diagram 5]3A and 3B are diagrams illustrating examples of character elements according to the present embodiment. [Figure 6] 5A and 5B are diagrams illustrating an example of data in a writing pressure characteristics storage unit according to the embodiment. [Figure 7] FIG. 2 is a diagram showing an overview of handwriting input processing of the tablet terminal according to the embodiment. [Figure 8] 10 is a flowchart illustrating an example of a handwriting input process of the tablet terminal according to the embodiment. [Figure 9] 11 is a first flowchart showing an example of a learning process of the tablet terminal in this embodiment. [Figure 10] 13 is a second flowchart showing an example of a learning process of the tablet terminal in this embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An information processing apparatus and a control method according to an embodiment of the present invention will be described below with reference to the drawings.

[0018] FIG. 1 is an external view showing an example of a tablet terminal 1 and an information processing system 100 according to the present embodiment. 1, the information processing system 100 includes a tablet terminal 1 and a pen 30. In this embodiment, the tablet terminal 1 will be described as an example of an information processing device.

[0019] The tablet terminal 1 has a touch screen 20 installed on one main surface of a housing CS1, and an application program such as a memo pad or drawing program is executed using a pen 30. The tablet terminal 1 is capable of handwritten input using the pen 30. Note that the pen 30 can be set to various types of brushes when inputting handwritten text.

[0020] The touch screen 20 includes a display unit 21 and a touch sensor unit 22, and the display unit 21 displays various types of information on a display screen DF. The touch sensor unit 22 is disposed over the display unit 21, and detects when the pen 30 touches the display screen DF of the display unit 21, and detects the touch position of the pen 30. The touch screen 20, the display unit 21, and the touch sensor unit 22 will be described in detail later.

[0021] Next, the main hardware configuration of the tablet terminal 1 and the information processing system 100 will be described with reference to FIG. FIG. 2 is a diagram showing an example of the main hardware configuration of the tablet terminal 1 and the information processing system 100 according to this embodiment.

[0022] 2, the information processing system 100 includes a tablet terminal 1 and a pen 30. The tablet terminal 1 also includes a processor 11, a main memory 12, a flash memory 13, a touch screen 20, a peripheral device 23, an audio system 24, a microphone 25, a speaker 26, a baseband chip 27, a wireless unit 28, and a wireless communication unit 29.

[0023] The processor 11 is, for example, an application processor including a CPU (Central Processing Unit). The processor 11 controls the entire tablet terminal 1.

[0024] The main memory 12 is a writable memory used as a read area for the execution program of the processor 11 or as a work area for writing processing data for the execution program. The main memory 12 is composed of, for example, a plurality of DRAM (Dynamic Random Access Memory) chips. The execution program includes an OS (Operating System), various device drivers for operating the hardware of peripheral devices, various services / utilities, application programs (application software), and the like.

[0025] The flash memory 13 is, for example, a flash EEPROM (Electrically Erasable Programmable Read Only Memory), and stores the OS, various drivers, various services / utilities, application programs (hereinafter sometimes referred to as applications), and various data.

[0026] The display unit 21 is, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display, and displays a display screen based on drawing data (display data) output from the processor 11. The display unit 21 is capable of displaying, for example, the movement trajectory (drawing trajectory) of the pen 30.

[0027] The touch sensor unit 22 detects the position of an operation medium such as the pen 30 on the screen of the display unit 21 (on the display screen DF) and the contact of the pen 30 with the screen. The touch sensor unit 22 detects the contact position of the pen 30 on the screen of the display unit 21.

[0028] The peripheral device 23 is, for example, a Wireless Local Area Network (WLAN) module, a Global Positioning System (GPS) module, and sensors such as an acceleration sensor.

[0029] The audio system 24 is, for example, an audio IC (Integrated Circuit), and inputs, records, plays back, and outputs sound data. For example, a microphone 25 and a speaker 26 are connected to the audio system 24. For example, the audio system 24 outputs sound data collected by the microphone 25 to the processor 11 or the baseband chip 27. In addition, the audio system 24 converts sound data acquired from the processor 11 or the baseband chip 27 into a sound signal, and outputs the sound signal to the speaker 26.

[0030] The microphone 25 picks up sounds around the tablet terminal 1. For example, the microphone 25 picks up sounds such as the user's voice when audio integration with another terminal is performed. The speaker 26 outputs various sounds to the outside of the tablet terminal 1. For example, the speaker 26 outputs (emits) sounds received from other terminals when performing audio integration with other terminals.

[0031] The baseband chip 27 is a dedicated IC that controls wireless communication such as 4G (fourth generation mobile communication system) and 5G (fourth generation mobile communication system). The baseband chip 27 outputs audio data received using the wireless unit 28 to the speaker 26 via the audio system 24. The baseband chip 27 also acquires audio data collected from the microphone 25 via the audio system 24, and outputs the data from the mobile communication system using the wireless unit 28. The baseband chip 27 also communicates input / output data for data communication by the mobile communication system with the processor 11.

[0032] The radio unit 28 is a wireless communication device including an antenna for performing wireless communication in a mobile communication system. The wireless communication unit 29 is, for example, a Bluetooth (registered trademark) module, and performs wireless communication with the pen 30. The wireless communication unit 29 receives, for example, sensor information detected by the pen 30 (contact pressure, angle of the pen 30, etc.) from the pen 30.

[0033] The pen 30 is a pen-shaped operating medium, such as a touch pen, a stylus pen, etc. The pen 30 is capable of communicating with the tablet terminal 1 by wireless communication such as Bluetooth (registered trademark). The pen 30 includes a wireless communication unit 31, a sensor unit 32, a memory 33, and an MCU 34.

[0034] The wireless communication unit 31 is, for example, a Bluetooth (registered trademark) module, and performs wireless communication between the tablet terminal 1 and the pen 30. The wireless communication unit 31 transmits, for example, sensor information detected by the sensor unit 32 to the tablet terminal 1. The sensor unit 32 detects sensor information such as the contact pressure (writing pressure) of the pen 30, the angle of the pen 30, and the like.

[0035] The memory 33 is composed of, for example, a RAM, and stores part of the control program for the pen 30 and processing data for the MCU 34. Note that the memory 33 may include a flash memory, a ROM, and the like in addition to the RAM.

[0036] The MCU (Micro Controller Unit) 34 includes a CPU, memories such as ROM and RAM, and I / O related components, and generally controls the pen 30. The MCU 34 controls data communication with the tablet terminal 1 via the wireless communication unit 31. The MCU 34 transmits, for example, sensor information detected by the sensor unit 32 to the tablet terminal 1 via the wireless communication unit 31.

[0037] Next, the functional configuration of the information processing system 100 according to the present embodiment will be described with reference to FIG. FIG. 3 is a block diagram showing an example of the functional configuration of the tablet terminal 1 and the information processing system 100 according to this embodiment.

[0038] As shown in FIG. 3, the information processing system 100 includes a tablet terminal 1 and a pen 30. The tablet terminal 1 includes a control unit 10, a touch screen 20, and a storage unit 40.

[0039] The touch screen 20 includes a display unit 21 and a touch sensor unit 22 . The touch sensor unit 22 detects the contact and contact position of an operating medium (pen 30) on a detection surface on the screen of the display unit 21. The touch sensor unit 22 detects, for example, the contact position of the pen 30 on the screen of the display unit 21 as a pen input. The touch sensor unit 22 detects, for example, contact position information indicating two-dimensional coordinates on the display screen DF of the display unit 21 as the contact position of the pen 30. The touch sensor unit 22 outputs the contact position information to the control unit 10.

[0040] The memory unit 40 is a memory unit realized, for example, by the main memory 12 or the flash memory 13, and includes a brush information memory unit 41, a pen pressure characteristic memory unit 42, a CCE classification information memory unit 43, an SCE classification information memory unit 44, a handwritten stroke memory unit 45, a partial stroke memory unit 46, and a learning data memory unit 47.

[0041] The brush information storage unit 41 is, for example, a storage unit realized by the main memory 12, and stores brush information corresponding to character elements (CE), which are basic parts (basic patterns constituting a kanji character) obtained by dividing a kanji character. An example of data in the brush information storage unit 41 will now be described with reference to Fig. 4. Note that the character elements include simple character elements (SCE), which are simple character elements without corners and composed of one character element (CE), and complex character elements (CCE), which are complex shapes including corners and composed of multiple character elements (CE).

[0042] FIG. 4 is a diagram showing an example of data stored in the brush information storage unit 41 in this embodiment. As shown in FIG. 4, the brush information storage unit 41 stores a CEID (character element ID), a brush size, an aspect ratio, a standard brush angle, and a standard stroke angle in association with each other.

[0043] CEID is character element identification information that identifies a character element. Brush size indicates the size of the brush. Aspect ratio indicates the aspect ratio of the brush, that is, the ratio of width to height. Standard brush angle indicates the standard (reference) brush angle of the character element, and standard stroke angle indicates the standard (reference) angle of the stroke of the character element.

[0044] For example, the example shown in Figure 4 indicates that the brush size of the character element with CEID "SCE001" is "R1" and the aspect ratio is "XX1:YY1". It also indicates that the standard brush angle of the character element is "α1" and the standard stroke angle is "β1". Note that "SCE001" is a simple character element.

[0045] For example, the brush size of the character element with CEID "CCE001" is "R2" and the aspect ratio is "XX2:YY2". The standard brush angle of the character element is "α2" and the standard stroke angle is "β2". "CCE001" is a composite character element.

[0046] Now, with reference to FIG. 5, the character elements will be described. Fig. 5 is a diagram showing an example of character elements in this embodiment, in which character elements constituting the Chinese character "eternal" will be described.

[0047] As shown in Fig. 5, the Chinese character "eternity" is decomposed into character elements CE1 to CE8. Character elements CE1 to CE8 include strokes such as tome, hane, and harai, and corresponding brush information is defined for each of character elements CE1 to CE8 as shown in Fig. 4 above. Note that character elements CE1 to CE8 are simple character elements. Simple character elements have simple patterns with relatively few variations, so it is easy to generate a classification model by machine learning, but the recognition rate (hit rate) tends to be low.

[0048] In addition, in the Chinese character "yong", composite character elements formed by combining a plurality of simple character elements are composite character element CCE1, composite character element CCE2, etc. as shown in FIG. 5. Since composite character elements have many variations in complex patterns, it is difficult to generate a classification model for all variations by machine learning, but the recognition rate (hit rate) tends to be high.

[0049] In this embodiment, a predetermined number of composite character elements selected in advance, such as composite character element CCE1 and composite character element CCE2, are defined, and for each defined character element, by rendering (drawing), in the handwritten input of Chinese characters, beautiful character rendering is realized. In addition, in this embodiment, for parts that cannot be handled by composite character elements, a plurality of simple character elements including the above-described character elements CE1 to CE8 determined from various Chinese characters are defined, and for each defined simple character element, by rendering (drawing), beautiful characters are realized in the handwritten input of Chinese characters. Here, the "beautiful characters" refer to characters rendered to resemble a preferred font, for example.

[0050] Returning to the description of FIG. 3, the pen pressure characteristic storage unit 42 is a storage unit realized by, for example, the main memory 12, and stores a pressure curve (pen pressure characteristic) indicating the relationship between the stroke position and the pen pressure in a character element (simple character element or composite character element). The pen pressure characteristic storage unit 42 stores, for example, by associating the identification information of the character element with the pen pressure characteristic. Here, with reference to FIG. 6, a data example of the pen pressure characteristic storage unit 42 will be described.

[0051] FIG. 6 is a diagram showing a data example of the pen pressure characteristic storage unit 42 in this embodiment. As shown in FIG. 6, the pen pressure characteristic storage unit 42 stores by associating CEID with the pen pressure characteristic.

[0052] For example, in the example shown in Fig. 6, the pen pressure characteristic corresponding to CEID "SCE001" is shown to be a pressure curve like waveform W1. Note that the horizontal axis of the graph showing waveform W1 indicates stroke position, and the vertical axis indicates pen pressure (pressure). The pressure curve indicates that the higher the pen pressure, the thicker the stroke line will be when rendered, and the lower the pen pressure, the thinner the stroke line will be when rendered.

[0053] The pen pressure characteristic (waveform W1) corresponding to CEID "SCE001" indicates a character element (character element CE1 shown in FIG. 5) in which the stroke line becomes thicker as the stroke position advances and then suddenly thins at the end.

[0054] For example, the pen pressure characteristic corresponding to CEID "SCE002" indicates a pressure curve like waveform W2. The pen pressure characteristic (waveform W2) corresponding to CEID "SCE002" indicates a character element (character element CE2 shown in FIG. 5) in which the stroke line becomes thin once as the stroke position advances and then thickens again.

[0055] Returning to the description of Fig. 3 again, the CCE classification information storage unit 43 is, for example, a storage unit realized by the main memory 12, and stores information for classifying compound character elements. The CCE classification information storage unit 43 is, for example, a learned model generated by machine learning, and stores a CCE classification model for classifying handwritten strokes, which will be described later, into any one of a predetermined number of compound character elements selected in advance. The CCE classification information storage unit 43 is an example of a model storage unit.

[0056] The SCE classification information storage unit 44 is, for example, a storage unit realized by the main memory 12, and stores information for classifying simple character elements. The SCE classification information storage unit 44 is, for example, a learned model generated by machine learning, and stores an SCE classification model for classifying a partial stroke, which will be described later, into one of a plurality of simple character elements that is greater than a predetermined number of compound character elements. The SCE classification information storage unit 44 is an example of a model storage unit.

[0057] The handwritten stroke storage unit 45 is, for example, a storage unit realized by the main memory 12, and stores handwritten stroke data detected by the touch sensor unit 22 using the pen 30. The handwritten stroke storage unit 45 stores, as handwritten strokes, collection data of time-series detection points detected by the touch sensor unit 22 when handwriting inputting a kanji character, for example.

[0058] The partial stroke storage unit 46 is a storage unit realized by, for example, the main memory 12, and stores data of partial strokes generated from handwritten strokes. The partial strokes will be described in detail later.

[0059] The learning data storage unit 47 is, for example, a storage unit realized by the main memory 12, and stores learning data for generating the above-mentioned classification models (CCE classification model and SCE classification model). When generating an SCE classification model, the learning data storage unit 47 stores learning data in which partial strokes of a pre-specified character handwritten by a user correspond to simple character elements. When generating a CCE classification model, the learning data storage unit 47 stores learning data in which handwritten strokes of a pre-specified character handwritten by a user correspond to complex character elements.

[0060] Control unit 10 is a functional unit realized, for example, by processor 11 executing a program stored in main memory 12 or flash memory 13, and executes various processes based on an OS (for example, Android (registered trademark)). Control unit 10 includes an input processing unit 101, a CCE classification processing unit 102, a division processing unit 103, an SCE classification processing unit 104, a classification control unit 105, a rendering processing unit 106, a display processing unit 107, and a learning processing unit 108.

[0061] The input processing unit 101 is a functional unit that is realized by the processor 11 executing a program stored in the main memory 12 or the flash memory 13. The input processing unit 101 is, for example, a device driver that controls an input by the touch sensor unit 22, and detects the position and contact of the pen 30 on the screen of the display unit 21 (on the display screen DF) by the input from the touch sensor unit 22. The input processing unit 101 may acquire, for example, contact pressure (writing pressure) and angle information of the pen 30 from the pen 30.

[0062] The input processing unit 101 detects the input of a handwritten stroke based on the contact position of an operation medium (for example, the pen 30) detected by the touch sensor unit 22. That is, the input processing unit 101 stores the detected contact position information (position coordinates) of the pen 30 in the handwritten stroke storage unit 45 as a handwritten stroke.

[0063] In addition, a handwritten stroke, or simply a stroke, refers to a collection of data of detection points of the trajectory of the pen tip from the time the pen 30 makes contact (touch) to the time the pen 30 releases contact (up), and may include a display of the trajectory of the pen tip on the screen of the display unit 21 (on the display screen DF).

[0064] Furthermore, the input processing unit 101 detects the input of handwritten strokes, and displays a movement trajectory of the contact position of the operation medium as the input handwritten strokes on the display unit 21. The input processing unit causes the display unit 21 to display the input handwritten strokes via the display processing unit 107, which will be described later.

[0065] The CCE classification processing unit 102 (an example of a first classification processing unit) is a functional unit realized by the processor 11 executing a program stored in the main memory 12 or the flash memory 13. The CCE classification processing unit 102 uses a CCE classification model (first machine learning result) to identify and classify a compound character element that corresponds to an input handwritten stroke from among a predetermined number of compound character elements selected in advance, which are compound character elements that are basic patterns that constitute Chinese characters.

[0066] Here, the CCE classification model is a classification model that is generated by executing a learning process on learning data in which a predetermined number of preselected compound character elements are associated with corresponding handwritten strokes. The CCE classification processing unit 102 uses the CCE classification model stored in the CCE classification information storage unit 43 to classify handwritten strokes into compound character elements.

[0067] Specifically, as shown in step S11 of FIG. 7, the CCE classification processing unit 102 classifies, for example, a handwritten stroke ST1 into a compound character element CCE1 using the CCE classification model MD1.

[0068] The division processing unit 103 is a functional unit that is realized when the processor 11 executes a program stored in the main memory 12 or the flash memory 13. The division processing unit 103 divides handwritten strokes input by a user at corners to generate divided partial strokes. The division processing unit 103 divides handwritten strokes stored in the handwritten stroke storage unit 45 by the input processing unit 101 into partial strokes.

[0069] Specifically, as shown in step S12 of FIG. 7, the division processing unit 103 divides a handwritten stroke ST1 having two corners at the corners to generate partial strokes SS1, SS2, and SS3. The division processing unit 103 stores the divided partial strokes in the partial stroke storage unit 46.

[0070] The SCE classification processing unit 104 (an example of a second classification processing unit) is a functional unit that is realized when the processor 11 executes a program stored in the main memory 12 or the flash memory 13. The SCE classification processing unit 104 classifies the partial strokes divided by the division processing unit 103 into one of a plurality of simple character elements, which are basic patterns that constitute a kanji character.

[0071] That is, the SCE classification processing unit 104 uses an SCE classification model (second machine learning result) different from the CCE classification model to identify and classify simple character elements corresponding to partial strokes from among a plurality of character elements (simple character elements) that are greater than a predetermined number of compound character elements.

[0072] Here, the SCE classification model is a classification model that is generated by executing a learning process on learning data in which pre-selected character elements for partial strokes are associated with the corresponding partial strokes. The SCE classification processing unit 104 uses the SCE classification model stored in the SCE classification information storage unit 44 to classify partial strokes into simple character elements.

[0073] 7, the SCE classification processor 104 classifies, for example, the partial stroke SS1 into a character element CE2 and the partial stroke SS2 into a character element CE3 using the SCE classification model MD2. Also, the SCE classification processor 104 classifies, for example, the partial stroke SS3 into a character element CE4 using the SCE classification model MD2.

[0074] The classification control unit 105 is a functional unit that is realized by the processor 11 executing a program stored in the main memory 12 or the flash memory 13. The classification control unit 105 switches between the CCE classification processing (first classification processing) by the CCE classification processing unit 102 and the SCE classification processing (second classification processing) by the SCE classification processing unit 104, depending on whether or not the handwritten stroke includes a corner portion.

[0075] For example, when a handwritten stroke includes a corner portion, the classification control unit 105 causes the CCE classification processing unit 102 to execute a CCE classification process using a CCE classification model. Also, for example, when a handwritten stroke does not include a corner portion, the classification control unit 105 causes the SCE classification processing unit 104 to execute an SCE classification process using an SCE classification model.

[0076] In addition, for example, when the classification control unit 105 causes the CCE classification processing unit 102 to execute CCE classification processing using a CCE classification model and the confidence level of the CCE classification processing is less than a threshold, the classification control unit 105 causes the SCE classification processing unit 104 to execute SCE classification processing using the SCE classification model.

[0077] Furthermore, when the result of executing the CCE classification process shows that the number of corner parts contained in the handwritten stroke does not match the number of corner parts contained in the classified compound character element, the classification control unit 105 causes the SCE classification processing unit 104 to execute an SCE classification process using an SCE classification model.

[0078] Furthermore, when the confidence level of the CCE classification process is equal to or higher than a threshold value and the number of corner parts contained in the handwritten stroke matches the number of corner parts contained in the classified composite character element, the classification control unit 105 causes the rendering processing unit 106 to adjust the handwritten stroke using the classified composite character element and causes the handwritten stroke to be displayed on the display unit 21.

[0079] In addition, if the result of executing the SCE classification process is that the confidence level of the SCE classification process is less than a threshold, the classification control unit 105 causes the rendering processing unit 106 to display the handwritten strokes on the display unit 21 using a general method that does not use character elements.

[0080] In addition, when the confidence level of the SCE classification process is equal to or greater than a threshold, the classification control unit 105 causes the rendering processing unit 106 to adjust the handwritten strokes using the classified simple character elements and displays the handwritten strokes on the display unit 21.

[0081] The rendering processing unit 106 is a functional unit that is realized when the processor 11 executes a program stored in the main memory 12 or the flash memory 13. The rendering processing unit 106 adjusts the shape of a partial stroke based on brush information and writing pressure characteristics corresponding to a character element (complex character element or simple character element) classified by the CCE classification processing unit 102 or the SCE classification processing unit 104, and displays the handwritten stroke on the display unit 21.

[0082] The rendering processing unit 106, for example, acquires a pressure curve corresponding to a classified character element (simple character element) or a compound character element from the pen pressure characteristic storage unit 42, and adjusts the thickness of the handwritten stroke based on the acquired pressure curve. In addition, the rendering processing unit 106 acquires brush information corresponding to the classified character elements (simple character elements) or compound character elements from the brush information storage unit 41, for example, and adjusts the shape of the handwritten stroke based on the acquired brush information.

[0083] In addition, when the rendering processing unit 106 executes the SCE classification process, it adjusts the shape of the handwritten stroke based on the brush information and pen pressure characteristics corresponding to the character element (simple character element) classified by the SCE classification process, and displays the handwritten stroke with the adjusted shape on the display unit 21.

[0084] If the simple character element is normally classified by the SCE classification process, the rendering process unit 106 recombines the partial strokes with the brush angle and line thickness adjusted to generate an adjusted handwritten stroke. Furthermore, the rendering processing unit 106 causes the display unit 21 to display the adjusted handwritten strokes via the display processing unit 107, which will be described later.

[0085] 7, when a simple character element is classified correctly, the rendering processing unit 106 adjusts the partial strokes SS1, SS2, and SS3 and recombines them to generate a rendered handwritten stroke CST1. When a compound character element is classified correctly, the rendering processing unit 106 adjusts the handwritten stroke ST1 and generates a rendered handwritten stroke CST1. The rendering processing unit 106 replaces the input handwritten stroke with the handwritten stroke whose shape has been adjusted, and displays the handwritten stroke on the display unit 21.

[0086] The display processing unit 107 is a functional unit that is realized by the processor 11 executing a program stored in the main memory 12 or the flash memory 13. The display processing unit 107 causes the display unit 21 to display a movement trajectory on the screen where the pen 30 has come into contact with the screen and moved, based on the contact position of the pen 30 detected by the touch sensor unit 22.

[0087] Furthermore, the display processing unit 107 causes the display unit 21 to temporarily display the input handwritten stroke, and then causes the display unit 21 to display a handwritten stroke whose shape has been adjusted by replacing the input handwritten stroke.

[0088] The learning processing unit 108 is a functional unit realized by the processor 11 executing a program stored in the main memory 12 or the flash memory 13. The learning processing unit 108 executes a learning process on the learning data stored in the learning data storage unit 47 to generate a classification model.

[0089] Here, the learning data includes learning data for generating an SCE classification model and learning data for generating a CCE classification model. The learning data used to generate the SCE classification model is a correspondence between partial strokes of pre-specified characters handwritten by a user and simple character elements, and multiple characters including simple character elements are generated based on handwritten strokes that are handwritten and input multiple times.

[0090] In this case, the learning processing unit 108 generates an SCE classification model by executing a learning process on the learning data stored in the learning data storage unit 47. The learning processing unit 108 stores the generated SCE classification model (second machine learning result) in the SCE classification information storage unit 44.

[0091] In addition, the learning data used to generate the CCE classification model is generated by associating handwritten strokes of pre-specified characters handwritten by a user with compound character elements, and is generated by having a user handwrite multiple characters including compound character elements multiple times.

[0092] In this case, the learning processing unit 108 generates a CCE classification model by executing a learning process on the learning data stored in the learning data storage unit 47. The learning processing unit 108 stores the generated CCE classification model (first machine learning result) in the CCE classification information storage unit 43.

[0093] Next, the operation of the tablet terminal 1 according to this embodiment will be described with reference to the drawings. FIG. 7 is a diagram showing an overview of handwriting input processing of the tablet terminal 1 in this embodiment.

[0094] 7, tablet terminal 1 first classifies input handwritten strokes into compound character elements using CCE classification model MD1 (step S11). CCE classification processing unit 102 of tablet terminal 1 executes CCE classification processing to classify handwritten strokes into compound character elements using CCE classification model MD1 stored in CCE classification information storage unit 43.

[0095] If the CCE classification processing unit 102 of the tablet terminal 1 successfully classifies the handwritten stroke into a composite character element through the CCE classification processing, the CCE classification processing unit 102 proceeds to step S14, adjusts the shape of the handwritten stroke based on the classified composite character element, and displays it on the display unit 21.

[0096] Furthermore, if the handwritten stroke cannot be properly classified into a compound character element by the CCE classification process, the tablet terminal 1 advances the process to step S12, where the handwritten stroke is divided at its corners to generate partial strokes. The division processing unit 103 of the tablet terminal 1 divides the handwritten stroke to generate partial strokes.

[0097] Next, the tablet terminal 1 classifies the divided partial strokes into simple character elements using the SCE classification model MD2 (step S13). The SCE classification processing unit 104 of the tablet terminal 1 executes an SCE classification process to classify the partial strokes into simple character elements using the SCE classification model MD2 stored in the SCE classification information storage unit 44.

[0098] If the tablet terminal 1 classifies the partial stroke into a simple character element through the SCE classification process, the process proceeds to step S14, where it adjusts the shape of the handwritten stroke based on the classified simple character element and displays it on the display unit 21.

[0099] Next, with reference to FIG. 8, a detailed description will be given of the handwriting input process of the tablet terminal 1 shown in FIG. FIG. 8 is a flowchart showing an example of handwriting input processing of the tablet terminal 1 in this embodiment.

[0100] As shown in Fig. 8, the tablet terminal 1 first determines whether or not a handwritten stroke has been detected (step S101). The input processing unit 101 of the tablet terminal 1 determines whether or not a handwritten stroke has been detected by the touch sensor unit 22. If the input processing unit 101 detects a handwritten stroke (step S101: YES), the process proceeds to step S102. If the input processing unit 101 does not detect a handwritten stroke (step S101: NO), the process returns to step S101.

[0101] In step S102, the input processing unit 101 temporarily displays the handwritten strokes on the display unit 21. The rendering used at this time is performed with a thickness according to the conventional pen pressure, or with a uniform thickness regardless of the pen pressure. That is, the input processing unit 101 causes the display unit 21 to display the detected handwritten strokes via the display processing unit 107. The input processing unit 101 also causes the handwritten strokes to be stored in the handwritten stroke storage unit 45.

[0102] Next, the input processing unit 101 determines whether the pen 30 has been removed from the screen of the display unit 21 (step S103). That is, the input processing unit 101 determines whether the input of the single handwritten stroke has been completed. If the pen 30 has been removed from the screen of the display unit 21 (the input of the handwritten stroke has been completed) (step S103: YES), the input processing unit 101 advances the process to step S104. If the pen 30 has not been removed from the screen of the display unit 21 (the input of the handwritten stroke has not been completed) (step S103: NO), the input processing unit 101 returns the process to step S102.

[0103] In step S104, the classification control unit 105 of the tablet terminal 1 determines whether or not the handwritten stroke includes a corner. If the handwritten stroke includes a corner (step S104: YES), the classification control unit 105 advances the process to step S105. If the handwritten stroke does not include a corner (step S104: NO), the classification control unit 105 advances the process to step S109.

[0104] In step S105, the CCE classification processing unit 102 of the tablet terminal 1 classifies the handwritten strokes into corresponding compound character elements using the CCE classification model. The CCE classification processing unit 102 executes CCE classification processing to classify the handwritten strokes into compound character elements using the CCE classification model stored in the CCE classification information storage unit 43 (see step S11 in FIG. 7).

[0105] Next, classification control unit 105 determines whether or not the confidence level of the CCE classification process is equal to or greater than a threshold (step S106). If the confidence level of the CCE classification process is equal to or greater than the threshold (step S106: YES), classification control unit 105 advances the process to step S107. If the confidence level of the CCE classification process is less than the threshold (step S106: NO), classification control unit 105 advances the process to step S109.

[0106] In step S107, the classification control unit 105 determines whether the number of corners of the handwritten strokes matches the number of corners of the classified composite character element. If the number of corners of the handwritten strokes matches the number of corners of the classified composite character element (step S107: YES), the classification control unit 105 proceeds to step S108. If the number of corners of the handwritten strokes does not match the number of corners of the classified composite character element (step S107: NO), the classification control unit 105 proceeds to step S109.

[0107] In step S108, the rendering processing unit 106 of the tablet terminal 1 adjusts the shape of the handwritten stroke based on the brush information and pen pressure characteristics corresponding to the classified composite character element or character element (simple character element), and displays it on the display unit 21. The rendering processing unit 106 causes the handwritten stroke whose shape has been adjusted to be displayed on the display unit 21 via the display processing unit 107. After the processing of step S108, the rendering processing unit 106 returns the processing to step S101.

[0108] In step S109, the division processing unit 103 of the tablet terminal 1 divides the handwritten stroke at the corners to generate partial strokes. The division processing unit 103 divides the handwritten stroke to generate partial strokes, for example, as in step S12 shown in FIG. 7. The division processing unit 103 stores the generated partial strokes in the partial stroke storage unit 46.

[0109] Next, the SCE classification processing unit 104 of the tablet terminal 1 classifies the partial strokes into corresponding character elements (simple character elements) by the SCE classification model (step S110). The SCE classification processing unit 104 classifies the partial strokes stored in the partial stroke storage unit 46 into simple character elements using the SCE classification model stored in the SCE classification information storage unit 44 (see step S13 in FIG. 7).

[0110] Next, the classification control unit 105 determines whether the certainty of the SCE classification process is equal to or greater than a threshold (step S111). If the certainty of the SCE classification process is equal to or greater than the threshold (step S111: YES), the classification control unit 105 advances the process to step S108. If the certainty of the SCE classification process is less than the threshold (step S111: NO), the classification control unit 105 advances the process to step S112.

[0111] In step S112, the rendering processing unit 106 displays the handwritten strokes on the display unit 21 by a general drawing process. The rendering processing unit 106 may adjust the thickness of the handwritten strokes according to, for example, the contact pressure (writing pressure) acquired from the pen 30 and the angle of the pen 30, and display the handwritten strokes on the display unit 21. After the process of step S12, the rendering processing unit 106 returns the process to step S101.

[0112] In the example shown in FIG. 8, the determination processes in steps S106 and S107 are performed, but one of the determination processes in steps S106 and S107 may be omitted.

[0113] Next, the learning process of the tablet terminal 1 will be described with reference to FIGS. Fig. 9 is a first flowchart showing an example of the learning process of the tablet terminal 1 in this embodiment. In Fig. 9, the learning process of the CCE classification model will be described.

[0114] 9, the learning processing unit 108 of the tablet terminal 1 acquires handwritten strokes of a specific character multiple times for multiple users (step S201). The learning processing unit 108 acquires handwritten strokes of a specific character multiple times for multiple users via the input processing unit 101. Note that the specific characters are preferably multiple characters that can learn a predetermined number of compound character elements.

[0115] Next, the learning processing unit 108 labels each handwritten stroke and the corresponding compound character element to obtain learning data (step S202). The learning processing unit 108 stores the learning data in which the handwritten strokes and the labeling of the compound character element are associated with each other in the learning data storage unit 47.

[0116] Next, the learning processing unit 108 performs machine learning using the learning data to generate a CCE classification model that classifies compound character elements from handwritten strokes (step S203). The learning processing unit 108 executes machine learning processing using the above-mentioned learning data stored in the learning data storage unit 47, and generates a CCE classification model as a first machine learning result.

[0117] Next, the learning processing unit 108 stores the CCE classification model, which is the first machine learning result, in the CCE classification information storage unit 43 (step S204). After the process of step S204, the learning processing unit 108 ends the learning process.

[0118] 10 is a second flowchart showing an example of the learning process of the tablet terminal 1 in this embodiment. In FIG. 10, the learning process of the SCE classification model will be described.

[0119] 10, the learning processing unit 108 of the tablet terminal 1 acquires handwritten strokes of a specific character multiple times for multiple users (step S301). The learning processing unit 108 acquires handwritten strokes of a specific character multiple times for multiple users via the input processing unit 101. Note that the specific characters are preferably multiple characters that can learn multiple predetermined simple character elements.

[0120] Next, the learning processing unit 108 divides each handwritten stroke to generate partial strokes, and labels the corresponding simple character elements to obtain learning data (step S302). The learning processing unit 108 stores the learning data in which the partial strokes and the labeling of the simple character elements are associated with each other in the learning data storage unit 47.

[0121] Next, the learning processing unit 108 performs machine learning using the learning data as well, and generates an SCE classification model that classifies the partial strokes into simple character elements (step S303). The learning processing unit 108 executes machine learning processing using the learning data stored in the learning data storage unit 47, and generates an SCE classification model as a second machine learning result.

[0122] Next, the learning processing unit 108 stores the SCE classification model, which is the second machine learning result, in the SCE classification information storage unit 44 (step S304). After the process of step S304, the learning processing unit 108 ends the learning process.

[0123] As described above, the tablet terminal 1 (information processing device) according to the present embodiment includes a CCE classification processing unit 102 (first classification processing unit), a division processing unit 103, an SCE classification processing unit 104 (second classification processing unit), a classification control unit 105, and a rendering processing unit 106. The CCE classification processing unit 102 uses a CCE classification model (first machine learning result) to identify and classify a compound character element that corresponds to an input handwritten stroke from among a predetermined number of compound character elements that are compound character elements that are basic patterns constituting Chinese characters and that are selected in advance. The division processing unit 103 divides the handwritten stroke at its corners to generate divided partial strokes. The SCE classification processing unit 104 (second classification processing unit) uses an SCE classification model (second machine learning result) different from the CCE classification model to identify and classify a character element (for example, a simple character element) that corresponds to a partial stroke from among a plurality of character elements that is greater than a predetermined number. The classification control unit 105 switches between a CCE classification process (first classification process) by the CCE classification processing unit 102 and an SCE classification process (second classification process) by the SCE classification processing unit 104 depending on whether the handwritten stroke includes a corner portion. The rendering processing unit 106 adjusts the shape of the handwritten stroke based on the brush information and pen pressure characteristics corresponding to the classified composite character element or character element (for example, simple character element), and displays the handwritten stroke with the adjusted shape on the display unit 21.

[0124] As a result, the tablet terminal 1 according to this embodiment classifies handwritten strokes or partial strokes into compound character elements or character elements (simple character elements), and adjusts the shape of the handwritten stroke based on the brush information and pen pressure characteristics corresponding to the compound character element or character element (simple character element), so that when inputting handwritten characters, it is possible to display beautiful characters (especially kanji) while maintaining the quality of the handwriting.

[0125] In addition, the tablet terminal 1 according to the present embodiment adjusts the shape of the handwritten stroke in response to a compound character element or a character element (simple character element) based on the handwritten stroke, so that it is possible to display beautiful characters while maintaining the size and position of the handwritten stroke. Unlike the conventional technology that simply replaces the handwritten stroke with a specified font, the tablet terminal 1 according to the present embodiment does not lose the user's handwriting, and does not lose the freedom of handwriting, such as the aspect ratio of the characters and the size.

[0126] In addition, in the method using character elements as in this embodiment, the display quality of beautiful characters (kanji characters) depends on how accurately the classification process into compound character elements or simple character elements can be executed. Compound character elements are easy to classify with high accuracy, but there are many variations, and it is difficult to generate a classification model for classifying all compound character elements. In addition, for simple character elements, it is possible to learn almost all variations and generate a classification model, but the classification accuracy tends to be low.

[0127] Therefore, in the tablet terminal 1 according to the present embodiment, a predetermined number of classification models of complex character elements (CCE classification models) are generated, and the classification models of complex character elements (CCE classification models) and the classification models of simple character elements (SCE classification models) are used in combination (switched) to classify character elements (complex character elements or simple character elements) with high accuracy. As a result, the tablet terminal 1 according to the present embodiment can accurately execute character elements (complex character elements or simple character elements), and can display neater characters (Chinese characters).

[0128] In this embodiment, when a handwritten stroke includes a corner, the classification control unit 105 causes the CCE classification processing unit 102 to execute a CCE classification process using the CCE classification model. When a handwritten stroke does not include a corner, the classification control unit 105 causes the SCE classification processing unit 104 to execute an SCE classification process using the SCE classification model.

[0129] As a result, the tablet terminal 1 according to this embodiment can appropriately switch between CCE classification processing using the CCE classification model and SCE classification processing using the SCE classification model, and can display even clearer characters (kanji characters).

[0130] Furthermore, in this embodiment, when the confidence level of the CCE classification processing is less than a threshold as a result of causing the CCE classification processing unit 102 to execute the CCE classification processing using the CCE classification model, the classification control unit 105 causes the SCE classification processing unit 104 to execute the SCE classification processing using the SCE classification model. When the rendering processing unit 106 executes the SCE classification processing, it adjusts the shape of the handwritten stroke based on the brush information and writing pressure characteristics corresponding to the character element (simple character element) classified by the SCE classification processing, and displays the handwritten stroke with the adjusted shape on the display unit 21.

[0131] As a result, when the tablet terminal 1 of this embodiment is unable to properly classify a compound character element using the CCE classification process, it is able to properly adjust the shape of handwritten strokes using classification of character elements (simple character elements) using the SCE classification process, thereby enabling the display of neat characters while maintaining the quality of the handwriting.

[0132] Furthermore, in this embodiment, when the result of executing the CCE classification process shows that the number of corner parts contained in the handwritten stroke does not match the number of corner parts contained in the classified compound character element, the classification control unit 105 causes the SCE classification processing unit 104 to execute an SCE classification process using an SCE classification model.

[0133] As a result, when the tablet terminal 1 of this embodiment is unable to properly classify a complex character element using the CCE classification process, it can properly adjust the shape of a handwritten stroke using classification of character elements (simple character elements) using the SCE classification process.

[0134] In this embodiment, the CCE classification model is a classification model generated by executing a learning process on learning data in which a predetermined number of preselected composite character elements are associated with corresponding handwritten strokes.The SCE classification model is a classification model generated by executing a learning process on learning data in which preselected character elements for partial strokes (simple character elements) are associated with corresponding partial strokes.

[0135] As a result, the tablet terminal 1 according to this embodiment can appropriately generate a CCE classification model and an SCE classification model with a realistic amount of calculations, and by switching between the CCE classification model and the SCE classification model, the classification accuracy of character elements can be improved.

[0136] Furthermore, the tablet terminal 1 according to this embodiment includes a pen pressure characteristics storage unit 42 that stores character elements (simple character elements) or compound character elements in association with pressure curves indicating the relationship between stroke positions and pen pressure in the character elements (simple character elements) or compound character elements. The rendering processing unit 106 acquires the pressure curves corresponding to the classified character elements (simple character elements) or compound character elements from the pen pressure characteristics storage unit 42, and adjusts the thickness of the handwritten strokes based on the acquired pressure curves.

[0137] As a result, the tablet terminal 1 of this embodiment can appropriately adjust the thickness of a handwritten stroke (for example, changes in thickness midway through a stroke) according to the stroke position using a pressure curve corresponding to a character element (simple character element) or a compound character element, thereby enabling the display of neat characters while maintaining the quality of the handwriting.

[0138] Furthermore, the tablet terminal 1 according to this embodiment includes a brush information storage unit 41 that stores character elements (simple character elements) or compound character elements in association with brush information including the size, aspect ratio, and angle of a brush to be used. The rendering processing unit 106 acquires brush information corresponding to the classified character elements (simple character elements) or compound character elements from the brush information storage unit 41, and adjusts the shape of a handwritten stroke based on the acquired brush information.

[0139] As a result, the tablet terminal 1 according to this embodiment can change the brush depending on the character element (simple character element) or compound character element using brush information corresponding to the character element (simple character element) or compound character element, and appropriately adjust the shape of the handwritten stroke (for example, the shape of the corners at the beginning and end of the stroke, the crease, etc.), thereby displaying beautiful characters while maintaining the quality of the handwriting.

[0140] Moreover, the tablet terminal 1 according to this embodiment includes a display unit 21, a touch sensor unit 22, and an input processing unit 101. The touch sensor unit 22 detects the contact and contact position of an operation medium (e.g., a pen 30) on a detection surface on the screen (on the display screen DF) of the display unit 21. The input processing unit 101 detects the input of a handwritten stroke based on the contact position of the operation medium (e.g., a pen 30) detected by the touch sensor unit 22. The input processing unit 101 detects the input of a handwritten stroke and displays a movement trajectory of the contact position of the operation medium (e.g., a pen 30) on the display unit 21 as an input handwritten stroke. The rendering processing unit 106 replaces the input handwritten stroke and displays the handwritten stroke with the adjusted shape on the display unit 21.

[0141] As a result, the tablet terminal 1 according to this embodiment first displays the input handwritten strokes, making it possible to input handwritten characters while checking the display, making handwritten input easier, and by replacing the handwritten strokes with handwritten strokes based on the adjusted partial strokes, it is possible to display neat characters.

[0142] The control method according to the present embodiment is a control method for the tablet terminal 1 including the CCE classification processing unit 102, the division processing unit 103, and the SCE classification processing unit 104, and includes a classification control step and a rendering processing step. The CCE classification processing unit 102 uses the CCE classification model to identify and classify a compound character element corresponding to an input handwritten stroke from among a predetermined number of compound character elements that are compound character elements (simple character elements) that are basic patterns constituting Chinese characters. The division processing unit 103 divides the handwritten stroke at its corners to generate divided partial strokes. The SCE classification processing unit 104 uses an SCE classification model different from the CCE classification model to identify and classify a character element (simple character element) corresponding to a partial stroke from among a plurality of character elements (simple character elements) that are greater than a predetermined number. In the classification control step, the classification control unit 105 switches between the CCE classification processing by the CCE classification processing unit 102 and the SCE classification processing by the SCE classification processing unit 104 depending on whether the handwritten stroke includes a corner or not. In the rendering processing step, the rendering processing unit 106 adjusts the shape of the handwritten stroke based on the brush information and pen pressure characteristics corresponding to the classified composite character element or character element (simple character element), and displays the handwritten stroke with the adjusted shape on the display unit 21.

[0143] As a result, the control method according to this embodiment has the same effect as the tablet terminal 1 described above, and when inputting handwritten text, it is possible to display beautiful characters (especially kanji characters) while maintaining the quality of the handwritten text.

[0144] The present invention is not limited to the above-described embodiment, and can be modified without departing from the spirit of the present invention. For example, in the above embodiment, the information processing device is a tablet terminal 1, but the present invention is not limited to this. The information processing device may be, for example, a smartphone or a notebook personal computer equipped with a tablet mode.

[0145] In addition, in the above embodiment, an example of the OS used is described using Android (registered trademark), but this is not limited to this and may be applied to other OS such as iOS (registered trademark).

[0146] In the above embodiment, the processes of the CCE classification processing unit 102, the segmentation processing unit 103, the SCE classification processing unit 104, the classification control unit 105, and the rendering processing unit 106 may be realized by an application program, or part of the processes may be realized by a device driver or the like.

[0147] Furthermore, when the information processing device is a notebook personal computer, part of the processing of the CCE classification processing unit 102, the division processing unit 103, the SCE classification processing unit 104, the classification control unit 105, and the rendering processing unit 106 may be performed by an embedded controller.

[0148] In addition, in each of the above embodiments, an example has been described in which the tablet terminal 1 includes the learning processing unit 108, but the present invention is not limited to this, and for example, the tablet terminal 1 may not include the learning processing unit 108. In this case, for example, an external device such as a server device connectable to the tablet terminal 1 may include the learning processing unit 108.

[0149] Each of the components of the tablet terminal 1 described above has an internal computer system. A program for implementing the functions of each of the components of the tablet terminal 1 described above may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read into a computer system and executed to perform processing in each of the components of the tablet terminal 1 described above. Here, "reading a program recorded on a recording medium into a computer system and executing it" includes installing the program into a computer system. The "computer system" referred to here includes hardware such as an OS and peripheral devices.

[0150] Furthermore, a "computer system" may include multiple computer devices connected via a network including the Internet, WAN, LAN, dedicated lines, and other communication lines. Furthermore, a "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into a computer system. In this way, the recording medium storing the program may be a non-transitory recording medium such as a CD-ROM.

[0151] The recording medium also includes a recording medium installed inside or outside and accessible from a distribution server to distribute the program. The program may be divided into a plurality of parts, downloaded at different times, and then combined with each component of the tablet terminal 1, or each divided program may be distributed by a different distribution server. Furthermore, the "computer-readable recording medium" includes a recording medium that holds a program for a certain period of time, such as a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network. The above program may be a recording medium for realizing part of the above-mentioned functions. Furthermore, the recording medium may be a so-called difference file (difference program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

[0152] In addition, some or all of the above-mentioned functions may be realized as an integrated circuit such as LSI (Large Scale Integration). Each of the above-mentioned functions may be individually processed, or some or all of the functions may be integrated into a processor. The integrated circuit method is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. In addition, if an integrated circuit technology that can replace LSI appears due to advances in semiconductor technology, an integrated circuit based on that technology may be used. [Explanation of symbols]

[0153] 1. Tablet device 10 Control section 11 Processors 12. Main memory 13. Flash Memory 20. Touchscreen 21 Display section 22 Touch sensor section 23 Peripheral Devices 24 Audio System 25. Mike 26 Speaker 27 Baseband chip 28 Radio Section 29, 31 Wireless Communication Department 30 pens 32 Sensor section 33 Memory 34 MCU 40 Storage section 41 Brush information storage unit 42 Pen pressure characteristics memory section 43 CCE classification information storage section 44 SCE classification information storage section 45 Handwritten stroke memory 46 Partial Stroke Memory Unit 47 Learning data storage unit 100 Information Processing Systems 101 Input processing unit 102 CCE classification processing unit 103 Division Processing Unit 104 SCE classification processing unit 105 Classification control section 106 Rendering Processing Unit 107 Display processing unit 108 Learning processing unit

Claims

1. a first classification processing unit that uses the first machine learning result to identify and classify a compound character element that corresponds to the input handwritten stroke from among a predetermined number of compound character elements that are compound character elements that are basic patterns constituting Chinese characters; a division processing unit that divides the handwritten stroke at corners and generates divided partial strokes; a second classification processing unit that identifies and classifies the character elements corresponding to the partial strokes from among the plurality of character elements that is greater than the predetermined number, by using a second machine learning result different from the first machine learning result; a classification control unit that switches between a first classification process by the first classification processing unit and a second classification process by the second classification processing unit depending on whether the handwritten stroke includes the corner portion; a rendering processing unit that adjusts a shape of the handwritten stroke based on brush information and writing pressure characteristics corresponding to the classified composite character element or the character element, and displays the handwritten stroke whose shape has been adjusted on a display unit; An information processing device comprising:

2. The classification control unit is when the handwritten stroke includes the corner portion, causing the first classification processing unit to execute the first classification process using the first machine learning result; When the handwritten stroke does not include the corner portion, the second classification processing unit executes the second classification process using the second machine learning result. The information processing device according to claim 1 .

3. The classification control unit is when a degree of certainty of the first classification process is less than a threshold as a result of causing the first classification processing unit to execute the first classification process using the first machine learning result, causing the second classification processing unit to execute the second classification process using the second machine learning result; When the rendering processing unit has executed the second classification process, the rendering processing unit adjusts a shape of the handwritten stroke based on the brush information and the writing pressure characteristics corresponding to the character element classified by the second classification process, and displays the handwritten stroke with the adjusted shape on the display unit. The information processing device according to claim 2 .

4. The classification control unit is When the number of the corners included in the handwritten stroke does not match the number of the corners included in the classified composite character element as a result of executing the first classification process, the second classification processing unit is caused to execute the second classification process using the second machine learning result. The information processing device according to claim 3 .

5. the first machine learning result is a classification model generated by executing a learning process on learning data in which a predetermined number of the composite character elements are associated with the corresponding handwritten strokes, The second machine learning result is a classification model generated by executing a learning process on learning data in which the character elements for the partial strokes selected in advance are associated with the corresponding partial strokes. The information processing device according to claim 1 .

6. a writing pressure characteristics storage unit that stores the character element or the composite character element in association with a pressure curve that indicates a relationship between a stroke position and a writing pressure in the character element or the composite character element, The rendering processing unit acquires the pressure curve corresponding to the classified character element or the composite character element from the pen pressure characteristic storage unit, and adjusts a thickness of the handwritten stroke based on the acquired pressure curve. The information processing device according to claim 1 .

7. a brush information storage unit that stores the character element or the composite character element in association with the brush information including a size, an aspect ratio, and an angle of a brush to be used; The rendering processing unit acquires the brush information corresponding to the classified character element or the composite character element from the brush information storage unit, and adjusts a shape of the handwritten stroke based on the acquired brush information. The information processing device according to claim 1 .

8. The display unit; a touch sensor unit that detects a contact and a contact position of an operation medium on a detection surface on a screen of the display unit; an input processing unit that detects an input of the handwritten stroke based on a contact position of the operation medium detected by the touch sensor unit; Equipped with the input processing unit detects the input of the handwritten stroke, and displays a movement trajectory of a contact position of the operation medium on the display unit as the input handwritten stroke; The rendering processing unit replaces the input handwritten stroke with the handwritten stroke whose shape has been adjusted, and displays the handwritten stroke on a display unit. The information processing device according to claim 1 .

9. a first classification processing unit that uses a first machine learning result to identify and classify a compound character element that corresponds to an input handwritten stroke from among a preselected predetermined number of compound character elements, the compound character element being a compound character element formed by combining character elements that are basic patterns constituting a Chinese character; a segmentation processing unit that divides the handwritten stroke at corners to generate divided partial strokes; and a second classification processing unit that uses a second machine learning result different from the first machine learning result to identify and classify the character element that corresponds to the partial stroke from among a plurality of the character elements that is greater than the predetermined number, a classification control unit that switches between a first classification process by the first classification processing unit and a second classification process by the second classification processing unit depending on whether the handwritten stroke includes the corner portion; a rendering processing unit adjusting a shape of the handwritten stroke based on brush information and writing pressure characteristics corresponding to the composite character element or the character element classified by the first classification process or the second classification process, and displaying the handwritten stroke with the adjusted shape on a display unit; A control method comprising:

Citation Information

Patent Citations

  • Information processing device, information processing system, information processing method and information processing program

    JP2021043851A