Electronic device and information processing method

Abstract

The present invention provides an electronic device and an information processing method that achieve the goal of increasing the enthusiasm of users who use note-taking tools to perform writing, drawing, or other such activities, allowing them to enjoy learning and improve their skills. One embodiment of the present invention includes: a main body having a note-taking function and being in the shape of a pen, or capable of being attached to a note-taking tool; a detection unit provided in the main body for detecting the magnitude of a user's note-taking action; a storage unit for storing data including at least one of image data and numerical data; and a control unit for controlling the selection of image data or numerical data from the storage unit in accordance with the magnitude detected by the detection unit, and causing an image based on the selected image data or numerical data to be displayed on a display unit.

Description

Technical Field

[0001] The present invention relates to electronic devices and information processing methods, and in particular to electronic devices and information processing methods that can change information such as images on software in response to note-taking actions such as writing text or drawing pictures, and can also play games. Background Art

[0002] Patent Documents 1 and 2 disclose games for raising virtual creatures. For example, the image display device described in Patent Document 1 uses an electronic device such as an electronic notebook or electronic game console to display the growth process of organisms such as animals or plants as images. Furthermore, the cultivation simulation device described in Patent Document 2 raises and grows virtual creatures by allowing the player to respond to calls or requests from the virtual creatures on the screen.

[0003] Patent Document 3 discloses a technique for detecting the motion of a toy wand and influencing software, etc. Patent Document 4 discloses a technique for reading the motion of a pen and transmitting it to a server.

[0004] Patent Documents 5 and 6 disclose technologies for tracking handwriting. Specifically, Patent Document 5 discloses a method comprising: a step of transmitting light from a light source moving over a writing instrument to display the position and path of the writing instrument on a two-dimensional writing surface; a step of detecting the light using two or more sensors and generating a sequence of signals representing the detected light; and a step of applying technology to improve reading stability due to insufficient pixels. Furthermore, Patent Document 6 discloses an electronic addition and deletion system comprising: a unit that extracts and analyzes body movement information from electronic pen note information; and a display unit capable of displaying at least the answer status obtained by the analysis unit, the answer result input via the electronic pen, and the correct answer information to a question.

[0005] Patent Document 7 discloses a handwriting input device. Specifically, in the handwriting input device disclosed in Patent Document 7, if a handwriting is taken with an input pen, the handwriting is detected in real time and displayed on a TV as if it were a drawing.

[0006] Patent Document 8 discloses a handwritten text input game device that allows players to advance through handwritten text input. The device includes a unit that displays text used as a copybook, a unit that inputs handwritten text using a pen and simultaneously displays its content, a unit that recognizes the strokes of handwritten text, a unit that determines the strokes of the input text, a unit that stores the learning difficulty level and the copybook, and a unit that displays the determination result on a display unit and advances to the next copybook of the next difficulty level based on the determination result.

[0007] Patent Document 9 discloses a portable handwritten text input device, in which handwritten text is read as video data by a camera, stored in a memory, and the stored video data is transmitted to an external device via infrared rays.

[0008] Patent Document 10 discloses a controller for a conversational drawing game. The controller includes a sensor configured to track the controller's position in three-dimensional space and transmit sensor signals representing the positional information to a computer; and an actuator configured to output haptic effects to the user in response to control signals from a processor connected to the controller.

[0009] Furthermore, Patent Documents 11 to 14 disclose devices that detect the movement of a toothbrush to assist with toothbrushing.

[0010] [Patent Document 1] Japanese Patent Application Laid-Open No. 7-160853

[0011] [Patent Document 2] Japanese Patent Application Laid-Open No. 10-232595

[0012] [Patent Document 3] Japanese Patent Publication No. 2006-521162

[0013] [Patent Document 4] Japanese Patent Publication No. 2004-537342

[0014] [Patent Document 5] Japanese Patent Application Laid-Open No. 2009-187565

[0015] [Patent Document 6] Japanese Patent Application Laid-Open No. 2009-031614

[0016] [Patent Document 7] Japanese Patent Application Laid-Open No. 7-248875

[0017] [Patent Document 8] Japanese Patent Application Laid-Open No. 2006-204853

[0018] [Patent Document 9] Japanese Patent Application Laid-Open No. 2002-041230

[0019] [Patent Document 10] Japanese Patent Publication No. 2012-512724

[0020] [Patent Document 11] International Publication No. WO2016 / 104442

[0021] [Patent Document 12] Japanese Patent Application Publication No. 2017-060661

[0022] [Patent Document 13] Japanese Patent Application Publication No. 2017-060662

[0023] [Patent Document 14] Japanese Patent Application Laid-Open No. 2009-240760

[0024] When studying or drawing, we use pencils, mechanical pencils, ballpoint pens, and other notepads. When using these notepads to acquire knowledge and improve skills, it is important to cultivate enthusiasm.

[0025] Here, as in the technology described in Patent Document 1 or Patent Document 2, there are games such as raising animals, virtual creatures, pets, etc. on smartphones, personal computers, game consoles, etc., which have the fun of hatching virtual creatures from eggs, making them grow, and evolving. In addition, in the technology described in Patent Document 3 or Patent Document 4, the actions of electronic devices are read. However, these are limited to the gameplay of raising virtual creatures and have nothing to do with behaviors that help writing or drawing. Therefore, it is not possible to expect the effect of increasing enthusiasm for writing or learning.

[0026] Furthermore, although handwriting tracing devices described in Patent Documents 5 and 6, the handwriting input device described in Patent Document 7, and the handwritten character input device described in Patent Document 9 can perform handwriting tracing, they do not enhance the effect of improving character practice and learning enthusiasm.

[0027] In addition, in the text input game device described in Patent Document 8, the input character strokes are determined by inputting characters that become copybooks, but this determination is only for predetermined copybooks. In addition, in the controller associated with the conversational drawing game described in Patent Document 10, a tactile effect is output to the user. In addition, in the devices described in Patent Documents 11 to 14, assistance related to brushing teeth is provided. None of these technologies can fully associate various note-taking actions such as writing characters or drawing with the effect of improving learning enthusiasm such as learning or skills. Summary of the Invention

[0028] The present invention aims to provide an electronic device and an information processing method that allow users who use writing instruments to perform writing operations such as writing or drawing to improve their skills while learning with enthusiasm.

[0029] To address the aforementioned issues, one embodiment of the present invention provides an electronic device comprising: a main body having a note-taking function and being in the form of a pen or attachable to a note-taking device; a detection unit provided in the main body and configured to detect the magnitude of a user's note-taking action; a storage unit storing data comprising at least one of image data and numerical data; and a control unit controlling the selection of data from the storage unit in accordance with the magnitude detected by the detection unit, and causing an image based on the selected data to be displayed on a display unit. At least one of the storage unit, display unit, and control unit may be provided in the main body of the electronic device or in an external device (e.g., a portable terminal).

[0030] According to this configuration, an image is displayed on the display unit in accordance with the magnitude of the writing action of the writing instrument. In other words, the image displayed on the display unit changes according to the magnitude of the writing action based on text, drawing, etc.

[0031] Another embodiment of the present invention relates to an electronic device comprising: a panel portion having a detection portion for detecting handwriting; a storage portion for storing data comprising at least one of image data and numerical data; and a control portion for controlling the selection of data from the storage portion in accordance with a note amount value based on handwriting detected by the detection portion, and displaying an image based on the selected data on the display portion.

[0032] According to this configuration, handwriting is detected by the detection unit provided on the panel portion, and an image is displayed on the display portion in accordance with the amount of the note based on the detected handwriting. In other words, the image displayed on the display portion changes according to the amount of the note based on handwriting such as text or drawings written on the panel portion.

[0033] Another embodiment of the present invention relates to an information processing method for an electronic device, wherein the electronic device includes a sensor and a control unit, wherein the sensor detects a user's movements, and the control unit controls and changes an image displayed on a display unit based on information detected by the sensor.

[0034] In this information processing method, the control unit includes: a step of detecting a note-taking action in a user's action based on information obtained from a sensor; a step of calculating a magnitude of the note-taking action from the detected note-taking action; and a step of selecting an image to be displayed on the display unit corresponding to the magnitude.

[0035] With this configuration, a note-taking action is detected among the user's actions, and an image is displayed on the display unit in accordance with the magnitude of the detected note-taking action. Specifically, the image displayed on the display unit changes depending on the magnitude of the note-taking action, such as text or drawing, performed by the user.

[0036] An information processing method according to another embodiment of the present invention comprises: a step of detecting a note-taking action of a user when using an electronic device using a sensor, wherein the electronic device comprises a sensor for detecting the note-taking action; a step of calculating the value of the note-taking action from the note-taking action detected by the sensor; and a step of granting points to establish a corresponding relationship with the user and exchanging consideration corresponding to the points with a server via any network.

[0037] According to this configuration, the writing action among the user's actions is detected, and points corresponding to the amount of the detected writing action are awarded. Thus, the points can be exchanged for a desired consideration.

[0038] According to the present invention, it is possible to provide an electronic device and an information processing method that can increase the enthusiasm of users who use writing instruments to perform writing operations such as writing and drawing, and improve their skills while having fun learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a schematic diagram illustrating an electronic device according to the first embodiment.

[0040] Figure 2 This is a block diagram illustrating an electronic device according to the first embodiment.

[0041] Figure 3 This is a flowchart schematically showing the information processing method according to this embodiment.

[0042] Figure 4 This is a flowchart showing a detailed example (Part 1) of the information processing method according to the present embodiment.

[0043] Figure 5 This is a flowchart showing a detailed example (Part 1) of the information processing method according to the present embodiment.

[0044] Figure 6 This is a flowchart showing a detailed example (Part 1) of the information processing method according to the present embodiment.

[0045] Figure 7 This is a flowchart showing a detailed example (Part 1) of the information processing method according to the present embodiment.

[0046] Figure 8 (a) and (b) are schematic diagrams showing other configuration examples of the first embodiment.

[0047] Figure 9 It is a schematic diagram showing another configuration example of the first embodiment.

[0048] Figure 10is a schematic diagram illustrating an example of using a plurality of pen-type electronic devices.

[0049] Figure 11 This is a flowchart showing an example of processing when two pen-type electronic devices are close to each other.

[0050] Figure 12 is a schematic diagram illustrating auxiliary equipment.

[0051] Figure 13 is a block diagram illustrating auxiliary equipment.

[0052] Figure 14 This is a schematic diagram illustrating the configuration of a pen-type electronic device having a power generation function.

[0053] Figure 15 This is a schematic diagram illustrating an electronic device according to the second embodiment.

[0054] Figure 16 It is a schematic diagram showing another configuration example of the second embodiment.

[0055] Figure 17 It is a schematic diagram showing another configuration example of the second embodiment.

[0056] Figure 18 (a) and (b) are schematic diagrams showing other configuration examples of the second embodiment.

[0057] Figure 19 is a schematic diagram illustrating a hat-type electronic device.

[0058] Figure 20 This is a schematic diagram illustrating an electronic device according to a third embodiment.

[0059] Figure 21 This is a flowchart schematically showing the operation of the electronic device according to this embodiment.

[0060] Figure 22 Schematic diagram illustrating an example of using a plurality of panel-type electronic devices.

[0061] Figure 23 It is a schematic diagram showing another configuration example of the fourth embodiment.

[0062] Figure 24 Schematic diagrams illustrating examples of electronic devices using different methods.

[0063] Figure 25 (a) and (b) are schematic diagrams illustrating information exchange between electronic devices.

[0064] Figure 26 This is a schematic diagram illustrating information exchange between electronic devices.

[0065] Figure 27 This is a flowchart showing the information processing method involved in the fifth embodiment.

[0066] Figure 28 This is a flowchart illustrating the restriction process.

[0067] Figure 29 This is a flowchart illustrating the determination process of an example action.

[0068] Figure 30 This is a flowchart illustrating the determination process of an example action.

[0069] Figure 31 This is a flowchart illustrating the process of determining a learning subject.

[0070] Figure 32 is a diagram showing an example of a graph display.

[0071] Figure 33 This is a schematic diagram showing an example of handwriting recognition.

[0072] Figure 34 is a schematic diagram showing an example of information sharing.

[0073] Figure 35 is a diagram showing an example of a ranking display.

[0074] Figure 36 is a schematic diagram showing an example of information aggregation.

[0075] Figure 37 This is a flowchart showing an example of information confirmation processing.

[0076] Figure 38 This is a flowchart illustrating the exchange process for actual objects.

[0077] Figure 39 This is a schematic diagram showing an example of how growth is displayed.

[0078] Figure 40 is a diagram showing an example of a collection display.

[0079] Figure 41 (a) and (b) are schematic diagrams illustrating the distinction between note-taking actions.

[0080] Figure 42 This is a schematic diagram illustrating the classification of users of electronic devices.

[0081] Figure 43 This is a schematic diagram illustrating the correspondence between a pen-shaped electronic device and text information.

[0082] Figure 44This is a flowchart illustrating the correspondence between a pen-shaped electronic device and text information.

[0083] Figure 45 This is a schematic diagram showing an example of switching between acquisition modes.

[0084] Figure 46 : is a flowchart showing an example of correct answer determination processing.

[0085] Figure 47 is a flowchart showing an example of a limited function process.

[0086] Figure 48 This is a diagram showing an example of scoring for the pen spinning action.

[0087] Figure 49 is a schematic diagram showing an example of an electronic device having a built-in speaker.

[0088] Figure 50 This is a flowchart showing an example of processing for associating a dynamic image with a pen movement.

[0089] Figure 51 : is a flowchart showing an example of signature identification processing. DETAILED DESCRIPTION

[0090] Hereinafter, embodiments of the present invention will be described based on the drawings. In the following description, the same reference numerals are given to the same components, and description of components that have been described once will be omitted as appropriate.

[0091] (First embodiment)

[0092] Figure 1 This is a schematic diagram illustrating an electronic device according to the first embodiment.

[0093] Figure 2 This is a block diagram illustrating an electronic device according to the first embodiment.

[0094] The electronic device according to this embodiment is a pen-shaped electronic device 1 such as a mechanical pencil or a ballpoint pen. The pen-shaped electronic device 1 includes a main body 10 having a note-taking function. The main body 10 includes a display (display unit) 2, an information processing unit 4, a battery 5, and a sensor 6.

[0095] A pen tip 10p is provided at one end of the body 10. The user can hold the body 10 and use the pen tip 10p to draw text or pictures. A knocking portion 3 can also be provided at the other end of the body 10. That is, the pen tip 10p can be protruded by pressing the knocking portion 3.

[0096] The display 2 is a display unit using liquid crystal or organic EL. The display 2 is arranged, for example, near the other end of the main body 10 (a position where it is not hidden when the main body 10 is held). The display 2 displays predetermined images or text information.

[0097] The information processing unit 4 includes a control unit 41, a memory 42, and a communication unit 43. These may be formed from a single chip or from separate chips. The control unit 41 is, for example, a CPU, and controls each component through programs executed therein. The memory 42 includes ROM and RAM. The memory 42 stores programs executed by the control unit and data displayed on the display 2 (at least one of image data and numerical data).

[0098] The communication unit 43 is a part that performs information input and output with an external device (for example, the portable terminal 7). The communication unit 43 includes a sending unit 431 that sends information to the outside and a receiving unit 432 that receives information sent from the outside. As external devices, in addition to the portable terminal 7, for example, a smartphone, a tablet terminal, a game console, a dedicated terminal, and a personal computer can be cited. The communication unit 43 is, for example, a wireless communication unit (a communication unit that uses radio waves or light). As wireless communication units, Bluetooth (registered trademark) or Wi-Fi (registered trademark), an infrared communication unit, and a telephone line can be cited. The battery 5 is, for example, a storage battery, and is a source of power for driving each part.

[0099] The sensor 6 is a part that detects the movement of the main body 10. Examples of the sensor 6 include an acceleration sensor, a tilt sensor, a rotational track sensor, a gravity acceleration sensor, a gyroscope sensor, an inertial sensor, and a pressure sensor. In addition, the sensor 6 may also include a unit that detects the operation of the knocking part 3.

[0100] In this embodiment, sensor 6 is used to detect the magnitude of the user's note-taking action. Here, the magnitude of the note-taking action is at least one of the values representing the note-taking frequency, number of notes, amount of notes, text recognition, note-taking speed, note-taking acceleration, the angle of the main body 10, and the pressure during note-taking of the pen-type electronic device 1. In addition, sensor 6 can also be used to detect values based on the method of use and the mode of operation of the pen-type electronic device 1. These magnitudes are the output values from sensor 6 themselves, or values calculated based on the output values. The calculation to calculate the magnitude of the note-taking action can be performed by sensor 6, or by control unit 41 or other parts.

[0101] For example, a note action is considered "started" when the acceleration sensor's acceleration exceeds a certain value and changes more than a certain number of times, and "completed" when the acceleration sensor's acceleration remains below a certain value for a certain period of time. Each time this starts and completes occurs, the note frequency is incremented, and the number of times this starts and completes is the note frequency.

[0102] When the acceleration sensor, tilt sensor, and rotation sensor sense changes, the number of notes is counted. The amount of notes is counted based on how long the changes detected by the sensors continue to occur.

[0103] Methods for determining text recognition include, for example, determining if the acceleration changes over a consistent period or if actions such as lifting or pressing the pen occur regularly. For example, in the case of drawing, the acceleration changes occur relatively gradually, while in the case of writing Chinese characters, the acceleration changes occur continuously over short periods or repeatedly at angles close to 0° / 90°.

[0104] The motion method can be detected based on, for example, changes in acceleration, tilt, or rotational direction. Furthermore, by measuring the time until these changes occur, information such as the distance, direction, cycle, interval, rotation, and tilt of the pen tip 10p can be obtained. By detecting at least one of these pieces of information, the motion method of the pen tip 10p can be detected.

[0105] The magnitude of the user's note-taking action detected by sensor 6 can be stored in memory 42, within portable terminal 7, or on a server via a network. In this embodiment, control unit 41 controls the selection of data (at least one of image data and numerical data) from memory 42 in accordance with the magnitude of the user's note-taking action detected by sensor 6, and displays an image based on the selected data on display 2.

[0106] Furthermore, the control unit 41 can transmit information on the magnitude of the writing action to the portable terminal 7 via the communication unit 43. Application software is activated in the portable terminal 7, and upon receiving information transmitted from the pen-shaped electronic device 1 via wireless communication or the like, image data corresponding to the information is selected and displayed on the display 8. In this case, the image data or the like can be stored in the portable terminal 7 or downloaded from a server via a network (e.g., an intranet).

[0107] In addition to the above-mentioned configuration, the pen-type electronic device 1 of this embodiment includes Figure 2The power switch 205, image processing unit 201, charging unit 208, and antenna 209 are shown. The power switch 205 turns the pen-shaped electronic device 1 on and off. The image processing unit 201 processes image data read from the memory 42 according to instructions from the control unit 41 and transmits it to the display 2. The charging unit 208 has a line or cable connection terminal for charging the battery 5. The antenna 209 is connected to the communication unit 43 to transmit and receive information with external devices.

[0108] (Information Processing Method)

[0109] Figure 3 This is a flowchart schematically showing the information processing method according to this embodiment.

[0110] Furthermore, although the information processing method using the pen-type electronic device 1 is exemplified here, the same information processing method can be applied to electronic devices of other types.

[0111] First, when the pen-type electronic device 1 starts operating (step S1), it is determined whether the sensor 6 detects the operation (step S2). If the operation is detected, the sensor 6 acquires operation information based on the operation (step S3). The operation information includes the magnitude of the writing operation.

[0112] Next, the control unit 41 selects data (at least one of image data and numerical data) corresponding to the magnitude of the note action. In the present embodiment, as an example, the image of a creature or character (hereinafter referred to simply as "character, etc.") on the software is changed (step S4). That is, the control unit 41 selects image data of a character, etc. on the software from the memory 42, etc., corresponding to the magnitude of the note action, to display an image based on the image data on the display 2. For example, an image of a character, etc. that grows as the magnitude of the note action increases is displayed. In this way, an image display of a character, etc. that grows together with the note action can be obtained. In addition, the control unit 41 can also obtain numerical data corresponding to the magnitude of the note action from the memory 42, etc. to graph the numerical data and then display it on the display 2.

[0113] Next, it is determined whether the pen-type electronic device 1 is being used continuously (step S5). If it is being used continuously, the process returns to step S2 and the subsequent processes are repeated. Otherwise, if it is not being used continuously, the process is completed (step S6).

[0114] Through such actions, based on information related to the use or action of the pen-type electronic device 1, an image display or a graphical display of any changes of a character, such as being born from an egg, growing, becoming stronger, getting older, having personality changes, changing properties, evolving, etc., is performed.

[0115] Here, a detailed example of the information processing method according to this embodiment will be described.

[0116] Figure 4 This is a flowchart showing a detailed example (Part 1) of the information processing method according to the present embodiment.

[0117] exist Figure 4 In the detailed example (part 1) of the information processing method shown, first, when the pen-shaped electronic device 1 begins operating (step S541), it is determined whether the sensor 6 detects the motion (step S542). If motion is detected, motion information is acquired based on the motion detected by the sensor 6 (step S543), and the acquired motion information is stored (step S544). The motion information can be stored in the memory 42 within the pen-shaped electronic device 1 or in an external device (such as the portable terminal 7 or a server) via a network.

[0118] Next, based on the accumulated information, the information processing unit 4 performs analysis and identification of the action (step S545). This identification includes analyzing the note action to determine whether it is a marked page, a language other than Japanese such as English, a highly recognizable word, and the subject of study.

[0119] Next, it is determined whether to transmit information (step S546). If information is to be transmitted, the pen-shaped electronic device 1 transmits information (action information or determination information) to the portable terminal 7, or transmits information from the portable terminal 7 to the server via the network.

[0120] Next, the device receiving the information (portable terminal 7, server, etc.) performs motion analysis and identification (step S547). Even in the device receiving the information, similar to step S545, the analyzed note-taking motion is identified, including whether it is a page markup, a language other than English or Japanese, a highly recognizable character, or a learning subject. Furthermore, the device receiving the information may also perform pen motion analysis and identification using an algorithm different from that used by the pen-shaped electronic device 1.

[0121] Next, based on the discriminated information, it is determined whether or not to change the image or the like (step S548). That is, if the discriminated information satisfies the conditions for changing the image or the like, it is determined that the image or the like is to be changed.

[0122] Next, the control unit 41 performs processing corresponding to the change determination (step S549). For example, processing may be performed to change the image of the character, etc. on the software, or to obtain points as numerical data, obtain items, or display them in a graph. This may result in an image display showing that the character, etc. grows along with the note action, or points increase along with the note action, or items are obtained.

[0123] Next, it is determined whether the pen-type electronic device 1 is being used continuously (step S550). If it is being used continuously, the process returns to step S542 and the subsequent processing is repeated. Otherwise, if it is not being used continuously, the process is completed (step S551).

[0124] Figure 5 This is a flowchart showing a detailed example (part 2) of the information processing method according to this embodiment.

[0125] exist Figure 5 In the detailed example (part 2) of the information processing method shown, Figure 4 In the detailed example (part 1) of the information processing method shown in FIG5 , information transmission (step S546) and information analysis / judgment (step S547) are performed. Specifically, the detailed example (part 2) includes the following steps: starting the operation of the pen-type electronic device 1 (step S541), detecting the operation (step S542), acquiring the operation information (step S543), accumulating the operation information (step S544), analyzing and distinguishing the information (step S545), determining a change (step S548), processing corresponding to the change determination (step S549), determining whether to continue (step S550), and completing the process (step S551).

[0126] Figure 6 This is a flowchart showing a detailed example (part 2) of the information processing method according to this embodiment.

[0127] exist Figure 6 In the detailed example (3) of the information processing method shown, Figure 4 The information analysis and determination (step S545) in the process of the detailed example (part 1) of the information processing method shown in FIG. Specifically, the process of the detailed example (part 3) includes starting the operation of the pen-type electronic device 1 (step S541), detecting the operation (step S542), acquiring the operation information (step S543), accumulating the operation information (step S544), transmitting the information (step S546), analyzing and determining the information (step S547), determining a change (step S548), processing corresponding to the change determination (step S549), determining whether to continue (step S550), and completing the process (step S551).

[0128] Figure 7 This is a flowchart showing a detailed example (part 2) of the information processing method according to this embodiment.

[0129] exist Figure 7 In the detailed example (4) of the information processing method shown, Figure 4The information analysis and judgment (step S547) in the process of the detailed example (part 1) of the information processing method shown in FIG. Specifically, the process of the detailed example (part 4) includes starting the operation of the pen-type electronic device 1 (step S541), detecting the operation (step S542), acquiring the operation information (step S543), accumulating the operation information (step S544), analyzing and judging the information (step S545), sending the information (step S546), judging a change (step S548), processing corresponding to the judgment of the change (step S549), judging whether to continue (step S550), and completing the process (step S551).

[0130] (Other Configuration Examples of the First Embodiment)

[0131] Figure 8 as well as Figure 9 It is a schematic diagram showing another configuration example of the first embodiment.

[0132] Figure 8 The pen-type electronic device 1 shown in (a) is an example in which a sensor 6 and an information processing unit 4 are provided on a main body 10. Figure 8 In the pen-type electronic device 1 shown in (a), no Figure 1 Even if the display 2 is not provided on the main body 10 of the pen-type electronic device 1, images can be displayed on the display unit of an external device such as the portable terminal 7. Since the pen-type electronic device 1 does not have a display 2, it can achieve weight reduction and cost reduction.

[0133] Figure 8 The pen-shaped electronic device 1 shown in (b) has a power feeder 15 for charging the battery 5. A USB connector or a wireless power feeder can be used as the power feeder 15. A USB connector allows for wired power feeding, while a wireless power feeder allows for contactless power feeding.

[0134] Figure 9 The pen-shaped electronic device 1 shown in the figure is provided with an interface unit 16 for connecting to an external device (eg, a portable terminal 7) and a cable C. Figure 9 The pen-type electronic device 1 shown here performs information input and output with an external device via a cable C. Figure 9 The pen-shaped electronic device 1 shown may or may not include a wireless communication unit. In addition, the interface unit 16 may also serve as the power supply unit 15 for charging the battery 5 .

[0135] Figure 10 This is a schematic diagram illustrating an example of using multiple pen-type electronic devices.

[0136] like Figure 10As shown, when the sensor 6 detects an action other than a writing action, such as the first pen-shaped electronic device 11 and the second pen-shaped electronic device 12 approaching to a predetermined distance or touching each other, characters etc. approach (touch) each other on the software, and a specific action occurs.

[0137] For example, when the first pen-shaped electronic device 11 and the second pen-shaped electronic device 12 touch each other, the acceleration of the contact is detected by the sensor 6, and an image showing that an egg or child is being born, such as a character, is displayed on the software. After the egg or child is born, the egg is hatched by writing actions using the pen-shaped electronic devices 11 and 12, and the image changes, such as the character growing from a child to a larger size.

[0138] Alternatively, an image of a separate character or the like may be displayed on one or both of the first pen-shaped electronic device 11 and the second pen-shaped electronic device 12 in response to the proximity or contact between the two devices.

[0139] Furthermore, when the first pen-type electronic device 11 and the second pen-type electronic device 12 come into proximity or contact with each other, information from one device may be transmitted to the other device.

[0140] Figure 11 This is a flowchart showing an example of processing when two pen-shaped electronic devices are brought into close proximity. The processing shown in this flowchart is program processing of one of the two pen-shaped electronic devices. Furthermore, close proximity also includes contact.

[0141] First, the control unit 41 of one pen-shaped electronic device detects whether another pen-shaped electronic device is approaching (step S8 ). If approaching is detected, the control unit 41 acquires information about the approaching other pen-shaped electronic device (step S9 ).

[0142] Next, the control unit 41 grasps the information such as the mutual roles of the two pen-shaped electronic devices that are close to each other (step S10). Next, the control unit 41 determines the generation of eggs or children based on the information such as the mutual roles (step S11). In the case where no eggs or children are generated in the mutual roles, etc., the process proceeds to step S15. On the other hand, in the case where it is determined that an egg or child is generated based on the mutual roles, etc., the control unit 41 determines whether part of the information is inherited (step S12). Inheriting part of the information means, for example, inheriting part of the personality of the parent's role when an egg is born from the role, etc. or a child is born.

[0143] If part of the information has been inherited, image data of a new character (egg or child) that has inherited the part of the information is created (or selected) and displayed on the display 2 (step S13). On the other hand, if part of the information has not been inherited, image data of a new character (egg or child) that has not inherited the part of the information is created (or selected) and displayed on the display 2 (step S14).

[0144] Next, it is determined whether the pen-type electronic device 1 is to be used continuously (step S15). If it is to be used continuously, the process returns to step S2 and the subsequent processing is repeated. Otherwise, the process is completed (step S16).

[0145] By this process, when the two pen-shaped electronic devices 11 and 12 are brought close together, an image of an egg or child of a born character, or a child character that inherits part of the characteristics or traits of its parent character, is displayed on the display 2 or the display unit of an external device. Thus, the born character egg or child can be nurtured on the software.

[0146] In addition, you can use the characters in the software to play games and fight with other creatures or characters. In addition, you can also use the communication function or the network to fight with creatures or characters raised by others and enjoy fighting games.

[0147] As another example, a connection can be established between the first pen-shaped electronic device 11 and the second pen-shaped electronic device 12 by bringing them into proximity or contact. The established connection information can also be inherited in software, allowing for the sharing of various connection-based information, such as sending messages (e.g., encouraging each other), switching roles, and sharing study time or charts.

[0148] Figure 12 is a schematic diagram illustrating auxiliary equipment.

[0149] like Figure 12 As shown, the pen-type electronic device 1 according to this embodiment can also be used in connection with a dedicated auxiliary device 17 .

[0150] The auxiliary device 17 includes a charging terminal 18 and a display 19. The display 19 may also be a touch panel. For example, the pen-shaped electronic device 1 is inserted into the storage portion 17a of the auxiliary device 17 and connected to the charging terminal 18. This allows charging.

[0151] The display 19 of the auxiliary device 17 displays an image of a character or the like stored in the connected pen-shaped electronic device 1, allowing the user to check the character's growth or play a game. If a game function is provided, the touch panel of the display 19, the sensor (accelerometer) 20, or the keyboard 23 may be used in advance.

[0152] Furthermore, the auxiliary device 17 may include a communication unit 21 for connecting to a network (eg, a wireless LAN network) or an external device, or a portable battery 22 .

[0153] Figure 13 The auxiliary device 17 may also include the following components in addition to the above components: Figure 13 The image processing unit 50, memory 52, control unit 53, power switch 56, charging unit 59, power feeding unit 60, and antenna 61 are shown. The power switch 56 is used to turn the auxiliary device 17 on and off. The power switch 56 turns on when the pen-shaped electronic device 1 is inserted into the storage unit 17a and turns off when it is removed.

[0154] The image processing unit 50 processes the image data read from the memory 52 according to the instruction of the control unit 53 and sends it to the display 19. The charging unit 59 has a line or cable connection terminal for charging the battery 22. Figure 12 The antenna 61 is connected to the charging terminal 18. The antenna 61 is connected to the communication unit 21 to transmit and receive information with external equipment.

[0155] Figure 14 This is a schematic diagram illustrating the configuration of a pen-type electronic device having a power generation function.

[0156] Right now, Figure 14 The pen-shaped electronic device 1 shown includes a power generation unit 71. The power generation unit 71 of the pen-shaped electronic device 1 generates electricity by actions such as writing, carrying, and shaking the main body 10, and charges the battery 5. The battery 5 is charged by the power generated by the power generation unit 71.

[0157] (Second embodiment)

[0158] Figure 15 This is a schematic diagram illustrating an electronic device according to the second embodiment.

[0159] The electronic device according to this embodiment is a grip-type electronic device 81 that is attached to a writing instrument 84 such as a mechanical pencil or a ballpoint pen. The grip-type electronic device 81 includes a body 80 that can be attached to the pen body of the writing instrument 84. The body 80 is provided with a sensor 82 and a memory 83.

[0160] As with the first embodiment, examples of the sensor 82 include an acceleration sensor, a tilt sensor, a rotational track sensor, and a gravity acceleration sensor. When a user uses the writing instrument 84 to take a note, the sensor 82 detects the movement of the body 80 moving along with the writing instrument 84, and obtains the magnitude of the writing movement.

[0161] The value of the writing action is at least one of the frequency, number of times, amount of writing, and character recognition of the writing tool 84. The sensor 82 may also detect the use method of the writing tool 84 and the value based on the action mode.

[0162] The magnitude of the user's note-taking action detected by the sensor 82 may also be stored in the memory 83. Information related to the magnitude of the note-taking action stored in the memory 83 may be transmitted to an external device via a communication unit (transmitting unit) not shown, or may be transmitted to an external device via a cable. In the external device, an image is selected and displayed based on the information related to the magnitude of the transmitted note-taking action. The grip-type electronic device 81 may also have a function of receiving information sent from the outside using a communication unit (receiving unit) not shown.

[0163] With this grip-type electronic device 81, a user can attach it to a note-taking tool 84 and perform a note-taking action. Sensor 82 detects information related to the note-taking action, and displays or changes images corresponding to the magnitude of the note-taking action. For example, a character in a software can be displayed with various changes, such as birth from an egg, growth, strength, aging, personality changes, and evolution, depending on the magnitude of the note-taking action.

[0164] In addition, in this embodiment, similarly to the first embodiment, a plurality of grip-type electronic devices 81 may be brought into close proximity (contact) to generate a specific action.

[0165] (Other Configuration Examples of the Second Embodiment)

[0166] Figures 16 to 18 It is a schematic diagram showing another configuration example of the second embodiment.

[0167] Figure 16 The illustrated handheld electronic device 81 has a configuration in which a display 87 is provided in a main body 80. By providing the display 87 in the main body 80, an image of a character or the like on software can be checked using the handheld electronic device 81.

[0168] That is, based on the magnitude of the note action detected by the sensor 82 of the grip-type electronic device 81, it is possible to confirm on the display 87 of the main body 80 any image of a character being born from an egg, growing, becoming stronger, getting older, having personality changes, property changes, evolving, etc.

[0169] In addition to the sensor 82, the main body 80 may also be equipped with a battery 88, a communication unit 90, and a power feeder 91. The battery 88, such as a storage battery, is the source of power for driving various components. The communication unit 90 is the part (transmitter and receiver) that inputs and outputs information with an external device (e.g., the portable terminal 14). The power feeder 91, which utilizes a USB connector or a wireless power feeder, charges the battery 88 using power drawn from the outside via the power feeder 91.

[0170] Figure 17 The illustrated handheld electronic device 81 includes an interface unit 92 for connecting to an external device (e.g., a portable terminal 14) via a cable C. Thus, the handheld electronic device 81 can input and output information to and from the external device via the cable C. Furthermore, the interface unit 92 can also serve as the power feeder 91 for charging the battery 88.

[0171] Figure 18 (a) is a schematic diagram illustrating auxiliary equipment.

[0172] like Figure 18 As shown in (a) of FIG. 8 , the grip-type electronic device 81 according to this embodiment can also be used in connection with a dedicated auxiliary device 93 .

[0173] The auxiliary device 93 includes a charging terminal 94 and a display 19. The display 19 can also be a touch panel. For example, the handheld electronic device 81, while attached to the writing instrument 84, is inserted into the storage space of the auxiliary device 93 and connected to the charging terminal 94. This allows charging.

[0174] The auxiliary device 93 displays an image of a character or other device stored in the connected grip-type electronic device 81 on the display 19, allowing users to check the character's growth or play games. If the device has a gaming function, it may also utilize the touch panel of the display 19, a sensor (accelerometer) 20, or a keyboard 23. Furthermore, it may include a communication unit 21 or a battery 22 for portability.

[0175] Figure 18 (b) is a schematic diagram illustrating the configuration of a grip-type electronic device having a power generation function.

[0176] That is, Figure 18The grip-type electronic device 81 shown in (b) includes a power generation unit 95. The power generation unit 95 of the grip-type electronic device 81 generates electricity by actions such as writing, carrying, or shaking the main body 80, and charges the battery 88. The battery 88 is charged by the power generated by the power generation unit 95.

[0177] Figure 19 is a schematic diagram illustrating a hat-type electronic device.

[0178] exist Figures 15 to 18 In the example shown, the grip-type electronic device 81 mounted on the writing instrument 84 is described, but it may also be Figure 19 The cap-shaped electronic device 85 is shown attached to a writing instrument 84. The cap-shaped electronic device 85 is inserted into the pen body of the writing instrument 84 when the writing instrument 84 is in use. Furthermore, when the writing instrument 84 is not in use, the front end (the pen tip 10p side) of the writing instrument 84 is inserted into the writing instrument 84 and used as a pen cap.

[0179] (Third embodiment)

[0180] Figure 20 This is a schematic diagram illustrating an electronic device according to a third embodiment.

[0181] The electronic device according to this embodiment is a panel-type electronic device 100 having a detection unit such as a touch panel. The panel-type electronic device 100 includes a panel unit 105 , a control unit 110 , and a memory 120 .

[0182] Panel unit 105 is provided with a display unit that displays images and the like, and a detection unit 106 that detects the position (coordinates) of contact on the display unit. In other words, panel unit 105 is a touch panel. When the tip of a writing instrument 101, such as a stylus pen, is touched on panel unit 105 to write, detection unit 106 detects the handwriting. Control unit 110 identifies the handwriting detected by detection unit 106 and counts the value.

[0183] Memory 120 stores image data and the like. Memory 120 may be provided within the panel-type electronic device 100 or in an external device. If memory 120 is provided in an external device, necessary image data and the like can be transmitted from memory 120 to panel-type electronic device 100 via a communication device (e.g., a network or cable).

[0184] In the panel-type electronic device 100, the control unit 110 detects the note volume value based on the handwriting detected by the detection unit 106. The note volume value based on the handwriting is to calculate the acceleration, direction, rotation trajectory, etc. of the handwriting based on the coordinates detected by the detection unit 106 of the panel unit 105 and its movement, and obtain at least one of the values representing the note frequency, number of notes, note volume and text recognition based on these values. In addition, the method of using the note stationery 101, the value based on the action method, and the pressure during note taking can also be detected based on the detection unit 106. In addition, the content of the written text or picture itself can be recorded, and it can also be analyzed and used.

[0185] The control unit 110 performs control to select image data from the memory 120 according to the handwriting detected by the detection unit 106 and the handwriting amount value, and displays an image based on the selected image data on the display unit of the panel unit 105 .

[0186] Figure 21 This is a flowchart schematically showing the operation of the electronic device according to this embodiment.

[0187] When panel-type electronic device 100, an example of an electronic device according to this embodiment, begins operation (step S101), it first determines whether handwriting (trajectory) is detected by detection unit 106 of panel unit 105 (step S102). If handwriting is detected, trajectory information is obtained based on the trajectory detected by detection unit 106 (step S103). The trajectory information is the amount of writing.

[0188] Next, the control unit 110 selects image data corresponding to the note amount value. In the present embodiment, as an example, the image of the character, etc. on the software is changed (step S104). That is, the control unit 110 selects image data of the character, etc. on the software from the memory 120 corresponding to the note amount value, and displays an image based on the image data on the display unit of the panel unit 105. For example, an image of the character, etc. that grows as the note amount value increases is displayed. In this way, an image display in which the character, etc. grows together with the note action can be obtained. In addition, the control unit 110 can also obtain numerical data corresponding to the note amount value from the memory 42, etc., to graph the numerical data and display it on the display 2.

[0189] Next, it is determined whether the panel-type electronic device 100 is to be continued to be used (step S105). If it is to be continued, the process returns to step S102 and the subsequent processing is repeated. Otherwise, the process is completed (step S106).

[0190] When a note is taken using the note-taking tool 101 on the panel portion 105 through the operation of the panel-type electronic device 100, the detection unit 106 detects information related to the note, and displays or changes images corresponding to the note value. For example, an image display can be made of a character in the software, such as being born from an egg, growing, becoming stronger, aging, changing personality, changing attributes, or evolving, in accordance with the note value.

[0191] Figure 22 Schematic diagram illustrating an example of using a plurality of panel-type electronic devices.

[0192] like Figure 22 As shown, when an action other than a note taking action is performed, such as bringing the first panel-type electronic device 102 and the second panel-type electronic device 103 close to each other within a predetermined distance and touching each other, characters etc. on the software approach (touch) each other and a specific action occurs.

[0193] For example, when the first panel-type electronic device 102 and the second panel-type electronic device 103 are brought into contact, an image of an egg or a child being born, such as a character, is displayed on the software. After the egg or child is born, the image changes such as the egg hatching or the character growing significantly from the child are produced by using the note-taking actions of each panel-type electronic device 102 and 103.

[0194] Alternatively, when the first panel-type electronic device 102 and the second panel-type electronic device 103 come close to or in contact with each other, one or both of them may display an image generated by a single character or the like.

[0195] Furthermore, when the first panel-type electronic device 102 and the second panel-type electronic device 103 come into proximity or contact with each other, information from one device may be transmitted to the other device.

[0196] (Other configuration examples of the fourth embodiment)

[0197] Figure 23 It is a schematic diagram showing another configuration example of the fourth embodiment.

[0198] This panel-type electronic device 152 is a laying-type device that is laid under a notebook or other note paper 151. The panel-type electronic device 152 includes a flat tray 150 that can be used as a laying device, a sensor 155 provided on the tray 150, a memory 156, and a control unit 157.

[0199] When the user takes notes using the writing instrument 153 with the tray 150 laid as a laying device under the writing paper 151 , the sensor 155 detects the writing pressure and the like to detect coordinates corresponding to the handwriting.

[0200] Memory 156 stores image data and the like. Memory 156 may be provided within disk portion 150 of panel-type electronic device 152 or in an external device. If memory 156 is provided in an external device, necessary image data and the like may be transmitted from memory 156 to panel-type electronic device 152 via a communication device (e.g., a network or cable).

[0201] Control unit 157 detects a handwriting volume value based on handwriting detected by sensor 155. This handwriting volume value is obtained by calculating the handwriting's acceleration, direction, rotational trajectory, and pressure during writing based on the coordinates and movement detected by sensor 155, and then obtaining at least one of the following values: frequency of writing, number of times of writing, amount of writing, and text legibility. Sensor 155 can also be used to detect a value based on the method of use and operation of writing tool 153.

[0202] Based on the detected handwriting, the control unit 157 selects image data from the memory 156 in accordance with the handwriting amount. The image data selected by the control unit 157 is transmitted to the external device and displayed on the display unit of the external device.

[0203] Alternatively, the control unit 157 may store a note size value based on the detected handwriting in the memory 156 in advance, and when the panel-type electronic device 152 is connected to an external device (via a network or cable), the note size value stored in the memory 156 may be transmitted to the external device. The external device processes the transmitted note size value, selects image data corresponding to the image data, and displays the image data on the display unit of the external device.

[0204] Using this panel-type electronic device 152, the user places the tray 150 under a piece of note paper 151 to form a laying device, and then uses the note-taking stationery 153 to take notes. When taking notes, sensor 155 detects information related to the note-taking action, and an external device displays or changes images corresponding to the note value. For example, an image display can be displayed of a character in an app, such as being born from an egg, growing, becoming stronger, aging, changing personality, changing attributes, or evolving, all in accordance with the note value.

[0205] Alternatively, the disk 150 may contain only magnetic information and the pen-shaped electronic device 1 may be used as a writing instrument 153. In this case, the magnetic information and other information of the disk 150 may be used to read the positional relationship between the disk 150 and the pen-shaped electronic device 1, thereby obtaining the writing information of the pen-shaped electronic device 1 and performing the various processes and communications described above.

[0206] In addition, in this embodiment, similar to the above embodiment, a specific action can be generated when multiple panel-type electronic devices 152 are brought into close proximity (contact). In addition, although writing is performed using the writing implements 101 and 153 on the panel-type electronic devices 100 and 152, writing can also be performed by copying with fingers.

[0207] Figure 24 Schematic diagrams illustrating examples of electronic devices using different methods.

[0208] Each type of electronic device, such as the pen-shaped electronic device 1, grip-shaped electronic device 81, or panel-shaped electronic device 100, described above, operates independently. Specifically, the software executed in each electronic device has the ability to perform any of the following changes to the character in the software, such as birth from an egg, growth, strength gain, aging, personality change, and evolution, based on information such as usage frequency, number of times used, amount of usage, method of use, and type of action performed.

[0209] Each electronic device can also be connected to server 161 via a network to play competitive games. As the characters in the software on each electronic device grow and become stronger, they contribute to the advancement of the software game. This also facilitates battles with other creatures. Furthermore, this can be beneficial when playing competitive games with creatures or characters cultivated by others over the network.

[0210] Figure 25 as well as Figure 26 This is a schematic diagram illustrating information exchange between electronic devices.

[0211] Figure 25 (a) shows an example of information exchange when electronic devices of the same type (for example, the pen-type electronic device 1) are close to each other. Figure 25 (b) shows an example of information exchange when electronic devices of different types (for example, the pen-type electronic device 1 and the grip-type electronic device 81) are close to each other.

[0212] For example, by bringing two electronic devices close together, specific actions such as contact or proximity of characters on the software screen can be performed, and images such as eggs being laid or babies being born can be displayed on the software screen. This can also help train the following characters.

[0213] Figure 26 This example illustrates an example of electronic devices exchanging information via a network without being in direct physical contact. For example, the pen-shaped electronic device 1 is connected to a server 161 connected to a network via a portable terminal 7, allowing virtual contact with other electronic devices via a communication line. This allows the characters of each electronic device in virtual contact to be displayed on the screen of the portable terminal 7. Figure 26In the example shown, a character 162 on the software of the pen-shaped electronic device 1 and a character 163 on the software of another electronic device virtually connected via a network are displayed on the screen of the portable terminal 7. A screen display of these characters 162 and 163 meeting each other is performed on the screen.

[0214] Thus, the image of an egg or child of a new character born from the two characters 162 and 163 is displayed on the screen of the portable terminal 7. The egg or child of the new character can also be set to inherit the characteristics of the parents.

[0215] (Fifth embodiment)

[0216] Information related to use or action of the pen-type electronic device 1 or the grip-type electronic device 81 or the panel-type electronic device 100 or handwriting (trajectory) information may be assigned points instead of directly developing characters on the software.

[0217] These points can be exchanged for digital information, electronic currency, or points from partner companies. Furthermore, these points can be exchanged for character development within the app, or for housing or furniture related to the character. Furthermore, these points can be exchanged for in-app nutritional supplements or food that aid the character's growth.

[0218] As mentioned above, if we list examples of digital information, they may be music, movies, etc., or games, puzzles, etc.

[0219] Alternatively, a function may be provided that opens a time limit for playing the game based on points.

[0220] Furthermore, the pen-shaped electronic device 1 may have a function of writing a letter or an electronic message and sending image data of a character or the like on the software or points along with the written letter or electronic message.

[0221] Figure 27 This is a flowchart showing the information processing method involved in the fifth embodiment.

[0222] This information processing method is realized by program processing executed by the control unit 41 .

[0223] The pen-type electronic device 1 , the grip-type electronic device 81 , and the panel-type electronic device 100 are connected to the server 161 via a network.

[0224] First, when processing begins (step S201), it is determined whether action / usage is detected (step S202). If action / usage is detected, action / usage information is obtained (step S203). The action / usage information represents the note value. Next, points are obtained based on the obtained action / usage information (step S204). Points are managed for each user by server 161.

[0225] Next, it is determined whether the points have been used (step S205). If used, digital information such as a creature or character, furniture, or video is obtained based on the points used (step S206). This process is, for example, a process for exchanging the points selected by the user for the corresponding consideration based on program processing executed by the server 161.

[0226] Next, it is determined whether to continue using (step S207). If it is continued, the process returns to step S202 and the subsequent processing is repeated. Otherwise, if it is not continued using, the operation is completed (step S208).

[0227] You can also assign characters or points on the software to the purchased electronic device in advance.

[0228] Furthermore, the weights of characters, their own growth levels, and points on these apps can be changed based on time, date, or period. This allows for habits like studying at designated times or completing summer vacation homework quickly.

[0229] Furthermore, by pre-associating electronic devices or characters in software with friends or specific people, users can earn more points and receive benefits by using them on the same day, at the same time, or under specific conditions. This creates a sense of unity and enhances motivation for learning.

[0230] Alternatively, you can make it so that if you solve a puzzle, you accumulate points or characters on the software. Alternatively, you can make it so that if you open a text or do a word or picture exercise, you accumulate points or characters on the software.

[0231] Furthermore, in this case, not only can electronic usage information from electronic devices be sent, but also written words or drawings can be sent by mail or in electronic form. In this case, it is also possible to allow users to accumulate more characters on the software or grant points and discounts if the words or drawings are highly recognizable.

[0232] In addition, these electronic devices and software may have a function of exchanging or transferring characters, etc. Similarly, they may also have a function of exchanging or transferring points obtained.

[0233] Such electronic devices or software have a configuration that allows not only the business owner but also a third party or user to join. For example, as follows.

[0234] (1) A third party or user can register a character or game on the software, thereby allowing the user or third party to enjoy the character or game created on the software, or more people to enjoy the game.

[0235] (2) Granting money or points to third parties or users who create games such as characters on the software.

[0236] (3) Pen-shaped, grip-shaped, or panel-shaped electronic devices can be designed with only the core electronic device, without the outer packaging of the pen, etc. This allows third parties to create pen-shaped or grip-shaped electronic devices with a degree of freedom in appearance.

[0237] (4) A configuration that changes the weight of the integration and the like described above.

[0238] These electronic devices or software may have a configuration that allows access by a third party and imposes security restrictions, etc. For example, as follows.

[0239] Specifically, the pen-shaped, grip-shaped, or floor-standing electronic device incorporates security features or individual identification capabilities, allowing it to operate only when it corresponds to the software. This prevents the pen-shaped electronic device or software from being used alone. Furthermore, it prevents unauthorized use.

[0240] Figure 28 This is a flowchart illustrating the restriction process.

[0241] First, the information built into the electronic device is obtained (step S251), and the built-in information is sent (step S252). Meanwhile, software information is obtained (step S253). Next, the information built into the electronic device is compared with the software information (step S254), and a determination is made as to whether permission is granted (step S255). If permission is denied, the action / use information is denied / not used (step S366). If permission is granted, the action / use information is permitted / used (step S257).

[0242] Alternatively, a third party may register characters or games on the software with the software and review the registration. Alternatively, the business owner may receive a portion of the registration fee when the registration is charged.

[0243] Alternatively, the business owner may have a configuration that allows him or her to verify the characters on the software or the content of the game.

[0244] Furthermore, the electronic device involved in this embodiment can distinguish whether a page is marked, a language other than Japanese, such as English, or highly recognizable characters. For example, if a page is marked, actions such as smearing within a certain area can be recognized. Whether a page is a highly recognizable character is determined by the speed of the action or the uniform spacing of the action. Furthermore, whether a page is in English is determined by short intervals or short-period lateral shifts, which are different from Chinese characters.

[0245] Figure 29 as well as Figure 30This is a flowchart illustrating the determination process of an example action.

[0246] This determination process is realized by program processing executed in the control unit 41 .

[0247] exist Figure 29 In the process of obtaining the action / use information (steps S301 to S302), the action is determined (step S303). In case (a), the information "used in the markup page" is added (step S304). In case (b), the information "used in English" is added (step S305). In case (c), the information "drawing" is added (step S306). Figure 30 In, with Figure 29 Similarly, the action is judged (step S303) starting from the acquisition of action / usage information (steps S301 to S302), and in case (d) it is judged as a "highly recognizable word" (step S308), and in case (e) it is judged as a "sloppy word" (step S309).

[0248] As a discrimination method, AI (Artificial Intelligence), machine learning, deep learning, etc. can be listed. This can more accurately discriminate usage information. Thus, the discrimination result can also be used as learning records or game data.

[0249] There are also cases where these discrimination information is added to the information of the action or use. In this case, the points given to the added information may be changed or the speed of the cultivation may be changed.

[0250] Furthermore, the information acquired by the electronic device according to this embodiment can also be used to determine a learning subject.

[0251] Figure 31 This is a flowchart illustrating the process of determining a learning subject.

[0252] This determination process is realized by program processing executed by the control unit 41 .

[0253] Right now, Figure 31 As shown, first, the action is judged (step S503) starting from the acquisition of action / usage information (steps S501 to S502), and information such as "English learning was carried out" is added in case (a) (step S504), information such as "science learning was carried out" is added in case (b) (step S505), and information such as "mathematics / arithmetic learning was carried out" is added in case (c) (step S506).

[0254] Examples of identification methods include AI, machine learning, and deep learning. This allows for more accurate identification of learning subjects based on the unique note-taking behavior of each subject. Furthermore, the identification results can be used as learning records or game data.

[0255] As described above, the note content type (marked page, English, picture, legible characters, illegible characters, learning subject, etc.) determined based on the note-taking action is the note type. The memory 42 or external device (portable terminal 7 or server 161), serving as the storage unit of the electronic device according to this embodiment, stores the time (note time) divided based on the magnitude of the note-taking action and the note type determined by the control unit 41. The control unit 41 may also perform at least one of graphing and sorting based on the note time and note type stored in the storage unit.

[0256] Figure 32 is a diagram showing an example of a graph display.

[0257] That is, based on the information (note-taking time, etc.) obtained by the electronic device involved in this embodiment (for example, the pen-type electronic device 1) or the type of note identified (information on learning subjects, etc.), at least one of the learning time or learning subjects, the time of concentration, the time of writing highly recognizable characters, etc. is graphed. Figure 32 In the example, a chart is displayed on the display 8 of the portable terminal 7. The chart can be a learning record for each time, or a learning record for each day, month, etc. In addition, learning and the like can be input manually without using the electronic device involved in this embodiment.

[0258] Furthermore, the text itself can be identified using information acquired by the electronic device according to this embodiment.

[0259] Figure 33 This is a schematic diagram showing an example of handwriting recognition.

[0260] For example, if the character "あ" is written on paper 400 on an electronic device (e.g., pen-shaped electronic device 1) according to this embodiment, the pen itself, or the electronic device that transmits it (e.g., portable terminal 7), or a computer in the cloud can recognize it as "あ." This recognition can be performed based on a specific rule base, AI, machine learning, deep learning, etc. Furthermore, not only can the character be recognized, but the handwriting can also be fully reproduced.

[0261] Information acquired from the electronic device (e.g., the pen-shaped electronic device 1) according to this embodiment, information obtained by analyzing the information, and information manually input as described above, can be shared among various devices via wireless communication (network). For example, sharing can be done with pre-registered friends, parents, etc.

[0262] Figure 34 is a schematic diagram showing an example of information sharing.

[0263] For example, information acquired by the electronic device involved in this embodiment (e.g., pen-shaped electronic device 1) is transmitted to the portable terminal 7 via wireless communication, and is then shared from the portable terminal 7 to the portable terminal 14 of a friend or parent via the network base station 600. Shared information includes information (messages, guidance, etc.) notified on the Web by terminals such as smartphones, tablet computers, and game consoles, or from the electronic device involved in this embodiment, the pen-shaped device, and the like.

[0264] Sorting or ranking is performed based on information acquired from the electronic device (for example, the pen-type electronic device 1 ) according to the present embodiment, information obtained by analyzing the information, and information manually input as described above.

[0265] Figure 35 is a diagram showing an example of a ranking display.

[0266] In this example, the ranking is displayed on the display 8 of the mobile terminal 7. The ranking display may be a ranking with friends obtained via the network, a ranking with people all over the country, or a ranking with one's own past records.

[0267] Information acquired from the electronic device (for example, the pen-type electronic device 1 ) according to the present embodiment, information obtained by analyzing the information, and information manually input as described above can be collected via a network.

[0268] Figure 36 is a schematic diagram showing an example of information aggregation.

[0269] Information acquired by the electronic device according to this embodiment (e.g., pen-shaped electronic device 1) is transmitted to portable terminal 7 via wireless communication, and then transmitted from portable terminal 7 to specific terminal 500 via network base station 600. The information transmitted to specific terminal 500 can be collected and confirmed as statistical information.

[0270] The information thus collected can be checked, for example, by the cram school or the parent's terminal 500. The cram school or the parent may not have the electronic device according to this embodiment, but can refer to the information by pre-registering the electronic device or the user of the electronic device.

[0271] Figure 37 This is a flowchart showing an example of information confirmation processing.

[0272] First, the process begins (step S511), detecting the electronic device (e.g., pen-type electronic device 1) or user involved in this embodiment (step S512). If detected, the pen or user is registered (step S513). Next, it is determined whether additional registrations exist (step S514). If so, the process returns to step S512 and repeats the subsequent process. If no additional registrations exist, the registered pen or user information is confirmed (step S515).

[0273] In addition, people connected via the network can also send feedback to each other. For example, a friend, a parent, a cram school, etc. presses a button indicating any intention display. By pressing the button indicating the intention display, the user of the electronic device involved in this embodiment is informed of the situation. It is also possible to obtain benefits in the training game through the intention display. For example, it is also possible to obtain items when the number of intention displays obtained exceeds a specified value, or to defeat the enemy together with the user who obtained the intention display in the game and the user who generated the intention display.

[0274] Furthermore, the development of characters and the like on these software programs or the games using them may include a free part, a paid part, or both.

[0275] In the case where both the free part and the paid part are mixed, it is possible to obtain a benefit if charging (paying money) is performed.

[0276] If we list an example of the benefits, there are the following benefits.

[0277] (1) It can speed up the hatching of eggs.

[0278] (2) Capable of hatching multiple eggs.

[0279] (3) Accelerate the development of roles, etc.

[0280] (4) Cultivate multiple eggs or characters, etc.

[0281] (5)Accumulate points.

[0282] (6) The proportion of accumulated points increases.

[0283] By sending this information to external parties via a network or other means, characters you create on the software can be transformed into physical objects such as toys or stationery, allowing you to take them to hand. An example of a toy is a plush toy, and a beloved character created on the software can also be taken to hand as a physical object rather than software. Another example of stationery is a pencil case or pen with the character printed on it. Other products include any type of product, from daily necessities to rides like bicycles.

[0284] Figure 38This is a flowchart illustrating the exchange process for actual objects.

[0285] Once the process is started (step S311), a character on the software is selected (step S312), and then a selection of a real product is made (step S313). Next, payment is made (step S314).

[0286] Next, determine whether the selected product already exists (step S315). If it is an existing product, then the product is delivered (step S317). If it is not an existing product, then create the product (step S316) and deliver it (step S317).

[0287] Furthermore, the software could impose some kind of limit on the number of eggs that can be hatched, the number of characters that can be raised simultaneously, etc. This could encourage faster hatching and raising of creatures. Furthermore, if the upper limit could be raised by charging a fee, this could encourage charging fees.

[0288] Furthermore, the number of characters that can be simultaneously cultivated can be adjusted based on a point system, such as how characters hatch from eggs, grow, become stronger, age, change their personalities, change their attributes, and evolve. Information such as the frequency, number of uses, amount of use, method of use, and trajectory of pen-type or grip-type electronic devices, touchscreen displays, and pressure-sensitive devices can be converted into points to determine the appropriate style of use. For example, points can be used to hatch eggs, or to improve characters. Furthermore, enabling the purchase of points for a fee can enhance the promotion of subscriptions.

[0289] On a display provided in a pen-type or grip-type electronic device, or on a display of an auxiliary device or a portable terminal (smartphone, tablet, etc.), the appearance of changes in characters etc. on the software can be viewed.

[0290] Figure 39 This is a schematic diagram showing an example of displaying growth patterns. In this example, the display is displayed on the display 2 of the pen-shaped electronic device 1, but the same applies to displays of other electronic devices. Here, the display 2 sequentially displays changes such as egg hatching, character growth, and evolution at predetermined intervals. Alternatively, the display can be sequentially displayed by pressing a predetermined button.

[0291] Furthermore, electronic devices or the like may be equipped with a function that allows the characters on the software to be saved and the appearance of the assembly to be confirmed. Figure 40 2 is a schematic diagram showing an example of a collection display. In this example, images of characters and the like stored are displayed in a list on the display 211 of the portable terminal 204.

[0292] Characters, eggs, and points earned through growth in the app can also be accumulated and sold as physical items, such as pencils and mechanical pencil leads, exercise books, and textbooks. Furthermore, this can be combined with not only stationery and books but also food and daily necessities. When selling physical items, the app can simply reflect the item's lucky bag, password, or barcode by reading it into a pen-shaped electronic device, handheld electronic device, smartphone, tablet, or dedicated terminal.

[0293] In addition, points can be awarded to the character etc. when the character answers the question and inputs the answer to the question and obtains the correct answer.

[0294] The pen-shaped electronic device 1 may not have a separate power switch, but may have a function of turning on the power by striking the knocking portion 3, such as by extending or retracting the pen core, or by shaking the pen-shaped electronic device 1 itself. Alternatively, the pen-shaped electronic device 1 may have a function of automatically turning off the power if there is no change in acceleration, striking, or clamping for a certain period of time.

[0295] Figure 41 (a) and (b) are schematic diagrams illustrating the distinction between note-taking actions.

[0296] In the electronic device according to this embodiment, the control unit 41 distinguishes between a note-taking action and an action not taking a note. As an example, the control unit 41 distinguishes between the note-taking actions as follows.

[0297] First, if Figure 41 As shown in (a) of FIG. 1 , the control unit 41 detects a certain plane SF based on the detection value of the sensor 6. For example, the plane SF is detected based on the principle that three points (P1, P2, and P3) of the three-dimensional coordinates read by the sensor 6 or three or more points connected to form a plane.

[0298] Therefore, when the coordinates of the pen tip 10p are located on the plane SF according to the detection value of the sensor 6, it is distinguished as a writing action. Figure 41 As shown in (b), even when the coordinates of the pen tip 10p are within a certain distance D from the plane SF, it is distinguished as a writing motion.

[0299] On the other hand, if the pen tip 10p moves in a three-dimensional direction with its coordinates different from the detected plane SF and moves away from the detected plane SF by a certain distance D, the movement is considered as a writing movement. The specific three-dimensional distance D is, for example, 10 cm.

[0300] Furthermore, if the same action at a certain period, direction, and distance reaches a long distance or a long time, it can be determined not to be treated as a note-taking action. For example, a long distance is 10 cm, and a long time is 5 seconds.

[0301] For example, if you're not writing text but are just drawing a meaningless circle intermittently, it won't be considered a note-taking action that can help you earn points and develop your character. This is determined by whether the change in acceleration persists for at least a certain period of time. Furthermore, if the acceleration vector points in a certain direction, or if an edge moves up and down repeatedly, and the change in the acceleration vector continues at a predetermined interval, these situations are also not considered note-taking actions.

[0302] When multiple people use the same electronic device and each person earns points for a different role on the software, the following procedure is performed, for example.

[0303] (1) Figure 42 This is a schematic diagram illustrating the classification of users of electronic devices. Figure 42 The pen-shaped electronic device 301 shown is provided with a recognition tapping unit 302. For example, as in a multi-functional ballpoint pen (multi-color ballpoint pen), multiple parts to be tapped by the pen butt or a part of the rotating pen are provided to distinguish users (persons / individuals).

[0304] (2) When writing or drawing, the action preferences are distinguished, and users (persons / individuals) are distinguished based on the writing preferences. Here, "preferences" refer to the speed of pen movement, the intervals between movements, the period, the handwriting, etc. when writing or drawing, all of which are information with a unique individual changing pattern. Based on the values related to the movements detected by the sensor 6, the control unit 41 calculates information corresponding to the above-mentioned "preferences" and performs similarity determination or pattern matching with the information of the "preferences" of pre-registered individuals. In this way, users (persons / individuals) are distinguished.

[0305] Furthermore, pen-type, grip-type, or panel-type electronic devices have the function of controlling software operations by detecting specific movements or specific trajectories. For example, the lock is unlocked only when a specific movement is made, so that information can be stored or sent.

[0306] In addition, by tracing a specific pattern (code), etc., the characters on the software can be acquired, cultivated, grown, or points can be obtained.

[0307] Alternatively, a specific action (for example, shaking the electronic device in a pre-registered manner) may be performed to identify the user (person / individual).

[0308] Figure 43This is a schematic diagram illustrating the correspondence between a pen-shaped electronic device and text information.

[0309] Figure 44 This is a flowchart illustrating the correspondence between a pen-shaped electronic device and text information.

[0310] By using the pen-type electronic device 1 according to the present embodiment, electronic information (text data) of characters or symbols can be automatically obtained based on the writing action detected by the sensor 6 .

[0311] Specifically, when writing characters or marks using the pen-shaped electronic device 1, motion information detected by the sensor 6 is accumulated. This information is then associated with text data, and when writing is performed using the pen-shaped electronic device 1, the text data of the characters or marks corresponding to the writing action is detected and converted into the text data.

[0312] In order to facilitate the correspondence between note taking actions and text data, such as Figure 43 As shown, a start mark 351 and an end mark 352 are input in advance on a piece of paper or the like, and characters for tracking are set from the top. The user starts detection by tracking the start mark 351 on the pen-shaped electronic device 1.

[0313] After marking the tracking start mark 351, the user sequentially tracks the characters 353 pre-set on the pen-shaped electronic device 1. For example, assume that the character "あ" is set as the first character and is tracked. At this point, the detection value of sensor 6 is associated with the text data of "あ." Similarly, by pre-tracking multiple characters, information establishing the association between the detection value of sensor 6 and the text data is accumulated. When the user marks the tracking completion mark 352, data accumulation is complete.

[0314] Along Figure 44 Explain the data storage and character recognition processing.

[0315] This processing is realized according to program processing executed by the control unit.

[0316] First, the process starts (step S351), and when the start mark is detected (step S352), the action / use information of the note action is acquired by the sensor 6 (step S353). This process is repeated until the end mark is detected (step S354).

[0317] After the completion mark is detected, the action / usage information of the note action acquired by the sensor 6 is transmitted to the server 161 or the like (step S355). Next, a correspondence is established between the transmitted action / usage information of the note action and the text information (text data) to be noted, which is sandwiched between the start mark and the completion mark (steps S356 and S357), and the associated information is accumulated (step S358).

[0318] By accumulating information associating the action / usage information of the writing action with the text data, it is possible to identify the text data of characters based on the information of the sensor 6 when the writing action is performed using the pen-shaped electronic device 1 (step S359).

[0319] This method of utilization enables the pen-shaped electronic device 1 of this embodiment to have an OCR (Optical Character Recognition) function. For example, by using a text booklet such as a Chinese character exercise book, information is stored that establishes a correspondence between action information (detection values from the sensor 6) generated by writing text using the pen-shaped electronic device 1 and previously known text data. This correspondence information is then used as a database, and text data corresponding to writing actions using the pen-shaped electronic device 1 can be obtained.

[0320] Regarding software using the electronic device (eg, the pen-shaped electronic device 1 ) according to this embodiment, a development game using the software can benefit from connections with other electronic devices or others (eg, registration on the software).

[0321] (1) If learning is done at the same time or within an hour before or after, the virtual creatures will be cultivated as quickly as possible, which will be beneficial for fighting against virtual enemies.

[0322] (2) If you have registered in advance, you will receive assistance in battling virtual enemies on the software.

[0323] (3) Points or items can be earned regularly based on the presence or absence of registrations, the number of registrants, etc.

[0324] The software using the electronic device (for example, the pen-type electronic device 1 ) according to the present embodiment can have a function of enabling a training game using the software to be activated in a specific environment.

[0325] For example, when it is determined that the study notebook is identified using the camera of the smartphone, a game using virtual reality (AR: Augmented Reality) can be played on the notebook.

[0326] Alternatively, the pen-type electronic device 1, smart monitor, etc. may be read to play games related to the electronic device.

[0327] Regarding the information obtained by the electronic device involved in this embodiment (for example, the pen-type electronic device 1), there is also a case where its acquisition cycle is not always the same cycle. That is, in the control unit, when obtaining note information including note frequency, note number, note amount, text recognition, note speed, note acceleration, note angle (pen angle), note angle, and at least any one of the pressure during note taking, a plurality of acquisition modes with different sampling frequencies can also be switched. For example, it is also possible to switch between a first acquisition mode that obtains note information with a first sampling number every certain time and a second acquisition mode that obtains note information with a second sampling number less than the first sampling number every certain time. In addition, in the case of switching more than three acquisition modes, the first acquisition mode, the second acquisition mode, ..., the Nth acquisition mode (N natural number) are prepared in advance and switched.

[0328] As a result, a period in which information is acquired at a high frequency and a period in which information is acquired at a low frequency are mixed.

[0329] Figure 45 This is a schematic diagram showing an example of switching between acquisition modes.

[0330] Figure 45 The horizontal axis represents time, and one vertical line represents a timing of obtaining note information.

[0331] When acquiring information at a high frequency (first acquisition mode M1), a sampling frequency of approximately 50 Hz to 5000 Hz is envisioned. On the other hand, when acquiring information at a low frequency (second acquisition mode M2), a sampling frequency of approximately 0.01 Hz to 50 Hz is envisioned. This high frequency (first acquisition mode M1) and low frequency (second acquisition mode M2) switching can be periodic or random.

[0332] Furthermore, the acquisition of information itself may be performed by accumulating information at regular intervals or by changing the frequency when transmitting data.

[0333] Furthermore, the type of data acquired may be changed at each time. For example, at one time, all acceleration data for the X, Y, and Z axes may be acquired, and at another time, only acceleration data for the X and Y axes may be acquired.

[0334] These periodic changes are considered to be used separately, for example, to discriminate learning subjects or characters during a high-frequency period (first acquisition pattern M1 ) and to measure learning time during a low-frequency period (second acquisition pattern M2 ).

[0335] This makes it possible to achieve both acquisition of information required for processing and reduction of the data amount.

[0336] The software for utilizing the electronic device involved in this embodiment (for example, the pen-type electronic device 1) can analyze the accumulated data on the learning frequency or learning amount, learning speed, attention, etc. for each learning subject, and propose better learning methods based on the analyzed data.

[0337] For example, if there is less time to learn the mother tongue, then promote mother tongue learning; if attention is lost, then promote rest, etc.

[0338] In addition, this behavior can be input by parents or cram schools, or it can be automatically determined by AI.

[0339] Learning records can be linked to performance indicators. This can include the results of regular lesson input by the student, their guardian, or a cram school. Furthermore, it can be linked to the correct answers to each question. In this case, information entered by the student, their guardian, or the cram school, as well as information such as circle marks and cross marks, can be read. Marked pages can also be read.

[0340] Figure 46 : is a flowchart showing an example of correct answer determination processing.

[0341] First, the process starts (step S521), the motion information read by the sensor 6 is accumulated (step S522), and the motion information is analyzed (step S523).

[0342] Next, the correct answer information is read from the database (step S524), and the information obtained based on the analysis result of the action information is compared with the correct answer information to determine whether it is the correct answer (step S525).

[0343] If the answer is correct, a specified value is provided (step S526), and the process proceeds to the next step (step S527). For example, if the answer is correct, the training speed is accelerated, and more items or points are accumulated. On the other hand, if the answer is not correct, a specified value, no value, or other value is provided (step S528), and the process proceeds to the next step (step S529). This process can link learning records with performance reminders.

[0344] The software for the electronic device (e.g., pen-shaped electronic device 1) according to this embodiment can be equipped with functions for recognizing text recognition, speed, angle, and pen grip. Furthermore, it can store information about people with good text recognition and beautiful handwriting, analyze this data using machine learning, and compare it with other data to make a distinction.

[0345] Regarding the software for utilizing the electronic device involved in this embodiment (for example, the pen-type electronic device 1), it has a goal setting function that uses information obtained from the electronic device (note-taking time, study time, etc.), or determines the type of note (marked page, language, study subject, text recognition, ranking, etc.) or graphical information to perform reverse calculations based on the career one wants to work in or the university one wants to attend.

[0346] For example, the amount of study Tokyo university students put in during their high school years can be measured and used as a goal, allowing high school students to set daily study targets. Furthermore, for those aiming to become programmers, in addition to essential information on math and arithmetic, programming textbooks are provided. Indicators for achieving these goals can be sent online or set by the user.

[0347] The software for the electronic device (eg, the pen-type electronic device 1 ) according to this embodiment may have a function of associating a development game with a learning log or the contents of the learning log.

[0348] (1) The virtual creatures on the software are given professions and taught subjects related to their professions to strengthen the virtual creatures.

[0349] (2) The virtual creatures in the software are given occupations. If the creatures do not study subjects related to their occupations, the creatures will not be born.

[0350] (3) The virtual creatures on the software cannot learn and cannot operate in the virtual places where they can operate on the software after being released for a specific period of time.

[0351] (4) Virtual creatures on the software can run in virtual places on the software but cannot run without learning specific subjects.

[0352] (5) The virtual creatures that can be obtained on the software vary depending on the subjects studied.

[0353] Software used in the electronic device (for example, the pen-shaped electronic device 1 ) according to this embodiment has a function that disables or restricts the use of the software after a certain time, a certain period, or completion of a certain process.

[0354] Figure 47 is a flowchart showing an example of a limited function process.

[0355] First, the process begins (step S531), allowing the use of the software involved in this embodiment (step S532). Next, the process determines whether restrictions have been introduced (step S533), and determines whether restrictions have been introduced. If there are no restrictions, the process returns to step S532 and the use of the software involved in this embodiment continues. On the other hand, if there are restrictions, the process proceeds to step S534, where restrictions on the use of the software are imposed.

[0356] Furthermore, when the software according to this embodiment is used until a certain time or a certain period / a certain process is completed, the use of the smartphone or the game console itself can be restricted. For example, the restriction may require the input of a password.

[0357] A pulse monitoring element may be mounted on the grip portion of the electronic device (for example, the pen-type electronic device 1 ) according to this embodiment to measure the pulse.

[0358] That is, the control unit acquires biological information including at least one of the user's blood flow and pulse in addition to the written information. This makes it possible to measure the user's attention using at least one of the written information and the biological information.

[0359] The electronic device involved in this embodiment (for example, the pen-type electronic device 1) can also be used to obtain pen pressure. In this way, it is possible to accurately determine whether a word is written, and also accurately know when the writing starts and ends. In addition, it is also possible to obtain information about the strength of the pen pressure, which can help provide feedback on the text recognition or a better writing method. As a unit for obtaining pen pressure, for example, a pressure sensor or a deformation sensor pre-configured at the part close to the pen tip 10p can be cited as the sensor 6.

[0360] The grip strength can also be acquired using the electronic device according to this embodiment (eg, the pen-shaped electronic device 1). To acquire the grip strength, a pressure sensor can be provided at the grip portion.

[0361] The electronic device involved in this embodiment (for example, the pen-shaped electronic device 1) can also be used to determine the user's attention. In addition, the determined attention can be fed back to the user, parents, cram school, etc. Based on this information, learning can be improved. The determination of attention can be done by analyzing the movement of the pen, or by using information such as the pulse, pen pressure, and grip strength as described above.

[0362] When a so-called pen spinning action is performed using the electronic device according to this embodiment (for example, the pen-shaped electronic device 1 ), the function of reading the action, identifying the technique name, and scoring may be provided.

[0363] Figure 48 This is a diagram showing an example of scoring for the pen spinning action.

[0364] For example, by utilizing an accelerometer or gravity acceleration sensor as the sensor 6 of the pen-shaped electronic device 1, it is possible to detect how many times the pen has been moved in which direction and at what angle. The pen-shaped electronic device 1 or the portable terminal 7 has the function of analyzing this information, identifying the skill name, and scoring the skill proficiency.

[0365] As an alternative to the electronic device described in this embodiment, a smart monitor, a ring-type device, or a strap-on device can be used. In this case, while the accuracy is lower than when using a pen, existing electronic devices can be utilized. Furthermore, a wrist-wrap device can capture information about the movements of the hand's muscles and tendons for motion detection. Furthermore, a wrist-wrap device can also be equipped with a sensor for detecting blood flow or pulse.

[0366] The electronic device according to this embodiment (for example, the pen-type electronic device 1 ) may have a built-in speaker.

[0367] Figure 49 is a schematic diagram showing an example of an electronic device having a built-in speaker.

[0368] For example, the main body 10 of the pen-shaped electronic device 1 has a built-in speaker 9. Thus, sounds, music, and the like can be output from the speaker 9. The built-in speaker 9 allows the use of the pen-shaped electronic device 1 or other electronic device according to this embodiment to produce sound effects, music to enhance learning, or voice calls from characters.

[0369] The electronic device according to this embodiment (for example, the pen-type electronic device 1 ) may also have a function of combining still images and moving images transmitted from a server via a network with data acquired by the electronic device. This function may be integrated into a server connected to the network.

[0370] Figure 50 This is a flowchart showing an example of the process of associating a dynamic image with a pen movement.

[0371] First, the transmission of the moving image begins (step S561), and the position and time information of the moving image are obtained (step S562). On the other hand, if the operation of the pen-shaped electronic device 1 as an example of an electronic device begins (step S563), the operation, usage status, and time information of the pen are obtained (step S564).

[0372] Next, the timing of the dynamic image obtained in step S562 and the timing of the pen movement obtained in step S564 are associated with each other to link the dynamic image and the pen movement information (step S565).

[0373] Next, the pen movement is analyzed (character type, recognition, etc.) (step S566). The analysis results can be used to improve the content, and are fed back to the business owner who sent the dynamic image (step S567), or used as learning information for the user of the electronic device (step S568), or other feedback is provided.

[0374] This processing allows the relationship between the dynamic image and the pen's movements to be understood, such as where in the dynamic image the pen moves and writes text. This allows, for example, information on which parts of the dynamic image are being focused on and which parts are being missed. This information can also be used by educational companies, for example, to improve their dynamic images.

[0375] Furthermore, in this embodiment, since it is possible to establish a correspondence between pen movements and dynamic images, for example, by recognizing gestures from pen movements, the dynamic images can be controlled based on the recognized gestures. For example, if the pen is determined to be inactive for a certain period of time, the dynamic image can be automatically stopped. Furthermore, the dynamic image can be controlled to temporarily stop, fast forward, or rewind actions such as by using specific pen gestures.

[0376] The electronic device involved in this embodiment (e.g., pen-shaped electronic device 1) can be used to prevent dementia caused by rehabilitation. For example, it detects pen movements and, based on the analyzed results, applies vibration to the main body 10, outputs sound, light, images, and other signals to provide feedback to the user. Furthermore, by transmitting information such as the degree of writing and hand movements of the user of the pen-shaped electronic device 1 to terminals owned by nurses, caregivers, and other attendants, it is possible to understand the relationship between pen movements and the status of rehabilitation or care.

[0377] The electronic device (e.g., pen-shaped electronic device 1) of this embodiment can be used in situations where personal identification is required, such as when using a credit card. In such situations, the electronic device (e.g., pen-shaped electronic device 1) of this embodiment can detect the pen's movements during signature processing. This improves security, such as preventing signatures from being signed on behalf of others.

[0378] Figure 51 : is a flowchart showing an example of signature identification processing.

[0379] First, a pre-signature is obtained (step S571). This process allows the user to write a signature using the electronic device (eg, pen-shaped electronic device 1) according to this embodiment because the correct signature is registered in advance.

[0380] Next, the signature is analyzed (step S572). Specifically, information about the user's pen movements when writing their signature is detected. This pen movement information includes at least one of character legibility, writing speed, writing acceleration, writing angle (pen angle), and writing pressure. This writing information is considered unique to the user. The analysis results are then stored (step S573).

[0381] Next, a signature for user confirmation is obtained (step S574). That is, when using a credit card, for example, the user writes a signature using the electronic device involved in this embodiment (for example, the pen-shaped electronic device 1) for user confirmation.

[0382] Next, the signature is analyzed (step S575). This process detects the pen movement information used to confirm the signature. The pen movement information is the same as the note information stored in step S573.

[0383] Next, it is determined whether the signatures match (step S576). That is, the degree of match between the correct signature note information stored in step S573 and the signature note information analyzed for identity verification in step S575 is determined.

[0384] Therefore, when the degree of consistency exceeds the specified value, the signatures are considered consistent and the authentication is OK (passed) (step S577). When the degree of consistency is below the specified value, the signatures are considered inconsistent and the authentication is NG (failed) (step S578).

[0385] In the signature discrimination process using the electronic device involved in this embodiment (for example, the pen-type electronic device 1), because the note information of the pen action is used (including at least any one of the note direction, note speed, note acceleration, text recognition, note angle (pen angle), and note pressure), the handwriting comparison discrimination becomes higher. For example, this is because it is believed that the signature of another person is imitated, so the drawing speed is slower than the person's own. In addition, it is also believed that the pen angle and the pressure when writing are different from those of the person himself. In this embodiment, even if the shape of the drawing is the same as the person's signature, such note information will be combined to determine whether it is the person's signature. Therefore, compared with simple handwriting comparison, it can be distinguished with high precision.

[0386] In addition, the process of determining consistency based on the note information (step S576) may also be performed by AI, machine learning, deep learning, or the like.

[0387] Here, an example of the specific size of the electronic device according to this embodiment is shown.

[0388] Since the pen-type electronic device 1 and the grip-type electronic device 81 are designed for handheld use, the following size and weight are preferable. For example, a length of 50 cm or less and a weight of 1 kg or less are preferable.

[0389] Furthermore, in the pen-shaped electronic device 1, the sensor 6 for sensing motion is preferably located closer to the tip (toward the pen tip 10p) than the center of the body 10, or closer to the rear end (opposite the pen tip 10p) of the body 10, to facilitate sensing motion of the pen tip 10p. For example, the sensor 6 is located within 5 cm of the pen tip 10p of the body 10, or within 5 cm of the rear end of the body 10.

[0390] As described above, by utilizing the electronic device involved in this embodiment, it is possible to display images of any changes of characters on the software, such as hatching from eggs, growing, becoming stronger, getting older, changing personality, changing properties, and evolving, corresponding to the value of the note action.

[0391] Furthermore, software, pen-shaped, or grip-type electronic devices can determine the legibility of text and the beauty of images, and use this information to influence the character's development, strength, and style. For example, if a character draws a lot of beautiful images, they become a beautiful creature. If a character writes a lot of legible text, they develop a good personality.

[0392] Furthermore, the characters hatched from the eggs may be influenced, for example, by drawing a picture to produce a character of a certain type or style, or by writing a letter to produce a character of a different type or style.

[0393] In addition, the image displayed on the display 2 is not only a living thing or a character, but also includes a vehicle or aircraft or other ride, a robot or machinery or other non-living things, plants, etc. In addition, it can also be a real person or a historical person.

[0394] According to the electronic device according to the embodiment described above, the following effects can be obtained.

[0395] (1) By using a grip-type electronic device 81 and a cap-type electronic device 85 on a writing implement such as a pen-type electronic device 1 or a pencil, a mechanical pencil, or a ballpoint pen, it is possible to use a sensor to detect any action such as whether the writing implement is being used, the frequency of use, the number of times the writing implement is being used, the amount of writing implement being used, and the method of writing implement being used. Thus, based on the information read by the sensor, a character on the software, etc. can be grown. The growth of the character, etc. can be confirmed through a display, a smartphone, a game console, a dedicated terminal, etc., which is mounted on the pen. In addition, a game using the character, etc. can be played.

[0396] (2) By bringing pens with the same function close to or in contact with each other, eggs can be hatched, babies can be born, or the next character can be raised.

[0397] (3) Information exchange between the pen and a dedicated terminal or smartphone can be performed through wired or wireless communication.

[0398] (4) Through the above (1) to (3), it is possible to be inspired to use the pen, and further to study or draw. In addition, because the degree of growth, growth process, personality, and nature of the character etc. change according to the method used, it is possible to be inspired to study seriously and write highly recognizable characters.

[0399] In addition, the above description describes the present embodiment and its application examples (variations, specific examples), but the present invention is not limited to these examples. For example, in the present embodiment, an image is displayed on the display 2, but a voice or sound may be output instead of the image display or together with the image display. In addition, as an image displayed corresponding to the value of the note action, in addition to characters, etc., text information or numerical information may also be displayed. For example, in the case of repeatedly noting the same Chinese character, it is possible to identify the situation in which the same Chinese character is notated according to the action of the electronic device, and the number of times the Chinese character is notated is displayed on the display 2, and the number of times is output as sound. In addition, for each of the aforementioned embodiments or their application examples (variations, specific examples), those skilled in the art can appropriately add, delete, change the design of the constituent elements, or appropriately combine the features of each embodiment, and as long as they have the gist of the present invention, they are included in the scope of the present invention.

[0400] Industrial Applicability

[0401] In addition to writing utensils, the present invention can be preferably applied to objects that are used by hand, such as toothbrushes, spoons, forks, and chopsticks. In addition, in situations where habituation is required, such as alarm clocks and watches, the same system as the present invention can also be applied. In addition, the software described in the embodiment can be executed by a computer, recorded in a recording medium, and sent via a network. In addition, the software and application programs described above can be implemented without using electronic devices such as the pen-type electronic device 1 or the panel-type electronic device 100. For example, instead of inputting information using the acceleration of the pen-type electronic device 1, the data typed using a personal computer can be treated as text information, and the information entered using a touch pen on a tablet terminal can be treated as text and picture information. In this way, learning using a personal computer or a tablet can also achieve the same effect as in the present embodiment, which can enhance the promotion of learning.

[0402] [Description of Reference Numerals]

[0403] 1…Pen-shaped electronic device; 2…Display; 3…Tap unit; 4…Information processing unit; 5…Battery; 6…Sensor; 7…Portable terminal; 8…Display; 9…Speaker; 10…Main unit; 10p…Pen tip; 11…First pen-shaped electronic device; 12…Second pen-shaped electronic device; 14…Portable terminal; 15…Power feeding unit; 16…Interface unit; 17…Auxiliary device; 17a…Storage unit; 18…Charging terminal; 19…Display; 20…Sensor; 21…Communication unit; 22… Battery; 23…Keyboard; 41…Control unit; 42…Memory; 43…Communication unit; 50…Image processing unit; 52…Memory; 53…Control unit; 56…Power switch; 59…Charging unit; 60…Power feeding unit; 61…Antenna; 71…Power generation unit; 80…Main unit; 81…Grip-type electronic device; 82…Sensor; 83…Memory; 84…Notebook; 85…Hat-type electronic device; 87…Display; 88…Battery; 90…Communication unit; 91…Power feeding unit; 92…Connector Mouth; 93… auxiliary equipment; 94… charging terminal; 95… power generation unit; 100… panel-type electronic device; 101… writing instrument; 102… first panel-type electronic device; 103… second panel-type electronic device; 105… panel unit; 106… detection unit; 110… control unit; 120… memory; 150… disk unit; 151… note paper; 152… panel-type electronic device; 153… writing instrument; 155… sensor; 156… memory; 157… control unit ; 161…server; 162…role; 163…role; 201…image processing unit; 204…portable terminal; 205…power switch; 208…charging unit; 209…antenna; 211…display; 301…pen-type electronic device; 302…tapping unit for identification; 351…start mark; 352…completion mark; 353…text; 400…paper; 500…terminal; 600…base station; C…cable; D…distance; SF…plane; P1, P2, P3…point.

Claims

1. An information processing method for an electronic device including a sensor and a control unit, wherein: The sensor detects a movement of a user using the writing instrument or a movement of the writing instrument, and the control unit performs control to change an image displayed on the display unit based on information detected by the sensor. The control unit includes: A step of distinguishing the time of an action not treated as a note action including text from the time when the acceleration detected by the sensor exceeds a certain value and this state continues for a certain time, and counting the time to obtain the magnitude of the note action; and The step of displaying images or graphs of the character's birth from an egg, growth, strengthening, life extension, personality change, property change, and evolution on the display unit corresponding to the value, The step of obtaining the magnitude of the writing action includes the following processing: according to the value detected by the sensor, in any of the following cases, it is determined that the writing action is not the text, when the value exceeds a certain three-dimensional space within a time period of a preset three-dimensional coordinate of the pen tip, when the same action at a certain period, direction, and distance enters a long distance and for a long time, when the change in acceleration lasts for at least a certain time, and when the vector change of acceleration lasts at a predetermined interval. An action time not to be treated as the note action is determined based on a periodicity of change in acceleration data according to a value detected by the sensor.

Images