Handwritten content sharing system
The handwriting content sharing system addresses the lack of real-time handwritten content interaction in remote settings by using a detection mechanism and terminals to transmit and display handwritten data with writer context, enhancing educational and communicative effectiveness.
Patent Information
- Application Number
- JP2024105808
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-06-29
- Publication Date
- 2026-01-16
AI Technical Summary
Existing video conferencing systems fail to replicate the educational and communicative benefits of in-person writing, such as enhanced learning and teaching effectiveness, as they do not allow remote participants to see and interact with handwritten content in real time.
A handwriting content sharing system that includes a detection mechanism to capture handwritten data using a dot pattern on paper, a writer-side terminal to acquire and transmit this data, and a viewer-side terminal to display and output the handwritten content along with the writer's appearance and audio, all connected via telecommunications.
Enables remote participants to communicate and share information while actually writing, enhancing learning and teaching by allowing real-time viewing of handwritten content and the writer's context, thus improving comprehension and interaction.
Smart Images
Figure 2026006670000001_ABST
Abstract
Description
[Technical Field]
[0001] The present invention relates to a writing content sharing system that enables multiple people located in remote locations to preferably communicate or share information through writing. [Background technology]
[0002] Video call or video conferencing systems that allow multiple people in remote locations to exchange audio and video in real time over telecommunications lines are well known (see, for example, the following prior art documents). The use of these types of systems to hold online classes, lectures, training sessions, speeches, etc. is becoming commonplace. [Prior art documents] [Non-patent literature]
[0003] [Non-Patent Document 1] "Using FaceTime on iPhone or iPad" (iPhone, iPad, and FaceTime are registered trademarks), [online], Apple Inc., [searched February 1, 2024], Internet<URL:https: / / support.apple.com / ja-jp / 105088> Summary of the Invention [Problem to be solved by the invention]
[0004] For learners and participants receiving lectures or instruction from teachers and instructors, having a writing instrument and writing on notebooks or memo paper is considered important for enhancing the effectiveness of knowledge imparted. Writing by hand stimulates not only the senses of sight and hearing, but also the sense of touch, and by incorporating muscle movements, activates multiple areas of the brain simultaneously, enhancing concentration and allowing what one needs to learn to be imprinted deeply in the brain. The commonly-used concepts of "moving your hands" and "writing to remember" are empirically known. In particular, for children and students who are still growing, the repeated act of writing itself improves their writing ability and promotes intelligence. Strong writing ability allows them to think while writing and think while writing. It is conceivable that improved writing ability also leads to improved reading comprehension.
[0005] On the other hand, the act of writing with a writing instrument in hand can also be beneficial for teachers and lecturers. The speed at which characters are written, or in other words, the number of characters output per unit time, is slower than the speed at which one speaks orally. Therefore, when writing, teachers and lecturers are forced to minimize the number of characters and summarize the main points. This restriction leads to deeper reflection on the part of teachers and lecturers, enabling them to convey knowledge to learners and participants more efficiently. Another advantage is that learners and participants can see how teachers and lecturers are writing and what they have written, which makes it easier for them to understand the content.
[0006] When a teacher or lecturer writes something, it is with the intention of showing it to the other party, i.e., the learner or participant, and explaining it to them. Conversely, it is natural that teachers and students will want to watch the learner or participant write and confirm the content. If both parties are in the same place, for example, in a classroom at school or cram school, it is possible to directly see the act of writing and the content.
[0007] However, when both parties are in different locations, it is physically impossible to directly see what the other person is writing. It is difficult to recreate the environment of a real school, cram school, or private tutoring when the parties are far apart. This applies not only to educational settings, but also to meetings within and between companies, and other situations where you need to show someone what you have written.
[0008] The present invention was made in consideration of the above circumstances, particularly the importance of actually writing, and aims to provide a writing content sharing system that allows multiple people located in remote locations to communicate or share information while actually writing. [Means for solving the problem]
[0009] The present invention provides a handwriting content sharing system including, as its elements, a detection mechanism for detecting the handwriting of a writer who writes on paper using a writing implement, a writer-side terminal which is an information processing terminal located beside the writer, and a viewer-side terminal which is an information processing terminal located beside a viewer who views the content of the handwriting written by the writer, wherein the writer-side terminal includes a handwriting acquisition unit which acquires the handwriting of the writer on paper via the detection mechanism, an input control unit which photographs the appearance of the writer or the circumstances around the writer at the same time as the writer is writing via a camera separate from the detection mechanism and / or collects the voice of the writer or the voice around the writer at the same time via a microphone, and the handwriting data acquired by the handwriting acquisition unit and the image data (video) photographed via the input control unit. and a transmitting unit that transmits the handwriting data (which may be an image, or one or more still images in a time series) and / or collected audio data over a telecommunications line, and the viewer terminal is equipped with a receiving unit that receives the handwriting data, as well as the image data and / or the audio data, provided from the writer terminal over the telecommunications line, a handwriting display unit that displays the handwriting of the writer on paper on the screen of a display device based on the handwriting data received by the receiving unit, and an output control unit that displays the appearance of the writer or the situation around the writer on the screen of a display device based on the image data received by the receiving unit and / or outputs the voice of the writer or the voice around the writer from a speaker based on the audio data.
[0010] It should be noted that an information processing server may be interposed between the writer's terminal and the viewer's terminal. That is, the handwriting content sharing system may include an information processing server connected to both the writer's terminal and the viewer's terminal via a telecommunications line, and the information processing server includes a receiving unit that receives handwriting data and image data or collected audio data transmitted from the writer's terminal via the telecommunications line, a storage unit that stores the handwriting data and image data or collected audio data received by the receiving unit, and a transmitting unit that transmits the handwriting data and image data or collected audio data stored in the storage unit to the viewer's terminal via the telecommunications line.
[0011] The detection mechanism, for example, reads a dot pattern printed in advance on paper using a photoelectric sensor attached near the tip of the writing instrument used by the writer, detects the position coordinates on the paper indicated by the dot pattern, and by repeating this process, obtains time-series data of the position coordinates of the tip of the writing instrument, and outputs this data as the handwriting of the writer on the paper.
[0012] The program (or program product) used to configure the handwriting content sharing system of the present invention causes the information processing terminal on the writer's side to function as a handwriting acquisition unit that acquires the handwriting of the writer on paper using a writing instrument via a detection mechanism, an input control unit that photographs the writer's appearance or the situation around the writer at the same time as the writer is writing via a camera separate from the detection mechanism and / or collects the writer's voice or the sound around the writer at the same time via a microphone, and a transmission unit that transmits the handwriting data acquired by the handwriting acquisition unit, as well as the image data photographed and / or the sound data collected via the input control unit, via a telecommunications line.
[0013] It is then possible to provide a combination of a detection mechanism that reads a dot pattern printed in advance on paper using a photoelectric sensor attached near the tip of the writing instrument used by the writer, detects the position coordinates on the paper indicated by the dot pattern, and repeats this process to obtain time-series data on the position coordinates of the tip of the writing instrument, which then outputs this data as the handwriting of the writer on the paper, together with the above-mentioned program.
[0014] In addition, the program (or program product) used to configure the handwriting content sharing system of the present invention causes the information processing terminal on the viewer's side to function as a receiving unit that receives handwriting data of the handwriting on paper by the writer, which is provided via a telecommunications line from the writer's side terminal, which is an information processing terminal located next to the writer, as well as image data captured via a camera of the writer's appearance or the situation around the writer at the same time as the writer is writing and / or audio data collected via a microphone of the writer's speech or the sound around the writer at the same time; a handwriting display unit that displays the handwriting of the writer on the screen of a display device based on the handwriting data received by the receiving unit; and an output control unit that displays the writer's appearance or the situation around the writer on the screen of a display device based on the image data received by the receiving unit and / or outputs the writer's speech or the sound around the writer from a speaker based on the audio data. [Effects of the Invention]
[0015] According to the present invention, a suitable system for sharing written content is realized that enables a plurality of people located in remote locations to communicate or share information while actually writing. [Brief explanation of the drawings]
[0016] [Figure 1] 1 is a diagram schematically illustrating the main components of a handwritten content sharing system according to an embodiment of the present invention; [Figure 2] 3A and 3B are diagrams illustrating the effects of the handwritten content sharing system according to the embodiment. [Figure 3]3A and 3B are diagrams illustrating dot patterns on paper that constitute a detection mechanism in the embodiment. [Figure 4] 2 is a diagram illustrating hardware resources included in the transcriber terminal, the viewer device, and the information processing server of the embodiment. FIG. [Figure 5] FIG. 2 is a functional block diagram of the handwriting content sharing system according to the embodiment. [Figure 6] FIG. 2 is a diagram illustrating an example of how the handwritten content sharing system according to the embodiment is used. [Figure 7] FIG. 2 is a diagram illustrating an example of how the handwritten content sharing system according to the embodiment is used. [Figure 8] FIG. 2 is a diagram illustrating an example of how the handwritten content sharing system according to the embodiment is used. DETAILED DESCRIPTION OF THE INVENTION
[0017] An embodiment of the present invention will be described with reference to the drawings. As shown in Figures 1 and 2, the handwriting content sharing system of this embodiment allows a viewer S, who is located at a remote location separate from the writer W, to view the content T of writing written by a writer W on paper P using a writing implement. At the same time, the situation around the writer W at the same time as the writer W is writing can be transmitted to the viewer S. In other words, the viewer S can view an image M (which may be a moving image or a still image) of the appearance of the writer W or the surroundings of the writer W, and / or an utterance of the writer or a sound V of the surroundings of the writer, together with the content T of the handwriting.
[0018] A viewer S using this system can view and watch the content T of the writing and the words and actions M and V of the scribe W in real time (or in near real time with a slight time lag) while the scribe W is writing. Alternatively, the viewer S can view and watch the content T of the writing made by the scribe W in the past and the words and actions M and V of the scribe W after the writing act has occurred.
[0019] A specific application of this system is the transmission of written content T during lessons, lectures, instruction, etc. in educational settings (similar to classrooms in schools, cram schools, and individual instruction settings) where teachers, lecturers, etc. impart knowledge and insight to learners, participants (including children, pupils, and students). In this case, the teacher, lecturer, etc. can be the transcriber W, and the learners, participants, etc. can be the viewers S who view the written content T.
[0020] Conversely, a learner or a student may be the scribe W, and a teacher or lecturer may be the viewer S who views the written content T.
[0021] Of course, this system can be used in general situations outside of the educational setting, such as in meetings within a company or between companies, where written material T needs to be shown to others.
[0022] As shown in Figure 1, the main elements of this system are a detection mechanism that detects handwriting T of a writer W who writes on paper P using a writing implement, an information processing terminal 1 located near the writer W who writes, an information processing terminal 2 located near a viewer who views the content T written by the writer, and an information processing server 3 that acts as an intermediary between the two information processing terminals 1 and 2. The writer's terminal 1, the viewer's terminal 2, and the information processing server 3 are communicatively connected via a telecommunications line 4. The telecommunications line 4 includes a PAN (Personal Area Network), a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, etc.
[0023] This section describes a detection mechanism for detecting the content T of writings made by a writer W. A special dot pattern is printed in advance on the paper P on which the writer W writes. The paper P may be, for example, a notebook page, loose-leaf paper, report paper, memo pad, or other stationery, and is similar to ordinary paper except for the fine dot pattern. The dot pattern indicates the position coordinates of the location on the paper P where the dot pattern is printed. The dot pattern is scattered throughout the paper P and is printed across almost the entire surface of the paper P. Figure 3 shows an enlarged example of the dot pattern on a portion of the paper P. The dot pattern consists of countless tiny circular dots arranged according to a certain rule. The arrangement pattern of these dots within a given area (e.g., a 2 mm square) indicates the vertical and horizontal position coordinates of the location on the paper P. In the illustrated example, a virtual grid (not necessarily printed on the paper P) arranged at equal intervals vertically and horizontally or its intersections are set as reference points, and the relative positional relationship of each dot as viewed from this reference point is used to indicate the coordinate pattern. The dots that make up the dot pattern are printed using carbon ink or other pigments or dyes, but they may also be printed using a material that is transparent to the human eye and can be read by a photoelectric sensor on the writing instrument (one that can read the dot pattern by irradiating it with light waves of a specified wavelength and receiving the reflected waves; such as ink that glows in infrared or ultraviolet light).
[0024] The writing instrument side device 0 attached to the writing instrument used by the writer W can read the dot pattern on the paper surface P through a photoelectric sensor (a solid-state image sensor, which can be thought of as an ultra-small camera, and may be accompanied by an illumination light source) such as a CCD (Charge-Coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor).
[0025] The writing function of the writing instrument itself is to apply pigment (including graphite) or dye from the tip of the writing instrument onto paper P, that is, to leave visible handwriting T on paper P. The photoelectric sensor of the writing instrument side device 0 is mounted on the writing instrument side device 0 attached to the writing instrument that is held and used by the writer W. The writing instrument side device 0 is fixed to the writing instrument. The writing instrument side device 0 is a microcomputer system that includes a lens and a photoelectric sensor for capturing an image of the paper surface near the tip of the writing instrument, a sensor (such as a pressure sensor or a distance sensor; the lens and photoelectric sensor may serve both functions) for detecting that the tip of the writing instrument is in contact with the paper and writing is being performed, a memory (main memory and / or an auxiliary storage device such as a flash memory) for storing values indicated by the dot pattern on the paper read by the photoelectric sensor while writing with the writing instrument, a communication interface for transmitting the values stored in the memory to an information processing terminal 1 located next to the writer W, a processor for controlling these, and a battery as a power source for supplying the necessary power. The communication interface is a transceiver for wireless communication such as Bluetooth (registered trademark) or wireless LAN, or a connection interface for wired communication such as USB (Universal Serial Bus, registered trademark), which performs information communication with the information processing terminal 1 located next to the writer W. The writing instrument side device 0 may be integrated into the writing instrument body or may be detachable from the writing instrument body (pencil, mechanical pencil, ballpoint pen, etc.). When a writer uses a writing instrument to write, the writing instrument side device 0 reads a dot pattern printed on the paper P to be written on, and detects the coordinates of the current position of the tip of the writing instrument on the paper P, which is indicated by the dot pattern. By repeating this process, time-series data of the coordinates of the position of the tip of the writing instrument is obtained, and this is output to the information processing terminal 1 as handwriting data of the writing on the paper P by the writer W.
[0026] The combination of the paper P on which the dot pattern is printed and the writing implement side device 0 that constitute the detection mechanism of this embodiment, and the acquisition of handwriting data using these are existing technologies.
[0027] The specific embodiment of the detection mechanism is not limited to the one described above. For example, a known pressure-sensitive, electromagnetic induction, or capacitance touch panel, pen tablet, digitizer, or the like may be placed under the paper P to be written on, and the position coordinates of the tip of the writing implement when the writer W writes may be obtained via the touch panel, and time-series data of the position coordinates may be output to the information processing terminal 1. Another method for obtaining time-series data of the position coordinates of the tip of the writing implement is to use a triangulation method involving distance measurement using infrared or ultrasonic waves. Alternatively, an acceleration sensor may be installed in the writing implement device 0, and time-series data of the position coordinates of the tip may be obtained by detecting the movement of the writing implement. Although no dot pattern is printed on the paper surface P, a photoelectric sensor of the writing instrument side device 0 or a camera sensor independent of the writing instrument (which may be attached to the information processing terminal 1) may capture images of the paper surface P and / or the pigment or dye applied to the paper surface P through a lens, and the obtained images (time series) may be analyzed using AI (autificial intelligence) or the like to measure data on the handwriting T or perform optical character recognition.
[0028] The information processing terminals 1 and 2 used by the transcriber W and the viewer S, respectively, are, for example, general-purpose mobile phone terminals, particularly smartphones, tablets, personal computers, video game consoles, and other computers. As shown in Figure 4, the information processing terminals 1 and 2 include hardware resources such as a processor a, a main memory b, an auxiliary storage device c, an operation input device d, an audio codec e, a video codec f, a communication interface g, a speaker h, a display i as a display device, a microphone j, and a camera k, all of which are controlled and coordinated by a controller m (e.g., a system controller or an I / O controller). The auxiliary storage device c may be a flash memory, a hard disk drive, an optical disk drive, or the like. The video codec f includes a graphics processing unit (GPU) that generates the screen to be displayed based on drawing instructions received from the processor a and sends the screen signal to the display i, and a video memory that temporarily stores screen and image data. The audio codec e encodes audio signals input from the microphone j to generate audio data and decodes the encoded audio data to output audio from the speaker h. The video codec f and the audio codec e can each be implemented as software rather than hardware.
[0029] The operation input device d is a touch panel (which may be superimposed on the display i) that the user operates with his / her fingers, a push button, a pointing device such as a mouse, a keyboard, a horizontal button, or the like.
[0030] The communication interface g is a device for performing data communication between the writing instrument side device 0 attached to the writing instrument, and external devices or computers 1, 2, 3, and is, for example, a wireless transceiver for performing wireless communication via Bluetooth, wireless LAN, mobile phone network, WiMAX (registered trademark) network, etc., or a connection interface for performing wired communication such as Ethernet (registered trademark) or USB.
[0031] In the information processing terminal 1, a program to be executed by the processor a is stored in the auxiliary storage device c, and when the program is executed, it is read from the auxiliary storage device c into the main memory b and decoded by the processor a. The program may be pre-installed on the information processing terminal 1, or may be downloaded from an external computer (including the information processing server 3) via a telecommunications line. A known web browser may be used as part of the program. The program may also include scripts written in HTML (HyperText Markup Language), CSS (Cascading Style Sheets), JavaScript (registered trademark), etc.
[0032] The writer's terminal 1 located beside the writer W operates hardware resources in accordance with a program, and performs the functions of a handwriting acquisition unit 101, an input control unit 102, and a transmission unit 103 shown in FIG.
[0033] The handwriting acquisition unit 101 acquires handwriting data of writing on paper P by a writer W using a writing implement via a detection mechanism. Specific aspects of the detection mechanism are as described above. The handwriting acquisition unit 101 receives, as handwriting data, a time series of position coordinates of the tip of the writing implement on paper P via wireless communication (particularly, short-range wireless communication such as Bluetooth) or wired communication between the writing implement device 0 and the writer's terminal 1.
[0034] The input control unit 102 uses a camera k to capture images of the appearance of the writer W or the circumstances around the writer W at the same time that the writer W is writing, and obtains the image data (which may be moving image data or still image data). This camera k is separate from the camera of the writing instrument side device 0 (a photoelectric sensor mounted on the writing instrument side device 0) that constitutes the detection mechanism. The input control unit 102 also uses a microphone j to collect the voice of the writer W or the voices around the writer W at the same time that the writer W is writing, and obtains the voice data.
[0035] The transmission unit 103 transmits the handwriting data acquired by the handwriting acquisition unit 101, as well as the image data captured via the input control unit 102 and / or the audio data collected, via the telecommunications line 4 to the information processing server 3 and ultimately to the viewer terminal 2.
[0036] The information processing server 3 mediates between the transcriber terminal 1 located near the transcriber W and the viewer terminal 2 located near the viewer S. The information processing server 3 is, for example, a general-purpose personal computer, a workstation, a web server, or other computer. The information processing server 3 may be a cluster or a so-called cloud consisting of multiple computers connected to each other so as to be able to communicate via a LAN, WAN, or other telecommunications line 4. As shown in FIG. 4, the information processing server 3 also includes hardware resources such as a processor a, a main memory b, an auxiliary storage device c, an operation input device d, an audio codec e, a video codec f, a communication interface g, a speaker h, a display i, a microphone j, and a camera k, which operate in conjunction with each other under the control of a controller m.
[0037] In the information processing server 3, a program to be executed by the processor a is stored in the auxiliary storage device c, and when the program is executed, it is read from the auxiliary storage device c into the main memory b and decoded by the processor a. The program may be pre-installed on the information processing terminal 3 or may be downloaded from an external computer via a telecommunications line. A known web server (HTTP (HyperText Transfer Protocol) daemon) may be used as part of the program. The program may also include scripts written in HTML, CSS, JavaScript, etc. The information processing server 3 may transmit HTML, CSS, JavaScript, or other required programs or scripts to the transcriber terminal 1 or the viewer device 2.
[0038] The information processing server 3 operates hardware resources in accordance with a program, and performs the functions of a receiving unit 301, a storage unit 302, and a transmitting unit 303 shown in FIG.
[0039] The receiving unit 301 receives handwriting data of the writing content T by the writer W, image data photographed next to the writer W, and / or collected audio data, which are provided from the information processing terminal 1 on the writer W's side via the telecommunications line 4.
[0040] The storage unit 302 stores the handwriting data, image data and / or voice data received by the receiving unit 301 in a required storage area of the main memory b or the auxiliary storage device c.
[0041] The transmitting unit 303 transmits the handwriting data, image data and / or voice data received by the receiving unit 301 or stored in the storage unit 302 to the viewer terminal 2 via the telecommunications line 4 .
[0042] In the information processing terminal 2, a program to be executed by the processor a is stored in the auxiliary storage device c, and when the program is executed, it is read from the auxiliary storage device c to the main memory b and decoded by the processor a. The program may be pre-installed in the information processing terminal 2, or may be downloaded from an external computer (including the information processing server 3) via a telecommunications line. A known web browser may be used as part of the program. The program may also include scripts written in HTML, CSS, JavaScript, etc.
[0043] The viewer terminal 2 located near the viewer S operates hardware resources in accordance with a program, and performs the functions of a receiving unit 201, a handwriting display unit 202, and an output control unit 203 shown in FIG.
[0044] The receiving unit 201 receives handwriting data of the writing content T by the writer W, image data photographed next to the writer W, and / or collected audio data, which are provided from the information processing terminal 1 on the writer W's side (either via the information processing server 3 or without via the information processing server 3) via the telecommunications line 4.
[0045] The handwriting display unit 202 displays handwriting T written by the writer W on paper P on the screen of a display i, which is a display device, based on the handwriting data received by the receiving unit 202. Note that the display device that displays the handwriting T is not limited to the display i, and may be a projector, an electronic whiteboard, or the like.
[0046] The output control unit 203 displays the appearance of the writer W or the situation M around the writer W at the same time when the writer W is writing on the screen of the display i, which is a display device, based on the image data received by the receiving unit 202. Note that the display device that displays the situation M around the writer W is not limited to the display i, and may be a projector, an electronic whiteboard, or the like connected to the information processing terminal 2. In addition, the output control unit 203 outputs, from a speaker, the speech of the writer W or the voice S around the writer W at the same time when the writer W is writing, based on the audio data received by the receiving unit 202.
[0047] The mechanism for connecting the writer terminal 1 and the viewer terminal 2 may be similar to that of an existing video call or video conference system. For example, the information processing server 3 may act as a relay device between the writer terminal 1 and the viewer terminal 2. In this case, the information processing server 3 receives handwriting data, image data, and / or audio data transmitted from the writer terminal 1 and transmits the data to the viewer terminal 2. The writer W using the writing instrument terminal 1 and the viewer S using the viewer terminal 2 each transmit an identifier (account information, so-called ID (Identifier) and password, digital certificate, biometric information of the user, etc.) or an identifier for identifying an individual communication session (session ID, which may take the form of a Uniform Resource Identifier (URI))) to the information processing server 3. Upon receiving the identifier, the information processing server 3 authenticates the writing instrument terminal 1 and the viewer terminal 2 and establishes a communication connection and relay between a certain writing instrument terminal 1 and a certain viewer terminal 2.
[0048] In an aspect in which the information processing server 3 serves as a relay device, handwriting data and sets of image data and / or audio data can be stored and accumulated in the storage unit 302. After the writer W performs the writing act, the viewer S can receive the data stored in the storage unit 302 from the information processing server 3 and subsequently view the handwriting content T of the writer W and the image M and / or audio V around the writer W. The contents written by the writers W, W1, and W2 are displayed on a display i or the like, which is a display device of the viewer side terminal 2, and an image M of the situation around the writers W, W1, and W2 and / or a sound VM of the situation around the writers W, W1, and W2 and / or In contrast, the information processing server 3 can transmit an IP (Internet Protocol) address or the like indicating the destination of the communication between the other party to each of the note-taker terminal 1 and the viewer terminal 2, which are permitted to communicate with each other, and after establishing a communication connection between the two terminals, allowing direct communication between the note-taker terminal 1 and the viewer terminal 2, handwriting data, image data and / or audio data can be exchanged; in other words, the information processing server 3 can be configured not to act as a relay device for such data.
[0049] Naturally, the transcriber terminal 1 and the viewer terminal 2 within a facility such as a school or cram school may be connected via a LAN or WAN within the facility. It may also be possible to connect the transcriber terminal 1 and the viewer terminal 2 via P2P (Peer to Peer) from the beginning without using the information processing server 3.
[0050] The handwriting data, image data and / or audio data acquired by the transcriber terminal 1 is not necessarily sent immediately to the information processing server 3 or the viewer terminal 2. The handwriting data, image data and / or audio data may be temporarily stored in a required storage area of the storage unit of the transcriber terminal 1, i.e., the main memory b or the auxiliary storage device c, and then later transmitted from the transcriber terminal 1 to the information processing server 3 or the viewer terminal 2, which may then receive it.
[0051] In other words, the content T of writing by the scribe W and the image M / audio V of the surroundings of the scribe W are not necessarily viewed / listened to by the viewer S at approximately the same time (in real time) as the scribe W is writing, and there may be a time lag between the scribe W's writing act and the viewer S's viewing / listening. Even so, the data of the content T of writing by the scribe W and the data of the image M of the surroundings of the scribe W and / or the data of the audio V of the surroundings of the scribe W are synchronized in time. The viewer S can view the content T of writing by the scribe W, as well as the image M contemporaneous with the writing act, and / or listen to the audio V contemporaneous with the writing act.
[0052] In the output control unit 203 of the viewer side terminal 2, the contents of the writing made by the writer W are displayed on a display device such as a display i, for example, as follows: Animation processing: Time series data of the position coordinates of the tip of the writing implement on the paper P is sent from the writer's terminal 1 to the viewer's terminal 2. The viewer's terminal 2 displays a time series reproduction of the miracle of the tip of the writing implement based on the time series data (by drawing dots, etc., at the position coordinates of the tip of the writing implement at each point in time). Static image processing: Time series data of the position coordinates of the tip of the writing implement on the paper P is sent from the writer's terminal 1 to the viewer's terminal 2. The viewer's terminal 2 generates image data that depicts the image of the tip of the writing implement (drawing dots, etc., sequentially at the position coordinates of the tip at each point in time) based on the time series data, and displays the image. Note that such still image data may be generated in the writer's terminal 1 or the information processing server 3, and the image data may be sent to the viewer's terminal 2, which then displays the image. Processing as text data: Time series data of the position coordinates of the tip of the writing implement on the paper P is sent from the writer's terminal 1 to the viewer's terminal 2. Based on the time series data, the viewer's terminal 2 determines and deciphers the characters written by the writer W using AI or OCR, etc., and displays the character string on the screen in the required font. It is preferable that the font size is reflected according to the size of the characters written by the writer W. Note that such character string data may be generated in the writer's terminal 1 or the information processing server 3, and the text data may be sent to the viewer's terminal 2, which then displays the character string upon receiving it. It is possible that:
[0053] This system can be easily realized by utilizing an existing video call or video conference system. In an existing video call or video conference system, an image M captured by one terminal 1, 2 or a collected audio V can be transmitted to the other terminal 2, 1, and the image M or audio V can be output on the other terminal 2, 1. Furthermore, a display screen T of one terminal 1, 2 can be transmitted to the other terminal 2, 1, and the screen T can be displayed on the other terminal 2, 1, thereby performing so-called screen sharing (desktop sharing). Therefore, this system can be realized by displaying the handwritten content T by the scribe W on the screen of the scribe terminal 1 and displaying that display screen on the viewer terminal 2.
[0054] In operating this system, as shown in FIG. 2, one form is to connect the transcriber terminal 1 and the viewer terminal 2 one-to-one, and allow the contents of writing by the transcriber W using the former terminal 1 (as well as the image M and / or audio V of the surroundings of the transcriber W) to be viewed by the viewer S using the latter terminal 2. However, as shown in FIGS. 6 and 7, it is also possible to connect the transcriber terminal 1 and the viewer terminal 2 one-to-many or many-to-many. In the example shown in FIG. 6, there is at least one transcriber terminal 1 and multiple viewer terminals 2, and a communication session connects these transcriber terminals 1 and 2. It is then possible to allow multiple viewers S1 and S2 to simultaneously view the contents T of writing by the transcriber W, as well as the image M and / or audio V of the surroundings of the transcriber W at the same time as the transcriber W was writing, whether simultaneously or not. This can be used, for example, to allow a lecture content of a lecturer, a scribe W, to be viewed by a plurality of viewers S1 and S2, who are students.
[0055] 7, it is also possible to realize a communication session connecting at least one viewer terminal 2 and multiple writer terminals 1, as shown in FIG. 7. It is also possible to allow a viewer S to view the contents T1 and T2 written by multiple writers W1 and W2, respectively, as well as images M and / or audio V of the surroundings of each writer W1 and W2 at the same time as each writer W1 and W2 is writing. This can be used, for example, to allow a viewer S, who is an instructor, to view the contents T1 and W2 written by multiple writers W1 and W2, who are students.
[0056] Incidentally, it is also possible to reflect the color and line type (line thickness, solid line / chain line / dotted line, etc., or font type and color) of the handwriting T to be displayed on the viewer terminal 2, i.e., the handwriting T written by the writer W. In the example illustrated in FIG. 8, the handwriting T is a mixture of a certain color or line type TA and a different color or line type TB, and these are displayed on the display i of the viewer terminal 2. As described above, in this system, the writer W actually writes on the paper P using a writing implement. The detection mechanism detects the current position coordinates of the tip of the writing implement via the writing implement device 0, and provides the time series of the position coordinates as data of the handwriting T to the writer terminal 1 and, ultimately, to the viewer terminal 2. However, the detection mechanism cannot necessarily detect the color or line types TA and TB of the pigment or dye actually applied to the paper P from the tip of the writing implement. Therefore, in this embodiment, The writer's terminal 1 accepts, via the operation input device d, an operation to specify the color and line types TA and TB of the writing implement used by the writer W currently writing, according to the program. After accepting the operation, the data of the handwriting T is associated with information identifying the color or line types TA and TB. Different writing implements are prepared for different colors and line types TA and TB, and each writing implement is provided with a writing implement device 0. When the writing implement device 0 transmits data of handwriting T to the writer's terminal 1, it transmits information identifying the writing implement device 0, i.e., information specifying the color or line type TA or TB of handwriting T, along with the data of handwriting T, and the writer's terminal 1 receives this information. A photoelectric sensor (camera) mounted on the writing implement device 0 captures an image of the color and line types TA and TB of the pigment or dye applied to the paper P from the tip of the writing implement, and the writing implement device 0 determines the color or line types TA and TB. The writing implement device 0 then transmits information specifying the color or line types TA and TB of the handwriting T together with the data of the handwriting T, which is received by the writer's terminal 1. In either of the above cases, the data of the handwriting T transmitted from the writer terminal 1 to the viewer terminal 2 is accompanied by information on the color or line type TA, TB representing the handwriting T. Upon receiving this, the viewer terminal 2 draws the handwriting T on the screen of the display i in the specified color or line type TA, TB according to the program.
[0057] The present invention is not limited to the above-described embodiment, and the specific configuration of each unit and the processing procedure can be modified in various ways without departing from the spirit of the present invention. [Explanation of symbols]
[0058] 0…Writing implements 1...Transcriber's terminal 101...Handwriting acquisition part 102...input control unit 103...Transmitter 2...Viewer's terminal 201...Receiver 202...Handwriting display area 203...Output control unit 3...Information processing server 301...Receiver 302...Storage section 303...Transmitter
Claims
1. A writing content sharing system including, as elements, a detection mechanism for detecting handwriting of a writer who writes on paper using a writing implement, a writer-side terminal which is an information processing terminal located near the writer, and a viewer-side terminal which is an information processing terminal located near a viewer who views the contents of the handwriting by the writer, The scribe side terminal, a handwriting acquisition unit that acquires handwriting on paper by the writer via the detection mechanism; an input control unit that captures an image of the scribe or the surroundings of the scribe at the same time as the scribe is writing using a camera separate from the detection mechanism, or collects the scribe's speech or the sounds around the scribe at the same time using a microphone; a transmitting unit that transmits the handwriting data acquired by the handwriting acquisition unit, as well as the photographed image data or the collected audio data via the input control unit, through a telecommunications line; Equipped with The viewer side terminal, a receiving unit for receiving the handwriting data and the image data or the voice data transmitted from the writer's terminal via a telecommunications line; a handwriting display unit that displays the handwriting of the writer on a screen of a display device based on the handwriting data received by the receiving unit; an output control unit that displays the appearance of the writer or the situation around the writer on a screen of a display device based on the image data received by the receiving unit, or outputs the voice of the writer or the voice around the writer from a speaker based on the voice data; A writing content sharing system comprising:
2. A note sharing system including an information processing server connected to both the note-taker terminal and the viewer terminal via a telecommunications line, The information processing server, a receiving unit that receives the handwriting data and image data or collected voice data transmitted from the writer's terminal via a telecommunications line; a storage unit for storing the handwriting data and image data received by the receiving unit or the collected voice data; a transmitting unit for wirelessly transmitting the handwriting data and image data or collected voice data stored in the storage unit to the viewer terminal via a telecommunications line; 2. The system for sharing handwritten notes according to claim 1, comprising:
3. The writing content sharing system described in claim 1 or 2, wherein the detection mechanism reads a dot pattern printed in advance on paper using a photoelectric sensor attached near the tip of the writing instrument used by the writer, detects the position coordinates on the paper indicated by the dot pattern, and by repeating this process, obtains time-series data of the position coordinates of the tip of the writing instrument, and outputs this data as the handwriting of the writer on the paper.
4. 2. A system for sharing handwritten notes according to claim 1, comprising: an information processing terminal; a handwriting acquisition unit that acquires handwriting written by a writer on paper using a writing implement via a detection mechanism; an input control unit that captures the appearance of the writer or the surroundings of the writer at the same time as the writer is writing using a camera separate from the detection mechanism, or collects the voice of the writer or the voices around the writer at the same time using a microphone; a transmission section that transmits the handwriting data acquired by the handwriting acquisition section, as well as the image data captured via the input control section or the voice data collected via the input control section, via a telecommunications line; A program that functions as a
5. A detection mechanism that reads a dot pattern printed on paper in advance with a photoelectric sensor attached near the tip of the writing implement used by the writer, detects the position coordinates on the paper indicated by the dot pattern, and repeats this process to obtain time-series data of the position coordinates of the tip of the writing implement, and outputs this data as the handwriting of the writer on the paper; A set with the program according to claim 3.
6. 2. A system for sharing handwritten notes according to claim 1, comprising: an information processing terminal; a receiving unit that receives handwriting data of the handwriting on paper by the scribe, which is transmitted via a telecommunications line from a scribe's terminal, which is an information processing terminal located next to the scribe, as well as image data of the scribe's appearance or the scribe's surroundings taken via a camera at the same time as the scribe is writing, or audio data of the scribe's speech or the sounds of the scribe's surroundings taken via a microphone at the same time; a handwriting display unit that displays the handwriting of the writer on paper on a screen of a display device based on the handwriting data received by the receiving unit; an output control unit that displays the appearance of the writer or the situation around the writer on a screen of a display device based on the image data received by the receiving unit, or outputs the voice of the writer or the voice around the writer from a speaker based on the voice data; A program that functions as a