Terminal, input display method and input display system

By implementing input mode selection and display functions in the communication terminal, the problem of inability to infer input time of the other terminal is solved, and the communication between the terminals is improved.

CN116391352BActive Publication Date: 2024-11-22MITSUBISHI ELECTRIC CORP
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202080107071.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-11-19
Publication Date
2024-11-22
Estimated Expiration
2040-11-19

AI Technical Summary

Technical Problem

The existing communication terminal device cannot infer the waiting time required by the sender before completing the message input, resulting in a lack of realism in communication between the terminals.

Method used

By realizing the input mode selection, notification, acceptance, mode acquisition and display control unit in the terminal, it notifies and displays the input mode to the communication side and allows the user to infer the information input time of the other terminal based on experience.

Benefits of technology

By displaying the input mode of the other terminal, the user can roughly estimate the time required for the input of the other terminal to improve the realism of communication between the terminals.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116391352B_ABST
    Figure CN116391352B_ABST
Patent Text Reader

Abstract

The terminal (2) comprises: an input mode selection unit (3) which accepts selection of an input mode from a plurality of input modes; a notification unit (5) which notifies the terminal (2) on the other side of the communication of the input mode; an input acceptance unit (4a, 4b) which accepts information input based on the input mode selected by the input mode selection unit (3); a mode acquisition unit (7) which acquires the input mode notified from the terminal (2) on the other side of the communication; and a display control unit (8) which generates display information for displaying the information input accepted by the input acceptance unit (4a, 4b) on a display area of ​​a display (10). When the mode acquisition unit (7) acquires the input mode notified from the terminal (2) on the other side of the communication, the display control unit (8) generates display information for displaying the acquired input mode on the display area, and displays the input mode notified from the terminal (2) on the other side of the communication on the display (10) according to the generated display information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to a terminal, an input display method and an input display system. Background Art

[0002] A communication terminal device for communicating between two separate locations by communication is known. For example, Patent Document 1 describes a communication terminal device that notifies the other party's communication terminal device of the state of editing or generating a message as a transfer process, generates and transmits a message, and receives and reproduces a message generated by the other party's communication terminal device.

[0003] In the communication terminal device described in Patent Document 1, the recipient of a message can identify the sender of a message by the state of the message being edited or generated. For example, the recipient of a message can recognize the sender's status such as "message is being input." or "message has been deleted.", and can communicate with the other party in consideration of their status.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2006-140748 Summary of the invention

[0007] Problem that the invention aims to solve

[0008] However, the communication terminal device described in Patent Document 1 has the following problem: although the recipient of the message can recognize that the sender is inputting a message, it is impossible to guess how much waiting time is spent before the sender completes the input of the message and receives the message. This makes the communication between the terminals lack a sense of reality.

[0009] The present disclosure is used to solve the above-mentioned problem, and its object is to obtain a terminal, an input display method, and an input display system that can estimate the input time of information in a terminal of a communication partner.

[0010] Means used to solve problems

[0011] The terminal disclosed in the present invention comprises: an input mode selection unit, which accepts selection of an input mode from a voice input mode and a character input mode; a notification unit, which notifies a terminal on the other side of the communication of the input mode selected by the input mode selection unit; an input acceptance unit, which accepts information input based on the input mode selected by the input mode selection unit; a mode acquisition unit, which acquires the input mode notified from the terminal on the other side of the communication; and a display control unit, which generates display information for displaying the information input accepted by the input acceptance unit in a display area of ​​a display. When the mode acquisition unit acquires the input mode notified from the terminal on the other side of the communication, the display control unit generates display information for displaying the acquired input mode in the display area, and displays the input mode notified from the terminal on the other side of the communication on the display according to the generated display information.

[0012] Effects of the Invention

[0013] According to the present disclosure, the display control unit generates display information for displaying the input mode notified from the terminal of the communication partner on the display area, and the display displays the input mode notified from the terminal of the communication partner according to the generated display information. Based on the experience of using a common communication terminal, the user roughly grasps the time required for inputting information for each of the various input modes provided by the terminal. Therefore, the terminal of the present disclosure can infer the input time of information in the terminal of the communication partner by displaying the input mode notified from the terminal of the communication partner on the display. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a block diagram showing the configuration of the input display system according to the first embodiment.

[0015] Figure 2 This is a block diagram showing the structure of a terminal according to Embodiment 1.

[0016] Figure 3 This is a flowchart showing the input display method according to the first embodiment.

[0017] Figure 4 It is an explanatory diagram showing a screen of a terminal on the transmitting side and a screen of a terminal on the receiving side for performing voice input.

[0018] Figure 5 It is an explanatory diagram showing a screen of a terminal on the transmission side and a screen of a terminal on the reception side for performing keyboard input.

[0019] Figure 6 It is an explanatory diagram showing a screen of a terminal for performing keyboard input and a screen of a terminal for performing voice input.

[0020] Figure 7 This is a block diagram showing a hardware configuration for realizing the functions of the terminal according to the first embodiment.

[0021] Figure 8 This is a block diagram showing a hardware configuration for executing software that realizes the functions of the terminal according to the first embodiment.

[0022] Fig. 9 This is a block diagram showing the structure of a terminal according to the second embodiment.

[0023] Fig.10 This is a flowchart showing the input display method of the second embodiment.

[0024] Fig.11 It is an explanatory diagram showing a screen of a terminal on the transmission side and a screen of a terminal on the reception side that draw graphics.

[0025] Fig.12 It is an explanatory diagram showing a screen of a terminal on the transmission side and a screen of a terminal on the reception side in which an image selection operation is performed.

[0026] Fig.13 It is an explanatory diagram showing the screen of a terminal on the transmitting side and the screen of a terminal on the receiving side performing camera photography.

[0027] Fig.14 The diagram is an explanatory diagram showing a tool selection, a screen of a transmitting terminal for drawing a graphic using the selected tool, and a screen of a receiving terminal. DETAILED DESCRIPTION

[0028] Implementation method 1.

[0029] Figure 1 This is a block diagram showing the configuration of the input display system 1 according to the first embodiment. Figure 1 The input display system 1 shown is a system that displays information on each other's displays through communication between terminals 2. In the input display system 1, when the terminal 2 on the sending side inputs information (message, etc.) to be sent to the terminal 2 on the receiving side, information indicating the input mode of the information is sent to the terminal 2 on the receiving side. The terminal 2 is, for example, a smartphone, a tablet terminal, or a PC (Personal Computer). The terminal 2 on the receiving side displays the input mode received from the terminal 2 on the sending side on the display. Figure 1 , a case where two terminals 2 communicate with each other is shown. However, the input display system 1 may allow three or more terminals 2 to share display information with each other.

[0030] In the input display system 1, when one of the various input modes is selected, the terminal 2 notifies the selected input mode to the terminal 2 on the other side of the communication, and starts the information input of the input mode. When the terminal 2 on the other side of the communication notifies the input mode, the terminal 2 displays the notified input mode on the display. As input modes that can be used in terminals 2 such as smartphones, tablet terminals or PCs, there are, for example, voice input mode and keyboard input mode. The voice input mode is a mode in which a character string corresponding to a voice is input through voice recognition. The keyboard input mode is a mode in which input of a character string using a software keyboard or a hardware keyboard is accepted.

[0031] In the keyboard input mode, input is performed on a character-by-character basis using the keyboard. On the other hand, in the voice input mode, only a voice is spoken into the microphone. That is, the voice input mode can shorten the time required for information input compared to the keyboard input mode. Users who are used to using smartphones, tablet terminals, or PCs have an experience of understanding the difference in the time required for information input in each of these input modes. Therefore, when terminal 2 obtains the input mode notified from terminal 2 on the other party's side of the communication, it generates display information for displaying the obtained input mode on the display, and displays the input mode notified from the terminal on the other party's side of the communication on the display according to the generated display information. As a result, the user can, for example, roughly estimate the time it takes to complete the information input in terminal 2 on the other party's side of the communication.

[0032] Figure 2 This is a block diagram showing the structure of the terminal 2 according to the first embodiment. Figure 1 The input display system 1 shown in the figure has terminals 2 each having Figure 2 The terminal 2 includes an input mode selection unit 3, a voice input unit 4a, a keyboard input unit 4b, a notification unit 5, a communication unit 6, a mode acquisition unit 7, a display control unit 8, a character string temporary storage unit 9, a display 10, an operation receiving unit 11, a character management database 12 (hereinafter referred to as character management DB12), a character management unit 13, and a trajectory determination unit 14.

[0033] The input mode selection unit 3 accepts selection of an input mode from a plurality of input modes executable by the terminal 2. For example, the input mode selection unit 3 accepts selection of any one of a voice input mode and a keyboard input mode based on the selection operation (1). For example, the input mode selection unit 3 accepts a selection operation using a pointing device, and starts an input acceptance unit corresponding to the input mode accepted in the voice input unit 4a and the keyboard input unit 4b. As the pointing device, for example, a touch panel is assumed. However, this is only an example, and a joystick, a pointing stick, a touchpad, a stylus, a data glove, a trackball, a digitizer, a mouse, a light pen, or a joystick can be used for the pointing device.

[0034] The voice input unit 4a is an input receiving unit that receives input of a character string corresponding to a voice through voice recognition. For example, the voice input unit 4a detects the voice of the user and performs voice recognition on the detected voice. A voice recognition dictionary database (hereinafter referred to as voice recognition dictionary DB) is used for voice recognition. The voice recognition dictionary DB, for example, registers an acoustic model that describes the acoustic features of phonemes, which are units of human speech, and a recognition dictionary that describes words for voice recognition. The voice input unit 4a has a voice recognition engine that recognizes the input voice.

[0035] The voice input unit 4a analyzes the input voice, calculates the acoustic features of the voice, and searches for a word having an acoustic feature closest to the calculated acoustic feature from the words described in the recognition dictionary registered in the voice recognition dictionary DB. The voice input unit 4a outputs the character string representing the searched word as a character string representing the voice recognition result to the display control unit 8.

[0036] In addition, the voice input unit 4a may also include a data transceiver instead of the voice recognition engine and the voice recognition dictionary DB. The data transceiver is a communication device that can transmit and receive data with a voice recognition server (not shown) via a communication line such as the Internet, and may be, for example, the communication unit 6. The data transceiver sends data representing the voice detected by the voice input unit 4a to the voice recognition server. The voice recognition server includes a voice recognition engine for recognizing voice, and when receiving data representing the voice sent from the data transceiver, the voice represented by the received data is recognized, and a character string representing the recognition result of the voice is sent to the data transceiver. The data transceiver outputs the character string representing the voice recognition result received from the voice recognition server to the display control unit 8.

[0037] The keyboard input unit 4b is an input receiving unit that receives input of a character string using a software keyboard or a hardware keyboard. For example, in a software keyboard, character input is received through a touch panel for a keyboard displayed on the display 10. In the following description, it is assumed that the keyboard input unit 4b uses a software keyboard.

[0038] The notification unit 5 notifies the terminal 2 on the communication partner side of the input mode selected by the input mode selection unit 3. For example, the notification unit 5 establishes a communication connection via the communication unit 6, thereby notifying the terminal 2 on the communication partner side of information indicating the input mode. The communication unit 6 is a device that communicates with the terminal 2 on the communication partner side. The communication method performed by the communication unit 6 may be any one of wireless communication and wired communication.

[0039] The mode acquisition unit 7 acquires the input mode notified from the terminal 2 on the communication partner side. For example, the mode acquisition unit 7 establishes a communication connection via the communication unit 6, thereby receiving information indicating the input mode from the terminal 2 on the communication partner side. The input mode acquired by the mode acquisition unit 7 is the input mode selected by the input mode selection unit 3 provided in the terminal 2 on the communication partner side.

[0040] The display control unit 8 generates display information for displaying information (e.g., a character string) inputted by the voice input unit 4a and the keyboard input unit 4b on the display area of ​​the display 10. The character string temporary storage unit 9 temporarily stores data representing the character string inputted by the voice input unit 4a or the keyboard input unit 4b. For example, data representing the character string is stored in the character string temporary storage unit 9 until the curve information is outputted from the trajectory determination unit 14. The display 10 displays information according to the display information generated by the display control unit 8.

[0041] The operation accepting unit 11 accepts the operation (2) of drawing a line as the user's operation, and outputs the content of the accepted operation to the trajectory determination unit 14. For example, the operation accepting unit 11 accepts the operation (2) using a pointing device. The character management DB 12 is a database in which character attribute table information indicating the correspondence between character type, character size, character height, and character width is registered.

[0042] The character size of the characters displayed on the display 10 is preset in the character management unit 13. The character management unit 13 refers to the character attribute table information registered in the character management DB 12, outputs the character attribute information indicating the character height corresponding to the preset character size to the trajectory determination unit 14, and outputs the character attribute information indicating the character height and character width corresponding to the preset character size to the display control unit 8.

[0043] The trajectory determination unit 14 determines the trajectory of the line input to the display area of ​​the display 10 through the operation accepted by the operation acceptance unit 11. For example, the trajectory determination unit 14 obtains the time series data of the point cloud constituting the line input through the user's operation from the pointing device. The trajectory determination unit 14 generates curve information representing the length and shape of the curve based on the time series data of the point cloud obtained from the pointing device, and outputs the generated curve information to the display control unit 8. The curve represents the trajectory of the line input through the user's operation. The curve also includes straight lines or curved lines. That is, the curve determined by the trajectory determination unit 14 includes lines of various shapes that can be calculated using the time series data of the point cloud on the display area of ​​the display 10.

[0044] Figure 3 This is a flowchart showing the input display method according to the first embodiment.

[0045] The input mode selection unit 3 accepts the selection of the input mode (step ST1). Next, the notification unit 5 notifies the terminal 2 on the other side of the communication connection via the communication unit 6 that the input mode selection unit 3 has accepted the selection of the input mode (step ST2). When the selection of the voice input mode is accepted, the voice input unit 4a accepts the input of information based on the voice input mode (step ST3). When the selection of the keyboard input mode is accepted, the keyboard input unit 4b accepts the input of information based on the keyboard input mode (step ST3).

[0046] If the input receiving unit corresponding to the input mode that has accepted the selection among the voice input unit 4a and the keyboard input unit 4b has not completed the input of information (step ST4; No), the input of information continues. When the information input is completed (step ST4; Yes), the input receiving unit corresponding to the input mode that has accepted the selection outputs information indicating that the information input is completed to the input mode selection unit 3. The input mode selection unit 3 outputs information indicating that the information input is completed to the notification unit 5. The notification unit 5 is connected to the communication unit 6 via the communication unit, thereby notifying the terminal 2 on the other side of the communication of the information indicating that the information input is completed (step ST5).

[0047] Next, the display control unit 8 generates display information for displaying information that the input is accepted by the input accepting unit corresponding to the selected input mode among the voice input unit 4a and the keyboard input unit 4b on the display 10 (step ST6). The display 10 displays information that the input is accepted by the input accepting unit corresponding to the selected input mode among the voice input unit 4a and the keyboard input unit 4b according to the display information generated by the display control unit 8 (step ST7).

[0048] The mode acquisition unit 7 checks whether the input mode notified from the communication partner terminal 2 is acquired (step ST8). For example, the mode acquisition unit 7 performs communication connection via the communication unit 6, thereby receiving information indicating the input mode from the communication partner terminal 2. The information indicating the input mode acquired by the mode acquisition unit 7 is output to the display control unit 8.

[0049] The display control unit 8 generates display information for displaying the acquired input mode on the display 10 (step ST9). The display 10 displays the input mode notified from the terminal 2 of the communication partner side according to the display information generated by the display control unit 8 (step ST10). The user can visually confirm the input mode displayed on the display 10, and can understand in which input mode the terminal 2 of the communication partner side is inputting information, and can roughly estimate the input time of the information based on the understood input mode.

[0050] The mode acquisition unit 7 confirms whether the input of information is completed in the terminal 2 of the communication partner side (step ST11). For example, when the communication unit 6 receives information indicating that the input of information is completed from the terminal 2 of the communication partner side (step ST11; yes), the mode acquisition unit 7 notifies the display control unit 8 that the input of information is completed in the terminal 2 of the communication partner side. In addition, if the communication unit 6 does not receive information indicating that the input of information is completed from the terminal 2 of the communication partner side (step ST11; no), the mode acquisition unit 7 repeats the above confirmation process.

[0051] When the input of information is completed in the terminal 2 of the communication partner side, the display control unit 8 generates display information for not displaying the input mode notified from the terminal 2 of the communication partner side, and outputs it to the display 10. The display 10 clears (does not display) the display of the input mode notified from the terminal 2 of the communication partner side from the screen according to the display information generated by the display control unit 8 (step ST12).

[0052] Figure 4 1 is an explanatory diagram showing a screen 10A of a terminal 2 on the transmitting side and a screen 10B of a terminal 2 on the receiving side for performing voice input. The input mode selection unit 3 is implemented as a user interface using a display 10 and a touch panel. Figure 4 In the example shown, the display control unit 8 generates display information for displaying the button image 3a corresponding to the voice input mode and the button image 3b corresponding to the keyboard input mode. The display 10 displays the button image 3a and the button image 3b according to the display information generated by the display control unit 8.

[0053] When the user presses the button image 3a or the button image 3b using the touch panel, the operation information detected by the touch panel is output to the input mode selection unit 3. The input mode selection unit 3 accepts the selection of the input mode based on the operation information. Figure 4 , the button image 3a is pressed, so the input mode selection unit 3 accepts the selection of the voice input mode. The notification unit 5 notifies the receiving terminal 2 of the input mode (voice input mode) that the selection has been accepted via the communication unit 6. The receiving terminal 2 displays information indicating the voice input mode notified from the transmitting terminal 2 on the screen 10B. For example, text information such as "voice input in progress" is displayed on the screen 10B.

[0054] In the terminal 2 on the transmitting side, the voice input unit 4a performs voice recognition on the voice such as "ABCD" uttered by the user, and outputs the character string M as the voice recognition result to the display control unit 8. As a result, the voice input is completed, and therefore, the notification unit 5 notifies the terminal 2 on the receiving side of the completion of the voice input via the communication unit 6. Next, the display control unit 8 uses the curve information obtained from the trajectory determination unit 14 and the character string M obtained from the voice input unit 4a to generate display information in which the character string M is arranged along the curve C represented by the curve information. The display information generated by the display control unit 8 is output to the display 10, and further transmitted to the terminal 2 on the receiving side via the communication unit 6.

[0055] like Figure 4 As shown, the display 10 displays the character string M on the screen 10A according to the display information generated by the display control unit 8. When the receiving terminal 2 is notified of the completion of the voice input from the transmitting terminal 2, the input mode display is cleared from the screen 10B. Thereafter, the receiving terminal 2 displays the character string M on the screen 10B according to the display information received from the transmitting terminal 2.

[0056] Figure 5 10A and 10B of the terminal 2 on the receiving side. Figure 4 When the button image 3b shown in FIG. 1 is displayed, the input mode selection unit 3 accepts the selection of the keyboard input mode. The notification unit 5 notifies the receiving terminal 2 of the keyboard input mode that the selection has been accepted via the communication unit 6. The receiving terminal 2 displays information indicating the input mode (keyboard input mode) notified from the transmitting terminal 2 on the screen 10B. For example, text information such as "Keyboard input in progress" is displayed on the screen 10B.

[0057] In the terminal 2 on the transmitting side, the keyboard input unit 4b accepts character input using the software keyboard 15, and outputs the character string M of "ABCD" that has been input to the display control unit 8. Next, the notification unit 5 notifies the terminal 2 on the receiving side of the completion of the keyboard input via the communication unit 6. The display control unit 8 uses the curve information obtained from the trajectory determination unit 14 and the character string M obtained from the keyboard input unit 4b to generate display information in which the character string M is arranged along the curve C represented by the curve information. The display information generated by the display control unit 8 is output to the display 10, and is further transmitted to the terminal 2 on the receiving side via the communication unit 6.

[0058] like Figure 5As shown, the display 10 displays the character string M on the screen 10A according to the display information generated by the display control unit 8. When the receiving terminal 2 is notified of the completion of the keyboard input from the sending terminal 2, the input mode display ("keyboard input in progress") is cleared (not displayed) from the screen 10B. Thereafter, the receiving terminal 2 displays the character string M on the screen 10B according to the display information received from the sending terminal 2.

[0059] Figure 6 1 is an explanatory diagram showing a screen 10A of a terminal 2 for keyboard input and a screen 10B of a terminal 2 for voice input. In the terminal 2 having the screen 10A, the input mode selection unit 3 accepts the selection of the keyboard input mode. The notification unit 5 notifies the terminal 2 having the screen 10B of information indicating that the selected keyboard input mode has been accepted via the communication unit 6. The terminal 2 displays information indicating the notified input mode (keyboard input mode) on the screen 10B.

[0060] In the terminal 2 having the screen 10B, the input mode selection unit 3 accepts the selection of the voice input mode. The notification unit 5 notifies the terminal 2 having the screen 10A of the voice input mode that has been selected via the communication unit 6. The terminal 2 displays information indicating the notified input mode (voice input mode) on the screen 10A. Thus, the user of each terminal 2 can estimate the rough input time based on the input mode of the communication partner. Thus, the realism of the communication based on the communication between the terminals 2 can be improved.

[0061] Figure 7 2 is a block diagram showing a hardware structure for realizing the functions of the terminal 2. Figure 7 In the embodiment, the processing circuit 107 obtains data input to the input mode selection unit 3 and the operation receiving unit 11 via the input interface 100. The processing circuit 107 outputs display information to the display 102 via the output interface 101. The display 102 is Figure 2 The processing circuit 107 inputs and outputs data transmitted and received between the communication device 104 and the terminal 2 on the other side of the communication via the communication interface 103. The communication device 104 is Figure 2 The processing circuit 107 inputs and outputs data to and from the storage device 106 via the input / output interface 105. The storage device 106 stores Figure 2 The character management DB 12 is shown.

[0062] Figure 8 2 is a block diagram showing the hardware structure of the software that executes the functions of the terminal 2. Figure 8In the embodiment, the processor 108 obtains data input to the input mode selection unit 3 and the operation receiving unit 11 via the input interface 100. The processor 108 outputs display information to the display 102 via the output interface 101. The display 102 is Figure 2 The processor 108 inputs and outputs data transmitted and received between the communication device 104 and the terminal 2 on the other side of the communication via the communication interface 103. The communication device 104 is Figure 2 The processor 108 inputs and outputs data to and from the storage device 106 via the input / output interface 105. The storage device 106 stores Figure 2 The character management DB 12 is shown.

[0063] The functions of the input mode selection unit 3, the voice input unit 4a, the keyboard input unit 4b, the notification unit 5, the mode acquisition unit 7, the display control unit 8, the character string temporary storage unit 9, the operation receiving unit 11, the character management unit 13 and the trajectory determination unit 14 are realized by the processing circuit. Figure 3 The processing circuit of each process from step ST1 to step ST12 shown in the figure may be dedicated hardware or a CPU (Central Processing Unit) that executes a program stored in a memory.

[0064] The processing circuit is Figure 7 In the case of the dedicated hardware processing circuit 107 shown, the processing circuit 107 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0065] The functions of the input mode selection unit 3, sound input unit 4a, keyboard input unit 4b, notification unit 5, mode acquisition unit 7, display control unit 8, character string temporary storage unit 9, operation acceptance unit 11, character management unit 13 and trajectory determination unit 14 possessed by the terminal 2 can be implemented by different processing circuits, or these functions can be unified by one processing circuit.

[0066] The processing circuit is Figure 8In the case of the processor 108 shown, the functions of the input mode selection unit 3, the voice input unit 4a, the keyboard input unit 4b, the notification unit 5, the mode acquisition unit 7, the display control unit 8, the character string temporary storage unit 9, the operation receiving unit 11, the character management unit 13 and the trajectory determination unit 14 of the terminal 2 are realized by software, firmware or a combination of software and firmware. In addition, the software or firmware is recorded in the form of a program and stored in the memory 109.

[0067] The processor 108 implements the functions of the input mode selection unit 3, sound input unit 4a, keyboard input unit 4b, notification unit 5, mode acquisition unit 7, display control unit 8, character string temporary storage unit 9, operation acceptance unit 11, character management unit 13 and trajectory determination unit 14 of the terminal 2 by reading and executing the program stored in the memory 109.

[0068] For example, the terminal 2 has a memory 109 for storing a program, which, when executed by the processor 108, results in the execution of Figure 3 The processing of step ST1 to step ST12 in the flowchart shown.

[0069] These programs enable the computer to execute steps or methods of the input mode selection unit 3, voice input unit 4a, keyboard input unit 4b, notification unit 5, mode acquisition unit 7, display control unit 8, character string temporary storage unit 9, operation acceptance unit 11, character management unit 13 and trajectory determination unit 14.

[0070] In addition, the memory 109 can also be a computer-readable storage medium storing a program, which is used to enable the computer to function as an input mode selection unit 3, a sound input unit 4a, a keyboard input unit 4b, a notification unit 5, a mode acquisition unit 7, a display control unit 8, a character string temporary storage unit 9, an operation acceptance unit 11, a character management unit 13 and a trajectory determination unit 14.

[0071] The memory 109 corresponds to, for example, RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable ROM), EEPROM (Electrically-EPROM), and other non-volatile or volatile semiconductor memories, magnetic disks, floppy disks, optical disks, high-density disks, mini disks, DVDs, and the like.

[0072] Alternatively, part of the functions of the input mode selection unit 3, sound input unit 4a, keyboard input unit 4b, notification unit 5, mode acquisition unit 7, display control unit 8, character string temporary storage unit 9, operation acceptance unit 11, character management unit 13 and trajectory determination unit 14 of the terminal 2 may be implemented by dedicated hardware, and the remaining part may be implemented by software or firmware.

[0073] For example, the input mode selection unit 3, the sound input unit 4a, the keyboard input unit 4b, the notification unit 5, the mode acquisition unit 7, and the display control unit 8 realize their functions by the processing circuit 107 as dedicated hardware, and the character string temporary storage unit 9, the operation receiving unit 11, the character management unit 13, and the trajectory determination unit 14 realize their functions by the processor 108 reading and executing the program stored in the memory 109. In this way, the processing circuit can realize the above functions by hardware, software, firmware, or a combination thereof.

[0074] As described above, in the terminal 2 of the first embodiment, the display control unit 8 generates display information for displaying the input mode notified from the terminal 2 of the communication partner side on the display area, and displays the input mode notified from the terminal 2 of the communication partner side on the display 10 according to the generated display information. The user has a rough understanding of the input time of each of the various input modes based on experience. Therefore, by the terminal 2 displaying the input mode notified from the terminal 2 of the communication partner side on the display 10, the user can, for example, estimate the input time of the information in the terminal 2 of the communication partner side. As a result, the realism of the communication based on the communication between the terminals 2 can be improved.

[0075] In the terminal 2 of the first embodiment, the display control unit 8 generates display information for displaying the character string received input along the curve input to the display area of ​​the display 10. The display 10 displays the character string arranged along the curve on the screen according to the above display information. Thus, the character string arranged along the curve drawn on the screen of the display 10 can be displayed.

[0076] Implementation method 2.

[0077] Fig. 9 2 is a block diagram showing the structure of a terminal 2A according to Embodiment 2. Fig. 9 In the Figure 2 The same structural elements are marked with the same reference numerals and their descriptions are omitted. Figure 1 The input display system 1 shown in the figure has terminals 2 each having Fig. 9The terminal 2A includes an input mode selection unit 3, a voice input unit 4a, a keyboard input unit 4b, a notification unit 5, a communication unit 6, a mode acquisition unit 7, a display control unit 8, a character string temporary storage unit 9, a display 10, an operation reception unit 11, a character management DB 12, a character management unit 13, a trajectory determination unit 14, an operation mode selection unit 17, a graphic drawing unit 18a, an image selection unit 18b, a shooting processing unit 18c, and various processing units 18d.

[0078] The operation mode selection unit 17 accepts selection of an operation mode from a plurality of operation modes executable by the terminal 2A. For example, the operation mode selection unit 17 accepts selection of any of the graphics drawing mode, the image selection mode, the shooting mode, and the various processing modes based on the selection operation (3). For example, the operation mode selection unit 17 accepts a selection operation using a pointing device and activates a function execution unit corresponding to the operation mode selected by the graphics drawing unit 18a, the image selection unit 18b, the shooting processing unit 18c, and the various processing units 18d.

[0079] The function execution unit executes a function corresponding to the operation mode selected by the operation mode selection unit 17. The function execution unit provided in the terminal 2A includes, for example, a graphics drawing unit 18a, an image selection unit 18b, a photographing processing unit 18c, and various processing units 18d. The graphics drawing unit 18a is a function execution unit corresponding to the graphics drawing mode, and accepts a line drawing operation using a touch panel. For example, the graphics drawing unit 18a is implemented by a processor provided in the terminal 2A executing a drawing application.

[0080] The image selection unit 18b is a function execution unit corresponding to the image selection mode, and accepts the selection of image information using the touch panel from the plurality of image information stored in the memory. For example, the image selection unit 18b displays selection candidates from the plurality of image information stored in the memory on the screen of the display 10, and accepts the selection of image information from the selection candidates using the touch panel. The memory may be provided in the terminal 2A, or may be an external storage device capable of communication connection with the terminal 2A.

[0081] The photographing processing unit 18c is a function execution unit corresponding to the photographing mode, and photographs through the camera provided by the terminal 2A. For example, the photographing processing unit 18c receives operation input using a touch panel, and photographs the subject by controlling the camera provided by the terminal 2A. The photographed image can be displayed on the display 10, and can also be sent to the terminal 2A on the other side of the communication using the communication unit 6. The photographing processing unit 18c can photograph static images and dynamic images.

[0082] The various processing units 18d are function execution units corresponding to various processing modes, and perform various processing. Various processing is processing other than the graphic drawing mode, the image selection mode, and the shooting mode, for example, a process of translating a character string input by voice or keyboard into a specified language. The various processing units 18d accept a language designation operation and a translation start operation using the touch panel, and perform the translation of the character string. The translation result can be displayed on the display 10, and can also be sent to the terminal 2A on the communication partner side using the communication unit 6.

[0083] Fig.10 This is a flowchart showing the input display method of the second embodiment.

[0084] The operation mode selection unit 17 accepts the selection of the operation mode (step ST1a). Next, the notification unit 5 notifies the terminal 2A on the other side of the communication connection via the communication unit 6 of the operation mode selected by the operation mode selection unit 17 (step ST2a). For example, when the selection of the graphic drawing mode is accepted, the graphic drawing unit 18a performs graphic drawing (step ST3a). When the selection of the image selection mode is accepted, the image selection unit 18b performs image selection (step ST3a). When the selection of the shooting mode is accepted, the shooting processing unit 18c performs camera shooting (step ST3a). When the selection of various processing modes is accepted, the various processing units 18d perform various processing (step ST3a).

[0085] If the function execution unit that has accepted the selection of the operation mode in the graphic drawing unit 18a, the image selection unit 18b, the imaging processing unit 18c, and the various processing units 18d has not completed the execution of the function (step ST4a; No), it continues to execute the function. In the case where the execution of the function is completed (step ST4a; Yes), the function execution unit that has accepted the selection of the operation mode outputs information indicating the completion of the execution of the function to the operation mode selection unit 17. The operation mode selection unit 17 outputs information indicating the completion of the execution of the function to the notification unit 5. The notification unit 5 is connected to the communication unit 6 via the communication unit, thereby notifying the terminal 2A on the other side of the communication of the information indicating the completion of the execution of the function (step ST5a).

[0086] The mode acquisition unit 7 checks whether the operation mode selected by the terminal 2A on the communication partner side is acquired (step ST6a). For example, the mode acquisition unit 7 performs communication connection via the communication unit 6, thereby receiving information indicating the operation mode from the terminal 2A on the communication partner side. The information indicating the operation mode acquired by the mode acquisition unit 7 is output to the display control unit 8.

[0087] The display control unit 8 generates display information for displaying the acquired operation mode on the display 10 (step ST7a). The display 10 displays the information indicating the operation mode notified from the terminal 2A on the communication partner side according to the display information generated by the display control unit 8 (step ST8a). The user can recognize which operation mode function is being executed in the terminal 2A on the communication partner side by visually confirming the operation mode displayed on the display 10.

[0088] For example, by including the depicted information depicted in the graphic depiction mode, the image information selected in the image selection mode, and the captured image information captured in the shooting mode in the message sent to the communication partner, the communication unit 6 can be used to send the above information to the terminal 2A on the communication partner side. In the graphic depiction mode, the depicted information is generated by drawing a line using a touch panel. On the other hand, in the image selection mode, the image information is obtained only by selecting from the selection candidates, and the shooting mode obtains the captured image information only by shooting with a camera. That is, the image selection mode and the shooting mode obtain the sent information in a shorter time compared to the graphic depiction mode. Users who are used to using smartphones, tablet terminals or PCs have experienced the difference in the time required to obtain information for each operation mode as described above. Therefore, the terminal 2A displays the operation mode notified from the terminal 2A on the communication partner side on the display 10. As a result, the user can roughly estimate the completion time of the operation in the terminal 2A on the communication partner side.

[0089] The mode acquisition unit 7 confirms whether the execution of the function is completed in the terminal 2A on the communication partner side (step ST9a). For example, when the communication unit 6 receives information indicating that the execution of the function is completed from the terminal 2A on the communication partner side (step ST9a; yes), the mode acquisition unit 7 notifies the display control unit 8 that the execution of the function is completed in the terminal 2A on the communication partner side. In addition, if the communication unit 6 does not receive information indicating that the execution of the function is completed from the terminal 2A on the communication partner side (step ST9a; no), the mode acquisition unit 7 repeats the above confirmation process.

[0090] When the execution of the function is completed in the terminal 2A on the communication partner side, the display control unit 8 generates display information for not displaying the operation mode notified from the terminal 2A on the communication partner side, and outputs it to the display 10. The display 10 clears (does not display) the display of the operation mode notified from the terminal 2A on the communication partner side from the screen according to the display information generated by the display control unit 8 (step ST10a).

[0091] In addition, the functions of the input mode selection unit 3, the sound input unit 4a, the keyboard input unit 4b, the notification unit 5, the communication unit 6, the mode acquisition unit 7, the display control unit 8, the operation reception unit 11, the character management unit 13, the trajectory determination unit 14, the operation mode selection unit 17, the graphic drawing unit 18a, the image selection unit 18b, the shooting processing unit 18c, and the various processing units 18d of the terminal 2A are realized by the processing circuit. That is, the terminal 2A has a circuit for executing Fig.10 The processing circuit for the processing of steps ST1a to ST10a shown in FIG. Figure 7 The dedicated hardware processing circuit 107 shown may also be a processor that performs Figure 8 The memory 109 is shown storing programs for the processor 108 .

[0092] Fig.11 1 is an explanatory diagram showing a screen 10A of a terminal 2A on the sending side and a screen 10B of a terminal 2A on the receiving side that depicts a graphic. The operation mode selection unit 17 is implemented as a user interface using the display 10 and the touch panel. For example, the display control unit 8 generates display information for displaying a button image corresponding to each operation mode. The display 10 displays the button image according to the display information generated by the display control unit 8. When the touch panel detects an operation of pressing the button image, it outputs the detected operation information to the operation mode selection unit 17. The operation mode selection unit 17 accepts the selection of the operation mode based on the operation information.

[0093] When the operation mode selection unit 17 accepts the selection of the graphic drawing mode, the notification unit 5 notifies the receiving terminal 2A of the accepted selected graphic drawing mode via the communication unit 6. The receiving terminal 2A displays the operation mode (graphic drawing mode) notified from the sending terminal 2A on the screen 10B. For example, text information such as "drawing input" is displayed on the screen 10B. The communication unit 6 is used to send a message containing the drawing information drawn in the sending terminal 2A to the receiving terminal 2A. The user of the receiving terminal 2A can roughly estimate the completion time of the operation in the sending terminal 2A by referring to the operation mode displayed on the screen 10B.

[0094] Fig.12It is an explanatory diagram showing a screen 10A of a terminal 2A on the sending side and a screen 10B of a terminal 2A on the receiving side performing an image selection operation. When the operation mode selection unit 17 accepts the selection of the image selection mode, the notification unit 5 notifies the terminal 2A on the receiving side of the image selection mode that has been selected via the communication unit 6. The terminal 2A on the receiving side displays the operation mode (image selection mode) notified from the terminal 2A on the sending side on the screen 10B. For example, text information such as "image selection in progress" is displayed on the screen 10B. Using the communication unit 6, a message containing the image information selected in the terminal 2A on the sending side is sent to the terminal 2A on the receiving side. The user of the terminal 2A on the receiving side can roughly estimate the completion time of the operation in the terminal 2A on the sending side by referring to the operation mode displayed on the screen 10B.

[0095] Fig.13 It is an explanatory diagram showing a screen 10A of a terminal 2A on the sending side and a screen 10B of a terminal 2A on the receiving side that perform camera photography. When the operation mode selection unit 17 accepts the selection of the photography mode, the notification unit 5 notifies the terminal 2A on the receiving side of the accepted selected photography mode via the communication unit 6. The terminal 2A on the receiving side displays the operation mode (photography mode) notified from the terminal 2A on the sending side on the screen 10B. For example, a text message such as "Camera photography" is displayed on the screen 10B. The communication unit 6 is used to send a message containing the captured image information obtained in the terminal 2A on the sending side to the terminal 2A on the receiving side. The user of the terminal 2A on the receiving side can roughly estimate the completion time of the operation in the terminal 2A on the sending side by referring to the operation mode displayed on the screen 10B.

[0096] Fig.14 This is an explanatory diagram showing a screen 10A of a terminal 2A on the sending side, which shows tool selection, drawing of a graphic using the selected tool, and a screen 10B of a terminal 2A on the receiving side. The graphic drawing operation includes a tool selection operation that accepts selection of the thickness of the drawn line, the type of line, the display effect, etc. When the operation mode selection unit 17 accepts the selection of the graphic drawing mode and accepts the selection of the tool selection operation mode, the notification unit 5 notifies the receiving side terminal 2A of the accepted selected operation mode via the communication unit 6. The receiving side terminal 2A displays the operation mode (tool selection operation mode) notified from the sending side terminal 2A on the screen 10B. For example, if Fig.14 As shown in the upper part of , text information such as “Tool is being selected” is displayed on the screen 10B.

[0097] For example, the tool selection operation is performed by operating the tool box 20 displayed on the screen 10A using the touch panel. The tool box 20 displays the thickness of the drawn line, the line type, the display effect, etc. as selection candidates. When the tool selection operation is completed and the graphic drawing is started, the notification unit 5 notifies the receiving terminal 2A of the graphic drawing mode via the communication unit 6. The receiving terminal 2A sets the text information such as "Tool selection is in progress" to non-display, and displays the graphic drawing mode notified from the sending terminal 2A on the screen 10B. For example, Fig.14 As shown in the middle layer of , in screen 10B, text information such as “drawing input in progress” is displayed.

[0098] like Fig.14 As shown in the lower layer of , when the drawing of the graphics is completed on the screen 10A of the terminal 2A on the sending side, the notification unit 5 notifies the terminal 2A on the receiving side of the completion of the drawing of the graphics via the communication unit 6. When the terminal 2A on the receiving side receives the notification that the drawing of the graphics is completed in the terminal 2A on the sending side, the text information such as "drawing input" is set not to be displayed on the screen 10B. As a result, the user of the terminal 2A on the receiving side can infer that the editing or generation of the message in the terminal 2A on the sending side has been completed.

[0099] Furthermore, when the terminal 2A on the communication partner side notifies the user of an input mode different from the previous one before the user is notified that the information input in the terminal 2A on the communication partner side is completed, or when the terminal 2A on the communication partner side notifies the user of an input mode before the user is notified that the operation in the terminal 2A on the communication partner side is completed, the display control unit 8 generates display information for displaying the newly notified input mode in the display area of ​​the display 10, and displays the newly notified input mode on the display 10 according to the generated display information. Thus, the change of the input mode in the terminal 2A is successively notified, so that the user can estimate the waiting time for the message input based on the change of the display of the input mode on the display 10.

[0100] Furthermore, when an operation mode different from the previous one is notified from the terminal 2A on the communication partner side before the completion of the operation in the terminal 2A on the communication partner side is notified, or when an operation mode is notified from the terminal 2A on the communication partner side before the completion of the information input in the terminal 2A on the communication partner side is notified, the display control unit 8 generates display information for displaying the newly notified operation mode on the display 10, and displays the newly notified operation mode on the display 10 according to the generated display information. Thus, the change of the operation mode in the terminal 2A is successively notified, so that the user can estimate the waiting time until the operation is completed based on the change of the display of the operation mode on the display 10.

[0101] As described above, in the terminal 2A of the second embodiment, the display control unit 8 generates display information for displaying the operation mode notified from the terminal 2A on the communication partner side in the display area, and the display 10 displays the operation mode notified from the terminal 2A on the communication partner side according to the generated display information. The user has a rough grasp of the execution time of each of the various operation modes based on experience. Therefore, the user can infer the completion time of the operation in the terminal 2A on the communication partner side by displaying the operation mode notified from the terminal 2A on the communication partner side on the display 10 by the terminal 2A. As a result, the realism of the communication based on the communication between the terminals 2A can be improved.

[0102] Furthermore, the embodiments may be combined or arbitrary components of the embodiments may be modified, or arbitrary components may be omitted in the embodiments.

[0103] Industrial Applicability

[0104] The terminal of the present disclosure can be used, for example, in a system in which terminals communicate with each other through communication.

[0105] Description of Reference Numerals

[0106] 1 Input display system, 2, 2A terminal, 3 Input mode selection unit, 3a, 3b button image, 4a Sound input unit, 4b Keyboard input unit, 5 Notification unit, 6 Communication unit, 7 Mode acquisition unit, 8 Display control unit, 9 Character string temporary storage unit, 10 Display, 10A, 10B screen, 11 Operation acceptance unit, 12 Character management database (DB), 13 Character management unit, 14 Trajectory determination unit, 15 Keyboard, 17 Operation mode selection unit, 18a Graphic drawing unit, 18b Image selection unit, 18c Photo processing unit, 18d Various processing units, 20 Toolbox, 100 Input interface, 101 Output interface, 102 Display, 103 Communication interface, 104 Communication device, 105 Input and output interface, 106 Storage device, 107 Processing circuit, 108 Processor, 109 Memory.

Claims

1. A terminal, characterized in that: The terminal has: an input mode selection unit that receives selection of an input mode from a voice input mode and a character input mode; a notification unit configured to notify a terminal on the other party side of the input mode selected by the input mode selection unit; an input accepting unit that accepts information input based on the input mode selected by the input mode selecting unit; a mode acquisition unit that acquires an input mode selected from a voice input mode and a character input mode notified from a terminal on the communication partner side; as well as a display control unit for generating display information for displaying information inputted by the input accepting unit in a display area of ​​a display, The display control unit generates display information for displaying the acquired input mode in the display area when the mode acquisition unit acquires the input mode notified from the terminal on the communication partner side, and displays the input mode notified from the terminal on the communication partner side on the display according to the generated display information. The terminal also has: an operation mode selection unit that accepts selection of an operation mode from a plurality of operation modes that respectively represent a plurality of operations performed by a user on the terminal, wherein the plurality of operations are operations other than voice and character input; as well as a function execution unit that executes a function corresponding to the operation mode selected by the operation mode selection unit; the notification unit notifies the terminal on the communication partner side of the operation mode selected by the operation mode selection unit; The operation mode selection unit accepts selection of an operation mode from among an operation mode in which a user draws graphics on a screen of a terminal, an operation mode in which a user selects image information from a plurality of image information via the terminal, and an operation mode in which a user takes a picture using a camera of the terminal. The function execution unit is a graphics drawing unit that executes graphics drawing, an image selection unit that selects the image information from a plurality of image information, and a photography processing unit that performs photography by a camera.

2. The terminal according to claim 1, characterized in that The display control unit generates display information for displaying the character string inputted by the input accepting unit along a curved line in a display region of the display, and displays the character string arranged along the curved line on the display according to the generated display information.

3. The terminal according to claim 1, characterized in that: The input accepting unit is a voice input unit that accepts input of a character string corresponding to a voice through voice recognition, and a keyboard input unit that accepts input of a character string using a software keyboard or a hardware keyboard.

4. The terminal according to claim 1, characterized in that: When the display control unit is notified that the information input in the terminal on the communication partner side is completed, it generates display information for preventing the input mode notified from the terminal on the communication partner side from being displayed, and according to the generated display information, the input mode notified from the terminal on the communication partner side is not displayed in the display.

5. The terminal according to claim 1, characterized in that: When the display control unit is notified that an operation in the terminal on the communication partner side is completed, it generates display information for preventing the operation mode notified from the terminal on the communication partner side from being displayed, and according to the generated display information, the operation mode notified from the terminal on the communication partner side is not displayed in the display.

6. The terminal according to claim 1, characterized in that: The display control unit generates display information for displaying the newly notified input mode on the display area, and displays the newly notified input mode on the display according to the generated display information, when the terminal on the communication partner side is notified of an input mode different from the previous input mode before being notified that the information input in the terminal on the communication partner side is completed, or when the terminal on the communication partner side is notified of an input mode before being notified that the operation in the terminal on the communication partner side is completed. The display control unit generates display information for displaying the newly notified operation mode on the display area, and displays the newly notified operation mode on the display according to the generated display information, when the terminal on the communication side is notified of an operation mode different from the previous operation mode before being notified of completion of the operation in the terminal on the communication side, or when the terminal on the communication side is notified of an operation mode before being notified of completion of information input in the terminal on the communication side.

7. An input display method, characterized in that: The input display method comprises the following steps: The input mode selection unit receives a selection of an input mode from a voice input mode and a character input mode; a notification unit notifying a terminal on the communication partner side of the input mode selected by the input mode selection unit; The input receiving unit receives information input in the input mode selected by the input mode selection unit; The mode acquisition unit acquires an input mode selected from a voice input mode and a character input mode notified from the terminal on the communication partner side; as well as The display control unit generates display information for displaying the information inputted by the input accepting unit in a display area of ​​a display, The display control unit generates display information for displaying the acquired input mode in the display area when the mode acquisition unit acquires the input mode notified from the terminal on the communication partner side, and displays the input mode notified from the terminal on the communication partner side on the display according to the generated display information. The input display method further comprises the following steps: The operation mode selection unit accepts selection of an operation mode from a plurality of operation modes respectively representing a plurality of operations performed by a user on the terminal, wherein the plurality of operations are operations other than voice and character input; as well as The function execution unit executes a function corresponding to the operation mode selected by the operation mode selection unit. the notification unit notifies the terminal on the communication partner side of the operation mode selected by the operation mode selection unit; The operation mode selection unit accepts selection of an operation mode from among an operation mode in which a user draws graphics on a screen of a terminal, an operation mode in which a user selects image information from a plurality of image information via a terminal, and an operation mode in which a user takes a picture using a camera of a terminal. The function execution unit is a graphics drawing unit that executes graphics drawing, an image selection unit that selects the image information from a plurality of image information, and a photography processing unit that performs photography by a camera.

8. An input display system that displays information on each other's displays through communication between terminals, characterized in that: The input display system comprises: A communication unit that communicates with a terminal on the other side of the communication; an input mode selection unit that receives selection of an input mode from a voice input mode and a character input mode; a notification unit that transmits and notifies the input mode selected by the input mode selection unit to the terminal on the communication partner side through the communication unit; An input receiving unit that receives information input in the input mode selected by the input mode selection unit; a mode acquisition unit that acquires an input mode selected from a voice input mode and a character input mode received from a terminal on the communication partner side via the communication unit; as well as a display control unit configured to generate display information for displaying information inputted by the input accepting unit in a display area of ​​the display, The display control unit generates display information for displaying the acquired input mode in the display area when the mode acquisition unit acquires the input mode notified from the terminal on the communication partner side, and displays the input mode notified from the terminal on the communication partner side on the display according to the generated display information. The input display system further comprises: an operation mode selection unit that accepts selection of an operation mode from a plurality of operation modes that respectively represent a plurality of operations performed by a user on the terminal, wherein the plurality of operations are operations other than voice and character input; as well as a function execution unit that executes a function corresponding to the operation mode selected by the operation mode selection unit; the notification unit notifies the terminal on the communication partner side of the operation mode selected by the operation mode selection unit; The operation mode selection unit accepts selection of an operation mode from among an operation mode in which a user draws graphics on a screen of a terminal, an operation mode in which a user selects image information from a plurality of image information via the terminal, and an operation mode in which a user takes a picture using a camera of the terminal. The function execution unit is a graphics drawing unit that executes graphics drawing, an image selection unit that selects the image information from a plurality of image information, and a photography processing unit that performs photography by a camera.