Input display control device, input display control method and input display system

Abstract

When the display control unit (12) determines that a portion of the character string obtained by the character string obtaining unit (10) cannot be displayed along the curve because the length of the curve represented by the curve information obtained by the curve information obtaining unit (9) is too short, the display control unit (12) determines the shape of the extended portion of the curve based on the shape of the curve represented by the curve information obtained by the curve information obtaining unit (9), extends the curve, and generates display information for displaying the portion of the character string along the curve of the extended portion.

Description

[0001] This application is a divisional application of the invention patent application with the application date of February 8, 2016, application number 201680080670.6, and invention name "Input display control device, input display control method and input display system". Technical Field

[0002] The present invention relates to an input display control device, an input display control method and an input display system for generating display information for displaying a character string. Background Art

[0003] The following Patent Document 1 discloses a display device that displays a character string along a line such as a curve.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Laid-Open No. 10-124033 Summary of the invention

[0007] Problems to be solved by the invention

[0008] The conventional display device is configured as described above, so if the length of the curve is longer than the length of the string, the entire string can be displayed along the curve. However, if the length of the curve is shorter than the length of the string, there is a problem that a part of the string cannot be displayed along the curve.

[0009] The present invention is made to solve the above-mentioned problems, and its object is to obtain an input display control device, an input display control method and an input display system that can display the entire character string along the curve even when the length of the acquired curve is shorter than the length of the character string.

[0010] Means for solving problems

[0011] The input display control device of the present invention comprises: a curve information acquisition unit, which acquires curve information representing a curve; a character string acquisition unit, which acquires a character string; and a display control unit, which generates display information for displaying the character string acquired by the character string acquisition unit along the curve represented by the curve information acquired by the curve information acquisition unit. When the display control unit determines that a part of the character string acquired by the character string acquisition unit cannot be displayed along the curve because the length of the curve represented by the curve information acquired by the curve information acquisition unit is short, the display control unit determines the shape of an extended portion of the curve based on the shape of the curve, extends the curve, and generates display information for displaying a part of the character string along the curve of the extended portion.

[0012] Effects of the Invention

[0013] According to the present invention, when the length of the curve represented by the curve information obtained by the curve information obtaining unit is short and it is determined that a part of the character string obtained by the character string obtaining unit cannot be displayed along the curve, the display control unit determines the shape of the extended part of the curve according to the shape of the curve, extends the curve, and generates display information for displaying a part of the character string along the curve of the extended part. Therefore, it has the effect of being able to display the entire character string along the curve even when the length of the obtained curve is shorter than the length of the character string. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 1 is a block diagram showing an input display system including an input display control device according to Embodiment 1 of the present invention.

[0015] Figure 2 This is a hardware configuration diagram of the input display system according to the first embodiment of the present invention.

[0016] Figure 3 This is a hardware configuration diagram of a computer when the input display system is implemented by software, firmware, etc.

[0017] Figure 4 This is a flowchart (part 1) showing the processing contents of the input display system according to the first embodiment of the present invention, that is, the input display control method.

[0018] Figure 5 This is a flowchart (part 2) showing the processing contents of the input display system according to the first embodiment of the present invention, that is, the input display control method.

[0019] Figure 6 2 is an explanatory diagram showing a character attribute table stored in the character management DB 2 .

[0020] Figure 7 It is an explanatory diagram showing a display process in the case where a graph image is output before a character string indicating a recognition result of a voice is output.

[0021] Figure 8 It is an explanatory diagram showing a display process in the case where a character string indicating a result of voice recognition is output before a graph image is output.

[0022] Fig. 9A This is an explanatory diagram showing that if the length of the curve is longer than the length of the character string, all characters constituting the character string can be displayed in an overlapping manner on the curve image. Fig. 9B This is an explanatory diagram showing that if the length of the curve is shorter than the length of the character string, some characters constituting the character string cannot be displayed in an overlapping manner on the curve image.

[0023] Fig. 10A1 is an explanatory diagram showing position coordinates of the leading end and the trailing end of a curve used to calculate the length L of the curve when the shape of the curve is a straight line. Fig. 10B 1 is an explanatory diagram showing position coordinates of the front end, the terminal end, and the middle point of a curve used to calculate the length L of the curve when the shape of the curve is curved.

[0024] Fig.11 It is an explanatory diagram showing the processing contents of the extended curve shape determination unit 14 of the display control unit 12 .

[0025] Fig. 12A is an explanatory diagram showing an example in which the shape of the extended curve is the shape of the tangent line of the terminal portion of the original curve. Fig. 12B is an explanatory diagram showing an example in which the shape of the extended curve has the same curvature as the shape of the original curve. Fig. 12C is an explanatory diagram showing an example in which the shape of the extended curve is a wave line shape having the same period as the shape of the original curve. Fig.12D This is an explanatory diagram showing an example in which the shape of the extended curve is an arc shape.

[0026] Fig.13 It is an explanatory diagram showing the processing contents of the curve extending unit 17 of the display control unit 12 .

[0027] Fig.14 1 is a block diagram showing an input display system including an input display control device according to a second embodiment of the present invention.

[0028] Fig.15 This is a hardware configuration diagram of an input display system according to Embodiment 2 of the present invention.

[0029] Fig.16 1 is an explanatory diagram showing an example of display processing in the display processing unit 53 of the display control unit 51 .

[0030] Fig.17 1 is an explanatory diagram showing an example of display processing in the display processing unit 53 of the display control unit 51 .

[0031] Fig.18 It is an explanatory diagram showing an example of displaying a character string indicating a translation result.

[0032] Fig.19 It is an explanatory diagram showing an example of displaying a character string indicating a translation result.

[0033] Fig. 20 This is an explanatory diagram showing an example of display processing in the display processing unit 53 when the display 32 is composed of two back-to-back screens.

[0034] Fig.21 It is an explanatory diagram showing an example of displaying a character string indicating a translation result.

[0035] Fig. 22 1 is a block diagram showing an input display system including an input display control device according to a third embodiment of the present invention.

[0036] Fig.23 This is a hardware configuration diagram of an input display system according to Embodiment 3 of the present invention.

[0037] Fig.24 This is an explanatory diagram showing an example of a process of displaying a character string indicating a result of character recognition.

[0038] Fig.25 1 is a block diagram showing an input display system including an input display control device according to a fourth embodiment of the present invention.

[0039] Fig.26 This is an explanatory diagram showing an example of a display process of a character string indicating a translation result. DETAILED DESCRIPTION

[0040] Hereinafter, in order to explain the present invention in more detail, a mode for carrying out the present invention will be described with reference to the accompanying drawings.

[0041] Implementation Method 1

[0042] Figure 1 is a structural diagram showing an input display system including an input display control device according to Embodiment 1 of the present invention, Figure 2 This is a hardware configuration diagram of the input display system according to the first embodiment of the present invention.

[0043] exist Figure 1 and Figure 2 In the embodiment, the operation accepting unit 1 is realized by the pointing device 21 and performs processing for accepting an operation of drawing a line as a user operation.

[0044] In this embodiment 1, a touch panel is assumed to be used as the indicating device 21, but it is not limited to a touch panel, and for example, a joystick, an indicating stick (track point), a touchpad, a stylus, a data glove, a trackball, a handwriting tablet, a mouse, a light pen, a handle, etc. can be used.

[0045] The character management DB 2 is realized by, for example, the storage processing circuit 22 , and stores a character attribute table indicating the correspondence relationship among character type, character size, character height, and character width.

[0046] The character management unit 3 is implemented by, for example, the character management processing circuit 23. The character management unit 3 performs the following processing: the size of the character displayed on the display 32 is set in advance, and the character attribute table stored in the character management DB2 is referred to, and the character attribute information indicating the character height corresponding to the character size is output to the trajectory determination unit 4, and the character attribute information indicating the character height and character width corresponding to the character size is output to the input display control device 100.

[0047] The trajectory determination unit 4 is realized by, for example, the image generation processing circuit 24, and performs the following processing: determining the trajectory of the line drawn by the operation accepted by the operation acceptance unit 1, and outputting curve information representing the curve such as the length and shape of the curve representing the trajectory to the input display control device 100. It is assumed that the curve includes lines of all shapes such as straight lines and broken lines.

[0048] Then, the trajectory identification unit 4 generates a curve image which is an image showing a curve of the trajectory, and outputs the curve image to the input display control device 100. The curve image is included in the curve information and is output to the input display control device 100.

[0049] The line width of the curved line image corresponds to the character height indicated by the character attribute information output from the character management unit 3 .

[0050] The character string recognition unit 5 includes a voice detection unit 6 , a voice recognition dictionary DB 7 , and a voice recognition unit 8 , and performs a process of outputting a character string indicating a voice recognition result to the input display control device 100 as a display target character string.

[0051] The voice detection unit 6 is realized by, for example, a microphone 25 and detects the voice of the user.

[0052] The voice recognition dictionary DB7 is realized by, for example, the storage processing circuit 22 and is composed of, for example, an acoustic model that describes acoustic features of phonemes, which are small units of human speech, and a recognition dictionary that describes words to be voice recognized.

[0053] The voice recognition unit 8 is realized by, for example, a voice recognition processing circuit 26 , and is equipped with a voice recognition engine that recognizes the voice detected by the voice detection unit 6 .

[0054] That is, the sound recognition unit 8 implements the following processing: analyzes the sound detected by the sound detection unit 6, calculates the acoustic features of the sound, searches for words having acoustic features closest to the calculated acoustic features from the words recorded in the recognition dictionary of the sound recognition dictionary DB7, and outputs the character string representing the word as a character string representing the recognition result of the sound to the input display control device 100.

[0055] exist Figure 1 In FIG. 1 , an example is shown in which the voice recognition dictionary DB 7 and the voice recognition unit 8 are implemented in the character string recognition unit 5 . However, a data transmission and reception unit may be implemented instead of the voice recognition dictionary DB 7 and the voice recognition unit 8 .

[0056] The data transmission and reception unit is a network communication device such as a network card capable of transmitting and receiving data with a voice recognition server (not shown) via a communication path such as the Internet or a LAN, and transmits data representing the voice detected by voice detection unit 6 to the voice recognition server.

[0057] The voice recognition server is equipped with a voice recognition engine for recognizing voice, and upon receiving data representing voice transmitted from the data transmission and reception unit, recognizes the voice and transmits a character string representing the recognition result of the voice to the data transmission and reception unit.

[0058] The data transmission and reception unit receives a character string indicating a result of voice recognition transmitted from a voice recognition server, and then outputs the character string to the input display control device.

[0059] The input display control device 100 includes a curve information acquisition unit 9 , a character string acquisition unit 10 , a character attribute information acquisition unit 11 , and a display control unit 12 .

[0060] The curve information acquisition unit 9 is realized by an input interface circuit 27 having a LAN (Local Area Network) port or a USB (Universal Serial Bus) port, for example, and performs processing for acquiring the curve information output from the trajectory determination unit 4 .

[0061] The character string acquisition unit 10 is realized by, for example, the input interface circuit 27 , and performs processing for acquiring the character string output from the character string recognition unit 5 .

[0062] The character attribute information acquisition unit 11 is realized by, for example, the input interface circuit 27 , and performs processing for acquiring character attribute information output from the character management unit 3 .

[0063] The display control unit 12 includes a length comparison unit 13, an extended curve shape determination unit 14, a curve extension unit 17, and a display processing unit 18, and performs the following processing: generates display information for displaying the character string output from the character string recognition unit 5 along the curve represented by the curve information output from the trajectory determination unit 4, and displays the character string along the curve according to the display information. In other words, the display control unit 12 performs the following processing: generates display information for displaying the character string output from the character string recognition unit 5 superimposed on the curve image included in the curve information acquired by the curve information acquisition unit 9, and displays the character string superimposed on the curve image according to the display information.

[0064] Furthermore, the display control unit 12 implements the following processing: when it is determined that a portion of the character string output from the character string recognition unit 5 cannot be displayed along the curve because the length of the curve represented by the curve information obtained by the curve information acquisition unit 9 is short, the shape of the extended portion of the curve is determined based on the shape of the curve, the curve is extended, and display information for displaying a portion of the character string along the curve of the extended portion is generated, and the portion of the character string is displayed based on the display information.

[0065] Furthermore, the display control unit 12 performs a process of, after displaying the character string superimposed on the graph image, generating display information for erasing the display of the graph image, and erasing the graph image in accordance with the display information.

[0066] The length comparison unit 13 is realized by, for example, a length comparison processing circuit 28 , and performs a process of comparing the length of the curve indicated by the curve information acquired by the curve information acquisition unit 9 with the length of the character string acquired by the character string acquisition unit 10 .

[0067] The extended curve shape determination unit 14 is implemented by, for example, a shape determination processing circuit 29, and includes a curve storage unit 15 and a shape determination processing unit 16. The extended curve shape determination unit 14 performs the following processing: if the comparison result of the length comparison unit 13 indicates that the length of the curve is shorter than the length of the character string, the shape of the curve of the extended portion, i.e., the extended curve, is determined based on the shape of the curve represented by the curve information acquired by the curve information acquisition unit 9.

[0068] The curve storage unit 15 stores a plurality of curves having different shapes in advance.

[0069] The shape determination processing unit 16 performs the following processing: comparing the shapes of the multiple curves stored in the curve storage unit 15 with the shape of the curve represented by the curve information obtained by the curve information acquisition unit 9, selecting the shape of the curve corresponding to the curve represented by the curve information obtained by the curve information acquisition unit 9 from the shapes of the multiple curves stored in the curve storage unit 15, and determining the shape of the extended curve based on the shape of the selected curve.

[0070] The curve extension unit 17 is implemented, for example, by a curve extension processing circuit 30, and performs the following processing: if the comparison result of the length comparison unit 13 indicates that the length of the curve is shorter than the length of the character string, an extended curve whose shape is determined by the extended curve shape determination unit 14 is added to the terminal of the curve represented by the curve information obtained by the curve information acquisition unit 9.

[0071] The display processing unit 18 is realized by, for example, a display processing circuit 31, and performs processing such as generating display information for causing the display 32 to display a character string along the curve represented by the curve information obtained by the curve information obtaining unit 9, and performing processing such as generating display information for causing the display 32 to display a character string obtained by the character string obtaining unit 10 along the curve represented by the curve information obtained by the curve information obtaining unit 9. In other words, the display processing unit 18 performs processing such as generating display information for causing the display 32 to display a character string superimposed on a curve image, and performing processing such as superimposing on a curve image obtained by the curve information obtaining unit 9, and performing processing such as display information for causing the display 32 to display a character string superimposed on a curve image.

[0072] Furthermore, the display processing unit 18 performs the following processing: if the comparison result of the length comparison unit 13 indicates that the length of the curve is shorter than the length of the character string, the following display information is generated and a part of the character string is displayed on the display 32 according to the display information. The display information is used to display a part of the character string that cannot be displayed overlappingly on the curve image on the display 32 along the extended curve added by the curve extension unit 17.

[0073] Furthermore, the display processing unit 18 performs a process of generating display information for erasing the display of the graph image after causing the display 32 to display the character string, and causing the display 32 to erase the graph image according to the display information.

[0074] The character string temporary storage unit 19 is implemented by, for example, a storage processing circuit 22 , and temporarily stores the character string acquired by the character string acquisition unit 10 .

[0075] exist Figure 1 In the above, it is assumed that the operation receiving unit 1, the character management DB 2, the character management unit 3, the trajectory determination unit 4, the character string recognition unit 5, the curve information acquisition unit 9, the character string acquisition unit 10, the character attribute information acquisition unit 11, the length comparison unit 13, the extended curve shape determination unit 14, the curve extension unit 17, the display processing unit 18 and the character string temporary storage unit 19, which are the structural elements of the input display system, are respectively Figure 2 The dedicated hardware shown, namely, the indicating device 21, the storage processing circuit 22, the character management processing circuit 23, the image generation processing circuit 24, the microphone 25, the sound recognition processing circuit 26, the input interface circuit 27, the length comparison processing circuit 28, the shape determination processing circuit 29, the curve extension processing circuit 30, the display processing circuit 31 and the display 32 are implemented.

[0076] Here, the storage processing circuit 22 is, for example, a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), a magnetic disk, a floppy disk, an optical disk, a high-density disk, a mini disk, a DVD (Digital Versatile Disc), etc.

[0077] Furthermore, the character management processing circuit 23, the image generation processing circuit 24, the sound recognition processing circuit 26, the length comparison processing circuit 28, the shape determination processing circuit 29, the curve extension processing circuit 30 and the display processing circuit 31 are, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate array), or a combination thereof.

[0078] Furthermore, the structural elements of the input display system are not limited to those implemented by dedicated hardware, and the input display system may also be implemented by software, firmware, or a combination of software and firmware.

[0079] Software and firmware are stored as programs in the memory of a computer. Computer means hardware that executes programs, such as CPU (Central Processing Unit), central processing unit, processing unit, computing unit, microprocessor, microcomputer, processor, DSP (Digital Signal Processor), etc.

[0080] Figure 3 This is a hardware configuration diagram of a computer when the input display system is implemented by software, firmware, etc.

[0081] When the input display system is implemented by software or firmware, a character management DB2, a voice recognition dictionary DB7 and a character string temporary storage unit 19 are formed on the computer's memory 41, and a program for causing the computer to execute the processing steps of the character management unit 3, the trajectory determination unit 4, the voice recognition unit 8, the curve information acquisition unit 9, the character string acquisition unit 10, the character attribute information acquisition unit 11, the length comparison unit 13, the extended curve shape determination unit 14, the curve extension unit 17 and the display processing unit 18 is stored in the memory 41, and the processor 42 executes the program stored in the memory 41.

[0082] Figure 4 and Figure 5 This is a flowchart showing the processing contents of the input display system according to the first embodiment of the present invention, that is, the input display control method.

[0083] And, in Figure 2 An example in which the structural elements of the input display system are respectively implemented by dedicated hardware is shown in Figure 3 , an example is shown in which the input display system is implemented by software or firmware, but some components of the input display system may be implemented by dedicated hardware, and the remaining components may be implemented by software or firmware.

[0084] For example, the operation accepting unit 1 and the character string recognition unit 5 can be implemented by dedicated hardware, and the character management DB 2, the character management unit 3, the trajectory determination unit 4, the input display control device 100, and the character string temporary storage unit 19 can be implemented by software or firmware. However, the combination of dedicated hardware and software is arbitrary.

[0085] Next, the operation will be described.

[0086] Figure 6 2 is an explanatory diagram showing a character attribute table stored in the character management DB 2 .

[0087] exist Figure 6 In the character attribute table, for example, the character height and character width corresponding to the character size are described for each character type. As the character type, for example, Ming Dynasty type, bold type, etc. are considered.

[0088] The character type and character size of the character displayed on the display 32 are set in advance in the character management unit 3. The character management unit 3 refers to the character attribute table stored in the character management DB 2, outputs the character attribute information indicating the character height corresponding to the character type and character size to the trajectory determination unit 4, and outputs the character attribute information indicating the character height and character width corresponding to the character type and character size to the input display control device 100 ( Figure 4 Step ST1).

[0089] For example, if the character type of the pre-set character is "A" and its character size is "10", the character attribute information indicating that the character height is "5mm" is output to the trajectory determination unit 4, and the character attribute information indicating that the character height is "5mm" and the character width is "5mm" is output to the input display control device 100.

[0090] Furthermore, if the character type of the pre-set character is "B" and its character size is "5", the character attribute information indicating that the character height is "2mm" is output to the trajectory determination unit 4, and the character attribute information indicating that the character height is "2mm" and the character width is "1.25mm" is output to the input display control device 100.

[0091] Upon receiving the character attribute information from the character management unit 3 , the trajectory determination unit 4 sets the line width of the curve image to be generated to a line width corresponding to the character height indicated by the character attribute information (step ST2 ).

[0092] For example, when the character height indicated by the character attribute information is "10mm", if the upper and lower margins of the character are set to "0.5mm", the calculation is 10+0.5+0.5=11=(mm), and the line width of the curve image is set to "11mm".

[0093] Here, an example is shown in which blanks are provided above and below the characters. However, it is also possible to make the character height indicated by the character attribute information and the line width of the curved line image coincide with each other without providing blanks.

[0094] In this embodiment 1, it is assumed that the indicating device 21 that implements the operation acceptance unit 1 is a touch panel. Therefore, when the user uses his or her finger or an input pen for the touch panel to draw a line on the touch panel (step ST3: "Yes" case), the operation acceptance unit 1 accepts the operation of drawing a line and outputs the content of the operation to the trajectory determination unit 4 (step ST4).

[0095] The output content of the line drawing operation is, for example, time series data indicating the coordinates of the position of the finger or input pen in contact with the touch panel, that is, the continuous change of the position coordinates on the touch panel.

[0096] When receiving the details of the operation of drawing a line from the operation accepting unit 1 , the trajectory determination unit 4 determines the trajectory of the line drawn by the operation based on the details of the operation (step ST5 ).

[0097] As described above, the content of the operation of drawing a line represents a continuous change in the contact position with respect to the touch panel, and therefore, the trajectory of the drawn line can be determined by connecting the contact positions in time series.

[0098] After determining the trajectory of the drawn line, the trajectory determination unit 4 generates curve information indicating the curve, such as the length and shape of the curve representing the trajectory.

[0099] Then, the trajectory determination unit 4 generates a curve image which is an image representing a curve of the trajectory, and outputs curve information including the curve image to the input display control device 100 (step ST6 ).

[0100] The line width of the curve image is the line width set in step ST2. In the above example, the line width of the curve image is set to "11 mm". Therefore, a curve image with a line width of "11 mm" is generated.

[0101] When the sound detection unit 6 detects the user's sound (in the case of "Yes" in step ST7), the sound recognition unit 8 of the character string recognition unit 5 performs the recognition process of the sound using the sound recognition dictionary DB7, thereby generating a character string representing the recognition result of the sound, and outputting the character string to the input display control device 100 (step ST8).

[0102] The sound recognition process itself performed by the sound recognition unit 8 is a well-known technique, so a detailed description is omitted. However, for example, consider the following method: Analyze the sound detected by the sound detection unit 6, calculate the acoustic characteristics of the sound, and search for the word with the acoustic characteristics closest to the calculated acoustic characteristics from the words described in the recognition dictionary of the sound recognition dictionary DB7, and output the character string representing the word as the character string representing the recognition result of the sound.

[0103] In this Embodiment 1, for the sake of easy explanation, it is assumed that the user makes a speech such as "春になったら桜を見に行きましょう" (When spring comes, let's go see the cherry blossoms). The sound recognition unit 8 outputs a character string representing "春になったら桜を見に行きましょう".

[0104] After the input display control device 100 outputs the curve information including the curve image from the trajectory determination unit 4, it causes the display 32 to display the curve image, and overlaps and displays the character string output from the character string recognition unit 5 on the curve image.

[0105] Next, the processing content of the input display control device 100 will be specifically described.

[0106] Figure 7 It is an explanatory diagram showing the display process in the case where the curve image is output before the character string representing the recognition result of the sound is output.

[0107] Figure 8 It is an explanatory diagram showing the display process in the case where the character string representing the recognition result of the sound is output before the curve image is output.

[0108] FIG. 9 is an explanatory diagram showing the character string overlapped and displayed on the curve image. Fig. 9A It shows that if the length of the curve is longer than the length of the character string, all the characters constituting the character string can be overlapped and displayed on the curve image. Fig. 9B It shows that if the length of the curve is shorter than the length of the character string, some of the characters constituting the character string cannot be overlapped and displayed on the curve image.

[0109] FIG. 10 is an explanatory diagram showing the processing content of the length comparison unit 13 of the display control unit 12. Fig. 10A Shows the position coordinates of the front end and the terminal end of the curve used to calculate the length L of the curve when the shape of the curve is a straight line. Fig. 10B Shows the position coordinates of the front end, the terminal end, and the midpoint of the curve used to calculate the length L of the curve when the shape of the curve is curved.

[0110] Fig.11 Is an explanatory diagram showing the processing content of the extended curve shape determination unit 14 of the display control unit 12.

[0111] FIG. 12 is an explanatory diagram showing an example of the shape of the extended curve determined by the extended curve shape determination unit 14 of the display control unit 12.

[0112] Fig.13 Is an explanatory diagram showing the processing content of the curve extension unit 17 of the display control unit 12.

[0113] Before the string is output from the voice recognition unit 8 of the string recognition unit 5, curve information including a curve image is output from the trajectory discrimination unit 4. Thus, when the curve information acquisition unit 9 acquires the curve information ( Figure 5 In step ST11: "Yes" case), the display processing unit 18 of the display control unit 12 generates display information for displaying the curve image, and according to this display information, as Figure 7 Shown in step A, the curve image is displayed on the display 32 (step ST12).

[0114] In this Embodiment 1, for the sake of easy explanation, it is assumed that the length of the line drawn by the user's operation is short. Therefore, the length of the curve is shorter than the length of the string representing the recognition result of the voice.

[0115] In addition, when the display processing unit 18 displays the curve image, the entire curve image can be displayed at the same time. However, the curve image can also be displayed in an animation manner that is sequentially displayed starting from the beginning side of the string to be displayed later.

[0116] For example, as Figure 7 Shown in step B, when the user makes a speech such as "春になったら桜を見に行きましょう", the display processing unit 18 receives the string representing "春になったら桜を見に行きましょう" from the voice recognition unit 8 ( Figure 5 In step ST13: "Yes" case), generates display information for overlapping and displaying the string on the curve image, and according to this display information, as Figure 7 Shown in step C, the string is overlapped and displayed on the curve image (step ST14).

[0117] However, in Figure 7 In the example, the length of the curve is shorter than the length of the string representing the voice recognition result. Therefore, the string "春になったら桜を見に行き" is overlapped and displayed on the curve image, but "ましょう" is not overlapped and displayed on the curve image.

[0118] After the string acquisition unit 10 acquires the string representing "春になったら桜を見に行きましょう" output from the voice recognition unit 8, the length comparison unit 13 of the display control unit 12 refers to the character width represented by the character attribute information acquired by the character attribute information acquisition unit 11 after being output from the character management unit 3, and calculates the length of this string ( Figure 5 step ST15).

[0119] Here, the string output from the voice recognition unit 8 is "春になったら桜を見に行きましょう", and the number of characters in this string is 16. Therefore, for example, if the character width represented by the character attribute information output from the character management unit 3 is "5mm", the length of the string becomes 80mm as shown in the following formula (1).

[0120] Length of string = 5mm × 16 = 80mm (1)

[0121] Next, the length comparison unit 13 calculates the length of the curve represented by the curve information acquired by the curve information acquisition unit 9 (step ST16).

[0122] If the shape of the curve is, for example, Fig. 10A a straight line as shown, the length L of the curve can be calculated according to the position coordinates (Xs, Ys) of the front end and the position coordinates (Xe, Ye) of the terminal end of the curve as shown in the following formula (2). In the case of the curve image displayed in Figure 7 step A, the left end of the curve is the front end of the curve, and the right end of the curve is the terminal end of the curve. These position coordinates are in the same unit as the unit representing the character width, etc. Here, the unit is mm.

[0123]

[0124] In formula (2), the calculation formula for the length L of the curve in the case where the shape of the curve is a straight line is shown. However, if the shape of the curve is, for example, Fig. 10B a curve as shown, the position coordinates (Xs, Ys) of the front end and the position coordinates (Xe, Ye) of the terminal end of the curve and the position coordinates (Xn, Yn) (n = 1,..., N: N is an integer greater than or equal to 1) of multiple intermediate points from the front end to the terminal end of the curve are used, and the shape of the curve is approximated by a straight line, whereby the length L of the curve can be calculated.

[0125] For example, when N=2, as shown in the following formula (3), the length L of the curve can be calculated based on the position coordinates of the front end of the curve (Xs, Ys), the position coordinates of the middle point (X1, Y1), the position coordinates of the middle point (X2, Y2), and the position coordinates of the terminal (Xe, Ye).

[0126]

[0127] In formula (3), Xs <X1<X2<Xe。

[0128] exist Fig. 10B In FIG. 1 , an example of N=2 is shown. However, in order to improve the approximation accuracy, N may be set to 3 or more and the position coordinates (Xn, Yn) of a plurality of intermediate points may be used.

[0129] After calculating the length of the curve, the length comparison unit 13 compares the length of the curve with the length of the character string ( Figure 5 Step ST17).

[0130] If the comparison result of the length comparison unit 13 indicates that the length of the curve is shorter than the length of the character string ( Figure 5 If "Yes" in step ST18), the extended curve shape determination unit 14 determines the shape of the extended curve, that is, the shape of the extended curve, based on the shape of the curve represented by the curve information acquired by the curve information acquisition unit 9 (step ST19).

[0131] That is, the shape determination processing unit 16 of the extended curve shape determination unit 14 compares the shapes of the plurality of curves stored in the curve storage unit 15 with the shape of the curve represented by the curve information acquired by the curve information acquisition unit 9, thereby determining the shape of the curve that has the highest degree of consistency with the shape of the curve represented by the curve information from the shapes of the plurality of curves stored in the curve storage unit 15, and selecting the shape of the curve with the highest degree of consistency. The process of comparing the shapes of the curves and determining the shape of the curve with the highest degree of consistency is a well-known technique, and therefore detailed description thereof is omitted.

[0132] The shape determination processing unit 16 selects a shape from the shapes of the plurality of curves stored in the curve storage unit 15. Fig.11 After selecting the shape of the curve shown, the shape of the extended curve is determined based on the shape of the selected curve.

[0133] For example, if the lengths of the plurality of curves stored in the curve storage unit 15 are much longer than the curve representing the trajectory of the line drawn by the operation, the shape of the portion of the selected curve that is longer than the curve represented by the curve information is determined to be the shape of the extended curve.

[0134] exist Fig.11In the example, a dotted line is used to represent a curve showing the trajectory of a line drawn by the operation, that is, a curve showing the trajectory identified by the trajectory identification unit 4, and a solid line is used to represent a curve selected with the highest degree of consistency, so that the shape of the extended portion of the curve represented by the dotted line is the same as the shape of the curve represented by the solid line.

[0135] In this embodiment 1, the following example is shown: among the shapes of multiple curves stored in the curve storage unit 15, the shape of the curve that has the highest consistency with the shape of the curve representing the trajectory identified by the trajectory identification unit 4 is selected, and the shape of the extended curve is determined based on the shape of the selected curve. However, this is just an example, and the shape of the curve of the extended part, that is, the shape of the extended curve, can also be determined by other methods.

[0136] For example, a method may be considered in which a change tendency of the curvature of the shape of the curve representing the trajectory identified by the trajectory identification unit 4 is calculated, and the shape of the extended curve is estimated based on the change tendency of the curvature.

[0137] FIG. 12 shows an example of determining the shape of the extended curve. Fig. 12A In the example, the shape of the tangent line at the terminal end of the gently curved line is set to the shape of the extended curve.

[0138] exist Fig. 12B In the example, the curvature of the entire curve is constant, and therefore the curvature of the shape of the extended curve is made the same as the curvature of the shape of the original curve which is the curve representing the trajectory determined by the trajectory determination unit 4 .

[0139] exist Fig. 12C In the example, the shape of the curve is a periodic wave line shape, so the shape of the extended curve is made into a wave line shape with the same period as the original curve.

[0140] exist Fig.12D In the example, the curve is curved in an arc shape near the terminal end, so the shape of the extended curve is made to be the same arc shape as the shape near the terminal end of the original curve.

[0141] After the extension curve shape determination unit 14 determines the shape of the extension curve, the curve extension unit 17 adds the extension curve to the terminal end of the curve represented by the curve information acquired by the curve information acquisition unit 9 to extend the curve (step ST20 ).

[0142] exist Fig.13 In the example, the length of the curve is 60 mm, the length of the string is 80 mm, and the length of the curve is 20 mm shorter than the length of the string, so the curve is extended by 20 mm.

[0143] After the curve extension unit 17 extends the curve, the display processing unit 18 generates display information for displaying a part of the character string, "ましょう", which is not displayed due to the short curve, along the extended part of the curve. Figure 7 As shown in step D, the display 32 is caused to display "ましょう", which is a part of the character string that is not displayed because the curve is short, along the curve of the extended portion (step ST21).

[0144] Here, the display 32 is shown to display a part of the character string that is not displayed along the extended curve added by the curve extension part 17, but it is also possible to Figure 7 As shown in the right figure in step D, a curve image representing the extended curve is displayed on the display 32, and a part of the character string that is not displayed is superimposed on the curve image.

[0145] After the display processing unit 18 displays all characters constituting the character string on the curve image in an overlapping manner, it generates display information for removing the curve image. In the display 32, according to the display information, Figure 7 As shown in step E, the curve image is eliminated and only the character string is displayed (step ST22).

[0146] When the display processing unit 18 erases the graph image, the entire graph image may be erased at once, or the graph image may be erased in an animation in which the graph image is erased sequentially from the head of the character string.

[0147] Here, it is assumed that after the curved image and the character string are displayed in an overlapping manner, the curved image is erased after a certain period of time. However, the curved image may be erased immediately after the display of the character string is completed.

[0148] like Figure 8 As shown in step A of FIG. 1 , the user speaks before performing the line drawing operation, thereby, before the curve information including the curve image is output from the trajectory determination unit 4, the voice recognition unit 8 of the character string recognition unit 5 outputs a character string, thereby, when the character string acquisition unit 10 acquires the character string ( Figure 5 In the case of step ST11: "No"), the display processing unit 18 stores the character string in the character string temporary storage unit 19 and waits until the curve information is output from the trajectory determination unit 4 (step ST23: "No").

[0149] Then, the user starts an operation of drawing a line, and after the operation is completed, curve information is output from the trajectory determination unit 4 .

[0150] By outputting curve information from the locus discrimination unit 4, when the curve information acquisition unit 9 acquires the curve information (in the case of "Yes" in step ST23), the display processing unit 18 reads out the string from the string temporary storage unit 19, generates display information for overlapping and displaying the string on the curve image, and according to this display information, as shown in step B of Figure 8 the string is overlapped and displayed on the curve image. That is, the display processing unit 18 overlaps and displays the curve image and the string at the same position (step ST24).

[0151] However, in the Figure 8 example, the length of the curve is shorter than the length of the string representing the voice recognition result. Therefore, "春になったら桜を見に行き" in the string is overlapped and displayed on the curve image, but "ましょう" is not overlapped and displayed on the curve image.

[0152] When the display processing unit 18 overlaps and displays the curve image and the string at the same position, the entire curve image and the string can be displayed simultaneously, but the curve image and the string can also be displayed in an animation manner by sequentially displaying them starting from the beginning side of the string.

[0153] Moreover, the curve image and the string can be sequentially displayed in a manner that emerges from below the finger in combination with the user's finger drawing.

[0154] In addition, in Figure 7 an example is shown where the user speaks after finger drawing, and in Figure 8 an example is shown where the user performs finger drawing after speaking, but finger drawing and speaking can also be performed simultaneously.

[0155] After the string acquisition unit 10 acquires the string representing "春になったら桜を見に行きましょう" output from the voice recognition unit 8, the length comparison unit 13 refers to the character width represented by the character attribute information obtained by the character attribute information acquisition unit 11 after being output from the character management unit 3, and calculates the length of the string according to the above formula (1) ( Figure 5 step ST15).

[0156] Moreover, after receiving the curve information acquired by the curve information acquisition unit 9, the length comparison unit 13 calculates the length of the curve represented by the curve information according to the above formula (2) or formula (3) (step ST16).

[0157] After the length comparison unit 13 calculates the length of the curve, it compares the length of the curve with the length of the string ( Figure 5 step ST17).

[0158] If the comparison result of the length comparison unit 13 indicates that the length of the curve is shorter than the length of the character string ( Figure 5 If "Yes" in step ST18), the extended curve shape determination unit 14 determines the shape of the extended curve, that is, the shape of the extended curve, based on the shape of the curve represented by the curve information acquired by the curve information acquisition unit 9 (step ST19).

[0159] After the extension curve shape determination unit 14 determines the shape of the extension curve, the curve extension unit 17 adds the extension curve to the terminal end of the curve represented by the curve information acquired by the curve information acquisition unit 9, thereby extending the curve (step ST20).

[0160] exist Fig.13 In the example, the length of the curve is 60 mm, the length of the string is 80 mm, and the length of the curve is 20 mm shorter than the length of the string, so the curve is extended by 20 mm.

[0161] After the curve extension unit 17 extends the curve, the display processing unit 18 generates display information for displaying a part of the character string, "ましょう", which is not displayed due to the short length of the curve, along the extended portion of the curve. Figure 8 As shown in step C, the display 32 is caused to display "ましょう", which is a part of the character string that has not been displayed, along the curve of the extended portion (step ST21).

[0162] Here, the display 32 is shown to display a part of the character string that is not displayed along the extended curve added by the curve extension part 17, but it is also possible to Figure 8 As shown in the right figure in step C of , a curve image representing the extended curve is displayed on the display 32, and a part of the character string that is not displayed is superimposed and displayed on the curve image.

[0163] After the display processing unit 18 displays all characters constituting the character string on the curve image in an overlapping manner, it generates display information for removing the curve image. In the display 32, according to the display information, Figure 8 As shown in step D, the curve image is eliminated and only the character string is displayed (step ST22).

[0164] When the display processing unit 18 erases the graph image, the entire graph image may be erased at once, or the graph image may be erased in an animation in which the graph image is erased sequentially from the head of the character string.

[0165] Here, it is assumed that after the curved image and the character string are displayed in an overlapping manner, the curved image is erased after a certain period of time. However, the curved image may be erased immediately after the display of the character string is completed.

[0166] In this first embodiment, an example is shown in which the length of the curve is shorter than the length of the character string representing the recognition result of the sound. However, when the length of the curve is greater than the length of the character string representing the recognition result of the sound, there is no need to extend the length of the curve, and all the characters constituting the character string can be displayed overlappingly on the curve image.

[0167] Therefore, if the comparison result of the length comparison unit 13 indicates that the length of the curve is greater than the length of the character string ( Figure 5 (If "No" in step ST18), the processing of steps ST19 to ST21 is not implemented, and the process is transferred to step ST22, the curve image is eliminated, and only the character string is displayed.

[0168] In the first embodiment, it is assumed that the characters constituting the character string are not separated and the length between the characters is 0 mm. The length comparison unit 13 calculates the length of the character string according to the formula (1). However, if the length between the characters is greater than 0 mm, the length of the character string may be calculated by taking the length between the characters into consideration. For example, if the length between the characters is 0.5 mm, the length of the character string becomes 87.5 mm as shown in the following formula (4).

[0169] The length of the string = (5mm × 16) + (0.5mm × 15) = 87.5mm (4)

[0170] Similar to formula (1), formula (4) shows the case where the number of characters in the character string is 16 and the character width is "5 mm".

[0171] As can be seen from the above, according to the first embodiment, when the display control unit 12 determines that a part of the character string obtained by the character string obtaining unit 10 cannot be displayed along the curve because the length of the curve represented by the curve information obtained by the curve information obtaining unit 9 is short, the display control unit 12 determines the shape of the extended part of the curve according to the shape of the curve represented by the curve information obtained by the curve information obtaining unit 9, extends the curve, and generates display information for displaying a part of the character string along the curve of the extended part. Therefore, the effect of being able to display the entire character string along the curve is achieved even when the length of the curve is shorter than the length of the character string.

[0172] Furthermore, according to the first embodiment, when the curve information including the curve image is output from the trajectory determination unit 4 before the voice recognition unit 8 of the character string recognition unit 5 outputs the character string, the input display control device 100 displays the character string output from the voice recognition unit 8 of the character string recognition unit 5 on the curve image after the curve image is displayed on the display 32. On the other hand, when the character string is output from the voice recognition unit 8 of the character string recognition unit 5 before the curve information including the curve image is output from the trajectory determination unit 4, the input display control device 100 waits until the curve information is output from the trajectory determination unit 4, and displays the curve image on the display 32 after the curve information is output from the trajectory determination unit 4, and displays the character string on the curve image on the curve image. Therefore, when the user performs the line drawing operation before speaking, the position of the character string indicating the recognition result of the voice can be grasped in advance. Therefore, the effect of the user being able to grasp the display image of the character string is achieved. Furthermore, when the user speaks before performing the line drawing operation, the curve image and the character string are displayed on the same position on the curve image after the line drawing operation is performed, so that the user can be given a highly responsive operation feeling.

[0173] In the first embodiment, the input display control device 100 displays the character string superimposed on the curve image. However, the character string may be displayed around the curve image along the curve image instead of superimposing the character string on the curve image.

[0174] Furthermore, the curved line image may be a colorless and transparent image if the user cannot confirm the curved line image. Furthermore, the curved line image may be a colored and transparent image.

[0175] When the curve image is colorless and transparent, the display processing unit 18 of the display control unit 12 can cause the display 32 to display a curve without width representing the trajectory determined by the trajectory determination unit 4 or a curve without width representing the trajectory extended by the curve extension unit 17 .

[0176] Implementation Method 2

[0177] In the first embodiment, the display 32 displays the character string acquired by the character string acquisition unit 10 along the curve represented by the curve information acquired by the curve information acquisition unit 9. However, the character string may be translated and the display 32 may display the character string representing the translation result.

[0178] Fig.14 is a structural diagram showing an input display system including an input display control device according to a second embodiment of the present invention, Fig.15 This is a hardware configuration diagram of an input display system according to Embodiment 2 of the present invention.

[0179] exist Fig.14 and Fig.15 In, with Figure 1 and Figure 2 The same reference numerals denote the same or corresponding parts, and thus their descriptions are omitted.

[0180] The translation unit 52 is realized by, for example, a translation processing circuit 34 composed of a semiconductor integrated circuit or a one-chip microcomputer having a CPU mounted thereon, and performs a process of translating the character string acquired by the character string acquisition unit 10 .

[0181] The display control unit 51 includes a length comparison unit 13, an extended curve shape determination unit 14, a curve extension unit 17, a translation unit 52 and a display processing unit 53. Figure 1 In addition to the same processing as that performed by the display control unit 12, the following processing is also performed: generating a curve for displaying a translation string having a length greater than the length of the string representing the translation result of the translation unit 52, generating display information for displaying the string representing the translation result of the translation unit 52 along the curve for displaying the translation string on the display 32, and causing the display 32 to display the string representing the translation result of the translation unit 52 according to the display information.

[0182] The display processing unit 53 of the display control unit 51 is implemented by, for example, the display processing circuit 31, and performs Figure 1 The display processing unit 18 performs the same processing.

[0183] Furthermore, the display processing unit 53 and Figure 1 Different from the display processing unit 18, the following processing is also implemented: a curve for displaying a translation string having a length greater than the length of the string representing the translation result of the translation unit 52 is generated, and the display 32 displays the string representing the translation result of the translation unit 52 along the curve for displaying the translation string.

[0184] exist Fig.14 In the above, it is assumed that the operation receiving unit 1, the character management DB 2, the character management unit 3, the trajectory determination unit 4, the character string recognition unit 5, the curve information acquisition unit 9, the character string acquisition unit 10, the character attribute information acquisition unit 11, the length comparison unit 13, the extended curve shape determination unit 14, the curve extension unit 17, the translation unit 52, the display processing unit 53 and the character string temporary storage unit 19, which are the structural elements of the input display system, are respectively Fig.15The dedicated hardware shown, namely, the indicating device 21, the storage processing circuit 22, the character management processing circuit 23, the image generation processing circuit 24, the microphone 25, the sound recognition processing circuit 26, the input interface circuit 27, the length comparison processing circuit 28, the shape determination processing circuit 29, the curve extension processing circuit 30, the display processing circuit 31, the display 32 and the translation processing circuit 34 are implemented, but the input display system can also be implemented by software, firmware, or a combination of software and firmware.

[0185] When the input display system is implemented by software or firmware, Figure 3 The computer shown in the figure has a memory 41 which is composed of a character management DB 2, a voice recognition dictionary DB 7, and a character string temporary storage unit 19, and a program for causing the computer to execute the processing steps of the operation receiving unit 1, the character management unit 3, the trajectory determination unit 4, the voice detection unit 6, the voice recognition unit 8, and the display control unit 51 is stored in the memory 41. Figure 3 The processor 42 shown may execute the program stored in the memory 41 .

[0186] Next, the operation will be described.

[0187] However, except for the translation unit 52 and the display processing unit 53, since it is the same as the above-mentioned embodiment 1, here, the processing content of the translation unit 52 and the display processing unit 53 will be mainly described.

[0188] After the character string acquisition unit 10 acquires the character string indicating the recognition result of the voice output from the character string recognition unit 5 , the translation unit 52 translates the character string and outputs the character string indicating the translation result to the length comparison unit 13 and the display processing unit 53 .

[0189] If the character string indicating the result of voice recognition is "I went to the sea in summer vacation", the character string indicating the result of its translation is, for example, "I went to the sea in summer vacation".

[0190] The display processing unit 53 and Figure 1 Similarly, the display processing unit 18 displays the character string indicating the recognition result of the voice acquired by the character string recognition unit 5 in a superimposed manner on the graph image acquired by the graph information acquisition unit 9 .

[0191] Furthermore, if the length of the curve represented by the curve information acquired by the curve information acquisition unit 9 is shorter than the length of the character string and a part of the character string cannot be displayed along the curve, Figure 1 Similarly, the display processing unit 18 causes the display processing unit 53 to display a part of the character string along the extended curve added by the curve extending unit 17 on the display 32 .

[0192] When receiving the character string indicating the translation result from the translation unit 52 , the display processing unit 53 generates a curve for displaying the translated character string having a length greater than the length of the character string indicating the translation result.

[0193] At this time, it is assumed that character attribute information indicating the character widths of characters constituting the character string indicating the translation result is received from the character management unit 3 .

[0194] For example, when the character string representing the translation result is "I went to the sea in summer vacation", the number of characters including spaces is 36. Therefore, if the character width of the characters constituting the character string representing the translation result is set to "2 mm", the length of the character string representing the translation result becomes 72 mm as shown in the following formula (5).

[0195] The length of the character string representing the translation result = 2 mm × 36 = 72 mm (5)

[0196] Therefore, in this case, the length of the curved line for displaying the translated character string becomes a length of 72 mm or more.

[0197] Furthermore, the shape of the curve for displaying the translated character string is set to be the same shape as the curve representing the trajectory of the line drawn by the user's operation, that is, the shape of the curve represented by the curve information acquired by the curve information acquisition unit 9. In the case where the curve is extended by the curve extension unit 17, the shape is set to be the same as the shape of the curve extended by the curve extension unit 17.

[0198] For example, if the shape of the curve representing the trajectory of a line drawn by the user's operation is an arc shape, the shape of the curve used to display the translation string is also an arc shape; if the shape of the curve representing the trajectory of a line drawn by the user's operation is a wavy line shape, the shape of the curve used to display the translation string is also a wavy line shape.

[0199] In addition, in a case where the curve is extended by the curve extending unit 17 and the length of the curve for displaying a translation string is longer than the length of the curve extended by the curve extending unit 17, or in a case where the curve is not extended by the curve extending unit 17 and the length of the curve for displaying a translation string is longer than the length of the curve represented by the curve information acquired by the curve information acquiring unit 9, the shape of the extended portion of the curve for displaying a translation string is determined based on the shape of the curve for displaying a translation string by the same method as in the case of extending the curve represented by the curve information, and the curve for displaying a translation string is extended.

[0200] After generating the curve for displaying the translated string, the display processing unit 53 generates display information for displaying the string indicating the translation result on the display 32 along the curve for displaying the translated string, and displays the string indicating the translation result on the display 32 according to the display information.

[0201] here, Fig.16 1 is an explanatory diagram showing an example of display processing in the display processing unit 53 of the display control unit 51 .

[0202] In step A, a curve image representing the trajectory of the line drawn by the user's operation is displayed on the display 32 .

[0203] After the recognition process of the user's voice is performed in step B, a character string indicating the recognition result of the voice is superimposed and displayed on the curve image in step C. Steps A to C are the same as those in the above-mentioned embodiment 1. However, here, an example is shown in which the curve information including the curve image is output before the character string indicating the recognition result of the voice is output.

[0204] In step D, the display 32 is caused to display a character string indicating a translation result along a curve for displaying a translated character string having the same shape as a curve representing the trajectory of a line drawn by the user's operation.

[0205] exist Fig.16 In the example, the display processing unit 53 places the curve for displaying the translated character string in parallel on the lower side of the curve representing the trajectory of the line drawn by the user's operation, but it can also be placed in parallel on the upper side of the curve representing the trajectory of the line drawn by the user's operation, and can also be placed in parallel on the left or right side.

[0206] In step E, the curve image representing the curve of the trajectory of the line drawn by the user's operation is eliminated.

[0207] In the second embodiment, the display processing unit 53 causes the display 32 to display the curve for displaying the translated character string. However, the curve for displaying the translated character string may be transparent.

[0208] In the second embodiment, a character string indicating the result of voice recognition and a character string indicating the result of translation are displayed side by side. Fig.17 As shown, after displaying the character string indicating the voice recognition result in step C, in step D the character string indicating the voice recognition result may be deleted from the beginning and the character string indicating the translation result may be displayed superimposed on the curve image from the beginning.

[0209] In this case, the character string finally displayed is only the character string indicating the translation result.

[0210] Furthermore, in this embodiment 2, if Fig.18 As shown, the display processing unit 53 may display the character string indicating the result of the back-translation in parallel with the character string indicating the result of the voice recognition and the character string indicating the result of the translation.

[0211] In this second embodiment, an example of translation from Japanese to English is shown, and therefore, back translation in this case means translating the translated English back into Japanese.

[0212] exist Fig.18 In the example of , in step D, a character string indicating the translation result in English is displayed below a character string indicating the recognition result in Japanese, and a character string indicating the back translation in Japanese is displayed below the English.

[0213] And, if Fig.19 As shown, in step D, after eliminating the character string representing the recognition result of the sound in Japanese that is overlapped and displayed on the curve image, the character string representing the translation result in English on the left and right sides is overlapped and displayed on the curve image, and the character string representing the back translation in Japanese is displayed below the English.

[0214] As can be seen from the above, according to the second embodiment, there is a translation unit 52 that translates the character string obtained by the character string acquisition unit 10, and the display processing unit 53 generates a curve for displaying the translation character string having a length greater than the length of the character string representing the translation result of the translation unit 52, and generates display information for displaying the character string representing the translation result of the translation unit 52 along the curve for displaying the translation character string on the display 32. Therefore, similar to the above-mentioned first embodiment, the following effect is achieved, that is, even if the length of the curve representing the trajectory of the line drawn by the operation of drawing the line is shorter than the character string representing the translation result of the translation unit 52, the entire character string can be displayed along the curve.

[0215] In the second embodiment, a character string indicating the result of voice recognition and a character string indicating the result of translation are displayed side by side. Fig. 20 As shown, in the case of a back-to-back structure in which the display 32 is composed of two screens and the upper sides of the two screens are connected to each other, for example, a character string representing the sound recognition result can be displayed in the lower screen in the figure of the two back-to-back screens, and a character string representing the translation result can be displayed in the upper screen in the figure of the two back-to-back screens.

[0216] Alternatively, a character string indicating the translation result may be displayed on the lower screen in the two back-to-back screens, and a character string indicating the voice recognition result may be displayed on the upper screen in the two back-to-back screens.

[0217] However, since the two screens are back-to-back, the characters are rotated 180 degrees in the upper screen of the two screens in the figure.

[0218] And, in the case of two pictures back to back, such as Fig.21 As shown, a character string representing the result of the back translation may also be displayed.

[0219] Implementation 3

[0220] In the first embodiment, the character string recognition unit 5 acquires a character string representing a recognition result of a voice as a display target character string. However, a character string representing a recognition result of a character drawn by an operation accepted by the operation accepting unit 1 may be acquired as a display target character string.

[0221] Fig. 22 is a structural diagram showing an input display system including an input display control device according to a third embodiment of the present invention, Fig.23 This is a hardware configuration diagram of an input display system according to Embodiment 3 of the present invention.

[0222] exist Fig. 22 and Fig.23 In, with Figure 1 and Figure 2 The same reference numerals denote the same or corresponding parts, and thus their descriptions are omitted.

[0223] The character string recognition unit 60 includes a character recognition processing unit 61 , and performs processing for outputting a character string indicating a recognition result of a character drawn by an operation accepted by the operation accepting unit 1 as a display target character string.

[0224] The character recognition processing unit 61 is implemented by, for example, a character recognition processing circuit 35 equipped with an optical character recognition (OCR) function, and performs the following processing: recognizing characters drawn by the operation accepted by the operation acceptance unit 1, and outputting a character string representing the recognition result of the character as a display target character string.

[0225] exist Fig. 22 In the above, it is assumed that the operation receiving unit 1, the character management DB 2, the character management unit 3, the trajectory determination unit 4, the character string recognition unit 60, the curve information acquisition unit 9, the character string acquisition unit 10, the character attribute information acquisition unit 11, the length comparison unit 13, the extended curve shape determination unit 14, the curve extension unit 17, the display processing unit 18 and the character string temporary storage unit 19, which are the structural elements of the input display system, are respectively Fig.23The dedicated hardware shown, namely, the indicating device 21, the storage processing circuit 22, the character management processing circuit 23, the image generation processing circuit 24, the microphone 25, the sound recognition processing circuit 26, the input interface circuit 27, the length comparison processing circuit 28, the shape determination processing circuit 29, the curve extension processing circuit 30, the display processing circuit 31, the display 32 and the character recognition processing circuit 35, but the input display system can also be implemented by software, firmware, or a combination of software and firmware.

[0226] When the input display system is implemented by software or firmware, Figure 3 The computer memory 41 shown in the figure is provided with a character management DB 2 and a character string temporary storage unit 19, and the memory 41 is provided with a program for causing the computer to execute the processing steps of the operation receiving unit 1, the character management unit 3, the trajectory identification unit 4, the character recognition processing unit 61 and the display control unit 12. Figure 3 The processor 42 shown may execute the program stored in the memory 41 .

[0227] Fig.24 This is an explanatory diagram showing an example of a process of displaying a character string indicating a result of character recognition.

[0228] Next, the operation will be described.

[0229] In this embodiment 3, it is assumed that the indicating device 21 that implements the operation acceptance unit 1 is a touch panel. Therefore, when the user uses his or her own fingers or an input pen for the touch panel to draw characters on the touch panel as an operation of drawing lines, the operation acceptance unit 1 accepts the operation of drawing characters and outputs the content of the operation to the trajectory determination unit 4 and the character string recognition unit 60.

[0230] The output content of the operation of drawing a character is, for example, time series data indicating the coordinates of the position of a finger or an input pen in contact with the touch panel, that is, the continuous change of the position coordinates on the touch panel.

[0231] exist Fig.24 In the example, in step A, the user handwrites a string like "private and food べたいです".

[0232] The character recognition processing unit 61 of the character string recognition unit 60 recognizes a plurality of characters drawn by the operation accepted by the operation accepting unit 1 , obtains a character string indicating the recognition result of the plurality of characters, and outputs the character string to the trajectory determination unit 4 and the input display control device 100 .

[0233] After receiving the content of the operation of drawing one or more characters from the operation accepting unit 1 and the character string from the character recognition processing unit 61 , the trajectory determination unit 4 performs a process of separating the content of the operation of drawing one or more characters for each character.

[0234] Specifically, it is as follows.

[0235] First, the trajectory determination unit 4 determines the coordinate X having the smallest X coordinate among the coordinates of positions on the touch panel represented by the time series data, which is the content of the operation of drawing a character. LEFT and the coordinate X with the largest X coordinate RIGHT , from the coordinate X RIGHT Subtract coordinate X LEFT , thereby calculating the length of the string in the X direction.

[0236] Then, the trajectory determination unit 4 divides the length of the character string in the X direction by the number of characters to calculate the length of each character in the X direction, and calculates the length of each character in the X direction according to the length of each character in the X direction and the coordinate X. LEFT , calculate the coordinate range of each character in the X direction. For example, (the length of each character in the X direction × (n-1) + X LEFT )~(the length of each character in the X direction × n + X LEFT ) to calculate the X-coordinate range of the nth character from the left end of the string.

[0237] Then, the trajectory identification unit 4 finds the above-mentioned X-direction coordinate range that respectively includes each position coordinate represented by the time series data, and allocates each position coordinate represented by the time series data to the corresponding X-direction coordinate range, thereby separately depicting the content of the operation of more than one character for each character.

[0238] Next, the trajectory identification unit 4 calculates the average values ​​of the X coordinates and the Y coordinates of the position coordinates represented by the time series data included in the coordinate range in the X direction as the center coordinates of the character corresponding to the coordinate range.

[0239] After the trajectory determination unit 4 calculates the center coordinates of each character constituting the character string, Fig.24 As shown in step B, a line is generated by connecting the center coordinates of each character, and this line is used as a curve representing the trajectory of the character string, and curve information representing the curve is output to the input display control device 100.

[0240] Here, assuming that the size of each character constituting the string is equal, the length of the string in the X direction is divided by the number of characters to calculate the length of each character in the X direction. In the case of handwritten characters, the size of each character is not necessarily equal, so the coordinate range unit of the X direction may not be consistent with that of each character constituting the string.

[0241] However, in the third embodiment, even if the center coordinates of each character are not strictly known, if the center coordinates of each area obtained by dividing the area where the character string is located in the X direction are known, a curve representing the parallel direction of the character string can be obtained. Therefore, even if the sizes of the characters are not equal, the curve can be obtained by the above method.

[0242] Furthermore, here, when the trajectory determination unit 4 obtains the curve representing the parallel direction of the character string, it separates the position coordinates represented by the time series data by the coordinate range unit in the X direction. However, for example, in the case where the handwritten characters are written vertically, the position coordinates represented by the time series data may be separated by the coordinate range unit in the Y direction.

[0243] The method of separating the position coordinates described here is just an example, and other methods can of course be used.

[0244] In this third embodiment, a curve is obtained based on the content of the operation for drawing characters. However, when the user performs an operation for drawing a line shown in the first embodiment, which is different from the operation for drawing characters, and the operation for drawing characters and the operation for drawing a line are accepted by the operation acceptance unit 1, the trajectory determination unit 4 can also determine the curve that presents the trajectory of the drawn line based on the content of the operation for drawing the line accepted by the operation acceptance unit 1, as in the first embodiment.

[0245] After the character string acquisition unit 10 acquires the character string output from the character recognition processing unit 61 of the character string recognition unit 60, the display processing unit 18 of the display control unit 12 generates display information for displaying the character string on the display 32 along the curve represented by the curve information acquired by the curve information acquisition unit 9, and displays the character string on the display 32 according to the display information.

[0246] exist Fig.24 In step C, a curve showing the trajectory is drawn in parallel with the handwritten character to display the character string, but the present invention is not limited to this. For example, after the handwritten character is erased, a curve may be drawn at the position of the handwritten character to display the character string.

[0247] Furthermore, the curve representing the trajectory can be transparent.

[0248] In this third embodiment, a curve can be obtained based on the operation content of drawing characters. However, as in the first embodiment described above, the recognized character string is displayed using characters with predetermined attributes. Therefore, when the length of the obtained curve is shorter than the length of the character string, as in the first embodiment described above, the display processing unit 18 cannot display part of the characters constituting the character string along the curve.

[0249] In this case, similarly to the first embodiment, a part of the character that is not displayed is displayed by extending the curved line.

[0250] That is, the length comparison unit 13 compares the length of the curve and the length of the character string, the extension curve shape determination unit 14 determines the shape of the extension curve, and the curve extension unit 17 extends the curve by adding the extension curve to the curve represented by the curve information acquired by the curve information acquisition unit 9. Then, the display processing unit 18 displays a part of the character that is not displayed along the extension curve.

[0251] As can be seen from the above, according to the third embodiment, the character string recognition unit 60 has a character recognition processing unit 61. Therefore, when displaying handwritten characters, the same effect as the above-mentioned first embodiment is also achieved, wherein the character recognition processing unit 61 recognizes the characters drawn by the operation accepted by the operation acceptance unit 1, and obtains the character string representing the recognition result of the character as the display object character string.

[0252] Implementation 4

[0253] In the above-mentioned Embodiment 3, the character string recognition unit 60 having the character recognition processing unit 61 is applied to the above-mentioned Embodiment 1. Figure 1 However, the character string recognition unit 60 having the character recognition processing unit 61 may also be applied to the input display control device of the second embodiment. Fig.14 The input display control device.

[0254] Fig.25 1 is a block diagram showing an input display system including an input display control device according to a fourth embodiment of the present invention.

[0255] Fig.26 This is an explanatory diagram showing an example of a display process of a character string indicating a translation result.

[0256] In the third embodiment, the display 32 is made to display a character string indicating the recognition result of a plurality of characters along the curve indicated by the curve information obtained by the curve information obtaining unit 9. However, in the fourth embodiment, the difference is that Fig.26As shown in step C, the display processing unit 53 of the display control unit 51 causes the display 32 to display the character string indicating the translation result output from the translation unit 52 along the curve indicated by the curve information acquired by the curve information acquisition unit 9 .

[0257] According to this fourth embodiment, similar to the above-mentioned second embodiment, even when the length of the curve represented by the curve information acquired by the curve information acquisition unit 9 is shorter than the character string representing the translation result of the translation unit 52, the entire character string can be displayed along the curve.

[0258] In addition, the present invention can freely combine the various embodiments, modify the arbitrary components of the various embodiments, or omit the arbitrary components of the various embodiments within the scope of the invention.

[0259] Industrial Applicability

[0260] The input display control device, input display control method and input display system of the present invention are suitable for displaying character strings.

[0261] Description of symbols

[0262] 1: Operation receiving unit; 2: Character management DB; 3: Character management unit; 4: Trajectory identification unit; 5: Character string recognition unit; 6: Voice detection unit; 7: Voice recognition dictionary DB; 8: Voice recognition unit; 9: Curve information acquisition unit; 10: Character string acquisition unit; 11: Character attribute information acquisition unit; 12: Display control unit; 13: Length comparison unit; 14: Extended curve shape determination unit; 15: Curve storage unit; 16: Shape determination processing unit; 17: Curve extension unit; 18: Display processing unit; 19: Character string temporary storage unit; 21: Indicator; 22: Storage processing circuit; 23: character management processing circuit; 24: image generation processing circuit; 25: microphone; 26: voice recognition processing circuit; 27: input interface circuit; 28: length comparison processing circuit; 29: shape determination processing circuit; 30: curve extension processing circuit; 31: display processing circuit; 32: display; 34: translation processing circuit; 35: character recognition processing circuit; 41: memory; 42: processor; 51: display control unit; 52: translation unit; 53: display processing unit; 60: string recognition unit; 61: character recognition processing unit; 100: input display control device.

Claims

1. A display control device, wherein: The display control device comprises: a curve information acquisition unit that acquires curve information representing the curve; a character string acquisition unit that acquires a character string; a translation unit that translates the character string acquired by the character string acquisition unit; and a display control unit configured to generate display information for displaying the character string translated by the translation unit along a curve obtained by rotating the curve represented by the curve information acquired by the curve information acquisition unit by 180 degrees, When it is determined that the length of the curve represented by the curve information obtained by the curve information obtaining unit is short and the part of the character string translated by the translation unit cannot be displayed along the curve obtained by rotating 180 degrees, the display control unit determines the shape of the extended part of the curve obtained by rotating 180 degrees according to the shape of the curve obtained by rotating 180 degrees, extends the curve obtained by rotating 180 degrees, and generates display information for displaying the part of the translated character string along the curve of the extended part. The display control unit comprises: a length comparison unit for comparing the length of the curve obtained by rotating the curve by 180 degrees with the length of the character string acquired by the character string acquisition unit; an extended curve shape determining unit that determines the shape of the extended curve of the curve as the extended portion based on the shape of the curve rotated by 180 degrees if the comparison result of the length comparing unit indicates that the length of the curve obtained by rotating by 180 degrees is shorter than the length of the character string; A curve extension part, which adds an extended curve whose shape is determined by the extended curve shape determination part to the terminal end of the curve obtained by rotating 180 degrees; as well as a display processing unit for generating display information for displaying a part of the character string along the extended curve added by the curve extending unit, The extended curve shape determining unit comprises: A curve storage unit storing a plurality of curves of different shapes; as well as A shape determination processing unit compares the shapes of the multiple curves stored in the curve storage unit with the shape of the curve obtained by rotating 180 degrees, selects the shape of the curve corresponding to the shape of the curve obtained by rotating 180 degrees from the shapes of the multiple curves stored in the curve storage unit, and determines the shape of the extended curve based on the shape of the selected curve.

2. The display control device according to claim 1, wherein: The display control unit further generates display information for displaying the character string acquired by the character string acquisition unit along the curve indicated by the curve information.

3. The display control device according to claim 2, wherein: The display control unit also generates display information having a first screen and a second screen, and generates the following display information, which is used to display the character string along the curve on the first screen, and display the translated character string obtained by rotating 180 degrees from the displayed character string along the curve obtained by rotating 180 degrees on the second screen.

4. The display control device according to claim 3, wherein: The first screen and the second screen generated by the display control unit have the same rectangular shape, the first screen is arranged at the lower side, and the second screen is arranged at the upper side.

5. The display control device according to claim 3, wherein: The display control unit generates display information for displaying a character string obtained by back-translating the translated character string displayed on the second screen in parallel with the character string along the curved line displayed on the first screen.

6. The display control device according to claim 4, wherein: The display control unit generates display information for displaying a character string obtained by back-translating the translated character string displayed on the second screen in parallel with the character string along the curved line displayed on the first screen.

7. The display control device according to any one of claims 1 to 6, wherein: The display control device further includes a display that displays the display information generated by the display control unit.

8. A display control method, wherein: Get the curve information representing the curve, Get the string, Translating the obtained character string; and generating display information for displaying the translated character string along a curve obtained by rotating the curve represented by the acquired curve information by 180 degrees, When it is determined that the length of the curve represented by the obtained curve information is short and the part of the translated character string cannot be displayed along the curve obtained by rotating 180 degrees, the shape of the extended part of the curve obtained by rotating 180 degrees is determined based on the shape of the curve obtained by rotating 180 degrees, the curve obtained by rotating 180 degrees is extended, and display information for displaying the part of the translated character string along the curve of the extended part is generated, The method further comprises: comparing the length of the curve obtained by rotating 180 degrees with the length of the obtained character string; If the comparison result indicates that the length of the curve obtained by rotating 180 degrees is shorter than the length of the character string, determining the shape of an extended curve of the curve as the extended portion according to the shape of the curve obtained by rotating 180 degrees; Adding an extended curve that determines the shape to the terminal end of the curve obtained by rotating 180 degrees; and generating display information for displaying a portion of the character string along an additional extended curve, The method further comprises: storing a plurality of curves having different shapes; and The shapes of the stored multiple curves are compared with the shape of the curve obtained by rotating 180 degrees, and the shape of the curve corresponding to the shape of the curve obtained by rotating 180 degrees is selected from the shapes of the stored multiple curves, and the shape of the extended curve is determined based on the shape of the selected curve.

9. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the steps of the method according to claim 8 are implemented.

Images