Input display device, input display method, and input display program

Abstract

Provided are an input display device, an input display method, and an input display program that improve operability of drawing a graphic representing the position indicated by a character string.　An input display device (100) according to the present disclosure comprises: a region setting unit (40) that sets a display region of a character string as a character string region; a trajectory input unit (51) that receives an input of a trajectory that connects the inside of the character string region and an indication position, which is the position indicated by a character string positioned outside the character string region; and a display control unit (70) that generates, on the basis of the trajectory positioned outside the character string region, an indication graphic representing the indication position, and displays the generated indication graphic.

Description

Input display device, input display method and input display program

[0001] The present disclosure relates to an input display device, an input display method, and an input display program.

[0002] There are input display devices that display graphics such as lines, arrows, speech bubbles, etc. around the character string to represent the position indicated by the character string. For example, Patent Literature 1 discloses a device that automatically displays graphics (speech bubbles) of a predetermined shape around the character string.

[0003] Japanese Patent Application Laid-Open No. 2021-152861

[0004] The device described in Patent Document 1 has a problem in that it is not easy to draw a figure that represents the position indicated by a character string. For example, the device described in Patent Document 1 displays a figure of a predetermined shape, so it is necessary to edit the displayed figure in order to represent the position indicated by the character string.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide an input display device, an input display method, and an input display program that can improve the operability of drawing a figure that represents the position indicated by a character string.

[0006] The input display device according to the present disclosure includes an area setting unit that sets a display area for a character string as a character string area, a trajectory input unit that accepts input of a trajectory that connects the inside of the character string area with an indication position, which is a position indicated by a character string located outside the character string area, and a display control unit that generates an indication figure that represents the indication position based on the trajectory located outside the character string area and displays the generated indication figure.

[0007] The input display method according to the present disclosure includes an area setting step of setting a display area for a character string as a character string area, a trajectory input step of accepting input of a trajectory connecting the inside of the character string area with a designated position, which is a position indicated by a character string located outside the character string area, and a display control step of generating an indication figure representing the designated position based on the trajectory located outside the character string area and displaying the generated indication figure.

[0008] The input display program of the present disclosure enables a computer to realize an area setting function that sets a display area for a character string as a character string area, a trajectory input function that accepts input of a trajectory that connects the inside of the character string area with an indication position, which is a position indicated by a character string located outside the character string area, and a display control function that generates an indication figure that represents the indication position based on the trajectory located outside the character string area and displays the generated indication figure.

[0009] The input display device, input display method, and input display program according to the present disclosure can improve the operability of drawing a figure that represents a position indicated by a character string.

[0010] 1 is a system block diagram illustrating an input display device according to a first embodiment of the present disclosure. FIG. 2 is a hardware configuration diagram illustrating an input display device according to a first embodiment of the present disclosure. FIG. 3 is an explanatory diagram illustrating a character attribute table stored in a character management DB according to a first embodiment of the present disclosure. FIG. 4 is an explanatory diagram illustrating a flow of inputting a trajectory according to a first embodiment of the present disclosure. FIG. 5 is an explanatory diagram illustrating a flow of inputting a trajectory according to a first embodiment of the present disclosure. FIG. 6 is an explanatory diagram illustrating a flow of inputting a trajectory according to a first embodiment of the present disclosure. FIG. 7 is an explanatory diagram illustrating a flow of inputting a trajectory according to a first embodiment of the present disclosure. FIG. 8 is an explanatory diagram illustrating a flow of inputting a trajectory according to a first embodiment of the present disclosure. FIG. 9 is an explanatory diagram illustrating a flow of inputting a trajectory according to a first embodiment of the present disclosure. FIG. 1 is an explanatory diagram showing a flow of regeneration of an indication shape corresponding to a movement of the position of a character string according to the first embodiment of the present disclosure. FIG. 2 is an explanatory diagram showing a flow of regeneration of an indication shape corresponding to a movement of the designated position according to the first embodiment of the present disclosure. FIG. 3 is an explanatory diagram showing a flow of regeneration of an indication shape corresponding to a movement of the designated position according to the first embodiment of the present disclosure. FIG. 4 is an explanatory diagram showing a flow of regeneration of an indication shape corresponding to a movement of the designated position according to the first embodiment of the present disclosure. FIG. 5 is an explanatory diagram showing a flow of regeneration of an indication shape corresponding to a movement of the designated position according to the first embodiment of the present disclosure. FIG. 6 is an explanatory diagram showing a flow of regeneration of an indication shape corresponding to a movement of the designated position according to the first embodiment of the present disclosure.FIG. 1 is a flowchart showing an input display method according to embodiment 1 of the present disclosure. FIG. 2 is a flowchart showing an input display method corresponding to a shape change according to embodiment 1 of the present disclosure. FIG. 3 is a flowchart showing an input display method corresponding to a movement of a character string position or a pointed position according to embodiment 1 of the present disclosure. FIG. 4 is a flowchart showing an input display method corresponding to a change in balloon size according to embodiment 1 of the present disclosure. FIG. 5 is a system block diagram showing an input display device according to a modified example of embodiment 1 of the present disclosure. FIG. 6 is a hardware configuration diagram showing an input display device according to a modified example of embodiment 1 of the present disclosure. FIG. 7 is a flowchart showing an input display method according to a modified example of embodiment 1 of the present disclosure.

[0011] Hereinafter, examples of an input display device, an input display method, and an input display program according to the present disclosure will be described with reference to the drawings. In the drawings, the same or corresponding parts are designated by the same reference numerals, and descriptions thereof will not be repeated.

[0012] First Embodiment First, an input display device 100 according to a first embodiment of the present disclosure will be described with reference to Fig. 1 and Fig. 2. Fig. 1 is a system block diagram of the input display device 100, and Fig. 2 is a hardware configuration diagram of the input display device 100.

[0013] The input display device 100 is a device that displays a character string and a figure (hereinafter referred to as an indication figure S) representing a position (hereinafter referred to as an indication position P) indicated by the character string on a display 201. As shown in Fig. 1 , the input display device 100 includes a character string input unit 10, a character string acquisition unit 20, a character management unit 30, a character management DB (Database) 81, an area setting unit 40, an indication figure shape DB 82, an operation acceptance unit 50, an operation switching unit 60, a display control unit 70, and a temporary storage unit 83.

[0014] First, a description will be given of the configuration of the input display device 100 for displaying a character string on the display 201. The display of a character string is realized by the character string input unit 10, the character string acquisition unit 20, the character management unit 30, the character management DB (DataBase) 81, and the display processing unit 74 of the display control unit 70.

[0015] The character string input unit 10 is realized by, for example, a character string input processing circuit 202 shown in FIG.

[0016] The character string acquisition unit 20 is realized by, for example, a character string acquisition processing circuit 203 shown in Fig. 2. The character string acquisition unit 20 acquires character string information of a character string accepted as input by the character string input unit 10. Here, the character string information includes the number of characters, character arrangement, etc.

[0017] The character management unit 30 is realized, for example, by the character management processing circuit 205 shown in Fig. 2. The character management unit 30 sets the character type and character size of the character string to be displayed on the display 201. Specifically, the character management unit 30 refers to a character attribute table stored in the character management DB 81 and acquires character attribute information including the character height and character width corresponding to the set character type and character size.

[0018] The character management DB 81 is realized, for example, by the storage processing circuit 204 shown in Fig. 2. The character management DB 81 stores a character attribute table that indicates the correspondence between character types, character sizes, character heights, and character widths. An example of the character attribute table stored in the character management DB 81 is shown in Fig. 3. As shown in Fig. 3, the character attribute table describes the character height and character width corresponding to the character size for each character type (e.g., Mincho, Gothic, etc.).

[0019] For example, if the character type predetermined by the character management unit 30 is "A" and the character size is "10," the character management unit 30 acquires character attribute information indicating that the character height and character width are "5 mm." Also, if the character type predetermined by the character management unit 30 is "B" and the character size is "5," the character management unit 30 acquires character attribute information indicating that the character height is "2 mm" and the character width is "1.25 mm."

[0020] The display processing unit 74 of the display control unit 70 is realized by, for example, the display processing circuit 212 shown in Fig. 2. The display processing unit 74 displays a character string on the display 201 based on the character string information acquired by the character string acquisition unit 20 and the character attribute information acquired by the character management unit 30.

[0021] Next, a description will be given of the configuration of the input display device 100 for displaying the instruction figure S on the display 201. The display of the instruction figure S is realized by a character string acquisition unit 20, a character management unit 30, a character management DB (DataBase) 81, an area setting unit 40, an instruction figure shape DB 82, an operation acceptance unit 50, an operation switching unit 60, a display control unit 70, and a temporary storage unit 83.

[0022] As described above, the character string acquisition unit 20 acquires character string information from the character string input unit 10 in order to display a character string. Furthermore, the character string acquisition unit 20 acquires character string information displayed on the display 201 from the display processing unit 74 of the display control unit 70 in order to display the indication figure S. Specifically, the character string acquisition unit 20 acquires character string information including the number of characters and character string arrangement from the display processing unit 74 of the display control unit 70. Note that the character string acquisition unit 20 may acquire character string information for a character string of multiple lines. The character string information for a character string of multiple lines includes the number of lines of the character string, the number of characters per line, the length of the line spacing, and the like in addition to the number of characters and the character string arrangement.

[0023] As described above, the character management unit 30 sets the character type and character size of the character string to be displayed on the display 201, and refers to the character attribute table stored in the character management DB 81 to obtain character attribute information including the character height and character width corresponding to the set character type and character size.

[0024] The area setting unit 40 is realized by, for example, the area setting processing circuit 206 shown in Fig. 2. The area setting unit 40 sets a display area for character strings (hereinafter referred to as character string area A) on the display 201. Specifically, the area setting unit 40 sets the character string area A based on the character string information acquired by the character string acquisition unit 20 and the character attribute information acquired by the character management unit 30. More specifically, the area setting unit 40 generates area information, which is absolute coordinate information for the character string area A in the display area of ​​the display 201, based on the number of characters, character arrangement, character height, and character width. Here, the character string area A is an area that includes the display area for the character string, and may also be an area that includes the margins around the character string.

[0025] An example will be described in which the region setting unit 40 sets an area including margins around a character string as the character string region A. For example, if the character attribute information indicates a character height of "5 mm" and the preset margins around the character string, i.e., the margins above and below the character, are "0.5 mm," the region setting unit 40 sets the height of the character string region A to "6 mm." If the character string information indicates the number of characters is "10," the character width indicated by the character attribute information is "5 mm," and the preset margins around the character string, i.e., the margins on the left and right of the character string, are "0.5 mm," the region setting unit 40 sets the length of the character string region A to "51 mm." Note that, although an example in which margins are set around characters has been described, margins do not necessarily have to be set. Furthermore, the character string does not have to be linear, but may be curved. If the character string is curved, the character string region A is set to an equal width along the character string.

[0026] The region setting unit 40 may set a display region for multiple lines of character strings as the character string region A. When setting the character string region A for multiple lines of character strings, the height of the character string region A is set based on the number of lines and the line spacing indicated by the character string information, the character height indicated by the character attribute information, and the predetermined margins above and below the character string. For example, if the number of lines indicated by the character string information is "2," the line spacing indicated by the character string information is "10 mm," the character height indicated by the character attribute information is "5 mm," and the predetermined margins around the character string, i.e., the margins above and below the character, are "0.5 mm," the region setting unit 40 sets the height of the character string region A to "21 mm." The length of the character string region A is set based on the maximum number of characters per line indicated by the character string information, the character width indicated by the character attribute information, and the predetermined margins around the character string. For example, if the maximum number of characters per line indicated by the character string information is "10," the character width indicated by the character attribute information is "5 mm," and the preset margins around the character string, i.e., the left and right margins of the character string, are "0.5 mm," the region setting unit 40 sets the length of the character string region A to "51 mm." The above describes an example in which the length of the character string region A is set based on the maximum number of characters per line. The length of the character string region A may also be set based on the number of characters per line. That is, the character string region A of a multi-line character string may be set so that each line has the same length, or may be set so that each line has a different length based on the number of characters per line. Furthermore, margins may not be provided around the characters. The character string does not have to be linear, but may also be curved.

[0027] The operation reception unit 50 is realized, for example, by the pointing device 207 shown in Fig. 2. The operation reception unit 50 receives user operations and outputs the contents of the received operations to the display control unit 70. The user operations received by the operation reception unit 50 include an operation to input a trajectory (hereinafter referred to as trajectory L), an operation to set the shape of the instruction figure S, an operation to move the position of the character string or the instruction position P, and an operation to edit the shape of the instruction figure S, which are respectively realized by a trajectory input unit 51, a shape setting unit 52, a movement unit 53 and a shape editing unit 54 described later.

[0028] The trajectory input unit 51 accepts, as a user operation, an operation to input a trajectory L. Specifically, in order to display a pointing figure S on the display 201, the trajectory input unit 51 accepts input of a trajectory L that connects the inside of the character string area A with a pointing position P located outside the character string area A, and acquires trajectory information that is absolute coordinate information of the trajectory L in the display area of ​​the display 201. Here, the inside of the character string area A includes an area line that is a boundary line between the outside of the character string area A and the inside of the character string area A.

[0029] Furthermore, the trajectory input unit 51 may accept an operation to select a character string for displaying an instruction figure S before accepting input of the trajectory L. When accepting an operation to select a character string for displaying an instruction figure S, the trajectory input unit 51 accepts input of a trajectory L that connects the inside of the character string area A of the selected character string with an instruction position P located outside the character string area A of the selected character string.

[0030] Here, the flow of the input operation of the trajectory L by the trajectory input unit 51 will be described with reference to FIG. 4 . FIG. 4 is an explanatory diagram of the operation of the trajectory input unit 51. In FIG. 4 , the character string area A is indicated by a dotted line, and the trajectory L is indicated by a dashed line. FIG. 4( a ) shows a state immediately before the user performs an operation to select a character string for which an instruction figure S is to be displayed. The operation of selecting a character string is, for example, an operation of touching the inside of the character string area A. When the user selects a character string, a trajectory acceptance symbol M is displayed on the display 201 as shown in FIG. 4( b ). In detail, when the trajectory input unit 51 accepts the operation of selecting a character string, it outputs a signal to the display processing unit 74 of the display control unit 70 to display the trajectory acceptance symbol M on the display 201, and the display processing unit 74 displays the trajectory acceptance symbol M. Here, the trajectory acceptance symbol M is a symbol displayed to assist in input of the trajectory L connecting the inside of the character string area A to an external instruction position, and is displayed inside the character string area A. After the trajectory acceptance symbol M is displayed, the user drags the trajectory acceptance symbol M as shown in FIG. 4( c ). More specifically, the trajectory input unit 51 acquires time-series data indicating continuous changes in the position of the trajectory reception symbol M as trajectory information.

[0031] 4B shows an example in which the trajectory acceptance symbol M is displayed in the center of the character string area A, but the display position of the trajectory acceptance symbol M is not limited to the center of the character string area A as long as it is inside the character string area A including the area line. Also, the trajectory acceptance symbol M does not have to be displayed. When the trajectory input unit 51 does not display the trajectory acceptance symbol M, the trajectory input unit 51 accepts a drag operation from any position inside the character string area A as an input operation of the trajectory L, rather than a drag operation of the trajectory acceptance symbol M.

[0032] The shape setting unit 52 accepts a user operation to set the shape of the instruction figure S. Specifically, the shape setting unit 52 sets the shape of the instruction figure S to be displayed on the display 201 by the display control unit 70 from among the shapes of multiple instruction figures S stored in the instruction figure shape DB 82. For example, the user sets the shape of the instruction figure S by touching the instruction figure S that the user wants to set from among the multiple instruction figures S displayed on the display 201. By accepting the operation to set the shape of the instruction figure S, the shape setting unit 52 acquires set shape information, which is relative coordinate information indicating the size, width, height, etc. of the instruction figure S.

[0033] The instruction figure shape DB 82 is realized, for example, by the storage processing circuit 204 shown in FIG. 2 . The instruction figure shape DB 82 stores information on the shapes of a plurality of instruction figures S. FIG. 5 shows examples of the shapes of the instruction figure S stored in the instruction figure shape DB 82. As shown in FIG. 5 , the instruction figure shape DB 82 stores lines ( FIG. 5( a) ), arrows ( FIG. 5( b) ), speech bubbles ( FIGS. 5( c ), 5( d ), 5( e ), and 5( f )), etc., as the shapes of the instruction figure S. Here, a speech bubble, which is an example of the shape of an instruction figure, is a figure composed of a speech bubble main body, which is a portion that surrounds a character string, and a speech bubble indicator, which is a portion that indicates the instruction position P. FIGS. 5( c ) and 5 ( d ) show examples in which the speech bubble indicator is horn-shaped and protruding from the speech bubble main body, and FIGS. 5( e ) and 5 ( f ) show examples in which the speech bubble indicator is linear. The speech bubble instruction section may be displayed only on the outside of the speech bubble main body, as shown in FIG. 5(e), or may be displayed so as to protrude from the inside of the speech bubble main body to the outside, as shown in FIG. 5(f).

[0034] Note that FIG. 5( b ) shows an example in which an arrowhead symbol is displayed at the end of a line far from the character string as an example in which the shape of the instruction figure S is an arrow, but the position of the arrowhead symbol is not limited to this. The arrowhead symbol may be displayed at the end of the line close to the character string. The arrowhead symbol may also be displayed at both ends of the line. Furthermore, FIGS. 5( c ) and 5 ( d ) show examples in which the shape of the instruction figure S is a speech bubble, where the shape of the speech bubble main body is elliptical and rectangular, but the shape of the speech bubble main body is not limited to these. Furthermore, FIGS. 5( e ) and 5 ( f ) show examples in which the shape of the instruction figure S is a speech bubble, where the speech bubble indicator is a line, but this is not limited to this. For example, the speech bubble indicator may be an arrow.

[0035] The operation switching unit 60 is realized by the operation switching processing circuit 208 shown in Fig. 2. The operation switching unit 60 accepts a user operation for switching the operation accepted by the operation acceptance unit 50, and switches the operation accepted by the operation acceptance unit 50. Specifically, the operation switching unit 60 switches the operation acceptance unit 50 to one of the trajectory input unit 51, the shape setting unit 52, the movement unit 53, and the shape editing unit 54 described below. The user operation for switching the operation accepted by the operation acceptance unit 50 is an operation corresponding to a contact time of the pointing device 207 or a contact position of the pointing device 207 that is predetermined for each operation of the operation acceptance unit 50.

[0036] The display control unit 70 displays the character string and the indication figure S on the display 201. The display control unit 70 includes a indication figure generation unit 71, a trajectory change unit 72 (described later), a character string line break unit 73 (described later), and a display processing unit 74.

[0037] The indicator figure generation unit 71 is realized by, for example, the image generation processing circuit 209 shown in FIG. 2 . The indicator figure generation unit 71 generates an indicator figure S based on area information indicating the character string area A set by the area setting unit 40, trajectory information indicating the trajectory L input by the trajectory input unit 51, and set shape information indicating the shape of the indicator figure S set by the shape setting unit 52. Specifically, the indicator figure generation unit 71 generates the indicator figure S based on the shape of the indicator figure S set by the shape setting unit 52 and the trajectory L located outside the character string area A. More specifically, the indicator figure generation unit 71 generates graphic information, which is absolute coordinate information of the indicator figure S in the display area of ​​the display 201, based on the area information, trajectory information, and set shape information. Here, the trajectory L located outside the character string area A refers to a portion of the trajectory L that connects the intersection of the character string area A and the trajectory L with the indicated position P. When the trajectory L received by the trajectory input unit 51 is a trajectory L that connects an area line included inside the character string area A with a designated position P located outside the character string area A, the trajectory L located outside the character string area A becomes the entire trajectory L received by the trajectory input unit 51. When the trajectory L received by the trajectory input unit 51 is a trajectory L that connects the inside of the character string area A excluding the area line with a designated position P located outside the character string area A, the trajectory L located outside the character string area A becomes a part of the trajectory L received by the trajectory input unit 51.

[0038] The flow of generating the indication figure S by the indication figure generating unit 71 will be described with reference to FIG. 6 . In FIG. 6 , the character string area A is indicated by a dotted line, the locus L by a dashed line, and the indication figure S by a solid line. Here, the character string area A and the locus L are not displayed on the display 201, but are shown in FIG. 6 for clarity. Note that FIG. 6( a) shows a case where the shape of the indication figure S set by the shape setting unit 52 is a line, FIG. 6( b) shows a case where the shape of the indication figure S set by the shape setting unit 52 is an arrow, FIG. 6( c) shows a case where the shape of the indication figure S set by the shape setting unit 52 is a speech bubble with a horn-shaped speech bubble indicator, and FIG. 6( d) shows a case where the shape of the indication figure S set by the shape setting unit 52 is a speech bubble with a linear speech bubble indicator. When the shape of the indication figure S set by the shape setting unit 52 is a line ( FIG. 5( a)), as shown in FIG. 6( a), the indication figure generating unit 71 generates a locus L connecting the intersection of the character string area A and the locus L with the indicated position P as the indication figure S. When the shape of the instruction figure S set by the shape setting unit 52 is an arrow ( FIG. 5( b)), the instruction figure generation unit 71 generates, as the instruction figure S, a figure obtained by adding an arrowhead symbol to the locus L connecting the intersection of the character string area A and the locus L with the instruction position P, as shown in FIG. 6( b). Here, the arrowhead symbol is combined with at least one of the intersection of the character string area A and the locus L and the instruction position P. When the shape of the instruction figure S set by the shape setting unit 52 is a speech bubble having a horn-shaped speech bubble instruction portion ( FIGS. 5( c) and 5(d)), the instruction figure generation unit 71 generates, as the instruction figure S, a figure that surrounds the locus L connecting the intersection of the character string area A and the locus L with the instruction position P, and the character string area A, as shown in FIG. 6( c). Furthermore, when the shape of the instruction figure S set by the shape setting unit 52 is a speech bubble having a linear speech bubble instruction portion (Figures 5(e) and (f)), as shown in Figure 6(d), the instruction figure generation unit 71 generates a figure as the instruction figure S by combining a figure surrounding the character string area A with a trajectory connecting the intersection of the character string area A and the trajectory L with the instruction position P.

[0039] When generating a pointer figure S for a multi-line character string, the pointer figure S is generated based on a character string area A of the multi-line character string set by the area setting unit 40, as shown in Fig. 7. Fig. 7(a) shows a case where the shape of the pointer figure S set by the shape setting unit 52 is a line, Fig. 7(b) shows a case where the shape of the pointer figure S set by the shape setting unit 52 is an arrow, and Fig. 7(c) shows a case where the shape of the pointer figure S set by the shape setting unit 52 is a speech bubble with a horn-shaped speech bubble indicator. Except for the use of a character string area A of a multi-line character string, the process is the same as the flow of generating the pointer figure S described in Fig. 6.

[0040] The display processing unit 74 performs processing for displaying the character string described above on the display 201, as well as processing for displaying the instruction figure S on the display 201. Specifically, the display processing unit 74 displays the instruction figure S on the display 201 based on the figure information generated by the instruction figure generation unit 71.

[0041] The temporary storage unit 83 is realized by, for example, the storage processing circuit 204 shown in Fig. 2. The temporary storage unit 83 temporarily stores the contents of the processing performed by the display control unit 70.

[0042] Next, a configuration of the input display device 100 will be described, which regenerates and displays the instruction figure S based on the content of an operation received from the user after displaying the instruction figure S. The redisplay of the instruction figure S is performed by the operation switching unit 60 switching the operation reception unit 50 to the shape setting unit 52, the movement unit 53, or the shape editing unit 54. When the shape setting unit 52, the movement unit 53, or the shape editing unit 54 receives an operation to set the shape of the instruction figure S, an operation to move the position of the character string or the instruction position P, or an operation to edit the shape of the instruction figure S, the instruction figure generation unit 71 regenerates the instruction figure S based on the content of the operation received by the operation reception unit 50.

[0043] First, a configuration for displaying the instruction figure S, accepting a user's setting operation for the shape of the instruction figure S, and then displaying the instruction figure S again will be described with reference to Fig. 1 and Figs. 8-9. In Figs. 8-9, the character string area A is shown by dotted lines, the locus L by dashed lines, and the instruction figure S by solid lines. Here, the character string area A and the locus L are not shown on the display 201, but are shown in Figs. 8-9 for ease of understanding.

[0044] The redisplay of the indication figure S in response to the setting operation of the shape of the indication figure S is realized by the shape setting unit 52 , the indication figure generating unit 71 and the display processing unit 74 .

[0045] The shape setting unit 52 allows the operation receiving unit 50 to receive an operation for setting the shape of the instruction figure S even after the instruction figure S is displayed on the display 201. In other words, the shape setting unit 52 receives an operation for changing the instruction figure S displayed on the display 201. As shown in FIGS. 8( a) to 9(a), the shape setting unit 52 receives an operation for changing the instruction figure S when the shape setting unit 52 receives an operation in which the user touches the instruction position P. When the user touches the instruction position P, options for the shape of the instruction figure S are displayed near the instruction position P. The user can select the shape of the instruction figure S to be changed from the options for the shape of the instruction figure S displayed near the instruction position P by touching the shape of the instruction figure S that the user wishes to change.

[0046] Even after generating the instruction figure S once, the instruction figure generating unit 71 generates the instruction figure S again based on the set shape information received from the shape setting unit 52. In detail, the instruction figure generating unit 71 generates the instruction figure S based on the area information indicating the character string area A, the trajectory information indicating the trajectory L, and the set shape information of the instruction figure S received again.

[0047] The display processing unit 74 displays the instruction figure S reflecting the change operation of the instruction figure S on the display 201, as shown in Figures 8(b)-9(b), based on the graphic shape of the instruction figure S regenerated by the instruction figure generation unit 71.

[0048] Next, a configuration for accepting a user's operation to move the position of the character string or the designated position P after displaying the instruction figure S and then re-displaying the instruction figure S will be described with reference to FIGS. 1 and 10 to 21. FIGS. 10(a1) to 21(a1) are diagrams illustrating a state before accepting a user's operation to move the position of the character string or the designated position P. FIGS. 10(a2) to 21(a2) are diagrams illustrating a change in the trajectory L for accepting a user's operation to move the position of the character string or the designated position P and re-displaying the instruction figure S. FIGS. 10(a3) to 21(a3) are diagrams illustrating a state after accepting a user's operation to move the position of the character string or the designated position P and re-displaying the instruction figure S. FIGS. 10(a1), (a2), (a3) ​​and 21(a1), (a2), and (a3) ​​illustrate a case where the shape of the instruction figure S is an arrow, and FIGS. 10(b) to 21(b) illustrate a case where the shape of the instruction figure S is a speech bubble. 10 to 21, the character string area A is indicated by a dotted line, the locus L by a broken line, and the pointer figure S by a solid line. Although the character string area A and the locus L are not displayed on the display 201, they are displayed in FIGS. 10(a2) to 21(a2) for ease of understanding.

[0049] The re-display of the indication figure S in response to the operation of moving the position of the character string or the indication position P is performed by the movement unit 53 , the locus change unit 72 , the indication figure generation unit 71 and the display processing unit 74 .

[0050] The movement unit 53 accepts a user operation to move the position of the character string or the instruction position P after displaying the instruction figure S on the display 201, and outputs position movement information, which is absolute coordinate information of the character string or the instruction position P in the display area of ​​the display 201.

[0051] The trajectory modification unit 72 is realized, for example, by the trajectory modification processing circuit 210 shown in FIG. 2 . The trajectory modification unit 72 modifies at least one of the size and position of the trajectory L so that the trajectory L becomes similar to the movement received by the movement unit 53. Modifying the trajectory so that the trajectory L becomes similar means performing at least one of enlarging, reducing, inverting, and rotating. Specifically, the trajectory modification unit 72 generates modified trajectory information that is absolute coordinate information of the trajectory L in the display area of ​​the display 201 and that reflects position movement information, which is absolute coordinate information output from the movement unit 53, on trajectory information, which is absolute coordinate information output from the trajectory input unit 51.

[0052] When the movement unit 53 accepts a movement of the position of the character string, the trajectory change unit 72 changes at least one of the size and the position of the trajectory L so that the relative positions of the designated position P and one end of the trajectory L located within the character string area A with respect to the character string area A are the same before and after the change of the trajectory L, as shown in FIGS. 10(a2) to 15(a2). For example, as shown in FIGS. 10(a2) to 11(a2), the trajectory change unit 72 may enlarge and invert the trajectory L before the change so that the trajectories before and after the change are similar. Furthermore, as shown in FIGS. 12(a2) to 15(a2), the trajectory change unit 72 may enlarge and rotate the trajectory L before the change so that the trajectories before and after the change are similar.

[0053] When the movement unit 53 accepts the movement of the designated position P, the trajectory modification unit 72 changes at least one of the size and position of the trajectory L so that the position of one end of the trajectory L located inside the character string area A is the same before and after the change of the trajectory L, as shown in FIGS. 16(a2) to 21(a2). For example, as shown in FIGS. 16(a2) to 18(a2), the trajectory modification unit 72 may enlarge and invert the trajectory L before the change so that the trajectories before and after the change are similar. Furthermore, as shown in FIGS. 19(a2) to 21(a2), the trajectory modification unit 72 may enlarge and rotate the trajectory L before the change so that the trajectories before and after the change are similar. Here, the designated position P is the position of the other end of the trajectory L located outside the character string area A.

[0054] When the instruction figure generation unit 71 receives the position movement information generated by the movement unit 53 and the changed trajectory information generated by the trajectory change unit 72, the instruction figure generation unit 71 regenerates the instruction figure S based on the area information, changed trajectory information, and set shape information that reflect the position movement information. Here, the area information that reflects the position movement information is absolute coordinate information of the character string area A in the display area of ​​the display 201. When the movement unit 53 accepts a movement of the character string, the character string area A moves in conjunction with the movement of the character string. Therefore, by regenerating the instruction figure S based on area information that reflects the position movement information, which is absolute coordinate information output from the movement unit 53, in the area information that is absolute coordinate information output by the area setting unit 40, the instruction figure S corresponding to the operation accepted by the movement unit 53 can be generated.

[0055] The display processing unit 74 displays the instruction figure S reflecting the change operation of the instruction figure S on the display 201, as shown in Figures 10 (a3) ​​to 21 (a3), based on the graphic shape of the instruction figure S regenerated by the instruction figure generation unit 71.

[0056] In the above, we have explained the case where the shape of the instruction figure S indicated by the set shape information is an arrow, but if the shape of the instruction figure S indicated by the set shape information is a speech bubble, the speech bubble is displayed again, deformed in accordance with the movement of the position of the character string or the instruction position P, as shown in Figures 10(b) to 21(b).

[0057] Next, a configuration for accepting an edit operation of the shape of the instruction figure S from the user after displaying the instruction figure S and redisplaying the instruction figure S and the character string will be described with reference to Figs. 1 and 22. In Fig. 22, the character string area A (A1, A2) is indicated by dotted lines, and the instruction figures S (S1, S2) are indicated by solid lines. Although the character string area A is not displayed on the display 201, it is shown in Fig. 22 for ease of understanding. The redisplay of the instruction figure S in response to an edit operation of the shape of the instruction figure S is performed by the shape editing unit 54, the character string line break unit 73, the instruction figure generating unit 71, and the display processing unit 74.

[0058] The shape editing unit 54 accepts an editing operation of the shape of the instruction figure S as a user operation after the instruction figure S is displayed on the display 201. Here, editing the shape of the instruction figure S includes changing the size and aspect ratio of the instruction figure S once it has been displayed. The editing operation of the shape of the instruction figure S is realized, for example, by selecting the instruction figure S to be edited by long pressing, and then dragging the default shape of the selected instruction figure S.

[0059] The shape editing unit 54 outputs edited set shape information, which is information about the shape of the instruction figure S that has been edited, to the instruction figure generating unit 71 and the character string line break unit 73 of the display control unit 70. Here, the edited set shape information is relative coordinate information that indicates the size, width, height, etc. of the instruction figure S, and reflects changes to the size, width, and height of the instruction figure included in the set shape information stored in the instruction figure shape DB 82.

[0060] The instruction figure generation unit 71 regenerates the instruction figure S based on the edit setting shape information generated by the shape editing unit 54. Specifically, the instruction figure generation unit 71 regenerates the instruction figure S based on the area information, the trajectory information, and the edit setting shape information. For example, after displaying a speech bubble S1 as the instruction figure S as shown in FIG. 22( a), if the shape editing unit 54 receives an operation to change the aspect ratio of the speech bubble S1 as an editing operation for the shape of the instruction figure S, the instruction figure generation unit 71 regenerates a speech bubble S2 corresponding to the shape editing operation as shown in FIG. 22( b).

[0061] The character string line break unit 73 is realized, for example, by the character string line break processing circuit 211 shown in FIG. 2 . The character string line break unit 73 breaks a character string in response to an editing operation of the shape of the instruction figure S received by the shape editing unit 54. Specifically, when the shape of the instruction figure S is a speech bubble and a character string is displayed outside the speech bubble, the character string line break unit 73 generates line break information for breaking the character string displayed outside the speech bubble so that the character string is displayed inside the speech bubble. In other words, when the character string does not fit within the speech bubble main body, which is the part of the speech bubble that surrounds the character string, the character string line break unit 73 generates line break information for breaking the character string so that the character string fits within the speech bubble main body. For example, as shown in FIG. 22B , when a character string does not fit within the speech bubble main body of the regenerated speech bubble S2 as a result of regeneration of the speech bubble S2 and is displayed outside the speech bubble S2, the character string line break unit 73 breaks the character string so that the character string is displayed inside the speech bubble. That is, the character string line breaker 73 changes the character string area A from a character string area A1 having a height equivalent to one line without a line break to a character string area A2 having a height equivalent to two lines after the line break.

[0062] Based on the graphic shape of the instruction figure S regenerated by the instruction figure generation unit 71, the display processing unit 74 displays on the display 201 the instruction figure S reflecting the change operation of the instruction figure S and a line break of the character string, as shown in Figure 22 (c).

[0063] 1, the components of the input display device 100, namely, the character string input unit 10, the character string acquisition unit 20, the character management DB 81, the character management unit 30, the area setting unit 40, the instruction figure shape DB 82, the operation acceptance unit 50, the operation switching unit 60, the instruction figure generation unit 71, the locus change unit 72, the character string line break unit 73, the display processing unit 74, and the temporary storage unit 83, are each realized by dedicated hardware shown in Fig. 2. That is, it is assumed that the input display device 100 is realized by a display 201, a character string input processing circuit 202, a character string acquisition processing circuit 203, a storage processing circuit 204, a character management processing circuit 205, an area setting processing circuit 206, a pointing device 207, an operation switching processing circuit 208, an image generation processing circuit 209, a locus change processing circuit 210, a character string line break processing circuit 211, and a display processing circuit 212.

[0064] Here, the storage processing circuit 204 may be, for example, a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), or an EEPROM (Electrically Erasable Programmable Read Only Memory), or a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, or a DVD (Digital Versatile Disc).

[0065] The pointing device 207 is, for example, a touch panel. Note that the pointing device 207 is not limited to a touch panel and may be a joystick, a pointing stick (trackpoint), a touchpad, a stylus, a data glove, a trackball, a pen tablet, a mouse, a light pen, a joypad, or the like.

[0066] The character management processing circuit 205, area setting processing circuit 206, operation switching processing circuit 208, image generation processing circuit 209, trajectory change processing circuit 210, character string line feed processing circuit 211, and display processing circuit 212 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0067] Furthermore, the components of the input display device 100 are not limited to those realized by dedicated hardware, and the input display device 100 may be realized by software, firmware, or a combination of software and firmware. Software or firmware is stored as a program in the memory of a computer. A computer refers to hardware that executes a program, and includes a CPU (Central Processing Unit), central processing unit, processing device, arithmetic unit, microprocessor, microcomputer, processor, DSP (Digital Signal Processor), etc.

[0068] The hardware configuration of the input display device 100 when implemented by software, firmware, or the like will be described with reference to FIG. 23 . FIG. 23 is a hardware configuration diagram of a computer. As shown in FIG. 23 , when the input display device 100 is implemented by software, firmware, or the like, a character management DB 81, a reference figure shape DB 82, and a temporary storage unit 83 are configured on a memory 213 of the computer. Also, a program for causing the computer to execute the processing procedures of the character string input unit 10, the character string acquisition unit 20, the character management unit 30, the area setting unit 40, the operation acceptance unit 50, the operation switching unit 60, the reference figure generation unit 71, the trajectory change unit 72, the character string line break unit 73, and the display processing unit 74 is stored in the memory 213. Then, the processor 214 executes the program stored in the memory 213.

[0069] 2 shows an example in which each of the components of the input / display system is implemented by dedicated hardware, while FIG. 23 shows an example in which the input / display system is implemented by software, firmware, or the like. However, this is merely an example, and some of the components of the input / display system may be implemented by dedicated hardware, and the remaining components may be implemented by software, firmware, or the like. For example, the character string input unit 10, the operation reception unit 50, and the operation switching unit 60 may be implemented by dedicated hardware, and the character string acquisition unit 20, the character management DB 81, the character management unit 30, the area setting unit 40, the indication figure shape DB 82, the indication figure generation unit 71, the trajectory change unit 72, the character string line break unit 73, the display processing unit 74, and the temporary storage unit 83 may be implemented by software, firmware, or the like. However, any combination of dedicated hardware and software, etc., is possible.

[0070] Next, an input display method using the input display device 100 according to the first embodiment of the present disclosure will be described with reference to Fig. 24. Fig. 24 is a flowchart showing the procedure of the input display method according to the first embodiment.

[0071] 24 , first, the display processing unit 74 displays the character string input by the character string input unit 10 on the display 201 (step ST1). Specifically, the display processing unit 74 displays the character string on the display 201 based on the character string information input by the character string input unit 10 and acquired by the character string acquisition unit 20, and the character attribute information output from the character management unit 30.

[0072] Next, the character string obtaining unit 20 obtains character string information, which is information about the character string displayed on the display 201 (step ST2). Specifically, the character string obtaining unit 20 obtains the character string information, including the number of characters and the arrangement of the character string, from the display control unit 70.

[0073] Next, character management unit 30 acquires character attribute information (step ST3), which is character height and character width. Character management unit 30 references a character attribute table stored in character management DB 81 to acquire character attribute information including character height and character width corresponding to the character type and character size of the character string displayed on display 201.

[0074] The area setting unit 40 sets the character string area A based on the character string information acquired by the character string acquisition unit 20 and the character attribute information acquired by the character management unit 30 (step ST4). Specifically, the area setting unit 40 sets, as the character string area A, an area corresponding to the number of characters and character arrangement indicated by the character string information and the character height and character width indicated by the character attribute information.

[0075] Next, the shape setting unit 52 determines whether or not it has received a setting operation for the shape of the indication figure S (step ST5). If the shape setting unit 52 has received a setting operation for the shape of the indication figure S (Yes in step ST5), the shape setting unit 52 sets the shape of the indication figure S for which the setting operation has been received to be displayed on the display 201 (step ST6). If the shape setting unit 52 has not received a setting operation for the shape of the indication figure S (No in step ST5), the shape setting unit 52 does not change the setting of the shape of the indication figure S, and sets the shape of the indication figure S to be displayed on the display 201 as the default. For example, if an arrow is set as the default indication figure S, the user performs a shape setting operation if they want to display an indication figure S other than an arrow on the display 201, but does not perform any operation if they want to display an arrow on the display 201.

[0076] After the character string area A is set (steps ST2-ST4) and the shape of the indication figure S is set (steps ST5-ST6), the locus input unit 51 receives input of the locus L (steps ST7-ST9).

[0077] First, the trajectory input unit 51 determines whether or not an operation to select a character string for displaying the instruction graphic S has been received (step ST7). The operation to select a character string for displaying the instruction graphic S is performed, for example, by touching the inside of the character string area A with a touch panel. The trajectory input unit 51 waits until an operation to select a character string is received (No in step ST7). If the trajectory input unit 51 has received an operation to select a character string (Yes in step ST7), the trajectory input unit 51 outputs a signal to the display processing unit 74 to cause a trajectory acceptance symbol M to be displayed on the display 201, and the display processing unit 74 displays the trajectory acceptance symbol M on the display 201 (step ST8).

[0078] After the trajectory acceptance symbol M is displayed on the display 201, the trajectory input unit 51 determines whether or not input of a trajectory L has been accepted (step ST9). Here, the trajectory L accepted by the trajectory input unit 51 is a trajectory L that connects the inside of the character string area A with a designated position P located outside the character string area A. The trajectory input unit 51 waits until it accepts an input operation of the trajectory L (No in step ST9). If the trajectory input unit 51 accepts an input operation of the trajectory L (Yes in step ST9), it generates a designated figure S based on the trajectory L located outside the character string area (step ST10).

[0079] Specifically, the instruction figure generation unit 71 generates the instruction figure S based on the character string region A set by the region setting unit 40, the locus L received by the locus input unit 51, and the shape of the instruction figure S set by the shape setting unit 52. For example, if the shape of the instruction figure S set by the shape setting unit 52 is a line ( FIG. 5A ), the instruction figure generation unit 71 generates, as the instruction figure S, a locus L connecting the intersection of the character string region A and the locus L with the instruction position P, as shown in FIG. 6A . If the shape of the instruction figure S set by the shape setting unit 52 is an arrow ( FIG. 5B ), the instruction figure generation unit 71 generates, as the instruction figure S, a figure obtained by adding an arrowhead symbol to the locus L connecting the intersection of the character string region A and the locus L with the instruction position P, as shown in FIG. 6B . Here, the arrowhead symbol is superimposed on at least one of the intersection of the character string region A and the locus L and the instruction position P. When the shape of the instruction figure S set by the shape setting unit 52 is a speech bubble having a horn-shaped speech bubble instruction portion (FIGS. 5(c) and 5(d)), as shown in FIG. 6(c), the instruction figure generation unit 71 generates a figure that surrounds the character string area A and a locus L connecting the character string area A and the intersection of the locus L with the instruction position P, as the instruction figure S. When the shape of the instruction figure S set by the shape setting unit 52 is a speech bubble having a linear speech bubble instruction portion (FIGS. 5(e) and 5(f)), as shown in FIG. 6(d), the instruction figure generation unit 71 generates a figure that combines the locus connecting the character string area A and the intersection of the locus L with the instruction position P with a figure that surrounds the character string area A, as the instruction figure S.

[0080] The display processing unit 74 displays the character string acquired by the character string acquiring unit 20 and the indication figure S generated by the indication figure generating unit 71 on the display 201 (step ST11).

[0081] As described above, the input / display method according to the first embodiment includes an area setting step (steps ST3-ST4) of setting a display area for a character string as the character string area A, a trajectory input step (steps ST7-ST9) of receiving input of a trajectory L connecting the inside of the character string area A with a designated position P located outside the character string area A, and a display control step (steps ST10-ST11) of generating a pointer figure S representing the designated position P based on the trajectory L located outside the character string area A and displaying the generated pointer figure S. According to the above-described input / display method, the pointer figure S can be generated based on the trajectory L input by the user. Therefore, the pointer figure S can be generated without performing an operation to edit a pointer figure S of a predetermined shape, thereby improving the operability of drawing the pointer figure S, i.e., the figure representing the position indicated by the character string. Furthermore, according to the above-described input / display method, the pointer figure S can be generated based on the trajectory L input by the user. For example, the pointer figure S can be generated and displayed based on a complex trajectory L as shown in FIG. 9 . Therefore, according to the above-described input / display method, a pointer figure S of a complex shape can be drawn with a simple operation.

[0082] Although the above describes an example in which the region setting unit 40 uses the number of characters, character arrangement, character height, and character width to set the character string region A, this is not limiting. The region setting unit 40 may set the character string region A as a display region for the character string, and may set the character string region using known techniques. Furthermore, the above describes an example in which the character string acquisition unit 20 and the character management unit 30 output the number of characters, character arrangement, character height, and character width to the region setting unit 40. Alternatively, the character string acquisition unit 20 may acquire the number of characters, character arrangement, character height, and character width and output them to the region setting unit 40. When the character string acquisition unit 20 acquires the number of characters, character arrangement, character height, and character width, the input display device 100 does not need to include the character management unit 30 and the character management DB. Furthermore, in the above example, the character management unit 30 acquires the character height and character width corresponding to the character type and number of characters predetermined by the character management unit 30, but the character management unit 30 may also acquire the character height and character width corresponding to the character type and character size included in the character string information acquired by the character string acquisition unit 20.

[0083] Although the above example shows that the area setting unit 40 sets the character string area A for the character string acquired by the character string acquisition unit 20, the area setting unit 40 may set the character string area A for only the selected character string. Furthermore, when the area setting unit 40 sets the character string area A for only the selected character string, the trajectory input unit 51 may display the trajectory acceptance symbol M inside the character string in which the character string area A is set, without accepting the selection of the character string.

[0084] In addition, in the above example, the processing is performed in the order of setting the character string area A (steps ST2-ST4), setting the shape of the instruction figure S (steps ST5-ST6), and receiving the input of the locus (steps ST7-ST9), but this is not limiting. It is sufficient if the character string area A, the locus L, and the instruction figure S can be output to the instruction figure generation unit 71 before the instruction figure generation unit 71 generates the instruction figure S. Therefore, the setting of the character string area A (steps ST2-ST4), setting the shape of the instruction figure S (steps ST5-ST6), and receiving the input of the locus L (steps ST7-ST9) may be performed in parallel, or may be performed in a different order.

[0085] Next, an input display method for changing the shape of the indication figure S after the display control step (steps ST10-ST11) is completed, i.e., after the indication figure S is displayed on the display 201, will be described with reference to Fig. 25. Fig. 25 is a flowchart showing the input display method for changing the shape of the indication figure S after it is displayed.

[0086] As shown in FIG. 25 , after completing the display control steps (steps ST10-ST11), the shape setting unit 52 determines whether it has received an operation to change the shape of the indication figure S (step ST12). The operation to change the shape of the indication position is performed, for example, by touching the indication position P and selecting the desired shape of the indication figure S from among the shape options of the indication figure S displayed near the indication position P. If the shape setting unit 52 has not received an operation to change the shape of the indication position P (No in step ST12), the character string and the indication figure S displayed on the display 201 in step ST11 are displayed without change. If the shape setting unit 52 has received an operation to change the shape of the indication figure S (Yes in step ST12), the indication figure generation unit 71 regenerates the indication figure S based on the shape setting of the indication position P changed by the shape setting unit 52 (step ST13). The display processing unit 74 displays the character string and the regenerated indication figure S on the display 201 (step ST14). The specific processing contents of the instruction figure generating section 71 and the display processing section 74 are the same as those in steps ST10 and ST11.

[0087] As described above, according to the input display method for changing the shape of a pointer figure realized by the shape setting unit 52, the pointer figure generation unit 71, and the display control unit 70, even after the pointer figure S is displayed on the display 201, the shape of the pointer figure S can be appropriately changed and displayed without performing an operation to edit the pointer figure S displayed on the display 201. Therefore, according to the above input display method, the operability of drawing the pointer figure S can be further improved.

[0088] Next, an input display method for moving the position of the character string or the designated position P after the display control step (steps ST10-ST11) is completed, i.e., after displaying the indication figure S on the display 201, will be described with reference to Fig. 26. Fig. 26 is a flowchart showing the input display method for moving the position of the character string or the designated position P after displaying the indication figure S.

[0089] As shown in FIG. 26 , after completing the display control steps (steps ST10-ST11), the movement unit 53 determines whether or not it has received an operation to move the position of the character string or the designated position P (step ST15). The operation to move the position of the character string or the designated position P is performed, for example, by long-pressing the position of the character string or the designated position P to be moved. If the movement unit 53 has not received an operation to move the position of the character string or the designated position P (No in step ST15), the character string and the designated figure S displayed on the display 201 in step ST11 are displayed without change. If the movement unit 53 has received an operation to move the position of the character string or the designated position P (Yes in step ST15), the trajectory change unit 72 changes at least one of the size and position of the trajectory L so that the trajectory L becomes similar to the movement received by the movement unit 53 (step ST16).

[0090] Specifically, when the moving unit 53 accepts a movement of the position of the character string, the trajectory change unit 72 changes at least one of the size and the position of the trajectory L so that the relative positions of the designated position P and one end of the trajectory L located within the character string area A with respect to the character string area A remain the same before and after the change of the trajectory L. Furthermore, when the moving unit 53 accepts a movement of the designated position P, the trajectory change unit 72 changes at least one of the size and the position of the trajectory L so that the position of one end of the trajectory L located within the character string area A remains the same before and after the change of the trajectory L.

[0091] When the locus change unit 72 changes at least one of the size and position of the locus L, the indication figure generation unit 71 regenerates the indication figure S based on the changed locus L (step ST17). The display processing unit 74 displays the character string and the regenerated indication figure S on the display 201 (step ST18). Note that the specific processing contents of the indication figure generation unit 71 and the display processing unit 74 are the same as those of steps ST10 and ST11.

[0092] As described above, according to the input display method for moving the position of the character string or the designated position P, which is realized by the movement unit 53, the trajectory change unit 72, and the indication figure generation unit 71, even if the position of the character string or the designated position P is changed after the indication figure S is displayed on the display 201, it is possible to automatically display the indication figure S corresponding to the movement of the position of the character string or the designated position P, without performing an operation to edit the indication figure S displayed on the display 201. Therefore, according to the above input display method, it is possible to further improve the operability of drawing the indication figure S.

[0093] Next, an input display method for editing the shape of the indication figure S after the display control step (steps ST10-ST11) is completed, i.e., after the indication figure S is displayed on the display 201, will be described with reference to Fig. 29. Fig. 27 is a flowchart showing an input display method for editing the shape of the indication figure S after its display.

[0094] As shown in FIG. 27 , after completing the display control steps (steps ST10-ST11), the shape editing unit 54 determines whether or not it has accepted an edit request for the shape of the instruction figure S (step ST19). Editing the shape of the instruction figure S is performed, for example, by long-pressing the instruction figure S to be edited. If the shape editing unit 54 has not accepted an edit request for the shape of the instruction figure S (No in step ST19), the character string and instruction figure S displayed on the display 201 in step ST11 are displayed without change. If the shape editing unit 54 has accepted an edit request for the shape of the instruction figure S (Yes in step ST19), the instruction figure S is regenerated based on the edit information for the shape of the instruction figure S output from the shape editing unit 54 (step ST20). Next, the character string line break unit 73 determines whether or not the shape of the instruction figure S output from the shape setting unit 52 is a speech bubble (step ST21). If the shape of the instruction figure S is a speech bubble (Yes in step ST21), the character string line break unit 73 determines whether or not a character string is displayed outside the speech bubble (step ST22). If the character string line break unit 73 determines that a character string is displayed outside a speech bubble (Yes in step ST22), the character string line break unit 73 outputs information for breaking the character string so that the character string is displayed inside the speech bubble to the display processing unit 74 (step ST23). The display processing unit 74 displays the character string broken into lines so that the character string is displayed inside the speech bubble and the edited instruction figure S based on the character string line break information output from the character string line break unit 73 and the change instruction figure information output from the instruction figure generation unit 71 (step ST24). If the character string line break unit 73 determines that the shape of the instruction figure S is not a speech bubble (No in step ST21), the display processing unit 74 displays the same character string as in step ST11 and the instruction figure with a changed size based on the character string information output from the character string acquisition unit 20 and the change instruction figure information output from the instruction figure generation unit 71 (step ST25).

[0095] As described above, according to the input display method realized by the shape editing unit 54, the string line break unit 73, and the indication figure generation unit 71, even if the shape of the indication figure S is edited after the indication figure S is displayed on the display 201, a string corresponding to the size of the indication figure S can be automatically displayed without editing the arrangement of the string displayed together with the indication figure S.

[0096] The functions and effects of the input display device 100, the input display method, and the input display program according to the first embodiment of the present disclosure will be described.

[0097] The input display device 100 according to the first embodiment of the present disclosure includes an area setting unit 40 that sets a display area for a character string as a character string area A, a trajectory input unit 51 that accepts input of a trajectory L that connects the inside of the character string area A with an indication position P, which is a position indicated by a character string located outside the character string area A, and a display control unit 70 that generates an indication figure S that represents the indication position P based on the trajectory L located outside the character string area A and displays the generated indication figure S.

[0098] The input display method according to the first embodiment of the present disclosure includes an area setting step for setting a display area for a character string as a character string area A, a trajectory input step for accepting input of a trajectory L connecting the inside of the character string area A and a designated position P, which is a position indicated by a character string located outside the character string area A, and a display control step for generating an indication figure S representing the designated position P based on the trajectory L located outside the character string area A and displaying the generated indication figure S.

[0099] The input display program according to the first embodiment of the present disclosure provides a computer with an area setting function for setting a display area for a character string as a character string area A, a trajectory input function for accepting input of a trajectory L connecting the inside of the character string area A with a designated position P, which is a position indicated by a character string located outside the character string area A, and a display control function for generating a designated figure S representing the designated position P based on the trajectory L located outside the character string area A and displaying the generated designated figure S.

[0100] According to the input display device 100, the input display method, and the input display program of the first embodiment of the present disclosure, it is possible to generate a pointer figure S based on a trajectory input by a user. Therefore, it is possible to generate a pointer figure S without performing an operation to edit a pointer figure S of a preset shape. In other words, according to the input display device 100, the input display method, and the input display program of the first embodiment of the present disclosure, it is possible to display a figure representing a position indicated by a character string with a small number of operations, thereby improving the operability of drawing a figure representing a position indicated by a character string.

[0101] Furthermore, according to the input display device 100, input display method, and input display program of embodiment 1 of the present disclosure, an instruction figure S can be generated based on a trajectory input by the user, so that, for example, an instruction figure S of a complex shape such as that shown in Figure 9 can be generated without performing an operation to edit an instruction figure S of a predetermined shape.

[0102] Moreover, the input display device 100 according to the first embodiment of the present disclosure further includes a shape setting unit 52 that sets the shape of the instruction figure S to be displayed by the display control unit 70 from the shapes of a plurality of predetermined instruction figures S. According to the input display device 100 described above, even after the instruction figure S has been generated once, the shape of the instruction figure S can be appropriately changed and displayed without performing an operation to edit the instruction figure S once displayed on the display 201.

[0103] Furthermore, the input display device 100 according to the first embodiment of the present disclosure further includes a movement unit 53 that accepts movement of the position of the character string or the designated position P, and the display control unit 70 changes at least one of the size and the position of the locus L so that the locus L is similar in response to the movement accepted by the movement unit 53, and displays the instruction figure S based on the changed locus L. According to the input display device 100 described above, even if the position of the character string or the designated position P is changed after the instruction figure S has been generated, the instruction figure S can be displayed without performing an operation to edit the displayed instruction figure S.

[0104] The input display device 100 according to the first embodiment of the present disclosure further includes a shape editing unit 54 that accepts editing of the shape of the indication figure S, and when the indication figure S is a speech bubble and the character string does not fit in the portion of the speech bubble that encloses the character string, the display control unit 70 displays the character string with line breaks corresponding to the size of the portion of the speech bubble that encloses the character string. According to the input display device 100 described above, even when the shape of the indication figure S is edited, the character string corresponding to the size of the indication figure S can be displayed without the user having to perform an editing operation to change the position of the displayed character string.

[0105] In the above example, the character string area A and the locus L are not displayed on the display 201, and only the character string and the instruction figure S are displayed on the display 201. However, the character string area A and the locus L may be displayed on the display 201. The display processing unit 74 may also display the character string area A and the locus L on the display 201 for only a certain period of time. By displaying the character string area A and the locus L on the display 201, the process for displaying the instruction figure S can be visually confirmed, and therefore the operability of drawing the instruction figure S can be further improved.

[0106] In the first embodiment, the input display device 100 that displays a character string and an indication figure S on the display 201 has been described. In a modification of the first embodiment, the input display device 101 that can display a character string in a layout freely chosen by the user and that displays the character string on the display 201 together with an indication figure S that corresponds to the character string will be described.

[0107] The input display device 101 according to the modification of the first embodiment differs from the input display device 100 according to the first embodiment in that it includes a display control unit 70a that generates a trajectory image indicating the trajectory L received by the trajectory input unit 51 and displays a character string superimposed on the generated trajectory image. The configuration of the input display device 101 according to the modification of the first embodiment will be described with reference to FIGS. 28 and 29 . FIG. 28 is a system block diagram of the input display device 101, and FIG. 29 is a hardware configuration diagram of the input display device 101. The input display device 101 according to the modification of the first embodiment differs from the input display device 100 according to the first embodiment in the internal configurations and operations of the character string input unit 10a, operation acceptance unit 50a, operation switching unit 60a, and display control unit 70a, but the other configurations and operations are similar.

[0108] 28 , the character string input unit 10a included in the input display device 101 acquires a character string indicating a speech recognition result as a character string to be displayed. Then, similar to the character string input unit 10 according to the first embodiment, the character string input unit 10a outputs the character string received by the character string input unit 10a to the character string acquisition unit 20.

[0109] As shown in FIG. 28 , the character string input unit 10 a included in the input display device 101 includes a voice detection unit 11, a voice recognition dictionary DB 84, and a voice recognition unit 12. The voice detection unit 11 is realized, for example, by a microphone 215 shown in FIG. 29 . The voice detection unit 11 detects the user's voice and outputs the detected voice to the voice recognition unit 12. The voice recognition dictionary DB 84 is realized, for example, by a storage processing circuit 204 shown in FIG. 29 . The voice recognition dictionary DB 84 includes, for example, an acoustic model describing the acoustic characteristics of phonemes, which are small units of human speech, and a recognition dictionary describing words to be voice recognized. The voice recognition unit 12 is realized, for example, by a voice recognition processing circuit 216 shown in FIG. 29 . The voice recognition unit 12 is equipped with a voice recognition engine that recognizes the voice detected by the voice detection unit 11. The speech recognition unit 12 analyzes the speech detected by the speech detection unit 11, calculates the acoustic features of the speech, and searches for a word having acoustic features closest to the calculated acoustic features from among the words described in the recognition dictionary of the speech recognition dictionary DB 84. The speech recognition unit 12 outputs a character string indicating the searched word to the display processing unit 74a of the display control unit 70a as a character string indicating the speech recognition result.

[0110] Although the character string input unit 10a shown in FIG. 28 includes a voice recognition dictionary DB 84 and a voice recognition unit 12, the character string input unit 10a may include a data transmission / reception unit instead of the voice recognition dictionary DB 84 and the voice recognition unit 12. Here, the data transmission / reception unit is a network communication device such as a network card capable of transmitting and receiving data to and from a voice recognition server (not shown) via a communication path such as the Internet or a local area network (LAN). The data transmission / reception unit transmits data indicating the voice detected by the voice detection unit 11 to the voice recognition server. The voice recognition server is equipped with a voice recognition engine that recognizes voice. Upon receiving the data indicating the voice transmitted from the data transmission / reception unit, the voice recognition server recognizes the voice indicated by the received data and transmits a character string indicating the voice recognition result to the data transmission / reception unit. Upon receiving the character string indicating the voice recognition result transmitted from the voice recognition server, the data transmission / reception unit outputs the character string to the display processing unit 74a of the display control unit 70a.

[0111] The operation accepting unit 50a accepts an operation to input a trajectory L for generating a trajectory image for displaying an overlapping character string, in addition to a user operation accepted by the operation accepting unit 50 according to the first embodiment. Specifically, the trajectory input unit 51a of the operation accepting unit 50a accepts input of a trajectory L for generating a trajectory image for displaying an overlapping character string, in addition to a trajectory for generating a pointer figure S. The operation accepting unit 50a outputs information about the accepted trajectory L to the display control unit 70a. Specifically, when the trajectory input unit 51a accepts a trajectory for generating a pointer figure S, i.e., a trajectory L connecting a pointer position located inside the character string area A and a pointer position located outside the character string area A, the trajectory input unit 51a outputs information about the accepted trajectory L to the pointer figure generation unit 71 of the operation accepting unit 50a. Furthermore, when the trajectory input unit 51a accepts a trajectory L for generating a trajectory image for displaying an overlapping character string, the trajectory input unit 51a outputs information about the accepted trajectory L to the trajectory image generation unit 75 of the display control unit 70a.

[0112] In addition to the operations switched by the operation switching unit 60 according to the first embodiment, the operation switching unit 60a switches between an input operation of a locus L for generating an instruction figure S and an input operation of a locus L for generating a locus image on which a character string is superimposed and displayed, which are operations accepted by the locus input unit 51a. For example, when the operation switching unit 60a detects a long press operation of the pointing device 207, it performs a process of switching the operations accepted by the locus input unit 51a.

[0113] The display control unit 70a includes a trajectory image generation unit 75 in addition to the configuration of the display control unit 70 according to the first embodiment. The trajectory image generation unit 75 is realized, for example, by the image generation processing circuit 209 shown in FIG. 2. The trajectory image generation unit 75 generates a trajectory image indicating the trajectory L from an input operation of the trajectory L for generating a trajectory image on which the character strings accepted by the trajectory input unit 51a are superimposed and displayed. Similar to the area setting unit 40, the trajectory image generation unit 75 generates a trajectory image having a line width corresponding to the character height indicated by the character attribute information output from the character management unit 30.

[0114] In addition to the processing performed by the display processing unit 74 according to Embodiment 1, the display processing unit 74a performs processing to display the trajectory image generated by the trajectory image generating unit 75 on the display 201 and to display the character string acquired by the character string input unit 10a superimposed on the trajectory image. Furthermore, the display control unit 70a may perform processing to erase the display of the trajectory image after displaying the character string superimposed on the trajectory image.

[0115] Next, an input display method by the input display device 101 will be described with reference to Fig. 27. Fig. 27 is a flowchart showing the steps of the input display method according to a modification of the first embodiment.

[0116] In the input display method using the input display device 101, steps ST26 to ST31 in FIG. 30 are performed instead of step ST1 in FIG. 6 . As shown in FIG. 30 , first, the operation switching unit 60a switches the operation received by the trajectory input unit 51a to an input operation of a trajectory L for generating a trajectory image (step ST26). Specifically, the operation switching unit 60a detects a long press operation by the user, thereby switching to an input operation of a trajectory L for generating a trajectory image. Next, the trajectory input unit 51a determines whether or not an input operation of a trajectory L for generating a trajectory image has been received (step ST27). The trajectory input unit 51a waits until an input operation of a trajectory L for generating a trajectory image is received (No in step ST27). If the trajectory input unit 51a receives an input operation of a trajectory L for generating a trajectory image (Yes in step ST27), the trajectory image generation unit 75 generates a trajectory image (step ST28). Specifically, a trajectory image having a line width corresponding to the character height indicated by the character attribute information output from the character management unit 30 is generated. Subsequently, it is determined whether the voice detection unit 11 of the character string input unit 10 has detected the user's voice (step ST29). If the voice detection unit 11 detects the user's voice (Yes in step ST29), the voice recognition unit 12 uses the voice recognition dictionary DB 84 to perform a voice recognition process, thereby generating a character string indicating the voice recognition result, and outputs the character string to the display control unit 70a (step ST30). The display processing unit 74a of the display control unit 70a displays the trajectory image output from the trajectory image generation unit 75 and the character string output from the character string input unit 10, superimposed on each other, on the display 201 (step ST31).

[0117] The operation switching unit 60a waits until a long press operation is detected (No in step ST32). After the character string is displayed on the display 201, if the operation switching unit 60a detects a long press operation (Yes in step ST32), the operation accepted by the trajectory input unit 51a is switched to an operation for generating a pointer figure (step ST33). After the switching, the pointer figure S is generated based on the operation for generating a pointer figure accepted by the trajectory input unit 51a, as in the input display method by the input display device 100 according to embodiment 1. That is, steps ST2 to ST11 in FIG. 6 are executed.

[0118] The operation and effect of the input display device 101 according to the modified example of the first embodiment of the present disclosure will be described.

[0119] The input display device 101 according to the first embodiment of the present disclosure further includes, in addition to the components of the input display device 100 according to the first embodiment of the present disclosure, a character string acquisition unit 20a that accepts input of a character string, and a trajectory image generation unit 75 that generates a trajectory image showing the trajectory accepted by the trajectory input unit 51a, and the display control unit 70a displays the character string accepted by the character string input unit by superimposing it on the trajectory image.

[0120] According to the input display device 101 of the modified example of the first embodiment, it is possible to display a character string in a layout freely chosen by the user, and to automatically display an indication figure S corresponding to the character string displayed in the freely chosen layout. Furthermore, since the operation of inputting a character string and the operation of generating an indication figure S can be easily switched by the user performing a long press operation, the operation of inputting a character string and the operation of generating a figure can be performed with few steps.

[0121] Although the present disclosure has been described above based on the embodiments, the present disclosure is not limited to the embodiments. Furthermore, appropriate combinations, modifications, omissions, etc. of the embodiments are also included within the scope of the technical idea of ​​the present disclosure.

[0122] REFERENCE SIGNS LIST 10, 10a String input unit 20 String acquisition unit 30, 30a Character management unit 40 Area setting unit 50, 50a Operation reception unit 51, 51a Trajectory input unit 52 Shape setting unit 53 Movement unit 54 Shape editing unit 60, 60a Operation switching unit 70, 70a Display control unit 71 Pointing figure generation unit 72 Trajectory change unit 73 Character string line break unit 74, 74a Display processing unit 75 Trajectory image generation unit 81 Character management DB 82 Pointing figure shape DB 83 Temporary storage unit 84 Speech recognition dictionary DB 100, 101 Input display device 201 Display 202 String input processing circuit 203 String acquisition processing circuit 204 Storage processing circuit 205 Character management processing circuit 206 Area setting processing circuit 207 Pointing device 208 Operation switching processing circuit 209 Image generation processing circuit 210 Trajectory change processing circuit 211 Character string line break processing circuit 212 Display processing circuit 213 Memory 214 Processor 215 Microphone 216 Voice recognition processing circuit

Claims

1. An input display device comprising: an area setting unit that sets a display area for a character string as a character string area; a trajectory input unit that accepts input of a trajectory that connects the inside of the character string area with a designated position that is a position pointed to by the character string located outside the character string area; and a display control unit that generates an indication figure that represents the designated position based on the trajectory located outside the character string area, and displays the generated indication figure.

2. An input display device as described in claim 1, further comprising a shape setting unit that sets the shape of the indication figure to be displayed, and the display control unit generates the indication figure based on the shape of the indication figure set by the shape setting unit.

3. An input display device as described in claim 2, wherein the shape of the indication figure set by the shape setting unit is a line, and the display control unit displays the trajectory connecting the character string area and the intersection of the trajectory with the indication position as the indication figure.

4. The input display device of claim 2, wherein the shape of the indication figure set by the shape setting unit is an arrow, and the display control unit displays as the indication figure a figure in which an arrowhead symbol is added to the trajectory connecting the intersection of the character string area and the trajectory with the indication position, and the arrowhead symbol is displayed at least at either the intersection of the character string area and the trajectory or the indication position.

5. An input display device as described in claim 2, wherein the shape of the indication figure set by the shape setting unit is a speech bubble, and the display control unit displays as the indication figure a figure that surrounds the character string area and the trajectory connecting the intersection of the trajectory with the indication position and the character string area.

6. The input display device according to claim 2, wherein the shape of the indication figure set by the shape setting unit is a speech bubble, and the display control unit displays as the indication figure a figure that combines the character string area and the trajectory connecting the intersection of the trajectory with the indication position and a figure that surrounds the character string area.

7. An input display device according to any one of claims 1 to 6, further comprising a movement unit that accepts movement of the position of the character string or the indication position, wherein the display control unit changes at least one of the size and position of the trajectory so that the trajectory becomes similar in accordance with the movement accepted by the movement unit, and displays the indication figure based on the changed trajectory.

8. An input display device as described in claim 7, wherein the movement unit accepts movement of the position of the character string, and the display control unit changes at least one of the size and position of the trajectory so that the indication position located outside the character string area and the relative position of one end of the trajectory located inside the character string area with respect to the character string area are the same before and after the change of the trajectory, and displays the indication figure based on the changed trajectory.

9. An input display device as described in claim 7, wherein the movement unit accepts movement of the indication position, and the display control unit changes at least one of the size and position of the trajectory so that the position of one end of the trajectory located within the character string area remains the same before and after the change in the trajectory, and displays the indication figure based on the changed trajectory.

10. An input display device as described in any one of claims 1 to 9, further comprising a shape editing unit that accepts editing of the shape of the instruction figure, and wherein the display control unit, when the instruction figure is a speech bubble, causes the character string displayed outside the speech bubble to be displayed inside the speech bubble by inserting a line break.

11. An input display device according to any one of claims 1 to 10, further comprising: a character string input unit that accepts input of the character string; and a trajectory image generation unit that generates a trajectory image showing the trajectory accepted by the trajectory input unit, wherein the display control unit displays the character string accepted by the character string input unit superimposed on the trajectory image.

12. An input display method including: an area setting step of setting a display area for a character string as a character string area; a trajectory input step of accepting input of a trajectory connecting the inside of the character string area with a designated position that is a position pointed to by the character string located outside the character string area; and a display control step of generating an indication figure that represents the designated position based on the trajectory located outside the character string area, and displaying the generated indication figure.

13. An input display program for causing a computer to implement the following: an area setting function for setting a display area for a character string as a character string area; a trajectory input function for accepting input of a trajectory connecting the inside of the character string area with a designated position that is a position pointed to by the character string located outside the character string area; and a display control function for generating an indication figure that represents the designated position based on the trajectory located outside the character string area, and displaying the generated indication figure.

Images