Input system

Abstract

To provide an input system capable of improving specific inconvenience caused by data processing of a host processor.SOLUTION: An input system 10A includes: a first signal path SP1 through which a signal is transmitted from a sensor controller 18 to a display controller 20 via a host processor 22; a second signal path SP2 through which a signal is transmitted from the sensor controller 18 to the display controller 20 without passing through the host processor 22; an output selection unit 42 configured to be able to switch the first signal path SP1 and the second signal path SP2; and a mode control unit 44 for switching and executing a first output mode in which the first signal path SP1 is selected and a second output mode in which the second signal path SP2 is selected.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an input system using a touch panel display. [Background technology]

[0002] Patent Document 1 discloses a method for establishing a Transport Layer Security (TLS) connection between an electronic device equipped with a touch panel display and an external server, and for securely communicating data including confidential information. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-038036 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, when a host processor of an electronic device processes input data from a touch panel display, some technical or operational inconvenience may occur due to the data passing through the host processor.

[0005] SUMMARY OF THE PRESENT EMBODIMENT An object of the present invention is to provide an input system capable of improving certain inconveniences caused by data processing in a host processor. [Means for solving the problem]

[0006] The input system of the first invention is a system using a touch panel display in which sensor electrodes for detecting a position pointed to by a user are provided in a display area of ​​a display panel, and includes a sensor controller that acquires the pointed position within the display area based on a detection result of the sensor electrodes, a host processor that processes a data signal including the pointed position acquired by the sensor controller, a display controller that outputs a display signal indicating an image or video to the display panel, a first signal path that transmits a signal from the sensor controller to the display controller via the host processor, a second signal path that transmits a signal from the sensor controller to the display controller without passing through the host processor, and a switch unit configured to be able to switch between the first signal path and the second signal path.

[0007] The input system of the second invention is a system using a touch panel display in which sensor electrodes for detecting a position pointed to by a user are provided in a display area of ​​a display panel, and comprises a host processor for processing a data signal including the pointed position, a display controller connected to the host processor and outputting a display signal indicating an image or video to the display panel, and a sensor controller for obtaining the pointed position within the display area based on the detection result of the sensor electrodes, and the sensor controller is physically or logically connected to the host processor and also physically connected to the display controller. Effect of the Invention

[0008] The input system of the present invention alleviates certain inconveniences caused by data processing in a host processor. [Brief description of the drawings]

[0009] [Figure 1] 1 is an overall configuration diagram of an input system common to each embodiment of the present invention. [Diagram 2] FIG. 2 is a connection configuration diagram of the input system according to the first embodiment. [Diagram 3] FIG. 3 is a functional block diagram corresponding to FIG. 2. [Figure 4] FIG. 11 is a connection configuration diagram of an input system according to a second embodiment. [Diagram 5] FIG. 5 is a functional block diagram corresponding to FIG. 4. [Figure 6] FIG. 11 is a connection configuration diagram of an input system according to a third embodiment. [Figure 7] FIG. 7 is a functional block diagram corresponding to FIG. 6. [Figure 8] FIG. 11 is a sequence diagram when a first output mode is executed. [Figure 9] FIG. 13 is a diagram illustrating an example of an interpolation process. [Figure 10] 11A to 11C are diagrams showing transitions of a display image in response to a stylus operation. [Figure 11] FIG. 11 is a sequence diagram when a second output mode is executed. [Figure 12] 8 is a detailed functional block diagram of the partial image generating unit shown in FIGS. 3, 5 and 7. FIG. [Figure 13] 13A and 13B are diagrams for comparing writing follow-up performance with respect to high-speed movement of a stylus. [Figure 14] FIG. 13 is a diagram illustrating an example of an effect of additional processing. [Figure 15] FIG. 13 is a diagram illustrating an example of an effect of additional processing. [Figure 16] FIG. 2 is a diagram showing the overall configuration of a signature verification system in which the input system shown in FIG. 1 is incorporated. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] The input system of the present invention will be described below with reference to the accompanying drawings. The present invention is not limited to the embodiments and modifications described below, and can be freely modified without departing from the spirit of the present invention. Alternatively, each configuration may be arbitrarily combined within the scope of no technical contradiction.

[0011] [Configuration of Input System 10] <Overall composition> 1 is a diagram showing the overall configuration of an input system 10 common to each embodiment of the present invention. The input system 10 basically comprises an electronic device 14 having a touch panel display 12, and a stylus 16 which is a pen-type pointing device.

[0012] The electronic device 14 is, for example, a tablet terminal, a smartphone, or a personal computer. The stylus 16 is configured to be capable of one-way or two-way communication with the electronic device 14. A user can write pictures or characters on the electronic device 14 by holding the stylus 16 in one hand and moving the pen tip against the touch surface of the touch panel display 12.

[0013] In addition to the touch panel display 12, the electronic device 14 includes a sensor controller 18, a display controller 20, a host processor 22, a memory 24, and a communication module 26.

[0014] The touch panel display 12 includes a display panel 28 capable of visibly displaying content, and a sensor electrode 30 disposed above the display panel 28. The display panel 28 is capable of displaying monochrome or color images, and may be, for example, a liquid crystal panel or an organic EL (Electro-Luminescence) panel. The sensor electrode 30 includes a plurality of X-line electrodes for detecting an X-axis position in a sensor coordinate system, and a plurality of Y-line electrodes for detecting a Y-axis position.

[0015] The sensor controller 18 is an integrated circuit that controls the driving of the sensor electrodes 30. The sensor controller 18 drives the sensor electrodes 30 based on a control signal supplied from the host processor 22. In this way, the sensor controller 18 executes a "pen detection function" for detecting the state of the stylus 16 and a "touch detection function" for detecting a touch by a user's finger or the like.

[0016] The pen detection function includes, for example, a function of scanning the sensor electrodes 30, a function of receiving and analyzing downlink signals, a function of estimating the state of the stylus 16 (for example, position, posture, writing pressure), and a function of generating and transmitting uplink signals including commands to the stylus 16. In addition, the touch detection function includes, for example, a function of two-dimensionally scanning the sensor electrodes 30, a function of acquiring a detection map on the sensor electrodes 30, and a function of classifying areas on the detection map (for example, classification into fingers, palm, etc.).

[0017] The display controller 20 is an integrated circuit that controls the driving of the display panel 28. The display controller 20 drives the display panel 28 based on frame-by-frame image signals supplied from the host processor 22. As a result, an image or video is displayed within the display area R of the display panel 28. This image or video may include application windows, icons, a cursor, and also handwritten lines drawn by the user using the stylus 16.

[0018] In this way, a user interface (hereinafter, UI unit 32) is constructed by combining the input function of the stylus 16 and the sensor electrodes 30 with the output function of the display panel .

[0019] The host processor 22 is configured with a processing operation device including a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), and a GPU (Graphics Processing Unit). The host processor 22 can execute various functions including generating digital ink, creating image signals, and controlling the transmission and reception of data by reading and executing programs stored in the memory 24.

[0020] As will be described later, this input system 10 has a technical feature in that the sensor controller 18 is physically or logically connected to a host processor 22 and is also physically connected to a display controller 20.

[0021] The memory 24 is non-transient and is configured with a computer-readable storage medium. Here, the computer-readable storage medium is a storage device including a hard disk drive (HDD: Hard Disk Drive) and a solid state drive (SSD: Solid State Drive), or a portable medium such as a magneto-optical disk, a ROM, a CD-ROM, or a flash memory.

[0022] The communication module 26 is configured to be capable of transmitting and receiving electrical signals to and from an external device through wired or wireless communication, which enables the electronic device 14 to exchange various data with, for example, the validation server 122 (FIG. 16).

[0023] Next, first to third embodiments of the input system 10 in Fig. 1 will be described with reference to Fig. 2 to Fig. 7. It should be noted that in all of the figures, only the main components, that is, the sensor controller 18, the display controller 20, the host processor 22, the display panel 28, and the sensor electrodes 30, are shown, and other components are omitted.

[0024] First Embodiment 2 is a connection configuration diagram of an input system 10A in the first embodiment. The sensor controller 18 in the first embodiment is physically connected to the display controller 20, and is configured to be able to switch the output destination of a data signal to either the host processor 22 or the display controller 20.

[0025] Specifically, the sensor controller 18, the display controller 20, and the host processor 22 are physically connected by signal lines L1 to L3. The sensor controller 18 is connected to the host processor 22 via signal line L1. The host processor 22 is connected to the display controller 20 via signal line L2. The sensor controller 18 is connected to the display controller 20 via signal line L3. For example, the signal lines L1 and L3 are signal lines compatible with USB (Universal Serial Bus), and the signal line L2 is a signal line compatible with LVDS (Low Voltage Differential Signaling).

[0026] That is, the signal lines L1 and L2 constitute a first signal path SP1 that transmits a signal from the sensor controller 18 to the display controller 20 via the host processor 22. On the other hand, the signal line L3 constitutes a second signal path SP2 that transmits a signal from the sensor controller 18 to the display controller 20 without passing through the host processor 22.

[0027] Fig. 3 is a functional block diagram corresponding to Fig. 2. The sensor controller 18 functions as a position detection unit 40 and an output selection unit 42 (switch unit). The host processor 22 functions as a mode control unit 44 and a data processing unit 46. The display controller 20 functions as two types of output processing units 50, 52.

[0028] The output processing unit 50 is configured to be capable of executing output processing for outputting a display signal to the display panel 28. Specifically, the output processing unit 50 is configured to include a partial image generating unit 54, an additional processing unit 56, an image synthesis unit 58, and a signal generating unit 60. The additional processing unit 56 is a functional unit whose presence or absence can be selected.

[0029] The output processing unit 52 is configured to be capable of executing output processing for outputting a data signal to the host processor 22. Specifically, the output processing unit 52 is configured to include a storage unit 62 and an additional processing unit 64. Note that the additional processing unit 64 is a functional unit whose presence or absence can be selected.

[0030] <Second embodiment> 4 is a connection configuration diagram of an input system 10B in the second embodiment. The input system 10B in the second embodiment further includes a physical switch 66 (switch unit) having one input terminal and two output terminals.

[0031] Specifically, the sensor controller 18, the display controller 20, the host processor 22, and the physical switch 66 are physically connected by signal lines L4 to L8. The sensor controller 18 is connected to an input terminal of the physical switch 66 via a signal line L4. One output terminal of the physical switch 66 is connected to the host processor 22 via a signal line L5. The host processor 22 is connected to the display controller 20 via a signal line L6. The other output terminal of the physical switch 66 is connected to the display controller 20 via a signal line L7. The signal line L8 is a signal line through which the host processor 22 supplies a switch signal to the physical switch 66. For example, the signal lines L4, L5, and L7 are signal lines compatible with USB, and the signal line L6 is a signal line compatible with LVDS.

[0032] That is, the signal lines L4, L5, and L6 configure a first signal path SP1 that transmits a signal from the sensor controller 18 to the display controller 20 via the host processor 22. On the other hand, the signal lines L4 and L7 configure a second signal path SP2 that transmits a signal from the sensor controller 18 to the display controller 20 without passing through the host processor 22.

[0033] Fig. 5 is a functional block diagram corresponding to Fig. 4. The sensor controller 18 functions as a position detection unit 40. The host processor 22 functions as a mode control unit 44 and a data processing unit 46. The display controller 20 functions as two types of output processing units 50, 52. As can be seen from Figs. 3 and 5, the input system 10B has a different configuration from the input system 10A of the first embodiment in that the sensor controller 18 lacks the function of the output selection unit 42 (see Fig. 3) and instead a physical switch 66 is provided on the output side of the sensor controller 18.

[0034] <Third embodiment> 6 is a connection configuration diagram of an input system 10C in the third embodiment. The display controller 20 in the third embodiment is logically connected to the sensor controller 18, and is configured to be able to switch whether or not to supply a data signal from the sensor controller 18 to the host processor 22 by pass-through transmission.

[0035] Specifically, the sensor controller 18, the display controller 20, and the host processor 22 are physically connected by signal lines L9 to L11. The sensor controller 18 is connected to the display controller 20 via the signal line L9. The display controller 20 is connected to the host processor 22 via the signal line L10. The host processor 22 is connected to the display controller 20 via the signal line L11. For example, the signal lines L9 and L10 are signal lines compatible with USB, and the signal line L11 is a signal line compatible with LVDS.

[0036] That is, the signal lines L9, L10, and L11 constitute a first signal path SP1 that transmits a signal from the sensor controller 18 to the display controller 20 via the host processor 22. On the other hand, the signal line L9 constitutes a second signal path SP2 that transmits a signal from the sensor controller 18 to the display controller 20 without passing through the host processor 22.

[0037] Fig. 7 is a functional block diagram corresponding to Fig. 6. The sensor controller 18 functions as a position detection unit 40. The host processor 22 functions as a mode control unit 44 and a data processing unit 46. The display controller 20 functions as two types of output processing units 50, 52 and a logical switch 68. As can be understood from Figs. 3 and 7, this input system 10C has a different configuration from the input system 10A of the first embodiment in that the sensor controller 18 lacks the function of the output selection unit 42 (see Fig. 3) and instead has the function of the logical switch 68.

[0038] [Input System 10 Operation] The input system 10 in each embodiment is configured as described above. Next, the operation of the input system 10 will be described with reference to Fig. 8 to Fig. 13. The mode control unit 44 of the host processor 22 switches between a "first output mode" in which signal processing is performed using the first signal path SP1 and a "second output mode" in which signal processing is performed using the second signal path SP2.

[0039] In each of the above-described input systems 10A, 10B, and 10C, the host processor 22 is provided with the function of the mode control unit 44. Alternatively, the sensor controller 18 or the display controller 20 may be provided with this function.

[0040] <First output mode> 8 is a sequence diagram when the first output mode is executed. This diagram shows the operations of the sensor controller 18, the host processor 22, and the display controller 20 from left to right.

[0041] In step S01, the host processor 22 outputs a signal for requesting writing data including the position indicated by the stylus 16 (hereinafter also referred to as a request signal) to the sensor controller 18.

[0042] In step S02, after receiving the request signal supplied in step S01, the position detection unit 40 of the sensor controller 18 performs drive control of the sensor electrode 30 to detect the current position pointed to by the user. In this way, the sensor controller 18 obtains a position on the sensor coordinate system defined within the display area R of the display panel 28.

[0043] In step S03, the sensor controller 18 outputs a data signal including the indicated position detected in step S02 to the host processor 22. Prior to this output, for example, in the case of the input system 10A in Figs. 2 and 3, the output selection unit 42 of the sensor controller 18 switches the output destination of the data signal to the host processor 22. Also, in the case of the input system 10B in Figs. 4 and 5, the output terminal of the physical switch 66 is switched to the host processor 22. Also, in the case of the input system 10C in Figs. 6 and 7, the logical switch 68 of the display controller 20 is switched to "on".

[0044] This switching operation enables the sensor controller 18 to selectively output the data signal it generates to the host processor 22. Note that this data signal may include various information such as information on the writing state of the stylus 16 (hereinafter, writing information) and meta-information related to the user's writing. Examples of the writing information include the current pointing position, writing pressure, pen inclination, etc. Examples of the meta-information include the pen ID, data attributes, creator, creation date and time, etc.

[0045] In step S04, the data processing unit 46 of the host processor 22 obtains handwriting data including the indicated position from the data signal supplied in step S3, and performs an interpolation process to update the trajectory curve 80 using the obtained current indicated position. This interpolation process will be described in detail below with reference to FIG.

[0046] Fig. 9 is a diagram showing an example of the interpolation process. More specifically, Fig. 9(a) shows the shape of the locus curve 80 before the interpolation process, and Fig. 9(b) shows the shape of the locus curve 80 after the interpolation process. The XY coordinate system corresponds to a "sensor coordinate system" for specifying the detection position of the sensor electrode 30, that is, the indicated position on the display panel 28.

[0047] The locus curve 80 shown in FIG. 9(a) is a curve that sequentially connects discrete points P1, P2, and P3 that are past designated positions. For example, assume that discrete point P4, which is the current designated position, is added after discrete point P3, which is the previous designated position. Here, the data processing unit 46 performs an interpolation process that continuously connects the discrete points P3 and P4 using various methods including linear interpolation and spline interpolation. Then, as shown in FIG. 9(b), a new locus curve 80 is formed by combining the original locus curve 82 that passes through the discrete points P1, P2, and P3 with an interpolation curve 84 that passes through the discrete points P3 and P4.

[0048] Then, the data processing unit 46 generates an image signal on a frame-by-frame basis by synthesizing a stroke image showing a collection of strokes 90 (see FIG. 10(b)) formed by the above-mentioned interpolation process with a non-written image excluding the stroke image (e.g., an application window, a widget, a cursor, an icon, etc.).

[0049] In step S05 of FIG. 8, the host processor 22 outputs the image signal generated in step S04 to the display controller 20.

[0050] In step S06, the output processing unit 50 (more specifically, the signal generating unit 60) of the display controller 20 uses the image signal supplied in step S05 to generate a display signal for driving and controlling the display panel 28. In the first output mode, the signal generating unit 60 generates the display signal using the image signal that has passed through the image synthesis unit 58 as is.

[0051] In step S07, the output processing unit 50 outputs the display signal generated in step S06 to the display panel 28. As a result, the entire image including the stroke image is displayed in the display region R of the display panel 28. Thereafter, the input system 10 sequentially repeats steps S01 to S07 at intervals of one frame time.

[0052] Fig. 10 is a diagram showing the transition of a display image accompanying the operation of the stylus 16. More specifically, Fig. 10(a) shows the stationary state of the stylus 16, and Fig. 10(b) shows the moving state of the stylus 16.

[0053] As shown in Fig. 10(a), when the stylus 16 is in a stationary state, a cursor 88 is displayed in a position corresponding to the pen tip 86 within the display region R. The user moves the stylus 16 while pressing the pen tip 86 against the touch surface of the electronic device 14. As a result, as shown in Fig. 10(b), an image or video is successively updated following the pen tip 86 of the stylus 16. Specifically, the cursor 88 moves along the movement path of the stylus 16, and a stroke 90 corresponding to the trajectory of the indicated position appears to extend.

[0054] 8, when a predetermined processing timing is reached, the data processing unit 46 of the host processor 22 performs various processes on the handwriting data sequentially acquired from the sensor controller 18. Examples of such processes include a process of generating digital ink and a process of transmitting data via the communication module 26. In this manner, the input system 10 ends the operation according to the first output mode.

[0055] <Second output mode> However, when the host processor 22 of the electronic device 14 processes input data from the touch panel display 12, some technical or operational inconvenience may occur due to the processing going through the host processor 22. Therefore, the mode control unit 44 of the host processor 22 can improve the specific inconvenience by switching from the first output mode to the second output mode as necessary.

[0056] Fig. 11 is a sequence diagram when the second output mode is executed. Like Fig. 8, this diagram shows the operations of the sensor controller 18, the host processor 22, and the display controller 20 from left to right.

[0057] In step S11, the host processor 22 receives a command signal to transition from the first output mode to the second output mode. The trigger for generating this command signal will be described later.

[0058] In step S12, the host processor 22 outputs a flag signal indicating the start of execution of the second output mode to each unit involved in the operation of the second output mode. For example, in the case of the input system 10A in Figs. 2 and 3, the flag signal is output to the sensor controller 18 and the display controller 20. In addition, in the case of the input system 10B in Figs. 4 and 5, the flag signal is output to the display controller 20 and the physical switch 66. In addition, in the case of the input system 10C in Figs. 6 and 7, the flag signal is output to the display controller 20.

[0059] In step S13, the data processing unit 46 of the host processor 22 generates an image signal indicating an entire image Iorg (FIG. 12) in units of frames, and outputs the image signal to the display controller 20. Note that if handwriting input was being performed immediately before switching to the second output mode, the data processing unit 46 generates an image signal using the last updated stroke image.

[0060] In step S14, the host processor 22 outputs a request signal to the sensor controller 18 to request writing data including the position indicated by the stylus 16. This operation is the same as step S01 in FIG.

[0061] In step S15, the position detection unit 40 of the sensor controller 18 performs an operation for detecting the current position indicated by the user after receiving the request signal supplied in step S14. This operation is the same as the operation in step S02 in FIG.

[0062] In step S16, the sensor controller 18 outputs a data signal including the current indication position detected in step S15 to the display controller 20. Prior to this output, for example, in the case of the input system 10A in Figs. 2 and 3, the output selection unit 42 of the sensor controller 18 switches the output destination of the data signal to the display controller 20. Also, in the case of the input system 10B in Figs. 4 and 5, the output terminal of the physical switch 66 is switched to the display controller 20. Also, in the case of the input system 10C in Figs. 6 and 7, the logical switch 68 of the display controller 20 is switched to "off."

[0063] This switching operation enables the sensor controller 18 to selectively output the data signal generated by itself to the display controller 20. This data signal may contain the same type of information as in the first output mode (for example, the handwritten information and meta information described above), or may contain a different type of information.

[0064] In step S17, the output processing unit 50 (more specifically, the partial image generating unit 54) of the display controller 20 obtains handwriting data including the indicated position from the data signal supplied in step S16, and generates a partial image Ipar using the obtained current indicated position. This generation method will be described in detail below with reference to FIG.

[0065] Fig. 12 is a detailed functional block diagram of the partial image generating unit 54 shown in Fig. 3, Fig. 5, and Fig. 7. This partial image generating unit 54 includes an interpolator 92 that performs an interpolation process, and a combiner 94 that combines a plurality of small images.

[0066] The interpolator 92 generates a small image (hereinafter, referred to as a difference image Isub) showing an interpolation curve between the two points by continuously connecting the two discrete points using the time series of the designated positions. This difference image Isub means the difference between the stroke images, and in the example of Fig. 9(b), it corresponds to an image showing the interpolation curve 84 passing through the discrete points P3 and P4.

[0067] The combiner 94 sequentially combines the difference images Isub supplied from the interpolator 92 to generate an intermediate image (hereinafter referred to as a partial image Ipar) that indicates a collection of interpolation curves. In the example of FIG. 9(b), this partial image Ipar corresponds to an image that indicates the locus curve 80 after all of the discrete points P1 to P4 are connected. The generated partial image Ipar is supplied to the image synthesis unit 58 and is also fed back to the combiner 94.

[0068] In this manner, the partial image generating unit 54 generates a partial image Ipar that includes at least the currently designated position, and outputs this partial image Ipar to the image combining unit 58 (step S17).

[0069] 11, the output processing unit 50 (more specifically, the image synthesis unit 58) synthesizes the entire image Iorg supplied in step S13 with the partial image Ipar generated in step S17. Specifically, the image synthesis unit 58 generates an edited image Iprc by replacing an image of a corresponding area of ​​the entire image Iorg supplied once or sequentially from the host processor 22 with the partial image Ipar (see FIG. 12). Alternatively, instead of using the entire image Iorg, the image synthesis unit 58 may generate the edited image Iprc by performing padding processing on the partial image Ipar to fill in the remaining display area.

[0070] In step S19, the display controller 20 generates a display signal from the image signal indicating the processed image Iprc synthesized in step S18, and outputs the obtained display signal to the display panel 28. This operation is the same as the operations in steps S06 and S07 in Fig. 8. As a result, the processed image Iprc including the stroke image is displayed in the display region R of the display panel 28.

[0071] Thereafter, the input system 10 sequentially repeats steps S13 to S19 at intervals of one frame time. When the entire image Iorg is not updated sequentially, the display controller 20 stores one frame of image signals, so that step S13 can be omitted from the second time onwards.

[0072] Fig. 13 is a diagram comparing the writing followability with respect to high-speed movement of the stylus 16. More specifically, Fig. 13(a) shows a display image when the first output mode is executed, and Fig. 13(b) shows a display image when the second output mode is executed.

[0073] 13(a), in the first output mode, as the stylus 16 moves at high speed, the position of the cursor 88 shifts backward relative to the pen tip 86, and the leading position of the stroke 90 shifts further backward relative to the cursor 88. This may cause a user using the stylus 16 to feel uncomfortable due to the discrepancy in the indicated positions.

[0074] The former positional deviation may occur due to a response time difference corresponding to the frame interval, while the latter positional deviation may occur when, for example, the images of the cursor 88 and the stroke 90 are created using different designated positions with different acquisition timings due to hardware constraints including data processing and signal transmission.

[0075] 13(b), in the second output mode, a new stroke 98 is formed by adding a line segment 96 to the beginning of the original stroke 90 as the stylus 16 moves at high speed. In other words, although the positional deviation between the pen tip 86 and the cursor 88 remains, the positional deviation between the cursor 88 and the stroke 98 is largely eliminated. This alleviates the discomfort felt by the user due to the mismatch in the designated positions, and accordingly improves usability.

[0076] 11, the host processor 22 receives a command signal to transition from the second output mode to the first output mode. The trigger for generating this command signal will be described later.

[0077] In step S21, the host processor 22 outputs a flag signal indicating the start of execution of the first output mode to each unit involved in the operation of the first output mode. For example, in the case of the input system 10A in Figs. 2 and 3, the flag signal is output to the sensor controller 18 and the display controller 20. In addition, in the case of the input system 10B in Figs. 4 and 5, the flag signal is output to the display controller 20 and the physical switch 66. In addition, in the case of the input system 10C in Figs. 6 and 7, the flag signal is output to the display controller 20.

[0078] In step S22, the output processing unit 52 of the display controller 20 reads out the data temporarily stored in the storage unit 62, and outputs a data signal including a time series of the indicated position to the host processor 22 all at once.

[0079] In step S23, the data processing unit 46 of the host processor 22 obtains a set of handwriting data from the data signal supplied in step S22, and performs various processes on the handwriting data. For example, the data processing unit 46 may merge the handwriting data already obtained in the first output mode with the handwriting data newly obtained in the second output mode, and then perform a digital ink generation process and a data transmission process. In this manner, the input system 10 ends the operation corresponding to the second output mode.

[0080] <Effects of Input System 10> As described above, the input system 10 is a system using a touch panel display 12 having a sensor electrode 30 provided in a display area R of a display panel 28 for detecting a position pointed to by a user, and includes a sensor controller 18 that acquires the pointed position within the display area R based on the detection result of the sensor electrode 30, a host processor 22 that processes a data signal including the acquired pointed position, a display controller 20 that outputs a display signal indicating an image or video to the display panel 28, a first signal path SP1 that transmits a signal from the sensor controller 18 to the display controller 20 via the host processor 22, a second signal path SP2 that transmits a signal from the sensor controller 18 to the display controller 20 without passing through the host processor 22, and a switch unit (output selection unit 42, physical switch 66, logical switch 68) configured to be able to switch between the first signal path SP1 and the second signal path SP2.

[0081] In this way, a switch section capable of switching between the first signal path SP1 and the second signal path SP2 is provided, so that by selecting the second signal path SP2, data processing can be performed in a timely manner without passing through the host processor 22, thereby improving certain inconveniences caused by the data processing of the host processor 22.

[0082] The mode control unit 44 may switch between a first output mode in which the first signal path SP1 is selected by the switch unit and a second output mode in which the second signal path SP2 is selected by the switch unit. During the first output mode, the display controller 20 generates a display signal using the signal that has passed through the first signal path SP1, and outputs the display signal to the display panel 28. During the second output mode, the display controller 20 performs a predetermined process on the data signal that has passed through the second signal path SP2, and then outputs the processed display signal to the display panel 28. By using the second signal path SP2, which is shorter than the first signal path SP1, the time response of the display process is improved.

[0083] Furthermore, during execution of the second output mode, the host processor 22 outputs an image signal indicative of the entire image Iorg corresponding to the display region R to the display controller 20, and the display controller 20 may successively acquire the pointed position from the data signal passing through the second signal path SP2, generate a processed image Iprc by modifying at least a portion of the entire image Iorg that includes the current pointed position, and output a display signal indicative of the processed image Iprc to the display panel 28. It is only necessary to add a predetermined process for modifying only the portion that includes the current pointed position to the first output mode, and accordingly, an excessive increase in the processing load of the display controller 20 is suppressed.

[0084] Furthermore, the display controller 20 may generate a partial image Ipar including a stroke 98 (the locus of the indicated position) from the time series of the data signal that has passed through the second signal path SP2, and generate the processed image Iprc by synthesizing the entire image Iorg and the partial image Ipar. A predetermined process for generating the stroke 98 and synthesizing the images may be added to the first output mode, and an excessive increase in the processing load of the display controller 20 is suppressed accordingly.

[0085] [Examples of mode switching] A specific example of output mode switching timing will be described below. The mode control unit 44 of the host processor 22 may receive an instruction signal triggered by a user's operation (steps S11 and S20 in FIG. 11). This allows the user to recognize the state of the output mode through his or her own operation.

[0086] Hereinafter, an operation for changing from the first output mode to the second output mode will be referred to as a "first operation," and an operation for changing from the second output mode to the first output mode will be referred to as a "second operation." The second operation is the same as or different from the first operation. Furthermore, the first operation and the second operation may be hardware operations or software operations.

[0087] The hardware operations include pressing a hardware button (a so-called function switch) provided on the electronic device 14 or the stylus 16, or a pen-down operation or a pen-up operation of the stylus 16. The pen-down operation or the pen-up operation is detected by a pen pressure sensor (not shown) provided on the pen tip 86. On the other hand, the software operations include a specific operation that causes a state transition of an application running on the host processor 22, such as an operation of a user control on the display screen, or an operation to open or switch an application window.

[0088] For example, the first operation may be a pen-down operation of the stylus 16, and the second operation may be a pen-up operation corresponding to the pen-up operation. This makes it possible to timely switch the output mode for each series of stroke operations.

[0089] Furthermore, the mode control unit 44 may prohibit switching from the first output mode to the second output mode despite the receipt of a first operation while the contact state of the stylus 16 is detected. In addition to or separately from this, the mode control unit 44 may prohibit switching from the second output mode to the first output mode despite the receipt of a second operation while the contact state of the stylus 16 is detected. This prevents a block of writing data indicating one stroke 90 from being divided and transmitted to the host processor 22 and the display controller 20.

[0090] Furthermore, the mode control unit 44 may switch between the first output mode and the second output mode depending on the state of an application executed by the host processor 22. This allows the output mode to be switched at an appropriate time in conjunction with the application.

[0091] [Additional Processing] However, when a user writes confidential information having confidentiality or copyright through the UI unit 32, operational inconvenience may occur regarding the handling of the confidential information. Therefore, this inconvenience can be improved by executing the second output mode as necessary. The additional processing by the output processing units 50, 52 (additional processing units 56, 64) of the display controller 20 will be described below with reference to Figs. 14 to 16.

[0092] <1. Protection of visible information> If the secret information written by the user continues to be displayed on the display panel 28, there is a possibility that the contents may be intercepted by a third party. Therefore, the output processing unit 50 (more specifically, the additional processing unit 56) may perform additional processing to protect the visible information. In this case, the operation of step S24 shown below may be added between steps S17 and S18 in FIG. 11.

[0093] In step S24, the additional processing unit 56 performs additional processing on the partial image Ipar obtained in step S17 to protect visible information. Examples of this additional processing include concealment processing, unsharpening processing, and information addition processing. An example of the effect of the additional processing will be described below with reference to Figs. 14 and 15.

[0094] As shown in Fig. 14(a), content 100 showing a user's signature is displayed within display area R. A name and the handwriting of a signature are types of personal information, and personal information may be identified from the contents of a signed electronic document. Therefore, the display mode of content 100 may be changed over time during or after signing.

[0095] As shown in Fig. 14(b), after a predetermined time (e.g., 10 seconds) has elapsed since the end of the sign, a mask area 102 appears from the left side of the content 100 and conceals a part of the content 100. As time passes, the mask area 102 expands, and finally, as shown in Fig. 14(c), the mask area 102 conceals the entire content 100.

[0096] 14(d), the visibility of all or part of the content 100 may be reduced by applying an unsharpening process to the unsharpened region 104. Note that, in addition to the unsharpening process, various processes may be used that impart visual effects that relatively reduce the user's visibility.

[0097] In this way, the display controller 20 may generate a partial image Ipar in which the content 100 (the trajectory of the pointed position) indicated by the time series of the data signal that has passed through the second signal path SP2 is concealed or unsharpened, and may generate a processed image Iprc in which at least a part of the content is concealed or unsharpened by combining the entire image Iorg and the partial image Ipar. This makes it possible to suppress the visibility of the partial image Ipar, and to prevent information leakage due to a third party peeking at it.

[0098] As shown in Fig. 15, content 106 including a character string consisting of "This is a pencil." and a picture of a pencil is displayed in display area R. This content 106 may include highly confidential or copyrighted content (hereinafter, specific content 108). Therefore, a copyright mark 112 indicating the existence of a copyright may be displayed in a position near the specific content 108 within a designated area 110 designated by the user. Also, instead of copyright mark 112, a confidentiality mark generating a confidentiality obligation may be displayed.

[0099] In this way, the display controller 20 may generate a partial image Ipar to which information indicating confidentiality or copyright (e.g., copyright mark 112) is added near the specific content 108 (the trajectory of the pointed position) indicated by the time series of the data signal that has passed through the second signal path SP2, and generate a processed image Iprc to which the information is added to at least a part of the content by combining the entire image Iorg and the partial image Ipar. This provides an effect of preventing information leakage to a third party who views the partial image Ipar.

[0100] <2. Data Protection> For example, if the security of the electronic device 14 is breached, there is a high possibility that the handwriting data will be leaked, altered, or destroyed through the host processor 22. Therefore, the display controller 20 may sequentially acquire handwriting data including handwriting information on the writing state of the stylus 16 and meta-information related to the user's handwriting from the data signal passing through the second signal path SP2 while the second output mode is being executed, and output a data signal including a time series of the handwriting data that has been subjected to a predetermined process to the host processor.

[0101] Therefore, the output processing unit 52 (more specifically, the additional processing unit 56) performs additional processing to protect the data. In this case, the operation of step S25 shown below may be added between steps S21 and S22 in FIG.

[0102] In step S25, the additional processing unit 64 reads out the data temporarily stored in the storage unit 62, and performs additional processing to protect the data. Examples of this additional processing include irreversible processing, attachment processing, and encryption processing.

[0103] Here, "irreversible processing" refers to processing that irreversibly loses at least one of the handwriting information and meta information. Examples of irreversible processing include deleting pen IDs, deleting writing pressure, thinning or smoothing data strings, concealing the writing order, making strokes unsharpened, converting to raster images, and adding redundant information.

[0104] The term "imparting process" refers to a process of imparting information indicating confidentiality or copyright to the time series of handwritten data. Examples of the imparting process include adding a tag and embedding visible information including a mark.

[0105] Moreover, the "encryption process" can be performed using various methods including a public key method, a cryptographic key method, and a common key method. A specific example of the encryption process will be described below with reference to FIG.

[0106] Fig. 16 is an overall configuration diagram of a signature verification system 120 incorporating the input system 10 shown in Fig. 1. This signature verification system 120 is configured to include a verification server 122 in addition to the electronic device 14 and the stylus 16. A card reader 126 capable of reading a credit card 124 is connected to the electronic device 14. In addition, the electronic device 14 is capable of bidirectional communication with the verification server 122 via a relay device 128 and a network NW.

[0107] The electronic device 14 obtains the encryption key Ken stored in the credit card 124 through the card reader 126, and then temporarily stores the encryption key Ken in the memory 24. The host processor 22 reads out the encryption key Ken at a desired processing timing, performs encryption processing on the handwritten data, and outputs the obtained encrypted data D1 to the host processor 22. The host processor 22 performs processing to transmit the encrypted data D1 to the verification server 122 through the communication module 26.

[0108] Then, the verification server 122 performs a decryption process on the encrypted data D1 from the electronic device 14 using the decryption key Kde corresponding to the encryption key Ken. This allows the verification server 122 to obtain signature data D2 indicating the user's signature and to verify the validity of the signature. [Explanation of symbols]

[0109] 10 (A, B, C) input system, 12 touch panel display, 14 electronic device, 16 stylus, 18 sensor controller, 20 display controller, 22 host processor, 24 memory, 26 communication module, 28 display panel, 30 sensor electrode, 42 output selection unit (switch unit), 44 mode control unit, 46 data processing unit, 50, 52 output processing unit, 66 physical switch (switch unit), 68 virtual switch (switch unit), 80 trajectory curve, 90, 98 stroke, Ipar partial image, Iorg whole image, Iprc processed image, L1 to L11 signal lines, R display area, SP1 first signal path, SP2 second signal path

Claims

1. An input system using a touch panel display, the touch panel display having a sensor electrode disposed in a display area of ​​a display panel for detecting a position pointed to by a user, a sensor controller that acquires a pointed position within the display area based on a detection result of the sensor electrodes; a host processor for processing a data signal including the indicated position acquired by the sensor controller; a display controller that outputs a display signal representing an image or video to the display panel; a first signal path for transmitting a signal from the sensor controller to the display controller via the host processor; a second signal path for transmitting a signal from the sensor controller to the display controller without passing through the host processor; a switch unit configured to be able to switch between the first signal path and the second signal path; a mode control unit that switches between a first output mode in which the first signal path is selected by the switch unit and a second output mode in which the second signal path is selected by the switch unit; Equipped with While the mode control unit is executing the second output mode, the host processor outputs an image signal indicating an entire image corresponding to the display area to the display controller; The display controller sequentially acquires the indicated position from the data signal that has passed through the second signal path, generates a processed image by modifying at least a portion of the entire image that includes the current indicated position, and outputs a display signal indicative of the processed image to the display panel.

1. An input system comprising:

2. The input system according to claim 1, characterized in that the display controller generates a partial image including a trajectory of the indication position from a time series of data signals that have passed through the second signal path, and generates the processed image by synthesizing the entire image and the partial image.

3. The input system of claim 1, characterized in that the display controller generates a partial image in which the trajectory of the indication position indicated by the time series of the data signal that has passed through the second signal path is concealed or unsharpened, and generates the processed image in which at least a portion of the trajectory of the indication position is concealed or unsharpened by synthesizing the entire image and the partial image.

4. 2 . The input system according to claim 1 , wherein the mode control unit switches between the first output mode and the second output mode depending on a state of an application executed by the host processor.

5. The input system described in claim 1, characterized in that the display controller sequentially acquires writing data including writing information regarding the writing state of the stylus and meta information related to the user's writing from the data signal that passes through the second signal path during execution of the second output mode, and outputs a data signal including a time series of the writing data that has been subjected to a predetermined processing to the host processor.

6. 6. The input system according to claim 5, wherein the display controller performs irreversible processing to irreversibly delete at least one of the handwritten information and the meta-information.

7. 6. The input system according to claim 5, wherein the display controller performs an addition process for adding information indicating confidentiality or copyright to the time series of the handwritten data.

8. 6. The input system according to claim 5, wherein the display controller encrypts the time series of the handwritten data.

9. An input system using a touch panel display, the touch panel display having a sensor electrode disposed in a display area of ​​a display panel for detecting a position pointed to by a user, a sensor controller that acquires a pointed position within the display area based on a detection result of the sensor electrodes; a host processor for processing a data signal including the indicated position acquired by the sensor controller; a display controller that outputs a display signal representing an image or video to the display panel; a first signal path for transmitting a signal from the sensor controller to the display controller via the host processor; a second signal path for transmitting a signal from the sensor controller to the display controller without passing through the host processor; a switch unit configured to be able to switch between the first signal path and the second signal path; a mode control unit that switches between a first output mode in which the first signal path is selected by the switch unit and a second output mode in which the second signal path is selected by the switch unit; Equipped with The mode control unit is a pen-down operation of a stylus by a user is a trigger for switching from the first output mode to the second output mode, and a pen-up operation corresponding to the pen-down operation is a trigger for switching from the second output mode to the first output mode, While a contact state of the stylus is detected, switching from the second output mode to the first output mode is inhibited.

1. An input system comprising:

10. An input system using a touch panel display, the touch panel display having a sensor electrode disposed in a display area of ​​a display panel for detecting a position pointed to by a user, a host processor for processing a data signal including the indicated position; a display controller connected to the host processor and configured to output a display signal representing an image or video to the display panel; a sensor controller that acquires a pointed position within the display area based on a detection result of the sensor electrodes; Equipped with the sensor controller is physically or logically connected to the host processor and physically connected to the display controller, and is configured to be able to switch an output destination of the data signal between the host processor and the display controller; While the sensor controller switches the output destination of the data signal to the display controller, the host processor outputs an image signal indicating an entire image corresponding to the display area to the display controller; The display controller sequentially acquires the indicated position from the data signal transmitted from the sensor controller, generates a processed image by modifying at least a portion of the entire image including the current indicated position, and outputs a display signal showing the processed image to the display panel.

1. An input system comprising:

11. An input system using a touch panel display, the touch panel display having a sensor electrode disposed in a display area of ​​a display panel for detecting a position pointed to by a user, a host processor for processing a data signal including the indicated position; a display controller connected to the host processor and configured to output a display signal representing an image or video to the display panel; a sensor controller that acquires a pointed position within the display area based on a detection result of the sensor electrodes; a physical switch having an input terminal connected to the sensor controller, one output terminal connected to the host processor, and the other output terminal connected to the display controller; Equipped with While the output destination of the physical switch is connected to the display controller, the host processor outputs an image signal indicating an entire image corresponding to the display area to the display controller; The display controller sequentially acquires the indicated position from the data signal transmitted from the sensor controller, generates a processed image by modifying at least a portion of the entire image including the current indicated position, and outputs a display signal showing the processed image to the display panel.

1. An input system comprising:

Images