Information processing system, display device, and control method

The information processing system addresses cursor lag in EPDs by updating the background image less frequently than coordinate sampling, prioritizing cursor display to ensure smooth movement and clear visibility, thus enhancing user interaction.

JP2024163621AActive Publication Date: 2024-11-22LENOVO IRELAND INTERNATIONAL LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023079395
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-12
Publication Date
2024-11-22
Estimated Expiration
2043-05-12

AI Technical Summary

Technical Problem

Indirect and relative pointing in electro-phoretic displays (EPDs) results in delayed cursor display, making operations difficult due to cursor lag and multiple cursors appearing at different positions, which can hinder user recognition and operation.

Method used

An information processing system that samples coordinates using an input device, updates the background image at a cycle longer than the coordinate sampling cycle, and outputs an image with a cursor placed at the specified coordinates, prioritizing cursor display over background image updates to ensure smooth movement and visibility.

Benefits of technology

Enhances user identification of the cursor by ensuring smooth movement and reducing cursor overlap, thereby improving operational ease and reducing visual hindrance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024163621000001_ABST
    Figure 2024163621000001_ABST
Patent Text Reader

Abstract

To easily identify a cursor for operation with respect to a user.SOLUTION: A host system acquires a background image and samples coordinates indicated using an input device to notify a controller of the coordinates. The controller updates the background image with an update cycle longer than a sampling cycle of the coordinates and outputs an output image with a cursor placed at the coordinates on the background image to a display medium. Any embodiment, such as an information processing system, display device, and control method, can be achieved.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present application relates to an information processing system, a display device, and a control method, for example, display of a cursor related to an input operation. [Background technology]

[0002] Some information processing devices include an electrophoretic display (EPD). Patent Document 1 describes how the use of an EPD in an information processing device can reduce power consumption. However, with an EPD, it takes a long time to update the screen. Typically, the refresh rate of an EPD is about 8 Hz, whereas the refresh rate of a liquid crystal display (LCD) or an organic light emitting diode (OLED) display is 60 Hz or higher.

[0003] Additionally, in the user interface provided by the OS (Operating System), input operations are performed using indirect relative pointing. Indirect relative pointing is a function that does not directly identify the position indicated by the input device, but displays a cursor at the coordinates on the screen that correspond to that position, and identifies the coordinates of the displayed cursor in response to an input operation. Indirect relative pointing is mainly applied when a pointing device such as a mouse is used as the input device. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2015-64421 A Summary of the Invention [Problem to be solved by the invention]

[0005] Indirect / relative pointing can be considered as a feedback system that includes user actions. That is, the user operates the input device while tracking the movement of the cursor on the screen, and the system changes the position where the cursor is displayed according to the operation. In such a situation, the user (1) finds the cursor that appears on the screen, (2) operates the input device to move the cursor, and (3) places the cursor on the target (operation target) and performs an operation such as clicking. In information processing devices that employ EPD, the display of the cursor at the position specified by the operation is delayed, which often makes operation difficult.

[0006] When the indicated position moves, multiple cursors are displayed at different discrete positions (see Figure 6). As a result, the user may lose sight of the cursor that should be displayed at the position targeted for operation because he or she is unable to recognize it. Even if the user is able to recognize the cursor, operation may be hindered. One of the reasons for this is the relatively large delay in pixel luminance change. For example, black and white inversion takes more than 100 ms. When restoring the luminance of a pixel for which inversion of luminance has been instructed, it is necessary to wait until the previous luminance change is complete. Also, in the drawing process of the cursor that moves sequentially, after the drawing process of the cursor for a certain position is completed, the process of starting the drawing process for the latest position is repeated.

[0007] Therefore, instead of performing drawing processing on the entire screen including the cursor, it is possible to update the image by limiting it to a rectangular area around the cursor. In this method, a new rectangular area is determined at the stage where the movement of the specified position causes no overlap with the rectangular area including the cursor that was displayed immediately before, and the cursor is displayed within the determined rectangular area (see Figure 7). Since overlapping of the rectangular areas to be updated is prevented, the individual cursors are separated. It is also possible to reduce the size of the rectangular area to increase the drawing frequency and reduce the distance between multiple cursors that appear intermittently. However, it is still sometimes difficult to express smooth cursor movement. [Means for solving the problem]

[0008] The present application has been made to solve the above-mentioned problems, and an information processing system according to one aspect of the present application is an information processing system comprising a host system that samples coordinates indicated using an input device and acquires a background image, and a controller that displays an output image based on the background image on a display medium, wherein the host system notifies the controller of the coordinates, and the controller updates the background image at an update period longer than the sampling period of the coordinates, and outputs to the display medium the output image in which a cursor is placed at the coordinates relative to the background image.

[0009] In the above information processing system, the host system may output a drawing command indicating the coordinates and a cursor to the controller, and the controller may place the cursor at the coordinates indicated by the drawing command.

[0010] In the above information processing system, the controller may place the cursor at the latest coordinates when the display area of ​​the cursor based on the latest coordinates does not include a display area of ​​the cursor whose elapsed time from the time of placement is within the update period.

[0011] In the above information processing system, the controller may update the background image to be the output image to a latest background image when there is no cursor whose elapsed time from the time of placement is within the update period.

[0012] In the above information processing system, the controller may erase a cursor placed in a display area that does not include a display area of ​​the cursor based on the latest coordinates when the elapsed time from the time of placement becomes equal to or exceeds the update period.

[0013] In the above-mentioned information processing system, the controller may extract a partial image from the last updated background image having a size sufficient to encompass the cursor at the coordinates, superimpose the cursor on the partial image to construct a cursor icon, and place the cursor icon at the coordinates.

[0014] In the above information processing system, the host system may predict predicted coordinates a predetermined time after the present based on the coordinates of at least two samples, and the controller may place the cursor at the predicted coordinates with respect to the background image.

[0015] A display device according to a second aspect of the present application is a display device including a controller that displays an output image based on a background image on a display medium, wherein the controller updates the background image at an update period longer than a sampling period of a coordinate, and outputs an output image in which a cursor is placed at the coordinate to the display medium.

[0016] A control method according to a third aspect of the present application is a control method in an information processing system including a host system and a controller, wherein the host system samples coordinates indicated using an input device, notifies the controller of the coordinates, and acquires a background image, and the controller updates the background image at an update period longer than the sampling period of the coordinates, and outputs an output image in which a cursor is placed at the coordinates to a display medium. Effect of the Invention

[0017] According to the embodiment of the present application, it is possible for the user to easily identify the cursor related to the operation. [Brief description of the drawings]

[0018] [Figure 1] FIG. 2 is a schematic block diagram illustrating an example of a functional configuration of the information processing system according to the present embodiment. [Diagram 2] FIG. 2 is a schematic block diagram illustrating an example of a functional configuration of the information processing system according to the present embodiment. [Diagram 3] 10 is a flowchart illustrating an example of a cursor display process according to the embodiment. [Figure 4] 11 is an explanatory diagram for explaining an example of execution of a cursor display process according to the embodiment; FIG. [Diagram 5]FIG. 11 is an explanatory diagram for explaining a comparative example of cursor display processing. [Figure 6] 11A and 11B are explanatory diagrams showing examples of an output image and a display image including a cursor. [Figure 7] FIG. 11 is an explanatory diagram for explaining another comparative example of the cursor display process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, an embodiment of the present application will be described with reference to the drawings. First, a configuration example of an information processing system S1 according to an embodiment of the present application will be described. Fig. 1 is a schematic block diagram showing an example of the functional configuration of the information processing system S1 according to the present embodiment. The information processing system S1 includes a host system 10 and a display device 30. The host system 10 and the display device 30 are connected to each other so as to be able to transmit and receive various data between them. The mouse 40 is an example of an input device, and is connected to the host system 10 by wire or wirelessly.

[0020] An input signal is input to the host system 10 from the mouse 40. The host system 10 samples the coordinates indicated by the input signal from the mouse 40 at predetermined sampling intervals. The sampled coordinates indicate a relative position based on a predetermined reference point (e.g., the origin, the center of gravity, etc.) within the display screen of the display device 30. The host system 10 executes various programs to acquire a background image. The background image is mainly a moving image. The background image is updated at regular frame intervals. Each time the background image is updated, the host system 10 outputs display data indicating the updated background image to the display device 30. The host system 10 outputs coordinate information indicating the sampled coordinates to the display device 30.

[0021] The display device 30 includes a timing controller 36 and an EPD 38 . The timing controller 36 causes the EPD 38 to display an output image based on the background image shown in the display data input from the host system 10 . The timing controller 36 is an example of a controller that causes the EPD 38 to display an output image including the latest background image at each predetermined update period. The update period refers to the period for updating the output image or the background image used as an element of the output image, and corresponds to the reciprocal of the refresh rate of the EPD 38. The update period is longer than the sampling period of the coordinates. The timing controller 36 composes an output image by placing (overwriting) a cursor at coordinates specified by the coordinate information for the background image to be displayed on the EPD 38, and causes the EPD 38 to display the composed output image.

[0022] The information processing system S1 may be configured as a single information processing device, or the information processing device having the host system 10 and the display device 30 may be separate entities. These information processing devices may be realized in any form, such as a personal computer, a tablet terminal, or a mobile phone. In this application, the execution of processes instructed by instructions written in various programs by a processor or other hardware may be referred to as "executing a program" or "running a program."

[0023] In this application, "image" is not necessarily limited to spatial distribution of shading or color, but refers to visually recognizable information, i.e., characters, symbols, figures, patterns, or any combination thereof. "Background image" refers to an image on which the cursor is placed, and is not necessarily limited to an image that represents only the "background." Background images include so-called desktop images, various execution images that are displayed by executing application programs (sometimes referred to as "apps" in this application), etc.

[0024] The input device is not necessarily limited to the mouse 40. The input device may be any pointing device capable of detecting position information for determining the position at which a cursor is to be displayed on the display screen of the display device 30. The input device may be, for example, a joystick, a trackball, or the like. In the following description, the input device is the mouse 40, and the information processing system S1 is configured as a single information processing device.

[0025] Next, a hardware configuration example of the information processing system S1 will be described below. Fig. 2 is a schematic block diagram showing a hardware configuration example of the information processing system S1 according to this embodiment. The host system 10 includes a processor 12 , a main memory 14 , a chipset 20 , and a flash memory 22 .

[0026] The processor 12 controls the overall function of the information processing device including the host system 10. The processor 12 is, for example, one or more CPUs (Central Processing Units). The processor 12 executes a predetermined program and cooperates with the main memory 14 and other hardware to fulfill the functions of the host system 10.

[0027] The main memory 14 is a writable memory used as a working area for the processor 12, i.e., an area for reading programs to be executed and various setting data, and an area for writing processing data acquired by executing the programs. The main memory 14 is configured to include, for example, a plurality of DRAM (Dynamic Random Access Memory) chips. The executable programs include an OS, various device drivers for controlling peripheral devices, various services / utilities, applications, and the like.

[0028] The processor 12 and the main memory 14 function as a system device constituting a computer system that is the core of the information processing device, that is, a host system 10. The host system 10 includes a system device as hardware, and software such as an OS and a schedule task.

[0029] The chipset 20 includes one or more controllers and an input / output interface (I / F) that allows connection to multiple devices so as to input and output various data. The chipset 20 is connected to the processor 12, the flash memory 22, the display device 30, and the mouse 40 so as to input and output various data. The connection to the mouse 40 may use, for example, the IEEE802.15.1 standard.

[0030] The chipset 20 includes a general-purpose input / output I / F 20a and a graphic I / F 20b. The general-purpose input / output I / F 20a is mainly used for outputting coordinate information indicating sampled coordinates to the display device 30. The general-purpose input / output I / F 20a outputs the coordinate information in accordance with, for example, the USB (Universal Serial Bus) standard. The graphic I / F 20b is mainly used for outputting display data showing a background image to the display device 30. The graphic I / F 20b outputs the display data in accordance with, for example, the USB standard, the MIPI (Mobile Industry Processor Interface) standard, or the like.

[0031] Various programs and data are stored in the flash memory 22. The various programs include, for example, firmware, device drivers, services / utilities, and applications. These programs are executed by the processor 12. The stored data includes data to be processed by the processor 12 and data generated by the processing. The generated data may include not only final data, but also intermediate data that is generated up to a certain processing step and is used in a subsequent processing step.

[0032] The display device 30 includes a timing controller (T-CON) 36 and an EPD 38 . The timing controller 36 executes predetermined firmware to control the functions of the display device 30. Display data and coordinate information are input to the timing controller 36 from the host system 10. The timing controller 36 composes an output image by placing a cursor at coordinates specified by the coordinate information on a background image shown in the display data. The timing controller 36 causes the EPD 38 to display the composed output image.

[0033] The timing controller 36 specifies a signal value for each pixel constituting an output image at a predetermined refresh rate, and generates an output signal for instructing the timing of displaying the image at a density corresponding to the specified signal value. A refresh process is performed on the EPD 38 at each refresh period. The timing controller 36 outputs the generated output signal to a drive circuit for the EPD 38. The drive circuit includes, for example, a TTL (Transistor-Transistor Logic) circuit.

[0034] The EPD 38 has a shape in which its thickness is much thinner than its height and width, and a screen area is provided on its surface. A plurality of pixels are arranged two-dimensionally at regular intervals in the screen area. Each pixel includes an electrophoretic element and a pixel electrode. The EPD 38 includes a drive circuit. The drive circuit applies a drive voltage corresponding to a signal value specified for each pixel by the timing controller 36 to the pixel electrode of the pixel. The electrophoretic element is displayed with a brightness according to the drive voltage. A display image is expressed by the distribution of brightness for each pixel in the screen area.

[0035] The background image may be a monochrome binary image or a grayscale image. In a binary image, each pixel has two signal values. The two values ​​correspond to light and dark (black and white). In a grayscale image, each image has three or more signal values ​​(grayscale values) that indicate the degree of brightness. The background image may be a color image. A color image is represented by color signal values, for example, RGB signal values, that indicate the brightness of the color that each pixel can emit.

[0036] In addition, when display data representing a grayscale image is input to the timing controller 36, dithering may be performed on the grayscale value of each pixel to convert it into a binary image, and the resulting binary image may be used as the output image. Dithering is an example of binarization. Dithering includes the following steps: (i) setting the grayscale value as a judgment threshold, (ii) generating random numbers (uniform random numbers) that have a value range similar to the grayscale value and have a uniform occurrence probability within that value range, and (iii) setting the value of the binary information to 1 if the generated random number is equal to or greater than the judgment threshold, and setting the value of the binary information to 0 if the generated random number is less than the judgment threshold. The value of the binary information being 1 indicates that the pixel is to be displayed in white as a bright area, and the value of the binary information being 0 indicates that the pixel is to be displayed in black as a dark area.

[0037] When display data representing a color image is input to the timing controller 36, a known grayscale conversion may be performed on the color signal values ​​of each pixel to convert them into grayscale values. The grayscale conversion includes a process of calculating a weighted average of the color signal values ​​of each color as a grayscale value. The grayscale conversion converts the color image into a monochrome grayscale image having an intensity distribution for each pixel. The converted grayscale image may be binarized and converted into a binary image.

[0038] Next, an example of the functional configuration of the information processing system S1 according to this embodiment will be described below. Fig. 2 is a schematic block diagram showing an example of the functional configuration of the information processing system S1 according to this embodiment. The following describes an example of the functional configuration of the host system 10. The host system 10 includes an input processing unit 112, an OS processing unit 114, and an output processing unit .

[0039] The input processing unit 112 receives an input signal from the mouse 40 and samples the relative position indicated by the input signal at a predetermined sampling period. The sampling period is, for example, 8 to 30 ms (corresponding to a sampling frequency of 160 Hz to 30 Hz). The sampling period of the relative position and therefore the coordinates is usually equal to the frame period of the background image and shorter than the update period. The relative position corresponds to a displacement from a reference point in the screen space. As the reference point, a point (for example, the origin) in the screen area of ​​the EPD 38 set as the standard output from the host system 10 is used. The input processing unit 112 outputs coordinate information indicating the coordinates of each sampled sample to the OS processing unit 114. The function of the input processing unit 112 is realized by the CPU 12 executing a dedicated device driver on the OS.

[0040] The OS processing unit 114 executes a kernel that is the core of the OS by the processor 12, and provides basic functions of the OS. The basic functions of the OS processing unit 114 include control of the execution state of applications, firmware, device drivers, and other programs by the host system 10, allocation of computing resources of the host system 10, and the like. The host system 10 executes various programs under the control of the OS processing unit 114 (sometimes referred to as "on the OS" in this application). A background image is acquired by executing the programs. Acquiring a background image can refer to any function, such as generating a background image, receiving a background image from another device, or reading display data showing a background image previously stored in a storage medium of the host system. The function in question can depend on the program being executed.

[0041] The OS processing unit 114 outputs display data indicating the acquired background image to the display device 30 via the output processing unit 116. The frame period of the background image may be equal to the sampling period of the coordinates (for example, 8 to 30 ms (corresponding to a frame rate of 160 Hz to 30 Hz)). The OS processing unit 114 outputs the coordinate information input from the input processing unit 112 to the display device 30 via the output processing unit 116. The OS processing unit 114 may use the input coordinate information in processing during execution of a program. The background image may be changed by the OS processing unit 114 based on the coordinate information. When outputting display data in accordance with a program being executed, the OS processor 114 may call one or both of the functions of the input processor 112 and the output processor 116. The functions of the input processing unit 112 and the output processing unit 116 may each be described as an API (Application Programming Function).

[0042] The output processing unit 116 includes a cursor processing unit 116a and a display control unit 116b. The cursor processing unit 116a outputs the coordinate information input from the OS processing unit 114 to the display device 30. The cursor processing unit 116a may be preset with a cursor image and generate a drawing command (drawing command) indicating the acquired coordinate information and the cursor. The drawing command is a command for instructing the display device 30 to display a cursor at the coordinates indicated by the coordinate information. The cursor processing unit 116a outputs the generated drawing command to the display device 30. The function of the cursor processing unit 116a may be realized by the processor 12 executing a cursor drawing application on the OS and cooperating with the general-purpose input / output I / F 20a. The cursor drawing application may be configured as the above-mentioned API function and may be called from another program.

[0043] The display control unit 116b outputs display data input from the OS processing unit 114 to the display device 30. The function of the display control unit 116b may be realized by the processor 12 executing a device driver for the display device 30 on the OS in cooperation with the graphic I / F 20b. The device driver may be configured as the above-mentioned API function.

[0044] Next, a description will be given of an example of the functional configuration of the timing controller 36. The timing controller 36 includes a background image buffer 362, an update processing unit 364, and a frame buffer 366. The timing controller 36 executes predetermined firmware to realize its functions.

[0045] Background image buffer 362 stores display data input from host system 10. Every time display data is input to background image buffer 362, the stored display data is updated (overwritten, updated) with the input display data. Background image buffer 362 stores display data showing the latest background image at that time.

[0046] Update processing unit 364 performs processing related to updating the background image and cursor to be displayed on EPD 38. When coordinate information is not input from host system 10, update processing unit 364 writes the background image stored in background image buffer 362 to frame buffer 366 for each update cycle. Frame buffer 366 stores an output image to be displayed on EPD 38. Therefore, when the cursor is not displayed, frame buffer 366 can cause EPD 38 to display a new background image for each update cycle. Note that background image buffer 362 may hold the background image used for updating by writing it to frame buffer 366 at least until the next update.

[0047] When updated coordinate information is input from the host system 10, the update processing unit 364 places a cursor at the coordinates specified by the coordinate information in the output image stored in the frame buffer 366. Thus, the cursor is superimposed (overlaid) on the display area having those coordinates as a reference point. The reference point may be any position that represents the position of the cursor, such as the origin or center of gravity. When the coordinate information is input together with a drawing command, the update processing unit 364 may place the cursor transmitted by the drawing command instead of the cursor previously set in the update processing unit 364.

[0048] Since the coordinate information is updated every sampling period, the update processing unit 364 also executes processing for positioning the cursor based on the updated coordinate information every sampling period. That is, the position of the cursor displayed in the screen area of ​​the EPD 38 is updated every sampling period. The update period of the EPD 38 is longer than the response time of the EPD 38 and can be more than twice the sampling period. Therefore, multiple cursors may be displayed in the display image from the EPD 38. When the moving speed of the coordinates is low, simply displaying the cursor every update period may result in an overlapping area where the display area of ​​the cursor at a certain time point partially overlaps with the display area of ​​the cursor in the next sampling period, and the cursors here may not be distinguished, resulting in impaired visibility.

[0049] Therefore, when the time elapsed since the time point when the display area based on the latest updated coordinates was arranged (sometimes referred to as the "arrangement time point" in this application) does not include any part or all of the display area of ​​any cursor within the update period, the update processing unit 364 arranges the cursor at the latest updated coordinates. Since cursors at different positions are displayed without overlapping each other, visibility is not impaired.

[0050] 2, the update of the coordinate information is performed in parallel with the update of the background image stored in background image buffer 362. Update processor 364 prioritizes updating the cursor position over updating the background image. That is, when a cursor whose elapsed time since its placement is within the update period is present in the screen area, update processor 364 maintains the background image to be the output image at that time without updating it to the latest image. At this time, update processor 364 does not write the background image stored in background image buffer 362 to frame buffer 366.

[0051] When a cursor whose elapsed time from the placement time is within the update period does not exist in the screen area, the update processing unit 364 updates the background image to be output at that time to the latest image. In such cases, the coordinates indicated by the input coordinate information are maintained unchanged for a period of time equal to or longer than the update period, or the coordinate information indicating valid coordinates is not continuously input for a period of time equal to or longer than the update period. In this case, the update processing unit 364 does not write the background image stored in the background image buffer 362 to the frame buffer 366. By prioritizing the placement of the cursor based on the updated coordinates over the update of the background image, the occurrence of a cursor that causes an afterimage can be avoided, thereby suppressing a subjective deterioration in image quality.

[0052] The update processing unit 364 may generate a cursor icon. When generating the cursor icon, the update processing unit 364 extracts an image (sometimes referred to as a "partial image" in this application) within a partial area having the coordinates indicated by the coordinate information as a reference point from the background image used in the last update stored in the background image buffer 362. Then, the update processing unit 364 superimposes a cursor on the extracted partial image. The partial area has a predetermined shape (for example, a rectangle or a square) and is preferably as small as possible to include the entire cursor. The partial area may have a size that is inscribed in the outer edge of the cursor or a periphery with a certain width, for example. The update processing unit 364 can treat the generated cursor icon as an example of a cursor and subject it to processing such as placement, erasure, and position update in the information processing system S1.

[0053] The update processing unit 364 may erase a cursor that has been placed in a display area that does not overlap a part or all of the display area of ​​the cursor based on the latest coordinates and whose elapsed time from the time of placement is equal to or greater than the update period. For example, the update processing unit 364 identifies a cursor whose elapsed time from the time of placement is equal to or greater than the update period. The update processing unit 364 extracts a partial image having the same display area as the identified cursor from the background image used in the last update stored in the background image buffer 362. Then, the update processing unit 364 places the extracted partial image in the display area of ​​the identified cursor. As a result, the displayed cursor is replaced with the partial image that is a part of the latest background image.

[0054] The frame buffer 366 saves an output image to be output to the EPD 38. The frame buffer 366 has a storage medium that stores a signal value for each pixel that represents the output image, and thus has a storage area corresponding to the screen area of ​​the EPD 38. The timing controller 36 has an output signal generation circuit (not shown) that generates an output signal indicating an output image for each update period using the signal value for each pixel stored in the frame buffer 366. The output signal generation circuit outputs the generated output signal to a drive circuit for the EPD 38. In this embodiment, a background image, a background image with a cursor superimposed thereon, and a background image with a partial image further superimposed thereon can be output images.

[0055] Next, an example of cursor display processing according to this embodiment will be described. Fig. 3 is a flowchart showing an example of cursor display processing according to this embodiment. Fig. 3 shows an example in which the host system 10 outputs display data showing a background image for each frame period and a drawing command showing coordinates and a cursor for each sampling period to the display device 30. Then, the background image buffer 362 of the timing controller 36 stores the input display data every time the display data is input from the host system 10, and overwrites the original display data.

[0056] (Step S104) The update processing unit 364 judges whether the coordinates specified by the latest drawing command input from the host system 10 have changed from the coordinates specified by the immediately previous drawing command. If it is judged that they have changed (step S104 YES), the process proceeds to step S106. If it is judged that they have not changed (step S104 NO), the process proceeds to step S112.

[0057] (Step S106) Update processing unit 364 stops the process of updating the background image written in frame buffer 366 to the latest background image stored in background image buffer 362. (Step S108) Update processing unit 364 extracts a partial image within a partial area having the coordinates specified by the latest drawing command as a reference point from the last updated background image stored in background image buffer 362. Update processing unit 364 overlays the cursor transmitted by the drawing command on the extracted partial image to generate a cursor icon. Update processing unit 364 overwrites (places) the generated cursor icon in a storage area of ​​frame buffer 366 that corresponds to the partial area having the coordinates specified by the drawing command as a reference point.

[0058] (Step S110) The update processing unit 364 determines, among the cursor icons previously overwritten, that the elapsed time from the time of arrangement is equal to or longer than the update period and that is overwritten in a display area that does not include a display area of ​​a cursor icon based on the latest coordinates, as an updatable cursor icon. The update processing unit 364 erases the updatable cursor icon. When erasing the updatable cursor icon, the update processing unit 364 specifies the display area of ​​the cursor icon, and extracts a partial image within the specified display area from the latest background image stored in the background image buffer 362. The update processing unit 364 overwrites the extracted partial image in a storage area of ​​the frame buffer 366 corresponding to the specified display area. Then, the process returns to step S104.

[0059] (Step S112) The update processing unit 364 determines whether or not the background image can be updated. The update processing unit 364 can determine whether or not the background image can be updated based on whether or not a cursor whose elapsed time from the time of placement falls within the update period is present in the screen area. If it is determined that the background image can be updated (YES in step S112), the process proceeds to step S114. If it is not determined that the background image can be updated (NO in step S112), the process returns to step S104.

[0060] Fig. 4 is an explanatory diagram for explaining an example of the execution of the cursor display process according to this embodiment. Fig. 4 illustrates a case where the process of Fig. 3 is executed with an update period of 125 ms (corresponding to a refresh rate RR = 8 Hz) and a frame period of 16 ms (corresponding to a frame rate FR = 60 Hz). In this example, the frame period and the coordinate sampling period are equal. 4 shows in the first line the background image DT for each frame shown in the display data from the host system 10 and the coordinates OP shown in the drawing command. In this example, the background image DT changes very little over time, and the coordinates OP move from the top to the bottom of the screen area. The background image buffer 362 stores the latest background image DT at that point in time for each frame period.

[0061] The second row indicates the background image that is used to update the output image from among the background images stored in background image buffer 362. The background image is not updated in the first update cycle and the second update cycle. This is because the processing in steps S104 and S106 detects movement of the coordinates specified by the drawing command and stops updating the background image. The background image is updated in the third update cycle. This is because the processing in steps S112 and S114 determines that the cursor does not exist in the screen area and that the elapsed time from the time of placement is within the update cycle, and therefore the entire area of ​​the background image can be updated.

[0062] The third row indicates the elements of the output image to be written to the frame buffer 366. Here, the elements of the output image are a cursor icon, a partial image, and a background image. The cursor icon indicates different coordinates for each sampling period. The cursor icon is generated by the processing of step S108. The update processing unit 364 extracts a partial image within a partial area having the coordinates specified by the drawing command as the reference point from the background image used in the last update (step S108a). The update processing unit 364 superimposes the cursor transmitted by the drawing command on the extracted partial image (step S108b).

[0063] According to the update processing unit 364, a cursor icon that has been placed for an update period or more is determined to be updateable. The cursor icon determined to be updateable is erased by the process of step S110. That is, a partial image within the display area of ​​the cursor icon determined to be updateable, of the background image last updated from the background image buffer, is extracted. The extracted partial image is overwritten onto the display area of ​​the cursor icon. Then, when the series of input operations is ended and a cursor whose elapsed time from the placement time is within the update period no longer exists in the screen area, the latest background image related to the update is written into the frame buffer 366. Also, the cursor icon related to the coordinates indicated at that time is written.

[0064] The fourth row shows the display image displayed by the EPD 38. The display image is displayed based on the output image from the frame buffer 366. In the output image, a cursor whose position changes for each sampling period is displayed sequentially over time. However, the display of the cursor icon is maintained until the elapsed time from the start of display reaches the update period. Therefore, the cursors arranged at different coordinates for each sampling period in the display image accumulate over time. A cursor whose elapsed time from the start of display is equal to or greater than the update period is erased by overwriting the partial image. Therefore, the accumulation of the cursors is suppressed. Then, when the input operation ends and the period during which the movement of the specified coordinates stops continues for equal to or greater than the update period, the background image is updated and the display of the cursor at that position is maintained.

[0065] Next, a comparative example of cursor display processing will be described. FIG. 5 is an explanatory diagram for explaining a comparative example of cursor display processing. In the comparative example, the frame period, sampling period, and update period are the same as those in the example of FIG. 4. However, in the comparative example, the host system 10 outputs to the display device 30 an output image in which a cursor is superimposed on a background image at coordinates designated by an input operation for each frame period. The background image buffer 362 of the display device 30 stores the output image input from the display device 30 for each update period. The output image newly stored in the background image buffer 362 is output directly to the EPD 38 via the frame buffer 366.

[0066] The EPD 38 displays a display image based on the output image transmitted from the frame buffer 366. Since the display image is maintained within each update cycle, the position of the cursor appearing in the display image is fixed. Therefore, the position where the cursor appears changes discretely for each update cycle. In contrast, in this embodiment, the position of the cursor changes for each sampling cycle, so smoother cursor movement is expressed.

[0067] In the above description, the sampling period of the coordinates and the frame period of the background image are mainly the same, but this is not limited to the above. In this embodiment, the sampling period of the coordinates and the frame period of the background image may be different. In addition, the update processing unit 364 of the display device 30 may thin out a part of the latest coordinate information inputted thereto, and use it for cursor display at regular intervals. The sampling period is adjusted to be substantially longer by thinning out.

[0068] However, in this embodiment, the adjusted sampling period is set to be at least shorter than the update period. Generally, the shorter the sampling period, the smoother the cursor movement is displayed, but the number of cursors displayed for each update period increases. Therefore, as long as the cursor movement is displayed smoothly, the longest possible sampling period (for example, 40 to 80 ms (equivalent to 12 to 24 Hz)) should be set.

[0069] The smoothness of cursor movement required may vary depending on the function of the program executed by the host system 10. For example, smooth cursor movement is highly necessary for programs used in situations where the movement of coordinates indicated by an input device continues, such as drawing applications and game applications. For such programs, a short coordinate sampling period is preferable. On the other hand, smooth cursor movement is less necessary for programs in which the movement of coordinates indicated by an input device is rare, such as text-based document creation applications. For such programs, a long coordinate sampling period may be acceptable.

[0070] Therefore, a sampling period may be set in advance for each program in the cursor processing unit 116a of the host system 10, and the sampling period corresponding to the program executed by the OS processing unit 114 may be notified to the display device 30. The update processing unit 364 of the display device 30 employs the coordinate information input from the host system 10 for each notified sampling period.

[0071] In the above description, the update processing unit 364 displays the cursor at a position indicated by the coordinates notified from the host system 10, but the present invention is not limited to this. The update processing unit 364 may predict predicted coordinates after a predetermined elapsed time using a predetermined prediction model based on coordinates indicated in the coordinate information of two or more samples (e.g., the latest sample and the sample immediately before) up to that time. For example, a linear model can be used as the prediction model. The update processing unit 364 displays the cursor at a position indicated by the predicted coordinates instead of the coordinates indicated in the coordinate information. The delay until the cursor is displayed is compensated for by setting, for example, the delay time required from the detection of the coordinates to the display of the cursor as the elapsed time.

[0072] In the above description, the background image is mainly a moving image, but is not limited to this. The background image may be a still image. Even if the background image is a still image, the update processing unit 364 only needs to perform processing on the background image stored in the background image buffer 362. The cursor is not necessarily limited to an arrow. The cursor may have any shape, pattern, color, or combination thereof, and may be an image that can be clearly distinguished from other images. For example, the cursor may be an X mark, a ■ mark, or the like.

[0073] As described above, the information processing system S1 according to this embodiment includes a host system 10 that samples coordinates designated using an input device and acquires a background image, and a controller (e.g., timing controller 36) that displays an output image based on the background image on a display medium. The host system notifies the controller of the sampled coordinates. The controller updates the background image at an update period longer than the sampling period of the coordinates, and outputs an output image in which a cursor is positioned at the sampled coordinates for the background image to the display medium. The input device may be a mouse 40, and the display medium may be an EPD 38. According to this configuration, the controller updates the background image acquired from the host system 10 at an update period longer than the sampling period of the coordinates, and causes the display medium to display an output image in which a cursor is placed at the coordinates sampled for the background image. The display medium can display the cursor at each sampling period shorter than the update period of the background image. Since the movement of the cursor is expressed smoothly, the user can more easily identify the cursor indicating the indicated coordinates.

[0074] The host system 10 may output a drawing command indicating the sampled coordinates and a cursor to the controller, and the controller may place the cursor at the coordinates indicated by the drawing command. According to this configuration, a cursor specified by a drawing command input from host system 10 is displayed at the coordinates specified by the drawing command. Therefore, even if the cursor is not set in advance in the controller, a cursor having a desired form specified by host system 10 is displayed.

[0075] When the display area of ​​the cursor based on the latest coordinates does not include a display area of ​​the cursor where the elapsed time from the time of placement is within the update period, the controller may place the cursor at the latest coordinates. According to this configuration, the cursor is displayed at a new coordinate that is separate from the display area of ​​the already displayed cursor and does not overlap with the display area of ​​the previously displayed cursor. Since the individual cursors are displayed successively so that the display areas do not overlap, the visibility of the cursors is not impaired.

[0076] When there is no cursor whose elapsed time from the time of placement is within the update cycle of the display medium, the controller may update the background image that is an element of the output image to the latest background image. Therefore, updating of the background image is prioritized over displaying the cursor at the coordinates updated by sampling. Since the user's input operation performed by observing the movement of the cursor is not hindered, deterioration of the operability of the input device is suppressed.

[0077] The controller may erase a cursor that has been placed in a display area that does not include a display area of ​​the cursor based on the latest coordinates when the time that has elapsed since the placement becomes equal to or exceeds the update period. Therefore, the occurrence of afterimages due to erasure is suppressed while the cursor that is not currently being pointed to is erased, and therefore the accumulation of unnecessary cursors on the display image can be suppressed.

[0078] The controller may extract a partial image having a size sufficient to encompass the cursor at the most recently sampled coordinates from the last updated background image, superimpose the cursor on the partial image to construct a cursor icon, and place the cursor icon at those coordinates. Therefore, the cursor icon superimposed on the partial image extracted from the latest background image is placed on the background image. Thus, the area to be processed is the area of ​​the cursor icon, while the area in which the change is visible can be limited to the area of ​​the cursor. By introducing a partial image having a simpler shape than the cursor, the process related to writing the cursor can be simplified.

[0079] The host system 10 may predict predicted coordinates a predetermined time from now based on the coordinates of at least two samples, and the controller may position the cursor at the predicted predicted coordinates relative to a background image. With this configuration, the cursor is positioned at the predicted coordinates after the elapsed time, thereby compensating for the delay time from detection of the coordinates to display of the cursor.

[0080] Although the embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to the above embodiment, and the present invention also includes designs that do not deviate from the gist of the present invention. The configurations described in the above embodiment can be combined in any combination. [Explanation of symbols]

[0081] S1...information processing system, 10...host system, 12...processor, 14...main memory, 20...chipset, 30...display device, 36...timing controller, 38...EPD, 40...mouse, 112...input processing unit, 114...OS processing unit, 116...output processing unit, 116a...cursor processing unit, 116b...display control unit, 362...background image buffer, 364...update processing unit, 366...frame buffer

Claims

1. Sampling coordinates indicated using an input device; A host system for obtaining a background image; a controller for displaying an output image based on the background image on a display medium; An information processing system comprising: The host system includes: notifying the controller of the coordinates; The controller: updating the background image at an update period longer than a sampling period of the coordinates; The output image in which a cursor is placed at the coordinates on the background image is output to the display medium. Information processing system.

2. The host system includes: outputting a drawing command indicating the coordinates and a cursor to the controller; The controller: The cursor is placed at the coordinates specified by the drawing command. The information processing system according to claim 1 .

3. The controller: When the display area of ​​the cursor based on the latest coordinates does not include a display area of ​​the cursor whose elapsed time from the placement time is within the update period, the cursor is placed at the latest coordinates. The information processing system according to claim 1 .

4. The controller: When there is no cursor whose elapsed time from the placement time is within the update period, the background image to be the output image is updated to the latest background image. The information processing system according to claim 1 .

5. The controller: The time elapsed since placement is equal to or exceeds the update period, and the cursor placed in the display area that does not include the display area of ​​the cursor based on the latest coordinates is erased.

5. The information processing system according to claim 4.

6. The controller: a partial image having a size that includes the cursor at the coordinates is extracted from the last updated background image, and the cursor is superimposed on the partial image to form a cursor icon; Place the cursor icon at the coordinates. The information processing system according to claim 1 .

7. The host system includes: predicting predicted coordinates a predetermined time after the present based on the coordinates of at least two samples; The controller: The cursor is positioned at the predicted coordinates relative to the background image. The information processing system according to claim 1 .

8. the input device is a mouse; The display medium is an electrophoretic display. The information processing system according to claim 1 .

9. A display device including a controller that displays an output image based on a background image on a display medium, The controller: updating the background image at an update period longer than a sampling period of the coordinates; An output image in which a cursor is positioned at the coordinates is output to the display medium. Display device.

10. A control method in an information processing system including a host system and a controller, comprising: The host system includes: Sampling coordinates indicated using an input device; notifying the controller of the coordinates; Get the background image, The controller: updating the background image at an update period longer than a sampling period of the coordinates; An output image in which a cursor is positioned at the coordinates is output to a display medium. Control methods.

Citation Information

Patent Citations

  • Controller, display device, control method and program

    JP2015064421A