Information processing device, control method thereof, and program

Abstract

This information processing device is connected to a display device (14) capable of updating frame images at a variable refresh rate, and the device executes a process for rendering a frame image, transmits the rendered frame image to the display device (14), measures the rendering time required for the frame image rendering process, and changes the operating condition for the rendering process in accordance with the measured rendering time.

Description

[Technical Field]

[0001] The present invention relates to an information processing device that generates an image to be displayed on a screen of a display device, a control method thereof, and a program. [Background technology]

[0002] Generally, display devices such as home television sets display video by continuously updating still images (frame images) displayed on the screen at a predetermined cycle (refresh rate). Therefore, an information processing device that generates video to be displayed on the display device must render each frame image that constitutes the video at a fixed frame rate that matches this refresh rate. However, a fixed frame rate can lead to problems such as not being able to fully utilize the rendering capabilities of the information processing device, or, conversely, display disturbances such as stuttering and tearing can occur if the rendering capabilities of the information processing device are insufficient.

[0003] Therefore, video standards such as HDMI (registered trademark) have proposed a technology called variable refresh rate (VRR), which allows the refresh rate of a display device to be variable. This technology allows the display device to update the displayed image at any refresh rate, so the information processing device can dynamically change the frame rate according to its own drawing speed, etc., allowing it to draw frame images at a more desirable frame rate. Summary of the Invention [Problem to be solved by the invention]

[0004] The above-mentioned technology aims to efficiently draw frame images by actively changing the frame rate, but what kind of control is necessary to efficiently draw frame images when using variable refresh rate technology has not yet been fully explored.

[0005] The present invention has been made in consideration of the above-mentioned circumstances, and one of its objects is to provide an information processing device, a control method thereof, and a program that can efficiently perform drawing processing when displaying images at a variable refresh rate. [Means for solving the problem]

[0006] An information processing device according to one aspect of the present invention is an information processing device connected to a display device capable of updating frame images at a variable refresh rate, and is characterized by including a frame image drawing unit that executes drawing processing of the frame image and transmits the drawn frame image to the display device, a drawing time measurement unit that measures the drawing time required for drawing processing of the frame image, and a drawing processing control unit that changes the operating conditions of the drawing processing according to the measured drawing time.

[0007] An information processing method according to one aspect of the present invention is a control method for an information processing device connected to a display device capable of updating frame images at a variable refresh rate, and is characterized by including the steps of executing a drawing process for a frame image and transmitting the drawn frame image to the display device, measuring the drawing time required for the drawing process for the frame image, and changing the operating conditions of the drawing process according to the measured drawing time.

[0008] A program according to one aspect of the present invention is a program for causing a computer connected to a display device capable of updating frame images at a variable refresh rate to execute the steps of: drawing a frame image and transmitting the drawn frame image to the display device; measuring the drawing time required for the drawing process of the frame image; and changing operating conditions for the drawing process in accordance with the measured drawing time. This program may be provided by being stored on a computer-readable, non-transitory information storage medium. [Brief explanation of the drawings]

[0009] [Figure 1]1 is a block diagram showing a schematic configuration of an information processing device according to an embodiment of the present invention; [Figure 2] 1 is a functional block diagram showing functions realized by an information processing device according to an embodiment of the present invention; [Figure 3] FIG. 2 is a diagram illustrating an example of control executed by the information processing device according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0011] An information processing device 10 according to one embodiment of the present invention is a home game console, a personal computer, or the like, and, as shown in Fig. 1, includes a control unit 11, a storage unit 12, and an interface unit 13. The information processing device 10 is also connected to a display device 14 and an operation device 15 so as to be able to communicate data with them.

[0012] The control unit 11 includes at least one processor such as a CPU, and executes programs stored in the storage unit 12 to perform various types of information processing. In particular, in this embodiment, the control unit 11 includes a GPU (Graphics Processing Unit) 11a used for image rendering processing. Specific examples of processing performed by the control unit 11 in this embodiment will be described later. The storage unit 12 includes at least one memory device such as a RAM, and stores programs executed by the control unit 11 and data processed by the programs. The storage unit 12 also includes a buffer memory that temporarily stores frame images, which will be described later.

[0013] The interface unit 13 is an interface for data communication between the display device 14 and the operation device 15. The information processing device 10 is connected to the display device 14 and the operation device 15 via the interface unit 13 either wired or wirelessly. Specifically, the interface unit 13 includes a multimedia interface for transmitting a video signal supplied by the information processing device 10 to the display device 14. In particular, in this embodiment, the interface unit 13 includes an interface capable of transmitting a video signal in accordance with a standard compatible with a variable refresh rate, such as HDMI. The interface unit 13 also includes a data communication interface for receiving an operation signal indicating the content of an operation input by a user from the operation device 15. The interface unit 13 may further include a communication interface for transmitting and receiving data to and from another computer via a communication network, such as the Internet.

[0014] The display device 14 is a home television receiver, a monitor device for a personal computer, or the like, and displays an image based on a video signal received from the information processing device 10. In particular, in this embodiment, the display device 14 supports variable refresh rate technology, and updates frame images displayed on the screen at any timing determined according to the content of the signal input from the information processing device 10. In other words, the display device 14 can update the frame images it displays at a dynamically changing refresh rate, rather than at a predetermined fixed refresh rate.

[0015] The operation device 15 is a device for accepting operation inputs from the user of the information processing device 10, and may be, for example, a controller for a home game console, a keyboard, a mouse, etc. The operation device 15 has operation members such as buttons and a touchpad for accepting operation inputs from the user, and transmits operation signals indicating the content of the operation inputs from the user to the operation members to the information processing device 10.

[0016] Functions realized by the information processing device 10 in this embodiment will be described below with reference to the functional block diagram of FIG. 2. As shown in the diagram, the information processing device 10 functionally comprises a drawing command issuing unit 21, a frame image drawing unit 22, a drawing time measurement unit 23, and a drawing processing control unit 24. These functions are realized by the control unit 11 executing one or more programs stored in the storage unit 12. These programs may be provided to the information processing device 10 via a communication network such as the Internet, or may be provided to the information processing device 10 by storing them on an information storage medium such as an optical disc.

[0017] The drawing command issuing unit 21 determines the content of a frame image to be displayed on the screen of the display device 14, and issues the drawing command to the frame image drawing unit 22. The drawing command issuing unit 21 is realized by an application program such as a game program. As an example, when executing the processing of a game in which a character is controlled in a virtual space, the drawing command issuing unit 21 determines the positions and states of various objects placed in the virtual space, as well as the viewpoint position, and issues drawing commands to draw the state of the virtual space.

[0018] The drawing command issuing unit 21 repeatedly issues drawing commands for such frame images by continuously executing the loop processing. As a result, the frame images displayed on the screen of the display device 14 are updated sequentially and presented to the user as video. As mentioned above, because the display device 14 supports a variable refresh rate, the drawing command issuing unit 21 does not need to issue drawing commands at regular intervals, but can issue a drawing command for the next frame image to be displayed as appropriate when the frame image drawing unit 22 finishes drawing a frame image.

[0019] In response to drawing commands received from drawing command issuing unit 21, frame image drawing unit 22 draws a frame image to be displayed on the screen of display device 14, and transmits a video signal including data of the drawn frame image to display device 14. Frame image drawing unit 22 may be realized by a program independent of the program that realizes drawing command issuing unit 21, such as a graphics library or game engine. In particular, in this embodiment, at least a portion of the processing of frame image drawing unit 22 is realized by GPU 11a. In other words, GPU 11a executes at least a portion of the calculation processing required to draw the frame image.

[0020] In this embodiment, the frame image drawing unit 22 draws and transmits frame images at a variable frame rate. That is, upon completing the drawing of a frame image, the frame image drawing unit 22 immediately transmits the drawn frame image to the display device 14 and can start drawing the next frame image. The drawing command issuing unit 21 issues a drawing command for the next frame image in accordance with the timing at which the frame image drawing unit 22 completes drawing of the frame image. This allows the drawing command issuing unit 21 to continue issuing drawing commands in a cycle that matches the time required for the frame image drawing unit 22 to draw a frame image.

[0021] Each time frame image drawing unit 22 draws a new frame image, drawing time measurement unit 23 measures the time required for drawing the frame image (drawing time). Specifically, drawing time measurement unit 23 measures the time from the time frame image drawing unit 22 completes drawing the previous frame image to the time frame image drawing unit 22 completes drawing the next frame image as the drawing time required for drawing that frame image.

[0022] The rendering process control unit 24 controls the operating conditions of the frame image rendering process performed by the frame image rendering unit 22 based on the rendering time information measured by the rendering time measurement unit 23. If the frame image rendering time is shorter than expected, it is considered that the GPU 11a has a relatively large margin of processing power. Therefore, in such a case, the rendering process control unit 24 changes the operating conditions of the rendering process so as to reduce rendering performance when the frame image rendering unit 22 renders the next frame image. Generally, there is a trade-off between rendering performance and power consumption when rendering a frame image. Therefore, by reducing rendering performance, the frame image rendering time increases, but the power consumption required for rendering can be reduced. Conversely, if the time required for the frame image rendering process is longer than expected, the rendering process control unit 24 changes the operating conditions of the rendering process to improve the rendering performance of the frame image rendering unit 22, thereby shortening the time required for rendering subsequent frame images. This control reduces the power consumption of the information processing device 10 required for rendering frame images without excessively reducing the refresh rate of the video displayed on the display device 14.

[0023] Specifically, the rendering process control unit 24 determines in advance a target time Tx as a target value for the rendering time of a frame image. Then, each time the frame image rendering unit 22 renders a frame image, it compares the rendering time with the target time Tx and, if the rendering time is longer than the target time Tx, changes the operating conditions of the rendering process to improve rendering performance and shorten the rendering time, or, if the rendering time is shorter than the target time Tx, reduce power consumption and lengthen the rendering time. This makes it possible to control the rendering time of each frame image so that it approaches the target time Tx.

[0024] Here, the target time Tx may be a predetermined default value. For example, if the target refresh rate is 60 Hz, it is desirable to transmit a frame image to the display device 14 approximately once every 16.6 ms (=1000 ms / 60). In this case, the rendering process control unit 24 determines 16.6 ms as the target time Tx.

[0025] In this case, the default values ​​may be changeable depending on the settings input by the user in advance. Whether or not the rendering process control unit 24 actually executes control to change the operating conditions of the rendering process may also be determined depending on the settings input by the user. In this case, the information processing device 10 has a function for accepting user-specified operation settings, and stores the settings specified by the user in the storage unit 12. The rendering process control unit 24 then references the settings stored in the storage unit 12 and determines whether or not to execute control to change the operating conditions of the rendering process depending on the settings, and if so, what value to set for the target time Tx.

[0026] As an example, if the user preselects an operation mode that reduces power consumption, the rendering process control unit 24 executes control to change the operating conditions of the rendering process, but conversely, if the user selects to turn off this operation mode, the rendering process control unit 24 does not execute operation control. Also, when operating in an operation mode that reduces power consumption, the user specifies a frame rate value such as 60 Hz or 120 Hz, and the rendering process control unit 24 determines the target time Tx according to the specified frame rate value.

[0027] Alternatively, the rendering process control unit 24 may determine the target time Tx according to an application program that realizes the rendering command issuing unit 21 (i.e., an application program that determines the content of a frame image). The application program to be executed may assume a specific value as the refresh rate of the display device that displays the video. In this case, the rendering process control unit 24 determines the time corresponding to the refresh rate specified by the application program as the target time Tx. Specifically, for example, the rendering process control unit 24 may obtain the value of the refresh rate specified by the application program by referring to the setting information of the application program stored in the storage unit 12 or by receiving a notification from the application program itself that is being executed, and determine the target time Tx according to that value.

[0028] The rendering process control unit 24 may also determine the target time Tx according to the type of application program. For example, when the application program is a game program, the required frame rate tends to differ depending on the game genre. Specifically, for games that require real-time operability, such as action games and fighting games, it is desirable to update frame images at a high frame rate. On the other hand, for games where real-time performance is relatively unimportant, such as role-playing games and table games, it may be acceptable to render frame images at a relatively low frame rate. Therefore, when genre information about such application programs can be acquired, the rendering process control unit 24 may acquire the genre information of the application program being executed and select a target time Tx corresponding to the acquired genre information from among target times Tx predetermined for each genre as the target time Tx to be actually applied.

[0029] Furthermore, the rendering process control unit 24 may change the target time Tx in real time based on various conditions while the application program is running. For example, the rendering process control unit 24 may change the target time Tx in response to the content of the user's operation input. As a specific example, the rendering process control unit 24 monitors whether the user has input operation data to the operation device 15, and if no operation input has been performed for a certain period of time, changes the target time Tx to lower the frame rate to a given target value (e.g., 48 Hz). Conversely, while operation input is being performed, changes the target time Tx to raise the frame rate to a given target value (e.g., 120 Hz). In this way, the frame rate can be lowered and power consumption can be reduced while the user is not performing operations on the application program.

[0030] The rendering process control unit 24 may also change the target time Tx depending on whether or not there is a change in the frame image. As an example, the rendering process control unit 24 determines whether or not there is a change in the frame image by calculating a hash value for each new frame image that is rendered and comparing it with the previously calculated hash value. If the frame image remains unchanged for a predetermined period of time or longer, the rendering process control unit 24 changes the target time Tx so as to reduce the frame rate to a given target value (e.g., 48 Hz). In this way, if there is no change in the frame image displayed on the display device 14, the frequency of rendering the frame image can be reduced, thereby reducing power consumption.

[0031] As described above, when changing the target time Tx during the execution of an application program, it is also possible to change it stepwise by a predetermined amount, rather than changing it all at once to a target value determined depending on the situation.

[0032] Furthermore, the various criteria for determining the target time Tx described above may be used in combination. For example, when the target time Tx is changed in response to the content of the user's operation input, a value predetermined for each genre of application program may be used as the target value. Furthermore, when the target time Tx is changed in real time, the target value may be determined to be within a range between an upper limit and a lower limit specified by the application program being executed.

[0033] Below, specific examples of operating conditions for the rendering process that are the target of control by the rendering process control unit 24 will be described. For example, the rendering process control unit 24 may change the operating frequency of the GPU 11a as an example of an operating condition for the rendering process. In this case, the rendering process control unit 24 lowers the operating frequency of the GPU 11a when the rendering time is shorter than the target time Tx, and raises the operating frequency of the GPU 11a when the rendering time exceeds the target time Tx. It is known that lowering the operating frequency of the GPU 11a can significantly reduce the power consumption of the GPU 11a. Therefore, by changing the operating frequency of the GPU 11a according to the time required for each frame image rendering, it is possible to bring the rendering time of the frame image closer to the target time Tx, while reducing power consumption when there is a relatively sufficient amount of time for rendering.

[0034] Furthermore, the rendering process control unit 24 may change the number of arithmetic units (such as the number of shaders or the number of cores) used for rendering the frame images within the GPU 11a as an operating condition for the rendering process. The rendering process control unit 24 may also change the memory bandwidth, etc. Additionally, the rendering process control unit 24 may change various parameters that affect the rendering performance and power consumption of the GPU 11a according to the rendering time.

[0035] Furthermore, the rendering process control unit 24 may determine the amount of change in the parameters related to the operating conditions of the rendering process depending on the difference between the rendering time and the target time Tx. Specifically, the rendering process control unit 24 changes the parameters related to the operating conditions more significantly the greater the difference between the rendering time and the target time Tx. This allows the actual time required to render a frame image to quickly approach the target time Tx when it deviates significantly from the target time Tx.

[0036] A specific example of control executed by the information processing device 10 according to this embodiment will now be described with reference to Fig. 3. This figure shows, as an example, the control content when drawing three frame images starting from a reference time t0.

[0037] In the example shown in this figure, the frame image rendering unit 22 first renders the first frame image at an operating frequency of 2.0 GHz for the GPU 11a. As a result, the rendering time T1 required to render the first frame image falls short of the target time Tx. In response, the rendering process control unit 24 reduces the operating frequency of the GPU 11a at time t1 when rendering of the first frame image is completed. As a specific example, the operating frequency of the GPU 11a is changed to 1.8 GHz.

[0038] In response to this change, the frame image rendering unit 22 renders the second frame image at an operating frequency of 1.8 GHz for the GPU 11a. As a result, the rendering time T2 for the second frame image exceeds the target time Tx by a relatively large amount. In response to this, the rendering process control unit 24 increases the operating frequency of the GPU 11a at time t2 when rendering of the second frame image is completed. In this specific example, the rendering process control unit 24 increases the operating frequency of the GPU 11a by a relatively large amount, and as a result, the third frame image is rendered at 2.1 GHz, which is higher than the operating frequency when the first frame image was rendered. As a result, the third frame image is rendered in a relatively short rendering time T3.

[0039] With this type of control, it is expected that each frame image will be drawn in a drawing time that is close to the target time Tx overall. On the other hand, even if a frame image is drawn in a drawing time that is longer than the target time Tx, the display device 14 updates the frame images at a variable refresh rate, so that each frame image transmitted from the information processing device 10 can be displayed sequentially without dropping frames or the like.

[0040] As described above, according to the information processing device 10 of this embodiment, it is possible to reduce the power consumption required to draw frame images without significantly reducing the refresh rate at which the display device 14 displays video, thereby enabling efficient drawing processing.

[0041] It should be noted that the embodiments of the present invention are not limited to those described above. For example, in the above description, operating parameters such as the operating frequency of the GPU 11a are changed according to the frame image rendering time, but various other operating conditions related to the frame image rendering process may also be changed. Furthermore, multiple types of operating conditions may also be changed together.

[0042] In the above description, the operating conditions of the rendering process are changed according to the rendering time each time a frame image is rendered. However, the rendering process control unit 24 is not limited to this. The rendering process control unit 24 may change the operating conditions of the rendering process each time multiple frame images are rendered according to the total time required for the rendering process of those frame images. Alternatively, the rendering process control unit 24 may change the operating conditions of the rendering process each time multiple frame images are rendered according to the time required for the rendering process of the last frame image.

[0043] Furthermore, in the above description, the operating conditions of the rendering process are changed when the rendering time exceeds or falls short of the target time Tx even slightly, but this is not limiting and the rendering process control unit 24 may change the operating conditions of the rendering process based on the result of comparing the rendering time with a given threshold determined according to the target time Tx. Specifically, for example, the rendering process control unit 24 may reduce the performance of the rendering process when the rendering time exceeds a first threshold determined according to the target time Tx, and increase the performance of the rendering process when the rendering time falls short of a second threshold different from the first threshold. [Explanation of symbols]

[0044] 10 information processing device, 11 control unit, 11a GPU, 12 memory unit, 13 interface unit, 14 display device, 15 operation device, 21 drawing command issuing unit, 22 frame image drawing unit, 23 drawing time measurement unit, 24 drawing processing control unit.

Claims

1. An information processing device connected to a display device capable of updating frame images at a variable refresh rate, a frame image drawing unit that executes a drawing process of a frame image and transmits the drawn frame image to the display device; a drawing time measurement unit that measures a drawing time required for drawing the frame image; a rendering processing control unit that changes operating conditions of the rendering processing in accordance with the measured rendering time; Including, The drawing processing control unit changing the operating conditions of the rendering process in accordance with a result of comparing the measured rendering time with a given target time; The target time is changed in response to determining whether or not a user inputs an operation during execution of an application program that determines the content of the frame image.

1. An information processing device comprising:

2. In the information processing device according to claim 1, The rendering process control unit changes the operating conditions of the rendering process depending on the magnitude of the difference between the measured rendering time and the target time.

1. An information processing device comprising:

3. In the information processing device according to claim 1 or 2, The drawing process control unit determines the target time in accordance with a frame rate value designated by an application program that determines the content of the frame image.

1. An information processing device comprising:

4. In the information processing device according to claim 1 or 2, The drawing processing control unit determines the target time in accordance with the type of application program that determines the content of the frame image.

1. An information processing device comprising:

5. In the information processing device according to claim 1 or 2, The drawing processing control unit determines the target time in accordance with settings specified in advance by the user.

1. An information processing device comprising:

6. In the information processing device according to claim 1, The drawing processing control unit changes the target time in response to determining that the operation input has not been performed for a certain period of time.

1. An information processing device comprising:

7. In the information processing device according to claim 1, The drawing processing control unit changes the target time in accordance with the content of the operation input by the user.

1. An information processing device comprising:

8. In the information processing device according to claim 1, The drawing process control unit changes the target time in accordance with a change in the frame image drawn by the frame image drawing unit.

1. An information processing device comprising:

9. 2. The information processing device according to claim 1, the frame image drawing unit draws the frame image through calculation processing by a GPU; The rendering processing control unit changes the operating parameters of the GPU as the operating conditions of the rendering processing.

1. An information processing device comprising:

10. A control method for an information processing device connected to a display device capable of updating frame images at a variable refresh rate, comprising: a step of executing a drawing process of a frame image and transmitting the drawn frame image to the display device; measuring a drawing time required for the drawing process of the frame image; changing an operating condition of the drawing process in accordance with the measured drawing time; changing the operating conditions of the rendering process in response to a result of comparing the measured rendering time with a given target time; changing the target time in accordance with the content of an operation input by a user during execution of an application program that determines the content of the frame image; 10. A method for controlling an information processing device, comprising:

11. In the information processing device according to claim 1, The drawing processing control unit decreases the target time in response to determining that the operation input has been performed, and increases the target time in response to determining that the operation input has not been performed for a certain period of time.

1. An information processing device comprising:

12. A computer connected to a display device capable of updating frame images at a variable refresh rate, a step of executing a drawing process of a frame image and transmitting the drawn frame image to the display device; measuring a drawing time required for the drawing process of the frame image; changing an operating condition of the drawing process in accordance with the measured drawing time; changing the operating conditions of the rendering process in response to a result of comparing the measured rendering time with a given target time; changing the target time in response to determining whether or not a user has performed an operation input during execution of an application program that determines the content of the frame image; A program to execute.

Images