Content rendering control system and method
The system dynamically adjusts content rendering and input/output elements based on user interaction, addressing inefficiencies in existing systems by reducing power and computational usage while maintaining user engagement.
Patent Information
- Application Number
- GB2023006873
- Authority / Receiving Office
- GB · GB
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-10
- Publication Date
- 2024-11-13
AI Technical Summary
Current systems for controlling content rendering in entertainment devices continue to consume significant computational and power resources even when users are not actively interacting with the content, often entering standby mode too late or incorrectly due to inability to differentiate between intentional and inadvertent inputs, leading to inefficient power usage and user inconvenience.
A system that includes a hold detection unit and a modification unit to determine whether a peripheral device is being held by the user, allowing for real-time adjustment of content rendering quality and input/output element activation, reducing processing and power usage while maintaining user interaction capabilities.
This approach reduces computational and power consumption by dynamically adjusting rendering quality and input/output element activation based on user interaction, preventing accidental inputs and enhancing user experience by allowing quicker return to active interaction.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
BACKGROUND OF THE INVENTION Field of the invention The present invention relates to a systems and methods for controlling the rendering of content. Description of the Prior Art Users frequently interact with displayed content using various input devices. For example, users may use a games controller to play, and interact with, a videogame. It is typical for users to put down the input device at various times to take breaks from interacting with the content, e.g. to eat food, or go to the bathroom. However, despite the user no longer interacting with the content, the content may continue to be displayed. Since processing (e.g. rendering) the content is often a computationally intensive process, such prolonged display of the content may result in significant (e.g. computational and power) resource usage. To address this problem, some systems enter into a standby, reduced power, state if no inputs are received from the input device over a given period of time. However, this approach suffers from several drawbacks. The standby mode is typically entered after a relatively long period of time without inputs, which results in such systems continuing to operate at full power for longer and thus increased power consumption. In addition, such systems are often unable to differentiate between intentional and inadvertent user inputs (e.g. caused by a user sitting on the input device), which may result in the standby mode being entered at the wrong times (or not being entered when it should be). Further, the user is typically not able to interact with the displayed content in any way while the system is in standby mode, and it may take a considerable amount of time for the system to exit the standby mode-thus, users often find it cumbersome to return to interacting with the content. The present invention seeks to mitigate or alleviate these drawbacks. In particular, the present invention aims to provide more accurate and efficient control of interaction between displayed content and an input device, and an improved user experience. SUMMARY OF THE INVENTION Various aspects and features of the present invention are defined in the appended claims and within the text of the accompanying description and include at least: In a first aspect, a system for controlling the rendering of content is provided in accordance with claim 1. In another aspect, a method for controlling the rendering of content provided in accordance with claim 14. BRIEF DESCRIPTION OF THE DRAWINGS A more complete appreciation of the disclosure and many of the attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings, wherein: Figure 1 is a schematic diagram of an entertainment system in accordance with embodiments of the present description; Figure 2 is a schematic diagram of a peripheral device; Figure 3 is a schematic diagram of a system for controlling interaction; Figure 4 is a flow diagram of a method for controlling interaction; Figure 5 is a flow diagram of a method for modifying interaction; and Figure 6 is a flow diagram of a further method for controlling interaction. DESCRIPTION OF THE EMBODIMENTS Systems and methods for controlling interaction between displayed content and a peripheral device operable by a user are disclosed. In the following description, a number of specific details are presented in order to provide a thorough understanding of the embodiments of the present invention. It will be apparent, however, to a person skilled in the art that these specific details need not be employed to practice the present invention. Conversely, specific details known to the person skilled in the art are omitted for the purposes of clarity where appropriate. In an example embodiment of the present invention, a suitable system and / or platform for implementing the methods and techniques herein may be an entertainment device such as the Sony® PlayStation 5® videogame console. Referring now to the drawings, wherein like reference numerals designate identical or corresponding parts, Figure 1 shows an example of an entertainment system 10 which is a computer or console such as the Sony® PlayStation 5® (PS5). The entertainment system 10 comprises a central processor 20. This may be a single or multi core processor, for example comprising eight cores as in the PS5. The entertainment system also comprises a graphical processing unit or GPU 30. The GPU can be physically separate to the CPU, or integrated with the CPU as a system on a chip (SoC) as in the PS5. The entertainment device also comprises RAM 40, and may either have separate RAM for each of the CPU and GPU, or shared RAM as in the PS5. The or each RAM can be physically separate, or integrated as part of an SoC as in the PS5. Further storage is provided by a disk 50, either as an external or internal hard drive, or as an external solid state drive, or an internal solid state drive as in the PS5. The entertainment device may transmit or receive data via one or more data ports 60, such as a USB port, Ethernet® port, Wi-Fi® port, Bluetooth® port or similar, as appropriate. It may also optionally receive data via an optical drive 70. Audio / visual outputs from the entertainment device are typically provided through one or more A / V ports 90, or through one or more of the wired or wireless data ports 60. An example of a device for displaying content (e.g. images) output by the entertainment system is a head mounted display 'HMD' 120, such as the PlayStation VR 2 'PSVR2', worn by a user 1. It will be appreciated that the content may be displayed using various other devices-e.g. using a conventional television display connected to A / V ports 90. Where components are not integrated, they may be connected as appropriate either by a dedicated data link or via a bus 100. The user interacts with, and provides inputs to, the system typically using one or more peripheral devices (80, 130A), such as the DualSense® controller (80) in the case of the PS5, and / or one or more VR controllers (130A-L,R) in the case of the HMD. It will be appreciated that the same device may be used to both display content and receive user inputs. For example, a PlayStation Portable (PSP) device may be used to provide both of these functions. Referring now to Figure 2, a peripheral device may take one of a number of forms. Non-limiting examples include a wand-style controller such as the Sony PlayStation Move®, a joystick (which may be held freely in the hand or mounted on a fixed mount), a steering wheel (which again may be held freely in the hand or mounted on a fixed mount), or a handheld controller such as the DualSense® controller 80 shown in Figure 2. References to a peripheral device or the DualSense® controller herein may be taken to encompass other peripheral devices such as those listed above unless otherwise specified. Similarly as will be described later herein, some functions of a handheld controller 80 may be performed by a head-mounted display 802, and so references a handheld controller or the DualSense® controller herein may be taken to encompass a head mounted display as appropriate. In Figure 2, the DualSense® controller 80 is illustrated as an example of a handheld controller acting as a peripheral device. Such a controller typically has two handle sections 81L,R and a central body 81C. Various controls are distributed over the controller, typically in local groups. Examples include a left button group 82L, which may comprise directional controls 87L and / or one or more shoulder buttons (e.g. triggers) 83L, and similarly right button group 82R, which comprise function controls 85R and / or one or more shoulder buttons 83R. The controller also includes left and / or right joysticks 84L,R, which may optionally also be operable as buttons by pressing down on them. The controller (typically in the central portion of the device) may also comprise one or more system buttons 86, which typically cause interaction with an operating system of the entertainment device rather than with a game or other application currently running on it; such buttons may summon a system menu, or allow for recording or sharing of displayed content. Furthermore, the controller may comprise one or more other elements such as a touchpad 88, a light for optical tracking (not shown), a screen - i.e. display (not shown), haptic feedback elements (not shown), and the like. Each of the controls distributed over the controller may be used by a user to input commands to a device, such as the entertainment system 10, and as such may be referred to as input elements (that is, elements of a peripheral operable to provide an input). Further inputs may also be provided through the sensing of the motion of the controller (for example, using an accelerometer embedded within the controller, or an image-based tracking of the controller), as well as the sensing of audio via a microphone associated with the controller. Likewise, the controller 80 may comprise various output elements (that is, elements of a peripheral device operable to provide an output to the user), such as a screen, haptic feedback elements, lighting features (e.g. LED lights), or a loudspeaker. Embodiments of the present disclosure relate to modifying (in particular, reducing) aspects of interaction between displayed content and the peripheral device (e.g. reducing the quality of the displayed content, or deactivating input / output elements of the peripheral device) upon determining (i.e. detecting) that the peripheral device is not held by the user. Thus, the system is able to detect in real time whether a user wants to interact with, and control, the displayed content, and reduce aspects of the interaction on this basis. Reducing aspects of the interaction in this way allows reducing processing resource and power usage, both by the peripheral device itself, and any other devices used to process and display the content. In addition, reducing these aspects (e.g. pausing the content, or deactivating input elements of the peripheral device) may allow preventing inadvertent user input (e.g. by sitting on a controller), and protect the privacy of the user (e.g. by deactivating a microphone on the peripheral device when it is determined as not being held). A number of examples of the present disclosure relate to modifying rendering of the displayed content (e.g. reducing rendering quality of the displayed content) based on determining whether a peripheral device, operated by a user to interact with the content, is held by the user. Determining whether the device is held by the user allows more accurately determining whether the user is, or wants to, actively interact with the content, and modifying rendering of the content on this basis allows reducing the computational and power cost of displaying the content in a more nuanced manner where usage of processing resources is reduced to the extent possible while not sacrificing the user experience. In other words, the balance between reducing processing and the user experience is improved. For example, by modifying rendering of the content upon determining that the peripheral device is not held, as opposed to simply entering a standby mode after no inputs are received, processing is reduced whilst still displaying content (albeit with modified rendering) and so still allowing the user to interact with the content in a limited manner and allowing the user to more quickly return to actively interacting with the content (as opposed to exiting a standby mode which can take more time). The present disclosure is particularly applicable to videogame systems, where users often alternate between actively playing the game using a peripheral device, and putting the peripheral down to do other tasks. Thus, the present invention may allow more efficiently reducing the overall power and processing resource usage in videogame systems by modifying rendering of the videogame based on whether the peripheral is determined as being held. However, it will be appreciated that the present disclosure can equally be applied to control of any other displayed content using a peripheral device. For example, the present disclosure can be applied to user control of other rendered content (e.g. a film, or spreadsheet) displayed on a personal computer using any appropriate hand-held peripheral device such as a mouse. Figure 3 shows an example of a system for controlling interactions between displayed content and a peripheral device operable by a user in accordance with one or more embodiments of the present disclosure. The system comprises a hold detection unit 320 and a modification unit 330. The hold detection unit 320 determines whether a (hand-held) peripheral device is held by a user based on outputs by one or more sensors, as described in more detail below. For example, the hold detection unit 320 may determine whether the device is held by the user based on the output of a motion sensor and the motion of the device determined by the motion sensor. The modification unit 330 modifies one or more aspects of the interaction between the displayed content and the peripheral device upon the hold detection unit 320 determining that the peripheral device is not held by the user, as described in more detail below. In a number of examples, the modification unit 330 modifies rendering of the displayed content in dependence on the hold detection unit determining whether the peripheral device is held by the user. For example, the modification unit 330 may reduce the rendering quality (e.g. frame rate, or graphics quality) of the displayed content upon detection that the peripheral device is not held, in order to reduce processing resource and power usage. The system may further comprise an input unit 310 that receives one or more user inputs from the peripheral device. It will be appreciated that each of the input unit 310, hold detection unit 320, and modification unit 330 may be implemented as part of the peripheral device (e.g. one or more processors of the controller 80) and / or a further device (e.g. the CPU 20 of the entertainment system 10). More generally, it will be appreciated that these units may be implemented using one or more processors, distributed over one or more devices, as appropriate. It will also be appreciated that the one or more sensors (based on which the hold detection unit 320 determines whether the device is held by the user) may be provided as part of the peripheral device (e.g. as a motion or touch sensor on the device) and / or as part of a further device (e.g. as an external camera). Likewise, it will be appreciated that the one or more sensors may be provided on the hold detection unit 320 (e.g. in cases where the hold detection unit 320 is implement as part of the peripheral device) or separately to the hold detection unit (e.g. in cases where the hold detection unit 320 is implemented as part of a further device). Figure 4 shows an example of a method of controlling interaction between displayed content and a peripheral device using the system of Figure 3. At step 410, the input unit 310 may receive one or more user inputs from a peripheral device (for example, the controller 80 or one of VR controllers 130A-L,R) operated by a user. The inputs may comprise any suitable inputs for a particular peripheral device; examples include button presses, joystick manipulation, audio inputs, motion inputs (e.g. movements of the controller 80), and / or image inputs (e.g. gaze movements tracked using a camera installed in the HMD 120). It will be appreciated that step 410 is optional, and the method may start at the detection step 420 without first receiving inputs from the peripheral device. At step 420, the hold detection unit 320 determines (i.e. determines) whether or not the peripheral device is being held by the user - in other words, whether the user is holding the peripheral device. As used herein, the term "hold" (or similar terms such as "held", "holding", etc.) preferably connotes one or more of the user's hands being in contact with the peripheral device, optionally the user's hand(s) being in a position such that the user can operate the peripheral device to provide inputs to the system - e.g. the user's hands being arranged such that they can operate button inputs (e.g. the user's palms gripping handle sections 81L,R of the controller 80 and their fingers hovering over the various buttons (e.g. the triggers 83L,R)), or firmly holding the peripheral device such that they can provide motion inputs by moving the peripheral device. Thus, in some cases, the hold detection unit 320 may differentiate between a user's hand being merely in contact with the peripheral device (e.g. touching a handle section 81L of controller 80 using a finger) and the user's hand(s) being arranged in such way that the user can operate the input elements of the peripheral device. The hold detection unit 320 determines whether the peripheral device is held based on data captured by one or more sensors. The sensors capture data relating to usage of the peripheral device by the user, which is then processed by the hold detection unit 320 to determine whether the device is being held. In some cases, the hold detection unit 320 may comprise the sensors. In examples of the present disclosure, the sensors may comprise one or more of the following sensors (alone or in any combination): a motion sensor, touch sensor, camera (internal and / or external to the peripheral device), force sensor, temperature sensor, light sensor, and / or moisture sensor. Considering motion sensors, the peripheral device may comprise one or more motion sensors (e.g. an accelerometer, or an inertial measurement unit (IMU)) that detect motion of the peripheral device. The hold detection unit 320 may then determine whether the peripheral device is being held by the user based on the detected motion of the device. The use of motion sensors is advantageous as they can be used to accurately detect holding of the device, while being relatively cheap. The hold detection unit 320 may compare the detected motion against one or more predetermined conditions, and determine whether the peripheral device is held (or not held) based on whether the motion meets these conditions. For instance, the hold detection unit 320 may compare the detected motion against one or more predetermined thresholds and / or motion patterns. For example, the hold detection unit 320 may determine that the device is held if its detected motion (e.g. the acceleration of the device, or total distance moved) exceeds a predetermined threshold at a given instant or over a period of time (e.g. last five seconds), and not held if the detected motion is below this threshold (e.g. if the peripheral device has not moved in the last five, or ten seconds). In some cases, the hold detection unit 320 may determine that the device is not held if its detected motion (e.g. speed) exceeds a further predetermined threshold to detect instances where the device is moving but is not being held - for example, when the peripheral device has been thrown or falls (e.g. from a table). Alternatively, or in addition, the hold detection unit 320 may determine that the device is held (or not held) if its motion meets one or predetermined motion patterns. For example, the hold detection unit 320 may be trained using a machine learning model to identify motion patterns associated with the user holding (or not holding) the device and compare the detected motion to these identified patterns to determine whether they match. For example, the machine learning model may be trained to associate motion relating to the user periodically tilting the device side to side as corresponding to the peripheral device being held by the user, and motion corresponding to fast spinning of the device (e.g. as a result of being thrown) as corresponding to the device not being held. Considering touch sensors, the peripheral device may comprise one or more touch sensors that are configured to detect whether a user is touching the device. The hold detection unit 320 may then determine whether the user is holding the peripheral device based on whether, and / or which, touch sensors are detecting user touch. The touch sensors may comprise any suitable sensors for detecting user touch, such as capacitive or resistive touch sensors, infrared touch sensors, or surface acoustic wave (SAW) sensors. The one or more touch sensors may be arranged at locations on the peripheral device that allow detecting touch of the user's hand(s) in a position in which the user can operate the peripheral device to provide inputs to the system. For example, the touch sensors may be provided on handle sections 81L,R of the controller 80 as these are typically held by the user when the user operates the peripheral device to provide inputs. The hold detection 320 unit may determine whether the device is being held based on the occurrence (i.e. "touch detected" vs. "touch not detected"), duration, number, frequency, force, and / or pattern of user touches detected by the touch sensors. For example, the hold detection unit 320 may determine that the user is holding the device if one or more touch sensors have been activated for a period of time exceeding a predetermined threshold (e.g. 2 seconds), and that the user is not holding the device otherwise. Alternatively, or in addition, where multiple touch sensors are provided, the hold detection unit 320 may determine whether the user is holding the device based on the pattern of activated (i.e. touched) touch sensors. For example, the controller 80 may comprise a touch sensor on each of the handle portions 81L,R, and the hold detection unit 320 may determine that the user is holding the controller 80 only if touch sensors on both handle portions are activated. Considering force, light, and temperature sensors, these may operate in substantially the same way as the touch sensors described above. Force, or pressure, sensors may detect holding of the device by detecting application of a force on the device by the user (e.g. squeezing of the peripheral device while playing a game). For improved hold detection, force sensors may be provided on outer edges of the peripheral device as the user is more likely to apply a force at these locations. Light sensors may detect holding of the device by detecting whether the user's hands are occluding light from reaching the sensors. Likewise, temperature and moisture sensors may detect holding of the device by detecting an increase in temperature / moisture corresponding to the user holding the device (e.g. caused by the (typically higher than ambient) temperature and / or sweating of the user's hands). Considering cameras, the one or more sensors may comprise one or more cameras that capture images of the user and / or the peripheral device. The hold detection unit 320 may then process these images to determine whether the user is holding the peripheral device. The hold detection unit 320 may comprise a machine learning model trained to determine whether the user is holding the peripheral device based on images captured by the one or more cameras. The peripheral device may comprise the one or more cameras. The cameras may be arranged (e.g. positioned and oriented) to detect when the user is holding the device. For example, a camera may be arranged to capture images of part of the controller 80 (e.g. the handle sections 81L,R or the joysticks 84L,R) and the hold detection unit 320 may determine whether a hand (or part thereof) is present in the images, thus determining whether the user is holding the peripheral device. Alternatively, or in addition a camera may be arranged to capture images of the user (e.g. the user's face or hands) and detect whether the user is holding the device on this basis. For example, a camera may be provided on, or adjacent to, the touchpad 88 of the controller 80, and the hold detection unit 320 may determine that the device is held by the user if it detects the user's face or torso (optionally at a specific position) in images captured by the camera. Alternatively, or in addition, to cameras on the peripheral device, one or more cameras that are external to the peripheral device may be provided. These external camera(s) may capture images of the user and the peripheral device, and the hold detection unit 320 may determine whether the user is holding the device using appropriate image processing techniques and models, such as YOLOV3 or SSD object detection models. It will be appreciated that the various sensors described herein may be used in any appropriate combination. In one example, the sensors may comprise a motion sensor and a touch sensor provided on the peripheral device. The hold detection unit 320 may then determine whether the user is holding the device based on data captured by both the motion sensor and the touch sensor. For example, the hold detection unit 320 may determine whether the user is holding the device based on whether the detected motion meets one or more predetermined conditions and whether the touch sensor is activated. In some cases, the hold detection unit 320 may determine whether the user is holding the peripheral device further based on the current displayed content - in other words, the content being displayed at the time the sensors capture data and / or the application corresponding to this content. For example, the hold detection unit 320 may store different predetermined motion patterns for different applications (e.g. videogames), and determine whether the user is holding the device based on motion of the device as captured by a motion sensor and the current application (e.g. current videogame being played by the user) which determines the predetermined motion patterns. It will also be appreciated that the processing of the data captured by the sensors (e.g. by the hold detection unit 320) may be performed on the peripheral device and / or on one or more processors that are separate to the peripheral device, such as the CPU 20 of the entertainment system 10 of Figure 1. At step 430, the modification unit 330 modifies one or more aspects of the interaction between the displayed content and the peripheral device in dependence on whether the device is held by the user, as determined by the hold detection unit 320. In particular, the modification unit 330 may reduce one or more of the interaction aspects upon the hold detection unit 320 determining that the device is not held by the user. In this way, the system is able to react to the user not holding the device and reduce aspects of the interaction to reduce the processing resource and power usage. In some instances, rendering of displayed content for prolonged periods of time can result in damage to the rendering system (e.g. a console, or the entertainment system 10), for example due to overheating of the system's processors. Thus, by reducing power consumption and therefore heat generation upon determining that the user is not holding the peripheral device, the present system may allow avoiding such damage. In some cases, the modification unit 330 may reduce an interaction aspect upon the hold detection unit 320 determining that the peripheral device has not been held by the user for a predetermined period of time (e.g. 5 seconds, 20 seconds, or 1 minute). This allows avoiding pre-emptively reducing aspects of the interaction when a user might still want to interact with the displayed content. The predetermined period of time may be selected by a user. Alternatively, or in addition, modification unit 330 may determine this period of time based on historical data relating to holding the peripheral device (and / or one or more further peripheral devices) by the user (and / or one or more additional users), e.g. using an appropriately trained machine learning model. In some examples, the predetermined period may instead (or also) be selected in dependence upon the content being interacted with - for instance, a lower-intensity content (such as a game with a lot of cut-scenes, during which a user may not always hold a controller) may be associated with a longer predetermined period while higher-intensity content (such as a first-person shooter) may be associated with a shorter predetermined period. A further alternative or additional consideration may be the type of peripheral that is being used - for instance, a heavier or otherwise less comfortable controller may be associated with a longer predetermined period as it may be expected that a user would desire breaks from holding the controller. Similarly, a peripheral associated with virtual reality inputs or the like (that is, a peripheral which is often strapped to a user and / or used in a context in which it would be harder for the user to pick up the peripheral again due to restricted vision) may be associated with a shorter predetermined period as it may be considered less likely that the user stops holding the controller at any time other than during a break in play or interaction. In some cases, the modification unit 330 may reduce different aspects of the interaction after different predetermined periods of time during which the peripheral device is not held by the user. This allows the modification unit to progressively reduce more aspects as the peripheral device continues not being held. In this way, an improved balance between reducing processing resource usage and quality of the interaction and the user experience is provided. For example, if the device is not held only briefly (e.g. for 5 seconds), the frame rate of the displayed content may be reduced thus only partially reducing the quality of the interaction, but as the device is not held for longer periods of time additional aspects may be reduced (e.g. input elements of the peripheral device may be deactivated) to obtain further processing resource savings as it becomes less likely the user wants to, or will shortly want to, interact with the system and displayed content. An example of such progressive reduction of aspects is further illustrated and described with reference to Figure 6 below. Similarly, the modification unit 330 may modify a given aspect to a greater degree as the period of time during which the peripheral device has not been held increases. For example, the modification unit 330 may progressively reduce the resolution of the displayed content as this period of time increases (e.g. from 1080p to 720p after 10 seconds and from 720p to 360p after a further 20 seconds). A timer for this period of time may be reset each time the hold detection unit 320 determines that the user is holding the peripheral device. In some cases, the modification unit 330 may output a notification to the user that aspects of the interaction have been modified, e.g. by including this notification in the displayed content. In some cases, the modification unit 330 may modify the interaction aspects further based on the current displayed content- in other words, the content being displayed when the hold detection unit 320 determines that the user is not holding the peripheral device and / or the application associated with the current content. Different interaction aspects may be modified and / or interaction aspects may be modified in a different manner (e.g. after a different predetermined period of time or to a different degree) depending on the current content and / or application. For example, if the current application is a videogame where users frequently put down the peripheral device while still interacting with the content (e.g. a role playing game (RPG) such as Red Dead Redemption®), the modification unit 330 may reduce aspects of the interaction only after the peripheral is not held for a longer predetermined period of time (e.g. 2 minutes) and / or reduce aspects to a lesser degree (e.g. reduce rendering quality by reducing resolution of the content by 25%). Likewise, if the current application is a streaming service (e.g. Netflix®), the modification unit 330 may reduce aspects after a yet longer period of time (e.g. 2 hours). In contrast, where the application is a game where users are unlikely to not hold the peripheral while interacting with the content (e.g. a sports or shooting game), the modification unit 330 may reduce aspects after a shorter period of time (e.g. 10 seconds) and / or to a greater degree (e.g. reduce rendering quality by reducing resolution of the content by 50%). Similarly, on a more granular level, if the current displayed content corresponds to a storyline sequence in a game, the modification unit 330 may reduce aspects of the interaction only after the peripheral is not held for a longer predetermined period of time than in cases where the current displayed content corresponds to an action sequence in the game. Referring to Figure 5, the interaction aspects may be modified in a variety of ways, and modifying aspects of the interaction at step 430 may comprise one or more of the following steps (alone or in any combination): Step 432 - modifying one or more aspects relating to the displayed content; and / or Step 434 - modifying one or more aspects relating to the peripheral device (e.g. the controller 80). At step 432, the modification unit 330 modifies one or more aspects of the displayed content. This may comprise modifying rendering of the displayed content. As used herein, the term "modifying rendering" preferably connotes modifying one or more aspects of rendering the content - in other words, modifying one or more aspects of the process of rendering the displayed content (i.e. the rendering process), as opposed to modifying the target rendered content (such as modifying a game state through user interaction). Thus, modifying rendering of the displayed content may be performed independently to a user interaction, and differs from changing the target rendered content upon interaction with the user (or a peripheral device thereof), such as the user pausing the content or moving their in-game character (which would result in change to the target rendered content). Modifying rendering of the display content allows reducing processing resource usage whilst still allowing the user to interact with the content in a more limited manner. For example, when a user is playing a game, modifying the rendering by reducing the resolution allows reducing computational costs while still allowing the user to view the displayed content (e.g. a storyline segment in the game). In addition, modification of the rendering provides an indication to the user that they are not holding the device, thus, e.g., prompting them to return to actively interacting with the content by picking up the device, or, if they no longer with to interact with the content, to turn off the entertainment device 10 to further conserve power. Modifying rendering of the displayed content may include one or more (alone or in any combination) of: modifying (rendering) quality of the displayed content, modifying the rendered content, modifying one or more rendering operations, and / or modifying an allocation of processing resources to rendering the displayed content. Considering modifying rendering quality of the displayed content, the modification unit 330 may reduce the rendering quality upon the hold detection unit 320 determining that the peripheral device is not being held by the user. This allows reducing the amount of processing resources needed to display the content. To reduce rendering quality, the modification unit 330 may reduce one or more parameters of the displayed content. For example, the modification unit 330 may reduce the graphics quality of the displayed content. This may include reducing the resolution of the content (e.g. from 1080 pixels per frame to 720 pixels per frame), or reducing any other graphics quality, such as colouring of the pixels (e.g. from displaying images in colour to displaying them in black and white, or with reduced colour depth), or the size of the content (e.g. from filling the entire display to 50% of the display). Alternatively, or in addition, the modification unit 330 may reduce the frame rate of the displayed content - for example, from 60 frames per second to 30 frames per second; and / or the size of the displayed content. In some cases, reduction of the quality may be localised to one or more portions of the displayed content. For example, the modification unit 330 may first reduce the quality of the outer parts of the displayed content (e.g. the outer 25% or 50% of the display area) when the peripheral device is determined as not being held, and reduce the quality of the rest of the displayed content only upon the peripheral device not being held for a predetermined period of time. The modification unit 330 may store one or more rules for determining at which point in time (i.e. after what period of the device being determined as not being held) to reduce the quality of which parts of the displayed content. Considering modifying the rendered content, the modification unit 330 may determine the content for rendering based on whether the device is being held by the user. The modification unit 330 may determine the content for rendering based on one or more predetermined rules that dictate which content to render based on various conditions, such as the nature of the displayed content (e.g. a specific game being played and / or a specific point in that game), whether the device is held or not (and for how long), and / or user preferences (e.g. as previously provided by the user). It will be appreciated that the displayed content may comprise a plurality of graphical features of a virtual environment (e.g. the virtual environment in a videogame). In an example, the modification unit 330 may determine which of these graphical features to render in dependence on whether the peripheral device is held by the user. For example, upon determining that the user is not holding the device, the modification unit 330 may determine a subset of these graphical features for rendering (where by default, e.g. when the peripheral device is held by the user, all of these features are rendered). For example, upon determining that the user is not holding the device, the modification unit 330 may render graphical features corresponding to specific objects in the virtual environment (e.g. one or more player controller characters), and / or render graphical features within a predetermined threshold distance (in the virtual environment) from a virtual camera associated with the content (i.e. only render graphical features that are closer to the virtual camera); and not render any further graphical features to reduce processing resource usage. For example, when the user is playing a role playing game and the hold detection unit 320 determines that the user is not holding the device, the modification unit 330 may render only graphical features corresponding to the player-controller character, and not render other features relating to the scenery or other non-player controlled characters. In some cases, the predetermined threshold distance may be dependent on the period of time during which the controller is not held - for example, the distance decreasing (e.g. proportionally) as the period of time increases. It will be also appreciated that rendering of displayed content typically comprises one or more rendering operations in a graphics processing pipeline, such as one or more geometry operations (e.g. scene mesh generation, lighting, or clipping), shading operations (e.g. pixel shading), and / or rasterization operations. Considering modifying one or more rendering operations, this may comprise modifying at least one of these rendering operations based on whether the device is determined as being held by the user. For example, the modification unit 330 may reduce (i.e. limit extent of), or omit, the at least one rendering operation in dependence upon the hold detection unit 320 determining that the peripheral device is not held by the user. The one or more reduced, or omitted, rendering operations may comprise one or more selected from the list consisting of: shading, lighting, culling, and rasterization. For example, if the device is determined as not being held by the user, the modification unit 330 may reduce shading and lighting to reduce processing resource usage - e.g. the modification unit may perform pixel shading for only one in every four pixels, use fewer virtual light sources, and / or use a simpler polygonal shading model such as flat shading instead of a more complex model such as Gouraud shading. Reducing, or omitting, shading allows particularly reducing processing resource usage as shading is typically computationally-intensive. In some cases, modifying (rendering) quality of the displayed content, modifying the rendered content, and / or modifying one or more rendering operations as described elsewhere herein may be performed for only a subset of graphical features in the virtual environment associated with the displayed content. For example, the modification unit 330 may modify the rendering quality (e.g. reduce resolution) and / or rendering (e.g. reduce lighting operations) for only a subset of graphical features in the virtual environment, in order to reduce computational cost while minimising the subjective reduction in the appearance of the rendered content to the user and thus minimising the reduction in user experience. The modification unit 330 may select the subset of graphical features for which rendering is modified based on several factors, such as: a position of the graphical feature within the displayed content (e.g. whether it's positioned centrally or near the perimeter of the content), an object (e.g. the type and / or properties of the object) associated with the graphical feature (e.g. objects corresponding to a player-controlled character vs scenery element; or stationary vs moving objects), and / or a threshold distance in the virtual environment between each graphical feature and a virtual camera associated with the displayed content. The modification unit 330 may determine this subset of graphical features based on a predetermined mapping between features for which rendering is to be modified and the above-mentioned factors. For example, upon determining that the user is not holding the device, the modification unit 330 may reduce the rendering quality, one or more rendering operations, or not render at all, a subset of the graphical features in the virtual environment which correspond to scenery element objects and / or which are beyond a threshold distance from the virtual camera in the virtual environment. Considering modifying an allocation of processing resources to rendering the displayed content, the modification unit 330 may modify this allocation depending on whether the peripheral device is determined as being held by the user. Upon the hold detection unit 320 determining that the device is not being held by the user, the modification unit 330 may reduce an allocation of processing resources (e.g. of the CPU 20 of the entertainment device 10) to rendering the displayed content. For example, the modification unit 330 may reduce the allocation of the processing resources to generating the content (e.g. rendering frames of gameplay in a videogame) from 70% of the resources to 50%. In this way, the processing resource and power usage associated with displaying the content are reduced. In some cases, the modification unit 330 may reallocate at least part of the processing resources from rendering the content to one or more other processes. This allows these processing resources to be used for other processes when the user is not holding the peripheral device, thus allowing more efficient resource usage. For instance, the modification unit 330 may reallocate the processing resources to one or more other processes running on the device that renders the content (e.g. the entertainment device 10). For example, when a user is playing a first game and the hold detection unit 320 determines that the user is not holding the peripheral device, the modification unit 330 may reallocate processing resources to downloading or installing a different, second game. Alternatively, or in addition, the modification unit 330 may reallocate the processing resources to one or more other devices, such as other devices on the same network. For example, upon determining that the user is not holding the peripheral device, the modification unit 330 may make at least part (e.g. 30%) of the processing resources available to other devices on its local Wi-Fi network. This allows efficiently sharing processing resources between devices to ensure that the devices that require the most processing power for their current task have the requisite processing resources available. Advantageously, the various ways of modifying the rendering of the content may be combined to further reduce processing resource and power usage. For example, the quality of the displayed content may be reduced and simultaneously, or shortly after, processing resources may be reallocated from rendering the content to other processes. Combining the modifications in this way allows minimising performance reduction (e.g. avoiding lag of the displayed content) as the amount of processing required and the processing resources are reduced at the same time. In some cases, the modification unit 330 may modify rendering of the displayed content in dependence upon the hold detection unit 320 determining that the peripheral device has not been held by the user for a predetermined period of time. In other words, the modification unit 330 may only modify the rendering after the peripheral device has been determined as not being held for the predetermined period of time (e.g. 10 or 20 seconds, or 2 minutes). This allows avoiding prematurely modifying the rendering upon the user briefly letting go of the device. In some cases, the modification unit 330 may modify rendering of the displayed content to a degree depending on a period of time during which the peripheral device is determined as not being held. For example, the modification unit 330 may reduce rendering quality of the displayed content to a degree depending on the period of time during which the peripheral device is determined as not being held. For instance, the modification unit 330 may progressively reduce more parameters (such as frame rate or resolution), and / or reduce the parameters to a greater degree, as this period of time increases. For example, the modification unit 330 may reduce the resolution of the content by 25% after a first predetermined period of time (during which the peripheral device is not held) such as 5 seconds, reduce the frame rate by 50% after a second, greater period of time (e.g. 30 seconds), and reduce the resolution by a further 50% after a third, yet greater period of time (e.g. 2 minutes). Similarly, the modification unit 330 may reduce the allocation of processing resources to rendering the content, and / or increase the reallocation of these resources, to a progressively greater degree as the time period during which the peripheral device is determined as not being held increases. For example, the modification unit 330 may reduce the allocation of processing resources to rendering the content from 70% to 50% after 20 seconds and to 30% after a further 2 minutes (during which the peripheral device is not held). In some cases, the modification unit 330 may modify a first aspect of the rendering in dependence upon the hold detection unit 320 determining that the peripheral device has not been held by the user for a first predetermined period of time (e.g. modify rendering quality after the device is not held for 10 seconds), and to modify a second aspect of the rendering in dependence upon the hold detection unit determining that the peripheral device has not been held by the user for a second predetermined period of time (e.g. modify rendering operation after the device is not held for 1 minute). This may be extended to further aspects and further predetermined periods of time. The predetermined period(s) of time may be empirically determined, for example based on testing with a plurality of test users. It will be appreciated that, whilst the description primarily refers to the modification unit 330 modifying rendering of the displayed content (e.g. reducing rendering quality etc.) upon determining that the user is not holding the peripheral device, the modification unit 330 may likewise modify said rendering (e.g. increase rendering quality) upon the hold detection unit 320 determining that the device is held by the user. For example, after reducing the resolution of the display content as a result of determining that the peripheral device is not held, the modification unit 330 may increase the resolution back to its original value upon the hold detection unit 320 determining that the device is held by the user. In some cases, alternatively, or in addition, to modifying rendering of the displayed content, the modification unit 330 may interrupt display of the content. For example, the modification unit 330 may pause the displayed content, and / or close an application associated with displaying the content (e.g. the game currently being played by the user). This allows further reducing the amount of processing needed to display the content, as new content need not be generated (e.g. the next frames need not be rendered). The modification unit 330 may also save the user's current progress in the application (e.g. in the game being played). In cases where the user is interacting with content using the HMD 120, and controllers 130A-L,R, interrupting display of the content may comprise, upon determining that that one or both of the controllers 130A-L,R are not held, modifying a display mode of the HMD 120 to a "see-through" mode (i.e. a mode in which the user is able to see the surrounding environment) to allow the user to find and pick up the controller 130A-L,R easily even if they are still wearing the HMD headset 120. At step 434, the modification unit 330 modifies one or more aspects of the peripheral device. For example, upon the hold detection unit 320 determining that the device is not held by the user, the modification unit 330 may reduce on or more aspects of the device. This may include at least partially deactivating one or more input and / or output elements of the peripheral device, and / or discarding one or more inputs received from the peripheral device. Considering deactivating input / output elements of the peripheral device, the modification unit 330 may at least partially deactivate one or more input elements and / or one or more output elements of the peripheral device. Deactivating an element may comprise fully deactivating it - e.g. switching it off, or partially deactivating the element - e.g. reducing its power output (e.g. the output volume of a loudspeaker output element) and / or reducing its sensitivity to user input (e.g. reducing the sensitivity of a microphone input element). The specific input and / or output elements that are deactivated may depend on the particular peripheral device being used. For example, one or more of a microphone, loudspeaker, lighting features, display (i.e. screen), and / or a camera on the peripheral device may be at least partially deactivated as part of step 434. For instance, the microphone may be switched off, and any lighting features may be dimmed. This allows both reducing the power usage of the peripheral device and limiting the interaction between the peripheral device and displayed content. Further, this helps prevent accidental user inputs when the user is not holding the peripheral device (e.g. inputs resulting from the user sitting on the peripheral device). In addition, in some instances, this may allow improving the privacy of the user, as input elements (e.g. a microphone on the peripheral device) are automatically deactivated when the user is not holding the device (which indicates that the user does not wish to interact with the displayed content). Considering discarding user inputs, the modification unit 330 may discard one or more user inputs upon the hold detection unit 320 determining that the user is not holding the peripheral device. This allows limiting the interaction between the peripheral device and the displayed content when the user is not holding the peripheral device, and reducing the possibility of accidental user inputs from affecting the displayed content. Like deactivating the input elements, it may also improve privacy of the user as discarding user inputs (e.g. inputs provided via a microphone on the peripheral device) prevents further processing of these inputs and protects the privacy of the user. A user input may be discarded further in dependence on the content displayed at the time the input was received / determined and / or the input element (e.g. button on the controller 80) used to provide the input, as described elsewhere herein. Similarly to step 432, the extent of deactivation of input / output elements and / or discarding of the user inputs may progressively increase as the time period during which the peripheral device is determined as not being held increases. For example, the modification unit 330 may discard user inputs after 5 seconds, but only deactivate a microphone on the peripheral device after a further 20 seconds (during which the peripheral device is not held). Figure 6 shows a further example of a method of controlling interaction between displayed content and a peripheral device using the system of Figure 3. In particular, Figure 6 shows how different aspects of the interaction may be reduced after different predetermined periods of time during which the controller is determined as not being held. At step 610, the hold detection unit 320 starts determining whether the peripheral is held by the user. For example, the hold detection unit 330 may initiate operation of its one or more sensors. At step 620, the modification unit 330 determines whether the hold detection unit 320 has determined the device as not held for a time period greater than a first predetermined period of time (tl), such as 3, 5 or 10 seconds. If this condition is met (i.e. if this is determined to be true), the method proceeds to step 630. At step 630, the modification unit 330 reduces the rendering quality of the displayed content. This allows reducing the amount of processing needed to display the content. For example, the modification unit 330 may reduce the frame rate and / or resolution of the displayed content by 50%. At step 640, the modification unit 330 determines whether the hold detection unit 320 has determined the device as not held for a time period greater than a second, longer, predetermined period of time (t2), such as 30 seconds or 1 minute. If this condition is met (i.e. if this is determined to be true), the method proceeds to step 630. At step 650, the modification unit 330 deactivates one or more input / output (I / O) elements of the peripheral device. For example, the modification unit 330 may deactivate a microphone and a loudspeaker on the peripheral device. At step 660, the modification unit 330 determines whether the hold detection unit 320 has determined the device as not held for a time period greater than a third, yet longer, predetermined period of time (t3), such as 5 minutes. If this condition is met (i.e. if this is determined to be true), the method proceeds to step 630. At step 670, the modification unit 330 reallocates processing resources from rendering the content for display to other processes, for example the installation of a different application. It will be appreciated that at any of steps 620, 640, and 660, if the hold detection unit 320 determines that the device is held by the user, the method returns to step 610. Likewise, if at any of steps 620, 640, and 660, the device is determined as not being held for a period of time smaller or equal to the predetermined periods tl, t2, t3, the method remains at that step until the period of time exceeds the relevant threshold. Referring back to Figure 3, in a summary embodiment of the present invention, a system for controlling the rendering of content, such as content presented to a user via a display, may comprise the following: A hold detection unit 320 configured (for example by suitable software instruction) to determine whether a peripheral device is held by a user in dependence on outputs by one or more sensors, the peripheral device being operable by the user to interact with the content. And a modification unit 330 configured (for example by suitable software instruction) to modify rendering of the content in dependence on whether the peripheral device is held by the user, as described elsewhere herein. Of course, the functionality of these units may be realised by any suitable number of processors located at any suitable number of devices and any suitable number of devices as appropriate rather than requiring a one-to-one mapping between the functionality and a device or processor. It will be apparent to a person skilled in the art that variations in the above system corresponding to operation of the various embodiments of the method and / or system as described and claimed herein are considered within the scope of the present disclosure, including but not limited to that: the modification unit 330 is configured to modify the quality of the content in dependence on whether the peripheral device is held by the user, as described elsewhere herein; in this case, optionally, the quality of the content includes one or more selected from the list consisting of: a frame rate of the content, a graphics quality of the content, and a size of the content, as described elsewhere herein; rendering of the content comprises one or more rendering operations, and the modification unit 330 is configured to modify at least one of the rendering operations in dependence on whether the peripheral device is held by the user, as described elsewhere herein; in this case, optionally, the modification unit 330 is configured to reduce, or omit, the at least one rendering operation in dependence upon the hold detection unit 320 determining that the peripheral device is not held by the user, as described elsewhere herein; in this case, optionally, the at least one rendering operation comprises one or more selected from the list consisting of: shading, lighting, culling, and rasterization, as described elsewhere herein; the content comprises a plurality of graphical features of a virtual environment, and the modification unit 330 is configured to render a subset of the graphical features upon the hold detection unit 320 determining that the peripheral device is not held by the user, as described elsewhere herein; in this case, optionally, the modification unit 330 is configured to select the subset of graphical features based on one or more of selected from the list consisting of: a position of the each graphical feature in the content, an object associated with each graphical feature, and a threshold distance in the virtual environment between each graphical feature and a virtual camera associated with the content, as described elsewhere herein; the modification unit 330 is configured to modify rendering of the content to a degree depending on a period of time during which the peripheral device is determined as not being held by the user, as described elsewhere herein; the modification unit 330 is configured to modify rendering of the content in dependence upon the hold detection unit 320 determining that the peripheral device has not been held by the user fora predetermined period of time, as described elsewhere herein; the modification unit 330 is configured to modify a first aspect of the rendering in dependence upon the hold detection unit 320 determining that the peripheral device has not been held by the user for a first predetermined period of time, and to modify a second aspect of the rendering in dependence upon the hold detection unit 320 determining that the peripheral device has not been held by the user for a second predetermined period of time, as described elsewhere herein; the modification unit 330 is configured to modify rendering of the content further in dependence on the current content, as described elsewhere herein; the modification unit 330 is configured to modify an allocation of processing resources to rendering the content in dependence on whether the peripheral device is held by the user, as described elsewhere herein; in this case, optionally, the modification unit 330 is configured to reallocate at least part of the processing resources from rendering the content to one or more other processes in dependence upon the hold detection unit 320 determining that the peripheral device is not held by the user, as described elsewhere herein; the one or more sensors comprise a motion sensor for detecting motion of the peripheral device, as described elsewhere herein; in this case, optionally, the hold detection unit 320 is configured to detect whether the peripheral device is held by the user based on whether the detected motion meets one or more predetermined conditions, as described elsewhere herein; the one or more sensors comprise one or more touch sensors for detecting whether a user is touching the peripheral device, and / or one or more cameras for capturing one or more images of the peripheral device and / or the user, as described elsewhere herein; the hold detection unit 320 is configured to determine whether the peripheral device is held by the user further in dependence on the current (that is, currently displayed) content, as described elsewhere herein; the system comprises the one or more sensors, as described elsewhere herein; the one or more sensors are configured to obtain (i.e. capture) data relating to usage of the peripheral device (by the user), as described elsewhere herein; the one or more sensors are associated with the peripheral device, as described elsewhere herein; in this case, optionally, the peripheral device comprises the one or more sensors, as described elsewhere herein; the hold detection unit 320 comprises the one or more sensors, as described elsewhere herein; the peripheral device is a games controller, as described elsewhere herein; the content is part of a videogame, as described elsewhere herein; the content is displayed using the peripheral user device, as described elsewhere herein; and the content is virtual reality (VR) content, as described elsewhere herein. In a further summary embodiment of the present invention, a system for controlling interaction between displayed content and a peripheral device operated by a user may comprise the following: a hold detection unit 320 comprising one or more sensors for detecting whether the peripheral device is held by the user, as described elsewhere herein; and a modification unit 330 configured to modify (preferably reduce) one or more aspects of the interaction in dependence upon the hold detection unit detecting that the peripheral device is not held by the user, as described elsewhere herein. It will be apparent to a person skilled in the art that variations in the above system corresponding to operation of the various embodiments of the method and / or system as described and claimed herein are considered within the scope of the present disclosure, including but not limited to that: the modification unit is configured to reduce one or more aspects of one or more selected from the list consisting of: the peripheral device, and the displayed content, as described elsewhere herein; the modification unit is configured to reduce quality of the displayed content in dependence upon the hold detection unit detecting that the peripheral device is not held by the user in this case, optionally, the modification unit is configured to reduce one or more selected from the list consisting of: a frame rate of the displayed content, and a graphics quality of the displayed content, and a size of the displayed content, as described elsewhere herein; in this case, optionally, the modification unit is configured to reduce quality of the displayed content in dependence on a period of time during which the peripheral device is detected as not held, as described elsewhere herein; the modification unit is configured to pause the displayed content in dependence upon the hold detection unit detecting that the peripheral device is not held by the user, as described elsewhere herein; the modification unit is configured to reduce one or more aspects of the interaction in dependence upon the hold detection unit detecting that the peripheral device has not been held by the user for a predetermined period of time, as described elsewhere herein; in this case, optionally, the modification unit is configured to reduce a first aspect of the interaction in dependence upon the hold detection unit detecting that the peripheral device has not been held by the user for a first predetermined period of time, and to reduce a second aspect of the interaction in dependence upon the hold detection unit detecting that the peripheral device has not been held by the user for a second predetermined period of time, as described elsewhere herein; the modification unit is configured to reduce one or more aspects of the interaction further in dependence upon on the current displayed content, as described elsewhere herein; the modification unit is configured to reduce an allocation of processing resources to generating (e.g. processing and / or rendering) the content for display in dependence upon the hold detection unit detecting that the peripheral device is not held by the user, as described elsewhere herein; in this case, optionally, the modification unit is configured to reallocate at least part of processing resources from generating the content to one or more other processes, as described elsewhere herein; the modification unit is configured to at least partially deactivate one or more input / output elements of the peripheral device in dependence upon the hold detection unit detecting that the peripheral device is not held by the user, as described elsewhere herein; in this case, optionally, the one or more input / output elements comprise one or more selected from the list consisting of: a microphone, loudspeaker, lighting features, screen, and camera, as described elsewhere herein; and the modification unit is configured to discard one or more inputs received from the peripheral device in dependence upon the hold detection unit detecting that the peripheral device is not held by the user, as described elsewhere herein. Referring to Figure 4, in another summary embodiment of the present invention a method for controlling the rendering of content comprises the following steps. A step 420 comprises determining, in dependence on outputs by one or more sensors, whether a peripheral device is held by a user, the peripheral device being operable by the user to interact with the content, as described elsewhere herein. A step 430 comprises modifying rendering of the content in dependence on whether the peripheral device is held by the user, as described elsewhere herein. It will be appreciated that the above methods may be carried out on conventional hardware suitably adapted as applicable by software instruction or by the inclusion or substitution of dedicated hardware. Thus the required adaptation to existing parts of a conventional equivalent device may be implemented in the form of a computer program product comprising processor implementable instructions stored on a non-transitory machine-readable medium such as a floppy disk, optical disk, hard disk, solid state disk, PROM, RAM, flash memory or any combination of these or other storage media, or realised in hardware as an ASIC (application specific integrated circuit) or an FPGA (field programmable gate array) or other configurable circuit suitable to use in adapting the conventional equivalent device. Separately, such a computer program may be transmitted via data signals on a network such as an Ethernet, a wireless network, the Internet, or any combination of these or other networks. The foregoing discussion discloses and describes merely exemplary embodiments of the present invention. As will be understood by those skilled in the art, the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Accordingly, the disclosure of the present invention is intended to be illustrative, but not limiting of the scope of the invention, as well as other claims. The disclosure, including any readily discernible variants of the teachings herein, defines, in part, the scope of the foregoing claim terminology such that no inventive subject matter is dedicated to the public.
Claims
1. A system for controlling the rendering of content, the system comprising:a hold detection unit configured to determine whether a peripheral device is held by a user in dependence on outputs by one or more sensors, the peripheral device being operable by the user to interact with the content; anda modification unit configured to modify rendering of the content in dependence on whether the peripheral device is held by the user.
2. The system of claim 1, wherein the modification unit is configured to modify the quality of the content in dependence on whether the peripheral device is held by the user; wherein the quality of the content includes one or more selected from the list consisting of: a frame rate of the content, a graphics quality of the content, and a size of the content.
3. The system of any preceding claim, wherein rendering of the content comprises one or more rendering operations, and wherein the modification unit is configured to modify at least one of the rendering operations in dependence on whether the peripheral device is held by the user.
4. The system of claim 3, wherein the modification unit is configured to reduce, or omit, the at least one rendering operation in dependence upon the hold detection unit determining that the peripheral device is not held by the user.
5. The system of any preceding claim, wherein the content comprises a plurality of graphical features of a virtual environment, and wherein the modification unit is configured to render a subset of the graphical features upon the hold detection unit determining that the peripheral device is not held by the user; and wherein the modification unit is configured to select the subset of graphical features based on one or more of selected from the list consisting of: a position of the each graphical feature in the content, an object associated with each graphical feature, and a threshold distance in the virtual environment between each graphical feature and a virtual camera associated with the content.
6. The system of any preceding claim, wherein the modification unit is configured to modify rendering of the content to a degree depending on a period of time during which the peripheral device is determined as not being held by the user.
7. The system of any preceding claim, wherein the modification unit is configured to modify an allocation of processing resources to rendering the content in dependence on whether the peripheral device is held by the user.
8. The system of claim 7, wherein the modification unit is configured to reallocate at least part of the processing resources from rendering the content to one or more other processes in dependence upon the hold detection unit determining that the peripheral device is not held by the user.
9. The system of any preceding claim, wherein the one or more sensors comprise a motion sensor for detecting motion of the peripheral device.
10. The system of claim 9, wherein the hold detection unit is configured to detect whether the peripheral device is held by the user based on whether the detected motion meets one or more predetermined conditions.
11. The system of any preceding claim, wherein the one or more sensors comprise one or more touch sensors for detecting whether a user is touching the peripheral device, and / or one or more cameras for capturing one or more images of the peripheral device and / or the user.
12. The system of any preceding claim, wherein the hold detection unit is configured to determine whether the peripheral device is held by the user further in dependence on the current content.
13. The system of any preceding claim, wherein the wherein the peripheral device is a games controller; and wherein the content is part of a videogame.
14. A method for controlling the rendering of content device operable by a user, the method comprising: determining, in dependence on outputs by one or more sensors, whether a peripheral device is held by a user, the peripheral device being operable by the user to interact with the content; and modifying rendering of the content in dependence on whether the peripheral device is held by the user.
15. A computer program comprising computer executable instructions adapted to cause a computer system to perform the method of claim 14.
Citation Information
Patent Citations
ELECTRONIC Blackboard, VIDEO DISPLAY METHOD, AND RECORDING MEDIUM
CN110278403A
Electronic apparatus comprising a touch screen and method of controlling the same
EP2770418A1
Data processing apparatus, control device and method
GB2588807A
Image display apparatus and operating method for the same
KR102315923B1
Ultrasound adaptive power management systems and methods
US20170303899A1
