Media management based on user presence

Abstract

An information handling system can detect media playback, collect data from a user presence device, and determine a user presence relative to the information handling system based on the data from the user presence device. The system can also apply a policy based on the user presence by adjusting a setting associated with the media playback.

Description

Technical Field

[0001] This disclosure generally relates to information processing systems, and more specifically to media management based on user presence. Background Technology

[0002] As the value and use of information continue to grow, individuals and businesses seek additional ways to process and store information. One option is an information processing system. Information processing systems typically process, compile, store, or transmit information or data for business, personal, or other purposes. The technology and information processing needs and requirements may vary between different applications. Therefore, information processing systems may also differ in terms of what information is processed, how it is processed, how much information is processed, stored, or transmitted, and how quickly and efficiently it can be processed, stored, or transmitted. Variations in information processing systems allow them to be general-purpose or configured for specific users or purposes (such as financial transaction processing, airline ticketing, enterprise data storage, or global communications). Furthermore, information processing systems can include a variety of hardware and software resources that can be configured to process, store, and transmit information, and may include one or more computer systems, graphical interface systems, data storage systems, networking systems, and mobile communication systems. Information processing systems can also implement various virtualization architectures. Data and voice communications in information processing systems can be conducted via networks, which can be wired, wireless, or a combination thereof. Summary of the Invention

[0003] An information processing system can detect media playback, collect data from a user presence device, and determine the user's presence relative to the information processing system based on the data from the user presence device. The system can also apply a user presence-based strategy by adjusting settings associated with media playback. Attached Figure Description

[0004] It should be understood that, for the sake of simplicity and clarity, the components shown in the figures are not necessarily drawn to scale. For example, the dimensions of some components may be enlarged relative to other components. Regarding the embodiments shown and described herein in conjunction with the teachings of this disclosure, in the figures:

[0005] Figure 1 This is a block diagram illustrating an information processing system according to an embodiment of the present disclosure;

[0006] Figure 2 This is a block diagram illustrating an example of a system for user-presence-based media management according to an embodiment of this disclosure;

[0007] Figure 3 This is a sequence diagram for user-based media management according to an embodiment of this disclosure;

[0008] Figure 4 This is a diagram illustrating the environment of an information processing system for user-presence-based media management according to an embodiment of this disclosure;

[0009] Figure 5 This is a diagram illustrating the environment of an information processing system for user-presence-based media management according to an embodiment of this disclosure;

[0010] Figure 6 This is a flowchart illustrating a method for user-based media management according to an embodiment of this disclosure; and

[0011] Figure 7 This is a diagram illustrating a graphical user interface for user-based media management according to an embodiment of this disclosure.

[0012] Using the same reference numerals in different figures indicates similar or identical items. Detailed Implementation

[0013] The following description, taken in conjunction with the accompanying drawings, is provided to aid in understanding the teachings disclosed herein. The description focuses on specific implementations and embodiments of the teachings and is provided to help describe these teachings. This description should not be construed as limiting the scope or applicability of these teachings.

[0014] Figure 1An embodiment of an information processing system 100 is illustrated, comprising processors 102 and 104, a chipset 110, memory 120, a graphics adapter 130 connected to a video display 134, non-volatile RAM (NV-RAM) 140 of basic memory including an input / output system / expandable firmware interface (BIOS / EFI) module 142, a disk controller 150, a hard disk drive (HDD) 154, an optical disk drive 156, a disk emulator 160 connected to a solid-state drive (SSD) 164, an input / output (I / O) interface 170 connected to an expansion resource 174 and a trusted platform module (TPM) 176, a network interface 180, and a baseboard management controller (BMC) 190. Processor 102 is connected to chipset 110 via processor interface 106, while processor 104 is connected to chipset 110 via processor interface 108. In certain embodiments, processors 102 and 104 are connected together via high-capacity coherent structures such as HyperTransport links or QuickPath interconnects. Chipset 110 represents an integrated circuit or a group of integrated circuits that manage the data flow between processors 102 and 104 and other components of the information processing system 100. In a particular embodiment, chipset 110 represents a pair of integrated circuits, such as a northbridge component and a southbridge component. In another embodiment, some or all of the functions and features of chipset 110 are integrated with one or more of processors 102 and 104.

[0015] Memory 120 is connected to chipset 110 via memory interface 122. Examples of memory interface 122 include Double Data Rate (DDR) memory channels, and memory 120 represents one or more DDR dual in-line memory modules (DIMMs). In a particular embodiment, memory interface 122 represents two or more DDR channels. In another embodiment, one or more of processors 102 and 104 include a memory interface that provides dedicated memory for the processor. DDR channels and connected DDR DIMMs may conform to specific DDR standards, such as DDR3, DDR4, DDR5, etc.

[0016] Memory 120 may also represent various combinations of memory types, such as Dynamic Random Access Memory (DRAM) DIMMs, Static Random Access Memory (SRAM) DIMMs, Non-Volatile DIMMs (NV-DIMMs), Storage Class Memory Devices, Read-Only Memory (ROM) Devices, etc. Graphics adapter 130 is connected to chipset 110 via graphics interface 132 and provides video display output 136 to video display 134. Examples of graphics interface 132, as needed or desired, include a Peripheral Component Interconnect High Speed ​​(PCIe) interface, and graphics adapter 130 may include a quad-channel (x4) PCIe adapter, an eight-channel (x8) PCIe adapter, a 16-channel (x16) PCIe adapter, or another configuration. In a particular embodiment, graphics adapter 130 is disposed downwards on the system printed circuit board (PCB). Video display output 136 may include a Digital Video Interface (DVI), High Definition Multimedia Interface (HDMI), DisplayPort interface, etc., and video display 134 may include a monitor, smart TV, embedded display such as a laptop computer display, etc.

[0017] NV-RAM 140, disk controller 150, and I / O interface 170 are connected to chipset 110 via I / O channel 112. Examples of I / O channel 112 include one or more point-to-point PCIe links between chipset 110 and each of NV-RAM 140, disk controller 150, and I / O interface 170. Chipset 110 may also include one or more other I / O interfaces, including PCIe interfaces, Industry Standard Architecture (ISA) interfaces, Small Computer Serial Interface (SCSI) interfaces, and Inter-Integrated Circuit (I / O) interfaces. 2 C) Interfaces such as System Packet Interface (SPI), Universal Serial Bus (USB), another interface, or a combination thereof. NV-RAM 140 includes a BIOS / EFI module 142 that stores machine-executable code (BIOS / EFI code) that operates to detect resources of the information processing system 100, provides drivers for the resources, initializes the resources, and provides common access mechanisms for the resources. The functionality and features of the BIOS / EFI module 142 will be further described below.

[0018] Disk controller 150 includes a disk interface 152 that connects the disk controller to a hard disk drive (HDD) 154, an optical disk drive (ODD) 156, and a disk emulator 160. Examples of disk interfaces 152 include an Integrated Drive Electronics (IDE) interface, an Advanced Technology Attachment (ATA) interface such as a Parallel ATA (PATA) interface or a Serial ATA (SATA) interface, a SCSI interface, a USB interface, a proprietary interface, or a combination thereof. Disk emulator 160 allows SSD 164 to be connected to information processing system 100 via external interface 162. Examples of external interface 162 include a USB interface, an Institute of Electrical and Electronics Engineers (IEEE) 1394 (FireWire) interface, a proprietary interface, or a combination thereof. Alternatively, SSD 164 may be located within information processing system 100.

[0019] I / O interface 170 includes peripheral interface 172, which connects the I / O interface to expansion resource 174, TPM 176, and network interface 180. Peripheral interface 172 can be the same type of interface as I / O channel 112, or it can be a different type of interface. Thus, when peripheral interface 172 and I / O channel 112 are of the same type, I / O interface 170 expands the capability of the I / O channel; when they are of different types, the I / O interface converts information from a format suitable for the I / O channel to a format suitable for peripheral interface 172. Expansion resource 174 may include a data storage system, an additional graphics interface, a network interface card (NIC), a voice / video processing card, another expansion resource, or a combination thereof. Expansion resource 174 may be located on a main circuit board, on a separate circuit board or expansion card disposed within the information processing system 100, on a device external to the information processing system, or a combination thereof.

[0020] Network interface 180 represents a network communication device that is disposed within information processing system 100, located on the main circuit board of information processing system, integrated into another component such as chipset 110, located in another suitable location, or a combination thereof. Network interface 180 includes network channel 182, which provides an interface to devices external to information processing system 100. In a particular embodiment, network channel 182 is of a different type from peripheral interface 172, and network interface 180 converts information from a format suitable for the peripheral channel to a format suitable for the external device.

[0021] In a particular implementation, network interface 180 includes a NIC or host bus adapter (HBA), and examples of network channel 182 include an InfiniBand channel, Fibre Channel, Gigabit Ethernet channel, proprietary channel architecture, or a combination thereof. In another implementation, network interface 180 includes a wireless communication interface, and network channel 182 includes a Wi-Fi channel, a near field communication (NFC) channel, etc. Alternatively, it may use a Bluetooth Low Energy (BLE) channel, a cellular interface such as the Global System for Mobile Communications (GSM) interface, a Code Division Multiple Access (CDMA) interface, a Universal Mobile Telecommunications System (UMTS) interface, a Long Term Evolution (LTE) interface, or another cellular interface, or a combination thereof. Network channel 182 can connect to external network resources (not shown). Network resources may include another information processing system, a data storage system, another network, a grid management system, another suitable resource, or a combination thereof.

[0022] BMC 190 connects to multiple components of information processing system 100 via one or more management interfaces 192 to provide out-of-band monitoring, maintenance, and control of the components of information processing system. Thus, BMC 190 represents a processing device distinct from processors 102 and 104, which provides various management functions for information processing system 100. For example, BMC 190 may be responsible for power management, cooling management, etc. The term BMC is commonly used in the context of server systems, while in consumer devices, BMC can be referred to as an embedded controller (EC). A BMC included in a data storage system can be referred to as a storage enclosure processor. A BMC included in a blade server chassis can be referred to as a chassis management controller, while an embedded controller included in a blade of a blade server can be referred to as a blade management controller. The capabilities and functions provided by BMC 190 can vary considerably depending on the type of information processing system. BMC 190 can operate according to an Intelligent Platform Management Interface (IPMI). Examples of BMC 190 include integrated... Remote Access Controller (iDRAC).

[0023] Management interface 192 represents one or more out-of-band communication interfaces between BMC 190 and components of information processing system 100, and may include inter-integrated circuit (I2C) bus, system management bus (SMBUS), power management bus (PMBUS), low pin count (LPC) interface, serial bus such as Universal Serial Bus (USB) or Serial Peripheral Interface (SPI), network interface such as Ethernet interface, high-speed serial data link such as PCIe interface, network controller sideband interface (NC-SI), etc. As used herein, out-of-band access refers to operations performed on information processing system 100 separately from the BIOS / operating system execution environment (i.e., separately from the execution of code by processors 102 and 104 and the programs implemented on the information processing system in response to the executed code).

[0024] As needed or desired, the BMC 190 operates to monitor and maintain system firmware, such as code stored in the BIOS / EFI module 142, an option ROM for the graphics adapter 130, the disk controller 150, expansion resources 174, a network interface 180, or other components of the information processing system 100. Specifically, the BMC 190 includes a network interface 194, which can be connected to a remote management system as needed or desired to receive firmware updates. Here, the BMC 190 receives firmware updates, stores the updates in a data storage device associated with the BMC, transfers the firmware updates to the NV-RAM of the device or system to which the firmware updates are intended, thereby replacing the firmware of the currently operating device or system and restarting the information processing system, which then utilizes the updated firmware image.

[0025] The BMC 190 utilizes various protocols and application programming interfaces (APIs) to guide and control the processes used for monitoring and maintaining system firmware. Examples of protocols or APIs used for monitoring and maintaining system firmware, depending on need or expectation, include the graphical user interface (GUI) associated with the BMC 190, interfaces defined by the Distributed Management Task Force (DMTF) such as the Web Services Management (WSMan) interface, the Management Parts Transport Protocol (MCTP), or... Interfaces), various vendor-defined interfaces (such as the Dell EMC Remote Access Controller Manager (RACADM) utility, Dell EMC OpenManage Enterprise, Dell EMC OpenManage Server Manager (OMSS) utility, Dell EMC OpenManage Storage Service (OMSS) utility, or Dell EMC OpenManage Deployment Kit (DTK) suite), and other protocols or APIs such as BIOS setup utilities called by the “F2” boot option.

[0026] In certain implementations, the BMC 190 may be included on the main circuit board (such as a substrate, motherboard, or any combination thereof) of the information processing system 100, or integrated into another component (such as chipset 110) or another suitable component of the information processing system, as needed or desired. Thus, the BMC 190 may be part of an integrated circuit or chipset within the information processing system 100. Examples of the BMC 190 include iDRAC, etc. The BMC 190 may operate on a power plane separate from other resources in the information processing system 100. Therefore, the BMC 190 may communicate with the management system via network interface 194 while the resources of the information processing system 100 are powered down. Here, information may be sent from the management system to the BMC 190, and said information may be stored in RAM or NV-RAM associated with the BMC. Information stored in RAM may be lost after a power loss of the BMC 190's power plane, while information stored in NV-RAM can be preserved through power-down / power-up cycles of the BMC's power plane.

[0027] Information processing system 100 may include additional components and additional buses, not shown for clarity. For example, information processing system 100 may include multiple processor cores, audio devices, etc. Although specific arrangements of bus technologies and interconnections are shown for illustrative purposes, those skilled in the art will understand that the technologies disclosed herein are applicable to other system architectures. Information processing system 100 may include multiple central processing units (CPUs) and redundant bus controllers. One or more components may be integrated together. Information processing system 100 may include additional buses and bus protocols, such as I2C, etc. Additional components of information processing system 100 may include one or more storage devices that can store machine-executable code, one or more communication ports for communicating with external devices, and various input and output (I / O) devices (such as a keyboard, mouse, and video display).

[0028] For the purposes of this disclosure, the information processing system 100 may include any tool or set of tools operable to calculate, classify, process, transmit, receive, retrieve, generate, switch, store, display, indicate, detect, record, reproduce, dispose of, or utilize information, intelligence, or data of any form for commercial, scientific, control, entertainment, or other purposes. For example, the information processing system 100 may be a personal computer, laptop computer, smartphone, tablet device, or other consumer electronic device, network server, network storage device, switch, router, or other network communication device, or any other suitable device, and may vary in size, shape, performance, functionality, and price. Furthermore, the information processing system 100 may include processing resources for executing machine-executable code, such as processor 102, programmable logic array (PLA), embedded devices such as system-on-a-chip (SoC), or other control logic hardware. The information processing system 100 may also include one or more computer-readable media for storing machine-executable code such as software or data.

[0029] Previously, when a user was watching a media replay but temporarily looked away from the screen, they had to manually pause the replay by clicking a button or icon. When the user returned to the screen to continue watching, they might have to manually reset the video position back to where they were when they decided to leave the screen or look away. Gamers playing video games or employees using conference call applications often encounter similar situations. For example, a player might lose a video game while looking away or temporarily away because the game continues and the player hasn't paused it. Similarly, when an employee is working on another task or looking away from the screen, they must manually pause or mute the conference call application. In another example, after a certain point during a long media replay, the currently playing video might pause, prompting the user to ask if they still want to watch. The user must then interact with the media application by clicking a button or icon to resume playback.

[0030] In addition to these issues, the display device or screen of an information processing system often consumes a significant amount of power, resulting in unnecessary power consumption during media playback. Consequently, if media playback, video games, or teleconferences are not paused and the system is in power-saving mode, the battery of the mobile information processing system will be depleted more quickly. Therefore, it is advantageous for media playback to continue when a user is present but automatically pause when the user is absent. Therefore, this disclosure provides a system and method for automatically detecting the presence of a user and managing media, such as by adjusting media playback and power consumption accordingly.

[0031] Figure 2 A system 200 for user-presence-based media management is illustrated. System 200 includes an information processing system 100, a network 202, and a media streaming source 203. Information processing system 100 includes a media management service 205, a media playback source 250, a media application 255, a media content device 260, and a user presence device 280. Media management service 205 includes a near-field user awareness service 210, a far-field user awareness service 215, a background monitoring service 220, a video management service 225, and an operating system power management service 230. Components of system 200 may be implemented in hardware, software, firmware, or any combination thereof. The components shown are not drawn to scale, and system 200 may include more or fewer components. Additionally, connections between components may be omitted for clarity.

[0032] Media management service 205 can be configured to manage media based on information from a first set of services and one or more media management policies. Media management is performed to optimize user experience, improve user productivity, and optimize the power consumption of the information processing system 100. The first set of services includes near-field user awareness service 210, far-field user awareness service 215, background monitoring service 220, etc. Media management service 205 can determine one or more actions based on one or more media management policies, such as media management policy 235, and apply these actions using a second set of services (such as video management service 225, operating system power management service 230, video game control service, teleconferencing control service, teleconferencing service, etc.).

[0033] The actions to be applied may include adjusting one or more settings associated with the media currently playing or streaming. For example, settings associated with media content devices may be adjusted, such as increasing, decreasing, or muting the volume of speaker 270, dimming the display screen of display device 265, or muting microphone 275. These actions can be performed on various media playback sources and / or applications (such as video, music, video games, podcasts, teleconferencing, video calls, etc.). The actions may also include optimizing the power consumption of information processing system 100 by implementing energy-saving options (such as putting information processing system 100 into sleep or hibernation mode when a user leaves). Additionally, the actions may provide security measures, such as locking or dimming the display screen when another user is present and the currently displayed media content is considered confidential.

[0034] Data from one or more services in the first group can be used to determine user presence status, such as whether the user exists or does not exist relative to information processing system 100. Additionally, the data can be used to determine user presence status and / or user behavior regarding information processing system 100. User presence status can be based on the user's location within the information processing system, such as whether the user is in the near field or far field and / or whether there are multiple users in the near field or far field. User behavior can be based on individual actions performed by the user in response to media content, such as whether the user is looking at the display screen, whether the user is actively moving away from or towards information processing system 100, or whether the user is performing physical movements, such as gestures.

[0035] Media management policy 235 may include one or more rules to manage the settings, security, and / or power consumption of media application 255 based on one or more parameters (including user presence state and behavior) associated with the presence and / or absence of a user, and apply one or more actions accordingly. For example, when a user leaves the display device 265 or is no longer in front of the display device, media management policy 235 may dim or turn off the display device 265. In another example, when the absence of a user is detected, media management policy 235 may lock the information processing system 100 and guide it into a sleep mode or low-power state. Therefore, when the presence of a user is detected, media management policy 235 may wake the information processing system from sleep mode or low-power state and enable facial recognition.

[0036] The near-field user sensing service 210 can be configured to communicate with one or more user presence devices 280, such as a near-field sensor 290, and to assess user presence status, user presence state, and / or user behavior. For example, the near-field user sensing service 210 can collect and / or receive data outputs such as signals and / or sensor data from the near-field sensor 290. The near-field user sensing service 210 can process the data outputs, such as encoding, decoding, transmitting, filtering, and / or buffering all or part of the data, to determine whether a user is present in the near-field range and / or whether the user is looking at the display device 265 or making gestures towards the display device.

[0037] The far-field user sensing service 215 can be configured to communicate with one or more user presence devices, such as far-field sensor 292, and assess user presence status, user presence state, and / or user behavior. For example, the far-field user sensing service 215 can collect and / or receive data outputs such as signals and / or sensor data from the far-field sensor 292. The far-field user sensing service 215 can process the data outputs, such as encoding, decoding, transmitting, filtering, and / or buffering all or part of the data, to determine whether a user is within the far-field range of the information processing system 100, whether the user has walked away or returned, etc.

[0038] Background monitoring service 220 can be configured to communicate with one or more user presence devices 280, such as voice agent 285. For example, background monitoring service 220 can collect and / or receive outputs such as signal and / or sensor data from voice agent 285. Background monitoring service 220 can process sensor data, such as encoding, decoding, transmitting, filtering, and / or buffering all or part of the sensor data, to determine if a user is making a call. Background monitoring service 220 can also determine whether one or more users are present within a specific range, such as near-field and / or far-field range. Voice agents can also be used to detect keywords and can include software keyword positioners, hardware keyword positioners, and device driver interfaces that can be configured to support multiple voice agents. For example, one voice agent may be located near the near-field range, while another voice agent may be located near the far-field range.

[0039] Video management service 225 can be configured to adjust one or more settings of media application 255 based on user presence and behavior according to media management policy 235. For example, when a user moves away from information processing system 100, video management service 225 can pause currently playing / streaming media content. Once the user is determined to be in front of display device 265, video management service 225 can also resume media playback / streaming. As used herein, media playback includes media streaming. Video management service 225 can also rewind the video to the point when the user left, ensuring the user doesn't miss any part of the video.

[0040] The operating system power management service 230 can be configured to adjust various settings according to the media management policy 235, such as the brightness of the display device 265, speaker volume, speaker beam control, microphone beamforming, and power settings. For example, the operating system power management service 230 can be configured to dim the display device 265 when it receives information that the user has moved away from the information processing system 100, and increase the brightness of the display when the user's presence is detected. In another example, the operating system power management service 230 can put the information processing system 100 into a lower power mode, such as sleep or hibernation mode, when it receives information that the user has been away for a period of time and returns to normal power consumption, and enable facial recognition when the user's presence is detected.

[0041] Media playback source 250 includes local media files or content, such as audio, video, images, video games, teleconferencing, and combinations thereof. Media playback source 250 includes media files or media content that are locally accessible to information processing system 100, while also being accessible via network 202. Media content includes any data, text, sound, images, graphics, music, photos, or advertisements, and may also include video, audio, streaming content, webcasting, podcasts, blogs, online forums, and chat rooms. Media application 255 can be configured to support media content, such as playback and control. For example, media application 255 may include APIs for starting, pausing, resuming, searching for, and stopping media playback of video and / or audio files. Media application 255 can be a local media player or via a network such as... This allows for various applications to stream video. In some implementations, network 202 can be a public network, such as the Internet, a physical private network, a wireless network, a virtual private network, or any combination thereof.

[0042] Media content device 260 includes local and / or peripheral devices capable of receiving media content and providing media outputs such as visual and / or audio outputs. Media content device 260 includes display device 265, speaker 270, and microphone 275. Other functions and features of user presence device 280 are known in the art and will not be further disclosed herein unless required to illustrate the various embodiments disclosed herein. Display device 265 can be any type of dynamic display device capable of displaying images, video content, and / or graphical user interfaces on a display screen, and can include any type of light-emitting diode (LED), organic LED, liquid crystal display, electroluminescent, or other display technology. In this example, information processing system 100 uses display device 265 to provide visual outputs related to media application 255. Display device 265 can be physically connected or fixed to information processing system 100, or communicatively connected to it via one or more cables and / or intermediate components such as docking stations.

[0043] Speaker 270 can be any type of audio playback transducer, such as a loudspeaker, headphones, earphones, etc. Speaker 270 may include a single mono audio output device, a pair of devices providing stereo output, or an additional device providing surround sound effects. Video management service 225 may increase the volume of speaker 270 when it receives data output from background monitoring service 220 indicating that the user has left information processing system 100. Speaker beam control can be used to improve sound fidelity based on the user's position and / or gaze direction. For example, operating system power management service 230 may utilize speaker beam control to direct sound towards the user's position, such as directing it towards a specific direction 100 within the near or far field range around the information processing system, allowing the user to hear the sound better.

[0044] Microphone 275 can be any type of audio transducer capable of converting sound waves into electrical signals and may include a microelectromechanical system (MEMS) device or another type of audio capture device. For example, a user can use microphone 275 to answer or initiate calls (such as making a phone call or conference call), talk to other players while playing a video game, participate in conference calls (such as audio or video conferences), etc. Microphone 275 can be standalone or attached to another device, such as a headset. Therefore, microphone 275 can be used to detect whether a user is speaking. If the user is speaking, media management service 205 can pause video and / or audio content until the user stops speaking. Microphone 275 may include a single microphone, a pair of stereo microphones, or a microphone array. Based on the user's position and / or gaze direction, microphone beamforming can be used to improve the fidelity of the signal received from microphone 275. For example, the operating system power management service 230 can use microphone beamforming to be more sensitive to the sound in which the user is currently located, such as being more sensitive to a specific direction in the near field or far field, so that other participants in the conference call may be able to hear the user better.

[0045] User presence device 280 includes a voice agent 285, a near-field sensor 290, a far-field sensor 292, a camera sensor 295, an eye tracker 296, and a time-of-flight sensor 297. It may include other devices that can be used to detect user presence or user behavior, such as radar, magnetic, capacitive, and inductive sensors. For example, user presence device 280 may include a vision sensor, such as an Emza sensor, that can be configured to pick up a two-dimensional image of the object being measured. Visual sensors. Each of the user presence devices 280 can be communicatively coupled to the media management service 205 or its components via a wireless local area network such as 802.11b, g, n, and ac networks, a wireless personal area network such as Bluetooth or 802.11ad networks, or other communication networks. The user presence device 280 can transmit output in various formats (such as signals and / or data streams) to the media management service. For example, the data stream may be Extensible Markup Language. TM JavaScript Object Notation TM Other formats. Other functions and features of the user presence device 280 are known in the art and will not be further disclosed herein unless required to illustrate the various embodiments disclosed herein.

[0046] Ranging and proximity sensors, such as near-field sensor 290, mid-field sensor, and far-field sensor 292, can include any system, apparatus, or device configured to detect the presence, gaze, and other parameters such as direction, size, and speed of movement. The ranging and proximity sensors can transmit data outputs, such as signals, data streams, or both, to media management service 205. Near-field sensor 290 can provide ranging and proximity sensing of the user's presence within a first range around the information processing system, along with other parameters such as direction, size, and speed of movement. Near-field sensor 290 can transmit its output to near-field user perception service 210 using near-field communication protocols. Near-field sensor 290 can be a single sensor or a combination of multiple sensors, including proximity sensors, time-of-flight (TOF) sensors, eye trackers, charging voltage sensors, vision sensors, and motion sensors.

[0047] Far-field sensor 292 can provide ranging and proximity sensing of user presence in a second range, which can be farther than the first range around the information processing system, and in other parameters such as direction of movement, magnitude, and speed. Far-field sensor 292 can transmit outputs such as signals or data streams to far-field user awareness service 215. Far-field sensor 292 can be a single sensor or a combination of multiple sensors, including a camera, a Bluetooth user presence sensor, a Bluetooth Low Energy user presence sensor, etc. Similarly, a mid-field sensor can provide ranging and proximity sensing of user presence in a third range, which can be between the first and second ranges.

[0048] In addition to the presence or absence of a user, the time-of-flight sensor 297 (such as...) The TOF (Time-of-Flight) ranging sensor can also be configured to detect user movement. Specifically, the TOF sensor can be configured to scan infrared energy in a certain pattern at the expected user location and measure the reflection to detect the object. Additionally, the TOF sensor can enable facial recognition and can distinguish a person sitting in front of the display device 265 of the information processing system 100 from inanimate objects such as a chair.

[0049] Eye tracker 296 can be configured to measure eye movement to determine the location the user is looking at, the content the user is looking at, and the duration the user is fixating on a specific point. Eye tracker 296 can use near-infrared light and / or a high-definition camera to record the direction of reflection from the cornea and calculate the eye's position to determine its focus point. Camera sensor 295 can be configured to detect the user's presence and / or focus based on one or more signals or data streams from the camera. Camera sensor 295 can be one of a charge-coupled device (CCD), an electron-multiplying CCD, a complementary metal-oxide-semiconductor (CMOS), a back-illuminated CMOS, etc.

[0050] Figure 3 A sequence diagram 300 for user-presence-based media management is shown. The sequence diagram illustrates actions taken and / or messages transmitted between objects such as user 305, media playback source 250, video management service 225, near-field user-aware service 210, far-field user-aware service 215, background monitoring service 220, and operating system power management service 230. Sequence diagram 300 is an example of interaction between one or more objects, and those skilled in the art will understand that the sequence of interactions between objects can vary. Furthermore, sequence diagram 300 may include more or fewer objects and / or more or fewer interactions than depicted.

[0051] At point 310, user 305 can open media application 255 to play media playback source 250. At point 315, user 305 can click a button and / or press an icon in media application 255 to play media playback source 250, which can transmit a signal to video management service 225. At point 322, video management service 225 can play media playback source 250, also referred to as video in this figure, in response to the received signal. At point 320, user 305 can continue watching the video. At point 317, video management service 225 can transmit a message that video is playing to near-field user perception service 210. When user 305 is watching video at point 327, near-field sensor 290 can detect the presence or gaze of user 305.

[0052] At 325, user 305 can begin talking to another user or leave the cubicle. At 330, near-field sensor 290 no longer detects user 320's gaze and / or presence, which may be transmitted to or detected by near-field user perception service 210. In response, at 335, near-field user perception service 210 can transmit a message to video management service 225 that no user's gaze and / or presence has been detected, thereby pausing video at 336. At 337, near-field user perception service 210 can also transmit a message to far-field user perception service 215 that no user's gaze and / or presence has been detected. Simultaneously, voice agent 285 can be configured to detect the presence of multiple voices around voice commands and / or information processing system 100 at 332.

[0053] At point 345, far-field sensor 292 can scan user 305. At this point, far-field user perception service 215 can determine that it has not detected the presence and / or gaze of user 305, which can be transmitted to video management service 225 at point 342. This can be performed as a verification at point 335. In response, video management service 225 can continue to pause video at point 336. Far-field user perception service 215 can also transmit at point 345 to operating system power management service 230 that it has not detected the gaze or presence of user 305. In response, operating system power management service 230 can apply power-saving modes at point 350, such as dimming the display and putting information processing system 100 into low-power mode.

[0054] At 345, user 305 can resume looking at the display screen and / or return to the cubicle as user 305 walks back to the cubicle, where far-field sensor 292 can detect the user's presence at 355. Near-field sensor 290 can also detect the user's presence at 352, which can be used to verify 355. At 357, near-field user perception service 210 and / or far-field user perception service 215 can transmit to operating system power management service 230 that the presence and / or gaze of user 305 has been detected, and then operating system power management service can restore the brightness of the display screen at 370 and restore information processing system 100 to its normal power mode. In addition, operating system power management service 230 can send a signal to video management service 225 to resume video playback at 372, and video management service 225 resumes video playback at 375. User 305 can continue watching videos and / or playing video games until he or she closes the video at 380.

[0055] Figure 4 An environment 400 is shown that encompasses the field range associated with the information processing system 100. Environment 400 includes a near-field range 410 and a far-field range 415 surrounding the information processing system 100 and the user 305. The near-field range 410 can indicate... Figure 2 The near-field sensor 290 has a range. The far-field range 415 can indicate... Figure 2 The range of the far-field sensor 292. Environment 400 may include a mid-range range between the near-field range 410 and the far-field range 415. Settings associated with managing media content (such as the volume of the media / audio content, the time elapsed before the microphone is muted, and the distances of the near-field range 410 and the far-field range 415 from the information processing system 100) may be set by the user 305 or administrator using an interface according to default or preset settings before or during media playback in the media management service monitoring and / or management information processing system 100.

[0056] In one implementation, user 305 can watch video playback or play video games using information processing system 100 within near field range 410. While media playback is currently running, user 305 can walk away or turn away from the display screen of information processing system 100. The system can detect the user presence status of user 305, such as whether user 305 is in front of display device 265, within near field range 410, outside near field range 410 but within far field range 415, or outside far field range 415. The system can also detect whether user 305 is watching information processing system 100 when within near field range 410.

[0057] The media management service can apply media management policies based on user presence, absence, behavior, and / or user 305's position relative to field ranges such as near-field range 410 and far-field range 415. Media management policies can also be based on the type of media content being played. For example, if user 305 moves outside the range of the far-field proximity sensor during media playback, user 305 can be considered absent. Therefore, the media management service can pause media playback until the far-field proximity sensor detects user 305 within far-field range 415. In another example, the media management service can continue to pause media content until the near-field proximity sensor detects user 305 within near-field range 410. In yet another example, the media management service can continue to pause media content until the near-field proximity sensor detects user 305 within near-field range 410 and the user is viewing the display screen of the information processing system 100. Additionally, to pause media content, the display screen can be dimmed or turned off to save power until the media management service resumes media content. The media management service can also provide user 304 with a notification that media content has been paused.

[0058] In another implementation, user 305 can listen to audio playback, such as music or podcasts, at information processing system 100. During audio playback / streaming, user 305 can walk away from or turn away from the display screen of information processing system 100 while listening. As used herein, audio playback includes audio streaming. Similar to the above, the media management service can detect the user presence status of user 305, such as whether user 305 is within near-field range 410, outside near-field range 410 but within far-field range 415, or outside far-field range 415. The media management service can also detect whether user 305 is viewing information processing system 100 when within near-field range 310. Additionally, the media management service can detect whether user 305 is speaking on a telephone or similar device.

[0059] The media management service can adjust the speaker output based on the distance of user 305 from information processing system 100. For example, as user 305 moves from outside near-field range 410 towards far-field range 415, the media management service can increase the speaker volume and continue to increase the volume as the user moves further away but remains within far-field range 415. For example, the increase and / or decrease in volume can be proportional to the distance of the user from the information processing system or device. The volume can be adjusted upwards to a specific threshold, such as an appropriate volume for the distance within the hearing safety range. The threshold can be a default preset or set by user 305 or an administrator. When user 305 is outside far-field range 415, user 305 can be considered absent and audio playback can be paused. The opposite may occur when user 305 moves towards information processing system 100. For example, when a far-field proximity sensor detects user 305 within far-field range 415, the media management service can resume playback of audio content at a specific volume. As user 305 gets closer to information processing system 100, the volume of the audio content may become softer. Similar to the above, media management services can provide notifications when audio content is paused.

[0060] In another implementation, user 305 may be participating in a video conference using information processing system 100. Typically, the sound may be inconsistent as the user speaks and moves / paces. For example, the sound becomes louder when the user moves closer to the device or information processing system. Therefore, the sound becomes softer when the user moves away from the device or information processing system. Consequently, user 305 may have difficulty hearing other participants. Additionally, other participants may have difficulty hearing user 305. In this situation, the media management service can dynamically adjust the speaker output and microphone input of the information processing system based on the user's presence and location (such as presence and location relative to near-field range 410 and far-field range 415) to ensure that other participants can hear user 305 and user 305 can hear other participants. Furthermore, the microphone can be muted by default when user 305 is absent for a period of time. Additionally, notification settings associated with user 305's status, such as available, busy, away, etc., can be provided to other participants.

[0061] In an additional implementation, user 305 may wish to pause media content for one reason or another (such as when user 305 is cooking with his or her dominant hand, using a telephone, etc.) without pressing a button or clicking an icon. The media management service can be configured to detect gestures or track body movements, such as nodding, and apply media management policies accordingly. The media management service can apply different media management policies based on user 305's location and / or the type of media content currently being played. For example, the media management service could lower the volume, mute, or pause the media content in response to user 305 waving his or her other hand.

[0062] Figure 5 An environment 500 is shown, representing the field range associated with the information processing system 100. Environment 500 includes a near-field range 410 and a far-field range 415 surrounding the information processing system 100. Environment 500 also includes a user 305. In one embodiment, user 305 may view media content or listen to audio content at the information processing system 100. Another user (such as user 505) may approach user 305, and when far-field ranging and proximity sensors detect that user 505 is within the far-field range 415, a media management service may perform one or more actions based on a set of policies. For example, the media management service may lower the speaker volume and notify user 305 of user 505's approach.

[0063] Additionally, when the near-field proximity sensor detects that user 505 is within the near-field range 410, the media management service can perform one or more actions based on another set of policies. For example, in addition to the actions performed when user 505 is detected in the far-field range 415, the media management service can dim the display screen of the information processing system 100, mute the speakers and microphones, and / or pause media content for security reasons.

[0064] Figure 6 A method 600 for user-based media management is illustrated. Method 600 can be performed by one or more components of system 200, such as media management service 205. However, although embodiments of this disclosure are based on... Figure 2 This flowchart is described using the media management service 205, but it should be recognized that other systems can be used to perform the described methods. Those skilled in the art will understand that the flowchart illustrates a typical example and can be extended to advanced applications or services in practice.

[0065] Method 600 typically begins at box 605, wherein the method is for a media playback monitoring information processing system. At decision box 610, the method determines whether media playback is detected. If media playback is detected, a "yes" branch is used and the method proceeds to box 615. If media playback is not detected, a "no" branch is used and the method loops back to box 605.

[0066] At box 615, the method can collect or receive data output associated with user presence and behavior from one or more sensors and user presence devices. User presence includes user presence state. For example, the method can collect or receive data output associated with user presence and behavior from one or more sensors and user presence devices. Figure 2 User presence device 280 collects signals and / or sensor data. At decision box 620, the method determines whether a user exists. If the user exists, a "yes" branch is adopted and the method proceeds to box 615. If the user does not exist, a "no" branch is adopted and the method proceeds to decision box 625, whereby the method determines whether another user presence device, such as a camera and / or sensor, can be used for verification. If the user presence device can be used for verification, a "yes" branch is adopted and the method proceeds to box 630. If the user presence device is not available, a "no" branch is adopted and the method proceeds to box 645.

[0067] At box 630, the method can activate the user presence device and use it to assist in user presence verification. At decision box 635, the method can determine whether user presence is detected. If a user is detected, the "Yes" branch is adopted and the method proceeds to box 640. If no user is detected, the "No" branch is adopted and the method proceeds to box 645. At box 640, the method can collect signals and / or data from sensors and cameras to determine user presence status and / or user behavior.

[0068] At box 645, the method can determine the type of media and / or media content that the media application is playing / streaming. For example, the method can determine whether the media application is associated with video, video games, audio, teleconferencing, etc. At box 650, the method can determine and retrieve applicable media management policies to be applied based on user presence, user behavior, and / or the type of media content being played. Media management policies can be retrieved from data storage devices (such as databases) associated with the media management service. For example, if the media content is audio content, the media management policy may include rules associated with speaker volume, while if the media content is associated with a video game, the media management policy may include rules associated with microphone settings in addition to speaker volume, etc.

[0069] If a user is consuming multiple media content types, such as playing video games while listening to music, multiple media management policies can be determined and retrieved for each media type, unless a single media management policy covers all media content types the user is consuming. At box 655, the method can apply actions and / or adjust or update settings based on the media management policies. For example, the method can pause the video the user is watching. In another example, the method can lower the speaker volume. In yet another example, if the user is consuming multiple media content types, actions and / or adjustments can be performed for each media content type.

[0070] Figure 7A graphical user interface 700 for user-presence-based media management is illustrated. The graphical user interface 700 can be presented and displayed to a user on a display device of an information processing system. The graphical user interface 700 includes several fields, such as a toggle radio button 705, text fields 710 and 715, and a toggle radio button 720. Users can change different fields to manage user presence detection. Users may have different fields based on the type of media content. In this example, the fields can be associated with managing video content. For example, by toggling the toggle radio button 705, a user can specify whether the video can be automatically paused. Additionally, a user can determine the near-field range and / or far-field range via the toggle radio button 720 and text field 710. In this example, the near-field range is set to four feet. Furthermore, a user can set the amount of time before the parameters take effect. In this example, the video management interface can dim the display after five seconds using text field 715.

[0071] although Figure 6 In some implementations, an exemplary block of method 600 is shown, but with... Figure 6 Compared to the boxes depicted herein, method 600 may include additional boxes, fewer boxes, different boxes, or boxes with different arrangements. Those skilled in the art will understand that the principles presented herein can be implemented in any appropriately arranged processing system. Alternatively or additionally, two or more boxes of method 600 may be performed in parallel. For example, boxes 645 and 650 may be performed in parallel.

[0072] According to various embodiments of this disclosure, the methods described herein can be implemented by a computer system executable software program. Furthermore, in exemplary non-limiting embodiments, implementations may include distributed processing, component / object distributed processing, and parallel processing. Alternatively, virtual computer system processing may be configured to implement one or more of the methods or functionalities described herein.

[0073] When referred to as “device,” “module,” “unit,” “controller,” etc., the implementation described herein can be configured as hardware. For example, a part of an information processing system device can be hardware such as, for example, an integrated circuit (such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a structured ASIC, or a device embedded on a larger chip), a card (such as a peripheral component interface (PCI) card, a PCI-express card, a PCMCIA card, or other such expansion card), or a system (such as a motherboard, a system-on-a-chip (SoC), or a standalone device).

[0074] This disclosure is intended to include a computer-readable medium that includes instructions or instructions received and executed in response to a propagation signal, enabling devices connected to a network to transmit voice, video, or data over the network. Furthermore, instructions can be transmitted or received over the network via a network interface device.

[0075] Although a computer-readable medium is shown as a single medium, the term "computer-readable medium" includes single or multiple media, such as centralized or distributed databases, and / or associated caches and servers storing one or more instruction sets. The term "computer-readable medium" also includes any medium capable of storing, encoding, or carrying instruction sets for execution by a processor or for causing a computer system to perform any or more of the methods or operations disclosed herein.

[0076] In certain non-limiting exemplary embodiments, a computer-readable medium may include solid-state memory, such as a memory card or other package housing one or more non-volatile read-only memories. Additionally, a computer-readable medium may be random access memory or other volatile rewritable memory. Furthermore, a computer-readable medium may include magneto-optical or optical media (such as magnetic disks or magnetic tapes) or another storage device for storing information received via a carrier signal (such as a signal transmitted through a transmission medium). Digital file attachments to emails or other independent information archives or sets of archives can be considered as distributed media equivalent to tangible storage media. Therefore, this disclosure is considered to include any and more of computer-readable media or distributed media in which data or instructions can be stored, as well as other equivalents and successor media.

[0077] Although only a few exemplary embodiments have been described in detail above, those skilled in the art will readily understand that many modifications can be made to the exemplary embodiments without substantially departing from the novel teachings and advantages of the embodiments of this disclosure. Therefore, all such modifications are intended to be included within the scope of the embodiments of this disclosure as defined in the appended claims. In the claims, the device plus function clause is intended to cover not only structural equivalents but also equivalent structures as described herein.

Claims

1. A media management method based on user presence, comprising: The processor monitors the information processing system to detect whether a media application is playing media content. In response to detecting that the media application is playing the media content, data is collected from multiple sensors, including far-field sensors and near-field sensors, wherein the near-field sensors provide proximity sensing at a first range around the information processing system, and wherein the far-field sensors provide proximity sensing at a second range around the information processing system; The presence status of a first user within the second range relative to the information processing system is determined based on the data from the far-field sensor, wherein the data includes data from the camera of the far-field sensor; In response to verification of the presence of the first user within the first range associated with the near-field sensor, a first strategy based on the presence of the first user and the media content is applied by adjusting the power mode setting in the information processing system, wherein a power-saving mode is applied when the presence of the first user is not detected, and a normal power mode is restored when the presence of the first user is detected. When the presence of the first user is detected, the presence of the second user is determined to be within at least one of the second range and the first range; In response to confirming the presence of the second user within the second range based on data from the far-field sensor, a first set of operations is performed according to a second strategy, the first set of operations including at least one of the following: reducing the audio output associated with the media content and issuing a second user proximity notification to the first user; In response to confirming the presence of the second user within the first range based on data from the near-field sensor, a second set of operations is performed according to a third strategy. The second set of operations includes at least one of the following: dimming the display screen of the information processing system, muting the audio output associated with the media content, and pausing the media content.

2. The method of claim 1, further comprising determining the user presence state based on the data from the sensor.

3. The method of claim 1, further comprising determining user behavior based on the data from the sensor.

4. The method of claim 1, wherein applying the strategy includes adjusting the power consumption mode of the information processing system.

5. The method of claim 1, further comprising, after applying the strategy, monitoring the information processing system to detect whether the media application is still playing.

6. The method of claim 1, wherein adjusting the settings in the media application includes pausing the playback of the media content.

7. An information processing system, comprising: processor; as well as A memory coupled to the processor, the memory storing instructions that, when executed by the processor, cause the information processing system to: Detect media playback; In response to detecting the media playback, data is collected from a user presence device, a near-field sensor, and a far-field sensor, the near-field sensor providing proximity sensing at a first range around the information processing system, and the far-field sensor providing proximity sensing at a second range around the information processing system. The presence of a first user relative to the information processing system is determined based on the data from the user presence device. as well as In response to verification from another user presence device, a first strategy based on the presence of the first user is applied by adjusting the power mode setting, wherein a power saving mode is applied when the presence of the user is not detected, and wherein a normal power mode is restored when the presence of the user is detected; When the presence of the first user is detected, the presence of the second user is determined to be within at least one of the second range and the first range; In response to confirming the presence of the second user within the second range based on data from the far-field sensor, a first set of operations is performed according to a second strategy, the first set of operations including at least one of the following: reducing the audio output associated with the media content and issuing a second user proximity notification to the first user; In response to confirming the presence of the second user within the first range based on data from the near-field sensor, a second set of operations is performed according to a third strategy. The second set of operations includes at least one of the following: dimming the display screen of the information processing system, muting the audio output associated with the media content, and pausing the media content.

8. The information processing system of claim 7, wherein the instructions, when executed by the processor, are further configured to cause the information processing system to determine the user presence status based on the data from the user presence device.

9. The information processing system of claim 7, wherein the instructions, when executed by the processor, are further configured to cause the information processing system to determine user behavior based on the data from the user presence device.

10. The information processing system of claim 7, wherein applying the strategy includes adjusting the power consumption mode of the information processing system.

11. The information processing system of claim 7, wherein the instructions, when executed by the processor, are further configured to cause the information processing system to further monitor the information processing system to detect whether the media playback is still running after the strategy is applied.

12. A non-transitory computer-readable medium storing instructions executable to perform operations including: In response to detecting whether a media application is playing media content on an information processing system, data is collected from multiple sensors, including a far-field sensor, a near-field sensor, and a voice agent, wherein the near-field sensor provides proximity sensing at a first range around the information processing system, and wherein the far-field sensor provides proximity sensing at a second range around the information processing system. The presence status of a first user within the second range is determined based on the data from the far-field sensor and the voice agent; as well as In response to the verification of the presence of the first user within the first range by the near-field sensor, a first strategy based on the presence of the user and the media content is applied by adjusting the settings in the media application, wherein the strategy includes adjusting the power mode settings of the information processing system. When the presence of the first user is detected, the presence of the second user is determined to be within at least one of the second range and the first range; In response to confirming the presence of the second user within the second range based on data from the far-field sensor, a first set of operations is performed according to a second strategy, the first set of operations including at least one of the following: reducing the audio output associated with the media content and issuing a second user proximity notification to the first user; In response to confirming the presence of the second user within the first range based on data from the near-field sensor, a second set of operations is performed according to a third strategy. The second set of operations includes at least one of the following: dimming the display screen of the information processing system, muting the audio output associated with the media content, and pausing the media content.

13. The non-transitory computer-readable medium of claim 12, wherein the operation further comprises determining a user presence state based on the data from the sensor.

14. The non-transitory computer-readable medium of claim 12, wherein the operation further comprises determining user behavior based on the data from the sensor.

15. The non-transitory computer-readable medium of claim 12, wherein the operation further includes, after applying the strategy, monitoring the information processing system to detect whether the media application is still playing in the information processing system.

16. The non-transitory computer-readable medium of claim 12, wherein applying the strategy includes adjusting the power consumption mode.

17. The non-transitory computer-readable medium of claim 12, wherein the operation further includes, after applying the strategy, monitoring the information processing system to detect whether the media application is playing in the information processing system.

Images