Rendering content on an in-living environment display based on an out-of-living environment environment
Patent Information
- Application Number
- JP2023542973
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2021-01-22
- Filing Date
- 2022-01-21
- Publication Date
- 2024-12-17
- Estimated Expiration
- 2042-01-21
AI Technical Summary
【0006】 全体を通じて同じ要素を同じ参照符号によって示す添付図面を参照しながら本開示の以下の詳細な説明を検討することにより、本開示のこれらの及びその他の特徴及び利点を理解することができる。
Smart Images

Figure 0007601235000001 
Figure 0007601235000002 
Figure 0007601235000003
Abstract
Description
[Technical field]
[0001] CROSS REFERENCE TO RELATED APPLICATIONS / INCORPORATION BY REFERENCE none
[0002] Various embodiments of the present disclosure relate to display technology and home automation, and in particular to rendering content on a display within a living environment based on an environment outside the living environment. [Background technology]
[0003] Advances in display technology and home automation have diversified the use of display device(s) within living environments such as homes or office spaces. For example, the traditional use of televisions within homes is also changing. Many users prefer to use televisions as digital wallpapers, and there is a shift from traditional broadcasting or content streaming to art and aesthetics. Other uses of displays in living environments may be possible, depending on the needs of the users. Summary of the Invention [Problem to be solved by the invention]
[0004] The limitations and disadvantages of the conventional approach will become apparent to one skilled in the art by comparing the described system with certain aspects of the present disclosure illustrated in the remaining portions of this application and with reference to the drawings. [Means for solving the problem]
[0005] Provided is an electronic device and method for rendering content on a display within a living environment based on an environment outside the living environment, substantially as hereinbefore described and / or in connection with at least one of the figures and more fully as hereinbefore described in the claims.
[0006] These and other features and advantages of the present disclosure will become apparent from a consideration of the following detailed description of the disclosure when taken in conjunction with the accompanying drawings, in which like reference characters refer to like elements throughout. [Brief description of the drawings]
[0007] [Figure 1] FIG. 1 is a block diagram illustrating an example environment for rendering content on a display within a living environment based on an environment outside the living environment, according to an embodiment of the disclosure. [Diagram 2] FIG. 1 is a block diagram illustrating an example electronic device for rendering content on a display within a living environment based on an environment outside the living environment, according to an embodiment of the present disclosure. [Figure 3A] FIG. 2 illustrates an example scenario for rendering content on a display within a living environment based on an environment outside the living environment, according to an embodiment of the present disclosure. [Figure 3B] FIG. 2 illustrates an example scenario for rendering content on a display within a living environment based on an environment outside the living environment, according to an embodiment of the present disclosure. [Figure 4] FIG. 2 illustrates an example scenario for rendering content on a display within a living environment based on an environment outside the living environment, according to an embodiment of the present disclosure. [Diagram 5] 1 is a flowchart illustrating an exemplary method for rendering an image of an external environment on a display in a living environment, according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0008] The disclosed electronic devices for rendering media content based on an external environment and methods for so controlling the electronic devices may find implementation as described below. Exemplary aspects of the present disclosure provide an electronic device (e.g., a television (TV)) that can control a display screen associated with the electronic device within a living environment. For example, the living environment may be a residential space (such as an apartment or house), a commercial space (such as an office space, a hotel room, a concert hall, a particular room or space within the residential or commercial space, and the like.
[0009] The electronic device may be configured to track a head position or gaze of a user present within the living environment at any point in time based on facial regions detected from one or more images of the living environment. Such images may be received from a first image capture device (e.g., a closed circuit television (CCTV) camera or other image capture device) coupled to the electronic device. The first image capture device may be present inside the living environment. For example, the first image capture device may be a wall-mounted camera or an integrated camera unit of the electronic device. The electronic device may select a second image capture device from the multiple image capture devices based on the tracked head position and / or gaze. For example, the user may be facing or looking in a particular direction, and the electronic device may select a second image capture device (e.g., a CCTV camera) that may be present outside the living environment in a direction that matches the user's orientation, field of view (FoV), or gaze.
[0010] The electronic device can be configured to receive images from the selected second image capture device and control the display screen to display the received images. In some living environments, people living or staying there may not have windows or may not be able to open them, for example, if the view outside is not good or the light is very bright. In some cases, even if there is a window, it may face a side of the living environment that is not desirable. For example, the window may be on the west side but the view to the east may be better. To address this issue, the disclosed electronic device (and display screen) can operate as an artificial or digital window that provides a view of the external environment. The electronic device can also be controlled to dynamically select a display screen in the living environment to display a view of the external environment (i.e., a live or recorded image / video feed) by tracking head position or gaze.
[0011] FIG. 1 is a block diagram illustrating an example environment for rendering content on a display within a living environment based on an environment outside the living environment, according to an embodiment of the present disclosure. FIG. 1 illustrates a diagram of an environment 100. The environment 100 includes a living environment 102 and an external environment 104. The living environment 102 illustrates an electronic device 106, a first image capture device 108, a number of display screens 110, an audio playback device 112, and one or more electronic devices 114. The external environment 104 illustrates a number of image capture devices 116, a number of audio capture devices 118, and a number of sensors 120. The number of image capture devices 116 may include, for example, a second image capture device 122. Also illustrated within the living environment 102 is a first user 124.
[0012] 1 depicts electronic device 106, first image capture device 108, multiple display screens 110, audio playback device 112, and one or more electronic devices 114 as separate devices. In some embodiments, the functionality of at least one of first image capture device 108, multiple display screens 110, audio playback device 112, or one or more electronic devices 114 may be incorporated entirely within electronic device 106 without departing from the scope of the present disclosure. Environment 100 further depicts a server 126, which may be communicatively coupled to electronic device 106 and / or other devices within environment 100 via a communications network 128.
[0013] The living environment 102 may be a built environment that provides people with the ability to engage in various types of social, cultural, religious, economic, or political activities. Examples of the living environment 102 may include, but are not limited to, a residential space (such as an apartment or a house), a commercial space (such as an office space, a hotel room, or a concert hall), or a specific room or space within a residential or commercial space. The external environment 102 may be any physical space outside the living environment 102. Examples of the external environment 102 may include, but are not limited to, a terrace, a roof, a balcony, a garden, a park, a road, a forest, a sky, or a combination thereof.
[0014] The electronic device 106 may include suitable logic, circuitry, and / or interfaces that may be configured to control one or more display screens (among the multiple display screens 110) to display one or more images of an exterior view of the living environment 102. Such images may be acquired by an image capture device (such as the second image capture device 122).
[0015] In some embodiments, the electronic device 106 may be placed or attached to any support or supporting structure of the living environment 102 (e.g., walls, roofs, doors, ceilings, etc.). In an operational state, the electronic device 106 and display screen may appear to be or mimic the functionality of an opening (e.g., a window) to a view outside the living environment 102. The electronic device (and display screen) may be placed or attached within or over an opening in the living environment 102 such that in a non-operational state, the electronic device (and display screen) appears to be or functions as an opening. In such cases, the display screen may be implemented as a see-through or semi-transparent or translucent display.
[0016] In some embodiments, a display screen may be integrated into the electronic device 106. The size of the display screen should be sufficient to match the typical size of an opening such as a window, door, or skylight. Examples of implementations of such electronic device 106 include, but are not limited to, televisions (TVs), display or computer monitors, digital signage, digital mirrors (or smart mirrors), and video walls (consisting of two or more displays tiled or stacked in series to form one large screen).
[0017] In another embodiment, electronic device 106 may communicate with one or more display screens through a suitable network or I / O (input / output) interface. Examples of implementations of such electronic device 106 include, but are not limited to, a mobile phone, a smart phone, a tablet computing device, a personal computer, a projector, a gaming console, a media player, a sound system, a set-top box or digital TV receiver, a smart audio device, a home theater system, a digital camera, or any other consumer electronic device with media rendering and sharing capabilities.
[0018] The first image capture device 108 may include suitable logic, circuitry, code, and / or interfaces that may be configured to capture one or more first images of the living environment 102. In one embodiment, the first image capture device 108 may detect a first facial region of a first user 124 present in the living environment 102 based on the captured one or more first images. In another embodiment, the first image capture device 108 may transmit the captured one or more first images to the electronic device 106, which may detect the first facial region of the first user 124. The detection may be based on one or more image processing techniques, such as face detection methods, object detection methods, deep learning methods, and other methods known to those skilled in the art.
[0019] In some embodiments, the first image capture device 108 may be placed, attached, or secured to any fixture or support structure within the living environment 102. In another embodiment, the first image capture device 108 may be integrated into the electronic device 106 or a display screen coupled to the electronic device 106. Examples of the first image capture device 108 may include, but are not limited to, an image sensor, a wide-angle camera, an action camera, a closed circuit television (CCTV) camera, a camcorder, a digital camera, a camera phone, a camera drone, a camera unit integrated into a robot (e.g., a robot dog, a cleaning robot, an attendant robot, etc.), or a camera unit integrated into any consumer electronic device (e.g., but is not limited to, a refrigerator, a lighting fixture, a personal computer, or an in-room security device, etc.).
[0020] The multiple display screens 110 may include suitable logic, circuitry, and interfaces that may be configured to display the output of the electronic device 106. The multiple display screens 110 may be utilized to display one or more second images captured from a selected second image capture device 122. In some embodiments, the multiple display screens 110 may be external display screens associated with the electronic device 106. The multiple display screens 110 may be touch screens that may allow a user (such as the first user 124) to provide user input via the multiple display screens 110 to select an operational mode of the electronic device 106. Each of the multiple display screens 110 may be implemented through one of several known technologies, such as, but not limited to, liquid crystal display (LCD) displays, light emitting diode (LED) displays, plasma displays, or organic LED (OLED) display technologies. In some embodiments, each of the multiple display screens 110 may be one of a see-through display, a translucent or translucent display, or a display having variable transparency (i.e., transparency that may be electronically controlled).
[0021] The audio playback device 112 may include suitable logic, circuitry, and / or interfaces that may be configured to play audio. The audio playback device 112 may receive electrical signals or instructions (i.e., related to audio) from an audio source (such as, but not limited to, the audio capture devices 118 or a server) and convert the received electrical signals or instructions into an audio output. The audio playback device 112 may be associated with the electronic device 106.
[0022] In some embodiments, the audio playback device 112 may be integrated into the electronic device 106. In such implementations, multiple audio capture devices 118 may form internal components of the electronic device 106. In some embodiments, multiple audio capture devices 118 may be communicatively coupled to the electronic device 106 and located within the living environment 102. Examples of the audio playback device 112 may include, but are not limited to, a surround sound system with a particular speaker layout (e.g., an M:N layout including M speakers and N subwoofers), external wireless speakers, internal speakers, external wired speakers, woofers, subwoofers, tweeters, loudspeakers, monitor speakers, optical audio devices, or sound output devices that may be communicatively coupled to the electronic device 106 through the communications network 128 or integrated into the electronic device 106.
[0023] Each electronic device of the one or more electronic devices 114 may include suitable logic, circuitry, and / or interfaces that may be configured to function based on control instructions from the electronic device 106. Such instructions may be communicated to the one or more electronic devices 114 based on environmental information related to an environment (e.g., the external environment 104) outside the living environment 102. The environmental information may include, for example, luminosity information, temperature information, or airflow information. Such information may be received from a number of sensors 120. In some embodiments, the one or more electronic devices 114 may be coupled to the electronic device 106 and disposed within the living environment 102. Examples of each of the one or more electronic devices 114 may include, but are not limited to, a lighting device, an electric fan or air conditioning system, an electric heater, a thermostat device, an aroma diffuser device, an audio / video playback device, a heating, ventilation, and air conditioning (HVAC) system, or other consumer electronic devices.
[0024] Each of the plurality of image capture devices 116 may include suitable logic, circuitry, code, and / or interfaces that may be configured to capture one or more second images of the external environment 104 from a particular camera perspective and / or camera position. Examples of the plurality of image capture devices 116 may include, but are not limited to, image sensors, wide-angle cameras, action cameras, CCTV cameras, camcorders, digital cameras, camera phones, camera drones, time-of-flight cameras (ToF cameras), night vision cameras, and / or other image capture devices.
[0025] The multiple image capture devices 116 may be located outside the living environment 102 (e.g., the external environment 104). In some embodiments, at least one of the multiple image capture devices 116 may be located, attached, or fixed at a specific location on a nearby support structure that is part of the living environment 102 (e.g., near a gate of the house, a boundary wall, or a fence around the living environment 102). For example, if the support structure is a wall or roof of the house, the specific location may be the side of these walls or roofs that face the external environment 104. In some embodiments, at least one of the multiple image capture devices 116 may be located, attached, or fixed at the same part of the support structure (e.g., a wall) where a display screen may be placed to match a window view or view. In such an implementation, both the display screen and the image capture device may be located at the same part of the support structure, but the former may be inside the living environment 102 and the latter may be outside the living environment 102.
[0026] Each of the plurality of audio capturing devices 118 may include suitable logic, circuitry, code, and / or interfaces that may be configured to capture first audio present in an environment outside the living environment 102. The plurality of audio capturing devices 118 may generate first audio information associated with the first audio captured from the plurality of audio capturing devices 118. Examples of the plurality of audio capturing devices 118 may include, but are not limited to, electret microphones, dynamic microphones, carbon microphones, piezoelectric microphones, fiber microphones, microelectromechanical systems (MEMS) microphones, or other microphones.
[0027] The plurality of sensors 120 may include suitable logic, circuitry, code, and / or interfaces that may be configured to generate environmental information related to an environment outside the living environment 102 (e.g., the external environment 104). The environmental information may include, for example, light intensity information, temperature information, airflow information, and the like. The plurality of sensors 120 may be further configured to transmit the environmental information to the electronic device 106. The plurality of sensors 120 may include, for example, one or more of an infrared sensor, a radio frequency identification (RFID) sensor, a motion sensor, a temperature sensor, an ultrasonic sensor, an airflow sensor, and a light sensor, and the like. More details regarding the plurality of sensors 120 are described, for example, in FIG. 3A.
[0028] The server 126 may include suitable logic, circuitry, code, and / or interfaces that may be configured to transmit media content to the electronic device 106. In some embodiments, the server 126 may be configured to store and communicate to the electronic device 106 environmental information that may control the operation of one or more electronic devices 114 in the living environment 102. The server 126 may perform operations through web applications, cloud applications, HTTP requests, repository operations, file transfers, and the like. Example implementations of the server 126 may include, but are not limited to, a database server, a file server, a web server, a media server, an application server, a mainframe server, or a cloud server.
[0029] The communication network 128 may include a communication medium that allows the electronic device 106, the living environment 102, the external environment 104, and the server 126 to communicate with each other. The communication network 128 may be either a wired connection or a wireless connection. Examples of the communication network 128 may include, but are not limited to, the Internet, a cloud network, a Wireless Fidelity (Wi-Fi) network, a personal area network (PAN), a local area network (LAN), or a metropolitan area network (MAN). Various devices in the environment 100 may be configured to connect to the communication network 128 according to various wired and wireless communication protocols. Examples of such wired and wireless communication protocols include, but are not limited to, at least one of 4th generation Long Term Evolution (LTE), 5th generation New Wireless (NR), Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), ZigBee, EDGE, IEEE 802.11, Light Fidelity (Li-Fi), 802.16, IEEE 802.11s, IEEE 802.11g, multi-hop communication, wireless access point (AP), device-to-device communication, cellular communication protocols, and Bluetooth (BT) communication protocols.
[0030] In operation, the electronic device 106 can be configured to execute a workflow that controls a display screen(s) to display image(s) or video of an exterior view of the living environment 102. Execution of such a workflow can be based on event detection, schedule, or user input. For example, an event can be related to a weather condition (such as whether it is snowing or raining), a social meeting or gathering (such as a party or dinner), or a time-dependent condition (such as sunrise, sunset, full moon, solar eclipse, etc.). A user can also schedule the execution of a workflow for a specific date and time. Workflows are only briefly described herein.
[0031] The electronic device 106 may be configured to receive one or more first images of the living environment 102 from a first image capture device 108 that may be coupled to the electronic device 106 and located inside the living environment 102. Such an image of the living environment 102 may include multiple objects (such as a display screen, furniture items, decorative items, or user(s)). The electronic device 106 may be configured to detect a first facial region of a first user 124 in the living environment 102 based on the received one or more first images at any point in time. For example, as described in FIG. 3A, the electronic device 106 may detect the first facial region of the first user 124 based on different image processing techniques, such as, but not limited to, face detection methods, object detection methods, deep learning models, or other image processing methods.
[0032] The electronic device 106 may be configured to track at least one of a first head position or a first gaze of the first user 124 based on the detected first facial region. For example, the first head position or the first gaze may be tracked to identify a particular location or device (or a particular direction) at which the first user 124 is likely looking or facing while present in the living environment 102. The electronic device 106 may be configured to select the second image capture device 122 from the multiple image capture devices 116 (e.g., as described in FIG. 3A ) based on tracking of the first head position and / or the first gaze. The multiple image capture devices 116 may be located outside the living environment 102 (e.g., the external environment 104).
[0033] In some embodiments, the electronic device 106 may determine locations, areas, or physical structures within the living environment 102 to which the first user 124 is likely looking or facing based on the first head position or first line of sight. The selection of the second image capture device 122 may be based on whether a display screen is available at or near these locations, areas, or physical structures within the living environment 102. In these or other embodiments, the second image capture device 122 may be selected among the multiple image capture devices 116 based on a determination that the relative distance of the second image capture device 122 to these locations, areas, or physical structures is minimal. For example, if the first user 124 is looking at a television screen on the right wall, a camera located outside the home and closest to the television screen may be selected.
[0034] The electronic device 106 can receive one or more second images from the selected second image capture device 122 and control a display screen (e.g., the first display screen 110A) associated with the electronic device 106 to display the received one or more second images. Such images can include a view of the external environment 104. The disclosed electronic device 106 can be controlled to automatically select one of multiple display screens in the living environment 102 to display an image of the external environment 104 by tracking head position or eye gaze.
[0035] FIG. 2 is a block diagram illustrating an example electronic device for rendering content on a display within a living environment based on an environment outside the living environment, according to an embodiment of the present disclosure. FIG. 2 is described with reference to elements of FIG. 1. FIG. 2 illustrates a block diagram 200 of an electronic device 106. The electronic device 106 may include circuitry 202 and memory 204. In some embodiments, the electronic device 106 may also include an image capture device (such as the first image capture device 108), a display screen (such as the plurality of display screens 110), and / or an audio playback device 112. Additionally, the electronic device 106 may include a network interface 206 that allows the electronic device 106 to be connected to a communications network 128.
[0036] The circuitry 202 may include suitable logic, circuits, interfaces, and / or code that may be configured to execute program instructions associated with different operations performed by the electronic device 106. The circuitry 202 may include one or more specialized processing units that may be implemented as independent processors. In some embodiments, the one or more specialized processing units may be implemented as an integrated processor or a group of processors that collectively perform the functions of the one or more specialized processing units. The circuitry 202 may be implemented based on multiple processor technologies known in the art. Example implementations of the circuitry 202 may be an X86-based processor, a graphic processing unit (GPU), a reduced instruction set computing (RISC) processor, an application specific integrated circuit (ASIC) processor, a complex instruction set computing (CISC) processor, a microcontroller, a central processing unit (CPU), and / or other control circuitry.
[0037] The memory 204 may include suitable logic, circuitry, interfaces, and / or code that may be configured to store program instructions executed by the circuitry 202. In some embodiments, the memory 204 may also be configured to store images of the living environment or external environment, and environmental information related to the external environment. Example implementations of the memory 204 may include, but are not limited to, a random access memory (RAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a hard disk drive (HDD), a solid-state drive (SSD), a CPU cache, and / or a secure digital (SD) card.
[0038] The network interface 206 may include suitable logic, circuitry, interfaces, and / or code that may be configured to facilitate communication between the electronic device 106, the living environment 102, the external environment 104, and the server 126 via the communications network 128. The network interface 206 may be implemented using various known technologies that support wired or wireless communication between the electronic device 106 and the communications network 128. The network interface 210 may include, but is not limited to, an antenna, a radio frequency (RF) transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a coder-decoder (CODEC) chipset, a subscriber identity module (SIM) card, or a local buffer circuit.
[0039] The network interface 206 may be configured to communicate via wireless communication with a network, such as the Internet, an intranet, a wireless network, a cellular network, a wireless local area network (LAN), or a metropolitan area network (MAN). The wireless communication may be configured to use one or more of a number of communication standards, protocols, and technologies, such as Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (W-CDMA), Long Term Evolution (LTE), 4G LTE, 5G NR, Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Bluetooth, Wireless Fidelity (WiFi) (such as IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, or IEEE 802.11n), Voice over Internet Protocol (VoIP), Light Fidelity (Li-Fi), Worldwide Interoperability for Microwave Access (Wi-MAX), and the like.
[0040] Those skilled in the art will appreciate that electronic device 106 of Figure 2 may include other suitable components or systems in addition to those illustrated herein for purposes of describing and illustrating the functionality and operation of the present disclosure. A detailed description of other components or systems of electronic device 106 is omitted from this disclosure for the sake of brevity. The operation of circuit 202 is further described, for example, in Figures 3A, 3B, and 4.
[0041] FIG. 3A illustrates an exemplary scenario for rendering content on a display in a living environment based on an environment outside the living environment, according to an embodiment of the present disclosure. FIG. 3A is described in relation to elements of FIG. 1 and FIG. 2. FIG. 3A illustrates an exemplary scenario 300A. The exemplary scenario 300A illustrates a room 302 and an external environment 304, which may be examples of the living environment 102 and the external environment 104, respectively. Also illustrated is a first image capture device 306, which may be an exemplary implementation of the first image capture device 108 of FIG. 1. Also illustrated are multiple image capture devices (such as a second image capture device 308 and a third image capture device 310). Also illustrated are multiple display screens (such as a first display screen 312 and a second display screen 314), which may be exemplary implementations of the multiple display screens 110. Also illustrated are one or more electronic devices (such as a lighting device 322 and an air conditioner 324), which may be exemplary implementations of the one or more electronic devices 114. Additionally, multiple sensors (such as temperature sensor 328 and light sensor 326 ), which may be example implementations of multiple sensors 120 , are also shown.
[0042] Scenario 300A further illustrates walls 316, 318, and 320 surrounding room 302. Also illustrated is a first user 330 standing near a first display screen 312 located within room 302 and facing wall 316. It should be noted that lighting device 322 and air conditioner 324 are merely examples. Scenario 300A may include other types of electronic devices without departing from the scope of the present disclosure. Also, temperature sensor 328 and light sensor 326 are merely examples. Scenario 300A may include other types of sensors without departing from the scope of the present disclosure.
[0043] The circuitry 202 may be configured to receive, at any time, one or more first images of the room 302 from a first image capture device 306 (which may be coupled to the electronic device 106 and located inside the room 302). The first image capture device 306 may be configured to capture one or more images of the room 302. In an embodiment, the first image capture device 306 may be configured to detect multiple objects (such as furniture items, decorative items, or a user) present in the room 302 based on the captured image or images as object detection results. In an embodiment, the first image capture device 306 transmits the captured images and the object detection results to the circuitry 202, which may determine multiple objects in the room 302.
[0044] The circuit 202 may be configured to detect a first facial region of the first user 330 present in the room 302 based on the received one or more first images. In an embodiment, the circuit 202 may detect the first facial region of the first user 330 in the received one or more first images based on image processing techniques, such as, but not limited to, face detection, object detection, deep learning, and other image processing techniques. The circuit 202 may be configured to track at least one of a first head position or a first gaze of the first user 330 based on the detected first facial region. The first head position or the first gaze may be tracked based on the detected first facial region in the received one or more images of the room 302. The first head position or the first gaze may indicate a location or area in the room 302 where the first user 330 is believed to be looking or gazing at at a particular time. For example, the first head position may be associated with a facial orientation of the user in terms of a particular yaw, roll, or pitch. Specifically, in some instances, the first head position may be determined in terms of a position such as a straight face position, a downward facing position, an upward facing position, a position toward the right or left side of the body, or a face orientation over the shoulder.
[0045] In an embodiment, the circuit 202 can be configured to determine a first viewing direction of the first user 330 based on at least one of the tracked first head position or the first line of sight. The viewing direction can be toward a wall (such as walls 316, 318, and 320) of the room 302, where the selected second image capture device can be located. For example, in the scenario 300A, at least one of the tracked first head position or the first line of sight can be toward the wall 316 of the room 302. In an embodiment, the viewing direction of the first user 330 can be determined based on a first head position associated with the first user 330. The viewing direction can change due to a change in the head position. In some scenarios, the viewing direction of the first user 330 can be determined based on the first line of sight of the first user 330. The viewing direction can change due to a change in the first line of sight, whereas the first head position can remain unchanged or can change. For example, at a first time point, the first user 330 may have a first head position that is straight ahead and a first line of sight that is directed toward the left corner of the wall 316. The second image capture device 308 may capture one or more second images of the external environment 104 based on a perspective associated with the first head position and the first line of sight at the first time point.
[0046] In an embodiment, the circuitry 202 may be further configured to select a first display screen 312 from multiple display screens (in the living environment) based on at least one of a tracked first head position or a first line of sight of the first user 330. For example, scenario 300A indicates that the first head position or the first line of sight is directed towards a wall 316 of the room 302. In this case, the first display screen 312 may be selected.
[0047] The circuitry 202 may be configured to select a second image capture device from a plurality of image capture devices (e.g., second image capture device 308 and third image capture device 310) based on at least one of the tracked first head position or first line of sight. The plurality of image capture devices are located outside the room 302 (e.g., the external environment 304). For example, if the first head position or the first line of sight is toward a wall 316 of the room 302, the circuitry 202 may select the second image capture device 308 located outside the wall 316.
[0048] The circuitry 202 can be configured to receive one or more second images from the selected second image capture device 308. Such images can include a view of the external environment 304 captured by the selected second image capture device 308. In an embodiment, the circuitry 202 can be further configured to control the selected second image capture device 308 to capture one or more second images of the environment external to the room 302 based on at least one of the tracked first head position or first line of sight of the first user 330. Thereafter, the circuitry 202 can be configured to control the selected first display screen 312 associated with the electronic device 106 to display the received one or more second images.
[0049] In some embodiments, the circuit 202 can select a preset or configurable theme from among multiple preset or configurable themes based on event information (e.g., weather-related events) or user input / preferences. The circuit 202 can be configured to overlay animation or graphic items on the displayed image (i.e., the displayed second image or images) based on the selected preset or configurable theme. Such themes can include, for example, graphics (representing a window style or design, window architecture, window shape, or window material), glass colors and effects, animated objects (e.g., snow, rain, or firecrackers), color effects (e.g., color temperature changes), or combinations thereof. Also, such themes, when overlaid or applied on the displayed image, can make the first display screen 312 appear as if a particular type of window were present on the wall 316 (i.e., support structure) of the living environment 102. This allows the electronic device 106 to maintain and / or enhance the user's viewing experience and add to the overall aesthetics of the living environment 102.
[0050] The first display screen 312 can be dynamically controlled to display an image of the external environment based on the head position and / or first line of sight of the first user 330. The first user 330 can experience a desired view of the external environment from any particular location or direction in which the display screen and image capture device are located.
[0051] FIG. 3B illustrates an exemplary scenario for rendering content on a display within a living environment based on an environment outside the living environment, according to an embodiment of the present disclosure. FIG. 3B is described with reference to elements of FIG. 1, FIG. 2, and FIG. 3A. FIG. 3B illustrates an exemplary scenario 300B. The exemplary scenario 300B illustrates a room 302 that may include a first image capture device 306 (e.g., as described in FIG. 3A), a first display screen 312, a second display screen 314, a lighting device 322, and an air conditioner 324. The exemplary scenario 300B further illustrates an external environment 304 that may include a second image capture device 308, a third image capture device 310, a temperature sensor 328, and a light sensor 326 (e.g., as described in FIG. 3A).
[0052] 3B further illustrates a first user 330 who may be standing in the room 302 and facing a wall 320. In an embodiment, the circuitry 202 may be configured to select a third image capture device 310 coupled to the electronic device 106 based on at least one of the tracked first head position or first line of sight. The third image capture device 310 may be located on a wall 320 outside the room 302 (external environment 304). The circuitry 202 may be configured to receive one or more third images from the selected third image capture device 310 and control a second display screen 314 associated with the electronic device 106 to display the received one or more third images. The one or more third images may be captured from a perspective of the third image capture device 310.
[0053] In an embodiment, multiple image capture devices (e.g., second image capture device 308 and third image capture device 310) may be associated with multiple audio capture devices (e.g., multiple audio capture devices 118) located outside the living environment (e.g., room 302). Circuit 202 may be further configured to control a first audio capture device (e.g., first audio capture device 118A) associated with a selected second image capture device 308 to capture first audio present in the environment outside room 302.
[0054] The circuit 202 may be configured to receive, from the first audio capture device 118A, first audio information related to the captured first audio. The circuit 202 may be further configured to control an audio playback device (such as the audio playback device 112) to play the first audio based on the received first audio information. While viewing the displayed image on the second display screen 314, the first user 330 may hear sounds of the environment outside the room 302, which may enhance the experience of the first user 330.
[0055] In an embodiment, the circuit 202 may be further configured to control the first display screen 312 or the audio player 112 associated with the electronic device 106 to render media content based on at least one of a current time, a geographic location of the first user 330, or one or more objects present in the living environment (e.g., the room 302). For example, the media content may render a morning scene, an evening scene, a night scene, etc. based on the current time. The first display screen 312 may display a scene corresponding to the current time, and the audio player 112 may play audio information accordingly. In another example, the media content may be rendered based on the geographic location of the first user 330. The geographic location may be, for example, a particular beach, a rocky area, a mountain, a jungle, etc. In another example, the media content may be rendered based on one or more objects present in the room 302. For example, the circuit 202 may detect a plurality of objects in the room 302 and control the first display screen 312 to display media content based on the detected plurality of objects. If the detected object is a coffee mug in the room 302, the circuit 202 may display media content that may be related to the coffee mug. Such media content may include, for example, a morning scene, a cafe scene, or a restaurant scene. In another example, the circuit 202 may detect an object such as, but not limited to, a person, an ornament, a food, or a combination thereof. The circuit 202 may display media content for such an object such as, but not limited to, a party scene, a dance scene, or a romantic scene. Other examples of media content may include, but are not limited to, one or more pre-stored images, one or more pre-stored audio content, one or more pre-stored graphic / animation content, a first media content received from a user device, or a second media content received from a server (such as server 126).User device (not shown) may refer to a device associated with the first user 330, such as a mobile phone, a smart phone, a tablet computing device, a personal computer, a gaming console, a media player, a sound system, a smart audio device, a home theater system, a digital camera, or other consumer electronic device with media rendering capabilities.
[0056] In some embodiments, the circuit 202 may be further configured to receive environmental information related to an environment outside the living environment (such as the room 302) from one or more sensors (such as a light sensor 326 and a temperature sensor 328) associated with the selected second image capture device 308. The environmental information may include at least one of luminosity information, temperature information, or airflow information. The circuit 202 may be configured to control the operation of one or more electronic devices (such as an air conditioner 324 and a lighting device 322) in the room 302 based on the received environmental information. Examples of such electronic devices may include, but are not limited to, a lighting device, an electric fan, an air conditioning system, an electric heater, a thermostat device, an aroma diffuser device, or a heating, ventilation, and air conditioning (HVAC) system. For example, the lighting device 322 may adjust the luminosity of the room 302 based on the external environment. The electric fan (not shown) may adjust the airflow of the room 302 based on the airflow of the external environment. The air conditioner 324 may adjust the temperature of the room 302 based on the temperature of the external environment 304. An aroma diffuser (not shown) can emit a scent into the room 302 that matches the scent of the exterior environment 304. An HVAC system (not shown) can include one or more vents that can introduce hot air, cold air, and / or dehumidified air into the room 302.
[0057] In an embodiment, the circuit 202 can be configured to receive environmental information from a light sensor 326 or a temperature sensor 328 present in an environment outside the room 302. The environmental information can be received in the form of an analog or digital signal from one or more sensors (such as the light sensor 326 or the temperature sensor 328). The light sensor 326 can include suitable logic, circuitry, and / or interfaces that can be configured to detect luminosity information of the external environment 304. For example, as shown in FIG. 3A, the light sensor 326 can be located on the selected second image capture device 308 to detect luminosity information of the environment outside the room 302. The light sensor 326 can transmit a signal to the circuit 202 indicative of the luminosity of the environment outside the room 302. The circuit 202 can be configured to determine the luminosity of the external environment 304 based on the signal received from the light sensor 326. Examples of the light sensor 326 can include, but are not limited to, a photoresistor, a photodiode, a photocell, a phototransistor, or a photoelectric sensor.
[0058] The temperature sensor 328 may include suitable logic, circuitry, and / or interfaces that may be configured to detect temperature information of the external environment 304. For example, as shown in FIG. 3A, the temperature sensor 328 may be located on the selected second image capture device 308 to obtain temperature information of the environment outside the room 302. The temperature sensor 328 may transmit a signal indicative of the temperature (or time series data of temperature values) of the environment outside the room 302 to the circuit 202. The circuit 202 may be configured to determine the temperature of the external environment 304 based on the signal received from the temperature sensor 328. For example, the circuit 202 may receive temperature information as 22 degrees Celsius, and luminosity information as a categorical variable (e.g., dim light or low light) or a deterministic value (e.g., 800 lumens). The circuit 202 may be configured to control the air conditioner 324 to set the room temperature to 32° C. and turn on the lighting device 322 based on the received temperature and luminosity information.
[0059] In an embodiment, the circuit 202 may be further configured to determine an emotional state of the first user 330 based on the detected first facial region of the first user 330. The first facial region of the first user 330 may be detected based on the received one or more first images of the room 302. Examples of the emotional states may include, but are not limited to, a happy emotion, a sad emotion, an angry emotion, a calm emotion, a scared emotion, a neutral emotion, an excited emotion, a confused emotion, a stressed emotion, an annoyed emotion, a surprised emotion, an excited emotion, or a frightened emotion. The first facial region (e.g., face area) of the first user 330 may indicate an identity of the first user 330. In an embodiment, the memory 204 may be configured to store identification information (such as a name, a sample image, and / or a user profile) of the first user 330. The circuit 202 may recognize the first user 330 based on the detected first facial region and the stored identification information. Media content may be displayed on the first display screen 312 based on the recognized identity of the first user 330 .
[0060] The circuitry 202 may be configured to select a third image capture device (such as the third image capture device 310) from the plurality of image capture devices based on the determined emotional state of the first user 330. The circuitry 202 may then control the first display screen 312 to display one or more third images received from the third image capture device 310. For example, the circuitry 202 may determine the media content as "landscape movies" for a happy emotion, "standup-comedy episodes" for a sad emotion, "beach video blogs" for a calm emotion, and "party videos" for an exciting emotion.
[0061] In an embodiment, the circuit 202 can be further configured to receive a user input indicating a selection of an operation mode of the electronic device 106. The circuit 202 can be configured to control the operation of the electronic device 106 based on the selected operation mode. The user input can be received via an input device, such as a first display screen 312 associated with the electronic device 106 or a TV remote control, or a mobile device. The operation mode can include, for example, a TV mode or a windowed mode. The circuit 202 can be configured to control the operation of the electronic device 106 to switch to the TV mode or the windowed mode based on the user input.
[0062] In an embodiment, the circuitry 202 can be configured to detect a presence of a first user 330 within the living environment (such as the room 302) based on the received one or more first images. The circuitry 202 can be further configured to control a power saving mode of the electronic device 106 based on the detected presence of the first user 330. For example, the circuitry 202 can turn on a power saving mode of the electronic device 106 if it detects that the first user 330 is not present within the living environment (such as the room 302) based on the received one or more first images. The disclosed electronic device 106 can conserve power by turning on a power saving mode when no user is present within the living environment 102.
[0063] In an embodiment, the circuitry 202 may be configured to detect the presence of a first user 330 inside the living environment (such as the room 302) based on an occupancy signal. In particular, the circuitry 202 may be configured to receive an occupancy signal from a first image capture device 306. The occupancy signal may refer to an electrical or digital signal transmitted from the first image capture device 306 to the electronic device 106. The occupancy signal may indicate the presence of the first user 330 within the living environment 102.
[0064] In an embodiment, the circuit 202 can be configured to control a power saving mode of the electronic device 106 based on at least one of the tracked first head position or first line of sight of the first user 330. As an example, the circuit 202 may detect the presence of the first user 330 within the living environment (such as the room 302) but the first head position or first line of sight of the first user 330 may not change for a specified period of time. This may indicate that the first user 330 may be dozing or sleeping in the room 302. In such a case, the circuit 202 may be configured to turn on the power saving mode of the electronic device 106.
[0065] FIG. 4 illustrates an exemplary scenario for rendering content on a display in a living environment based on an environment outside the living environment, according to an embodiment of the present disclosure. FIG. 4 is described in relation to elements of FIG. 1, FIG. 2, FIG. 3A, and FIG. 3B. FIG. 4 illustrates an exemplary scenario 400. The exemplary scenario 400 illustrates a room 402, which may be an example of the living environment 102. Also illustrated is a first image capture device 404, which may be an example implementation of the first image capture device 108 of FIG. 1. Also illustrated are multiple display screens (such as a first display screen 406 and a second display screen 408), which may be an example implementation of the multiple display screens 110. Also illustrated are multiple electronic devices (such as a lighting device 416 and an air conditioner 418), which may be an example implementation of the multiple electronic devices 114.
[0066] Scenario 400 further illustrates a first user 420 who may be standing near a first display screen 406. Scenario 400 further illustrates a second user 422 who may be standing near a second display screen 408. Scenario 400 further illustrates walls 410, 412, and 414 surrounding a living environment (such as room 402). It is noted that lighting device 416 and air conditioner 418 are merely examples of electronic devices. Scenario 400 may be applied to other types of electronic devices without departing from the scope of this disclosure.
[0067] In an embodiment, the circuit 202 can be configured to receive one or more first images of the room 402 from a first image capture device 404 (which can be coupled to the electronic device 106 and disposed within the room 402). The circuit 202 can be configured to detect a first facial region of a first user 420 present within the room 402 based on the received one or more first images (e.g., as also described in FIG. 3A ). The circuit 202 can also be configured to detect a second facial region of a second user 422 present within the room 402 based on the received one or more first images.
[0068] The circuitry 202 may be configured to track at least one of a first head position or a first line of sight of the first user 420 based on a detected first facial region (e.g., as also described in FIG. 3A ). Similarly, the circuitry 202 may be configured to track at least one of a second head position or a second line of sight of the second user 422 based on a detected second facial region. The circuitry 202 may be configured to select a second image capture device (e.g., second image capture device 308) from the plurality of image capture devices 116 based on tracking at least one of a first head position or a first line of sight of the first user 420 (e.g., as also described in FIG. 3A ). Similarly, the circuitry 202 may be configured to select a third image capture device (e.g., third image capture device 310) from the plurality of image capture devices 116 based on tracking at least one of a second head position or a second line of sight of the second user 422.
[0069] The circuitry 202 can be configured to receive one or more second images from a selected second image capture device (e.g., as also described in FIG. 3A ). Similarly, the circuitry 202 can be configured to receive one or more third images from a selected third image capture device 310. The circuitry 202 can then be configured to control a first display screen 406 associated with the electronic device 106 (e.g., as also described in FIG. 3A ) to display the received one or more second images. Similarly, the circuitry 202 can be configured to control a second display screen 408 to display the one or more third images from the third image capture device 310. The electronic device 106 enables dynamic control of the display screen based on the viewing angle or head pose of each of the first user 420 and the second user 422.
[0070] Figure 5 is a flow chart illustrating an exemplary method for rendering an image of an external environment on a display inside a living environment, according to an embodiment of the present disclosure. Figure 5 is described with reference to elements of Figures 1, 2, 3A, 3B, and 4. Figure 5 illustrates a flow chart 500. The operations of flow chart 500 may be performed by a computer system, such as electronic device 106 of Figure 1 or 2. Operations may begin at 502 and proceed to 504.
[0071] At 504, one or more first images of the living environment (e.g., living environment 102) may be received. In one or more embodiments, the circuitry 202 may be configured to receive the one or more first images of the living environment 102 from a first image capture device 108 coupled to the electronic device 106 and disposed within the living environment 102, for example as described in FIG.
[0072] At 506, a first facial region of a first user (e.g., first user 124) present in the living environment 102 may be detected. In one or more embodiments, the circuit 202 may be configured to detect a first facial region of a first user 124 present in the living environment 102 based on one or more received first images, for example as described in FIG.
[0073] At 508, at least one of a first head position or a first gaze of the first user 124 may be tracked. In one or more embodiments, the circuitry 202 may be configured to track at least one of a first head position or a first gaze of the first user 124 based on the detected first face region, for example as described in FIG.
[0074] At 510, a second image capture device (e.g., second image capture device 122) may be selected from the plurality of image capture devices (e.g., plurality of image capture devices 116). In one or more embodiments, the circuitry 202 may be configured to select the second image capture device 122 from the plurality of image capture devices 116 based on at least one of the tracked first head position or first line of sight. For example, as described in FIG. 3A, the plurality of image capture devices 116 may be located outside the living environment 102.
[0075] At 512, one or more second images may be received from the selected second image capture device 122. In one or more embodiments, the circuit 202 may be configured to receive one or more second images from the selected second image capture device 122, for example, as described in FIG.
[0076] At 514, a first display screen (such as first display screen 110A) may be controlled. In one or more embodiments, the circuitry 202 may be configured to control the first display screen 110A associated with the electronic device 106 to display the received one or more second images. Control may proceed to an end.
[0077] Although flowchart 500 depicts discrete operations such as 504, 506, 508, 510, 512, and 514, the disclosure is not so limited. Thus, in some embodiments, such discrete operations may be further divided into additional operations, combined into fewer operations, or eliminated, depending on the particular implementation, without departing from the essence of the disclosed embodiments.
[0078] Various embodiments of the present disclosure may provide a non-transitory computer-readable medium and / or storage medium having instructions stored thereon that may be executed by a machine and / or computer (e.g., electronic device 106). The instructions may cause the machine and / or computer (e.g., electronic device 106) to perform operations including retrieving one or more first images of the living environment (e.g., living environment 102) from a first image capture device (e.g., first image capture device 108) that may be coupled to electronic device 106 and located within the living environment 102. The operations may further include detecting a first facial region of a first user (e.g., first user 124) present within the living environment 102 based on the received one or more first images. The operations may further include tracking at least one of a first head position or a first gaze of first user 124 based on the detected first facial region. The operations may further include selecting a second image capture device (second image capture device 122) from a plurality of image capture devices (e.g., a plurality of image capture devices 116) coupled to the electronic device 106 based on at least one of the tracked first head position or the first line of sight of the first user 124. The plurality of image capture devices 116 may be located outside the living environment 102. The operations may further include retrieving one or more second images from the selected second image capture device 122. The operations may further include controlling a first display screen (e.g., first display screen 110A) associated with the electronic device 106 to display the received one or more second images.
[0079] An exemplary aspect of the disclosure may include an electronic device (such as electronic device 106) that may include a circuit (such as circuit 202). Circuit 202 may be configured to receive one or more first images of a living environment (such as living environment 102) from a first image capture device (such as first image capture device 108) that may be coupled to electronic device 106 and located within living environment 102. Circuit 202 may be configured to detect a first facial region of a first user (such as first user 124) present within living environment 102 based on the received one or more first images. Circuit 202 may be configured to track at least one of a first head position or a first gaze of first user 124 based on the detected first facial region. The circuitry 202 may be configured to select a second image capture device (e.g., second image capture device 122) from a plurality of image capture devices (e.g., plurality of image capture devices 116) coupled to the electronic device 106 based on at least one of a tracked first head position or a first line of sight of the first user 124. The plurality of image capture devices 116 may be located outside the living environment 102. The circuitry 202 may be further configured to receive one or more second images from the selected second image capture device 122 and control a first display screen (e.g., first display screen 110A) associated with the electronic device 106 to display the received one or more second images.
[0080] According to one embodiment, the circuitry 202 may be further configured to control the selected second image capture device 122 to capture one or more second images of an environment outside the living environment 102 based on at least one of the tracked first head position or first line of sight of the first user 124.
[0081] According to an embodiment, the circuitry 202 may be further configured to determine a first viewing direction of the first user 124 based on at least one of the tracked first head position or first line of sight. The viewing direction may be towards a wall (such as wall 318) of the living environment (such as room 302), on which the selected second image capture device 308 may be located.
[0082] According to an embodiment, the circuit 202 may be further configured to detect a second facial region of a second user (e.g., second user 422) present within the living environment (e.g., room 402) based on the received one or more first images. The circuit 202 may be configured to track at least one of a second head position or a second line of sight of the second user 422 based on the detected second facial region. The circuit 202 may be further configured to select a third image capture device from the plurality of image capture devices 116 based on at least one of the tracked second head position or the second line of sight of the second user 422. The circuit 202 may be configured to control the first display screen 110A to display the received one or more second images and one or more third images received from the third image capture device.
[0083] According to one embodiment, the circuitry 202 may be further configured to select a first display screen 110A from multiple display screens (e.g., multiple display screens 110) based on at least one of a first head position or a first line of sight of the tracked first user 124.
[0084] According to an embodiment, the multiple image capture devices 116 may be associated with multiple audio capture devices (such as the multiple audio capture devices 118) located outside the living environment 102. The circuit 202 may be configured to control a first audio capture device (such as the first audio capture device 118A) associated with the selected second image capture device 122 to capture first audio present in an environment outside the living environment 102. The circuit 202 may be further configured to receive first audio information from the first audio capture device 118A related to the captured first audio. The circuit 202 may be further configured to control an audio playback device (such as the audio playback device 112) associated with the electronic device 106 to play the first audio based on the received first audio information.
[0085] According to an embodiment, the circuitry 202 may be further configured to receive environmental information related to an environment outside the living environment 102 from one or more sensors (e.g., the plurality of sensors 120) associated with the selected second image capture device 122. The environmental information may include at least one of luminosity information, temperature information, or airflow information. The circuitry 202 may be further configured to control operation of one or more electronic devices (e.g., one or more electronic devices 114) in the living environment 102 based on the received environmental information.
[0086] According to one embodiment, the one or more electronic devices 114 may include at least one of a lighting device, an electric fan or air conditioning system, an electric heater, a thermostat device, an aroma diffuser device, or a heating, ventilation, and air conditioning (HVAC) system.
[0087] According to an embodiment, the circuitry 202 may be further configured to control a first display screen 110A or an audio player 112 associated with the electronic device 106 to render media content based on at least one of a current time, a geographic location of the first user 124, or one or more objects present in the living environment 102. According to an embodiment, the media content may include at least one of one or more pre-stored images, one or more pre-stored audio content, one or more pre-stored graphic / animation content, a first media content received from a user device, or a second media content received from a server (e.g., server 126).
[0088] According to an embodiment, the circuitry 202 may be further configured to determine an emotional state of the first user 124 based on the detected facial region of the first user 124. The circuitry 202 may be configured to select a third image capture device from the plurality of image capture devices 116 based on the determined emotional state of the first user 124. The circuitry 202 may be further configured to control the first display screen 110A to display one or more third images received from the third image capture device.
[0089] According to an embodiment, the circuitry 202 can be further configured to receive a user input indicating a selection of an operational mode of the electronic device 106. The circuitry 202 can be further configured to control an operation of the electronic device 106 based on the selected operational mode.
[0090] According to an embodiment, the circuitry 202 may be further configured to detect a presence of the first user 124 in the living environment 102 based on the received one or more first images. The circuitry 202 may be further configured to control a power saving mode of the electronic device 106 based on the detected presence of the first user 124. According to an embodiment, the circuitry 202 may be further configured to control a power saving mode of the electronic device 106 based on at least one of a tracked first head position or a first line of sight of the first user 124.
[0091] The present disclosure can be realized in hardware or in a combination of hardware and software. The present disclosure can be realized in a centralized manner in at least one computer system, or in a distributed manner where different elements can be distributed across several interconnected computer systems. Any computer system or other device adapted to perform the methods described herein can be suitable. The combination of hardware and software can be a general-purpose computer system that includes a computer program that, when loaded and executed, can control a computer system to perform the methods described herein. The present disclosure can be realized in hardware, including parts of an integrated circuit that also perform other functions.
[0092] The present disclosure may also be embodied in a computer program product, which includes all features enabling the implementation of the methods described herein and which is capable of executing these methods when loaded into a computer system. A computer program in this context means any expression, in any language, code or notation, of a set of instructions intended to cause a system capable of processing information to execute a particular function, either directly, or after a) conversion into another language, code or notation, b) reproduction in a different content form, or both.
[0093] While the present disclosure has been described with reference to several embodiments, those skilled in the art will recognize that various modifications can be made and equivalents substituted without departing from the scope of the present disclosure. Many modifications can also be made to adapt a particular situation or material to the teachings of the present disclosure without departing from the scope of the present disclosure. Therefore, it is not intended that the present disclosure be limited to the particular embodiments disclosed, but rather, it is intended to include all embodiments falling within the scope of the appended claims. [Explanation of symbols]
[0094] 300A Scenario 302 rooms 304 External environment 306 First image capture device 308 Second Image Capture Device 310 Third image capture device 312 first display screen 314 Secondary Display Screen 316 Wall 318 Wall 320 Wall 322 Lighting Equipment 324 Air Conditioner 326 Optical Sensor 328 Temperature Sensor 330 First User
Claims
1. 1. An electronic device comprising: receiving one or more first images of the living environment from a first image capture device within the living environment; Detecting a first face region of a first user present within the living environment based on the received one or more first images; Tracking at least one of a first head position or a first line of sight of the first user based on the detected first face region; selecting a second image capture device from a plurality of image capture devices present outside the living environment and communicatively coupled to the electronic device based on at least one of the tracked first head position or the first line of sight; receiving one or more second images from the selected second image capture device; controlling a first display screen within the living environment associated with the electronic device to display the received one or more second images; The circuit is configured to 1. An electronic device comprising:
2. the circuitry is further configured to control the selected second image capture device to capture one or more second images of an environment outside the living environment based on at least one of the tracked first head position or the first line of sight of the first user.
2. The electronic device of claim 1.
3. the circuitry is further configured to determine a first viewing direction of the first user based on at least one of the tracked first head position or the first line of sight, the viewing direction being toward a wall of the living environment, the selected second image capture device being located on the wall.
2. The electronic device of claim 1.
4. The circuit comprises: detecting a second face region of a second user present in the living environment based on the received one or more first images; Tracking at least one of a second head position or a second line of sight of the second user based on the detected second face region; selecting a third image capture device from the plurality of image capture devices based on at least one of the tracked second head position or the second line of sight of the second user; controlling the first display screen to display the received one or more second images and one or more third images received from the third image capture device; The electronic device of claim 1 , further configured to:
5. the circuitry is further configured to select the first display screen from a plurality of display screens based on at least one of the tracked first head position or the first line of sight of the first user.
2. The electronic device of claim 1.
6. the plurality of image capture devices are associated with a plurality of audio capture devices located outside the living environment, the circuit comprising: controlling a first audio capture device associated with the selected second image capture device to capture first audio present in an environment external to the living environment; receiving first audio information from the first audio capture device, the first audio information being related to the captured first audio; controlling an audio playback device associated with the electronic device to play the first audio based on the received first audio information; The electronic device of claim 1 , further configured to:
7. The circuit comprises: receiving environmental information from one or more sensors associated with the selected second image capture device, the environmental information including at least one of light intensity information, temperature information, or airflow information associated with an environment outside the living environment; controlling operation of one or more electronic devices within the living environment based on the received environmental information; The electronic device of claim 1 , further configured to:
8. the one or more electronic devices include at least one of a lighting device, an electric fan or air conditioning system, an electric heater, a thermostat device, an aroma diffuser device, or a heating, ventilation and air conditioning (HVAC) system; 8. The electronic device of claim 7.
9. the circuitry is further configured to control the first display screen or an audio playback device associated with the electronic device to render media content based on at least one of a current time, a geographic location of the first user, or one or more objects present within the living environment.
2. The electronic device of claim 1.
10. the media content includes at least one of one or more pre-stored images, one or more pre-stored audio content, one or more pre-stored graphic / animation content, a first media content received from a user device, or a second media content received from a server; 10. The electronic device of claim 9.
11. The circuit comprises: determining an emotional state of the first user based on the detected facial region of the first user; selecting a third image capture device from the plurality of image capture devices based on the determined emotional state of the first user; controlling the first display screen to display one or more third images received from the third image capture device; The electronic device of claim 1 , further configured to:
12. The circuit comprises: receiving a user input indicating a selection of an operational mode of the electronic device; controlling an operation of the electronic device based on the selected operating mode. The electronic device of claim 1 , further configured to:
13. The circuit comprises: Detecting a presence of the first user in the living environment based on the received one or more first images; controlling a power saving mode of the electronic device based on the detected presence of the first user. The electronic device of claim 1 , further configured to:
14. the circuitry is further configured to control a power saving mode of the electronic device based on at least one of the tracked first head position or the first line of sight of the first user.
2. The electronic device of claim 1.
15. In an electronic device, receiving one or more first images of the living environment from a first image capture device within the living environment; detecting a first face region of a first user present within the living environment based on the received one or more first images; Tracking at least one of a first head position or a first gaze of the first user based on the detected first face region; selecting a second image capture device from a plurality of image capture devices present outside the living environment and communicatively coupled to the electronic device based on at least one of the tracked first head position or the first line of sight; receiving one or more second images from the selected second image capture device; controlling a first display screen within the living environment associated with the electronic device to display the received one or more second images; The method according to claim 1, further comprising:
16. and controlling the selected second image capture device to capture one or more second images of an environment outside the living environment based on at least one of the tracked first head position or the first line of sight of the first user. The method of claim 15.
17. determining a first viewing direction of the first user based on at least one of the tracked first head position or the first line of sight, the viewing direction being toward a wall of the living environment, the selected second image capture device being located on the wall. The method of claim 15.
18. determining an emotional state of the first user based on the detected facial region of the first user; selecting a third image capture device from the plurality of image capture devices based on the determined emotional state of the first user; controlling the first display screen to display one or more third images received from the third image capture device; The method of claim 15 further comprising:
19. receiving environmental information from one or more sensors associated with the selected second image capture device, the environmental information including at least one of light intensity information, temperature information, or airflow information associated with an environment outside the living environment; controlling operation of one or more electronic devices within the living environment based on the received environmental information; The method of claim 15 further comprising:
20. A non-transitory computer-readable medium having computer-executable instructions stored thereon, the computer-executable instructions, when executed by an electronic device, causing the electronic device to: receiving one or more first images of the living environment from a first image capture device within the living environment; detecting a first face region of a first user present within the living environment based on the received one or more first images; Tracking at least one of a first head position or a first gaze of the first user based on the detected first face region; selecting a second image capture device from a plurality of image capture devices present outside the living environment and communicatively coupled to the electronic device based on at least one of the tracked first head position or the first line of sight; receiving one or more second images from the selected second image capture device; controlling a first display screen within the living environment associated with the electronic device to display the received one or more second images; A non-transitory computer-readable medium for causing a computer to perform operations including:
Citation Information
Patent Citations
Device for analyzing human action and recording medium for recording human action analytic program
JP2001008197A
Virtual window system
JP2006211068A
Video display device, video display control method and program
JP2016066918A
System and method for creating and manipulating synthetic environments
US20100157063A1
