Display device and method for controlling same

Abstract

The present disclosure relates to a display device. The display device according to the present disclosure comprises: a display; a radar sensor positioned on one side of the display to remotely monitor a user's movements using radio waves; and a controller including one or more processors that receive and process data from the radar sensor, wherein the controller may include: a radar processing unit that processes data received from the radar sensor; a sleep state detection unit that processes the data to detect a sleep state of the user; and a health state detection unit that processes the data to detect a health state of the user.

Description

Display device and control method thereof

[0001] The present disclosure relates to a display device.

[0002] Digital TV services utilizing wired or wireless networks are becoming more widespread. Digital TV services can offer a variety of services not available with existing analog broadcasting services.

[0003] For example, IPTV (Internet Protocol Television), a type of digital TV service, and smart TV services offer interactivity, allowing users to actively choose the type of program they want to watch and when. Building on this interactivity, IPTV and smart TV services can also offer a variety of additional services, such as internet search, home shopping, and online games.

[0004] TVs are equipped with ToF sensors, which provide presence / absence information around the TV, and provide location information in specific areas. ToF (Time of Flight) sensors emit light and measure the time it takes for the light to reflect and return, calculating distance.

[0005] Therefore, there were many restrictions on the operation of monitoring services for users.

[0006] The present disclosure seeks to provide a display device and a control method thereof that can monitor at least one of a viewer's sleep state and health state by applying radar sensor technology.

[0007] The present disclosure provides a display device and a control method thereof that can perform an appropriate response process according to a viewer's monitoring status.

[0008] The present disclosure provides a display device and a control method thereof that can provide intelligent motion sensing functions such as automatic door opening and closing, security alarm, and fall detection by applying radar sensor technology.

[0009] A display device according to the present disclosure includes a display; a radar sensor located on one side of the display for remotely monitoring a user's movements using radio waves; a controller including one or more processors for receiving and processing data from the radar sensor, wherein the controller may include a radar processing unit for processing data received from the radar sensor; a sleep state detection unit for processing the data to detect a sleep state of the user; and a health state detection unit for processing the data to detect a health state of the user.

[0010] A method for controlling a display device including a radar sensor according to the present disclosure, the method comprising: receiving data about a user from the radar sensor; performing situational awareness monitoring based on the data; and providing a first response process based on the situational awareness monitoring, wherein the situational awareness monitoring may include at least one of determining movement of the user, collecting biometric data of the user, and determining presence of the user.

[0011] According to at least one of the various embodiments of the present disclosure, information and data can be provided by monitoring at least one of a viewer's sleep state and health state.

[0012] According to at least one of the various embodiments of the present disclosure, an appropriate response process such as content recommendation, notification provision, or status change can be performed based on the viewer's monitoring status.

[0013] According to at least one of the various embodiments of the present disclosure, additional response processes, such as providing emergency action guidance or analyzing the health status of the user, may be performed as the response process progresses.

[0014] According to at least one of the various embodiments of the present disclosure, radar sensor technology can be applied to provide intelligent motion sensing functions such as automatic door opening and closing, security alarm, and fall detection.

[0015] In addition to the effects described above, the specific effects of the present disclosure are described below together with specific details.

[0016] Figure 1 is a block diagram illustrating the configuration of a display device according to one embodiment of the present invention.

[0017] Figure 2 is a block diagram of a remote control device according to one embodiment of the present invention.

[0018] Figure 3 shows an example of an actual configuration of a remote control device according to one embodiment of the present invention.

[0019] Figure 4 shows an example of utilizing a remote control device according to an embodiment of the present invention.

[0020] FIG. 5 is a block diagram showing a detailed configuration of a display device according to one embodiment of the present invention.

[0021] FIG. 6 is a block diagram showing the detailed configuration and operation thereof of a display device according to one embodiment of the present invention.

[0022] Figure 7 is an operational flowchart showing a control method of a display device according to one embodiment of the present invention.

[0023] FIG. 8 is an operation flowchart showing an intelligent sensing service using a radar sensor according to one embodiment of the present invention.

[0024] Hereinafter, embodiments related to the present invention will be described in more detail with reference to the drawings. The suffixes "module" and "part" used in the following description for components are assigned or used interchangeably solely for the convenience of writing the specification, and do not in themselves have distinct meanings or roles.

[0025] A display device according to an embodiment of the present invention is, for example, an intelligent display device that adds computer-assisted functionality to its broadcast reception function. While faithfully performing the broadcast reception function, it can also be equipped with Internet functionality and other features, providing a more user-friendly interface, such as a manual input device, touch screen, or space remote control. Furthermore, with support for wired or wireless Internet functionality, it can connect to the Internet and a computer, enabling functions such as email, web browsing, banking, or gaming. A standardized, general-purpose operating system can be used for these various functions.

[0026] Accordingly, the display device described in the present invention can perform various user-friendly functions, for example, by allowing various applications to be freely added or deleted on a general-purpose operating system kernel. More specifically, the display device can be a network TV, HBB TV, smart TV, LED TV, OLED TV, etc., and in some cases, it can also be applied to smartphones.

[0027] FIG. 1 is a block diagram illustrating the configuration of a display device according to one embodiment of the present invention.

[0028] Referring to FIG. 1, the display device (100) may include a broadcast receiving unit (130), an external device interface (135), a memory (140), a user input interface (150), a controller (170), a wireless communication interface (173), a display (180), a speaker (185), and a power supply circuit (190).

[0029] The broadcast receiving unit (130) may include a tuner (131), a demodulator (132), and a network interface (133).

[0030] The tuner (131) can select a specific broadcast channel according to a channel selection command. The tuner (131) can receive a broadcast signal for the selected specific broadcast channel.

[0031] The demodulator (132) can separate the received broadcast signal into a video signal, an audio signal, and a data signal related to the broadcast program, and can restore the separated video signal, audio signal, and data signal into a form that can be output.

[0032] The external device interface (135) can receive an application or a list of applications within an adjacent external device and transmit it to the controller (170) or memory (140).

[0033] The external device interface (135) can provide a connection path between the display device (100) and the external device. The external device interface (135) can receive one or more of images and audio output from an external device connected wirelessly or wiredly to the display device (100) and transmit them to the controller (170). The external device interface (135) can include a plurality of external input terminals. The plurality of external input terminals can include an RGB terminal, one or more HDMI (High Definition Multimedia Interface) terminals, and a component terminal.

[0034] A video signal of an external device input through an external device interface (135) can be output through a display (180). A voice signal of an external device input through an external device interface (135) can be output through a speaker (185).

[0035] An external device that can be connected to the external device interface (135) may be any one of a set-top box, a Blu-ray player, a DVD player, a game console, a sound bar, a smartphone, a PC, a USB memory, and a home theater, but this is only an example.

[0036] The network interface (133) may provide an interface for connecting the display device (100) to a wired / wireless network including the Internet. The network interface (133) may transmit or receive data to or from other users or other electronic devices via the connected network or another network linked to the connected network.

[0037] Additionally, some content data stored in the display device (100) can be transmitted to a selected user or electronic device among other users or other electronic devices pre-registered in the display device (100).

[0038] The network interface (133) can access a predetermined web page through a connected network or another network linked to the connected network. That is, by accessing a predetermined web page through a network, data can be transmitted or received with the corresponding server.

[0039] In addition, the network interface (133) can receive content or data provided by a content provider or network operator. That is, the network interface (133) can receive content such as movies, advertisements, games, VOD, broadcast signals, etc. and information related thereto provided from a content provider or network provider via a network.

[0040] Additionally, the network interface (133) can receive firmware update information and update files provided by the network operator, and can transmit data to the Internet or content provider or network operator.

[0041] The network interface (133) can select and receive a desired application from among applications open to the public via a network.

[0042] The memory (140) stores a program for each signal processing and control within the controller (170), and can store signal-processed image, voice, or data signals.

[0043] In addition, the memory (140) may perform a function for temporary storage of video, audio, or data signals input from an external device interface (135) or a network interface (133), and may store information about a specific image through a channel memory function.

[0044] The memory (140) can store an application or a list of applications input from an external device interface (135) or a network interface (133).

[0045] The display device (100) can play content files (video files, still image files, music files, document files, application files, etc.) stored in the memory (140) and provide them to the user.

[0046] The user input interface (150) can transmit a signal input by the user to the controller (170) or transmit a signal from the controller (170) to the user. For example, the user input interface (150) can receive and process control signals such as power on / off, channel selection, and screen setting from the remote control device (200) according to various communication methods such as Bluetooth, Ultra Wideband (WB), ZigBee, Radio Frequency (RF) communication, or infrared (IR) communication, or process control signals from the controller (170) to be transmitted to the remote control device (200).

[0047] In addition, the user input interface (150) can transmit control signals input from local keys (not shown) such as a power key, channel key, volume key, and setting value to the controller (170).

[0048] An image signal processed by the controller (170) can be input to the display (180) and displayed as an image corresponding to the image signal. In addition, an image signal processed by the controller (170) can be input to an external output device through an external device interface (135).

[0049] The voice signal processed by the controller (170) can be output as audio to the speaker (185). In addition, the voice signal processed by the controller (170) can be input to an external output device through the external device interface (135).

[0050] In addition, the controller (170) can control the overall operation within the display device (100).

[0051] In addition, the controller (170) can control the display device (100) by a user command or internal program input through the user input interface (150), and can connect to a network to enable the user to download a desired application or application list into the display device (100).

[0052] The controller (170) enables the user-selected channel information, etc. to be output through a display (180) or speaker (185) together with processed video or audio signals.

[0053] In addition, the controller (170) allows a video signal or audio signal from an external device, for example, a camera or camcorder, input through the external device interface (135) to be output through the display (180) or speaker (185) in accordance with an external device video playback command received through the user input interface (150).

[0054] Meanwhile, the controller (170) can control the display (180) to display an image, for example, a broadcast image input through a tuner (131), an external input image input through an external device interface (135), an image input through a network interface, or an image stored in a memory (140) can be controlled to be displayed on the display (180). In this case, the image displayed on the display (180) can be a still image or a moving image, and can be a 2D image or a 3D image.

[0055] In addition, the controller (170) can control the playback of content stored in the display device (100), received broadcast content, or external input content input from outside, and the content can be in various forms such as broadcast video, external input video, audio file, still image, connected web screen, and document file.

[0056] The wireless communication interface (173) can communicate with an external device through wired or wireless communication. The wireless communication interface (173) can perform short-range communication with the external device. To this end, the wireless communication interface (173) can support short-range communication using at least one of Bluetooth™, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies. This wireless communication interface (173) can support wireless communication between the display device (100) and a wireless communication system, between the display device (100) and another display device (100), or between the display device (100) and a network in which the display device (100, or an external server) is located via a short-range wireless communication network (Wireless Area Network). The short-range wireless communication network can be a short-range wireless personal area network (Wireless Personal Area Network).

[0057] Here, the other display device (100) may be a wearable device (e.g., a smartwatch, smart glasses, a head-mounted display (HMD)) or a mobile terminal such as a smart phone that can exchange data with (or be linked to) the display device (100) according to the present invention. The wireless communication interface (173) may detect (or recognize) a wearable device capable of communication around the display device (100).

[0058] Furthermore, if the detected wearable device is a device certified to communicate with the display device (100) according to the present invention, the controller (170) can transmit at least a portion of the data processed in the display device (100) to the wearable device via the wireless communication interface (173). Accordingly, a user of the wearable device can utilize the data processed in the display device (100) via the wearable device.

[0059] The display (180) can generate a driving signal by converting a video signal, data signal, OSD signal processed by the controller (170) or a video signal, data signal, etc. received from an external device interface (135) into R, G, and B signals, respectively.

[0060] Meanwhile, the display device (100) may be equipped with a radar sensor (160). This radar sensor (160) is located on one side of the display (180), includes at least one antenna, and can remotely monitor the user's movements using radio waves.

[0061] Such radar sensors (160) can use wavelengths in the millimeter (mm) band. For example, the radar sensor (160) may be a millimeter wave radar sensor (mmWave Radar Sensor).

[0062] For example, mmWave radar sensors (160) are an alternative sensing technology that can detect and track health and behavior in the home. Radar sensors (160) can help address a variety of sensing challenges, such as determining whether and how many people are in a room, identifying motion signatures that can detect falls, and measuring vital signs to assess factors like sleep quality.

[0063] Furthermore, the mmWave radar sensor (160) is inherently radio frequency-based, enabling detection without physical contact. Because it does not provide any visually identifiable information, it can be installed even in spaces where privacy is a concern. Combining these features at the application level can enable home monitoring systems to ensure users are safe and healthy.

[0064] Meanwhile, the display device (100) may further include an additional environmental sensor (165). This environmental sensor (165) may include at least one of an ambient light sensor, a microphone, and a temperature sensor.

[0065] The controller (170) may have one or more processors that receive and process data from the radar sensor (160). In addition, the controller (170) may control the environmental sensor (165).

[0066] By controlling these radar sensors (160) and / or environmental sensors (165), the controller (170) can check the user's sleep state and / or the user's health state.

[0067] The specific process of checking the user's sleep state and / or health state by controlling at least one of the radar sensor (160) and the environmental sensor (165) by such a controller (170) will be described in detail later.

[0068] Meanwhile, since the display device (100) illustrated in FIG. 1 is merely an embodiment of the present invention, some of the illustrated components may be integrated, added, or omitted depending on the specifications of the display device (100) actually implemented.

[0069] That is, two or more components may be combined into a single component, or a single component may be subdivided into two or more components, as needed. Furthermore, the functions performed by each block are intended to illustrate embodiments of the present invention, and their specific operations or devices do not limit the scope of the present invention.

[0070] According to another embodiment of the present invention, the display device (100) may receive and play back an image through a network interface (133) or an external device interface (135) without having a tuner (131) and a demodulator (132), unlike that shown in FIG. 1.

[0071] For example, the display device (100) may be implemented separately as an image processing device, such as a set-top box, for receiving broadcast signals or contents according to various network services, and a content playback device for playing contents input from the image processing device.

[0072] In this case, the operating method of the display device according to the embodiment of the present invention described below may be performed by any one of the display device (100) described with reference to FIG. 1, as well as an image processing device such as the separated set-top box, or a content playback device having a display (180) and an audio output unit (185).

[0073] Next, a remote control device according to an embodiment of the present invention will be described with reference to FIGS. 2 and 3.

[0074] FIG. 2 is a block diagram of a remote control device according to an embodiment of the present invention, and FIG. 3 shows an example of an actual configuration of a remote control device (200) according to an embodiment of the present invention.

[0075] First, a state in which at least one of a radar sensor (160) and an environmental sensor (165) is provided in a display device (100) is briefly illustrated. The radar sensor (160) and / or the environmental sensor (165) may be positioned at the top of the display device (100), but the embodiment of the present invention is not limited thereto.

[0076] Meanwhile, referring to FIG. 2, the remote control device (200) may include a fingerprint recognition device (210), a wireless communication circuit (220), a user input interface (230), a sensor (240), an output interface (250), a power supply circuit (260), a memory (270), a controller (280), and a microphone (290).

[0077] Referring to FIG. 2, the wireless communication circuit (220) transmits and receives signals with any one of the display devices according to the embodiments of the present invention described above.

[0078] The remote control device (200) may be equipped with an RF circuit (221) capable of transmitting and receiving signals with the display device (100) according to RF communication standards, and an IR circuit (223) capable of transmitting and receiving signals with the display device (100) according to IR communication standards. In addition, the remote control device (200) may be equipped with a Bluetooth circuit (225) capable of transmitting and receiving signals with the display device (100) according to Bluetooth communication standards. In addition, the remote control device (200) may be equipped with an NFC circuit (227) capable of transmitting and receiving signals with the display device (100) according to NFC (Near Field Communication) communication standards, and a WLAN circuit (229) capable of transmitting and receiving signals with the display device (100) according to WLAN (Wireless LAN) communication standards.

[0079] In addition, the remote control device (200) transmits a signal containing information about the movement of the remote control device (200) to the display device (100) through a wireless communication circuit (220).

[0080] Meanwhile, the remote control device (200) can receive a signal transmitted by the display device (100) through the RF circuit (221), and, if necessary, can transmit commands for power on / off, channel change, volume change, etc. to the display device (100) through the IR circuit (223).

[0081] The user input interface (230) may be configured as a keypad, buttons, a touchpad, or a touch screen. The user can input commands related to the display device (100) to the remote control device (200) by operating the user input interface (230). If the user input interface (230) includes a hard key button, the user can input commands related to the display device (100) to the remote control device (200) by pushing the hard key button. This will be described with reference to FIG. 3.

[0082] Referring to FIG. 3, the remote control device (200) may include a plurality of buttons. The plurality of buttons may include a fingerprint recognition button (212), a power button (231), a home button (232), a live button (233), an external input button (234), a volume control button (235), a voice recognition button (236), a channel change button (237), a confirmation button (238), and a back button (239).

[0083] The fingerprint recognition button (212) may be a button for recognizing a user's fingerprint. In one embodiment, the fingerprint recognition button (212) may be capable of a push operation, and may receive a push operation and a fingerprint recognition operation.

[0084] The power button (231) may be a button for turning the power of the display device (100) on / off.

[0085] The home button (232) may be a button for moving to the home screen of the display device (100).

[0086] The live button (233) may be a button for displaying a real-time broadcast program.

[0087] The external input button (234) may be a button for receiving an external input connected to the display device (100).

[0088] The volume control button (235) may be a button for adjusting the size of the volume output by the display device (100).

[0089] The voice recognition button (236) may be a button for receiving a user's voice and recognizing the received voice.

[0090] The channel change button (237) may be a button for receiving a broadcast signal of a specific broadcast channel.

[0091] The confirmation button (238) may be a button for selecting a specific function, and the back button (239) may be a button for returning to the previous screen.

[0092] Let's explain Figure 2 again.

[0093] When the user input interface (230) has a touch screen, the user can input commands related to the display device (100) using the remote control device (200) by touching the soft keys of the touch screen. In addition, the user input interface (230) may have various types of input means that can be operated by the user, such as a scroll key or a jog key, and this embodiment does not limit the scope of the present invention.

[0094] The sensor (240) may include a gyro sensor (241) or an acceleration sensor (243), and the gyro sensor (241) may sense information about the movement of the remote control device (200).

[0095] For example, the gyro sensor (241) can sense information about the operation of the remote control device (200) based on the x, y, and z axes, and the acceleration sensor (243) can sense information about the movement speed of the remote control device (200). Meanwhile, the remote control device (200) can further include a distance measuring sensor, so as to sense the distance to the display (180) of the display device (100).

[0096] The output interface (250) can output a video or audio signal corresponding to an operation of the user input interface (230) or a signal transmitted from the display device (100).

[0097] The user can recognize whether the output interface (250) is manipulating the user input interface (230) or controlling the display device (100).

[0098] For example, the output interface (250) may include an LED (251) that lights up when the user input interface (230) is operated or a signal is transmitted and received with the display device (100) via the wireless communication unit (225), a vibrator (253) that generates vibrations, a speaker (255) that outputs sound, or a display (257) that outputs images.

[0099] In addition, the power supply circuit (260) supplies power to the remote control device (200), and power waste can be reduced by stopping the power supply when the remote control device (200) does not move for a predetermined period of time.

[0100] The power supply circuit (260) can resume power supply when a predetermined key provided in the remote control device (200) is operated.

[0101] The memory (270) can store various types of programs, application data, etc. required for the control or operation of the remote control device (200).

[0102] When the remote control device (200) wirelessly transmits and receives signals through the display device (100) and the RF circuit (221), the remote control device (200) and the display device (100) transmit and receive signals through a predetermined frequency band.

[0103] The controller (280) of the remote control device (200) can store and reference information regarding the frequency band that can wirelessly transmit and receive signals with the display device (100) paired with the remote control device (200) in the memory (270).

[0104] The controller (280) controls all matters related to the control of the remote control device (200). The controller (280) can transmit a signal corresponding to a predetermined key operation of the user input interface (230) or a signal corresponding to the movement of the remote control device (200) sensed by the sensor (240) to the display device (100) via the wireless communication unit (225).

[0105] Additionally, the microphone (290) of the remote control device (200) can acquire voice.

[0106] A plurality of microphones (290) may be provided.

[0107] Next, Figure 4 is described.

[0108] Figure 4 shows an example of utilizing a remote control device according to an embodiment of the present invention.

[0109] Figure 4 (a) illustrates that a pointer (205) corresponding to a remote control device (200) is displayed on a display (180).

[0110] The user can move or rotate the remote control device (200) up and down, left and right. The pointer (205) displayed on the display (180) of the display device (100) corresponds to the movement of the remote control device (200). This remote control device (200) can be called a space remote control because, as shown in the drawing, the pointer (205) moves and is displayed according to the movement in 3D space.

[0111] Figure 4 (b) illustrates that when a user moves the remote control device (200) to the left, the pointer (205) displayed on the display (180) of the display device (100) also moves to the left correspondingly.

[0112] Information about the movement of the remote control device (200) detected through the sensor of the remote control device (200) is transmitted to the display device (100). The display device (100) can calculate the coordinates of the pointer (205) from the information about the movement of the remote control device (200). The display device (100) can display the pointer (205) to correspond to the calculated coordinates.

[0113] Figure 4 (c) illustrates a case where a user moves the remote control device (200) away from the display (180) while pressing a specific button within the remote control device (200). As a result, a selection area within the display (180) corresponding to the pointer (205) can be zoomed in and displayed in an enlarged manner.

[0114] Conversely, when the user moves the remote control device (200) closer to the display (180), the selection area within the display (180) corresponding to the pointer (205) may be zoomed out and displayed in a reduced size.

[0115] Meanwhile, when the remote control device (200) moves away from the display (180), the selection area may be zoomed out, and when the remote control device (200) moves closer to the display (180), the selection area may be zoomed in.

[0116] Additionally, when a specific button within the remote control device (200) is pressed, recognition of up, down, left, and right movements may be excluded. That is, when the remote control device (200) moves away from or toward the display (180), up, down, left, and right movements may not be recognized, and only forward and backward movements may be recognized. When a specific button within the remote control device (200) is not pressed, only the pointer (205) moves in accordance with the up, down, left, and right movements of the remote control device (200).

[0117] Meanwhile, the movement speed or movement direction of the pointer (205) can correspond to the movement speed or movement direction of the remote control device (200).

[0118] Meanwhile, the pointer in this specification refers to an object displayed on the display (180) in response to the operation of the remote control device (200). Accordingly, objects of various shapes other than the arrow shape illustrated in the drawing can be used as the pointer (205). For example, the pointer may be a concept including a point, a cursor, a prompt, a thick outline, etc. In addition, the pointer (205) may be displayed corresponding to one point on the horizontal or vertical axis on the display (180), or may be displayed corresponding to multiple points such as lines or surfaces.

[0119] FIG. 5 is a block diagram showing a detailed configuration of a display device according to one embodiment of the present invention.

[0120] Referring to FIG. 5, the display device (100) may include a radar sensor (160) located on one side of the display (180) to remotely monitor the user's movements using radio waves. Although not shown separately, the radar sensor (160) may include at least one antenna.

[0121] The radar sensor (160) may include a radar transmitter (161) that transmits a first signal having a radar pulse, a radar receiver (162) that receives a second signal reflected from the measurement target and related to the first signal, and a signal processor (163) that processes a signal related to the bio-information of the measurement target based on the first and second signals transmitted and received from the radar transmitter (161) and the radar receiver (162).

[0122] At this time, the signal processing unit (163) can extract a frequency based on the first and second signals, and process a signal related to the bio-information of the measurement target according to the phase difference of the extracted frequency.

[0123] The display device (100) may further include an environmental sensor (165) for additional data processing along with data collected from the radar sensor (160).

[0124] These environmental sensors (165) may include at least one of an ambient light sensor, a microphone, and a temperature sensor, but may also include other sensors.

[0125] Additionally, the display device (100) may include a controller (170) including one or more processors that receive and process data from a radar sensor (160). This controller (170) may have the same configuration as the controller (170) described above with reference to FIG. 1.

[0126] This controller (170) may include a radar processing unit (1710) that processes data received from a radar sensor (160), a sleep state detection unit (1720) that processes the data to detect a user's sleep state, and a health state detection unit (1730) that processes the data to detect a user's health state.

[0127] FIG. 6 is a block diagram showing the detailed configuration and operation thereof of a display device according to one embodiment of the present invention.

[0128] Referring to FIG. 6, an example of a detailed configuration and corresponding operation within the controller (170) is illustrated. For example, data processed by a radar processing unit (1710) that processes data received from a radar sensor (160) may be transmitted to a sleep state detection unit (1720) and a health state detection unit (1730).

[0129] The controller (170) can perform situational awareness monitoring of a user or viewer (hereinafter collectively referred to as a user) using at least one of a sleep state detection unit (1720) and a health state detection unit (1730).

[0130] For example, the sleep state detection unit (1720) can determine whether a user is present or absent along with the sleep state. Accordingly, the sleep state detection unit (1720) may include a sleep state determination unit (1721) and a presence / absence determination unit (1722).

[0131] Meanwhile, the health status detection unit (1730) can calculate the user's stress index. The health status detection unit (1730) can also determine whether the user suffers from arrhythmia. Accordingly, the health status detection unit (1730) may include a stress index calculation unit (1731) and an arrhythmia determination unit (1732).

[0132] In some cases, the operation of the sleep state detection unit (1720) and / or health state detection unit (1730) may additionally utilize information received from an environmental sensor (165) in addition to data received from a radar sensor (160).

[0133] The controller (170) can provide a first response process (S10) based on such situational awareness monitoring.

[0134] For example, the first response process (S10) may be a process corresponding to a process of calculating a user's stress index, a process of determining whether the user has an arrhythmia, etc., which is performed in the health status detection unit (1730).

[0135] As an exemplary embodiment, content appropriate for the user's stress index can be recommended based on the process of calculating the user's stress index performed by the health status detection unit (1730) (S11). For example, content appropriate for relieving the user's stress index can be recommended.

[0136] In addition, depending on the process of determining whether the user has an arrhythmia, which is performed in the health status detection unit (1730), a notification can be provided to the user or another user associated with the user (S12).

[0137] For example, the first response process (S10) may be a process that responds based on the sleep state determination performed by the sleep state detection unit (1720) and the determination of whether the user is present or absent.

[0138] As an exemplary embodiment, the TV status can be changed based on the sleep status determination performed by the sleep status detection unit (1720) (S13). Meanwhile, the function of pausing the viewing can be performed based on the process of determining whether the user is present or absent performed by the sleep status detection unit (1720) (S14). Thereafter, if the user's absence continues for a certain period of time or longer, the TV status can be changed (S13).

[0139] In this way, the first response process (S10) may include at least one of content recommendation, notification provision, and display status change.

[0140] As an exemplary embodiment, the controller (170) may further provide a second response process (S20) for the first response process (S10).

[0141] For example, the second response process (S20) may provide an emergency response guide (S21) in response to a notification provision (S12) based on the user's determination of arrhythmia. That is, an emergency response guide for arrhythmia occurrence may be provided via the display (180).

[0142] As another example, the second response process (S20) can analyze the sleep state (S22) in response to the process (S13) of changing the TV status based on the sleep state assessment. For example, the user's specific sleep state, such as sleeping posture, breathing, and snoring, can be analyzed and provided via the display (180).

[0143] Figure 7 is an operational flowchart showing a control method of a display device according to one embodiment of the present invention.

[0144] Hereinafter, a control method for a display device according to an embodiment will be specifically described. The control method or process described herein can be performed by the controller (170) described above. Hereinafter, operations performed by the controller (170) will be described as an example.

[0145] At this time, the controller (170) can proceed with the operation step using the data received from the radar sensor (160). In addition, the controller (170) can additionally use information received from the environmental sensor (165) in addition to the data received from the radar sensor (160).

[0146] Referring to FIG. 7, first, data can be received from a radar sensor (160) (S50). At this time, information received from an environmental sensor (165) can be additionally received.

[0147] Thereafter, the controller (170) can perform situational awareness monitoring using at least one of data received from the radar sensor (160) and information received from the environmental sensor (165) (S100).

[0148] As described above, the controller (170) can perform situational awareness monitoring for the user by using at least one of the sleep state detection unit (1720) and the health state detection unit (1730).

[0149] Such situational awareness monitoring may include at least one of determining the user's movements, collecting the user's biometric data, and determining the user's presence.

[0150] As an exemplary embodiment, the situational awareness monitoring process (S100) can collect biometric data along with the user's movement determination (S110). As another example, the situational awareness monitoring process (S100) can collect the user's biometric data (S120, 130). As another example, the situational awareness monitoring process (S100) can determine whether the user is present or absent (S140).

[0151] The user's movement can be determined and biometric data such as the user's pulse rate and respiration rate can be collected according to the biometric data collection process (S110) that is performed together with the user's movement determination. The user's sleep state can be determined through this process (S110) (S111).

[0152] In this way, when the user's sleep state is determined, a notification that the user is sleeping can be provided as the first response process (S112).

[0153] Then, in response to this notification provision (S112), the status of the display (e.g., TV) can be changed as a second response process (S113). For example, if the user is determined to be sleeping, the TV can be turned off.

[0154] According to the process of collecting the user's biometric data (S120), the user's biometric data such as pulse rate, respiration rate, and stress index may be collected. Through this process (S120), the user's condition such as arrhythmia or premature beat can be determined (S121).

[0155] In this way, when the user's sleep state is determined, as a first response process, a notification regarding the user's health state, i.e., arrhythmia, premature beat, etc., can be provided (S122).

[0156] Then, in response to this notification provision (S122), an emergency action guide can be provided through a display as a second response process (S123).

[0157] Meanwhile, biometric data such as the user's stress level may be collected according to the process of collecting the user's biometric data (S130). Through this process (S130), the user's stress index can be determined (S131).

[0158] In this way, when the user's stress index is determined, a notification regarding the user's stress state can be provided as a first response process (S132).

[0159] Then, in response to the notification provision (S132), content corresponding to the stress index can be recommended through the display as a second response process (S133).

[0160] Depending on the process of determining whether a user is present or absent (S140), the determination of absence (S141) or presence (S144) of the user can be made through object detection.

[0161] For example, if the user is determined to be absent, the first response process may be to stop the content being played (S142).

[0162] Then, in response to this content pause (S142), the status of the display (e.g., TV) can be changed as a second response process (S143). For example, if the user is determined to be absent, the TV can be turned off.

[0163] Meanwhile, if the user is determined to be inactive, the first response process can be to play the stopped content (S145).

[0164] FIG. 8 is an operation flowchart showing an intelligent sensing service using a radar sensor according to one embodiment of the present invention.

[0165] As described above, an intelligent sensing service can be provided using at least one of data received from a radar sensor (160) and information received from an environmental sensor (165).

[0166] For example, such intelligent sensing services may be performed in the controller (170) of the display device (100). As another example, such intelligent sensing services may be performed through a separate controller.

[0167] In addition to the situational awareness monitoring (S100) described above, such intelligent sensing services can also perform user registration services (S200), including indoor / outdoor judgment, shape-based user registration, and visitor verification, and fall detection services (S300) using motion detection. These service operations (S100, S200, S300) can be performed in an integrated manner. At least some of these intelligent sensing services can be performed in AR (Augmented Reality) mode.

[0168] First, the process of the user registration service (S200) is explained.

[0169] Referring to FIG. 8, motion detection (S210) can be performed using a radar sensor (160).

[0170] As a result of motion detection (S210), the location and distance of a moving object can be determined (S220). Using the collected location / distance data, the indoor or outdoor location can be determined. At this time, the location and distance of the object can be determined upon motion detection, allowing for the distinction between indoor and outdoor locations.

[0171] In addition, using shape data collected using a radar sensor (160), it is possible to detect whether a moving object is a person (YES) or an animal (NO) (S230).

[0172] At this time, if the moving object is a person, it can be determined whether the person is a registered user (S240).

[0173] If the user is a registered user (YES), the door can be automatically opened and closed (S250) through visitor verification (S242). If the user is not a registered user (NO), the user can be registered through security processing or entry can be denied (S241).

[0174] Next, in the room entered, intelligent motion sensing (S310) can be performed using a radar sensor (160). This intelligent motion sensing (S310) can be divided into a motion detection (S320) process and situational awareness monitoring (S100). The situational awareness monitoring (S100) process may be the same as described above, so a duplicate description will be omitted.

[0175] After the motion detection process (S320), the location and distance of the moving object can be determined again (S330).

[0176] In addition, using shape data collected using a radar sensor (160), it is possible to detect whether a moving object is a person (YES) or an animal (NO) (S340).

[0177] Afterwards, a fall detection service (S350) can be provided along with the results of situational awareness monitoring (S100). At this time, a notification service can be provided based on registered users.

[0178] As described above, the content of the notification service may provide a fall detection service (S350) by providing notification (S122) and / or emergency action guide to the user and related subjects.

[0179] As described above, by using the display device and its control method according to the embodiment of the present disclosure, the sleep state information can be monitored by judging the viewer's movement and bio-signal data, and the TV power state can be switched to AR mode to reduce power consumption, and the sleep state can be maintained well by switching to a set sleep mode through control of a surrounding linked device.

[0180] Monitoring a user's heart rate and pulse in a painless, non-invasive way offers major benefits to both those at risk for and those with chronic conditions, allowing for early detection and intervention.

[0181] Additionally, current stress levels can be monitored and used for content recommendations, etc.

[0182] According to one embodiment of the present invention, the above-described method can be implemented as processor-readable code on a medium in which a program is recorded. Examples of processor-readable media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices.

[0183] The display device described above is not limited to the configuration and method of the embodiments described above, and the embodiments may be configured by selectively combining all or part of each embodiment so that various modifications can be made.

[0184] According to the above-described embodiment of the present invention, a display device capable of performing situational awareness monitoring using a radar sensor and providing fall detection service and integrated service can be provided.

Claims

1. In the display device, display; A radar sensor located on one side of the display that remotely monitors the user's movements using radio waves; A controller including one or more processors that receive and process data from the radar sensor, The above controller, A radar processing unit that processes data received from the radar sensor; A sleep state detection unit that processes the above data to detect the user's sleep state; and A health status detection unit that processes the above data to detect the health status of the user. Display device.

2. In paragraph 1, The above radar sensor, A radar transmitter for transmitting a first signal having a radar pulse; A radar receiver that receives a second signal reflected from the measurement target and related to the first signal; and A signal processing unit that processes a signal related to the bio-information of the measurement target based on the first and second signals transmitted and received from the radar transmitter and the radar receiver. Display device.

3. In paragraph 1, Further comprising an environmental sensor for additional data processing along with data collected from the above radar sensor. Display device.

4. In paragraph 3, The above environmental sensor includes at least one of an ambient light sensor, a microphone, and a temperature sensor. Display device.

5. In paragraph 1, The above controller, Performing situation awareness monitoring using at least one of the above sleep state detection unit and the above health state detection unit Display device.

6. In paragraph 5, The above situational awareness monitoring is, At least one of determining the user's movement, collecting the user's biometric data, and determining whether the user is present Display device.

7. In paragraph 6, The biometric data includes at least one of the user's pulse rate, respiration rate, and stress index. Display device.

8. In paragraph 1, The above controller provides a first response process according to the above situation awareness monitoring. Display device.

9. In paragraph 8, The above first response process includes at least one of content recommendation, notification provision, and display status change. Display device.

10. In paragraph 8, The above controller further provides a second response process for the first response process. Display device.

11. In paragraph 10, The above second response process includes either providing emergency action guidance or analyzing sleep status. Display device.

12. A method for controlling a display device including a radar sensor, the method comprising: A step of receiving data about a user from a radar sensor; A step of performing situational awareness monitoring based on the above data; and A step of providing a first response process based on the above situation awareness monitoring is included, The above situation awareness monitoring is, At least one of determining the user's movement, collecting the user's biometric data, and determining whether the user is present A method of controlling a display device.

13. In paragraph 11, The above first response process includes at least one of content recommendation, notification provision, and display status change. A method of controlling a display device.

14. In paragraph 11, Further comprising a step of providing a second response process for the first response process. A method of controlling a display device.

15. In paragraph 11, The above second response process includes either providing emergency action guidance or analyzing sleep status. A method of controlling a display device.

Images