Display device and operation method thereof

Abstract

A display device and an operation method thereof are disclosed. The display device according to at least one of various embodiments of the present disclosure may comprise: a display for outputting content; and a processor for acquiring a step signal of a user from a smart pad, calculating the number of steps of the user for a preset time from the step signal of the user, and controlling playback of the output content on the basis of the number of steps of the user.

Description

Display device and method of operation thereof

[0001] The present disclosure relates to a display device and a method of operating the same.

[0002] The functions of display devices have recently become more diverse, including data and voice communication, taking pictures and videos using cameras, voice recording, playing music files using speaker systems, and outputting images or videos to the display.

[0003] Some display devices have added electronic game play capabilities or perform multimedia player functions.

[0004] As display devices become more diverse in their functions, they are being implemented in the form of multimedia players with complex functions such as taking photos or videos, playing music or video files, playing games, and receiving broadcasts.

[0005] The present disclosure provides a display device and an operating method thereof that interactively controls playback of currently playing content based on user information.

[0006] Another object of the present disclosure is to provide a display device and an operating method thereof including an algorithm for increasing the accuracy of acquired user information.

[0007] A display device according to at least one of the various embodiments of the present disclosure may include a display for outputting content; and a processor for obtaining a user's step signal from a smart pad, calculating the number of steps of the user for a preset period of time from the user's step signal, and controlling playback of the outputted content based on the number of steps of the user.

[0008] Controlling playback of the content according to at least one of the various embodiments of the present disclosure may include controlling playback speed of the content.

[0009] The processor according to at least one of the various embodiments of the present disclosure may calculate the number of steps per minute based on the calculated number of steps of the user.

[0010] The processor according to at least one of the various embodiments of the present disclosure may determine a speed for controlling the playback speed of the content based on the calculated number of steps per minute.

[0011] The playback control of the content according to at least one of the various embodiments of the present disclosure may further include at least one of audio output level adjustment and resolution adjustment control of the content.

[0012] The processor according to at least one of the various embodiments of the present disclosure may determine the preset time by taking into account the size of the buffer.

[0013] A display device according to at least one of the various embodiments of the present disclosure may include a display for outputting content; and a processor for obtaining a user's step signal from a smart pad, calculating the user's step count in a preset interval for a preset time range from the user's step signal, and controlling playback of the outputted content based on the user's step count.

[0014] Controlling playback of the content according to at least one of the various embodiments of the present disclosure may include controlling playback speed of the content.

[0015] The processor according to at least one of the various embodiments of the present disclosure may set a plurality of intervals for the preset time range, wherein each interval may be set to accumulate from an initial time of the preset time range.

[0016] The processor according to at least one of the various embodiments of the present disclosure can calculate the number of steps per minute of the user's number of steps calculated for each of the intervals.

[0017] The processor according to at least one of the various embodiments of the present disclosure may calculate an average number of steps by applying a weight to the number of steps per minute calculated in each of the interval units.

[0018] The processor according to at least one of the various embodiments of the present disclosure may determine a playback speed of the content based on the average number of steps, and control the playback speed of the content based on the playback speed.

[0019] The processor according to at least one of the various embodiments of the present disclosure may determine the preset time by taking into account the size of the buffer.

[0020] The playback control of the content according to at least one of the various embodiments of the present disclosure may further include at least one of audio output level adjustment and resolution adjustment control of the content.

[0021] A method of operating a display device according to at least one of various embodiments of the present disclosure may include the steps of: outputting content; acquiring a user's step signal; calculating the number of steps of the user for a preset time range from the user's step signal; and controlling playback of the outputted content based on the number of steps of the user. In this case, the calculated number of steps of the user may be any one of the total number of steps of the user for the preset time range and the number of steps in each section for a plurality of sections accumulated and set for the preset time range.

[0022] According to at least one of the various embodiments of the present disclosure, the following effects(s) can be obtained.

[0023] First, it has the effect of increasing user satisfaction by controlling content playback to match the user's current situation when exposed to various situations.

[0024] Second, it has the effect of increasing user satisfaction by obtaining more accurate user information and controlling services based on it.

[0025] FIG. 1 is a block diagram of a display device according to an embodiment of the present disclosure;

[0026] FIG. 2 is a block diagram of a remote control device according to an embodiment of the present disclosure;

[0027] FIG. 3 is an example of an actual configuration of a remote control device according to an embodiment of the present disclosure;

[0028] FIG. 4 is an example of utilizing a remote control device according to an embodiment of the present disclosure;

[0029] FIG. 5 is a drawing for explaining the horizontal mode and vertical mode of a stand-type display device according to an embodiment of the present disclosure;

[0030] FIG. 6 is a schematic diagram of an interactive display system according to one embodiment of the present disclosure;

[0031] FIG. 7 is a flowchart illustrating an interactive content playback control method according to one embodiment of the present disclosure;

[0032] Figure 8 is a drawing illustrating the method of Figure 7;

[0033] FIG. 9 is a flowchart illustrating an interactive content playback control method according to another embodiment of the present disclosure;

[0034] Figures 10 and 11 are drawings illustrating the method of Figure 9; and

[0035] FIG. 12 is a drawing illustrating a method for processing two-legged motion according to one embodiment of the present disclosure.

[0036] Hereinafter, embodiments related to the present disclosure will be described in more detail with reference to the drawings. The suffixes "module" and "part" used in the following description for components are assigned or used interchangeably solely for the convenience of writing the specification, and do not in themselves have distinct meanings or roles.

[0037] FIG. 1 is a block diagram illustrating the configuration of a display device (100) according to one embodiment of the present disclosure.

[0038] Referring to FIG. 1, the display device (100) may include a broadcast receiving unit (130), an external device interface unit (135), a storage unit (140), a user input interface unit (150), a control unit (170), a wireless communication unit (173), a voice acquisition unit (175), a display unit (180), an audio output unit (185), and a power supply unit (190).

[0039] The broadcast receiving unit (130) may include a tuner (131), a demodulation unit (132), and a network interface unit (133).

[0040] The tuner (131) can select a specific broadcast channel according to a channel selection command. The tuner (131) can receive a broadcast signal for the selected specific broadcast channel.

[0041] The demodulator (132) can separate the received broadcast signal into a video signal, an audio signal, and a data signal related to the broadcast program, and can restore the separated video signal, audio signal, and data signal into a form that can be output.

[0042] The network interface unit (133) may provide an interface for connecting the display device (100) to a wired / wireless network including the Internet. The network interface unit (133) may transmit or receive data to or from other users or other electronic devices via the connected network or another network linked to the connected network.

[0043] The network interface unit (133) can access a predetermined web page through a connected network or another network linked to the connected network. That is, by accessing a predetermined web page through a network, data can be transmitted or received with the corresponding server.

[0044] In addition, the network interface unit (133) can receive content or data provided by a content provider or network operator. That is, the network interface unit (133) can receive content such as movies, advertisements, games, VOD (Video on Demand), broadcast signals, etc., and information related thereto provided from a content provider or network provider via a network.

[0045] Additionally, the network interface unit (133) can receive firmware update information and update files provided by the network operator, and transmit data to the Internet or content provider or network operator.

[0046] The network interface unit (133) can select and receive a desired application from among applications open to the public via a network.

[0047] The external device interface unit (135) can receive an application or a list of applications in an adjacent external device and transmit it to the control unit (170) or storage unit (140).

[0048] The external device interface unit (135) can provide a connection path between the display device (100) and the external device. The external device interface unit (135) can receive one or more of images and audio output from an external device connected wirelessly or wiredly to the display device (100) and transmit them to the control unit (170). The external device interface unit (135) can include a plurality of external input terminals. The plurality of external input terminals can include an RGB terminal, one or more HDMI (High Definition Multimedia Interface) terminals, and a component terminal.

[0049] The voice signal of an external device input through the external device interface unit (135) can be output through the display unit (180). The voice signal of an external device input through the external device interface unit (135) can be output through the audio output unit (185).

[0050] An external device that can be connected to the external device interface unit (135) may be any one of a set-top box, a Blu-ray player, a DVD player, a game console, a sound bar, a smartphone, a PC, a USB memory, and a home theater, but this is only an example.

[0051] Additionally, some content data stored in the display device (100) can be transmitted to a selected user or electronic device among other users or other electronic devices pre-registered in the display device (100).

[0052] The storage unit (140) stores programs for each signal processing and control within the control unit (170), and can store signal-processed images, voices, or data signals.

[0053] In addition, the storage unit (140) may perform a function for temporary storage of video, audio, or data signals input from an external device interface unit (135) or a network interface unit (133), and may also store information about a specific image through a channel memory function.

[0054] The storage unit (140) can store an application or a list of applications input from the external device interface unit (135) or the network interface unit (133).

[0055] The display device (100) can play content files (video files, still image files, music files, document files, application files, etc.) stored in the storage unit (140) and provide them to the user.

[0056] The user input interface unit (150) can transmit a signal input by a user to the control unit (170), or transmit a signal from the control unit (170) to the user. For example, the user input interface unit (150) can receive and process control signals such as power on / off, channel selection, and screen setting from the remote control device (200) according to various communication methods such as Bluetooth, Ultra Wideband (UWB), ZigBee, Radio Frequency (RF) communication, or infrared (IR) communication, or process the control signals from the control unit (170) to be transmitted to the remote control device (200).

[0057] In addition, the user input interface unit (150) can transmit control signals input from local keys (not shown) such as a power key, channel key, volume key, and setting value to the control unit (170).

[0058] An image signal processed by the control unit (170) can be input to the display unit (180) and displayed as an image corresponding to the image signal. In addition, an image signal processed by the control unit (170) can be input to an external output device through the external device interface unit (135).

[0059] The voice signal processed in the control unit (170) can be output as audio to the audio output unit (185). In addition, the voice signal processed in the control unit (170) can be input to an external output device through the external device interface unit (135).

[0060] In addition, the control unit (170) can control the overall operation within the display device (100).

[0061] In addition, the control unit (170) can control the display device (100) by a user command or internal program input through the user input interface unit (150), and can connect to a network to enable the user to download a desired application or application list into the display device (100).

[0062] The control unit (170) enables the channel information selected by the user to be output through the display unit (180) or audio output unit (185) together with the processed video or audio signal.

[0063] In addition, the control unit (170) allows a video signal or audio signal from an external device, for example, a camera or camcorder, input through the external device interface unit (135) to be output through the display unit (180) or audio output unit (185) in accordance with an external device video playback command received through the user input interface unit (150).

[0064] Meanwhile, the control unit (170) can control the display unit (180) to display an image, and for example, can control the display unit (180) to display a broadcast image input through the tuner (131), an external input image input through the external device interface unit (135), an image input through the network interface unit, or an image stored in the storage unit (140). In this case, the image displayed on the display unit (180) can be a still image or a moving image, and can be a 2D image or a 3D image.

[0065] In addition, the control unit (170) can control the playback of content stored in the display device (100), received broadcast content, or external input content input from outside, and the content can be in various forms such as broadcast video, external input video, audio file, still image, connected web screen, and document file.

[0066] The wireless communication unit (173) can communicate with an external device through wired or wireless communication. The wireless communication unit (173) can perform short-range communication with the external device. To this end, the wireless communication unit (173) can support short-range communication using at least one of Bluetooth™, BLE (Bluetooth Low Energy), RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB, ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies. This wireless communication unit (173) can support wireless communication between the display device (100) and a wireless communication system, between the display device (100) and another display device (100), or between the display device (100) and a network where the display device (100, or an external server) is located via a short-range wireless communication network (Wireless Area Network). The short-range wireless communication network may be a short-range wireless personal area network (Wireless Personal Area Network).

[0067] Here, the other display device (100) may be a wearable device (e.g., a smart watch, smart glasses, a head mounted display (HMD), a mobile terminal such as a smart phone) that can exchange data with the display device (100) according to the present disclosure (or can be linked). The wireless communication unit (173) may detect (or recognize) a wearable device capable of communication around the display device (100). Furthermore, if the detected wearable device is an authenticated device to communicate with the display device (100) according to the present disclosure, the control unit (170) may transmit at least a part of the data processed in the display device (100) to the wearable device through the wireless communication unit (173). Accordingly, a user of the wearable device may use the data processed in the display device (100) through the wearable device.

[0068] The voice acquisition unit (175) can acquire audio. The voice acquisition unit (175) can include at least one microphone (not shown) and can acquire audio around the display device (100) through the microphone (not shown).

[0069] The display unit (180) can generate a driving signal by converting a video signal, data signal, OSD signal processed by the control unit (170) or a video signal, data signal, etc. received from the external device interface unit (135) into R, G, B signals, respectively.

[0070] Meanwhile, since the display device (100) illustrated in FIG. 1 is merely an example of the present disclosure, some of the illustrated components may be integrated, added, or omitted depending on the specifications of the display device (100) actually implemented.

[0071] That is, two or more components may be combined into a single component, or a single component may be subdivided into two or more components, as needed. Furthermore, the functions performed by each block are intended to illustrate embodiments of the present disclosure, and their specific operations or devices do not limit the scope of the present disclosure.

[0072] According to another embodiment of the present disclosure, the display device (100) may receive and play back an image through a network interface unit (133) or an external device interface unit (135) without having a tuner (131) and a demodulator (132), unlike as shown in FIG. 1.

[0073] For example, the display device (100) may be implemented separately as an image processing device, such as a set-top box, for receiving broadcast signals or contents according to various network services, and a content playback device for playing contents input from the image processing device.

[0074] In this case, the method of operating the display device according to the embodiment of the present disclosure to be described below may be performed by any one of the display device (100) described with reference to FIG. 1, as well as an image processing device such as a separate set-top box, or a content playback device having a display unit (180) and an audio output unit (185).

[0075] The audio output unit (185) receives a signal processed by the control unit (170) and outputs it as voice.

[0076] The power supply unit (190) supplies power to the entire display device (100). In particular, it can supply power to a control unit (170) that can be implemented in the form of a system on chip (SOC), a display unit (180) for displaying images, and an audio output unit (185) for outputting audio.

[0077] Specifically, the power supply unit (190) may be equipped with a converter that converts AC power into DC power and a dc / dc converter that converts the level of the DC power.

[0078] Next, a remote control device according to an embodiment of the present disclosure will be described with reference to FIGS. 2 and 3.

[0079] FIG. 2 is a block diagram of a remote control device according to an embodiment of the present disclosure, and FIG. 3 shows an example of an actual configuration of a remote control device according to an embodiment of the present disclosure.

[0080] First, referring to FIG. 2, the remote control device (200) may include a fingerprint recognition unit (210), a wireless communication unit (220), a user input unit (230), a sensor unit (240), an output unit (250), a power supply unit (260), a storage unit (270), a control unit (280), and a voice acquisition unit (290).

[0081] Referring to FIG. 2, the wireless communication unit (220) transmits and receives signals with any one of the display devices according to the embodiments of the present disclosure described above.

[0082] The remote control device (200) may be equipped with an RF module (221) capable of transmitting and receiving signals with the display device (100) according to RF communication standards, and may be equipped with an IR module (223) capable of transmitting and receiving signals with the display device (100) according to IR communication standards. In addition, the remote control device (200) may be equipped with a Bluetooth module (225) capable of transmitting and receiving signals with the display device (100) according to Bluetooth communication standards. In addition, the remote control device (200) may be equipped with an NFC module (227) capable of transmitting and receiving signals with the display device (100) according to NFC (Near Field Communication) communication standards, and may be equipped with a WLAN module (229) capable of transmitting and receiving signals with the display device (100) according to WLAN (Wireless LAN) communication standards.

[0083] Additionally, the remote control device (200) transmits a signal containing information about the movement of the remote control device (200) to the display device (100) through the wireless communication unit (220).

[0084] Meanwhile, the remote control device (200) can receive a signal transmitted by the display device (100) through the RF module (221), and, if necessary, can transmit commands for turning the power on / off, changing the channel, changing the volume, etc. to the display device (100) through the IR module (223).

[0085] The user input unit (230) may be configured as a keypad, buttons, a touch pad, or a touch screen. The user can input commands related to the display device (100) to the remote control device (200) by operating the user input unit (230). If the user input unit (230) is equipped with a hard key button, the user can input commands related to the display device (100) to the remote control device (200) by pushing the hard key button. This will be described with reference to FIG. 3.

[0086] Referring to FIG. 3, the remote control device (200) may include a plurality of buttons. The plurality of buttons may include a fingerprint recognition button (212), a power button (231), a home button (232), a live button (233), an external input button (234), a volume control button (235), a voice recognition button (236), a channel change button (237), a confirmation button (238), and a back button (239).

[0087] The fingerprint recognition button (212) may be a button for recognizing a user's fingerprint. In one embodiment, the fingerprint recognition button (212) may be capable of a push operation, and thus may receive a push operation and a fingerprint recognition operation. The power button (231) may be a button for turning the display device (100) on / off. The home button (232) may be a button for moving to the home screen of the display device (100). The live button (233) may be a button for displaying a real-time broadcast program. The external input button (234) may be a button for receiving an external input connected to the display device (100). The volume control button (235) may be a button for adjusting the volume output by the display device (100). The voice recognition button (236) may be a button for receiving a user's voice and recognizing the received voice. The channel change button (237) may be a button for receiving a broadcast signal of a specific broadcast channel. The confirmation button (238) may be a button for selecting a specific function, and the back button (239) may be a button for returning to the previous screen.

[0088] Let's explain Figure 2 again.

[0089] When the user input unit (230) is equipped with a touch screen, the user can input commands related to the display device (100) using the remote control device (200) by touching the soft keys of the touch screen. In addition, the user input unit (230) may be equipped with various types of input devices that can be operated by the user, such as a scroll key or a jog key, and the present embodiment does not limit the scope of the present disclosure.

[0090] The sensor unit (240) may be equipped with a gyro sensor (241) or an acceleration sensor (243), and the gyro sensor (241) may sense information about the movement of the remote control device (200).

[0091] For example, the gyro sensor (241) can sense information about the operation of the remote control device (200) based on the x, y, and z axes, and the acceleration sensor (243) can sense information about the movement speed of the remote control device (200). Meanwhile, the remote control device (200) can further include a distance measuring sensor, so as to sense the distance to the display unit (180) of the display device (100).

[0092] The output unit (250) can output a video or audio signal corresponding to the operation of the user input unit (230) or to a signal transmitted from the display device (100). Through the output unit (250), the user can recognize whether the user input unit (230) is being operated or whether the display device (100) is being controlled.

[0093] For example, the output unit (250) may be equipped with an LED module (251) that lights up when the user input unit (230) is operated or a signal is transmitted and received with the display device (100) through the wireless communication unit (220), a vibration module (253) that generates vibration, a sound output module (255) that outputs sound, or a display module (257) that outputs an image.

[0094] In addition, the power supply unit (260) supplies power to the remote control device (200), and can reduce power waste by stopping the power supply when the remote control device (200) is not moved for a predetermined period of time. The power supply unit (260) can resume the power supply when a predetermined key provided in the remote control device (200) is operated.

[0095] The storage unit (270) can store various types of programs, application data, etc. required for the control or operation of the remote control device (200). If the remote control device (200) wirelessly transmits and receives signals through the display device (100) and the RF module (221), the remote control device (200) and the display device (100) transmit and receive signals through a predetermined frequency band.

[0096] The control unit (280) of the remote control device (200) can store and reference information regarding the frequency band, etc., that can wirelessly transmit and receive signals with the display device (100) paired with the remote control device (200), in the storage unit (270).

[0097] The control unit (280) controls all matters related to the control of the remote control device (200). The control unit (280) can transmit a signal corresponding to a predetermined key operation of the user input unit (230) or a signal corresponding to the movement of the remote control device (200) sensed by the sensor unit (240) to the display device (100) via the wireless communication unit (220).

[0098] Additionally, the voice acquisition unit (290) of the remote control device (200) can acquire voice.

[0099] The voice acquisition unit (290) may include at least one microphone (291) and may acquire voice through the microphone (291).

[0100] Next, Figure 4 is described.

[0101] FIG. 4 shows an example of utilizing a remote control device according to an embodiment of the present disclosure.

[0102] Figure 4 (a) illustrates that a pointer (205) corresponding to a remote control device (200) is displayed on a display unit (180).

[0103] The user can move or rotate the remote control device (200) up and down, left and right. The pointer (205) displayed on the display unit (180) of the display device (100) corresponds to the movement of the remote control device (200). As shown in the drawing, the pointer (205) moves and is displayed according to the movement in 3D space, so the remote control device (200) can be called a space remote control.

[0104] Figure 4 (b) illustrates that when a user moves the remote control device (200) to the left, the pointer (205) displayed on the display unit (180) of the display device (100) also moves to the left in response.

[0105] Information about the movement of the remote control device (200) detected by the sensor of the remote control device (200) is transmitted to the display device (100). The display device (100) can calculate the coordinates of the pointer (205) from the information about the movement of the remote control device (200). The display device (100) can display the pointer (205) to correspond to the calculated coordinates.

[0106] Figure 4 (c) illustrates a case where a user moves the remote control device (200) away from the display unit (180) while pressing a specific button within the remote control device (200). As a result, a selection area within the display unit (180) corresponding to the pointer (205) can be zoomed in and displayed in an enlarged manner.

[0107] Conversely, when the user moves the remote control device (200) closer to the display unit (180), the selection area within the display unit (180) corresponding to the pointer (205) may be zoomed out and displayed in a reduced size.

[0108] Meanwhile, when the remote control device (200) moves away from the display unit (180), the selection area may be zoomed out, and when the remote control device (200) moves closer to the display unit (180), the selection area may be zoomed in.

[0109] In addition, when a specific button within the remote control device (200) is pressed, recognition of up, down, left, and right movements may be excluded. That is, when the remote control device (200) moves away from or toward the display unit (180), up, down, left, and right movements may not be recognized, and only forward and backward movements may be recognized. When a specific button within the remote control device (200) is not pressed, only the pointer (205) moves in accordance with the up, down, left, and right movements of the remote control device (200).

[0110] Meanwhile, the movement speed or movement direction of the pointer (205) can correspond to the movement speed or movement direction of the remote control device (200).

[0111] Meanwhile, the pointer in this specification refers to an object displayed on the display unit (180) in response to the operation of the remote control device (200). Therefore, objects of various shapes other than the arrow shape illustrated in the drawing can be used as the pointer (205). For example, the pointer may be a concept including a point, a cursor, a prompt, a thick outline, etc. In addition, the pointer (205) may be displayed corresponding to one point on the horizontal or vertical axis on the display unit (180), or may be displayed corresponding to multiple points such as lines or surfaces.

[0112] FIG. 5(a) and FIG. 5(b) are drawings for explaining the horizontal mode and vertical mode of a stand-type display device according to an embodiment of the present disclosure.

[0113] Referring to FIG. 5(a) and FIG. 5(b), a stand-type display device (100) is illustrated.

[0114] A shaft (103) and a stand base (105) can be connected to the display device (100).

[0115] The shaft (103) can connect the display device (100) and the stand base (105). The shaft (103) can extend vertically.

[0116] The lower end of the shaft (103) can be connected to the edge of the stand base (105).

[0117] The lower end of the shaft (103) can be rotatably connected to the periphery of the stand base (105).

[0118] The display device (100) and shaft (103) can rotate around a vertical axis with respect to the stand base (105).

[0119] The upper part of the shaft (103) can be connected to the rear of the display device (100).

[0120] The stand base (105) can serve to support the display device (100).

[0121] The display device (100) may be configured to include a shaft (103) and a stand base (105).

[0122] The display device (100) can rotate around the point where the upper part of the shaft (103) and the rear part of the display (180) meet.

[0123] FIG. 5(a) shows that the display (180) operates in a landscape mode in which the horizontal length is greater than the vertical length, and FIG. 5(b) shows that the display (180) operates in a landscape mode in which the vertical length is greater than the horizontal length.

[0124] The user can move the stand-type display device (100). That is, unlike a fixed device, the stand-type display device (100) has improved mobility, so the user is not restricted by the placement location.

[0125] Traditional display devices, as output devices, passively provide content based on user requests, typically not actively responding to user situations or actions. However, these display device responses can sometimes be inconvenient, sometimes not meeting user intent, or require additional action, resulting in inconvenience.

[0126] Hereinafter, the present specification describes a display device (100) and an operating method thereof in which various embodiments of the present disclosure are implemented that are interactive and actively operated based on user information.

[0127] For convenience of explanation, the operation of the display device (100) according to the present disclosure is described by way of example, controlling playback of content.

[0128] User information may include, for example, user motion information. Furthermore, user motion information may represent information obtained from a signal detected and transmitted by the smart pad (400) described below. Depending on the embodiment, user information may function only when, for example, specific content is being executed.

[0129] In this specification, content may mean all forms of data or information that can be output from the display device (100). Such content may also include applications, and such applications may include intelligent fit applications related to physical diagnosis and / or health. Meanwhile, such applications may be native applications built into the display device (100), or applications provided by the manufacturer of the display device (100) or a third party. Furthermore, the latter applications may be downloaded and installed by a user on a display device (100) connected to the Internet. Depending on the embodiment, content may be limited to specific content or specific applications in relation to the present disclosure. However, the present disclosure is not limited to the above-described examples.

[0130] In other words, it is assumed that the user is walking on a smart pad (400 in FIG. 6) described below while watching content through the display device (100). At this time, the display device (100) reacts, i.e., performs an action, in response to the user's action.

[0131] FIG. 6 is a schematic diagram of an interactive display system (hereinafter referred to as “system”) according to one embodiment of the present disclosure.

[0132] A system according to one embodiment of the present disclosure may be configured to include a display device (100) and a smart pad (400). Depending on the embodiment, the system may further include a server (500). According to another embodiment, the system may further include an image acquisition device (300). In this case, the image acquisition device (300) may be used for other purposes or may be required to assist the smart pad (400).

[0133] For convenience of explanation, the system will be described below primarily with reference to the display device (100) and the smart pad (400). Even if not described separately, the server (500) may be responsible for processing all or part of the functions or operations of the display device (100) described herein, and may transmit the processing results to the display device (100).

[0134] The display device (100) may receive a content playback request. Alternatively, the display device (100) may receive an intelligent fit application execution request.

[0135] Hereinafter, the term "content" in this specification may refer to any video-type content provided through the screen of the display device (100) while the intelligent fit application is running in the foreground or background, for example. However, this is merely an example to aid understanding of the present disclosure, and the present disclosure is not limited thereto.

[0136] As described above, when an intelligent fit application is running in the foreground or background, the display device (100) can be paired or / and connected with the smart pad (400) and obtain signals containing various information therefrom.

[0137] The aforementioned content control request can be made using the remote control device (200) illustrated in FIGS. 2 and 3 or a terminal (not shown) held by the user. At this time, the terminal may be a terminal pre-registered with at least one of the display device (100) / smart pad (400) / server (500), or a terminal belonging to the same network as the display device (100) / smart pad (400). Preferably, the terminal is a terminal on which a remote control application or an intelligent fit application for controlling the display device (100) is installed. Such a terminal may be a fixed terminal such as a TV, PC, or signage, or a mobile terminal such as a smartphone or tablet PC. Alternatively, the terminal may be a wearable device such as a smartwatch or smartglasses. If the display (180) of the display device (100) is configured as a touch panel, the request can be acquired from the display device (100) by a user's touch input.

[0138] In FIG. 6, for convenience, one smart pad (400) is illustrated, but multiple smart pads (400) may be simultaneously connected to one display device (100). Depending on the embodiment, the configuration, performance, version, etc. of each smart pad (400) may or may not be identical.

[0139] The image acquisition device (300) may not necessarily be an essential component.

[0140] As mentioned above, the server (500) may not necessarily be an essential component. However, the server (500) may replace all or part of the interactive response operation or processing of the display device (100).

[0141] The server (500) can obtain a user's movement signal from the smart pad (400). The server (500) can transmit the user's movement signal thus obtained to the display device (100). At this time, during the transmission process, the user's movement signal may be transmitted to the display device (100) as raw data, or may be transmitted to the display device (100) after processing. During the processing process, the server (500) can process the obtained user's movement signal and transmit the analyzed result to the display device (100). The analyzed result may also include a control command for controlling the display device (100).

[0142] Hereinafter, the operation of controlling the playback of content according to a user's request or selection in the processor of the display device (100) will be described in more detail.

[0143] For convenience of explanation, content playback control is explained using content playback speed control as an example.

[0144] FIG. 7 is a flowchart illustrating an interactive content playback control method according to an embodiment of the present disclosure. FIG. 8 is a diagram illustrating the method of FIG. 7.

[0145] In operation S110, the processor can obtain a signal about the user's movement from the smart pad (400), set a target section (i.e., time zone) for interactive content playback control from the obtained signal, and temporarily store data corresponding to the set time zone in a buffer.

[0146] Here, the target section of interactive content playback control can be determined based on, for example, the number of buffers for temporarily storing data, the size of the buffer, etc. In the present disclosure, for convenience of explanation, the target section may represent a section in units of 4 seconds, but the present disclosure is not limited thereto.

[0147] Referring to FIG. 8, the target time interval for interactive content playback control within the entire time interval, for example, between 0 and 6 seconds, obtained from the smart pad (400) may be, for example, between 2 and 6 seconds (a total of 4 seconds) (820). However, this is only an example, and other time intervals may be selected. In addition, the processor may continuously set and temporarily store the target interval while accumulating it on a second-by-second basis, for example. That is, the processor may set and store data so that a predetermined interval is included in the target interval while accumulating it within the entire interval, such as between 0 and 4 seconds (first target interval), between 1 and 5 seconds (second target interval), and between 2 and 6 seconds (third target interval).

[0148] However, for convenience, the time between 2 and 6 seconds is used as an example.

[0149] Meanwhile, in the present disclosure, the interactive content playback control target sections may be set so as not to overlap with each other. For example, if a control target section is set to be between 2 and 6 seconds based on absolute time as described above, the next control target section may be set to be between 7 and 11 seconds, so that the absolute times of the control target sections do not overlap with each other.

[0150] In the S120 operation, the processor can calculate the number of steps within the target interval (4 seconds).

[0151] Referring to Figure 8, the number of steps (830) within the target section may be a total of 15 steps.

[0152] In the S130 operation, the processor can normalize the number of steps (830) within the target interval to objectify it, i.e., convert it into steps per minute. Since there are 15 steps in 4 seconds, calculating 15 * (4 / 60) can calculate 225 steps as steps per minute.

[0153] In operation S140, the processor can set or determine content playback control content and content playback speed (speed) based on the calculated number of steps per minute.

[0154] At this time, the content playback speed (i.e., speed) can be determined, for example, by comparing the calculated steps per minute with the speed-based steps per minute. For example, if the steps per minute is 100 and the speed is assumed to be 1x (normal speed), then if the steps per minute is 225 as described above, the interactive content playback speed will be 2.25x.

[0155] In S150 operation, the processor can control the playback speed of the content according to the determined content playback speed (speed).

[0156] FIG. 9 is a flowchart illustrating an interactive content playback control method according to another embodiment of the present disclosure. FIGS. 10 and 11 are diagrams illustrating the method of FIG. 9.

[0157] In Fig. 9, an embodiment different from Fig. 7 is illustrated. However, the basic processing process of the embodiment of Fig. 9 is also similar to that of Fig. 7 described above, so overlapping parts will be briefly described, and different parts will be mainly described.

[0158] In operation S210, the processor can obtain a signal about the user's movement from the smart pad (400), set a target section (i.e., time zone) for interactive content playback control from the obtained signal, and temporarily store data corresponding to the set time zone in a buffer.

[0159] In the S220 operation, the processor can calculate the number of steps within the target interval (4 seconds).

[0160] Unlike FIG. 8, in operation S220, the number of steps accumulated per second, rather than the total number of steps in the target section, can be calculated. Accordingly, as illustrated in FIG. 9, the processor can calculate the number of steps (831) in the last 1 second, the number of steps (832) in the last 2 seconds, the number of steps (833) in the last 3 seconds, and the number of steps (834) in the last 4 seconds, respectively. At this time, the number of steps (832) in the last 2 seconds may include the number of steps (831) in the last 1 second, the number of steps (833) in the last 3 seconds may include the number of steps (832) in the last 2 seconds, and the number of steps (834) in the last 4 seconds may include the number of steps (833) in the last 3 seconds. In other words, the number of steps can be calculated by accumulating backwards from a reference time (6 seconds or the current time).

[0161] In the S230 operation, the processor can then calculate an objective indicator for speed determination, namely steps per minute.

[0162] Even in the S230 operation, unlike in FIGS. 7 and 8, the processor can calculate the number of steps per minute for each item, i.e., the number of steps in the last 1 second (831), the number of steps in the last 2 seconds (832), the number of steps in the last 3 seconds (833), and the number of steps in the last 4 seconds (834).

[0163] In the S240 operation, the processor can calculate the average step count by applying preset weights.

[0164] Here, the preset weights can be set to different values ​​for each item, for example. This is because evaluating the influence (e.g., user intent or movement) on content playback control differently for times near and far from the reference time (e.g., 6 seconds) may actually be more in line with the user's intent or movement. In other words, even within the target section, weights can be set differently between the start and end sections so that content playback control can be performed in line with the user's recent movements or intent. For example, if the user's step count was 10 between 2 and 4 seconds but 5 between 4 and 6 seconds, this indicates that the user's step count has recently been halved. Therefore, even if the total is 15 steps, rather than determining content playback control based on the entire 15 steps, it is more likely to reflect the influence of recent values, which is more in line with the user's intent. Therefore, by assigning different weights, the user's intent can be identified.

[0165] As mentioned above, assigning a higher weight to times relatively close to the reference time (present) and assigning a lower weight to times relatively far from the reference time can yield an objective number of steps according to the user's movement.

[0166] Accordingly, the processor according to the present disclosure may, for example, assign a weight of 4 to the number of steps (831) taken in the last 1 second, a weight of 3 to the number of steps taken in the last 2 seconds (832), a weight of 2 to the number of steps taken in the last 3 seconds (833), and a weight of 1 to the number of steps taken in the last 4 seconds (834).

[0167] In another embodiment, when calculating the number of steps separately per second, there are 5 steps between 2 seconds and 3 seconds, 3 steps between 3 seconds and 4 seconds, 3 steps between 4 seconds and 5 seconds, and 4 steps between 5 seconds and 6 seconds. However, referring to FIG. 9, the number of steps in the last 1 second (831) according to the present disclosure is 4 steps, the number of steps in the last 2 seconds (832) is 7 steps in total, the number of steps in the last 3 seconds (833) is 10 steps in total, and the number of steps in the last 4 seconds (834) is 15 steps in total.

[0168] Rather than calculating each item as a step count for an accumulated time, as described above, the step count can be calculated independently for each second (e.g., between 2 and 3 seconds, between 3 and 4 seconds, between 4 and 5 seconds, and between 5 and 6 seconds), with higher weighting given to time periods closer to the present. For example, the weighting for steps taken between 5 and 6 seconds may be relatively higher than that for steps taken between 2 and 3 seconds.

[0169] In another embodiment, items (e.g., the last 1 second, the last 2 seconds, between 2 and 3 seconds, between 3 and 4 seconds, etc.) may be grouped and weighted differently for each group. During the grouping process, the processor may set a threshold (e.g., number of steps) and group cases above and below the threshold. Alternatively, the processor may preset multiple thresholds and weights for each threshold, and apply the preset weights based on which section each item falls into by comparing it to the threshold. Alternatively, the processor may compare the number of cases above and below the threshold, and if a specific case is relatively more numerous, adjust the preset threshold before grouping.

[0170] However, the present disclosure is not limited to the above examples.

[0171] Referring to FIGS. 9 and 10, the number of steps in the last 1 second (831) calculated by the processor in the S220 operation is 4 steps, the number of steps in the last 2 seconds (832) is 7 steps, the number of steps in the last 3 seconds (833) is 10 steps, and the number of steps in the last 4 seconds (834) is 15 steps.

[0172] Also, in the S230 operation, the number of steps per minute for the last 1 second calculated by the processor is 4 steps * 60 seconds / 1 second, which is 240 steps; the number of steps per minute for the last 2 seconds is 7 steps * 60 seconds / 2 seconds, which is 210 steps; the number of steps per minute for the last 3 seconds is 10 steps * 60 seconds / 3 seconds, which is 200 steps; and the number of steps per minute for the last 4 seconds is 15 steps * 60 seconds / 4 seconds, which is 225 steps.

[0173] Accordingly, the processor can calculate the average number of steps by applying the weight assigned to each item to the calculated number of steps per minute for each item in operation S230. For example, referring to FIGS. 9 and 10 , the average number of steps can be calculated as {(225*1)+(200*2)+(210*3)+(240*4)} / (1+2+3+4) which is 221.5 steps.

[0174] In operation S250, the processor can calculate a content playback speed control level, i.e., a speed, based on the number of steps per minute calculated in operation S230 and the average number of steps calculated using weights in operation S240.

[0175] As mentioned above, if the average number of steps is 221.5 and 100 steps per minute is 1x, the speed can be calculated as 2.21 (or 2.22).

[0176] In operation S260, the processor can control the playback speed of the content being played according to the calculated speed.

[0177] Figure 11 is a drawing illustrating step data (left foot, right foot) transmitted from a smart mat (400).

[0178] Referring to Figure 11, the left side shows step data sensed at predefined time intervals. The time interval is exemplified as 0.2 seconds in Figure 11, but is not necessarily limited thereto.

[0179] In Fig. 11, if the last 4 bits of each step data are '01 00' and data of the corresponding bits are entered at least twice in a row, it can be recognized as a left foot, and if the last 4 bits are '00 01' and data of the corresponding bits are entered at least twice in a row, it can be recognized as a right foot.

[0180] Referring to FIG. 11, the display device (100) inputs 'FE EF 01 00' at 00.4 seconds and 'FE EF 01 00' at 00.6 seconds. Since the value of the last 4 bits is '01 00' continuously input, it may indicate that the left foot came into contact with the smart mat (400) at 00.4 seconds.

[0181] On the other hand, the display device (100) inputs 'FE EF 00 01' at 01.0 seconds and 'FE EF 00 01' at 01.2 seconds. Since the value of the last 4 bits is '00 01' continuously input, it may indicate that the right foot came into contact with the smart mat (400) at 01.2 seconds.

[0182] The section 1110 in FIG. 11 represents step data in a 4-second section, such as FIG. 8 or FIG. 10, for example. That is, referring to FIG. 11, it can be seen that the right foot touches the smart pad (400) at 2.2 seconds, the left foot touches the smart pad (400) at 3.0 seconds, the right foot touches the smart pad (400) again at 3.6 seconds, the left foot touches the smart pad (400) again at 4.4 seconds, and finally, the right foot touches the smart pad (400) at 5.4 seconds. Accordingly, it can be seen that the user took a total of 5 steps during the 4-second section on the smart pad (400).

[0183] Meanwhile, the downward arrow in Figure 11 is shown to explain that the user's step data is accumulated in interval units.

[0184] FIG. 12 is a drawing illustrating a method for processing two-legged motion according to one embodiment of the present disclosure.

[0185] In Fig. 12(a), 1210 represents step data transmitted from a smart pad (400) to a display device (100).

[0186] Accuracy can be improved by adjusting the sensor sensitivity, as shown in Fig. 12(b), to sense more accurate data on the movement of both feet.

[0187] In Fig. 12(b), when the sensor sensitivity is 2, as in Fig. 11 described above, when it is '01 00' (1220, 1240), it is determined that the user's left foot has contacted (pressed) the smart pad (400), and according to the sensor sensitivity setting, it can be determined that the left foot has come off the smart pad (400) after contact only when '00 00' is input at least twice in succession (1230).

[0188] In the case corresponding to 1250 in Fig. 12(a), since '00 00' is inputted one-time and not continuously, the display device (100) can determine that the left foot is not completely lifted off the smart pad (400) in this case.

[0189] The display device (100) can distinguish between the left foot and the right foot based on data obtained from the smart pad (400) as shown in FIGS. 11 and 12, and can accurately distinguish when the left foot comes into contact (presses) with the smart pad (400) and then releases the contact (falls off), thereby obtaining more accurate user information and performing content playback control accordingly.

[0190] Meanwhile, in the above, the display device (100) may, for example, determine the number of steps in units of 4 seconds per second by the method of FIG. 7 or FIG. 9 as illustrated in FIG. 11 and control playback of the content, but may compare the number of steps in units of 4 seconds calculated at the current time (corresponding second) with the number of steps in units of 4 seconds calculated in the previous second, and if there is no change greater than a threshold, may not reflect the step number data in units of 4 seconds calculated at the current time in the content playback control or may ignore it and maintain the control according to the previous second. This process may be performed every time step number data in units of 4 seconds is calculated.

[0191] The above-described content mainly describes controlling the playback speed of content to be fast or slow based on the movement of a user walking while exercising using the smart pad (400) from the display device (100), i.e., user information.

[0192] However, the present disclosure is not limited thereto. For example, the display device (100) may control the playback speed of content based on the aforementioned user information, as well as increase or decrease the output audio level, and may also change the resolution.

[0193] For example, the display device (100) can lower the resolution when the user's movement (e.g., walking) on ​​the smart pad (400) obtained through FIG. 7 or 9 is faster than the reference, and can increase the resolution when the movement is slower.

[0194] In the present disclosure, the display device (100) can immediately control the playback speed, resolution, etc. of the content on a frame-by-frame basis. According to an embodiment, the display device (100) in the present disclosure can immediately control the playback speed, resolution, etc. of the content on a scene-by-scene basis. According to an embodiment, the display device (100) in the present disclosure can immediately control the playback speed, resolution, etc. of the content on a chapter-by-chapter basis.

[0195] In the present disclosure, the display device (100) can be immediately controlled on an object basis with respect to the playback speed, resolution, etc. of the content.

[0196] In the present disclosure, the display device (100) may be controlled differently depending on whether the current playback section is a main section or a non-main section of the content, with respect to the playback speed, resolution, etc. of the content. For example, the display device (100) may be controlled so that non-main sections are played faster or output at a lower resolution, and conversely, main sections are played slower or output at a higher resolution, even based on the same user's movement data.

[0197] In addition, according to an embodiment, the display device (100) in the present disclosure may control the playback speed, resolution, etc. of the content differently depending on the type or type of the content. Here, the control may include differently adjusting the reference speed or reference resolution, etc.

[0198] According to an embodiment, the display device (100) of the present disclosure may control the playback speed, resolution, etc. of the content differently depending on the remaining time until the end time of the content. For example, if the remaining time until the end time of the content is less than a threshold time, the reference speed or reference resolution may be adjusted.

[0199] In addition, according to an embodiment, the display device (100) in the present disclosure can determine whether to control the playback speed or the resolution depending on the type or type of the content when controlling the playback of the content.

[0200] In addition, according to an embodiment, the display device (100) in the present disclosure may control differently depending on whether there are multiple smart pads or only one that are currently connected and transmit signals containing user information, with respect to the playback speed, resolution, etc. of the content. For example, if two users are each performing a walking exercise through two smart pads and watching content through the display device (100), controlling the playback of the content based only on information obtained from a signal of a specific user may not conform to the intention of the other user. Therefore, in the above case, according to an embodiment, if the difference between the information related to content control obtained from the signals of the two smart pads is greater than a threshold, the display device (100) may ignore both pieces of information without reflecting them, but if the difference is less than the threshold, the content may be controlled based on one piece of information.

[0201] This specification describes a case where a user walks on a smart pad (400) as an example. However, the user may perform various movements or actions on the smart pad (400) in addition to walking. In this case, the display device (100) may calculate the calories burned based on the user's movement signals and control the playback of content based on the calories burned. In this case, the weighting may be applied differently depending on the user's physical conditions, such as weight, height, and gender.

[0202] Even if not specifically mentioned, the order of at least some of the operations disclosed in the present disclosure may be performed simultaneously, in a different order than the order described above, or some may be omitted / added.

[0203] According to one embodiment of the present disclosure, the above-described method can be implemented as processor-readable code on a medium in which a program is recorded. Examples of processor-readable media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices.

[0204] The display device described above is not limited to the configuration and method of the embodiments described above, and the embodiments may be configured by selectively combining all or part of each embodiment so that various modifications can be made.

[0205] The present disclosure relates to a display device that interacts with an external device such as a smart pad and controls playback of content according to a user's movements, and a method of operating the same, and has industrial applicability as it can be utilized for interactive motion control between a plurality of devices.

Claims

1. A display that outputs content; and A processor that obtains a user's step signal from a smart pad, calculates the number of steps of the user for a preset period of time from the user's step signal, and controls the playback of the output content based on the number of steps of the user. Display device.

2. In claim 1, To control playback of the above content, Including playback speed control of the above content, Display device.

3. In claim 2, The above processor, Calculating the number of steps per minute based on the number of steps the user calculated above. Display device.

4. In claim 3, The above processor, Based on the calculated number of steps per minute, the speed for controlling the playback speed of the content is determined. Display device.

5. In claim 2, To control playback of the above content, At least one of the audio output level adjustment and resolution adjustment controls of the above content is further included. Display device.

6. In claim 1, The above processor, Determining the preset time by considering the size of the buffer, 7. A display that outputs content; and A processor comprising: a processor for obtaining a user's step signal from a smart pad, calculating the number of steps of the user in a preset interval for a preset time range from the user's step signal, and controlling the playback of the output content based on the number of steps of the user. Display device.

8. In claim 7, To control playback of the above content, Including playback speed control of the above content, Display device.

9. In claim 8, The above processor, Setting multiple intervals for the above preset time range, each interval being set to accumulate from the initial time of the above preset time range. Display device.

10. In claim 9, The above processor, Calculate the number of steps per minute for each user step calculated for each of the above sections. Display device.

11. In claim 10, The above processor, The average number of steps is calculated by applying a weight to the number of steps per minute calculated for each of the above sections. Display device.

12. In claim 11, The above processor, Determining the playback speed of the content based on the average number of steps, and controlling the playback speed of the content based on the playback speed. Display device.

13. In claim 7, The above processor, Determining the preset time by considering the size of the buffer, 14. In claim 8, To control playback of the above content, At least one of the audio output level adjustment and resolution adjustment controls of the above content is further included. Display device.

15. Step of outputting content; Step of acquiring a user's step signal; A step of calculating the number of steps of the user for a preset time range from the step signal of the user; and A step of controlling the playback of the output content based on the number of steps of the user, The number of steps of the user calculated above is one of the total number of steps of the user in the preset time range and the number of steps in each section for the multiple sections set cumulatively for the preset time range. How the display device operates.

Images