Display device and method of operation thereof

Abstract

A display device according to an embodiment of the present disclosure may comprise a display; and a controller configured to: display a first image on a first split screen of the display, which a first user account has a control authority, display a second image on a second split screen of the display, which a second user account has the control authority, receive a voice command, control the first split screen based on a first analysis result of the voice command when a user account matching the received voice command is the first user account, and control the second split screen based on a second analysis result of the voice command when the user account matching the received voice command is the second user account.

Description

DISPLAY DEVICE AND METHOD OF OPERATION THEREOF

[0001] This disclosure relates to a display device, and more specifically, to a display device that provides a service that can separate the viewing environments of a plurality of users.

[0002] Recent display device divides the screen into multiple screens and provides a multi-view function that allows viewing multiple content screens. The multi-view function can split the screen to provide two broadcasts or content provided through a broadcast and an external input.

[0003] The multi-view function may be provided depending on the execution of a multi-view application installed on the display device.

[0004] In the past, when operating a multi-view function that provides a plurality of screens, in order to control each screen, the screen to be controlled was selected through a remote control device, and then the user uttered a voice to control the selected screen.

[0005] However, the user has the inconvenience of having to use a remote control device every time to control a specific screen under the multi-view function.

[0006] The purpose of the present disclosure may be to separate the viewing environment for multiple users through simple speech when providing a multi-view function.

[0007] The purpose of the present disclosure may be to individually control split screens by recognizing the user's voice characteristics in a multi-view environment.

[0008] The purpose of the present disclosure may be to recognize the user's voice characteristics in a multi-view environment and apply optimal setting values to the user to each split screen.

[0009] A display device according to an embodiment of the present disclosure may comprise a display; and a controller configured to: display a first image on a first split screen of the display, which a first user account has a control authority, display a second image on a second split screen of the display, which a second user account has the control authority, receive a voice command, control the first split screen based on a first analysis result of the voice command when a user account matching the received voice command is the first user account, and control the second split screen based on a second analysis result of the voice command when the user account matching the received voice command is the second user account.

[0010] A method of operating a display device according to an embodiment of the present disclosure may comprise displaying a first image on a first split screen of the display, which a first user account has a control authority, displaying a second image on a second split screen of the display, which a second user account has the control authority, receiving a voice command, controlling the first split screen based on a first analysis result of the voice command when a user account matching the received voice command is the first user account, and controlling the second split screen based on a second analysis result of the voice command when the user account matching the received voice command is the second user account.

[0011] According to an embodiment of the present disclosure, in an environment where multiple users are watching with one display device, each user can individually control the divided screen. Accordingly, an optimal viewing environment can be provided to each user.

[0012] According to an embodiment of the present disclosure, screen setting values can be applied individually to users in a multi-view environment, so that a viewing environment that the user prefers can be provided.

[0013] Figure 1 is a block diagram illustrating a configuration of a display device according to an embodiment of the present disclosure.

[0014] Figure 2 is a block diagram of a remote control device according to an embodiment of the present disclosure.

[0015] Figure 3 shows an example of an actual configuration of a remote control device according to an embodiment of the present disclosure.

[0016] Figure 4 shows an example of using a remote control device according to an embodiment of the present disclosure.

[0017] Figure 5 shows an artificial intelligence (AI) server according to an embodiment of the present disclosure.

[0018] Figure 6 is a diagram for explaining the configuration of a system according to an embodiment of the present disclosure.

[0019] Figure 7a is a sequence diagram for explaining a method of operating a system according to an embodiment of the present disclosure.

[0020] Figures 7b and 7c are diagrams illustrating a process of assigning control authority of each split screen to different users according to execution of the multi-view function.

[0021] Figure 8a is a sequence diagram for explaining a method of operating a system according to an embodiment of the present disclosure.

[0022] Figures 8b to 8e are diagrams explaining the process of controlling each split screen through a user account registered through voiceprint recognition.

[0023] Figure 9a is a diagram illustrating a process of individually controlling audio output from a split screen through voiceprint recognition according to an embodiment of the present disclosure, and Figure 9b is a diagram illustrating the process of individually controlling an external device linked to a user account through voiceprint recognition according to an embodiment of the present disclosure.

[0024] Figures 10a and 10b are diagrams illustrating a scenario for transferring screen control rights according to an embodiment of the present disclosure.

[0025] Figure 11 is a flowchart illustrating a method of operating a display device according to an embodiment of the present disclosure.

[0026] Figure 12 is a flowchart illustrating a method of operating a display device according to another embodiment of the present disclosure.

[0027] Hereinafter, the present disclosure will be described in more detail with reference to the drawings. Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. The suffixes "module" and "unit or portion" for components used in the following description are merely provided only for facilitation of preparing this specification, and thus they are not granted a specific meaning or function.

[0028] The display device according to an embodiment of the present disclosure is, for example, an intelligent display device in which a computer support function is added to a broadcast reception function, and is faithful to a broadcast reception function and has an Internet function added thereto, such as a handwritten input device, a touch screen Alternatively, a more user-friendly interface such as a spatial remote control may be provided. In addition, it is connected to the Internet and a computer with the support of a wired or wireless Internet function, so that functions such as e-mail, web browsing, banking, or games can also be performed. A standardized general-purpose OS may be used for these various functions.

[0029] Accordingly, in the display device described in the present disclosure, various user-friendly functions can be performed because various applications can be freely added or deleted, for example, on a general-purpose OS kernel. More specifically, the display device may be, for example, a network TV, HBBTV, smart TV, LED TV, OLED TV, and the like, and may be applied to a smart phone in some cases.

[0030] Figure 1 is a block diagram showing a configuration of a display device according to an embodiment of the present disclosure.

[0031] Referring to Figure 1, a display device 100 may include an image receiver 130, an external device interface 135, a memory 140, a user input interface 150, a controller 170, a wireless communication interface 173, a display 180, a speaker 185, and a power supply circuit 190.

[0032] The image receiver 130 may include a tuner 131, a demodulator 132, and a network interface 133.

[0033] The tuner 131 may select a specific broadcast channel according to a channel selection command. The tuner 131 may receive a broadcast signal for the selected specific broadcast channel.

[0034] The demodulator 132 may separate the received broadcast signal into an image signal, an audio signal, and a data signal related to a broadcast program, and restore the separated image signal, audio signal, and data signal to a format capable of being output.

[0035] The external device interface 135 may receive an application or a list of applications in an external device adjacent thereto, and transmit the same to the controller 170 or the memory 140.

[0036] The external device interface 135 may provide a connection path between the display device 100 and an external device. The external device interface 135 may receive one or more of images and audio output from an external device connected to the display device 100 in a wired or wireless manner, and transmit the same to the controller 170. The external device interface 135 may include a plurality of external input terminals. The plurality of external input terminals may include an RGB terminal, one or more High Definition Multimedia Interface (HDMI) terminals, and a component terminal.

[0037] The image signal of the external device input through the external device interface unit 135 may be output through the display 180. The audio signal of the external device input through the external device interface 135 may be output through the speaker 185.

[0038] The external device connectable to the external device interface 135 may be any one of a set-top box, a Blu-ray player, a DVD player, a game machine, a sound bar, a smartphone, a PC, a USB memory, and a home theater, but this is only an example.

[0039] The network interface 133 may provide an interface for connecting the display device 100 to a wired / wireless network including an Internet network. The network interface 133 may transmit or receive data to or from other users or other electronic devices through a connected network or another network linked to the connected network.

[0040] In addition, a part of content data stored in the display device 100 may be transmitted to a selected user among a selected user or a selected electronic device among other users or other electronic devices registered in advance in the display device 100.

[0041] The network interface 133 may access a predetermined web page through the connected network or the other network linked to the connected network. That is, it is possible to access a predetermined web page through a network, and transmit or receive data to or from a corresponding server.

[0042] In addition, the network interface 133 may receive content or data provided by a content provider or a network operator. That is, the network interface 133 may receive content such as movies, advertisements, games, VOD, and broadcast signals and information related thereto provided from a content provider or a network provider through a network.

[0043] In addition, the network interface 133 may receive update information and update files of firmware provided by the network operator, and may transmit data to an Internet or content provider or a network operator.

[0044] The network interface 133 may select and receive a desired application from among applications that are open to the public through a network.

[0045] The memory 140 may store programs for signal processing and control of the controller 170, and may store images, audio, or data signals, which have been subjected to signal-processed.

[0046] In addition, the memory 140 may perform a function for temporarily storing images, audio, or data signals input from an external device interface 135 or the network interface 133, and store information on a predetermined image through a channel storage function.

[0047] The memory 140 may store an application or a list of applications input from the external device interface 135 or the network interface 133.

[0048] The display device 100 may play back a content file (a moving image file, a still image file, a music file, a document file, an application file, or the like) stored in the memory 140 and provide the same to the user.

[0049] The user input interface 150 may transmit a signal input by the user to the controller 170 or a signal from the controller 170 to the user. For example, the user input interface 150 may receive and process a control signal such as power on / off, channel selection, screen settings, and the like from the remote control device 200 in accordance with various communication methods, such as a Bluetooth communication method, a WB (Ultra Wideband) communication method, a ZigBee communication method, an RF (Radio Frequency) communication method, or an infrared (IR) communication method or may perform processing to transmit the control signal from the controller 170 to the remote control device 200.

[0050] In addition, the user input interface 150 may transmit a control signal input from a local key (not shown) such as a power key, a channel key, a volume key, and a setting value to the controller 170.

[0051] The image signal image-processed by the controller 170 may be input to the display 180 and displayed as an image corresponding to a corresponding image signal. Also, the image signal image-processed by the controller 170 may be input to an external output device through the external device interface 135.

[0052] The audio signal processed by the controller 170 may be output to the speaker 185. Also, the audio signal processed by the controller 170 may be input to the external output device through the external device interface 135.

[0053] In addition, the controller 170 may control the overall operation of the display device 100.

[0054] In addition, the controller 170 may control the display device 100 by a user command input through the user input interface 150 or an internal program and connect to a network to download an application a list of applications or applications desired by the user to the display device 100.

[0055] The controller 170 may allow the channel information or the like selected by the user to be output through the display 180 or the speaker 185 along with the processed image or audio signal.

[0056] In addition, the controller 170 may output an image signal or an audio signal through the display 180 or the speaker 185, according to a command for playing back an image of an external device through the user input interface 150, the image signal or the audio signal being input from an external device, for example, a camera or a camcorder, through the external device interface 135.

[0057] Meanwhile, the controller 170 may allow the display 180 to display an image, for example, allow a broadcast image which is input through the tuner 131 or an external input image which is input through the external device interface 135, an image which is input through the network interface unit or an image which is stored in the memory 140 to be displayed on the display 180. In this case, an image being displayed on the display 180 may be a still image or a moving image, and may be a 2D image or a 3D image.

[0058] In addition, the controller 170 may allow content stored in the display device 100, received broadcast content, or external input content input from the outside to be played back, and the content may have various forms such as a broadcast image, an external input image, an audio file, still images, accessed web screens, and document files.

[0059] The wireless communication interface 173 may communicate with an external device through wired or wireless communication. The wireless communication interface 173 may perform short range communication with an external device. To this end, the wireless communication interface 173 may support short range communication using at least one of Bluetooth쪠, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Near Field Communication (NFC), Wi-Fi (Wireless-Fidelity), Wi-Fi(Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies. The wireless communication interface 173 may support wireless communication between the display device 100 and a wireless communication system, between the display device 100 and another display device 100, or between the display device 100 and a network in which the display device 100 (or an external server) is located through wireless area networks. The wireless area networks may be wireless personal area networks.

[0060] Here, the another display device 100 may be a wearable device (e.g., a smartwatch, smart glasses or a head mounted display (HMD), a mobile terminal such as a smart phone, which is able to exchange data (or interwork) with the display device 100 according to the present disclosure. The wireless communication interface 173 may detect (or recognize) a wearable device capable of communication around the display device 100.

[0061] Furthermore, when the detected wearable device is an authenticated device to communicate with the display device 100 according to the present disclosure, the controller 170 may transmit at least a portion of data processed by the display device 100 to the wearable device through the wireless communication interface 173. Therefore, a user of the wearable device may use data processed by the display device 100 through the wearable device.

[0062] The display 180 may convert image signals, data signals, and OSD signals processed by the controller 170, or image signals or data signals received from the external device interface 135 into R, G, and B signals, and generate drive signals.

[0063] Meanwhile, since the display device 100 shown in Figure 1 is only an embodiment of the present disclosure, some of the illustrated components may be integrated, added, or omitted depending on the specification of the display device 100 that is actually implemented.

[0064] That is, two or more components may be combined into one component, or one component may be divided into two or more components as necessary. In addition, a function performed in each block is for describing an embodiment of the present disclosure, and its specific operation or device does not limit the scope of the present disclosure.

[0065] According to another embodiment of the present disclosure, unlike the display device 100 shown in Figure 1, the display device 100 may receive an image through the network interface 133 or the external device interface 135 without a tuner 131 and a demodulator 132 and play back the same.

[0066] For example, the display device 100 may be divided into an image processing device, such as a set-top box, for receiving broadcast signals or content according to various network services, and a content playback device that plays back content input from the image processing device.

[0067] In this case, an operation method of the display device according to an embodiment of the present disclosure will be described below may be implemented by not only the display device 100 as described with reference to Figure 1 and but also one of an image processing device such as the separated set-top box and a content playback device including the display 180 the speaker 185.

[0068] Next, a remote control device according to an embodiment of the present disclosure will be described with reference to Figures 2 to 3.

[0069] Figure 2 is a block diagram of a remote control device according to an embodiment of the present disclosure, and Figure 3 shows an actual configuration example of a remote control device 200 according to an embodiment of the present disclosure.

[0070] First, referring to Figure 2, the remote control device 200 may include a fingerprint reader 210, a wireless communication circuit 220, a user input interface 230, a sensor 240, an output interface 250, a power supply circuit 260, a memory 270, a controller 280, and a microphone 290.

[0071] Referring to Figure 2, the wireless communication circuit 220 may transmit and receive signals to and from any one of display devices according to embodiments of the present disclosure described above.

[0072] The remote control device 200 may include an RF circuit 221 capable of transmitting and receiving signals to and from the display device 100 according to the RF communication standard, and an IR circuit 223 capable of transmitting and receiving signals to and from the display device 100 according to the IR communication standard. In addition, the remote control device 200 may include a Bluetooth circuit 225 capable of transmitting and receiving signals to and from the display device 100 according to the Bluetooth communication standard. In addition, the remote control device 200 may include an NFC circuit 227 capable of transmitting and receiving signals to and from the display device 100 according to the NFC (near field communication) communication standard, and a WLAN circuit 229 capable of transmitting and receiving signals to and from the display device 100 according to the wireless LAN (WLAN) communication standard.

[0073] In addition, the remote control device 200 may transmit a signal containing information on the movement of the remote control device 200 to the display device 100 through the wireless communication circuit 220.

[0074] In addition, the remote control device 200 may receive a signal transmitted by the display device 100 through the RF circuit 221, and transmit a command regarding power on / off, channel change, volume adjustment, or the like to the display device 100 through the IR circuit 223 as necessary.

[0075] The user input interface 230 may include a keypad, a button, a touch pad, a touch screen, or the like. The user may input a command related to the display device 100 to the remote control device 200 by operating the user input interface 230. When the user input interface 230 includes a hard key button, the user may input a command related to the display device 100 to the remote control device 200 through a push operation of the hard key button. Details will be described with reference to Figure 3.

[0076] Referring to Figure 3, the remote control device 200 may include a plurality of buttons. The plurality of buttons may include a fingerprint recognition button 212, a power button 231, a home button 232, a live button 233, an external input button 234, a volume control button 235, a voice recognition button 236, a channel change button 237, an OK button 238, and a back-play button 239.

[0077] The fingerprint recognition button 212 may be a button for recognizing a user's fingerprint. In one embodiment, the fingerprint recognition button 212 may enable a push operation, and thus may receive a push operation and a fingerprint recognition operation.

[0078] The power button 231 may be a button for turning on / off the power of the display device 100.

[0079] The home button 232 may be a button for moving to the home screen of the display device 100.

[0080] The live button 233 may be a button for displaying a real-time broadcast program.

[0081] The external input button 234 may be a button for receiving an external input connected to the display device 100.

[0082] The volume control button 235 may be a button for adjusting the level of the volume output by the display device 100.

[0083] The voice recognition button 236 may be a button for receiving a user's voice and recognizing the received voice.

[0084] The channel change button 237 may be a button for receiving a broadcast signal of a specific broadcast channel.

[0085] The OK button 238 may be a button for selecting a specific function, and the back-play button 239 may be a button for returning to a previous screen.

[0086] A description will be given referring again to Figure 2.

[0087] When the user input interface 230 includes a touch screen, the user may input a command related to the display device 100 to the remote control device 200 by touching a soft key of the touch screen. In addition, the user input interface 230 may include various types of input means that may be operated by a user, such as a scroll key or a jog key, and the present embodiment does not limit the scope of the present disclosure.

[0088] The sensor 240 may include a gyro sensor 241 or an acceleration sensor 243, and the gyro sensor 241 may sense information regarding the movement of the remote control device 200.

[0089] For example, the gyro sensor 241 may sense information about the operation of the remote control device 200 based on the x, y, and z axes, and the acceleration sensor 243 may sense information about the moving speed of the remote control device 200. Meanwhile, the remote control device 200 may further include a distance measuring sensor to sense the distance between the display device 100 and the display 180.

[0090] The output interface 250 may output an image or audio signal corresponding to the operation of the user input interface 230 or a signal transmitted from the display device 100.

[0091] The user may recognize whether the user input interface 230 is operated or whether the display device 100 is controlled through the output interface 250.

[0092] For example, the output interface 450 may include an LED 251 that emits light, a vibrator 253 that generates vibration, a speaker 255 that outputs sound, or a display 257 that outputs an image when the user input interface 230 is operated or a signal is transmitted and received to and from the display device 100 through the wireless communication unit 225.

[0093] In addition, the power supply circuit 260 may supply power to the remote control device 200, and stop power supply when the remote control device 200 has not moved for a predetermined time to reduce power consumption.

[0094] The power supply circuit 260 may restart power supply when a predetermined key provided in the remote control device 200 is operated.

[0095] The memory 270 may store various types of programs and application data required for control or operation of the remote control device 200.

[0096] When the remote control device 200 transmits and receives signals wirelessly through the display device 100 and the RF circuit 221, the remote control device 200 and the display device 100 transmit and receive signals through a predetermined frequency band.

[0097] The controller 280 of the remote control device 200 may store and refer to information on a frequency band capable of wirelessly transmitting and receiving signals to and from the display device 100 paired with the remote control device 200 in the memory 270.

[0098] The controller 280 may control all matters related to the control of the remote control device 200. The controller 280 may transmit a signal corresponding to a predetermined key operation of the user input interface 230 or a signal corresponding to the movement of the remote control device 200 sensed by the sensor 240 through the wireless communication unit 225.

[0099] Also, the microphone 290 of the remote control device 200 may obtain a speech.

[0100] A plurality of microphones 290 may be provided.

[0101] Next, a description will be given referring to Figure 4.

[0102] Figure 4 shows an example of using a remote control device according to an embodiment of the present disclosure.

[0103] In Figure 4, (a) illustrates that a pointer 205 corresponding to the remote control device 200 is displayed on the display 180.

[0104] The user may move or rotate the remote control device 200 up, down, left and right. The pointer 205 displayed on the display 180 of the display device 100 may correspond to the movement of the remote control device 200. As shown in the drawings, the pointer 205 is moved and displayed according to movement of the remote control device 200 in a 3D space, so the remote control device 200 may be called a space remote control device.

[0105] In (b) of Figure 4, it is illustrated that that when the user moves the remote control device 200 to the left, the pointer 205 displayed on the display 180 of the display device 100 moves to the left correspondingly.

[0106] Information on the movement of the remote control device 200 detected through a sensor of the remote control device 200 is transmitted to the display device 100. The display device 100 may calculate the coordinates of the pointer 205 based on information on the movement of the remote control device 200. The display device 100 may display the pointer 205 to correspond to the calculated coordinates.

[0107] In (c) of Figure 4, it is illustrated that a user moves the remote control device 200 away from the display 180 while pressing a specific button in the remote control device 200. Accordingly, a selected area in the display 180 corresponding to the pointer 205 may be zoomed in and displayed enlarged.

[0108] Conversely, when the user moves the remote control device 200 to be close to the display 180, the selected area in the display 180 corresponding to the pointer 205 may be zoomed out and displayed reduced.

[0109] On the other hand, when the remote control device 200 moves away from the display 180, the selected area may be zoomed out, and when the remote control device 200 moves to be close to the display 180, the selected area may be zoomed in.

[0110] Also, in a state in which a specific button in the remote control device 200 is being pressed, recognition of up, down, left, or right movements may be excluded. That is, when the remote control device 200 moves away from or close to the display 180, the up, down, left, or right movements are not recognized, and only the forward and backward movements may be recognized. In a state in which a specific button in the remote control device 200 is not being pressed, only the pointer 205 moves according to the up, down, left, or right movements of the remote control device 200.

[0111] Meanwhile, the movement speed or the movement direction of the pointer 205 may correspond to the movement speed or the movement direction of the remote control device 200.

[0112] Meanwhile, in the present specification, a pointer refers to an object displayed on the display 180 in response to an operation of the remote control device 200. Accordingly, objects of various shapes other than the arrow shape shown in the drawings are possible as the pointer 205. For example, the object may be a concept including a dot, a cursor, a prompt, a thick outline, and the like. In addition, the pointer 205 may be displayed corresponding to any one point among points on a horizontal axis and a vertical axis on the display 180, and may also be displayed corresponding to a plurality of points such as a line and a surface.

[0113] Figure 5 shows an artificial intelligence (AI) server according to an embodiment of the present disclosure.

[0114] Referring to Figure 5, the AI server 500 may refer to a device that trains an artificial neural network using a machine learning algorithm or uses a learned artificial neural network.

[0115] The AI server 500 may be composed of a plurality of servers to perform distributed processing, and may be defined as a 5G network.

[0116] The AI server 500 may be included as a part of the display device 100 and may perform at least part of the AI processing.

[0117] The AI server 500 may include a communication interface 510, a memory 530, a learning processor 540, and a processor 560.

[0118] The communication interface 510 may transmit and receive data with an external device such as the display device 100.

[0119] The memory 530 may include a model memory 531.

[0120] The model memory 531 may store a model (or artificial neural network, 531a) that is being trained or has been learned through the learning processor 540.

[0121] The learning processor 540 may train the artificial neural network 531a using training data. The learning model may be used while mounted on the AI server 500 of the artificial neural network, or may be mounted and used on an external device such as the display device 100.

[0122] The learning model may be implemented in hardware, software, or a combination of hardware and software. When part or all of the learning model is implemented as software, one or more instructions constituting the learning model may be stored in the memory 530.

[0123] The processor 560 may infer a result value for new input data using the learning model and generate a response or control command based on the inferred result value.

[0124] Figure 6 is a diagram for explaining the configuration of a system according to an embodiment of the present disclosure.

[0125] Referring to Figure 6, the system 60 may include a display device 100, an AI server 500, a Speech-To-Text (STT) server 620, a Natural Language Processing (NLP) server 630, and a speaker recognition server 600.

[0126] Each element constituting the system 60 may perform wireless communication with each other. The wireless communication may be an Internet communication.

[0127] The speaker recognition server 600 may be called a speaker recognition device. The speaker recognition server 600 may identify the speaker by comparing a voice feature of the voice signal with a pre-stored voice feature.

[0128] The AI server 500 may transmit the voice signal transmitted from the display device 100 to the speaker recognition server 600 or the STT server 620.

[0129] The AI server 500 may be a relay server that relays communication between the display device 100 and an external server.

[0130] AI server 500 may be omitted. In this case, the display device 100 may directly transmit a voice signal to the speaker recognition server 600 or the STT server 620.

[0131] The STT (Speech-To-Text) server 620 may convert a voice signal into a text data.

[0132] The STT server 620 may transmit the converted text data to the NLP server 630.

[0133] The Natural Language Processing (NLP) server 630 may obtain an intent analysis result based on text data received from the STT server 620 using a natural language processing (NLP) engine.

[0134] The NLP server 630 may transmit the obtained intention analysis result to the display device 100 through the AI server 500.

[0135] The speaker database 601 may be included in the speaker recognition server 600 or may be provided separately from the speaker recognition server 600.

[0136] The speaker database 601 may store a speaker profile corresponding to each speaker. The speaker profile may include one or more of the speaker's gender, age, nickname, payment information, email, and phone number. The speaker profile may be referred to as an account profile.

[0137] The speaker database 601 may store embedding vectors corresponding to each of a plurality of registered speakers and speaker identification information matched to each embedding vector.

[0138] The speaker profile may include an embedding vector representing the voice feature of the registered speaker.

[0139] Hereinafter, the speaker registration process may be a process of extracting features of a voice uttered by a specific speaker using voice recognition technology, matching the extracted features to speaker identification information, and storing them. The speaker registration process may be a voice print registration process.

[0140] The speaker identification process may be a process of identifying whether a pre-registered speaker matches the speaker currently speaking.

[0141] The speaker registration process and the speaker identification process may be processes for providing a personalized voice recognition service.

[0142] Figure 7a is a sequence diagram for explaining a method of operating a system according to an embodiment of the present disclosure.

[0143] In particular, Figure 7a is a sequence diagram to explain the process of granting control authority to each screen according to execution of the multi-view function.

[0144] Hereinafter, app is an abbreviation for application.

[0145] Referring to Figure 7a, the controller 170 of the display device 100 may display the first image on the entire screen of the display 180 (S701).

[0146] The first user account may be logged in to the display device 100.

[0147] The first image may be an image displayed according to execution of the first app executed through the first user account.

[0148] The controller 170 of the display device 100 may receive a voice command for executing the multi-view function (S703).

[0149] The multi-view function may be a function that provides a plurality of split screens on the entire screen of the display 180. An execution image according to the execution of a specific app may be displayed on each of the plurality of split screens.

[0150] The multi-view function may be provided by executing a multi-view application installed on the display device 100.

[0151] In one embodiment, the controller 170 may receive a voice command for executing the multi-view function from the remote control device 200 through the user input interface 150.

[0152] In another embodiment, the controller 170 may receive a voice command for executing the multi-view function through a microphone (not shown) provided in the user input interface 150.

[0153] In another embodiment, the controller 170 may receive a voice command for executing a multi-view function from an artificial intelligence agent device connected through the wireless communication interface 173.

[0154] A voice command may be a command requesting execution of a multi-view function or execution of a specific app.

[0155] The controller 170 of the display device 100 may transmit a voice signal corresponding to a voice command to the AI server 500 through the network interface 133 (S705).

[0156] In one embodiment, the processor 560 of the AI server 500 may transmit the received voice signal to the STT server 620.

[0157] In another embodiment, the controller 170 of the display device 100 may directly transmit a voice signal corresponding to a voice command to the STT server 620 through the network interface 133.

[0158] The STT server 620 may convert the voice signal into text data through the STT engine and transmit the converted text data to the NLP server 630.

[0159] The processor 560 of the AI server 500 may obtain the intent analysis result of the voice command from the voice signal (S707).

[0160] The processor 560 of the AI server 500 may receive an intention analysis result matching the text data of the voice signal from the NLP server 630.

[0161] In another embodiment, the processor 560 of the AI server 500 may convert a voice signal into text data through an STT engine and obtain an intent analysis result from the converted text data using an NLP engine.

[0162] The intent analysis result may be a command to execute a multi-view application or execute a specific app. The specific app may be the same as the first app or may be a different app.

[0163] The processor 560 of the AI server 500 may acquire voice feature information (S709).

[0164] In one embodiment, the processor 560 may extract voice feature information from a voice signal. The voice feature information may be either a voice feature vector or an embedding vector representing voice feature.

[0165] The processor 560 may determine whether the speaker who uttered the voice signal is a registered account through a voiceprint recognition process. The voiceprint recognition process may be a process of extracting a voice feature vector (or embedding vector) from a voice signal and identifying the speaker by comparing the extracted voice feature vector with a feature vector stored in the speaker database 601.

[0166] The processor 560 of the AI server 500 may determine whether the user account corresponding to the acquired voice feature information is the registered first user account (S711).

[0167] The processor 560 of the AI server 500 may determine whether the user account corresponding to the acquired voice feature information is an account already registered in the speaker database 601.

[0168] The processor 560 may measure the similarity between the extracted speech feature vector and the stored feature vector, and if the measured similarity is greater than or equal to a preset similarity, it may obtain speaker identification information matching the stored feature vector.

[0169] The speaker identification information may be information that identifies the user account that issued the voice command. The speaker identification account may include information about the user account. Information about the user account may include one or more of the user's ID or name.

[0170] If the user account corresponding to the acquired voice feature information is a pre-registered first user account, the AI server 500 may transmit the first speaker identification information identifying the first user account and the intent analysis result to the display device 100 through the interface 510 (S713).

[0171] The first speaker identification information identifying the first user account may include the ID and name of the first user account, or registration account information indicating that the first user account is an account already registered through a voiceprint recognition process.

[0172] The controller 170 of the display device 100 may assign the control authority for the first split screen and the second split screen to the first user account based on the received first speaker identification information (S715).

[0173] The controller 170 may assign the control authority of the second split screen to the first user account based on the received first speaker identification information. The control authority for the second split screen may be the authority to control the language of the video, audio, or video output from the second split screen.

[0174] The control authority of the second split screen may further include app control authority that can control the app running on the second split screen. The control authority of the app may be the permission to control the functions or actions of the app executed according to the execution of the multi-view function through voice.

[0175] The control authority of the second split screen may include the authority to control an external device linked to the second user account that has the control authority of the second split screen. The external device linked to the second user account may be a device that was previously connected through the second user account.

[0176] The controller 170 of the display device 100 may display the first image on the first split screen of the display 180 based on the received intention analysis result, and display the second image on the second split screen of the display 180 (S717).

[0177] The first image may be the image displayed on the entire screen in step S701. The second video may be an execution video of an app executed according to the result of analysis of the intention of the voice command.

[0178] That is, the controller 170 may display the first image, which was previously displayed on the full screen, on the first split screen according to the user's voice command for executing the multi-view function, and the second image, which is the execution image of a specific app, on the second split screen.

[0179] As the control authority for the second split screen is granted to the first user account, the controller 170 may apply setting values matching the first user account to the second split screen.

[0180] The memory 140 may store a setting value set matching each of a plurality of user accounts. The setting value set may include one or more of values of the picture quality elements of the display 180, a sound quality of the speaker 185, a volume, and a language, or information about an external device (e.g., an audio output device) connected to the user account. The information about the external device may include setting values related to the sound or operation of the external device.

[0181] If the first video is played by a first user account that is already logged in, the controller 170 may also assign the control authority of the first split screen to the first user account.

[0182] The controller 170 may further display a first user indicator on the display 180 indicating that control authority for the first and second split screens belongs to the first user account.

[0183] The controller 170 may display a first user indicator on one side of the second split screen indicating that the first user account has control authority for the second split screen provided according to execution of the multi-view function.

[0184] Meanwhile, when the first image is executed by a first user account that is already logged in, the controller 170 may display a first user indicator indicating that the control authority for the first split screen belongs to the first user account on one side of the first split screen.

[0185] Meanwhile, the AI server 500, if the user account corresponding to the acquired voice feature information is a pre-registered second user account (S719), the processor 560 may transmit the intent analysis result and second speaker identification information identifying the second user account to the display device 100 through the communication interface 510 (S721).

[0186] The second speaker identification information identifying the second user account may include the ID and name of the second user account, or registration account information indicating that the second user account is a previously registered account through a voiceprint recognition process.

[0187] The controller 170 of the display device 100 may assign the control authority of the first split screen to the first user account and the control authority of the second split screen to the second user account based on the second speaker identification information (S723).

[0188] The controller 170 may assign the control authority of the second split screen to a pre-registered second user account based on the received second speaker identification information.

[0189] When control authority for the second split screen is granted to the second user account, the controller 170 may apply a setting value set matching the second user account on the second split screen.

[0190] If the first video is played by the first user account that is already logged in, the controller 170 may assign the control authority of the first split screen to the first user account. Accordingly, the controller 170 may apply a setting value set matching the first user account on the first split screen.

[0191] The controller 170 of the display device 100 may display the first image on the first split screen of the display 180 and display the second image on the second split screen of the display 180 based on the intention analysis result (S725).

[0192] The controller 170 may display the first image that was previously displayed on the full screen on the first split screen according to the user's voice command for executing the multi-view function, and display the second image that is the execution image of a specific app on the second split screen.

[0193] The controller 170 of the display device 100 may further display a second user indicator on the display 180 indicating that control authority for the second split screen belongs to the second user account based on the received second speaker identification information.

[0194] The controller 170 may display a second user indicator on one side of the second split screen indicating that the second user account has control authority for the second split screen provided according to execution of the multi-view function.

[0195] When the first image is executed by a first user account that is already logged in, the controller 170 may display a first user indicator on one side of the first split screen indicating that the control authority for the first split screen is in the first user account.

[0196] Meanwhile, the already logged in first user account cannot control the second split screen owned by the second user account through voice commands. However, the already logged in first user account can control all operations other than controlling the second split screen through voice.

[0197] In another embodiment, if the first user account is an unregistered account that has not undergone voiceprint recognition and is a simple login account, the controller 170 may also assign the control authority of the first split screen to the second user account.

[0198] The controller 170 may apply a setting value set matching the second user account on the second split screen.

[0199] In another embodiment, the controller 170 may automatically connect a first external device linked to a first user account having the control authority of the first split screen to a second external device linked to a second user account having the control authority of the second split screen through the wireless communication interface 173.

[0200] Meanwhile, the AI server 500, the processor 560, may transmit an uncontrollable notification to the display device 100 through the communication interface 510 (S727) when the user account corresponding to the acquired voice feature information is an unregistered account (S719).

[0201] The controller 170 of the display device 100 may output the received uncontrollable notification (S729).

[0202] In one embodiment, the uncontrollable notification may be a notification indicating that the multi-view function cannot be executed.

[0203] In another embodiment, the uncontrollable notification may be a notification indicating that the speaker recognition service is not supported because of the unregistered voice.

[0204] The controller 170 may display an uncontrollable notification through the display 180 or output it as a voice through the speaker 185.

[0205] Meanwhile, if the already logged in first user account is an unregistered account that has not gone through a voiceprint recognition process, the controller 170 may assign the control authority of the first split screen to the unregistered account.

[0206] Figures 7b and 7c are diagrams illustrating a process of assigning control authority of each split screen to different users according to execution of the multi-view function.

[0207] Referring to Figure 7b, a first user account corresponding to the first user A may be logged in to the display device 100.

[0208] Figures 7b and 7c, the first app may be an app named Live TV that provides a real-time broadcast program, and the second app may be an app named tube that provides a video streaming service.

[0209] The display device 100 may display the first image 711, which is an execution image of the first app executed by the first user account, on the entire screen of the display 180. The display device 100 may further display a first user indicator 701 on the display 180 indicating a logged in state by the first user account.

[0210] If the first user account is an account that has already been registered through a voiceprint recognition process, the first user indicator 701 may indicate that the first user account has control rights of the entire screen.

[0211] The second user B may utter a voice command to execute the multi-view function. For example, the voice command for executing the multi-view function may be <Open tube by multi view>. A voice command may be a command requesting execution of a multi-view function or execution of a specific app.

[0212] The display device 100 may transmit a voice signal corresponding to a voice command to the AI server 500, and receive speaker identification information corresponding to the voice signal and an intention analysis result of the voice command from the AI server 500. .

[0213] When the speaker identification information includes information about a second user account registered through a voiceprint recognition process, the display device 100 may execute the multi-view function, as shown in Figure 7c.

[0214] The display device 100 may display the first image 711 on the first split screen 710 and the second image 731 on the second split screen 730 according to the execution of the multi-view function. The first split screen 710 may be a screen in which an execution image of the first app is provided, and the second execution screen 730 may be a screen in which an execution image of the second app is provided.

[0215] The display device 100 may further display text 721 corresponding to the voice command uttered by the second user B on the display 180.

[0216] If the second user account is an account already registered through voiceprint recognition (or voice identification), the display device 100 may assign the control authority of the second split screen 730 to the second user account.

[0217] If the already logged in first user account is an account already registered through voiceprint recognition, the display device 100 may assign the control authority of the first split screen 710 to the first user account.

[0218] The display device 100 may display a first user indicator 701 within the first split screen 710 or on one side of the first split screen 710, indicating that the control authority for the first split screen 710 is in the first user account.

[0219] The display device 100 may display a second user indicator 703 within the second split screen 730 or on one side of the second split screen 730, indicating that the control authority for the second split screen 730 is in the second user account.

[0220] The user can intuitively recognize who has the control authority over each divided screen through the first user indicator 701 and the second user indicator 703.

[0221] Figure 8a is a sequence diagram for explaining a method of operating a system according to an embodiment of the present disclosure.

[0222] In particular, Figure 8a is a sequence diagram for explaining the process of controlling each divided screen.

[0223] The controller 170 of the display device 100 may display a first image on the first split screen and a second image on the second split screen (S801).

[0224] The controller 170 may display the first image on the first split screen and the second image on the second split screen according to the execution of the multi-view function.

[0225] The control authority for the first split screen may reside in the first user account, and the control authority for the second split screen may reside in the second user account.

[0226] The controller 170 of the display device 100 may receive a voice command (S803) and transmit a voice signal corresponding to the received voice command to the AI server 500 through the network interface 133 (S805).

[0227] In one embodiment, the controller 170 may receive the voice command from the remote control device 200 through the user input interface 150.

[0228] In another embodiment, the controller 170 may receive the voice command through a microphone (not shown) provided in the user input interface 150.

[0229] In another embodiment, the controller 170 may receive the voice command from an artificial intelligence agent device connected through the wireless communication interface 173.

[0230] The voice command may be a command for controlling the split screen. For example, the voice command may be a command indicating a search request for content.

[0231] The processor 560 of the AI server 500 may obtain voice feature information based on the voice signal (S807).

[0232] The processor 560 of the AI server 500 may determine whether the account identified through the voice feature information is a pre-registered first user account (S809), and when the account identified through the voice feature information is the pre-registered first user account, obtain a first analysis result of the voice command (S811).

[0233] The processor 560 may receive an intention analysis result matching the text data of the voice signal from the NLP server 630. The processor 560 may obtain an analysis result for the voice command based on the intention analysis result.

[0234] The intent analysis result of the voice command may be one of a content search request, a request to control video or audio displayed on a split screen, or a request to control an external device linked to the user account.

[0235] If the intention analysis result is the content search request, the processor 560 may obtain the first analysis result including a content search result through the first app that is the source of the first split screen. The processor 560 may transmit the content search request to the server of the first app and receive the content search result from the server of the first app in response to the content search request.

[0236] That is, when the processor 560 recognizes that the voice command is uttered through the first user account, the processor 560 may provide the analysis result of the voice command through the first app that matches the first split screen.

[0237] If the intent analysis result indicates a volume adjustment of the audio being output on the first split screen, the processor 560 may transmit the first analysis result indicating the volume adjustment of the audio to the display device 100.

[0238] The processor 560 of the AI server 500 may transmit the first analysis result obtained through the communication interface 510 to the display device 100 (S813).

[0239] The controller 170 of the display device 100 may control the first split screen based on the received first analysis result (S815).

[0240] Based on the first analysis result, the controller 170 may control video and audio output from the first split screen, the operation of an app running on the first split screen, or an external device linked to the user account.

[0241] If the first analysis result is the content search result searched through the first app, the controller 170 may display the first analysis result on the first split screen.

[0242] If the first analysis result is a command to adjust the volume of audio being output on the first split screen, the controller 170 may adjust the volume of the audio being output on the first split screen.

[0243] The processor 560 of the AI server 500 determines that the account identified through voice feature information is not the pre-registered first user account (S809) and is determined to be a pre-registered second user account (S817), may obtain a second analysis result of the voice command (S819).

[0244] The processor 560 may receive an intention analysis result matching the text data of the voice signal from the NLP server 630. The processor 560 may obtain an analysis result for the voice command based on the intention analysis result. The intent analysis result of the voice command may be one of a content search request and a request to control video or audio displayed on a split screen.

[0245] If the intent analysis result is the content search request, the processor 560 may obtain a second analysis result including the content search result through the second app that is the source of the second split screen. The processor 560 may transmit the content search request to the server of the second app and receive the content search result from the server of the second app in response to the content search request.

[0246] That is, when the processor 560 recognizes that the voice command is uttered through the second user account, it may provide the analysis result of the voice command through the second app that matches the second split screen.

[0247] If the intent analysis result indicates volume adjustment of the audio being output on the second split screen, the processor 560 may transmit a second analysis result indicating audio volume adjustment to the display device 100.

[0248] The processor 560 of the AI server 500 may transmit the second analysis result obtained through the communication interface 510 to the display device 100 (S821).

[0249] The controller 170 of the display device 100 may control the second split screen based on the received second analysis result (S823).

[0250] Meanwhile, the AI server 500, the processor 560, if the user account corresponding to the acquired speaker identification information is an unregistered account (S817), may transmit an uncontrollable notification to the display device 100 through the communication interface 510 (S825).

[0251] The controller 170 of the display device 100 may output the received uncontrollable notification (S827).

[0252] In one embodiment, the uncontrollable notification may be a notification indicating that control of the split screen is not possible.

[0253] The uncontrollable notification may be a notification indicating that the speaker recognition service is not supported because the voice is not registered.

[0254] The controller 170 may display the uncontrollable notification through the display 180 or output it as a voice through the speaker 185.

[0255] Figures 8b to 8e are diagrams explaining the process of controlling each split screen through a user account registered through voiceprint recognition.

[0256] In Figures 8b to 8e, it is assumed that the control authority of the first split screen 710 belongs to the first user A with the first user account, and the control authority of the second split screen 730 belongs to the second user B with the second user account.

[0257] It is assumed that the first app is an app that provides a broadcast program, and the second app is an app that provides a video streaming service.

[0258] A first image 711, which is an execution image of the first app, may be displayed on the first split screen 710, and a second image 731, which is an execution image of the second app, may be displayed on the second split screen 730.

[0259] The first user indicator 701 may indicate that the control authority for the first split screen 710 is in the first user account, and the second user indicator 703 may indicate that the control authority for the second split screen 730 is in the first user account. 2 May indicate that you are in a user account.

[0260] First, Figures 8b and 8c will be described.

[0261] Referring to Figure 8b, the first user A utters a voice command indicating a search request for the content name <serach drama 1>.

[0262] The display device 100 may obtain the first content search result for the content <drama 1> through the first app in response to the voice command. In one embodiment, the display device 100 may receive the first content search result from the AI server 200.

[0263] The display device 100 may display the first content search result 810 on the first split screen 710, as shown in Figure 8c. The first content search result 810 may include broadcast content images 811, 813, and 815 searched through the first app.

[0264] The display device 100 may further display a search tab 820. The search tab 820 may further include a TV broadcast item 821, a replay item 823, and an app item 825.

[0265] The TV broadcast item 821 may be an item for providing a broadcast content image currently being broadcast for the content that is the subject of a search request. In Figure 8c, the TV broadcast item 821 may be selected.

[0266] The replay item 823 may be an item for providing a replay of content that is the subject of a search request.

[0267] The app item 825 may be an item for providing content that is the subject of a search request through a video streaming service app.

[0268] The display device 100 may further display a text 801 indicating that a content search result corresponding to the content name of <drama1> is provided through the first app according to a voice command uttered by the first user B on the display 180.

[0269] Next, Figures 8d and 8e will be described.

[0270] Referring to Figure 8d, the second user B utters a voice command indicating a search request for content <serach drama 1>.

[0271] The display device 100 may obtain a second content search result for the content <drama 1> through the second app in response to the voice command.

[0272] The display device 100 may display the second content search result 830 on the second split screen 730, as shown in Figure 8e. The second content search result 830 may include content images 831 and 833 searched through the second app.

[0273] The display device 100 may further display a text 803 indicating that a content search result corresponding to the content name of <drama1> is provided through the second app according to a voice command uttered by the second user B on the display 180.

[0274] As such, according to an embodiment of the present disclosure, in an environment in which a plurality of users watch with one display device 100, each user can individually control the divided screen. Accordingly, an optimal viewing environment can be provided to each user.

[0275] Meanwhile, in the situation of Figures 8b to 8e, when the first user A or the second user B utters a voice command indicating termination of the multi-view function, the display device 100 may terminates the multi-view function and display either the first image 711 or the second image 731 on the entire screen.

[0276] When the first user A utters a voice command indicating termination of the multi-view function, the display device 100 may display the first image 711 on the entire screen.

[0277] When the second user B utters a voice command indicating termination of the multi-view function, the display device 100 may display the second image 731 on the entire screen.

[0278] Figure 9a is a diagram illustrating a process of individually controlling audio output from a split screen through voiceprint recognition according to an embodiment of the present disclosure, and Figure 9b is a diagram illustrating the process of individually controlling an external device linked to a user account through voiceprint recognition according to an embodiment of the present disclosure.

[0279] In Figures 9a and 9b, it is assumed that the control authority of the first split screen 710 belongs to the first user A with the first user account, and the control authority of the second split screen 730 belongs to the second user B with the second user account.

[0280] It is assumed that the first app is an app that provides a broadcast program, and the second app is an app that provides a video streaming service.

[0281] A first image 711, which is an execution image of the first app, may be displayed on the first split screen 710, and a second image 731, which is an execution image of the second app, may be displayed on the second split screen 730.

[0282] The controller 170 of the display device 100 may transmit a first audio of the first image 711 displayed on the first split screen 710 on a first external device 901 of the first user A through the wireless communication interface 173.

[0283] The display device 100 may transmit a second audio for the second image 731 being displayed on the first split screen 730 to the second external device 903 of the second user B.

[0284] The first external device 901 and the second external device 903 may be audio output devices such as headphone that output audio.

[0285] The display device 100 and each external device 901 and 903 may communicate through low-power Bluetooth communication.

[0286] Referring to Figure 9a, the second user B may utter a voice command for volume control called <Increase audio volume>. The display device 100 may perform voiceprint recognition and intent analysis for the voice command based on a voice signal corresponding to the voice command. When the display device 100 determines that the account matching the voice signal is the second user account and the intention analysis of adjusting the audio output is confirmed, the display device 100 may transmit the second audio with the volume increased in response to the voice command to the second external device 903.

[0287] Accordingly, the second user B can listen to the second audio at an increased volume compared to the existing one through the second external device 903.

[0288] Referring to Figure 9b, the second user B may utter a voice command to turn off the headphone, <turn off headphone>. The display device 100 may perform voiceprint recognition and intent analysis for the voice command based on a voice signal corresponding to the voice command. If the display device 100 determines that the account matching the voice signal is the second user account and the intent analysis of turning off the headphone is confirmed, the display device 100 may transmit a power-off command to the second external device 903 in response to the voice command.

[0289] Accordingly, the power of the second external device 903 may be turned off.

[0290] As such, according to an embodiment of the present disclosure, external device linked to each split screen can be individually controlled through voiceprint recognition.

[0291] Figures 10a and 10b are diagrams illustrating a scenario for transferring screen control rights according to an embodiment of the present disclosure.

[0292] In Figures 10a and 10b, it is assumed that the control authority of the first split screen 710 belongs to the first user A with the first user account, and the control authority of the second split screen 730 belongs to the second user B with the second user account.

[0293] Referring to Figure 10a, the first user A utters a voice command <Give me control authority of the second split screen!> to transfer control authority of the second split screen 730 to the first user account.

[0294] The display device 100 may perform voiceprint recognition and intent analysis for the voice command.

[0295] In one embodiment, when the intent analysis indicates transfer of the control authority, the display device 100 may grant the control authority of the second split screen 830 to the first user account, regardless of the voiceprint recognition result.

[0296] In another embodiment, the display device 100 may grant the control authority of the second split screen 830 to the first user account, regardless of the voiceprint recognition result, if the intent analysis indicates transfer of the control authority and the second user B is not detected. The display device 100 may detect a presence of the second user B based on an image captured through a camera (not shown).

[0297] As shown in Figure 10b, the display device 100 may display a first user indicator 701 on one side of the second split screen 730, indicating that the control authority of the second split screen 730 is granted to the first user account. Additionally, the display device 100 may apply the setting value set of the first user account to the second split screen 730.

[0298] Figure 11 is a flowchart illustrating a method of operating a display device according to an embodiment of the present disclosure.

[0299] The controller 170 of the display device 100 may display the first image on the entire screen of the display 180 (S1101).

[0300] The first user account may be logged in to the display device 100.

[0301] The first image may be an image displayed according to execution of the first app executed through the first user account.

[0302] The controller 170 of the display device 100 may receive a voice command for executing the multi-view function (S1103).

[0303] The controller 170 of the display device 100 may obtain an intent analysis result of the voice command from the voice signal (S1105).

[0304] The controller 170 of the display device 100 may convert voice signal into text data through an STT engine and obtain the intent analysis result from the text data through an NLP engine.

[0305] The intent analysis result may be a command to execute a multi-view application or execute a specific app. The specific app may be the same as the first app or may be a different app.

[0306] The controller 170 of the display device 100 may acquire voice feature information (S1107).

[0307] The voice feature information may be either a voice feature vector or an embedder vector representing voice feature.

[0308] The controller 170 may determine whether the speaker who uttered the voice signal is a registered account through a voiceprint recognition process.

[0309] The controller 170 may extract a voice feature vector (or embedding vector) from the speech signal and compare the extracted voice feature vector with the feature vector stored in the memory 140.

[0310] The controller 170 of the display device 100 may determine whether the user account corresponding to the acquired voice feature information is the registered first user account (S1109).

[0311] When the controller 170 of the display device 100 determines that the user account corresponding to the acquired voice feature information is a registered first user account, the controller 170 may assigne the control authority of the first split screen and the second split screen to the first account (S1111).

[0312] If a matching feature vector exists in the memory 140, the controller 170 may obtain speaker identification information corresponding to the matching feature vector.

[0313] The speaker identification account may include information about the user account. The information about the user account may include one or more of the user's ID and name.

[0314] The controller 170 may assign the control authority of the second split screen to the first user account based on speaker identification information. The control authority for the second split screen may be the authority to control the language of the video, audio, or video output from the second split screen.

[0315] The control authority of the second split screen may further include app control authority that can control the app running on the second split screen. The control authority of the app may be the permission to control the functions or actions of the app executed according to the execution of the multi-view function through voice.

[0316] The control authority of the second split screen may include the authority to control an external device linked to the second user account that has the control authority of the second split screen.

[0317] The controller 170 of the display device 100 may display the first image on the first split screen of the display 180 based on the obtained intention analysis result, and display the second image on the second split screen of the display 180 (S1113).

[0318] The first image may be an image displayed on the entire screen in step S1101. The second image may be an execution image of an app executed according to the intention analysis result of the voice command.

[0319] That is, the controller 170 may display the first image, which was previously displayed on the full screen, on the first split screen according to the user's voice command for executing the multi-view function, and display the secondt image, which is the execution image of a specific app on the second split screen.

[0320] As control authority for the second split screen is assigned to the first user account, the controller 170 may apply setting values matching the first user account to the second split screen.

[0321] The memory 140 may store setting value set matching each of a plurality of user accounts. The setting value set may include one or more of the values of an image quality elements of the display 180, a sound quality the speaker 185, a volume, and a language, or information about a previously connected external device (e.g., an audio output device).

[0322] If the first image is played by the first user account that is already logged in, the controller 170 may also assigne the control authority of the first split screen to the first user account.

[0323] The controller 170 may further display a first user indicator on the display 180 indicating that control authority for the first and second split screens belongs to the first user account.

[0324] The controller 170 may display the first user indicator on one side of the second split screen indicating that the first user account has the control authority for the second split screen provided according to execution of the multi-view function.

[0325] Meanwhile, when the first image is executed by the first user account that is already logged in, the controller 170 may also display the first user indicator indicating that the control authority for the first split screen belongs to the first user account on one side of the first split screen.

[0326] If the user account corresponding to the acquired voice feature information is a pre-registered second user account (S1115), the controller 170 of the display device 100 may assign the control authority of the first split screen to the first user account, and assign the control authority of for the second split screen to the second user account (S1117).

[0327] When the control authority for the second split screen is assigned to the second user account, the controller 170 may apply a setting value set matching the second user account on the second split screen.

[0328] If the first image is played by the first user account that is already logged in, the controller 170 may assign the control authority of the first split screen to the first user account. Accordingly, the controller 170 may apply a setting value set matching the first user account on the first split screen.

[0329] The controller 170 of the display device 100 may display the first image on the first split screen of the display 180 and display the second image on the second split screen of the display 180 based on the intention analysis result (S1119).

[0330] The controller 170 may display the first image that was previously displayed on the full screen on the first split screen according to the user's voice command for executing the multi-view function, and display the second image that is the execution image of a specific app on the second split screen.

[0331] The controller 170 of the display device 100 may further display a second user indicator on the display 180 indicating that control authority for the second split screen belongs to the second user account based on the received second speaker identification information.

[0332] The controller 170 may display the second user indicator on one side of the second split screen indicating that the second user account has control authority for the second split screen provided according to execution of the multi-view function.

[0333] When the first image is executed by a first user account that is already logged in, the controller 170 may displays the first user indicator on one side of the first split screen indicating that the control authority for the first split screen is in the first user account.

[0334] If the account corresponding to the speaker identification information is determined as an unregistered account, the controller 170 of the display device 100 may output an uncontrollable notification (S1121).

[0335] Figure 12 is a flowchart illustrating a method of operating a display device according to another embodiment of the present disclosure.

[0336] Figure 12 may be an example performed after step S1119 of Figure 11.

[0337] The controller 170 of the display device 100 may display the first image on the first split screen and the second image on the second split screen (S1201).

[0338] The controller 170 may display the first image on the first split screen and the second image on the second split screen according to the execution of the multi-view function.

[0339] The control authority for the first split screen may reside in the first user account, and control authority for the second split screen may reside in the second user account.

[0340] The controller 170 of the display device 100 may receive a voice command (S1203) and obtain voice feature information based on the voice signal corresponding to the voice command (S1205).

[0341] The controller 170 of the display device 100 may determine whether the account identified through the voice feature information is a pre-registered first user account (S1207), and when the account identified through the voice feature information is the pre-registered first user account, may obtain the first analysis result of the voice command (S1209).

[0342] The controller 170 may use an NLP engine to obtain an intention analysis result matching the text data of the voice signal.

[0343] The controller 170 may obtain an analysis result for the voice command based on the intention analysis result. The intent analysis result of the voice command may be one of a content search request, a request to control video or audio displayed on a split screen, or a request to control an external device linked to the user account.

[0344] If the intention analysis result is the content search request, the controller 170 may obtain the first analysis result including a content search result through the first app that is the source of the first split screen. The controller 170 may transmit the content search request to the server of the first app through the network interface 133 and receive the content search result from the server of the first app in response to the content search request.

[0345] That is, when the controller 170 recognizes that the voice command is uttered through the first user account, it may provide the analysis result of the voice command through the first app that matches the first split screen.

[0346] The controller 170 of the display device 100 may control the first split screen based on the obtained first analysis result (S1211).

[0347] Based on the first analysis result, the controller 170 may control video and audio output from the first split screen, the operation of an app running on the first split screen, or an external device linked to the user account.

[0348] If the first analysis result is the content search result searched through the first app, the controller 170 may display the first analysis result on the first split screen.

[0349] If the first analysis result is a command to adjust the volume of audio being output on the first split screen, the controller 170 may adjust the volume of the audio being output on the first split screen.

[0350] If the account identified through the voice feature information is not determined as the pre-registered first user account (S1207), and the account identified through the voice feature information is determined as a pre-registered second user account (S1213), the controller 170 of the display device 100 obtain a second analysis result of the voice command (S1215).

[0351] The controller 170 may use an NLP engine to obtain the intention analysis result matching the text data of the voice signal.

[0352] The controller 170 may obtain an analysis result for the voice command based on the intention analysis result. The intent analysis result of the voice command may be one of a content search request, a request to control video or audio displayed on a split screen, or a request to control an external device linked to the user account.

[0353] If the intention analysis result is the content search request, the controller 170 may obtain a second analysis result including the content search result through the second app that is the source of the second split screen. The controller 170 may transmit the content search request to the server of the second app through the network interface 133 and receive the content search result from the server of the second app in response to the content search request.

[0354] That is, when the controller 170 recognizes that the voice command is uttered through the second user account, it may provide the analysis result of the voice command through the second app that matches the second split screen.

[0355] If the intention analysis result indicates adjusting the volume of audio being output on the second split screen, the controller 170 may transmit a second analysis result indicating adjusting the volume of the audio to the display device 100.

[0356] The controller 170 of the display device 100 may control the second split screen based on the received second analysis result (S1217).

[0357] If the account is not registered, the controller 170 of the display device 100 may output an uncontrollable notification (S1219).

[0358] The display device 100 according to an embodiment of the present disclosure may also be applied to a display provided in a home appliance or a display provided in a vehicle.

[0359] A display device 100 according to an embodiment of the present disclosure may comprise a display 180 and a controller 170 configured to: display a first image 711 on a first split screen 710 of the display 180, which a first user account has a control authority, display a second image 731 on a second split screen 730 of the display 180, which a second user account has the control authority, receive a voice command, control the first split screen 710 based on a first analysis result of the voice command when a user account matching the received voice command is the first user account, and control the second split screen 730 based on a second analysis result of the voice command when the user account matching the received voice command is the second user account.

[0360] The controller 170 may display the first analysis result 810 including a content search result on the first split screen 710 when the user account matching the received voice command is the first user account, and an intent analysis result of the voice command indicates a content search request, and display the second analysis result 830 including a content search result on the second split screen 730 when the user account matching the received voice command is the second user account and the intent analysis result of the voice command indicates the content search request.

[0361] The first image 711 may be an execution image of a first application, and the second image 731 may be an execution image of a second application, the first analysis result 810 may include a first content search result searched through the first application and the second analysis result 830 may include a second content search result searched through the second application.

[0362] The controller 170 may display, on the display 180, a first user indicator 701 indicating that the control authority of the first split screen 710 is in the first user account and a second user indicator indicating that the control authority of the second split screen 730 is in the second user account.

[0363] The controller 170 may extract a voice feature vector from the voice command, compare the extracted voice feature vector with a stored feature vector, and obtain speaker identification information matching the feature vector when the feature vector matching the voice feature vector extracted according to a comparison result is stored.

[0364] The display device may further comprise a memory 140 configured to store a setting value set corresponding to each user account, the controller 170 may further apply a first setting value set corresponding to the first user account to the first split screen 710, and apply a second setting value set corresponding to the second user account to the second split screen 730.

[0365] The setting value set may include one or more values of picture quality elements, a sound quality, a volume, a language, or information in an external device (e.g. ab audio ouput deivce) connected to the user account.

[0366] The controller 170 may transmit volume-adjusted audio to an external device linked to the second user account when the user account matching the received voice command is the second user account, and the second analysis result indicates audio volume adjustment.

[0367] The controller 170 may trnasmit a command for controlling the external device the external device when the user account matching the received voice command is the second user account, and the second analysis result indicates an operation control of an external device linked to the second user account.

[0368] The controller 170 may assign the control authority of the second split screen 730 to the first user account when a voice command for transfering the control authority of the second split screen 730 to the first user account is received.

[0369] According to an embodiment of the present disclosure, the above-described method may be implemented with codes readable by a processor on a medium in which a program is recorded. Examples of the medium readable by the processor include a ROM (Read Only Memory), a Random Access Memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0370] The above-described display device is not limited to the configuration and method of the above-described embodiments, but the embodiments may be configured by selectively combining all or part of each embodiment such that various modifications can be made.

Claims

1.A display device, comprising:a display; anda controller configured to:display a first image on a first split screen of the display, which a first user account has a control authority,display a second image on a second split screen of the display, which a second user account has the control authority,receive a voice command,control the first split screen based on a first analysis result of the voice command when a user account matching the received voice command is the first user account, andcontrol the second split screen based on a second analysis result of the voice command when the user account matching the received voice command is the second user account.2.The display device of claim 1, wherein the controller is further configured to:display the first analysis result including a content search result on the first split screenwhen the user account matching the received voice command is the first user account, and an intent analysis result of the voice command indicates a content search request, anddisplay the second analysis result including a content search result on the second split screen when the user account matching the received voice command is the second user account and the intent analysis result of the voice command indicates the content search request.3.The display device of claim 2, wherein the first image is an execution image of a first application, and the second image is an execution image of a second application, the first analysis result includes a first content search result searched through the first application and the second analysis result includes a second content search result searched through the second application.4.The display device of claim 1, wherein the controller is further configured to:display, on the display, a first user indicator indicating that the control authority of the first split screen is in the first user account and a second user indicator indicating that the control authority of the second split screen is in the second user account.5.The display device of claim 1, wherein the controller is further configured to:extract a voice feature vector from the voice command,compare the extracted voice feature vector with a stored feature vector, andobtain speaker identification information matching the feature vector when the feature vector matching the voice feature vector extracted according to a comparison result is stored.6.The display device of claim 1, further comprising a memory configured to store a setting value set corresponding to each user account,wherein the controller is further configured to apply a first setting value set corresponding to the first user account to the first split screen, and apply a second setting value set corresponding to the second user account to the second split screen.7.The display device of claim 6, wherein the setting value set includes one or more values of picture quality elements, a sound quality, a volume, a language, or information in an external device connected to the user account.8.The display device of claim 1, wherein the controller is further configured to:transmit volume-adjusted audio to an external device linked to the second user account when the user account matching the received voice command is the second user account, and the second analysis result indicates audio volume adjustment.9.The display device of claim 1, wherein the controller is further configured to:trnasmit a command for controlling the external device the external device when the user account matching the received voice command is the second user account, and the second analysis result indicates an operation control of an external device linked to the second user account.10.The display device of claim 1, wherein the controller is further configured to:assign the control authority of the second split screen to the first user account when a voice command for transfering the control authority of the second split screen to the first user account is received.11.A method of operating a display device,displaying a first image on a first split screen of the display, which a first user account has a control authority;displaying a second image on a second split screen of the display, which a second user account has the control authority;receiving a voice command;controlling the first split screen based on a first analysis result of the voice command when a user account matching the received voice command is the first user account; andcontrolling the second split screen based on a second analysis result of the voice command when the user account matching the received voice command is the second user account.12.The method of claim 11, wherein the step of controlling the first split screen comprises:displaying the first analysis result including a content search result on the first split screen when the user account matching the received voice command is the first user account, and an intent analysis result of the voice command indicates a content search request, andwherein the step of controlling the second split screen comprises:displaying the second analysis result including a content search result on the second split screen when the user account matching the received voice command is the second user account and the intent analysis result of the voice command indicates the content search request.13.The method of claim 12, wherein the first image is an execution image of a first application, and the second image is an execution image of a second application, the first analysis result includes a first content search result searched through the first application and the second analysis result includes a second content search result searched through the second application.14.The method of claim 11, further comprising:displaying a first user indicator indicating that the control authority of the first split screen is in the first user account and a second user indicator indicating that the control authority of the second split screen is in the second user account.15.The method of claim 11, further comprising:extracting a voice feature vector from the voice command,comparing the extracted voice feature vector with a stored feature vector, andobtaining speaker identification information matching the feature vector when the feature vector matching the voice feature vector extracted according to a comparison result is stored.16.The method of claim 11, further comprising:storing a setting value set corresponding to each user account,wherein the step of controlling the first split screen comprises:applying a first setting value set corresponding to the first user account to the first split screen, andthe step of controlling the second split screen comprises:applying a second setting value set corresponding to the second user account to the second split screen.17.The method of claim 16, wherein the setting value set includes one or more values of picture quality elements, a sound quality, a volume, a language, or information in an external device connected to the user account.18.The method of claim 11, wherein the step of controlling the second split screen comprises:transmitting volume-adjusted audio to an external device linked to the second user account when the user account matching the received voice command is the second user account, and the second analysis result indicates audio volume adjustment.19.The method of claim 11, wherein the step of controlling the second split screen comprises:trnasmitting a command for controlling the external device the external device when the user account matching the received voice command is the second user account, and the second analysis result indicates an operation control of an external device linked to the second user account.20.The method of claim 11, further comprising:assigning the control authority of the second split screen to the first user account when a voice command for transfering the control authority of the second split screen to the first user account is received.

Images