Display device and method for operating the same

Abstract

A display device according to an embodiment includes a communication unit, a display, a memory storing one or more instructions, and at least one processor executing the one or more instructions, where the at least one processor, by executing the one or more instructions, may: acquire a first user image of a first user, and a control unit that causes a communication unit to transmit capability information of the display device to a server, receive information on a configuration method of a group service determined based on the capability information from the server, acquire a first virtual image and a second virtual image corresponding to a user of another display device based on the received information, and display the acquired first virtual image and the acquired second virtual image on the basis of the acquired second virtual image. And causing a display to display a group image including the first virtual image and the second virtual image.

Description

Technical Field

[0001] Various embodiments relate to a display device capable of displaying a user image as an avatar image in a non-face-to-face group service, and an operating method of the display device. Background Art

[0002] Recently, in line with the era of non-face-to-face contact, non-face-to-face services have increased, and non-face-to-face group services that allow multiple users to participate together remotely have increased. In particular, due to the increase in interest in health, home fitness services that guide people to exercise in personal spaces have been actively developed.

[0003] Existing home fitness services involve a method of watching and following various guided exercise videos from a trainer or a method of group fitness by using a video conferencing system.

[0004] In the case of group fitness by using a video conferencing system, the image of the user captured by the camera may be shared with other users, which may cause problems in terms of personal privacy protection. In particular, when exercising with strangers, users may not want to share their appearance (e.g., their face or exercise clothes) with other users.

[0005] Therefore, a method is needed to provide non-face-to-face group services in which multiple users can participate together while protecting the privacy of individual users. Summary of the invention

[0006] Solution to the problem

[0007] The display device according to the embodiment may include a communication unit, a display, a memory storing one or more instructions, and at least one processor executing the one or more instructions.

[0008] The at least one processor may obtain a captured first user image of the first user by executing the one or more instructions.

[0009] The at least one processor may control the communication unit to transmit the capability information about the display device to a server by executing the one or more instructions.

[0010] The at least one processor may receive, from the server, information about a configuration method of a group service determined based on the capability information by executing the one or more instructions.

[0011] The at least one processor may obtain a first virtual avatar image and a second virtual avatar image corresponding to a user of another display device based on the received information by executing the one or more instructions.

[0012] The at least one processor may control the display to display a group image including the first virtual avatar image and the second virtual avatar image by executing one or more instructions.

[0013] The operating method of the display device according to the embodiment may include obtaining a captured first user image of a first user.

[0014] The operating method of the display device according to the embodiment may include transmitting capability information about the display device to a server.

[0015] The operating method of the display device according to the embodiment may include receiving, from the server, information about a configuration method of the group service determined based on the capability information.

[0016] The operating method of the display device according to the embodiment may include obtaining a first avatar image and a second avatar image corresponding to a user of another display device based on the received information.

[0017] The operating method of the display device according to the embodiment may include displaying a group image including the first avatar image and the second avatar image. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a diagram showing a group service providing system according to an embodiment.

[0019] Figure 2 is a flowchart illustrating a method in which a server determines a configuration method of a group service according to an embodiment.

[0020] Figure 3 is a flowchart illustrating a method in which a display device determines a service configuration method according to an embodiment.

[0021] Figure 4 is a flowchart illustrating an operation method of a plurality of display devices when a group service configuration method is based on device determination according to an embodiment.

[0022] Figure 5 is a diagram illustrating operations of a plurality of display devices when a configuration of a group service is device-based according to an embodiment.

[0023] Figure 6 is a diagram illustrating an example of a user interface screen for setting avatar information according to an embodiment.

[0024] Figure 7 is a flowchart illustrating an operation method of a plurality of display devices when a group service configuration method is based on device determination according to an embodiment.

[0025] Figure 8 is a diagram illustrating operations of a plurality of display devices when a configuration of a group service is device-based according to an embodiment.

[0026] Fig. 9 is a flowchart illustrating an operation method of a plurality of display devices and a server when a service configuration method is determined based on a server according to an embodiment.

[0027] Fig.10 is a flowchart illustrating an operation method of a plurality of display devices and a server when a group service configuration method is based on server determination according to an embodiment.

[0028] Fig.11 is a block diagram showing a configuration of a display device according to an embodiment.

[0029] Fig.12 is a block diagram showing a configuration of a display device according to an embodiment.

[0030] Fig.13 is a block diagram showing a configuration of a server according to an embodiment. DETAILED DESCRIPTION

[0031] The terms used in the specification are briefly explained, and the present disclosure is described in detail.

[0032] The terms used in the present disclosure are selected from the most widely used general possible terms while considering the functions of the present disclosure, but the terms may vary depending on the intention of those skilled in the art or precedents, the emergence of new technologies, etc. In addition, in some cases, there are terms arbitrarily selected by the applicant, and in such cases, the meanings of the terms are described in detail in the description of the relevant disclosure. Therefore, the terms used in the present disclosure should be defined based on the meanings of the terms and the overall content of the present disclosure, rather than simply defined based on the names of the terms.

[0033] Throughout the specification, when a part "includes" an element, unless otherwise specified, another element may be further included, rather than excluding the presence of another element. In addition, terms such as "unit", "module", etc. described in the specification may represent a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software.

[0034] Hereinafter, the embodiments are described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure can be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the accompanying drawings, parts irrelevant to the explanation are omitted to clearly explain the present disclosure, and the same reference numerals in the accompanying drawings represent the same elements.

[0035] In the embodiments of the present specification, the term "user" refers to a person who controls a system, function, or operation, and may include a developer, an administrator, or an installer.

[0036] In addition, in the embodiments of the present specification, 'image' or 'picture' may refer to a still image, a moving image composed of a plurality of consecutive still images (or frames), or a video.

[0037] Figure 1 is a diagram showing a group service providing system according to an embodiment.

[0038] Reference Figure 1 , the group service providing system according to the embodiment may include a server 200 and a plurality of display devices 101, 102, and 103. The plurality of display devices 101, 102, and 103 according to the embodiment may be devices connected to the group service. The group service according to the embodiment may be a service that provides a group activity in which a plurality of users can participate together in a non-face-to-face method.

[0039] The plurality of display devices 101, 102, and 103 connected to the group service may share user images with each other. Each of the plurality of display devices 101, 102, and 103 may display one group image including user images of the plurality of display devices 101, 102, and 103.

[0040] Each of the plurality of display devices 101 , 102 , and 103 according to the embodiment may be a device for displaying an image by processing an image signal, and may be a device including a display.

[0041] Each of the plurality of display devices 101, 102, and 103 according to the embodiment may be implemented in various forms, such as a television (TV) including a display, a smart monitor, a mobile phone, a smart phone, a tablet personal computer (PC), a digital camera, a video camera, a laptop computer, a desktop computer, an e-book terminal, a digital broadcast terminal, a personal digital assistant (PDA), a navigation device, an MP3 player, a digital video disk (DVD) player, a wearable device, a video wall, a digital signage, a digital information display (DID), a projector display, a refrigerator, a washing machine, etc. In addition, each of the plurality of display devices 101, 102, and 103 may be a fixed electronic device arranged at a fixed position or a mobile electronic device having a portable form, and may be a digital broadcast receiver capable of receiving digital broadcasting. However, the present disclosure is not limited thereto.

[0042] like Figure 1As shown, when the group service is a server providing group fitness, the plurality of display devices 101, 102, and 103 can capture exercise images of individual users and provide the exercise images of the plurality of users as a single group image. Therefore, the users of the plurality of display devices can perform home fitness in a non-face-to-face method while sharing their own exercise images and exercise images of other users.

[0043] At this time, the moving image of the user may not include an image capturing the actual user, but may include an avatar image in which an avatar generated based on the captured image of the actual user moves.

[0044] In a case where exercise images of a plurality of users included in a group fitness screen are displayed as captured images of actual users, some users may not want to show their faces, exercise clothes, etc. to other users. Therefore, the privacy of individuals can be protected by displaying the exercise images of users included in the group fitness screen as avatar images.

[0045] The avatar image according to an embodiment may be an image that moves like an actual user based on a captured image of the user. The server 200 or the plurality of display devices 101, 102, and 103 according to an embodiment may detect an object (e.g., a user) in the captured image of the user, obtain tracking information of the detected object, and generate the avatar image based on the tracking information.

[0046] Meanwhile, the server 200 according to the embodiment may determine whether the configuration of the group service is server-based or device-based based on the capability information of the plurality of display devices 101, 102, and 103. For example, the server 200 may determine whether the operation of generating the avatar image is to be performed by the server 200 (based on the service configuration of the server) or by each of the plurality of display devices 101, 102, and 103 (based on the service configuration of the device).

[0047] When determining to configure the service based on the server, each of the plurality of display devices 101 , 102 , and 103 may transmit a captured image of a user (user image) or information on the user image to the server 200 .

[0048] The server 200 according to an embodiment may generate an avatar image corresponding to a user image based on a user image or information about a user image received from each of the plurality of display devices 101, 102, and 103. The server 200 may generate a single group image including avatar images corresponding to a user or a plurality of devices, and transmit the generated group image to the plurality of display devices 101, 102, and 103.

[0049] The plurality of display devices 101 , 102 , and 103 may display a single group image including avatar images corresponding to a plurality of users received from the server 200 .

[0050] Alternatively, when it is determined to configure the service based on the device, each of the multiple display devices 101, 102, 103 may also generate a virtual avatar image corresponding to the user image based on the user image from each of the multiple display devices 101, 102, 103, not via the server 200. In addition, each of the multiple display devices 101, 102, 103 may also send the generated virtual avatar image to other display devices. Each of the multiple display devices 101, 102 and 103 may generate and display a single group image including the virtual avatar image generated by the display device and the virtual avatar images received from other display devices. However, the present disclosure is not limited to this.

[0051] A method for a display device to obtain a single group image including an avatar image of a user of the display device and an avatar image of a user of another display device according to an embodiment is described in detail below with reference to the accompanying drawings.

[0052] Figure 2 is a flowchart illustrating a method in which a server determines a configuration method of a group service according to an embodiment.

[0053] according to Figure 2 The display device 100 of the embodiment may be Figure 1 Any one of the multiple display devices 101, 102 and 103.

[0054] refer to Figure 2 According to the embodiment, the display device 100 may obtain device capability information (S210). For example, the device capability information may include at least one of the following: type information, capacity information, information about the number of cores, and performance information for at least one processor included in the display device 100; and type information, capacity information, and performance information for a memory included in the display device 100.

[0055] The display device 100 may pre-store the device capability information. When receiving a request for the device capability information from the server 200, the display device 100 may transmit the pre-stored device capability information to the server 200 (S220).

[0056] The server 200 may receive device capability information from the display device 100 , and determine a configuration method of the group service based on the received device capability information ( S230 ).

[0057] For example, the server 200 may store policy information (service determination policy information) for determining a service configuration method. The service determination policy information may include standards of device specifications required to perform operations included in the group service (e.g., generating a virtual image on a device, etc.) (e.g., type information, capacity information, information about the number of cores, and performance information for a processor; and type information, capacity information, and performance information for a memory). However, the present disclosure is not limited thereto.

[0058] Furthermore, the service determination policy information may vary depending on the number of concurrent users connected to the server, network conditions, etc. Furthermore, the service determination policy information may also vary as the functions provided by the group service change. However, the present disclosure is not limited thereto.

[0059] The server 200 may determine whether configuration of the group service is based on a device or a server based on the device capability information and the service determination policy information.

[0060] In addition, the server 200 may determine the configuration method of the group service by considering the capability information of other display devices connected to the group service. For example, when the device specifications of some display devices connected to the group service meet the standards required for performing operations included in the group service, such as generating a virtual image, etc., but the device specifications of the remaining display devices do not meet the standards, the server 200 may determine the configuration method of the group service based on the server to be suitable for display devices with low specifications. However, the present disclosure is not limited thereto.

[0061] The server 200 may transmit information about the determined service configuration method to the display apparatus 100 ( S240 ).

[0062] The display apparatus 100 and the server 200 according to the embodiment may perform operations according to the determined service configuration method ( S250 , S260 ).

[0063] For example, when the configuration of the group service is device-based, the display device 100 according to the embodiment may generate a first virtual avatar image for a captured user image of a user, and transmit the generated first virtual avatar image to another display device. In addition, the display device 100 may receive a second virtual avatar image for a user of another display device from another display device. The display device 100 may generate and display a group image including the first virtual avatar image and the second virtual avatar image.

[0064] On the contrary, when the configuration of the group service is based on a server, the display device 100 according to the embodiment may send a captured user image of a user or information about the user image to the server 200, and the server 200 may generate a first virtual image based on the user image or information about the user image. In addition, the server 200 may also generate a second virtual image by receiving a user image of a user of another display device or information about the user image from other display devices. The server 200 may generate a group image including the first virtual image image and the second virtual image image, and send the generated group image to the display device. The display device may display the group image received from the server 200.

[0065] Figure 3 is a flowchart illustrating a method in which a display device determines a service configuration method according to an embodiment.

[0066] according to Figure 3 The display device 100 of the embodiment may be Figure 1 Any one of the multiple display devices 101, 102 and 103.

[0067] refer to Figure 3 According to the embodiment, the display device 100 may obtain device capability information (S310). For example, the device capability information may include at least one of the following: type information, capacity information, information about the number of cores, and performance information for at least one processor included in the display device 100; and type information, capacity information, and performance information for a memory included in the display device 100.

[0068] The display device 100 may pre-store device capability information.

[0069] The display apparatus 100 according to the embodiment may receive service determination policy information from the server 200 ( S320 ).

[0070] For example, the server 200 may store service determination policy information for determining a service configuration method. The service determination policy information may include standards for device specifications required to perform operations included in the group service (such as generating a virtual image on a device, etc.) (e.g., type information, capacity information, information about the number of cores, and performance information for a processor; and type information, capacity information, and performance information for a memory). However, the present disclosure is not limited thereto.

[0071] In addition, the service determination policy information may vary depending on the number of concurrent users connected to the server, network conditions, etc. In addition, the service determination policy information may also vary as the functions provided by the group service change. However, the present disclosure is not limited thereto.

[0072] When receiving a request for the service determination policy information from the display device 100, the server 200 may transmit the service determination policy information to the display device 100. However, the present disclosure is not limited thereto.

[0073] The display device 100 according to the embodiment may determine a configuration method of a service based on the device capability information and the service determination policy information ( S330 ).

[0074] For example, the display device 100 may determine whether the configuration of the group service is device-based or server-based, and may transmit information about the determined service configuration method to the server 200 ( S340 ).

[0075] The display apparatus 100 and the server 200 according to the embodiment may perform operations according to the determined service configuration method ( S350 , S360 ).

[0076] For example, when the configuration of the group service is device-based, the display device 100 may generate a first virtual avatar image for a captured user image of a user, and send the generated first virtual avatar image to another display device. In addition, the display device 10 may also receive a second virtual avatar image for a user of another display device from the other display device. The display device 100 may generate and display a group image including the first virtual avatar image and the second virtual avatar image.

[0077] On the contrary, when the configuration of the group service is based on a server, the display device 100 according to the embodiment may send a captured user image of a user or information about the user image to the server 200, and the server 200 may generate a first virtual image based on the user image or information about the user image. In addition, the server 200 may also generate a second virtual image by receiving a user image of a user of another display device or information about the user image from other display devices. The server 200 may generate a group image including the first virtual image image and the second virtual image image, and send the generated group image to the display device. The display device may display the group image received from the server 200.

[0078] Figure 4 is a flowchart illustrating an operation method of a plurality of display devices when a group service configuration method is determined based on a device according to an embodiment.

[0079] The plurality of display devices according to an embodiment may include a first display device 101 and a second display device 102. The plurality of display devices may be devices connected to a group service.

[0080] refer to Figure 4According to an embodiment, the first display device 101 may obtain a captured first user image of a user (first user) of the first display device by using a camera (S410). At this time, the camera may be a camera installed on the first display device 101 or a camera connected to the first display device 101 via wired or wireless communication.

[0081] The first display device 101 according to the embodiment may detect an object (eg, a first user) from the first user image and obtain information about the first user (first tracking information) ( S420 ).

[0082] For example, the first display device 101 can detect the first user by analyzing the first user image using an object detection module. The object detection module may include suitable logic, circuits, interfaces and / or codes that enable detection of a specific object included in an image captured by a camera. The object detection module can detect the first user by analyzing the first user image using at least one of a machine learning, a neural network, or a deep learning algorithm as a rule-based or artificial intelligence algorithm.

[0083] In addition, the first display device 101 can obtain the first tracking information by using a tracking module. The tracking module may include suitable logic, circuits, interfaces and / or codes capable of tracking the movement of a specific object included in an image captured by a camera. The tracking module may obtain the movement information of the first user from the first user image by using at least one of a machine learning, a neural network or a deep learning algorithm as a rule-based or artificial intelligence algorithm.

[0084] For example, the tracking module may estimate a posture of the first user, identify a specific body part (eg, face, hand, etc.) of the first user based on the estimated posture, and detect motion with respect to the identified specific body part in more detail.

[0085] However, the above-mentioned object detection module and tracking module are merely examples, and the first display device 101 according to an embodiment may detect the first user from the first user image and obtain the first tracking information through various methods.

[0086] The first display device 101 according to an embodiment may generate a first avatar image corresponding to the first user image ( S430 ).

[0087] The first display device 101 may generate a first avatar image based on the avatar information set by the user and the first tracking information obtained in S420. At this time, the first avatar image may be an image moving corresponding to the first user in the first user image.

[0088] The first display device 101 according to the embodiment may transmit the first avatar image to the second display device 102 ( S440 ).

[0089] In addition, the second display device 102 according to the embodiment can obtain a captured second user image of the user (second user) of the second display device by using a camera (S415). At this time, the camera can be a camera installed on the second display device 102 or a camera connected to the second display device 102 via wired or wireless communication.

[0090] In addition, the second display device 102 may detect an object (eg, a second user) from the second user image and obtain tracking information (second tracking information) about the second user (S425).

[0091] For example, the second display device 102 may detect the second user by analyzing the second user image using an object detection module. In addition, the second display device 102 may obtain second tracking information by using a tracking module.

[0092] The description of the object detection module and the tracking module is given in detail in S420, so its detailed description is omitted. However, the object detection module and the tracking module are only examples, and the second display device 102 according to the embodiment can detect the second user from the second user image and obtain the second tracking information by various methods.

[0093] The second display device 102 according to the embodiment may generate a second avatar image corresponding to the second user image ( S435 ).

[0094] The second display device 102 may generate a second avatar image based on the avatar information set by the second user and the second tracking information obtained in S425. At this time, the second avatar image may be an image moving corresponding to the second user in the second user image.

[0095] The second display device 102 according to the embodiment may transmit the second avatar image to the first display device 101 ( S445 ).

[0096] The first display device 101 according to the embodiment may generate and display a group image including the generated first avatar image and the second avatar image received from the second display device 102 ( S450 ).

[0097] For example, the first display device 101 may divide a single screen into a plurality of regions including a first region and a second region, display a first virtual avatar image in the first region, and display a second virtual avatar image in the second region. At this time, the size of the first region may be larger than the sizes of other regions. In addition, the first region may be located at the center of the screen. Alternatively, the first display device 101 may generate a group image by combining the first virtual avatar image and the second virtual avatar image onto a background screen set by the first user. However, the present disclosure is not limited thereto.

[0098] In addition, the second display device 102 according to the embodiment may generate and display a group image including the generated second avatar image and the first avatar image received from the first display device 101 ( S460 ).

[0099] For example, the second display device 102 may divide a single screen into a plurality of regions including a first region and a second region, display the second virtual avatar image in the first region, and display the first virtual avatar image in the second region. At this time, the size of the first region may be larger than the sizes of the other regions. In addition, the first region may be located at the center of the screen. Alternatively, the second display device 102 may generate a group image by combining the first virtual avatar image and the second virtual avatar image onto a background screen set by the second user. However, the present disclosure is not limited thereto.

[0100] Figure 5 is a diagram illustrating operations of a plurality of display devices when a configuration of a group service is device-based according to an embodiment.

[0101] refer to Figure 5 , the plurality of display devices according to the embodiment may include a first display device 101, a second display device 102, and a third display device 103. However, the present disclosure is not limited thereto.

[0102] The first display device 101, the second display device 102, and the third display device 103 may be devices connected to a group service provided by the server 200. The first display device 101, the second display device 102, and the third display device 103 may be connected as a group based on respective pieces of identification information to be able to share user images with each other.

[0103] For example, the first to third display devices 101, 102, and 103 may each transmit user information to the server 200, and the server 200 may perform authentication on the first to third display devices 101, 102, and 103 based on the user information of each of the first to third display devices 101, 102, and 103. Therefore, the first to third display devices 101, 102, and 103 may log in to the group fitness service.

[0104] In addition, the server 200 according to the embodiment may determine that the operation of generating the avatar image is performed by each of the first to third display devices 101, 102, and 103 based on the capability information of each of the first to third display devices 101, 102, and 103. Figure 2 This has been described in detail, so its detailed description is omitted.

[0105] Each of the first to third display devices 101, 102, and 103 may obtain a captured image of a user. For example, the first display device 101 may obtain a captured first user image 511 of a first user. The first display device 101 may detect the first user from the first user image 511, and obtain tracking information of the first user to generate a first avatar image 521.

[0106] The first avatar image 521 may be generated as an image moving identically to the first user of the first user image 511 based on avatar information set by the user. Figure 6 A method for setting virtual image information is described in detail.

[0107] Figure 6 is a diagram illustrating an example of a user interface screen for setting avatar information according to an embodiment.

[0108] refer to Figure 6 , the first display device 101 may display a user interface screen (avatar setting screen) on which avatar information may be set.

[0109] like Figure 6 As shown, the first display device 101 may include a menu screen 610, which may set a basic role for the avatar. For example, the first display device 101 may display a list 620 including a plurality of roles having different genders, races, ages, and the like.

[0110] When the first display device 101 receives an input for selecting any one of a plurality of characters, the first display device 101 may display the selected character 630 and display a menu screen 615 which may set items related to the selected character.

[0111] The first display device 101 may display a list 640 including items that may decorate the selected character.

[0112] When the first display device 101 receives an input for selecting at least one of the plurality of items, the first display device 101 may display a selected character 630 to which the selected at least one item is applied.

[0113] The first display device 101 may save the final character as the first avatar information about the first user based on a user input selecting a save button 650 included in the menu screen 615 .

[0114] However, Figure 6 The illustrated user interface screen for setting the avatar information is merely an example, and the first display device 101 according to an embodiment may set the first avatar information by various methods.

[0115] Reference again Figure 5 , the second display device 102 may obtain a captured second user image 512 of the second user. The second display device 102 may detect the second user from the second user image 512 and obtain tracking information of the second user. The second display device 102 may generate a second avatar image 522 based on the obtained tracking information of the second user. The second avatar image 522 may be generated based on avatar information set by the user, and may be an image moving in the same manner as the second user detected from the second user image 512.

[0116] In addition, the third display device 103 may obtain a captured third user image 513 of the third user. The third display device 103 may detect the third user from the third user image 513 and obtain tracking information of the third user. The third display device 103 may generate a third avatar image 523 based on the obtained tracking information of the third user. The third avatar image 523 may be generated based on avatar information set by the user, and may be an image moving in the same manner as the third user detected from the third user image 513.

[0117] Each of the first to third display devices 101 , 102 , and 103 may transmit the generated avatar image to other display devices.

[0118] Each of the first to third display devices 101 , 102 , and 103 may request the server 200 for identification information about other display devices connected to the group service, and may receive identification information about the other display devices from the server 200 .

[0119] Each of the first to third display devices 101 , 102 , and 103 may transmit the generated avatar image to the other display devices based on the received identification information about the other display devices.

[0120] For example, the first display device 101 may transmit the first avatar image 521 to the second display device 102 and the third display device 103 , receive the second avatar image 522 from the second display device 102 , and receive the third avatar image 523 from the third display device 103 .

[0121] In addition, the second display device 102 may transmit the second avatar image 522 to the first display device 101 and the third display device 103 , receive the first avatar image 521 from the first display device 101 , and receive the third avatar image 523 from the third display device 103 .

[0122] In addition, the third display device 103 may transmit the third avatar image 523 to the first display device 101 and the second display device 102 , receive the first avatar image 521 from the first display device 101 , and receive the second avatar image 522 from the second display device 102 .

[0123] The first display device 101 may generate and display a first group image 531 including a first virtual avatar image 521, a second virtual avatar image 522, and a third virtual avatar image 523. The first group image 531 may be divided into three regions, such that the second virtual avatar image 522 may be displayed in the first region, the first virtual avatar image 521 may be displayed in the second region, and the third virtual avatar image 523 may be displayed in the third region. At this time, the size and position of the region displaying the first to third virtual avatar images 521, 522, and 523 may be set based on a user input.

[0124] In addition, the second display device 102 may generate and display a second group image 532 including a first virtual avatar image 521, a second virtual avatar image 522, and a third virtual avatar image 523. The second group image 532 may be divided into three regions, such that the first virtual avatar image 521 may be displayed in the first region, the second virtual avatar image 522 may be displayed in the second region, and the third virtual avatar image 523 may be displayed in the third region. At this time, the size and position of the region displaying the first to third virtual avatar images 521, 522, and 523 may be set based on user input.

[0125] In addition, the third display device 103 may generate a third group image 533 including the first virtual avatar image 521, the second virtual avatar image 522, and the third virtual avatar image 523. The third group image 533 may be divided into three areas, so that the first virtual avatar image 521 may be displayed in the first area, the third virtual avatar image 523 may be displayed in the second area, and the second virtual avatar image 522 may be displayed in the third area. At this time, the size and position of the area displaying the first to third virtual avatar images 521, 522, and 523 may be set based on user input.

[0126] Figure 7is a flowchart illustrating an operation method of a plurality of display devices when a group service configuration method is determined based on a device according to an embodiment.

[0127] The plurality of display devices according to an embodiment may include a first display device 101 and a second display device 102. The plurality of display devices may be devices connected to a group service.

[0128] refer to Figure 7 , the first display device 101 may obtain a captured first user image of the first user by using a camera (S710). At this time, the camera may be a camera installed on the first display device 101 or a camera connected to the first display device 101 via wired or wireless communication.

[0129] In addition, the first display device 101 may detect the first user from the first user image, and obtain tracking information (first tracking information) about the first user ( S720 ).

[0130] Because it has been referenced Figure 4 S420 describes in detail the method of obtaining tracking information about the first user, so the same description is omitted.

[0131] The first display device 101 may transmit the first tracking information to the second display device 102 ( S730 ).

[0132] In addition, the second display device 102 according to the embodiment can obtain a captured second user image of the user of the second display device (second user) by using a camera (S715). At this time, the camera can be a camera installed on the second display device 102 or a camera connected to the second display device 102 via wired or wireless communication.

[0133] In addition, the second display device 102 may detect the second user from the second user image, and obtain tracking information (second tracking information) about the second user (S725).

[0134] Because it has been referenced Figure 4 S425 describes in detail the method of obtaining tracking information about the user, so the same description is omitted.

[0135] The second display device 102 may transmit the second tracking information to the first display device 101 ( S735 ).

[0136] The first display apparatus 101 according to an embodiment may generate a first avatar image based on the first tracking information, and generate a second avatar image based on the second tracking information ( S740 ).

[0137] For example, the first display device 101 may generate a first virtual avatar image corresponding to the first user image based on the first virtual avatar information and the first tracking information set by the first user. In addition, the first display device 101 may obtain second virtual avatar information, and generate a second virtual avatar image corresponding to the second user image based on the second virtual avatar information and the second tracking information. At this time, the second virtual avatar information may be obtained by receiving the virtual avatar information set by the second user or may be obtained by the setting of the first user. However, the present disclosure is not limited thereto.

[0138] The first display device 101 may generate and display a first group image including a first avatar image and a second avatar image ( S750 ).

[0139] For example, the first display device 101 may generate the first group image by combining the first avatar image and the second avatar image onto a virtual background screen. However, the present disclosure is not limited thereto.

[0140] In addition, the second display device 102 according to the embodiment may generate a first avatar image based on the first tracking information, and generate a second avatar image based on the second tracking information (S745).

[0141] For example, the second display device 102 may obtain the first virtual avatar information and generate a first virtual avatar image corresponding to the first user image based on the first virtual avatar information and the first tracking information. At this time, the first virtual avatar information may be obtained by receiving the virtual avatar information set by the first user or may be obtained by the setting of the second user. However, the present disclosure is not limited thereto.

[0142] In addition, the second display device 102 may generate a second avatar image corresponding to the second user image based on the second avatar information and the second tracking information set by the second user.

[0143] The second display device 102 may generate and display a second group image including the first avatar image and the second avatar image ( S755 ).

[0144] For example, the second display device 102 may generate the second group image by combining the first avatar image and the second avatar image onto a virtual background screen. However, the present disclosure is not limited thereto.

[0145] Figure 8 is a diagram illustrating operations of a plurality of display devices when a configuration of a group service is device-based according to an embodiment.

[0146] refer to Figure 8, the plurality of display devices according to the embodiment may include a first display device 101, a second display device 102, and a third display device 103. However, the present disclosure is not limited thereto.

[0147] The first display device 101, the second display device 102, and the third display device 103 may be devices connected to a group service provided by the server 200. The first display device 101, the second display device 102, and the third display device 103 may be connected as a group based on respective pieces of identification information to be able to share user images with each other.

[0148] For example, the first to third display devices 101, 102, and 103 may each transmit user information to the server 200, and the server 200 may perform authentication on the first to third display devices 101, 102, and 103 based on the user information of each of the first to third display devices 101, 102, and 103. Therefore, the first to third display devices 101, 102, and 103 may log in to the group fitness service.

[0149] In addition, the server 200 according to the embodiment may determine that the operation of generating the avatar image is performed by each of the first to third display devices 101, 102, and 103 based on the capability information of each of the first to third display devices 101, 102, and 103. Figure 2 This has been described in detail, so its detailed description is omitted.

[0150] Each of the first to third display devices 101, 102, and 103 may obtain a captured image of a user. For example, the first display device 101 may obtain a captured first user image 811 of a first user. The first display device 101 may detect the first user from the first user image 811, and obtain tracking information about the first user (first tracking information 821). In addition, the second display device 102 may obtain a second user image 812. The second display device 102 may detect the second user from the second user image 812, and obtain tracking information about the second user (second tracking information 822). In addition, the third display device 103 may obtain a third user image 813. The third display device 103 may detect the third user from the third user image 813, and obtain tracking information about the third user (third tracking information 823).

[0151] In addition, although not shown in the figure, in the case where the device specification of the second display device 102 does not satisfy the standard for obtaining the second tracking information from the second user image 812, the second display device 102 may transmit the second user image 812 to an external electronic device linked to the second display device 102. The external electronic device may be a personal electronic device of the second user, and the device specification of the external electronic device may satisfy the standard for obtaining the second tracking information from the second user image 812. The external electronic device may detect the second user from the second user image, and obtain the second tracking information 822 about the second user. The external electronic device may transmit the second tracking information 822 to the second display device 102.

[0152] Each of the first to third display devices 101 , 102 , and 103 may transmit identification information to other display devices.

[0153] Each of the first to third display devices 101, 102, and 103 may request identification information about other display devices connected to the group service from the server 200, and may receive identification information about other display devices from the server 200. Each of the first to third display devices 101, 102, and 103 may transmit identification information to other display devices based on the received identification information about other display devices.

[0154] For example, the first display device 101 may transmit first tracking information 821 to the second display device 102 and the third display device 103 , receive second tracking information 822 from the second display device 102 , and receive third tracking information 823 from the third display device 103 .

[0155] In addition, the second display device 102 may transmit the second tracking information 822 to the first display device 101 and the third display device 103 , receive the first tracking information 821 from the first display device 101 , and receive the third tracking information 823 from the third display device 103 .

[0156] In addition, the third display device 103 may transmit third tracking information 823 to the first display device 101 and the second display device 102 , receive first tracking information 821 from the first display device 101 , and receive second tracking information 822 from the second display device 102 .

[0157] The first display device 101 may generate a first virtual avatar image based on the first tracking information 821 and the first virtual avatar information, generate a second virtual avatar image based on the second tracking information 822 and the second virtual avatar information, and generate a third virtual avatar image based on the third tracking information 823 and the third virtual avatar information. At this time, the first virtual avatar information may be set by the first user, and the second virtual avatar information and the third virtual avatar information may be set by the first user or received from the second display device 102 and the third display device 103, respectively. However, the present disclosure is not limited thereto.

[0158] The first display device 101 may generate and display the first group image 831 by combining the first to third avatar images on a virtual background screen.

[0159] The second display device 102 may generate a first virtual avatar image based on the first tracking information 821 and the first virtual avatar information, generate a second virtual avatar image based on the second tracking information 822 and the second virtual avatar information, and generate a third virtual avatar image based on the third tracking information 823 and the third virtual avatar information. At this time, the second virtual avatar information may be set by the second user, and the first virtual avatar information and the third virtual avatar information may be set by the second user or received from the first display device 101 and the third display device 103, respectively. However, the present disclosure is not limited thereto.

[0160] The second display device 102 may generate and display the second group image 832 by combining the first to third avatar images onto the virtual background screen.

[0161] Alternatively, when the second display device 102 may not generate a virtual avatar image or a group image based on the tracking information, the second display device 102 may transmit the first to third tracking information 821, 822, and 823 to an external electronic device linked to the second display device 102. The external electronic device may generate the first to third virtual avatar images based on the first to third tracking information 821, 822, and 823. In addition, the external electronic device may generate the second group image 832 by combining the first to third virtual avatar images onto a virtual background screen. The external electronic device may transmit the first to third virtual avatar images or the second group image 832 to the second display device 102.

[0162] The third display device 103 may generate a first virtual avatar image based on the first tracking information 821 and the first virtual avatar information, generate a second virtual avatar image based on the second tracking information 822 and the second virtual avatar information, and generate a third virtual avatar image based on the third tracking information 823 and the third virtual avatar information. At this time, the third virtual avatar information may be set by a third user, and the first virtual avatar information and the second virtual avatar information may be set by the third user or received from the first display device 101 and the second display device 102, respectively. However, the present disclosure is not limited thereto.

[0163] The third display device 103 may generate and display the third group image 833 by combining the first to third avatar images on the virtual background screen.

[0164] Fig. 9 is a flowchart illustrating an operation method of a plurality of display devices and a server when a group service configuration method is determined based on a server according to an embodiment.

[0165] The plurality of devices according to an embodiment may include a first display device 101 and a second display device 102 , and the first display device 101 and the second display device 102 may be devices connected to a group service.

[0166] refer to Fig. 9 According to an embodiment, the first display device 101 may obtain a captured first user image of a first user by using a camera (S910). At this time, the camera may be a camera installed on the first display device 101 or a camera connected to the first display device 101 via wired or wireless communication.

[0167] In addition, the first display device 101 may transmit the first user image to the server 200 ( S920 ).

[0168] The second display device 102 may obtain a captured second user image of the second user by using a camera (S915). At this time, the camera may be a camera installed on the second display device 102 or a camera connected to the second display device 102 via wired or wireless communication.

[0169] Furthermore, the second display device 102 may transmit the second user image to the server 200 ( S925 ).

[0170] The server 200 may detect the first user from the first user image and obtain first tracking information about the first user. In addition, the server 200 may detect the second user from the second user image and obtain second tracking information about the second user (S940).

[0171] For example, the server 200 may detect the first user by analyzing the first user image using an object detection module. The object detection module may include suitable logic, circuits, interfaces, and / or codes that enable detection of a specific object included in an image captured by a camera. The object detection module may detect the first user by analyzing the first user image using at least one of machine learning, neural networks, or deep learning algorithms as a rule-based or artificial intelligence algorithm. In addition, the server 200 may obtain the first tracking information by using a tracking module. The tracking module may include suitable logic, circuits, interfaces, and / or codes that are capable of tracking the motion of a specific object included in an image captured by a camera. The tracking module may obtain the motion information of the first user from the first user image by using at least one of machine learning, neural networks, or deep learning algorithms as a rule-based or artificial intelligence algorithm.

[0172] For example, the tracking module may estimate a posture of the first user, identify a specific body part (eg, face, hand, etc.) of the first user based on the estimated posture, and detect motion relative to the identified specific body part in more detail.

[0173] In addition, the server 200 may detect the second user by analyzing the second user image using the object detection module, and may obtain motion information of the second user from the second user image by using the tracking module.

[0174] Meanwhile, the first display device 101 may transmit first avatar information set by the first user to the server 200 ( S950 ), and the second display device 102 may transmit second avatar information set by the second user to the server 200 ( S955 ).

[0175] The server 200 may generate a first virtual avatar image based on the first tracking information and the first virtual avatar information, and may generate a second virtual avatar image based on the second tracking information and the second virtual avatar information (S960). At this time, the first virtual avatar image may be an image that moves corresponding to the first user in the first user image, and the second virtual avatar image may be an image that moves corresponding to the second user in the second user image.

[0176] The server 200 may generate a group image including the first avatar image and the second avatar image (S970). For example, the server 200 may generate the group image by combining the first avatar image and the second avatar image on a virtual background screen.

[0177] Alternatively, the server 200 may generate the first group image and the second group image, the first group image and the second group image having different sizes and positions of the first virtual avatar image and the second virtual avatar image. For example, the first virtual avatar image may be displayed larger than the second virtual avatar image in the first group image, and the second virtual avatar image may be displayed larger than the first virtual avatar image in the second group image. However, the present disclosure is not limited thereto.

[0178] The server 200 may transmit the group image to the first display device 101 and the second display device 102 ( S980 , S985 ). For example, the server 200 may transmit the first group image to the first display device 101 , and transmit the second group image to the second display device 102 .

[0179] The first display device 101 and the second display device 102 may display the group image received from the server 200 (S990, S995). For example, the first display device 101 may display the first group image, and the second display device 102 may display the second group image.

[0180] Fig.10 is a flowchart illustrating an operation method of a plurality of display devices and a server when a group service configuration method is determined based on a server according to an embodiment.

[0181] The plurality of devices according to an embodiment may include a first display device 101 and a second display device 102 , and the first display device 101 and the second display device 102 may be devices connected to a group service.

[0182] refer to Fig.10 , the first display device 101 according to the embodiment may obtain a captured first user image of a first user by using a camera ( S1010 ).

[0183] The first display device 101 may detect a first user from the first user image, and obtain first tracking information about the detected first user ( S1020 ).

[0184] For example, the first display device 101 can detect the first user by analyzing the first user image using an object detection module. The object detection module may include suitable logic, circuits, interfaces and / or codes that enable detection of a specific object included in an image captured by a camera. The object detection module can detect the first user by analyzing the first user image using at least one of a machine learning, a neural network, or a deep learning algorithm as a rule-based or artificial intelligence algorithm.

[0185] In addition, the first display device 101 can obtain the first tracking information by using a tracking module. The tracking module may include suitable logic, circuits, interfaces and / or codes capable of tracking the movement of a specific object included in an image captured by a camera. The tracking module may obtain the movement information of the first user from the first user image by using at least one of a machine learning, a neural network or a deep learning algorithm as a rule-based or artificial intelligence algorithm.

[0186] For example, the tracking module may estimate a posture of the first user, identify a specific body part (eg, face, hand, etc.) of the first user based on the estimated posture, and detect motion relative to the identified specific body part in more detail.

[0187] However, the above-mentioned object detection module and tracking module are merely examples, and the first display device 101 according to an embodiment may detect the first user from the first user image and obtain the first tracking information through various methods.

[0188] The first display device 101 may transmit the first tracking information to the server 200 ( S1030 ).

[0189] In addition, the first display device 101 may transmit first avatar information set by the first user to the server 200 ( S1050 ).

[0190] In addition, the second display device 102 according to the embodiment may obtain a captured second user image of the second user by using a camera ( S1015 ).

[0191] The second display device 102 may detect the second user from the second user image and obtain second tracking information about the second user ( S1025 ).

[0192] The second display device 102 may obtain second tracking information about the second user by using the object detection module and the tracking module described above with reference to S1020 .

[0193] The second display device 102 may transmit the second tracking information to the server 200 ( S1035 ).

[0194] In addition, the second display device 102 may transmit second avatar information set by the second user to the server 200 ( S1055 ).

[0195] The server 200 according to the embodiment may generate a first avatar image based on the first tracking information and the first avatar information, and may generate a second avatar image based on the second tracking information and the second avatar information (S1060). At this time, the first avatar image may be an image moving corresponding to the first user in the first user image, and the second avatar image may be an image moving corresponding to the second user in the second user image.

[0196] The server 200 may generate a group image including the first avatar image and the second avatar image (S1070). For example, the server 200 may generate the group image by compositing the first avatar image and the second avatar image onto a virtual background screen.

[0197] Alternatively, the server 200 may generate the first group image and the second group image, the first group image and the second group image having different sizes and positions of the first virtual avatar image and the second virtual avatar image. For example, the first virtual avatar image may be displayed larger than the second virtual avatar image in the first group image, and the second virtual avatar image may be displayed larger than the first virtual avatar image in the second group image. However, the present disclosure is not limited thereto.

[0198] The server 200 may transmit the group image to the first display device 101 and the second display device 102 ( S1080 , S1085 ). For example, the server 200 may transmit the first group image to the first display device 101 and the second group image to the second display device 102 .

[0199] The first display device 101 and the second display device 102 may display the group image received from the server 200 (S1090, S1095). For example, the first display device 101 may display the first group image, and the second display device 102 may display the second group image.

[0200] Fig.11 is a block diagram showing a configuration of a display device according to an embodiment.

[0201] according to Fig.11 The display device 100 of the embodiment may be Figures 1 to 10 Any one of the multiple display devices 101, 102 and 103.

[0202] refer to Fig.11 , the display device 100 according to the embodiment may include a camera 110 , a processor 120 , a memory 130 , a display 140 , and a communication unit 150 .

[0203] The camera 110 according to an embodiment may obtain image frames such as still images, videos, etc. For example, the camera 110 may capture an image of the outside of the display device 100, and may capture an image of a user of the display device 100. At this time, the image captured by the camera 110 may be processed by the processor 120 or a separate image processor.

[0204] The camera 110 according to an embodiment may be built-in or installed as a component of the display device 100. Alternatively, the camera 110 may be connected to the display device 100 via wired or wireless communication through the input / output interface or the communication unit 150 as a component separate from the display device 100. However, the present disclosure is not limited thereto.

[0205] The processor 120 according to an embodiment controls the overall operation of the display apparatus 100 and signal flows between internal components of the display apparatus 100 , and performs a function of processing data.

[0206] The processor 120 may include a single core, a dual core, a triple core, a quad core, and multiple cores thereof. In addition, the processor 120 may include a plurality of processors. For example, the processor 120 may be implemented as a main processor (not shown) and a sub-processor (not shown).

[0207] In addition, the processor 120 may include at least one of a central processing unit (CPU), a graphics processing unit (GPU), and a video processing unit (VPU). Alternatively, according to an embodiment, the processor 120 may be implemented in the form of a system on chip (SoC) integrating at least one of the CPU, GPU, and VPU. Alternatively, the processor 120 may also include a neural processing unit (NPU).

[0208] The memory 130 according to an embodiment may store various pieces of data, programs, or applications for driving and controlling the display device 100 .

[0209] In addition, the program stored in the memory 130 may include one or more instructions. The program (one or more instructions) or application stored in the memory 130 may be executed by the processor 120.

[0210] The processor 120 according to an embodiment may control the communication unit 150 to obtain device capability information of the display device 100 by executing one or more instructions stored in the memory 130 , and transmit the obtained device capability information to the server 200 .

[0211] According to an embodiment, the device capability information may include at least one of the following: type information, capacity information, information about the number of cores and performance information for the at least one processor included in the display device 100; and type information, capacity information and performance information for the memory included in the display device 100.

[0212] The processor 120 may receive information on a configuration method of the group service from the server 200. Alternatively, the processor 120 may determine a configuration method of the group service based on device capability information and service determination policy information received from the server 200.

[0213] When determining the group service configuration method according to the embodiment based on the device, the processor 120 can detect the user from the captured user image of the user and obtain tracking information. The processor 120 can detect the user by analyzing the user image using an object detection module. The object detection module may include suitable logic, circuits, interfaces and / or codes that enable detection of a specific object included in an image captured by the camera. The object detection module may detect the user by analyzing the user image using at least one of a machine learning, a neural network or a deep learning algorithm as a rule-based or artificial intelligence algorithm.

[0214] In addition, the processor 120 may obtain the tracking information by using a tracking module. The tracking module may include suitable logic, circuitry, interfaces and / or code capable of tracking the movement of a specific object included in an image captured by the camera. The tracking module may obtain the user's movement information from the user image by using at least one of a machine learning, neural network or deep learning algorithm as a rule-based or artificial intelligence algorithm.

[0215] For example, the tracking module may estimate the user's posture, identify a specific body part (eg, face, hand, etc.) of the user based on the estimated posture, and detect movement relative to the identified specific body part in more detail. However, the present disclosure is not limited thereto.

[0216] For example, the processor 120 may obtain the tracking information by using a neural network trained to obtain the tracking information from the user image. However, the present disclosure is not limited thereto.

[0217] The neural network according to the embodiment may be a network that has been trained based on various training data under various standards or conditions for obtaining tracking information from a user image. Here, "training" may mean training a neural network so that the neural network can discover or learn by itself to obtain a method of analyzing input data when various pieces of collected data are input to the neural network, a method of obtaining tracking information from a user image based on the input data, and the like. Therefore, the neural network that has completed training can obtain tracking information from a user image. At this time, the trained neural network may be pre-stored in the display device 100 and may be periodically updated through communication with an external server.

[0218] The processor 120 according to an embodiment may generate an avatar image corresponding to the user image. The processor 120 may generate an avatar image based on avatar information and tracking information set by the user. At this time, the avatar image may be an image that moves corresponding to the user in the user image.

[0219] The processor 120 according to an embodiment may control the communication unit 150 to transmit the avatar image to other display devices. In addition, the processor 120 may receive the avatar image of another user or information for generating another avatar image (e.g., tracking information about another user) from another display device through the communication unit 150.

[0220] The processor 120 may generate a group image by combining the avatar image and an avatar image of another user (another avatar image).

[0221] In contrast, when the group service configuration method according to the embodiment is determined on a device basis, the processor 120 may control the communication unit 150 to transmit the user image or tracking information detected from the user image to the server 200 .

[0222] The processor 120 may receive a group image including avatar images of a plurality of users from the server 200 , and may control the display 140 to display the received group image.

[0223] The display 140 according to an embodiment may generate a driving signal by converting an image signal, a data signal, an on screen display (OSD) signal, a control signal, etc., which have been processed by the processor 120 , and may display an image according to the driving signal.

[0224] The display 140 according to the embodiment may be implemented as a plasma display panel (PDP), a liquid crystal display (LCD), an organic light emitting diode (OLED), a flexible display, etc., and may also be implemented as a three-dimensional (3D) display. In addition, the display 140 is configured as a touch screen and may be used as an input device in addition to an output device.

[0225] The display 140 according to an embodiment may display a user interface screen in which avatar information may be set. In addition, the display 140 may display a group image including a plurality of avatar images for a plurality of users. For example, the display 140 may divide a single screen into a plurality of regions, and separately display different avatar images in each divided region.

[0226] Alternatively, the group image according to an embodiment may include an image in which a plurality of avatar images are synthesized onto a virtual background screen. However, the present disclosure is not limited thereto.

[0227] The communication unit 150 according to the embodiment can send data or signals to an external device or server and receive data or signals from an external device or server. For example, the communication unit 150 may include a Wi-Fi module, a Bluetooth module, an infrared communication module, a wireless communication module, a LAN module, an Ethernet module, a wired communication module, etc. At this time, each communication module can be implemented in the form of at least one hardware chip.

[0228] The Wi-Fi module and the Bluetooth module communicate via the Wi-Fi mode and the Bluetooth mode, respectively. When using the Wi-Fi module or the Bluetooth module, various pieces of connection information such as a service set identifier (SSID), a session key, etc. may be first sent and received, and various types of information may be sent and received after establishing communication by using the connection information. The wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards, such as Zigbee, third generation (3G), third generation partnership project (3GPP), long term evolution (LTE), LTE advanced (LTE-A), fourth generation (4G), fifth generation (5G), etc.

[0229] The communication unit 150 according to an embodiment may transmit device capability information to the server 200 , and may receive information on a group service configuration method from the server 200 .

[0230] In addition, the communication unit 150 may also transmit a captured user image of a user or tracking information obtained from the user image to the server 200 , and receive a group image including avatar images for a plurality of users from the server 200 .

[0231] In addition, the communication unit 150 may receive identification information about other display devices connected to the group service from the server 200 .

[0232] In addition, the communication unit 150 may transmit tracking information or an avatar image obtained from a user image to another display device, and may receive tracking information about another user or an avatar image for another user from a user display device. However, the present disclosure is not limited thereto.

[0233] Fig.12 is a block diagram showing a configuration of a display device according to an embodiment.

[0234] refer to Fig.12 , Fig.12 The display device 1200 may be Fig.11 Another embodiment of the display device 100 is shown.

[0235] Reference Fig.12 According to an embodiment, the display device 1200 may include a tuner unit 1240, a processor 1210, a display 1220, a communication unit 1250, a sensing unit 1230, an input / output unit 1270, a video processing unit 1280, an audio processing unit 1285, an audio output unit 1260, a memory 1290 and a power supply unit 1295.

[0236] Included in Fig.12 The camera unit 1232, the processor 1210, the display 1220, the communication unit 1250 and the memory 1290 in the sensing unit 1230 are respectively Fig.11 The components correspond to the camera 110, the processor 120, the display 140, the communication unit 150 and the memory 130, and thus the same description is omitted.

[0237] The tuner unit 1240 according to an embodiment may tune and select only the frequency of a channel to be received by the display device 1200 among radio wave components by amplifying, mixing, resonating, etc. a broadcast signal received by wire or wirelessly. The broadcast signal includes audio, video, and additional information (eg, electronic program guide (EPG)).

[0238] The tuner unit 1240 may receive broadcast signals from various sources such as terrestrial broadcasting, cable broadcasting, satellite broadcasting, Internet broadcasting, etc. The tuner unit 1240 may also receive broadcast signals from sources such as analog broadcasting or digital broadcasting.

[0239] The sensing unit 1230 may sense a user's voice, image, or interaction, and may include a microphone 1231 , a camera unit 1232 , and a light receiving unit 1233 .

[0240] The microphone 1231 receives a user's speech voice. The microphone 1231 may convert the received voice into an electric signal and output the electric signal to the processor 1210. The user's speech voice may include, for example, a voice corresponding to a menu or function of the display device 1200.

[0241] The camera unit 1232 may receive images (eg, continuous frames) corresponding to the user's motion including gestures within the camera recognition range. The processor 1210 may use the received motion recognition result to select a menu displayed on the display device 1200 or perform control corresponding to the motion recognition result.

[0242] The light receiving unit 1233 receives an optical signal (including a control signal) received from an external control device through an optical window (not shown) of a bezel of the display 1220 or the like. The light receiving unit 1233 may receive an optical signal corresponding to a user input (e.g., touch, press, touch gesture, voice, or motion) from the control device. The control signal may be extracted from the received optical signal under the control of the processor 1210.

[0243] The input / output unit 1270 receives video (e.g., moving images, etc.), audio (e.g., voice, music, etc.), and additional information (e.g., EPG, etc.) from the outside of the display device 1200. The input / output interface may include any one of a high-definition multimedia interface (HDMI), a mobile high-definition link (MHL), a universal serial bus (USB), a thunderbolt interface, a video graphics array (VGA) port, a red, green, and blue (RGB) port, a D-subminiature (D-SUB), a digital visual interface (DVI), a component jack, and a PC port.

[0244] The processor 1210 controls the overall operation of the display device 1200 and signal flows between internal components of the display device 1200 , and performs a function of processing data.

[0245] The processor 1210 may include a random access memory (RAM) that stores a signal or data input from the outside of the display device 1200 or is used as a storage area corresponding to various tasks performed in the display device 1200, a read-only memory (ROM) that stores a control program for controlling the display device 1200, and a processor.

[0246] The video processing unit 1280 processes the video data received by the display device 1200. The video processing unit 1280 may perform various image processing such as decoding, scaling, noise filtering, frame rate conversion, and resolution conversion on the video data.

[0247] The display 1220 according to the embodiment may generate a driving signal by converting an image signal, a data signal, an OSD signal, a control signal, etc., which have been processed by the processor 1210 , and may display an image according to the driving signal.

[0248] The audio processing unit 1285 according to an embodiment performs processing on the audio data. The audio processing unit 1285 may perform various processing on the audio data, such as decoding, amplification, and noise filtering. Meanwhile, the audio processing unit 1285 may include a plurality of audio processing modules to process audio corresponding to a plurality of contents.

[0249] The audio output unit 1260 outputs audio included in a broadcast signal received through the tuner unit 1240 under the control of the processor 1210. The audio output unit 1260 may output audio (e.g., voice and sound) input through the communication unit 1250 or the input / output unit 1270. In addition, the audio output unit 1260 may output audio stored in the memory 1290 under the control of the processor 1210. The audio output unit 1260 may include at least one of a speaker, a headphone output terminal, or a Sony / Philips Digital Interface (S / PDIF) output terminal.

[0250] The power supply unit 1295 supplies power input from an external power source to components inside the display device 1200 under the control of the processor 1210. In addition, the power supply unit 1295 may supply power output from one or more batteries (not shown) located inside the display device 1200 to components inside the display device 1200 under the control of the power supply unit 1295.

[0251] The memory 1290 may store various pieces of data, programs or applications for driving and controlling the display device 1200 under the control of the processor 1210. The memory 1290 may include a broadcast receiving module, a channel control module, a volume control module, a communication control module, a voice recognition module, a motion recognition module, a light receiving module, a display control module, an audio control module, an external input control module, a power control module, a power control module of a wirelessly connected external device (e.g., Bluetooth), a voice database (DB) or a motion DB, which is not shown in the figure. The modules and DB not shown in the memory 1290 may be implemented in the form of software to perform a control function of broadcast reception, a channel control function, a volume control function, a communication control function, a voice recognition function, a motion recognition function, a light receiving function, a display control function, an audio control function, an external input control function and a power control function in the display device 1200, or a power control function of a wirelessly connected external device (e.g., Bluetooth). The processor 1210 may perform the corresponding function by using the software stored in the memory 1290.

[0252] on the other hand, Fig.11 and Fig.12 The block diagrams of the display devices 100 and 1200 shown respectively are block diagrams for one embodiment. Each component of the block diagram may be integrated, added, or omitted depending on the specifications of the display devices 100 and 1200 to be actually implemented. That is, two or more components may be combined into one component, or one component may be divided into two or more components as required. In addition, the functions performed in each frame are intended to explain the embodiments, and their specific operations or devices do not limit the scope of the present disclosure.

[0253] Fig.13 is a block diagram showing a configuration of a server according to an embodiment.

[0254] refer to Fig.13 , the server 200 according to the embodiment may include a communication unit 210 , a processor 220 , and a memory 230 .

[0255] The communication unit 210 according to the embodiment can send data or signals to multiple display devices and receive data or signals from multiple display devices. For example, the communication unit 210 may include a Wi-Fi module, a Bluetooth module, an infrared communication module, a wireless communication module, a LAN module, an Ethernet module, a wired communication module, etc. At this time, each communication module can be implemented in the form of at least one hardware chip.

[0256] The Wi-Fi module and the Bluetooth module communicate via the Wi-Fi mode and the Bluetooth mode, respectively. When using the Wi-Fi module or the Bluetooth module, various pieces of connection information such as a service set identifier (SSID), a session key, etc. may be first sent and received, and various types of information may be sent and received after establishing communication by using the connection information. The wireless communication module may include at least one communication chip that performs communication according to various wireless communication standards (such as Zigbee, 3G, 3GPP, LTE, LTE-A, 4G, 5G, etc.).

[0257] The communication unit 210 according to an embodiment may receive device capability information from a plurality of display devices connected to the group service. In addition, the communication unit 210 may transmit information on a group service configuration method determined based on the received device capability information to the plurality of display devices.

[0258] In addition, the communication unit 210 may receive user images or tracking information obtained from the user images from a plurality of display devices. The communication unit 210 may transmit a group image including avatar images of a plurality of users generated based on the user images or the tracking information to the plurality of display devices.

[0259] The processor 220 according to an embodiment controls the overall operation of the server 200 and signal flows between internal components of the server 200 , and performs a function of processing data.

[0260] The processor 220 may include a single core, a dual core, a triple core, a quad core, and multiple cores thereof. In addition, the processor 220 may include a plurality of processors. For example, the processor 220 may be implemented as a main processor (not shown) and a sub-processor (not shown).

[0261] In addition, the processor 220 may include at least one of a CPU, a GPU, and a VPU. Alternatively, according to an embodiment, the processor 120 may be implemented in the form of a SoC integrating at least one of a CPU, a GPU, and a VPU. Alternatively, the processor 220 may also include an NPU.

[0262] The memory 230 according to an embodiment may store various pieces of data, programs, or applications for driving and controlling the server 200 .

[0263] In addition, the program stored in the memory 230 may include one or more instructions. The program (one or more instructions) or application stored in the memory 230 may be executed by the processor 220.

[0264] The processor 220 according to an embodiment may determine a configuration method of the group service based on the device capability information and the service determination policy information received from the display device by executing one or more instructions stored in the memory 230 .

[0265] According to an embodiment, the device capability information may include at least one of the following: type information, capacity information, information about the number of cores and performance information for the at least one processor included in the display device; and type information, capacity information and performance information for the memory included in the display device.

[0266] The server 200 may store service determination policy information for determining a service configuration method. The service determination policy information may include standards for device specifications required to perform operations included in the group service (such as generating a virtual image on a device, etc.) (e.g., type information, capacity information, information about the number of cores, and performance information for a processor; and type information, capacity information, and performance information for a memory). However, the present disclosure is not limited thereto.

[0267] The service determination policy information may vary depending on the number of concurrent users connected to the server, network conditions, etc. In addition, the service determination policy information may also vary as the functions provided by the group service change. However, the present disclosure is not limited thereto.

[0268] The processor 220 may determine whether the configuration of the group service is device-based or server-based by comparing the device capability information with the service determination policy information.

[0269] When it is determined that the configuration of the group service is based on the server, the processor 220 may detect the user from the user image received from the display device and obtain tracking information.

[0270] The processor 220 may detect the user by analyzing the user image using an object detection module. The object detection module may include suitable logic, circuitry, interfaces, and / or code that enables detection of a specific object included in an image captured by the camera. The object detection module may detect the user by analyzing the user image using at least one of a machine learning, neural network, or deep learning algorithm as a rule-based or artificial intelligence algorithm.

[0271] In addition, the processor 220 may obtain the tracking information by using a tracking module. The tracking module may include suitable logic, circuitry, interfaces and / or code capable of tracking the movement of a specific object included in an image captured by the camera. The tracking module may obtain the user's movement information from the user image by using at least one of a machine learning, neural network or deep learning algorithm as a rule-based or artificial intelligence algorithm.

[0272] For example, the tracking module may estimate the user's posture, identify a specific body part (eg, face, hand, etc.) of the user based on the estimated posture, and detect movement relative to the identified specific body part in more detail. However, the present disclosure is not limited thereto.

[0273] Alternatively, the processor 220 may generate an avatar image based on the tracking information and avatar information received from the display device. At this time, the avatar image may be an image that moves corresponding to the user in the user image.

[0274] The processor 220 may generate a group image by combining avatar images of a plurality of users, and control the communication unit 210 to transmit the group image to a plurality of display devices.

[0275] The display device according to the embodiment may include a communication unit, a display, a memory storing one or more instructions, and at least one processor executing the one or more instructions.

[0276] The at least one processor may obtain a captured first user image of the first user by executing the one or more instructions.

[0277] The at least one processor may control the communication unit to transmit the capability information of the display device to the server by executing the one or more instructions.

[0278] The at least one processor may receive, from the server, information about a configuration method of the group service determined based on the capability information by executing the one or more instructions.

[0279] The at least one processor may obtain the first virtual avatar image and a second virtual avatar image corresponding to a user of another display device based on the received information by executing the one or more instructions.

[0280] The at least one processor may control the display device to display a group image including the first avatar image and the second avatar image by executing one or more instructions.

[0281] The capability information of the display device may include at least one of the following: type information, capacity information, information about the number of cores and performance information for the at least one processor included in the display device; and type information, capacity information and performance information for the memory included in the display device.

[0282] The configuration method of the group service can be determined as any one of the following: a device-based configuration method, through which the display device generates the first virtual avatar image corresponding to the first user image; and a server-based configuration method, through which the server generates the first virtual avatar image and the second virtual avatar image corresponding to a user of another display device connected to the group service.

[0283] The at least one processor may detect the first user from the first user image and obtain the first avatar image based on tracking information of the first user by executing the one or more instructions based on determining the configuration method of the group service as a device-based configuration method.

[0284] The at least one processor may control the communication unit to receive at least one of the second avatar image and the information about the second avatar image from another display device by executing one or more instructions.

[0285] The at least one processor may generate a second avatar image based on receipt of information regarding the second avatar image by executing one or more instructions.

[0286] The at least one processor may generate the group image by combining the first avatar image, the second avatar image, and a virtual background image by executing the one or more instructions.

[0287] The at least one processor may control the communication unit to request the server for device information about the other display device and receive the device information about the other display device from the server by executing the one or more instructions.

[0288] The at least one processor may control the communication unit to transmit at least one of the first virtual avatar image or the information about the first virtual avatar image to another display device based on the device information about the other display device by executing one or more instructions.

[0289] The at least one processor may control the communication unit to send at least one of the first user image and the information about the first user detected from the first user image to the server based on determining the configuration method of the group service as a server-based configuration method by executing the one or more instructions.

[0290] The at least one processor may receive the group image including the first avatar image and the second avatar image from the server by executing the one or more instructions.

[0291] The at least one processor may receive an input for setting avatar information corresponding to the first avatar image by executing the one or more instructions.

[0292] The first avatar image may be an avatar image generated based on the set avatar information.

[0293] The display device may further include a camera which obtains the first user image by capturing the image of the first user.

[0294] The operating method of the display device according to the embodiment may include obtaining a captured first user image of a first user.

[0295] The operating method of the display device according to the embodiment may include transmitting capability information of the display device to a server.

[0296] The operating method of the display device according to the embodiment may include receiving information about a configuration method of the group service determined based on the capability information from a server.

[0297] The operating method of the display device according to the embodiment may include obtaining a first avatar image and a second avatar image corresponding to a user of another display device based on the received information.

[0298] The operating method of the display apparatus according to the embodiment may include displaying a group image including a first avatar image and a second avatar image.

[0299] The obtaining of the first avatar image and the second avatar image may include: based on determining the configuration method of the group service as the device-based configuration method, detecting the first user from the first user image and generating the first avatar image based on tracking information about the first user.

[0300] The obtaining of the first virtual avatar image and the second virtual avatar image may receive at least one of the second virtual avatar image and information about the second virtual avatar image from the other display device.

[0301] The obtaining of the first avatar image and the second avatar image may include generating the second avatar image based on receiving information about the second avatar image.

[0302] The operating method of the display device according to the embodiment may further include generating a group image by combining the first avatar image, the second avatar image, and the virtual background screen.

[0303] The operating method of the display device according to the embodiment may further include requesting device information about other display devices from a server and receiving the device information about the display devices from the server based on determining the configuration method of the group service as the device-based configuration method.

[0304] The method for operating a display device according to an embodiment may further include transmitting at least one of the first avatar image or information about the first avatar image to other display devices based on device information about the other display devices.

[0305] The obtaining of the first avatar image and the second avatar image may include transmitting at least one of the first user image and information about the first user detected from the first user image to the server based on determining the configuration method of the group service as the server-based configuration method.

[0306] The obtaining of the first avatar image and the second avatar image may include receiving a group image including the first avatar image and the second avatar image from a server.

[0307] The operating method of the display device according to the embodiment may include receiving an input for selecting avatar information corresponding to the first avatar image.

[0308] The first avatar image may be an avatar image generated based on the set avatar information.

[0309] The operating method of the display device according to the embodiment can variably configure the group service according to the device capability information of the display device, so that various display devices can use the group service. Therefore, even when the specifications of the device are not high, the group service can be used through the server, thereby expanding the range of services that can be provided.

[0310] In addition, the display apparatus and the operating method thereof according to the embodiment may optimize the configuration method of the group service by variably configuring the group service according to the network condition.

[0311] In addition, the display apparatus and the operating method thereof according to the embodiment may protect the privacy of each user by displaying a group image converted from a user image to an avatar image when using a group service.

[0312] The operating method of the display device according to the embodiment can be implemented in the form of a program command that can be executed by various computer units and recorded on a computer-readable medium. The computer-readable medium may include program commands, data files, data structures, etc., alone or in combination. The program commands recorded in the medium may be program commands specially designed and configured for the present disclosure or may be program commands known and available to technicians in the field of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks, and tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as optical magnetic floppy disks, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program commands include machine language codes, such as machine language codes generated by a compiler, and high-level language codes that can be executed by a computer using an interpreter, etc.

[0313] In addition, at least one of the operating methods of the display device according to the above-mentioned embodiments may be included and provided as a computer program product. The computer program product may be traded between a seller and a buyer as a commodity.

[0314] A computer program product may include a software (S / W) program and a computer-readable storage medium on which the S / W program is stored. For example, a computer program product may include a product in the form of a S / W program (e.g., a downloaded application) electronically distributed by a manufacturer of an electronic device or an electronic market (e.g., Google Play Store, App Store). At least a portion of the S / W program may be stored on a storage medium or temporarily generated for electronic distribution. In this case, the storage medium may be a storage medium of a manufacturer's server, a server of an electronic market, or a relay server that temporarily stores the S / W program.

[0315] The computer program product may include a storage medium of a server or a storage medium of a client device in a system configured with a server and a client device. Alternatively, when a third device (e.g., a smart phone) communicates and connects with the server or the client device, the computer program product may include a storage medium of the third device. Alternatively, the computer program product may include an S / W program that is sent from the server to the client device or the third device or from the third device to the client device.

[0316] In this case, one of the server, the client device, and the third device may execute a computer program product to perform the method according to the embodiment disclosed above. Alternatively, two or more of the server, the client device, and the third device may execute a computer program product to implement the method according to the embodiment disclosed above in a distributed manner.

[0317] For example, a server (eg, a cloud server or an artificial intelligence server) may execute a computer program product stored on the server to control a client device communicating with the server to perform the method according to the embodiments disclosed above.

[0318] Although the embodiments have been described in detail above, the scope of the present disclosure is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present disclosure defined in the appended claims also fall within the scope of the present disclosure.

Claims

1. A display device 100, comprising: Communication unit 150; Display 140; A memory 130 storing one or more instructions; as well as At least one processor 120, configured to execute one or more instructions, The at least one processor 120 executes the one or more instructions to: obtaining a captured first-user image of a first user, controlling the communication unit to send capability information about the display device to a server, receiving, from the server, information on a configuration method of a group service determined based on the capability information, Based on the received information, obtaining a first avatar image and a second avatar image corresponding to a user of another display device, and The display is controlled to display a group image including the first virtual avatar image and the second virtual avatar image.

2. The display device according to claim 1, wherein: The capability information of the display device includes at least one of the following: type information, capacity information, information about the number of cores and performance information for at least one processor included in the display device; and type information, capacity information and performance information for the memory included in the display device.

3. The display device according to claim 1 or claim 2, wherein: The configuration method of the group service is determined to be any one of the following: a device-based configuration method, through which the display device generates the first avatar image corresponding to the first user image; and a server-based configuration method by which the server generates the first avatar image and the second avatar image corresponding to a user of another display device connected to the group service.

4. The display device according to claim 3, wherein: The at least one processor 120 executes the one or more instructions to: based on determining the configuration method of the group service as the device-based configuration method, detecting the first user from the first user image and generating the first avatar image based on tracking information about the first user, controlling the communication unit to receive at least one of the second virtual avatar image and information about the second virtual avatar image from the other display device, and Based on the receipt of the information regarding the second avatar image, the second avatar image is generated.

5. The display device according to claim 4, wherein: The at least one processor 120 executes the one or more instructions to generate the group image by combining the first virtual avatar image, the second virtual avatar image, and a virtual background image.

6. The display device according to claim 4 or 5, wherein: The at least one processor 120 executes one or more instructions to control the communication unit 150 to request the device information about the other display device from the server, and receive the device information about the other display device from the server.

7. The display device according to claim 6, wherein: The at least one processor 120 executes the one or more instructions to control the communication unit 150 to transmit at least one of the first virtual avatar image or information about the first virtual avatar image to the other display device based on the device information about the other display device.

8. The display device according to any one of claims 3 to 7, wherein: The at least one processor 120 executes the one or more instructions to: based on determining the configuration method of the group service as the server-based configuration method, controlling the communication unit 150 to transmit at least one of the first user image and information about the first user detected from the first user image to the server, and The group image including the first avatar image and the second avatar image is received from the server.

9. The display device according to any one of claims 1 to 8, wherein: The at least one processor 120 executes the one or more instructions to receive an input selecting avatar information corresponding to the first avatar image, and the first avatar image includes an avatar image generated based on the selected avatar information.

10. The display device according to any one of claims 1 to 9, wherein: The display device further includes a camera 110 configured to obtain the first user image by capturing an image of the first user.

11. A method for operating a display device, comprising: obtaining a captured first-user image of a first user, sending capability information about the display device to a server, receiving, from the server, information on a configuration method of a group service determined based on the capability information, Based on the received information, obtaining a first avatar image and a second avatar image corresponding to a user of another display device, and A group image including the first avatar image and the second avatar image is displayed.

12. The method according to claim 11, wherein: The capability information about the display device includes at least one of the following: type information, capacity information, information about the number of cores and performance information for at least one processor included in the display device; and type information, capacity information and performance information for the memory included in the display device.

13. The method according to claim 11 or 12, wherein: The configuration method of the group service is determined to be any one of the following: a device-based configuration method, through which the display device generates the first avatar image corresponding to the first user image; and a server-based configuration method by which the server generates the first avatar image and the second avatar image corresponding to a user of another display device connected to the group service.

14. The method according to claim 13, wherein: Obtaining the first avatar image and the second avatar image includes: based on determining the configuration method of the group service as the device-based configuration method, detecting the first user from the first user image and generating the first avatar image based on tracking information about the first user, receiving at least one of the second virtual avatar image and information about the second virtual avatar image from the other display device, and Based on the receipt of the information regarding the second avatar image, the second avatar image is generated.

15. One or more computer-readable media storing a program allowing the execution of the method according to any one of claims 11 to 14.

Images