Electronic device and operating method thereof

Abstract

This disclosed electronic device is configured to: obtain a plurality of user images including an image corresponding to one or more users received from a server device for a video call and an image corresponding to a user of the electronic device; determine whether the capability of the electronic device for image processing corresponds to the plurality of user images to be decoded for the video call; on the basis of determining that the capability of the electronic device for the image processing does not correspond to the plurality of user images to be decoded for the video call, obtain a predetermined number of user images having deteriorated quality characteristics from among the plurality of user images; obtain a partial screen corresponding to the predetermined number of user images by processing the predetermined number of user images having the deteriorated quality characteristics by using one decoder; and output a multi-view screen including the partial screen.

Description

Electronic device and method of operation thereof

[0001] The disclosed embodiments relate to an electronic device providing a video call service and an operating method thereof. Specifically, the disclosed embodiments relate to an electronic device providing a video call service performed among multiple parties and an operating method thereof.

[0002] As wired and wireless communication networks and technologies advance, the use of video calling services between electronic devices is increasing. Specifically, video calling services between electronic devices are widely used to enable remote users to communicate with each other without face-to-face interaction.

[0003] For video call services, one electronic device and another electronic device can be interconnected via a wired or wireless communication network. The electronic device can be any electronic device that includes a display capable of providing a video call screen and can connect to a wired or wireless communication network to communicate with another electronic device in a remote location. Examples of electronic devices include portable computers such as laptops, netbooks, or tablet PCs; portable terminals such as smartphones or PDAs; and televisions and digital televisions.

[0004] When a video call is made between multiple electronic devices, for example, a first electronic device and a second electronic device, the first electronic device acquires an image of the first user and transmits the acquired image to the second electronic device. Furthermore, the second electronic device acquires an image of the second user and transmits the acquired image to the first electronic device. Accordingly, the first electronic device and the second electronic device each display a video call screen including images of the first user and the second user. Thus, the first user and the second user can make a call while viewing the image of the other user from a remote location.

[0005] According to one embodiment, an electronic device may include a memory storing one or more instructions, and at least one processor executing the one or more instructions.

[0006] According to one embodiment, when the one or more instructions are individually and / or collectively executed by the at least one processor, the electronic device can obtain a plurality of user images including images corresponding to one or more users received from a server device for a video call and images corresponding to a user of the electronic device.

[0007] According to one embodiment, when the one or more instructions are individually and / or collectively executed by the at least one processor, the electronic device can determine whether a capability for image processing of the electronic device corresponds to the plurality of user images to be decoded for the video call.

[0008] According to one embodiment, when the one or more instructions are individually and / or collectively executed by the at least one processor, the electronic device may obtain user images with degraded quality characteristics for a predetermined number of user images among the plurality of user images when it is determined that the capability for image processing of the electronic device does not correspond to the plurality of user images to be decoded for the video call.

[0009] According to one embodiment, when the one or more instructions are individually and / or collectively executed by the at least one processor, the electronic device can obtain a partial screen corresponding to the predetermined number of user images by processing the predetermined number of user images with degraded quality characteristics with one decoder.

[0010] According to one embodiment, when the one or more instructions are individually and / or collectively executed by the at least one processor, the electronic device can output a multi-view screen including the partial screen.

[0011] According to one embodiment, a method of operating an electronic device may include obtaining a plurality of user images, each of which includes an image corresponding to one or more users received from a server device for a video call and an image corresponding to a user of the electronic device.

[0012] A method of operating an electronic device according to one embodiment may include determining whether a capability for image processing of the electronic device corresponds to the plurality of user images to be decoded for the video call.

[0013] According to one embodiment, a method of operating an electronic device may include an operation of obtaining user images with degraded quality characteristics for a predetermined number of user images among the plurality of user images when it is determined that the capability for image processing of the electronic device does not correspond to the plurality of user images to be decoded for the video call.

[0014] According to one embodiment, a method of operating an electronic device may include an operation of obtaining a partial screen corresponding to a predetermined number of user images by processing the predetermined number of user images with degraded quality characteristics with a single decoder.

[0015] A method of operating an electronic device according to one embodiment may include outputting a multi-view screen including the partial screen.

[0016] According to one embodiment, a non-transitory computer-readable recording medium having recorded thereon a program including one or more instructions executable by a processor of an electronic device, wherein by executing the one or more instructions by at least one processor of the electronic device, the electronic device obtains a plurality of user images including images corresponding to one or more users received from a server device for a video call and images corresponding to users of the electronic device, determines whether a capability for image processing of the electronic device corresponds to the plurality of user images to be decoded for the video call, and when it is determined that the capability for image processing of the electronic device does not correspond to the plurality of user images to be decoded for the video call, obtains user images with degraded quality characteristics for a predetermined number of user images among the plurality of user images, processes the predetermined number of user images with degraded quality characteristics by one decoder, thereby obtaining partial screens corresponding to the predetermined number of user images, and outputs a multi-view screen including the partial screens.

[0017] The foregoing and other aspects, features and advantages of the embodiments of the present disclosure will become more apparent from the following detailed description taken in conjunction with the accompanying drawings.

[0018] FIG. 1 illustrates an example of a system that provides a video call service according to one embodiment.

[0019] Figure 2 illustrates an example of a system according to one embodiment.

[0020] FIG. 3 is an example of a block diagram of an electronic device according to one embodiment.

[0021] FIG. 4 illustrates an example of an image processing unit of an electronic device according to one embodiment.

[0022] Figure 5 illustrates an example of an image processing unit according to one embodiment.

[0023] FIG. 6 illustrates an example of a method of operating an electronic device according to one embodiment.

[0024] FIG. 7 is a flowchart illustrating an example of a method of operating an electronic device according to one embodiment.

[0025] FIG. 8 is a reference diagram for explaining an example of a method of operating an electronic device according to one embodiment.

[0026] FIG. 9 is a reference diagram for explaining an example of a method of operating an electronic device according to one embodiment.

[0027] FIG. 10 is a reference diagram for explaining an example of an operating method of an electronic device according to one embodiment. FIG. 10 is a reference diagram for explaining an example of an operating method of a tooth.

[0028] FIG. 11 is a reference diagram for explaining a method for a server device to transmit video of users participating in a video call based on the video processing capability of an electronic device according to one embodiment.

[0029] FIG. 12 is a flowchart illustrating an example of a method of operating an electronic device according to one embodiment.

[0030] FIG. 13 is a flowchart illustrating an example of a method of operating an electronic device according to one embodiment.

[0031] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings so that those skilled in the art can easily practice the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description are omitted to clearly explain the present invention, and similar parts are designated with similar reference numerals throughout the specification. In addition, throughout the drawings, the same reference numerals are assigned to the same components.

[0032] The terms used in this invention have been selected from widely used, current terms, taking into account the functions of the invention. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, in which case their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this invention should not be defined simply as names, but rather based on their inherent meanings and the overall content of the invention.

[0033] When a part of the specification is said to "include" a component, this does not exclude other components, but rather implies the inclusion of other components, unless otherwise specifically stated. Furthermore, terms such as "part," "module," etc., used in the specification refer to a unit that processes at least one function or operation, which may be implemented in hardware, software, or a combination of hardware and software.

[0034] Throughout the specification, when a part is said to be "connected" to another part, this includes not only the "direct connection" but also the "electrical connection" with other elements intervening between them. Furthermore, when a part is said to "include" a component, this does not exclude other components, but rather includes the possibility of including other components, unless otherwise stated.

[0035] The appearances of phrases such as “in an embodiment,” “in a disclosed embodiment,” “in some embodiments,” or “in one embodiment” in various places throughout this specification are not necessarily all referring to the same embodiment.

[0036] Some embodiments may be represented by functional box configurations and various processing steps. Some or all of these functional boxes may be implemented by various hardware and / or software configurations that perform specific functions. For example, the functional boxes of the present disclosure may be implemented by one or more processors or microprocessors, or by circuit configurations for performing the intended functions. Furthermore, for example, the functional boxes of the present disclosure may be implemented by various programming or scripting languages. The functional boxes may be implemented by algorithms that execute on one or more processors. Furthermore, the present disclosure may employ conventional techniques for electronic configuration, signal processing, and / or data processing. Terms such as modules and configurations may be used broadly and are not limited to mechanical and physical configurations.

[0037] Additionally, the connecting lines or connecting members between components depicted in the drawings are merely exemplary representations of functional connections and / or physical or circuit connections. In an actual device, connections between components may be represented by various functional connections, physical connections, or circuit connections that may be replaced or added.

[0038] Additionally, the description 'at least one of A, B, and C' means that it can be any one of 'A', 'B', 'C', 'A and B', 'A and C', 'B and C', and 'A, B, and C'.

[0039] The disclosed embodiment relates to a display device and an operating method thereof. Specifically, the disclosed embodiment relates to an operating method for a display device that outputs a multi-view screen to process a video call request when a video call is requested between multiple electronic devices, and to a display device that performs the same.

[0040] In the disclosed embodiment, an electronic device capable of providing a video call service may be any electronic device that includes a display for displaying a video call screen and can connect to a wired or wireless communication network to communicate with another electronic device located remotely. Furthermore, the electronic device may be a computing device capable of providing a video call service, including a display, and may exist in a wide variety of forms. Examples of electronic devices include, but are not limited to, wearable devices, smartphones, personal digital assistants (PDAs), media players, tablet PCs, laptop computers, media players, TVs, digital TVs, smart TVs, digital signage, and digital signs. Hereinafter, for the sake of convenience of explanation, a device capable of providing a video call service according to the disclosed embodiment will be referred to as an "electronic device."

[0041] Additionally, in the descriptions below, the electronic device may be an electronic device capable of conducting a video call by displaying a video call screen on a display. That is, the electronic device may be a wearable device, a smartphone, a personal digital assistant (PDA), a media player, a tablet PC, a laptop computer, a media player, a TV, a digital TV, a smart TV, a digital signage, a digital signage, etc.

[0042] Hereinafter, with reference to the attached drawings, a display device and its operation method according to the disclosed embodiment will be described in detail.

[0043] FIG. 1 illustrates an example of a system that provides a video call service according to one embodiment.

[0044] Referring to FIG. 1, the system may include an electronic device 100, a server device 200, and one or more user devices, for example, a second electronic device 10, a third electronic device 20, and a fourth electronic device 30. The electronic device 100 may be referred to as a first electronic device in relation to the second electronic device 10, the third electronic device 20, and the fourth electronic device 30.

[0045] Electronic device 100 is a device capable of providing a video call service. In the disclosed embodiment, a video call refers to a call in which multiple users located remotely from each other communicate while viewing each other's faces on screens via multiple electronic devices. Furthermore, the term "video call" as referred to in the disclosed embodiment can be applied to all fields in which users located remotely communicate while receiving images of each other, such as video conferencing, non-face-to-face communication, and non-face-to-face education, and can refer to any case in which multiple users located remotely communicate while viewing each other's images. Here, the images included in the video call screen do not necessarily have to be the faces of the users making the video call, but can also be images representing the environment in which the users making the video call are located, or images provided by the users. In general, a video call can be conducted between two different electronic devices. Alternatively, three or more users located remotely from each other can conduct a video call through three or more electronic devices.

[0046] A video call can be performed between a user's electronic device 100, which is an electronic device located at a remote location, and other users' electronic devices, i.e., a second electronic device 10, a third electronic device 20, and a fourth electronic device 30. In addition, the video call can be performed by transmitting and receiving data required for the video call using a communication network such as 3G, 4G, or 5G. Alternatively, the video call can be performed through a communication service of a telecommunications company. In addition, the video call can be performed through various applications provided through a server of a service provider, such as a video call application, a non-face-to-face communication application, a video education application, a video conferencing application, etc.

[0047] Electronic device 100 may execute a video call service application to provide a video call service and may also provide a multi-view screen. A multi-view screen may refer to a screen in which a screen output on a display included in the electronic device is divided into multiple sub-screens, and each of the multiple sub-screens outputs a corresponding video.

[0048] Referring to FIG. 1, the electronic device 100 can output a multi-view screen 15 including four partial screens (11, 12, 13, 14). Specifically, the electronic device 100 can display the multi-view screen 15 that can be visually recognized by a user. For example, the multi-view screen 15 can output different images to each of the four partial screens (11, 12, 13, 14). In the examples of the attached drawings and detailed description, the multiple images can be four different first images, second images, third images, and fourth images. The electronic device 100 according to the disclosed embodiment can split the entire screen corresponding to the entire display area into a plurality of partial screens through screen splitting, and output independent images to each of the plurality of partial screens.

[0049] Electronic device 100 can provide a video call service using a video call service application. If the electronic device 100 wishes to provide a video call service, it can connect to a server device 200 using the video call service application. For example, if four users, a first user, a second user, a third user, and a fourth user, participate in the video call service, the device corresponding to each user can obtain each user's video and transmit the obtained user's video to the server device 200. For example, an electronic device 100 (which may be referred to as a first electronic device based on its relationship with a second, third, and fourth electronic device) may obtain an image of a first user (the user of the electronic device 100 may be referred to as a first user based on its relationship with a second, third, and fourth user) and transmit the image to a server device 200, a second electronic device 200 may obtain an image of the second user and transmit the image to the server device 200, a third electronic device 300 may obtain an image of the third user and transmit the image to the server device 200, and a fourth electronic device 400 may obtain an image of the fourth user and transmit the image to the server device 200. Then, the server device 200 may transmit images of other peer users to each electronic device. A peer user refers to a user of an electronic device other than the user of the corresponding electronic device based on each electronic device. For example, a second user, a third user, and a fourth user based on the first user of the electronic device 100 (the first electronic device) may be peer users. Accordingly, the server device 200 can transmit the images of the second to fourth users to the electronic device 100, transmit the images of the first, third, and fourth users to the second electronic device 200, transmit the images of the first, second, and fourth users to the third electronic device 300, and transmit the images of the first to third users to the fourth electronic device 400.Among the participants using the video call service from the server device 200, each electronic device that receives a video of a peer user can display the video of the user of each electronic device and the video of the peer user. For example, the electronic device 100 can display the video of the first user acquired by the electronic device 100 using its camera and the videos of the second to fourth users, which are the videos of the peer users received from the server device 200, on its display. In this way, the electronic device 100 can provide a multi-party video call service including the first to fourth users.

[0050] The method in which the electronic device 100 displays images of the first to fourth users is specifically as follows. The electronic device 100 can decode a first image signal corresponding to the image of the first user acquired by its camera to generate a first partial screen 11, decode a second image signal corresponding to the image of the second user to generate a second partial screen 12, decode a third image signal corresponding to the image of the third user to generate a third partial screen 13, and decode a fourth image signal corresponding to the image of the fourth user to generate a fourth partial screen 14. In addition, the electronic device 100 can output a multi-view screen 15 including the first partial screen 11, the second partial screen 12, the third partial screen 13, and the fourth partial screen 14. At this time, in order to generate each partial screen, a decoder capable of decoding each image signal may be required as many as the number of image signals. For example, as illustrated in FIG. 1, when generating four partial screens based on four video signals, the electronic device 100 may require four decoders capable of decoding each of the four video signals. Therefore, in general, when the electronic device 100 receives and decodes video signals for participants participating in a video call service, it may require as many decoders as the number of participants. However, since the number of participants participating in a video call service may change dynamically, the electronic device 100 including a fixed number of decoders needs to dynamically process video signals corresponding to the participants using the fixed number of decoders. For example, when there are a total of five participants in the video call service, the electronic device 100 equipped with four decoders will need to process a total of five video signals using the four decoders. For example, when there are a total of four participants in the video call service, the electronic device 100 equipped with two decoders will need to process a total of four video signals using the two decoders.The embodiments disclosed in this disclosure aim to provide a system capable of responding to a case where a video signal corresponding to a video call service participant exceeds the capacity (capability) that can be processed by a decoder of an electronic device.

[0051] According to one embodiment, when the capacity of user images participating in a video call service exceeds the capacity that can be processed by a decoder of the electronic device 100, the electronic device 100 can process user images having a capacity greater than the decoder processing capacity of the electronic device 100 by degrading the quality characteristics of some user images among a plurality of user images and mixing and decoding some user images with degraded quality characteristics with a single decoder.

[0052] According to one embodiment, when the number of user videos participating in a video call service exceeds the number of decoders of the electronic device 100, the electronic device 100 can process a number of user videos greater than the number of decoders of the electronic device 100 by lowering the quality characteristics of some of the user videos among the plurality of user videos and decoding some of the user videos with the lowered quality characteristics with a single decoder. In this way, by processing videos of some of the users participating in the video call service with the lowered quality characteristics, videos for all users participating in the video call service can be processed and output without missing any user videos.

[0053] According to one embodiment, the electronic device 100 can output a user image of a speaking user without quality degradation by selecting a user image corresponding to a non-speaking user as a user image that will degrade quality characteristics.

[0054] According to one embodiment, the electronic device 100 can identify a user image corresponding to a user who does not speak based on at least one of information indicating whether a microphone is on / off, information indicating whether the amount of voice data exceeds a threshold, and information indicating whether the user speaks.

[0055] According to one embodiment, the electronic device 100 may select a predetermined number of images corresponding to other users or images corresponding to users of the electronic device 100 received from the server device as some user images that will degrade quality characteristics.

[0056] According to one embodiment, the electronic device 100 may obtain one or more user images with degraded quality characteristics from the server device by transmitting a signal indicating a request to degrade quality characteristics of one or more user images to the server device, and / or obtain a predetermined number of user images with degraded quality characteristics by degrading quality characteristics of images corresponding to users of the electronic device.

[0057] According to one embodiment, the electronic device 100 may acquire an image with a reduced resolution or frame rate of the user image to obtain an image with degraded quality characteristics.

[0058] According to one embodiment, the electronic device 100 may obtain one or more user images with reduced resolution from the server device by transmitting a signal indicating a request to reduce the resolution of one or more user images to the server device and / or obtain a user image with reduced resolution by reducing the resolution of an image corresponding to a user of the electronic device.

[0059] According to one embodiment, the electronic device 100 may obtain one or more user images with a reduced frame rate from the server device by transmitting a signal indicating a request to lower the frame rate of one or more user images to the server device and / or obtain a user image with a reduced frame rate by lowering the frame rate of an image corresponding to a user of the electronic device.

[0060] According to one embodiment, the electronic device 100 can decode multiple images with degraded quality characteristics by time-slicing them into a single decoder.

[0061] According to one embodiment, the electronic device 100 can obtain a partial screen corresponding to each user image by processing one or more user images with quality characteristics that are not degraded with a corresponding decoder, and can also obtain a partial screen corresponding to multiple user images by mixing multiple user images with degraded quality characteristics and processing them with a single decoder. In addition, the electronic device 100 can obtain a full screen including a partial screen corresponding to each user image obtained in this manner and a partial screen corresponding to multiple user images.

[0062] According to one embodiment, the electronic device 100 can configure the entire screen such that the size of a partial screen corresponding to a plurality of user images in the entire screen is larger than the size of a partial screen corresponding to each user image.

[0063] An example of mixing and processing some of the user images received from the server device 200 in the electronic device 100 when the amount of user images to be decoded in a video call exceeds the image processing capability of the electronic device 100 is described with reference to FIGS. 4 to 10.

[0064] According to one embodiment, the server device 200 may receive information about the image processing capabilities of the electronic device 100 from the electronic device 100.

[0065] According to one embodiment, the server device 200 may determine whether the amount of user videos participating in a video call that needs to be decoded exceeds the video processing capability of the electronic device 100.

[0066] According to one embodiment, when the server device 200 determines that the amount of user videos participating in a video call to be decoded exceeds the video processing capability of the electronic device 100, the server device 200 may mix some of the user videos to create a single video stream and provide the mixed video stream to the electronic device 100.

[0067] According to one embodiment, the server device 200 may mix some user videos into a single video stream by lowering the resolution of some user videos or lowering the frame rate of some user videos.

[0068] An example of mixing user videos on the server device 200 side to create a single stream and transmitting it to the electronic device 100 when the amount of user videos to be decoded participating in a video call exceeds the video processing capability of the electronic device 100 is described in detail with reference to FIGS. 11 to 13.

[0069] Figure 2 illustrates an example of a system according to one embodiment.

[0070] Referring to FIG. 2, the system may include an electronic device 100 and a server device 200 providing a video call service.

[0071] The electronic device 100 can output or display an image received from the server device 200. The electronic device 100 may include various types of electronic devices capable of receiving and outputting images, such as network TVs, smart TVs, internet TVs, web TVs, IPTVs, and PCs. The electronic device 100 may be referred to as a display device in terms of receiving and displaying images, and may also be referred to as an image receiving device, a sink device, a computing device, etc.

[0072] The electronic device 100 is a device that can display images or data according to a user's request, and may include a communication interface 110, a display 120, a memory 130, and a processor 140.

[0073] The communication interface 110 may include various communication circuits for communicating with at least one external device. Here, 'communication' may mean an operation of transmitting and / or receiving data, signals, requests, and / or commands.

[0074] The communication interface 110 can perform wired or wireless communication with at least one external device. The external device may be a server device 200.

[0075] For example, the communication interface 110 may include at least one of a communication module, a communication circuit, a communication device, an input / output port, and an input / output plug for performing wired or wireless communication with at least one external device.

[0076] For example, the communication interface 110 may include at least one wireless communication module, wireless communication circuit, or wireless communication device that performs wireless communication with at least one external device.

[0077] For example, the communication interface 110 may include a short-range communication module capable of receiving control commands from a remote controller located in close proximity, such as an input device, such as an IR (infrared) communication module. In this case, the communication interface 110 may receive a control signal from the remote control device.

[0078] As another example, the communication interface 110 may include at least one communication module that performs communication according to a wireless communication standard such as Bluetooth, Wi-Fi, BLE (Bluetooth Low Energy), NFC / RFID, Wi-Fi Direct, UWB, or ZIGBEE. Alternatively, the communication interface 110 may further include a communication module that performs communication with a server to support long-distance communication according to a long-distance communication standard. For example, the communication interface 110 may include a communication module that performs communication through a network for Internet communication. In addition, the communication interface 110 may include a communication module that performs communication through a communication network according to a communication standard such as 3G, 4G, 5G, and / or 6G.

[0079] As another example, the communication interface 110 may include at least one port for connecting to an external device via a wired cable in order to communicate with the external device via a wired connection. For example, the communication interface 110 may include at least one of an HDMI port (High-Definition Multimedia Interface port), a component jack, a PC port, and a USB port. Accordingly, the communication interface 110 may communicate with a wired external device through at least one port. Here, the port may refer to a physical device configuration into which a cable, communication line, or plug can be connected or inserted.

[0080] As described above, the communication interface 110 may include at least one support element for supporting communication between the electronic device 100 and an external device. Here, the support element may include the communication module, the communication circuit, the communication device, the port (for input / output of data), the cable port (for input / output of data), the plug (for input / output of data), etc., as described above. For example, the at least one support element included in the communication interface 110 may include an Ethernet communication module, a Wi-Fi communication module, a Bluetooth communication module, an IR communication module, a USB port, a tuner (or a broadcast receiver), an HDMI port, a DP (display port), a DVI (digital visual interface) port, etc.

[0081] The display 120 can output an image to the screen under the control of the processor 140. For example, the processor 140 can control the display 120 so that an intended image is output to the display 120.

[0082] The display 120 outputs a video onto a screen. For example, the display 120 may output an image corresponding to the video data via an internally included display panel so that a user can visually recognize the video data. Specifically, the video data forming the content may include a plurality of frame images, and the display 120 may play the video content by sequentially displaying the plurality of frame images under the control of the processor 140. For example, the display 120 may output a content image corresponding to the video content onto the screen under the control of the processor 140.

[0083] The display 120 is depicted as being arranged within the electronic device 100, but is not limited thereto. The display 120 may also be arranged externally to the electronic device 100 and connected to the electronic device 100 via wired or wireless communication.

[0084] Memory 130 may store at least one instruction, data, information, and / or application. For example, memory 130 may store at least one instruction to be executed by processor 140. For example, memory 130 may store at least one program to be executed by processor 140. For example, memory 130 may store an application for providing a certain service.

[0085] The memory 130 may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a RAM (Random Access Memory), a SRAM (Static Random Access Memory), a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk.

[0086] The processor 140 may control the execution of at least one instruction to perform an intended operation. Here, the at least one instruction may be stored in an internal memory included in the processor 140 or in a memory 130 included in the electronic device 100 separately from the processor 140.

[0087] The processor 140 can control at least one component included in the electronic device 100 to perform an intended operation by executing at least one instruction. Accordingly, even if an example is given of the processor 140 performing predetermined operations, this may mean that the processor 140 controls at least one component included in the electronic device 100 to perform the predetermined operations.

[0088] Processor 140 may include various processing circuits and / or multiple processors. For example, the term "processor" as used herein, including in the claims, may include various processing circuits, including at least one processor. At least one processor, one or more processors may be configured to perform various functions described herein, individually and / or collectively, in a distributed fashion. As used herein, "processor," "at least one processor," and "one or more processors" may be configured to perform multiple functions. However, these terms encompass, without limitation, situations where one processor performs some of the functions and other processor(s) perform other parts of the functions, and situations where a single processor may perform all of the functions. Furthermore, the at least one processor may include a combination of processors that perform various of the disclosed functions in a distributed manner. At least one processor may execute program instructions to achieve or perform various functions.

[0089] For example, the processor 140 may include a RAM that stores signals or data input from the outside of the electronic device 100, a storage area corresponding to various tasks performed in the electronic device 100, a control program for controlling the electronic device 100, an application for providing a predetermined function or service, and / or a ROM that stores a plurality of instructions, and at least one processor. The processor 140 may include a graphics processor (Graphics Processing Unit) for graphics processing corresponding to video. The processor 140 may be implemented as a SoC (System On Chip) that integrates a core and a GPU. In addition, the processor 140 may include multiple cores more than a single core. For example, the processor 140 may include a dual core, a triple core, a quad core, a hexa core, an octa core, a deca core, a dodeca core, a hexa decimal core, etc.

[0090] In an embodiment of the present disclosure, the processor 140 may store one or more instructions in an internally provided memory, and may control the operations of the electronic device 100 to be performed by executing one or more instructions stored in the internally provided memory. That is, the processor 140 may perform a predetermined operation by executing at least one instruction or program stored in an internal memory or memory 130 provided within the processor 140.

[0091] According to one embodiment, the processor 140 may obtain a plurality of user images including images corresponding to one or more users received from a server device for a video call and images corresponding to users of the electronic device by executing one or more instructions stored in the memory 130.

[0092] According to one embodiment, the processor 140 may determine whether the capability for image processing of the electronic device corresponds to the plurality of user images to be decoded for the video call by executing one or more instructions stored in the memory 130.

[0093] According to one embodiment, the processor 140 may obtain user images with degraded quality characteristics for a predetermined number of user images among the plurality of user images when it is determined that the capability for image processing of the electronic device does not correspond to the plurality of user images to be decoded for the video call by executing one or more instructions stored in the memory 130.

[0094] According to one embodiment, the processor 140 may be configured to process a predetermined number of user images with degraded quality characteristics by executing one or more instructions stored in the memory 130, thereby obtaining partial screens corresponding to the predetermined number of user images, and outputting a multi-view screen including the partial screens.

[0095] According to one embodiment, the processor 140 may, by executing one or more instructions stored in the memory 130, identify whether the number of the plurality of user images to be decoded for a video call exceeds the number of decoders of the electronic device, and, upon identifying that the number of the plurality of user images exceeds the number of decoders of the electronic device, determine that the capability for image processing of the electronic device does not correspond to the plurality of user images to be decoded for the video call.

[0096] According to one embodiment, the processor 140 may select a user image corresponding to a non-speaking user as the user image for which the quality characteristic is to be degraded by executing one or more instructions stored in the memory 130.

[0097] According to one embodiment, the processor 140 may identify a user image corresponding to a user who does not speak based on at least one of information indicating whether a microphone is on / off, information indicating whether the amount of voice data exceeds a threshold, and information indicating whether the speaker is speaking by executing one or more instructions stored in the memory 130.

[0098] According to one embodiment, the processor 140 may acquire one or more user images with degraded quality characteristics from the server device by transmitting a signal indicating a request to degrade quality characteristics of one or more user images by executing one or more instructions stored in the memory 130, and / or may acquire a predetermined number of user images with degraded quality characteristics by degrading quality characteristics of images corresponding to users of the electronic device.

[0099] According to one embodiment, the processor 140 may acquire one or more user images with reduced resolution from the server device by transmitting a signal indicating a request to reduce the resolution of the one or more user images by executing one or more instructions stored in the memory 130, and / or may acquire a predetermined number of user images with reduced quality characteristics by reducing the resolution of images corresponding to users of the electronic device.

[0100] According to one embodiment, the processor 140 may acquire one or more user images with a reduced frame rate from the server device by transmitting a signal indicating a request to reduce the frame rate of the one or more user images to the server device by executing one or more instructions stored in the memory 130, and / or may acquire a predetermined number of user images with reduced quality characteristics by reducing the frame rate of images corresponding to users of the electronic device.

[0101] According to one embodiment, the processor 140 may decode a predetermined number of user images with degraded quality characteristics by time-slicing them into a single decoder by executing one or more instructions stored in the memory 130.

[0102] According to one embodiment, the processor 140 may obtain one or more partial screens including each partial screen corresponding to each user image of the one or more user images by executing one or more instructions stored in the memory 130, thereby processing one or more user images whose quality characteristics are not degraded with a corresponding decoder, and obtain the multi-view screen including the partial screens corresponding to the predetermined number of user images and the one or more partial screens corresponding to each user image.

[0103] The electronic device 100 may be any type of device that performs a function, including a processor and memory. The electronic device 100 may be a fixed or portable device. For example, the electronic device 100 may be a device that has a display and can display image content, video content, game content, graphic content, etc. The electronic device 100 may output or display images or content received from a server device 200. The electronic device 100 may include various types of electronic devices that can receive and output content, such as televisions such as network TVs, smart TVs, Internet TVs, web TVs, and IPTVs; computers such as desktops, laptops, and tablets; smartphones, cellular phones, game players, music players, video players, medical equipment, home appliances, and various other smart devices. The electronic device 100 may be referred to as a display device in that it receives and displays content, and may also be referred to as a content receiving device, a sink device, a display device, a computing device, etc.

[0104] The block diagram of the electronic device 100 illustrated in FIG. 2 is a block diagram for one embodiment. Each component of the block diagram may be integrated, added, or omitted depending on the specifications of the electronic device 100 actually implemented. For example, two or more components may be combined into one component, or one component may be subdivided into two or more components, as needed. In addition, the functions performed by each block are for the purpose of explaining embodiments, and the specific operations or devices thereof do not limit the scope of the present invention.

[0105] Now let's describe server device 200.

[0106] The server device 200 can provide content such as video or audio to the electronic device 100. The server device 200 can include various types of electronic devices that can provide content to the electronic device 100. The server device 200 can be referred to as a source device in terms of providing content, and can also be referred to as a host device, a content providing device, an electronic device, a storage device, a computing device, a server computer, etc.

[0107] The server device 200 may include a communication interface 210, memory 220, and a processor 230. However, the server device 200 may be implemented with more components than those illustrated, and is not limited to the examples described above. For example, the server device 200 may include a separate image processing unit for image processing of application images executed on the server device 200.

[0108] The communication interface 210 may include various communication circuits included in one or more modules that enable wireless communication between the server device 200 and the electronic device 100. In one embodiment, the communication interface 210 may perform communication with the electronic device 100 according to an Internet protocol. In one embodiment, the communication interface 210 may perform communication with the input device 10 according to an Internet protocol.

[0109] The memory 220 can store a program for processing and controlling the processor 230, and can store data input to or output from the server device 200.

[0110] The memory 220 may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a RAM (Random Access Memory), a SRAM (Static Random Access Memory), a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk.

[0111] Processor 230 controls the overall operation of server device 200. For example, processor 230 may perform functions of server device 200 described in the present disclosure by executing one or more instructions stored in memory 220.

[0112] Processor 230 may include various processing circuits and / or multiple processors. For example, the term "processor" as used herein, including in the claims, may include various processing circuits, including at least one processor. At least one processor, one or more processors may be configured to perform various functions described herein, individually and / or collectively, in a distributed fashion. As used herein, "processor," "at least one processor," and "one or more processors" may be configured to perform multiple functions. However, these terms encompass, without limitation, situations where one processor performs some of the functions and other processor(s) perform other parts of the functions, and situations where a single processor may perform all of the functions. Furthermore, the at least one processor may include a combination of processors that perform various of the disclosed functions in a distributed manner. At least one processor may execute program instructions to achieve or perform various functions.

[0113] In an embodiment of the present disclosure, the processor 230 can store one or more instructions in an internally provided memory, and control the execution of the one or more instructions stored in the internally provided memory so that the aforementioned operations are performed. That is, the processor 230 can perform a predetermined operation by executing at least one instruction or program stored in an internal memory provided within the processor 230 or in the memory 220.

[0114] According to one embodiment, the processor 230 may receive a request from the electronic device 100 to lower the quality characteristics of a predetermined number of user images by executing one or more instructions stored in the memory 220, process the requested user images to lower the quality characteristics, and transmit the user images with lowered quality characteristics to the electronic device 100. At this time, the quality characteristics may include a resolution indicating the sharpness of the image expressed in pixels, a frame rate (FPS) indicating the number of frames displayed per second (which may be referred to as a frame rate), a bit rate indicating the amount of data transmitted per second, etc. Accordingly, the processor 230 may lower the resolution of a predetermined number of user images or lower the frame rate of a predetermined number of user images by executing one or more instructions stored in the memory 220, and transmit the processed user images to the electronic device 100.

[0115] According to one embodiment, the processor 230 may receive information about an image processing capability indicating an image processing capability of the electronic device 100 from an electronic device 100 participating in a video call by executing one or more instructions stored in the memory 220. The processor 230 may receive the information about the image processing capability from the electronic device 100 during a communication connection process for the video call or may receive the information about the image processing capability from the electronic device 100 during a video call service. That is, when the electronic device 100 receives user images corresponding to one or more users participating in the video call from the server device 200 and displays a multi-view screen, and determines that the received user images exceed its capabilities, the electronic device 100 may transmit information about its image processing capability to the server device 200.

[0116] According to one embodiment, the processor 230 may transmit streams corresponding to each of the user images participating in the video call to the electronic device 100 by executing one or more instructions stored in the memory 220.

[0117] According to one embodiment, the processor 230 may determine whether images of users participating in a video call exceed the image processing capabilities of the electronic device 100 by executing one or more instructions stored in the memory 220.

[0118] According to one embodiment, the processor 230 may transmit a video stream corresponding to each user video to the electronic device 100 by executing one or more instructions stored in the memory 220, if the videos of users participating in the video call do not exceed the video processing capability of the electronic device 100.

[0119] According to one embodiment, the processor 230 may, by executing one or more instructions stored in the memory 220, mix some of the multiple user images into one image stream and transmit it to the electronic device 100 when the images of the users participating in the video call exceed the image processing capability of the electronic device 100.

[0120] The block diagram of the server device 200 illustrated in FIG. 2 is a block diagram for one embodiment. Each component of the block diagram may be integrated, added, or omitted depending on the specifications of the server device 200 actually implemented. For example, two or more components may be combined into one component, or one component may be subdivided into two or more components, as needed. In addition, the functions performed by each block are for the purpose of explaining embodiments, and the specific operations or devices thereof do not limit the scope of the present invention.

[0121] FIG. 3 is an example of a block diagram of an electronic device according to one embodiment.

[0122] Referring to FIG. 3, the electronic device 100 may include, in addition to a communication interface 110, a display 120, a memory 130, and a processor 140, an image processing unit 150, an audio processing unit 160, an audio output unit 170, a receiving unit 180, and a detection unit 190.

[0123] The communication interface 110 may include various communication circuits included in one or more modules that enable wireless communication between the electronic device 100 and a wireless communication system or between the electronic device 100 and a network in which another electronic device is located. For example, the communication interface 110 may include a mobile communication module 111, a wireless Internet module 112, and a short-range communication module 113.

[0124] The mobile communication module 111 transmits and receives wireless signals with at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signals may include various types of data, such as voice call signals, video call signals, or text / multimedia message transmission and reception.

[0125] The wireless Internet module 112 refers to a module for wireless Internet access, which may be built into or external to the device. Wireless Internet technologies that may be used include WLAN (Wireless LAN) (WiFi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), and HSDPA (High Speed ​​Downlink Packet Access). Through the wireless Internet module 112, the electronic device 100 can establish a Wi-Fi P2P (Peer to Peer) connection with other devices.

[0126] Short-range communication module 113 refers to a module for short-range communication. Short-range communication technologies that can be used include Bluetooth, BLE (Bluetooth Low Energy), RFID (Radio Frequency Identification), IrDA (Infrared Data Association), UWB (Ultra Wideband), and ZigBee.

[0127] The display 120 can display a video signal received from the server device 200 on the screen.

[0128] Memory 130 can store programs related to the operation of the electronic device 100 and various data generated during the operation of the electronic device 100.

[0129] Memory 130 can store at least one instruction. Additionally, memory 140 can store at least one program executed by processor 150. Additionally, memory 150 can store an application for providing a specific service.

[0130] According to one embodiment, the memory 130 may include a video calling application 131 comprising one or more instructions executable by the processor 140 to provide the video calling service operations disclosed in the present disclosure.

[0131] Specifically, the memory 130 may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a RAM (Random Access Memory), a SRAM (Static Random Access Memory), a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk.

[0132] The processor 140 may include various processing circuits for controlling the overall operation of the electronic device 100. For example, the processor 140 may perform the functions of the electronic device 100 described in the present disclosure by executing one or more instructions stored in the memory 130.

[0133] In an embodiment of the present disclosure, the processor 140 can store one or more instructions in an internally provided memory, and control the execution of operations of the display device by executing one or more instructions stored in the internally provided memory. That is, the processor 140 can perform a predetermined operation by executing at least one instruction or program stored in the internal memory or memory 130 provided within the processor 140.

[0134] The processor 140 controls the overall operation of the electronic device 100 and the signal flow between internal components (not shown) of the electronic device 100, and performs the function of processing data. When there is a user input or a preset stored condition is satisfied, the processor 140 can execute the OS (Operating System) stored in the memory 130 and various applications.

[0135] According to one embodiment, the processor 140 may perform operations for providing a video call service of the electronic device 100 disclosed in the present disclosure by executing one or more instructions included in a video call application 131 stored in the memory 130.

[0136] Processor 140 may include a graphic processor (Graphic Processing Unit) for graphics processing corresponding to video. The graphic processor generates a screen including various objects such as icons, images, and text using a calculation unit and a rendering unit. The calculation unit uses user interactions detected by the detection unit to calculate attribute values ​​such as coordinate values, shape, size, and color for each object to be displayed according to the screen layout. The rendering unit generates a screen including various layouts of objects based on the attribute values ​​calculated by the calculation unit.

[0137] The image processing unit 150 can process an image signal received from the receiving unit 110 or the communication interface 110 under the control of the processor 140 and output it to the display 130. The image processing unit 150 can include various image processing circuits.

[0138] The audio processing unit 160 may, under the control of the processor 140, convert an audio signal received from the receiving unit 110 or the communication interface 110 into an analog audio signal and output it to the audio output unit 180. The audio processing unit 160 may include various audio processing circuits.

[0139] The audio output unit 170 can output audio (e.g., voice, sound) input through the communication interface 110 or the receiver 180. In addition, the audio output unit 170 can output audio stored in the memory 130 under the control of the processor 140. The audio output unit 170 can include at least one of a speaker, a headphone output terminal, or an S / PDIF (Sony / Philips Digital Interface:) output terminal, or a combination thereof.

[0140] The receiving unit 180 can receive video (e.g., moving images, etc.), audio (e.g., voice, music, etc.), and additional information (e.g., EPG, etc.) from the outside of the electronic device 100 under the control of the processor 140. The receiving unit 180 may include one of an HDMI port (High-Definition Multimedia Interface port, 181), a component jack (component jack, 182), a PC port (PC port, 183), and a USB port (USB port, 184), or a combination of one or more thereof. In addition to the HDMI port, the receiving unit 180 may further include a DisplayPort (DP), Thunderbolt, and MHL (Mobile High-Definition Link).

[0141] The detection unit 190 detects the user's voice, the user's image, or the user's interaction, and may include a microphone 191, a camera 192, and an optical receiver 193.

[0142] The microphone 191 receives the user's uttered voice. The microphone 191 can convert the received voice into an electrical signal and output it to the processor 140. The user's voice may include, for example, a voice corresponding to a menu or function of the electronic device 100. According to one embodiment, the microphone 191 may acquire the user's voice of the electronic device 100 for a video call service under the control of the processor 140 and provide the acquired user's voice to the processor 140. The microphone 191 may be installed inside the electronic device 100 as illustrated in FIG. 3, or may be installed externally and connected to the processor 140 for communication.

[0143] The camera 192 can receive an image (e.g., consecutive frames) corresponding to a user's motion including a gesture within the camera recognition range. The processor 140 can use the recognition result of the received motion to select a menu displayed on the electronic device 100 or perform a control corresponding to the motion recognition result. According to one embodiment, the camera 192 can capture an image of a user of the electronic device 100 for a video call service under the control of the processor 140 and provide the captured user image to the processor 140. The camera 192 may be installed inside the electronic device 100 as illustrated in FIG. 3, or may be installed externally and connected to the electronic device 100 via communication. The optical receiver 193 receives an optical signal (including a control signal) received from an external control device. The optical receiver 193 can receive an optical signal corresponding to a user input (e.g., a touch, a press, a touch gesture, a voice, or a motion) from the control device. A control signal can be extracted from the received optical signal under the control of the processor 140.

[0144] FIG. 4 illustrates an example of an image processing unit of an electronic device according to one embodiment.

[0145] Referring to FIG. 4, the image processing unit 150 can perform image processing on video data. The image processing unit 150 can include a demultiplexer 410, a multi-decoder 420, a mixer 430, and a scaler 440.

[0146] The electronic device 100 can receive one or more video signals corresponding to one or more users from the server device 200 via the communication interface 110. For example, the server device 200 can provide a second video signal corresponding to a second user, a third video signal corresponding to a third user, and a fourth video signal corresponding to a fourth user.

[0147] The communication interface 110 can receive a second video signal corresponding to a second user, a third video signal corresponding to a third user, and a fourth video signal corresponding to a fourth user from the server device 200 via a communication network. The video signals received by the communication interface 110 can be transmitted to the video processing unit 150 and decoded under the control of the processor 140.

[0148] The image processing unit 150 may receive an image signal corresponding to one or more user images from the server device 200 and / or may receive an image signal corresponding to an image of a user of the electronic device 100 obtained from the camera 192.

[0149] The image processing unit 150 decodes and processes the input image signal, and can scale the decoded image signal so that it is sized into an image frame to be output from the display 120. Accordingly, each of the plurality of image signals input to the image processing unit 150 can be output as a plurality of images displayed on a plurality of partial screens.

[0150] According to one embodiment, the image processing unit 150 may include a demultiplexer 410, a multi-decoder 420, a mixer 430, and a scaler 440. The detailed configuration and arrangement of the demultiplexer 410, the multi-decoder 420, the mixer 430, and the scaler 440 included in the image processing unit 150 may change depending on product specifications, product grade, etc.

[0151] The video signal received by the electronic device 100 may be received in the form of a stream, multiplexed, and compressed according to a predetermined codec (or standard). Accordingly, the video processing unit 150 may receive a video signal in the form of a stream generated by multiplexing and compressing, demultiplex and decompress the input video signal, and output the image data in the form of RGB or YUV that can be displayed on the display 120.

[0152] The demultiplexer 410 can receive one or more video signals and demultiplex one or more video signals, respectively. Specifically, the demultiplexer 140 can include a plurality of demuxers connected in parallel, for example, a first demuxer 411, a second demuxer 412, and a third demuxer 413. Here, the demuxers can demultiplex one video signal from a multiplexed video signal. For example, the first demuxer 411 can demultiplex a first video signal from a multiplexed video signal, the second demuxer 412 can demultiplex a second video signal from the multiplexed video signal, and the third demuxer 413 can demultiplex a third video signal from a multiplexed video signal.

[0153] The multi-decoder 420 may include multiple decoders. Specifically, the multi-decoder 420 may include multiple decoders having a predetermined resolution. Here, the "decoder having a predetermined resolution" may be a decoder capable of decoding and processing a video signal having a predetermined resolution. In other words, the "decoder having a predetermined resolution" may refer to a decoder that supports processing of a video signal having a predetermined resolution. Each of the multiple decoders may have the same or different resolutions.

[0154] A video can be viewed as a collection of still images sequentially arranged in time, and each still image is called a 'frame'. The number of these frames displayed per second, or the speed at which the frames are displayed, is called the frames per second or frame rate, and is expressed in frames per second (fps). 2K video, also known as FHD (full HD), refers to a video with pixels / frame of 1920*1080 and fps of 30. Here, K means Kilo, or 1000, and generally indicates the number of pixels in the horizontal resolution. UHD (Ultra High Definition) video includes 4K video, which has a resolution about 4 times higher than 2K video, and 8K video, which has a resolution about 16 times higher than 2K video. UHD 4K video refers to a video with 3840*2160 pixels per screen and 60 fps, and UHD 8K video refers to a video with 7680*4320 pixels per screen and 60 fps.

[0155] For example, referring to FIG. 4, the multi-decoder 420 may include a first decoder 421, a second decoder 422, and a third decoder 423.

[0156] The number and specifications of decoders included in the multi-decoder 420 may vary depending on the product specifications, product grade, release year, etc. of the electronic device 100, and may exist in various forms. In addition, the specifications of the decoder may vary depending on the bitrates that can be processed, the resolutions that can be supported, the codecs that can be supported (e.g., HEVC, H.264, MEPG, etc.), etc. Each decoder included in the multi-decoder 420 is designed to be able to decompress different compressions for at least one codec, and may also support decompression for multiple codecs.

[0157] For example, the multi-decoder 420 may include a 4K decoder capable of decoding a video signal having a resolution of 4K or lower, a 2K decoder capable of decoding a video signal having a resolution of 2K or lower than FHD resolution, and a software decoder that performs decoding using software and is designed to perform decoding corresponding to various codecs. In contrast to the software decoder, the 4K decoder or the 2K decoder may be referred to as a hardware decoder. Each hardware decoder may decode two or more video signals using time division. For example, one 4K decoder may be used to decode four video signals corresponding to 2K video. Each decoder of the multi-decoder 420 may decode the received video signal to generate one partial screen. For example, referring to FIG. 4, a first decoder 421 can decode a first video signal to generate a first partial screen, a second decoder 422 can decode a second video signal to generate a second partial screen, and a third decoder 423 can decode a third video signal to generate a third partial screen.

[0158] The mixer 430 can receive one or more partial screens corresponding to one or more video signals decoded through a multi-decoder, and determine the positions of the received one or more partial screens to generate one full screen.

[0159] The scaler 440 can scale the entire screen provided by the mixer 430 to fit the image frame output from the display 120 and output it to the display 120.

[0160] Figure 5 illustrates an example of an image processing unit according to one embodiment.

[0161] Referring to FIG. 5, for example, let's say that the first decoder 421, the second decoder 422, and the third decoder 423 of the multi-decoder 420 are all decoders capable of decoding video signals having a resolution of UHD 4K or lower. For example, the first decoder 421 can decode a first user video having a UHD resolution received through a camera to generate a first partial screen corresponding to the first user. The second decoder 422 can decode a second user video having a UHD resolution received through a communication interface to generate a second partial screen corresponding to the second user. The third decoder 423 can decode a third user video having a UHD resolution received through a communication interface to generate a third partial screen corresponding to the third user.

[0162] The mixer 430 receives the first partial screen, the second partial screen, and the third partial screen generated in this manner, and sets and positions the first partial screen, the second partial screen, and the third partial screen on the entire screen to generate the entire screen, and the scaler 440 adjusts the size of the entire screen to fit an image frame to be output to the display 120, thereby outputting the multi-view screen 500. The first partial screen 510, the second partial screen 520, and the third partial screen 530 included in the multi-view screen 500 can have the same size in response to the same UHD resolution video.

[0163] In the disclosed embodiment, an application that performs a service, function, or operation that provides a multi-view screen may be referred to as a 'multi-view application'. Specifically, the multi-view application may refer to an application that divides a screen output from an electronic device 100 according to a predetermined layout and controls multiple user images to be output through each of the divided partial screens in order to provide a multi-view screen. The multi-view application may perform operations for providing a multi-view screen and control at least one of hardware resources and software resources of the electronic device 100 required for providing the multi-view screen. For example, hardware resources required for providing a multi-view screen may include multiple decoders for decoding multiple user images output to the multi-view screen, a memory for storing images to be output to the multi-view screen, and the like. In addition, software resources required for providing a multi-view screen may include a software decoder for software-based decoding of multiple user images output to the multi-view screen. In the disclosed embodiment, a multi-view application stored or included in the electronic device 100 can control the overall creation and output of the multi-view screen.

[0164] As described above, the electronic device 100 can provide a multi-view screen by controlling the generation of multiple partial screens by dividing the screen and managing the decoding operations of multiple decoders included internally. The layout of the multi-view screen can be determined by considering the number of partial screens, the aspect ratio supported by the display, the aspect ratio of the content output through each partial screen, the aspect ratio of the partial screen, the aspect ratio of the window (execution window) of the Internet portal transmitted through the Internet server, etc. In addition, the layout of the multi-view screen can be determined according to the user's settings or the electronic device 100's own settings. Referring to FIG. 5, when the number of split screens is three, that is, when the multi-view screen 500 includes three partial screens, the layout represents a case where the multi-view screen is evenly divided into three partial screens. However, the layout may divide the multi-view screen unequally. For example, the layout may include one main screen and two sub-screens.

[0165] As in the example illustrated in FIG. 5, when the number of decoders included in the multi-decoder of the electronic device 100 is sufficient to decode user images to be output by the electronic device 100, the electronic device 100 can output a multi-view screen including partial screens corresponding to multiple user images by having each decoder decode a corresponding image signal. However, when the number of decoders included in the multi-decoder of the electronic device 100 is not sufficient to decode image signals corresponding to user images to be output by the electronic device 100, the electronic device 100 can decode image signals corresponding to a greater number of user images than the number of decoders by having one or more decoders included in the multi-decoder 420 perform a decoding operation in a time-division manner. For example, in a situation where the multi-decoder 420 includes three UHD decoders as illustrated in FIG. 5, when it is necessary to decode videos having four or more UHD resolutions, the electronic device 100 can adjust the quality characteristics of the videos to enable decoding of multiple videos with a single decoder. For example, the electronic device 100 can decode multiple videos with lower resolutions or lower frame rates by acquiring them by lowering the resolutions or frame rates of some of the videos, thereby time-dividing them and decoding them with a single decoder. For example, a UHD decoder having UHD resolution can time-divide and decode up to four FHD videos. Therefore, when the electronic device 100 receives four UHD videos, two of the UHD videos can be decoded as they are with the UHD decoders. And the electronic device 100 can decode four images using three decoders in total by decoding two FHD images using one UHD decoder by converting the remaining two UHD images into FHD images and obtaining them. For example, a decoder capable of processing images with a frame rate of 60 fps can process 60 frames per second.Therefore, a 60fps decoder can decode two videos with a frame rate of 30fps by time-division. Therefore, when the electronic device 100 receives four 60fps videos, it decodes two of the 60fps videos as they are by the decoder, and converts the remaining two 60fps videos into 30fps videos and obtains them, thereby decoding two 30fps videos using one decoder, and decoding a total of four videos using three decoders.

[0166] FIG. 6 illustrates an example of a method of operating an electronic device according to one embodiment.

[0167] Referring to FIG. 6, in operation 610, the electronic device 100 may obtain multiple user images including images corresponding to one or more users received from a server for a video call and images corresponding to users of the electronic device.

[0168] According to one embodiment, the electronic device 100 may receive, through a communication interface, a video corresponding to one or more users participating in a video call from the server device 200 to provide a video call service. When receiving a video corresponding to one user from the server device 200, the electronic device 100 may provide a one-to-one video call service. When receiving videos corresponding to multiple users from the server device 200, the electronic device 100 may provide a one-to-many video call service or a video conference service.

[0169] According to one embodiment, the electronic device 100 may capture an image of a user of the electronic device 100 using a camera, thereby obtaining an image corresponding to the user of the electronic device, and transmit the obtained image corresponding to the user of the electronic device to the server device 200.

[0170] In operation 620, the electronic device 100 may determine whether the capability of the decoder provided by the electronic device is sufficient to decode multiple user videos for a video call. The capability of the decoder may indicate the capacity of the decoder to perform video processing. For example, the capability of the decoder may be indicated by the maximum resolution of the video that the decoder can process or the maximum frame rates of the video that the decoder can process. The electronic device 100 may include a multi-decoder including one or more decoders to decode multiple videos. When the multi-decoder includes multiple decoders, the decoding capacities of the multiple decoders may be different from each other. The decoding capacity of the decoder may indicate the maximum resolution of the video that the decoder can decode. For example, if a multi-decoder includes two decoders, both decoders may be decoders capable of decoding UHD video, or one decoder may be a decoder capable of decoding UHD video and the other decoder may be a decoder capable of decoding FHD video. The multiple user images that the electronic device 100 acquires for decoding may also all have the same resolution, or may have different resolutions.

[0171] Accordingly, the electronic device 100 can determine whether the total capabilities of one or more decoders included in the electronic device are sufficient to decode multiple user videos for a video call. For example, if the electronic device 100 includes two UHD decoders, it can be determined that the capabilities are sufficient to decode two UHD user videos. However, if the electronic device 100 includes two UHD decoders, it can be determined that the capabilities are not sufficient to decode three UHD videos.

[0172] According to one embodiment, if the resolution of an image that can be decoded by each decoder of one or more decoders provided in the electronic device matches the resolution of a user image to be decoded, the electronic device 100 can determine whether the number of decoders provided in the electronic device 100 is equal to or greater than the number of multiple user images to be decoded.

[0173] In operation 630, the electronic device 100 may select a predetermined number of user images from among the plurality of user images for which quality characteristics are to be degraded, as it is determined that the capabilities of the decoder provided by the electronic device are not sufficient to decode the plurality of user images for a video call.

[0174] According to one embodiment, if the electronic device 100 determines that the capabilities of a decoder provided by the electronic device 100 are not sufficient to decode multiple user images for a video call, the electronic device 100 may select a predetermined number of user images to decode some of the multiple user images by time-splitting them with one decoder. For example, if the electronic device 100 is provided with two UHD decoders and needs to decode three UHD user images, the electronic device 100 may select the predetermined number of user images for which quality characteristics will be degraded, since one UHD user image can be decoded with one UHD decoder.

[0175] In one embodiment, the quality characteristic may include the resolution or frame rate of the image. For example, if the electronic device 100 has two UHD decoders and needs to decode three UHD user images, the electronic device 100 can decode one UHD user image with one UHD decoder, and thus can select the remaining two UHD user images as the predetermined number of user images for which the resolution is to be reduced. For example, if the electronic device 100 has two UHD decoders and needs to decode three UHD user images, the electronic device 100 can decode one UHD user image with one UHD decoder, and thus can select the remaining two UHD user images as the predetermined number of user images for which the frame rate is to be reduced.

[0176] In one embodiment, the electronic device 100 may be able to randomly select a predetermined number of user images from among a plurality of user images for which quality characteristics, such as resolution or frame rate, will be degraded.

[0177] According to one embodiment, the electronic device 100 may select a predetermined number of user images for which quality characteristics, such as resolution or frame rate, are to be reduced based on whether the user speaks or the frequency of the user speaking. For example, the electronic device 100 may select an image of a user who is not speaking as the user image for which quality characteristics are to be reduced. For example, the electronic device 100 may select an image of a user who has a relatively low speaking frequency as the user image for which quality characteristics are to be reduced. Since an image of a user who does not speak may be relatively less important than an image of a user who speaks during a video call, it may be preferable to select an image of a user who has a low speaking frequency as the user image for which quality characteristics are to be reduced. The electronic device 100 may obtain information about whether the user speaks or the frequency of speaking from audio information received together with the user image from a server device.

[0178] In one embodiment, the electronic device 100 may select a predetermined number of user images from the camera acquired by the electronic device 100 of the user, which may degrade quality characteristics such as resolution or frame rate.

[0179] At operation 640, the electronic device 100 can acquire a predetermined number of user images with degraded quality characteristics.

[0180] According to one embodiment, the electronic device 100 may transmit a request to the server device 200 to lower the quality characteristics of a predetermined number of user images for which quality characteristics are to be lowered, and in response to the request, may receive a predetermined number of user images with lowered quality characteristics from the server device 200. For example, the electronic device 100 may transmit a request to the server device 200 to lower the quality characteristics of a first user image and a second user image among a plurality of user images received from the server device 200, and in response to the request, may receive the first user image and the second user image with lowered quality characteristics from the server device 200.

[0181] According to one embodiment, the electronic device 100 may perform an operation to lower the quality characteristics of a predetermined number of user images for which quality characteristics are to be lowered, thereby obtaining images with lowered quality characteristics.

[0182] According to one embodiment, if the user image of the electronic device 100 is obtained from a camera and the user image of the electronic device 100 is to have a quality characteristic degraded, the electronic device 100 may perform processing to degrade the quality of the user image of the electronic device 100 obtained from the camera.

[0183] In operation 650, the electronic device 100 can obtain one partial screen corresponding to the predetermined number of user images by processing the predetermined number of user images with degraded quality characteristics with one decoder.

[0184] According to one embodiment, the electronic device 100 can generate a partial screen corresponding to one user image using one decoder from an original user image with uncompromised quality characteristics.

[0185] According to one embodiment, the electronic device 100 can generate partial screens corresponding to multiple user images by decoding a predetermined number of user images with degraded quality characteristics in a time-division manner using a single decoder. The partial screens corresponding to multiple user images can include images corresponding to multiple users.

[0186] In operation 660, the electronic device 100 can output a multi-view screen including partial screens.

[0187] According to one embodiment, the electronic device 100 can output a multi-view screen including a partial screen corresponding to one user and partial screens corresponding to multiple users. For example, if the electronic device 100 has two 4K decoders and needs to decode three 4K user images, the electronic device 100 can obtain a first partial screen corresponding to the first user image by decoding the first user image at its original 4K resolution using the first 4K decoder, and can obtain a second partial screen including the second user image and the third user image by degrading the second user image and the third user image to a 2K resolution and time-division decoding them using the second 4K decoder. For example, if the electronic device 100 has two 4K decoders and needs to decode three 4K user videos, the electronic device 100 can obtain a first partial screen corresponding to the first user video by decoding the first user video at the original 60 fps using the first 4K decoder, and can obtain a second partial screen including the second user video and the third user video by time-division decoding the second user video and the third user video at a frame rate lowered to 30 fps using the second 4K decoder.

[0188] FIG. 7 is a flowchart illustrating an example of a method of operating an electronic device according to one embodiment.

[0189] Referring to FIG. 7, in operation 701, the electronic device 100 may receive a video call request from a user. The video call request may be a control signal requesting the initiation or execution of a video call service. The electronic device 100 may initiate a video call service between multiple users in response to the video call request. As described above, the video call may be performed through the video call function of the electronic device 100 itself, or an application for providing a video call service, etc. For example, an application for providing a video call service may be referred to as a video call application. The electronic device 100 may store the video call application. The video call application may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™). In the case of online distribution, the distribution may be made through a manufacturer's server, an application store's server, a relay server, etc.

[0190] A video call request may be received based on a user input entered through the user interface of the electronic device 100. For example, a user of the electronic device 100 may wish to make a video call with a friend while watching a broadcast screen. Then, the user may input a user input requesting a video call through the user interface of the electronic device 100. Then, the electronic device 100 may recognize that a video call request has been received based on the user input. Alternatively, the video call request may be received from an electronic device of a party with whom the user of the electronic device wishes to make a video call (e.g., 10, 20, 30 of FIG. 1). That is, the video call request may be received based on a request from at least one of the user of the electronic device 100 or the user of the other party's electronic device.

[0191] In operation 702, the electronic device 100 may transmit a video call request to the server device 200.

[0192] The server device 200 can communicate with the electronic device 100 via a wireless communication network. For example, the server device 200 may be a server that provides a video call or a video call service, or supports communication required for a video call service. For example, at least two devices among the electronic device 100 and another user's electronic device are connected via a mobile communication network such as 3G, 4G, or 5G, and a video call can be performed between the electronic device 100 and the other user's electronic device. In this case, the server device 200 may be a communication server that supports mobile communication according to a communication standard such as 3G, 4G, or 5G. The operations required for performing the aforementioned video call can be performed using the communication relay of the server device 200. Although the communication relay operation of the server device 200 is not illustrated in FIG. 7, the communication relay operation of the server device 200 may be included in transmitting and receiving data or signals between two different devices. Specifically, the server device 200 may be responsible for transmitting data or signals to support the execution (or switching execution) of a video call service. For example, the server device 200 may perform a relay operation to transmit a user video signal generated by the electronic device 100 to the other party's device. In addition, the server device 200 may transmit or forward various data and signals used to perform the video call service to at least one of another user electronic device and the electronic device 100.

[0193] In operation 703, the electronic device 100 can obtain an image of the user using a camera.

[0194] In operation 704, the electronic device 100 can transmit an image signal corresponding to the acquired user image to the server device 200.

[0195] In operation 705, the electronic device 100 may receive video signals corresponding to videos of other users participating in the video call service, i.e., one or more users, from the server device 200. For example, if the electronic device 100 receives a video signal corresponding to the video of one user, the electronic device 100 may provide a one-to-one video call service. For example, if the electronic device 100 receives video signals corresponding to videos of three users, the electronic device 100 may provide a video call service for a total of four participants, including the user of the electronic device 100. The video of the user of the electronic device 100 may be transmitted from the server device 200, so that the video of all users participating in the video call service may be received from the server device 200 at once. Alternatively, the video of the user of the electronic device 100 may be used directly in the electronic device 100 without going through the server device 200.

[0196] In operation 706, the electronic device 100 can determine whether the number of user videos requiring decoding exceeds the number of decoders in the electronic device. If the resolution of the user videos to be used for the video call service and the maximum resolution that the decoder provided by the electronic device 100 can decode are the same, the electronic device 100 can determine whether the number of user videos requiring simultaneous decoding exceeds the number of decoders in the electronic device. Through this determination, the electronic device 100 can determine whether the decoder capacity for decoding the user videos is sufficient to provide the video call service. For example, if the electronic device 100 includes three 4K decoders and needs to decode a total of three 4K user videos to provide the video call service, the electronic device 100 can determine that the number of user videos requiring decoding does not exceed the number of decoders in the electronic device. For example, if electronic device 100 includes three 4K decoders and needs to decode a total of four or more 4K user videos to provide a video call service, electronic device 100 may determine that the number of user videos requiring decoding exceeds the number of decoders in the electronic device.

[0197] If it is determined in operation 706 that the number of user images requiring decoding does not exceed the number of decoders in the electronic device, the process may proceed to operation 712.

[0198] In operation 712, the electronic device 100 may process each received user image with each decoder to generate multiple partial screens. Each partial screen of the multiple partial screens may correspond to one user image. For example, if the electronic device 100 includes three 4K decoders and acquires a total of three UHD user images, i.e., a first user image, a second user image, and a third user image, for providing a video call service, the electronic device 100 may decode the first user image using the first 4K decoder to generate a first partial screen corresponding to the first user, decode the second user image using the second 4K decoder to generate a second partial screen corresponding to the second user, and decode the third user image using the third 4K decoder to generate a third partial screen corresponding to the third user. In addition, in operation 713, the electronic device 100 may output a multi-view screen including multiple partial screens. For example, in the example described above, the electronic device 100 can output a multi-view screen including a first partial screen, a second partial screen, and a third partial screen.

[0199] If it is determined as a result of the judgment in operation 706 that the number of user images to be decoded exceeds the number of decoders in the electronic device, the process may proceed to operation 707.

[0200] In operation 707, the electronic device 100 may select a user video to degrade its quality characteristics from among multiple user videos to be decoded. Since the electronic device 100 does not have enough decoders to process each of the multiple user videos for a video call with a single decoder, some of the multiple user videos may be time-divisionally processed by a single decoder. To this end, the electronic device 100 may degrade the quality characteristics of the multiple user videos to be time-divisionally processed by a single decoder. The electronic device 100 may select multiple user videos to degrade their quality characteristics. For example, if an electronic device 100 includes three decoders capable of processing first resolution images and needs to decode a total of four first resolution user images to provide a video call service, the electronic device 100 can decode two first resolution images using one decoder each capable of processing first resolution images, and degrade the quality characteristics of the remaining two first resolution images, thereby enabling the two images to be decoded by one decoder. For example, if the four first resolution user images include a first user image, a second user image, a third user image, and a fourth user image, the first user image can be decoded using the first decoder, the second user image can be decoded using the second decoder, and the third user image and the fourth user image can be selected as images whose quality characteristics are to be degraded so that they can be processed by the third decoder.

[0201] In one embodiment, the quality characteristic of the image to be degraded may include the resolution. For example, if the electronic device 100 includes three 4K decoders and needs to decode a total of four 4K user images to provide a video call service, the electronic device 100 may decode two 4K user images as they are using the two 4K decoders, and may select the remaining two 4K images to be degraded to 2K in order to decode the remaining two 4K images using one 4K decoder. One 4K decoder can decode up to four 2K images. For example, if the electronic device 100 includes two 4K decoders and needs to decode a total of eight 4K user videos to provide a video call service, the electronic device 100 can select eight 4K videos to be downgraded to 2K in resolution in order to decode the eight videos with the two 4K decoders.

[0202] In one embodiment, the quality characteristic of the video to be degraded may include the frame rate. For example, if the electronic device 100 includes three 4K decoders and needs to decode a total of four UHD4K user videos to provide a video call service, the electronic device 100 may decode two 4K user videos as they are using the two 4K decoders, and select the two remaining 4K videos as the videos to be degraded in frame rate in order to decode the remaining two 4K videos with one 4K decoder. For example, if one 4K decoder is a decoder capable of processing 60 frames per second (60fps), the electronic device 100 may select two 4K videos as the videos to be degraded in quality characteristic to 30fps.

[0203] In one embodiment, the electronic device 100 may randomly select an image for which quality characteristics are to be degraded. That is, the electronic device 100 may randomly select an image for which quality characteristics are to be degraded from among multiple images, without considering the properties of the user image itself. The properties of the user image may include, for example, whether or not the user speaks or the frequency of speech.

[0204] According to one embodiment, the electronic device 100 may select an image for which quality characteristics are to be lowered by considering whether or not the user speaks or the frequency of speech. For example, the electronic device 100 may preferentially select an image for which quality characteristics are to be lowered by considering whether or not the user speaks. For example, the electronic device 100 may preferentially select an image for which quality characteristics are to be lowered by considering the frequency of speech of a user who speaks less frequently. In other words, among multiple users participating in a video call, the image for a user who speaks is more important, so the image for a user who speaks is output without degrading the quality characteristics as much as possible, and the image for a user who does not speak is output with degraded quality characteristics, thereby allowing the image for a less important user to be displayed in a smaller size or at a lower frame rate.

[0205] The electronic device 100 can determine whether or not the user speaks or the frequency of speaking based on a user image received from a server device 200 or a user image acquired using a camera of the electronic device 100.

[0206] In one embodiment, the electronic device 100 may determine whether the user is speaking or the frequency of speaking based on the amount of voice data. For example, the electronic device 100 may determine whether the user is speaking or the frequency of speaking based on whether the amount of voice data corresponding to the user's image exceeds a threshold. For example, if the amount of voice data corresponding to the user's image exceeds a threshold, the electronic device 100 may determine that the user is speaking or that the user's speaking frequency is high.

[0207] According to one embodiment, the electronic device 100 may determine whether or not a user is speaking or speaking frequently based on whether a microphone displayed in the user video is turned on or off. For example, users participating in a video call service may be able to set whether or not the microphone is turned on or off, and this on / off setting may be displayed in a portion of the user video. For example, the electronic device 100 may determine whether or not the microphone is turned on or off by analyzing the user video. The electronic device 100 may determine that a user video with the microphone turned on corresponds to a user who speaks or speaks frequently, and determine that a user video with the microphone turned off corresponds to a user who does not speak.

[0208] According to one embodiment, the electronic device 100 may determine whether or not the user has spoken or the frequency of speaking based on information received from the server device 200. For example, user devices participating in a video call service may each transmit information about whether or not the user of the corresponding electronic device has spoken to the server device 200 along with the user video, and the server device 200 may provide information about whether or not the user has spoken to other users' electronic devices along with the user video. For example, the electronic device 100 may determine whether or not the user has spoken based on information about whether or not other users have spoken, which is transmitted from the server device 200 along with other users' videos.

[0209] At operation 708, the electronic device 100 may transmit a request to the server device 200 to degrade the quality characteristics of the selected image.

[0210] According to one embodiment, the electronic device 100 may transmit a request to the server device 200 to lower the resolution of a selected image. For example, the electronic device 100 may transmit a request to the server device 200 to lower two UHD images from three UHD images transmitted from the server device 200 to FHD.

[0211] According to one embodiment, the electronic device 100 may transmit a request to the server device 200 to lower the frame rate of the selected images. For example, the electronic device 100 may transmit a request to the server device 200 to lower the frame rate of two UHD images among three UHD images received at 60 fps from the server device 200 to 30 fps.

[0212] In operation 709, the server device 200 may receive a request from the electronic device 100 and perform processing to lower the quality characteristics of the selected image. For example, upon receiving a request to lower the resolution of the selected image from the electronic device 100, the server device 200 may perform processing to downscale the selected image to lower the resolution of the selected image. For example, upon receiving a request to lower the frame rate of the selected image from the electronic device 100, the server device 200 may perform processing to downsample the selected image.

[0213] In operation 710, the server device 200 transmits a user image with degraded quality characteristics to the electronic device 100, and the electronic device 100 can receive it.

[0214] In operation 711, the electronic device 100 can process multiple user images with degraded quality characteristics using a single decoder to generate a single partial screen corresponding to the multiple user images.

[0215] According to one embodiment, the electronic device 100 can generate a partial screen including multiple user images by decoding multiple user images with degraded quality characteristics using a single decoder in a time-division manner. In operation 712, the electronic device 100 generates a partial screen including a single user image by decoding a single user image with undegraded quality characteristics using a single decoder, whereas in operation 711, the electronic device 100 generates a partial screen including multiple user images with degraded quality characteristics.

[0216] In operation 713, the electronic device 100 can generate and output a multi-view screen including multiple partial screens. Among the multiple partial screens forming the multi-view screen, some partial screens may be partial screens including a single user image, and some partial screens may be partial screens including multiple user images.

[0217] FIG. 8 is a reference diagram for explaining an example of a method of operating an electronic device according to one embodiment.

[0218] Referring to FIG. 8, according to an example, the image processing unit 150 may include three 4K decoders capable of decoding images with a resolution of 4K or lower. Accordingly, the image processing unit 150 may decode up to three images with a resolution of 4K.

[0219] The processor 140 can identify that the number of 4K user videos that need to be decoded to provide a video call service is greater than the number of decoders included in the video decoder 150. For example, the processor 140 can identify the number of users participating in the video call service at the beginning of the communication connection with the server device 200. Alternatively, the processor 140 can identify that a user participating in the video call service is added while the video call service is in progress after the communication connection with the server device 200. If it is identified through this identification that the number of 4K user videos that need to be decoded to provide a video call service is greater than the number of decoders included in the video decoder 150, the processor 140 can select multiple user videos for which quality characteristics are to be lowered in order to decode multiple user videos instead of one user video with one decoder, and transmit a request for lowering the quality characteristics of the selected multiple user videos to the server device 200 through the communication interface 110. Of course, a user image obtained through the camera 192 of the electronic device 100 can also be selected as a user image that degrades quality characteristics. For convenience of explanation, the following description will describe selecting a user image that degrades quality characteristics among the user images received from the server device 200.

[0220] For example, if users participating in a video call service include a first user, a second user, a third user, and a fourth user of the electronic device 100, the electronic device 100 can acquire a first user video through the camera 192, and receive a second user video, a third user video, and a fourth user video through the communication interface 110. At this time, it is assumed that the first user video, the third user video, and the fourth user video are all 4K. Since the number of user videos to be decoded is greater than the number of decoders, the processor 140 can transmit a request to the server device 200 to reduce the resolution of some videos, for example, the third user video and the fourth user video, to 2K. Accordingly, the electronic device 100 can receive a 4K second user video, a 2K third user video, and a 2K fourth user video through the communication interface 110.

[0221] The first decoder 421 can decode a 4K first user image to generate a partial screen 810 including the first user image. The second decoder 422 can decode a 4K second user image to generate a partial screen 820 including the second user image. The third decoder 423 can time-division decode a 2K third user image and a 2K fourth user image to generate a partial screen 830 including the third user image and the fourth user image. The third decoder 423 having a 4K resolution can decode up to four 2K images, but in the example, since there are two 2K ​​images, the first sub-decoder 424 and the second sub-decoder 425 can be used for decoding. That is, the first sub decoder 424 can decode the third user video to generate a partial screen 831 corresponding to the third user video, and the second sub decoder 425 can decode the fourth user video to generate a partial screen 832 corresponding to the fourth user video. The mixer 428 can receive and mix the partial screen 831 corresponding to the third user video and the partial screen 832 corresponding to the fourth user video, thereby generating a partial screen 830 including the partial screen 831 and the partial screen 832.

[0222] The mixer 430 receives a first partial screen 810 corresponding to a first user image, a second partial screen 820 corresponding to a second user image, and a third partial screen 830 corresponding to a third user image and a fourth user image, mixes each partial screen into one full screen, and positions each partial screen at a predetermined position on the full screen to create one multi-view screen.

[0223] The scaler 440 can create a multiview screen 800 to be output to the display by scaling the multiview screen output from the mixer 430 to fit the size of the display screen.

[0224] The layout of the multi-view screen 800 illustrated in FIG. 8 is an example, and the layout of the multi-view screen 800 can be determined in various ways. However, since the 2K user video has a lower resolution than the 4K user video, the resolution can be reduced by making it smaller than the 4K user video.

[0225] FIG. 9 is a reference diagram for explaining an example of a method of operating an electronic device according to one embodiment.

[0226] Referring to FIG. 9, according to an example, the image processing unit 150 may include three 4K decoders capable of decoding images with a resolution of 4K or lower. Accordingly, the image processing unit 150 may decode up to three images with a resolution of 4K.

[0227] The processor 140 can identify that the number of 4K user videos that need to be decoded to provide a video call service is greater than the number of decoders included in the video decoder 150. For example, the processor 140 can identify the number of users participating in the video call service at the beginning of the communication connection with the server device 200. Alternatively, the processor 140 can identify that a user participating in the video call service is added while the video call service is in progress after the communication connection with the server device 200. If it is identified through this identification that the number of 4K user videos that need to be decoded to provide a video call service is greater than the number of decoders included in the video processing unit 150, the processor 140 can select multiple user videos for which quality characteristics are to be lowered in order to decode multiple user videos instead of one user video with one decoder, and transmit a request for lowering the quality characteristics of the selected multiple user videos to the server device 200 through the communication interface 110. Of course, a user image obtained through the camera 192 of the electronic device 100 can also be selected as a user image that degrades quality characteristics. For convenience of explanation, the following description will describe selecting a user image that degrades quality characteristics among the user images received from the server device 200.

[0228] For example, if users participating in a video call service include a first user, a second user, a third user, and a fourth user of an electronic device 100, the electronic device 100 can acquire a first user video through a camera 192, and receive a second user video, a third user video, and a fourth user video through a communication interface 110. At this time, it is assumed that the first user video, the third user video, and the fourth user video are all 4K. Since the number of user videos to be decoded is greater than the number of decoders, the processor 140 can transmit a request to the server device 200 to lower the fps of some videos, for example, the third user video and the fourth user video, to 30. Since the fps of a 4K video is 60, lowering the fps to 30 allows two videos to be decoded with one 4K decoder. Accordingly, the electronic device 100 can receive 4K 60 fps second user video, 4K 30 fps third user video, and 4K 30 fps fourth user video through the communication interface 110.

[0229] The first decoder 421 can decode a 4K first user video to generate a partial screen 910 including the first user video. The second decoder 422 can decode a 4K second user video to generate a partial screen 920 including the second user video. The third decoder 423 can time-division decode a 4K 30 fps third user video and a 4K 30 fps fourth user video to generate a partial screen 930 including the third user video and the fourth user video. That is, the first sub-decoder 424 can decode the third user video to generate a partial screen 931 corresponding to the third user video, and the second sub-decoder 425 can decode the fourth user video to generate a partial screen 932 corresponding to the fourth user video. The mixer 428 can receive and mix a partial screen 931 corresponding to a third user image and a partial screen 932 corresponding to a fourth user image, thereby generating a partial screen 930 including the partial screen 931 and the partial screen 932.

[0230] The mixer 430 receives a first partial screen 910 corresponding to a first user image, a second partial screen 920 corresponding to a second user image, and a third partial screen 930 corresponding to a third user image and a fourth user image, mixes each partial screen into one full screen, and positions each partial screen at a predetermined position on the full screen to create one multi-view screen.

[0231] The scaler 440 can create a multiview screen 900 to be output to the display by scaling the multiview screen output from the mixer 430 to fit the size of the display screen.

[0232] The layout of the multi-view screen 900 illustrated in FIG. 9 is an example, and the layout of the multi-view screen 900 can be determined in various ways. However, since the 30 fps user video has a lower frame rate than the 60 fps user video, the video playback may not be relatively smooth. Therefore, it may be desirable to select a user video corresponding to a non-speaking user as one with a lower frame rate.

[0233] FIG. 10 is a reference diagram for explaining an example of a method of operating an electronic device according to one embodiment.

[0234] The example illustrated in Fig. 10 shows an example of selecting an image for which quality characteristics are to be degraded by referring to the user's speech information.

[0235] The electronic device 100 may receive information about users' speech from the server device 200 while operating as illustrated in FIG. 9. Based on the information about users' speech, the electronic device 100 may identify, for example, a third user as the speaking user. However, referring to FIG. 9, the third user is being played at 30 fps with a reduced quality characteristic. Since the speaking user video is an important video, it may be necessary to improve the quality characteristic of the speaking user video. Accordingly, the electronic device 100 may transmit a request to the server device 200 to adjust the quality characteristic of the video to 60 fps for the third user video and, for example, to 30 fps for the second user video. Accordingly, the electronic device 100 may receive, from the server device 200, a second user video at 30 fps, a third user video at 60 fps, and a fourth user video at 30 fps. Then, the processor 140 can control the second demux 412 to demultiplex the third user video at 60 fps, and the third demux 413 to demultiplex the second user video at 30 fps and the fourth user video at 30 fps.

[0236] The first decoder 421 can decode a 4K first user video to generate a partial screen 1010 including the first user video. The second decoder 422 can decode a 4K third user video to generate a partial screen 1020 including the third user video. The third decoder 423 can time-division decode a 4K 30 fps second user video and a 4K 30 fps fourth user video to generate a partial screen 1030 including the second user video and the fourth user video. That is, the first sub-decoder 424 can decode the second user video to generate a partial screen 1031 corresponding to the second user video, and the second sub-decoder 425 can decode the fourth user video to generate a partial screen 1032 corresponding to the fourth user video. The mixer 428 can receive and mix the partial screen 1031 corresponding to the second user image and the partial screen 1032 corresponding to the fourth user image, thereby generating the partial screen 1030 including the partial screen 1031 and the partial screen 1032.

[0237] The mixer 430 receives a first partial screen 1010 corresponding to a first user image, a second partial screen 1020 corresponding to a third user image, and a third partial screen 1030 corresponding to the second user image and the fourth user image, mixes each partial screen into one full screen, and positions each partial screen at a predetermined position on the full screen to create one multi-view screen.

[0238] The scaler 440 can create a multiview screen 1000 to be output to the display by scaling the multiview screen output from the mixer 430 to fit the size of the display screen.

[0239] The layout of the multi-view screen 1000 illustrated in Fig. 10 is an example, and the layout of the multi-view screen 1000 can be determined in various ways.

[0240] According to one embodiment, when the server device 200 needs to transmit videos of users participating in a video call that exceed the video processing capability of the electronic device 100 based on the video processing capability of the client, i.e., the electronic device 100, some of the user videos may be mixed and provided to the electronic device 100. The video processing capability may indicate information about the ability or performance of the electronic device to process videos. For example, the video processing capability may include at least one of the number of videos that the electronic device can decode, the resolution, and the frame rate information. For example, if the electronic device 100 has two 4K decoders, the video processing capability of the electronic device 100 may be two 4K videos and 60 fps.

[0241] FIG. 11 is a reference diagram for explaining a method for a server device to transmit video of users participating in a video call based on the video processing capability of an electronic device according to one embodiment.

[0242] Referring to FIG. 11, an electronic device (first electronic device) 100 may provide capability information of the electronic device 100 to a server device 200 for a video call service. Then, the server device 200 may determine whether the capability of the electronic device 100 is sufficient to process multiple user videos participating in the video call. If the determination result shows that the capability of the electronic device 100 is sufficient to process multiple user videos participating in the video call, the server device 200 may transmit multiple user videos participating in the video call service to the electronic device 100 as is. If the capability of the electronic device 100 is not sufficient to process multiple user videos participating in the video call, the server device 200 may degrade the quality characteristics of some of the multiple user videos and mix and transmit the degraded multiple user videos. The term "mixing" may refer to processing or encoding multiple user videos so that a single decoder of the electronic device 100 can process them.

[0243] Referring to FIG. 11, the electronic device 100 can transmit three 4K videos and 60fps to the server device 200 as its capability information. The server device 200 can confirm that there are four user videos corresponding to multiple users participating in the video call, namely, a first user video, a second user video, a third user video, and a fourth user video. If each user video is 4K, the server device 200 can determine that the capability of the electronic device 100 is not sufficient to process multiple user videos that need to be decoded for the video call. Therefore, the server device 200 can mix some of the user videos to enable decoding of four user videos with three 4K decoders. For example, the server device 200 can leave two of the four videos as they are, lower the resolution of the remaining two videos, and mix the two videos with the lowered resolution. For example, the server device 200 can leave the first user video and the second user video as they are, lower the third user video and the fourth user video to FHD respectively, and mix the third user video and the fourth user video. In addition, the server device 200 can transmit the first user video, the second user video, and the mixed video of the third user video and the fourth user video to the electronic device 100.

[0244] Then, the electronic device 100 can receive a first user image, a second user image, and a mixed image of a third user image and a fourth user image from the server device 200. Then, the electronic device 100 can decode the first user image with a first decoder to generate a first partial screen 1110 corresponding to the first user, decode the second user image with a second decoder to generate a second partial screen 1120 corresponding to the second user, and decode a mixed image of the third user image and the fourth user image with a third decoder to generate third partial screens 1130 corresponding to the third and fourth users. Then, the electronic device 100 can output a multi-view screen including the partial screen 1110, the second partial screen 1120, and the third partial screen 1130.

[0245] FIG. 12 is a flowchart illustrating an example of a method of operating an electronic device according to one embodiment.

[0246] Referring to FIG. 12, in operation 1201, the electronic device 100 may receive a video call request from a user. The video call request may be a control signal requesting the initiation or execution of a video call service. The video call request may be received based on a user input input through the user interface of the electronic device 100. For example, a user of the electronic device 100 may wish to make a video call with a friend while watching a broadcast screen. Then, the user may input a user input requesting a video call through the user interface of the electronic device 100. Then, the electronic device 100 may recognize that a video call request has been received based on the user input. Alternatively, the video call request may be received from an electronic device of a counterparty with whom the user of the electronic device wishes to make a video call (e.g., 10, 20, 30 of FIG. 1). That is, the video call request may be received based on a request from at least one of the user of the electronic device 100 or the user of the counterparty's electronic device.

[0247] In operation 1202, the electronic device 100 may transmit a video call request to the server device 200.

[0248] The server device 200 can communicate with the electronic device 100 via a wireless communication network. For example, the server device 200 may be a server that provides a video call or a video call service, or supports communication required for a video call service. For example, at least two devices among the electronic device 100 and another user's electronic device are connected via a mobile communication network such as 3G, 4G, or 5G, and a video call can be performed between the electronic device 100 and the other user's electronic device. In this case, the server device 200 may be a communication server that supports mobile communication according to a communication standard such as 3G, 4G, or 5G. The operations required for performing the aforementioned video call can be performed using the communication relay of the server device 200. Although the communication relay operation of the server device 200 is not illustrated in FIG. 7, the communication relay operation of the server device 200 may be included in transmitting and receiving data or signals between two different devices. Specifically, the server device 200 may be responsible for transmitting data or signals to support the execution (or switching execution) of a video call service. For example, the server device 200 may perform a relay operation to transmit a user video signal generated by the electronic device 100 to the other party's device. In addition, the server device 200 may transmit or forward various data and signals used to perform the video call service to at least one of another user electronic device and the electronic device 100.

[0249] In operation 1203, the electronic device 100 may transmit image processing capability information of the electronic device. The image processing capability information may indicate information about the performance or ability of the electronic device 100 to process images. For example, the image processing capability information may include at least one of the number of decoders, resolution information of images that each decoder can decode, and fps information of images that each decoder can decode. One or more decoders included in the electronic device 100 may have the same or different image processing capabilities. For example, if the electronic device 100 has two 4K UHD decoders, the image processing capability information may be as follows: number of decoders = 2, resolution information = UHD, fps information = 60 fps.

[0250] In operation 1204, the electronic device 100 may acquire an image of the user using a camera.

[0251] In operation 1205, the electronic device 100 can transmit a video signal corresponding to the acquired user image to the server device 200.

[0252] In operation 1206, the server device 200 can determine whether the capacity of user images to be decoded for a video call service exceeds the capability of the electronic device. For example, if the resolution of the decoder equipped with the electronic device 100 is the same as the resolution of the user images to be decoded, the server device 200 can determine whether the number of user images corresponding to users participating in the video call exceeds the number of decoders of the electronic device. For example, if the capability information of the electronic device 100 indicates that it is equipped with two UHD 4K decoders and the images corresponding to users participating in the video call are three or more images with a total of UHD 4K resolution, the server device 200 can determine that the number of user images to be decoded for the video call service exceeds the capability of the electronic device. For example, if the resolution of the decoder equipped with the electronic device 100 is not the same as the resolution of the user videos to be decoded, the server device 200 can determine whether the total decoding amount of user videos corresponding to users participating in the video call exceeds the capacity that can be decoded by the decoders of the electronic device. For example, if the capability information of the electronic device 100 indicates that it is equipped with one UHD 4K decoder and one FHD 2K decoder, and the videos corresponding to users participating in the video call are three or more videos with a total resolution of FHD 2K, the server device 200 can determine that the user videos to be decoded for the video call service do not exceed the capability of the electronic device. This is because one UHD 4K decoder can decode up to four FHD 2K videos.

[0253] As a result of the judgment of operation 1206, if the capacity of the user videos to be decoded for the video call service exceeds the capability of the electronic device, the process may proceed to operation 1207, and if it does not exceed the capacity, the process may proceed to operation 1208.

[0254] In operation 1207, the server device 200 may degrade the quality characteristics of some user videos among the videos of multiple users participating in a video call and transmit the videos with degraded quality characteristics to an electronic device. At this time, the server device may perform a process of mixing the multiple videos with degraded quality characteristics into a single video stream. To degrade the quality characteristics of the videos, the server device 200 may perform a process of lowering the resolution or frames per second of some user videos. The server device 200 may select a video among the multiple videos for which the quality characteristics are to be degraded. For example, the server device 200 may select a video of a user who does not speak among the multiple videos as the video for which the quality characteristics are to be degraded. For example, if the capability information of the electronic device 100 indicates that it is equipped with two UHD 4K decoders and there are three UHD 4K resolution videos corresponding to users participating in a video call, the server device 200 can process two of the three UHD 4K videos by downscaling them to 2K videos, mixing the two 2K ​​videos, and processing them into a single video stream. If, for example, the capability information of the electronic device 100 indicates that it is equipped with two UHD 4K decoders and there are eight UHD 4K resolution videos corresponding to users participating in a video call, the server device 200 can downscaling each of the eight UHD 4K videos to 2K videos, mixing the four 2K videos, and processing them into a single video stream, and also mixing the four other 2K videos, and processing them into another video stream.

[0255] In operation 1208, the server device 200 may transmit the original user video and / or the mixed user video to the electronic device 100. For example, if the capacity of the user videos to be decoded for a video call service does not exceed the capability of the electronic device, the server device 200 may transmit the original user video as is to the electronic device 100. For example, if the capacity of the user videos to be decoded for a video call service exceeds the capability of the electronic device, the server device 200 may mix portions of a plurality of user videos and transmit the original user video and the mixed video to the electronic device 100.

[0256] In operation 1209, the electronic device 100 may receive a user video participating in a video call from the server device 200, decode the received user video, and output the decoded video to a multi-view screen. The electronic device 100 may receive a stream corresponding to an original user video, i.e., an unmixed user video, and decode the stream with a decoder to generate a partial screen corresponding to a single user video. The electronic device 100 may receive a stream obtained by mixing multiple user videos, and decode the stream with a decoder to generate a partial screen corresponding to multiple user videos. The electronic device 100 may output a multi-view screen including multiple partial screens generated in this manner.

[0257] FIG. 13 is a flowchart illustrating an example of a method of operating an electronic device according to one embodiment.

[0258] Referring to FIG. 13, in operation 1301, the electronic device 100 may receive a video call request from a user. Operation 1301 corresponds to operation 1201 of FIG. 12, and thus further description thereof is omitted.

[0259] In operation 1302, the electronic device 100 may transmit a video call request to the server device 200. Operation 1302 corresponds to operation 1202 of FIG. 12, and thus further description thereof is omitted.

[0260] In operation 1303, the electronic device 100 may acquire an image of the user using a camera.

[0261] In operation 1304, the electronic device 100 can transmit an image signal corresponding to the acquired user image to the server device 200.

[0262] In operation 1305, when the server device 200 receives a user video of the electronic device and videos of other users, it can encode the videos of the users participating in the video call and transmit them to the electronic device 100.

[0263] In operation 1306, the electronic device 100 can determine whether the capacity of user images to be decoded for a video call service exceeds the capability of the electronic device. For example, if the resolution of the decoder equipped with the electronic device 100 is the same as the resolution of the user images to be decoded, the electronic device 100 can determine whether the number of user images corresponding to users participating in the video call exceeds the number of decoders of the electronic device. For example, if the capability information of the electronic device 100 indicates that it is equipped with two UHD 4K decoders and the images corresponding to users participating in the video call are three or more images with a total of UHD 4K resolution, the electronic device 100 can determine that the number of user images to be decoded for the video call service exceeds the capability of the electronic device. For example, if the resolution of the decoder equipped with the electronic device 100 is not the same as the resolution of the user videos to be decoded, the electronic device 100 can determine whether the total decoding amount of user videos corresponding to users participating in the video call exceeds the capacity that can be decoded by the decoders of the electronic device. For example, if the capability information of the electronic device 100 indicates that it is equipped with one UHD 4K decoder and one FHD 2K decoder, and the videos corresponding to users participating in the video call are three or more videos with a total resolution of FHD 2K, the electronic device 100 can determine that the user videos to be decoded for the video call service do not exceed the capability of the electronic device. This is because one UHD 4K decoder can decode up to four FHD 2K videos.

[0264] As a result of the judgment of operation 1306, if the capacity of the user videos to be decoded for the video call service exceeds the capability of the electronic device, the process may proceed to operation 1307, and if it does not exceed the capacity, the process may proceed to operation 1310.

[0265] In operation 1307, the electronic device 100 may transmit image processing capability information of the electronic device. The image processing capability information may indicate information about the performance or ability of the electronic device 100 to process images. For example, the image processing capability information may include at least one of the number of decoders, resolution information of images that each decoder can decode, and fps information of images that each decoder can decode. One or more decoders included in the electronic device 100 may have the same or different image processing capabilities. For example, if the electronic device 100 has two 4K UHD decoders, the image processing capability information may be as follows: number of decoders = 2, resolution information = UHD, fps information = 60 fps.

[0266] In operation 1308, the server device 200 may perform a process of mixing into a single video stream some of the user videos among the plurality of user videos participating in the video call by lowering the quality characteristics of the user videos. In order to lower the quality characteristics of the video, the server device 200 may perform a process of lowering the resolution or fps of some of the user videos. For example, if the capability information of the electronic device 100 indicates that it has two UHD 4K decoders, and there are three videos corresponding to the users participating in the video call as UHD 4K resolution videos in total, the server device 200 may lower the resolution of two of the three UHD 4K videos to 2K videos, mix the two 2K ​​videos, and process them into a single video stream. For example, if the capability information of the electronic device 100 indicates that it has two UHD 4K decoders, and the videos corresponding to the users participating in the video call are eight videos of UHD 4K resolution, the server device 200 can lower the resolution of each of the eight UHD 4K videos to 2K videos, mix four 2K videos, process them into one video stream, and also mix four other 2K videos and process them into another video stream.

[0267] In operation 1309, the server device 200 may transmit the original user video and / or the mixed user video to the electronic device 100. For example, if the capacity of the user videos to be decoded for a video call service does not exceed the capability of the electronic device, the server device 200 may transmit the original user video as is to the electronic device 100. For example, if the capacity of the user videos to be decoded for a video call service exceeds the capability of the electronic device, the server device 200 may mix portions of a plurality of user videos and transmit the original user video and the mixed video to the electronic device 100.

[0268] In operation 1310, the electronic device 100 may receive a user video participating in a video call from the server device 200, decode the received user video, and output it as a multi-view screen. The electronic device 100 may receive a stream corresponding to an original user video, i.e., an unmixed user video, and decode it with a decoder to generate a partial screen corresponding to a single user video. The electronic device 100 may receive a stream obtained by mixing multiple user videos, and decode it with a decoder to generate a partial screen corresponding to multiple user videos. The electronic device 100 may output a multi-view screen including multiple partial screens generated in this manner.

[0269] The method for operating a display device according to one embodiment of the present disclosure may be implemented in the form of program commands that can be executed via various computer means and recorded on a computer-readable medium. Furthermore, the embodiment of the present disclosure may be a computer-readable recording medium having recorded thereon one or more programs containing commands for executing the method for operating a display device.

[0270] The above computer-readable medium may include program commands, data files, data structures, etc., either singly or in combination. The program commands recorded on the medium may be those specially designed and configured for the present invention, or may be those known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of program commands include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc.

[0271] Here, the device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, "non-transitory" simply means that the storage medium is a tangible device and does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is permanently stored in the storage medium and cases where data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0272] According to one embodiment, the method of operating a display device according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be at least temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0273] Specifically, the disclosed embodiment can be implemented as a computer program product including a recording medium storing a program for performing a method of operating a display device.

[0274] Although the embodiments have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements made by those skilled in the art using the basic concept of the present invention defined in the following claims also fall within the scope of the present invention.

Claims

1. In electronic device 100, Memory 130 for storing one or more instructions, At least one processor 140 comprising one or more of the above instructions, When said one or more instructions are individually and / or collectively executed by said at least one processor, said electronic device, Obtaining a plurality of user images including images corresponding to one or more users received from a server device for a video call and images corresponding to users of the electronic device, Determine whether the capability for image processing of the electronic device corresponds to the plurality of user images to be decoded for the video call, When it is determined that the capability for image processing of the electronic device does not correspond to the plurality of user images to be decoded for the video call, a predetermined number of user images with degraded quality characteristics among the plurality of user images are acquired, By processing a predetermined number of user images with degraded quality characteristics with one decoder, a partial screen corresponding to the predetermined number of user images is obtained, An electronic device configured to output a multi-view screen including the above partial screen.

2. In paragraph 1, When said one or more instructions are individually and / or collectively executed by said at least one processor, said electronic device, Identifying whether the number of the plurality of user images to be decoded for a video call exceeds the number of decoders of the electronic device; An electronic device configured to determine that the capability for image processing of the electronic device does not correspond to the plurality of user images to be decoded for the video call, when the number of the plurality of user images is identified as exceeding the number of decoders of the electronic device.

3. In paragraph 1 or 2, When said one or more instructions are individually and / or collectively executed by said at least one processor, said electronic device, An electronic device configured to select a user image corresponding to a non-speaking user as a user image that will degrade the quality characteristics.

4. In any one of paragraphs 1 to 3, When said one or more instructions are individually and / or collectively executed by said at least one processor, said electronic device, An electronic device configured to identify a user image corresponding to a user who does not speak based on at least one of information indicating whether a microphone is on / off, information indicating whether the amount of voice data exceeds a threshold, and information indicating whether the speaker is speaking.

5. In any one of paragraphs 1 to 4, When said one or more instructions are individually and / or collectively executed by said at least one processor, said electronic device, An electronic device configured to acquire one or more user images with degraded quality characteristics from a server device by transmitting a signal indicating a request to degrade quality characteristics of one or more user images to the server device and / or to acquire a predetermined number of user images with degraded quality characteristics by degrading quality characteristics of images corresponding to users of the electronic device.

6. In any one of paragraphs 1 to 5, The above quality characteristics include the resolution of the image, When said one or more instructions are individually and / or collectively executed by said at least one processor, said electronic device, An electronic device configured to acquire one or more user images with reduced resolution from the server device by transmitting a signal indicating a request to reduce the resolution of the one or more user images to the server device and / or to acquire user images with reduced quality characteristics of the predetermined number by reducing the resolution of an image corresponding to a user of the electronic device.

7. In any one of paragraphs 1 to 6, The above quality characteristics include the frame rate (Frame Per Second) of the video, When said one or more instructions are individually and / or collectively executed by said at least one processor, said electronic device, An electronic device configured to acquire one or more user images with a reduced frame rate from the server device by transmitting a signal indicating a request to lower the frame rate of the one or more user images to the server device and / or to acquire user images with a predetermined number of reduced quality characteristics by lowering the frame rate of an image corresponding to a user of the electronic device.

8. In any one of paragraphs 1 to 7, When said one or more instructions are individually and / or collectively executed by said at least one processor, said electronic device, An electronic device configured to decode a predetermined number of user images having degraded quality characteristics by time-slicing them into one decoder.

9. In any one of paragraphs 1 to 8, When said one or more instructions are individually and / or collectively executed by said at least one processor, said electronic device, By processing one or more user images with the quality characteristics not degraded by a corresponding decoder, one or more partial screens including each partial screen corresponding to each user image of the one or more user images are obtained, An electronic device configured to obtain the multi-view screen including a partial screen corresponding to the predetermined number of user images and one or more partial screens corresponding to each user image.

10. A method for operating an electronic device 100, An operation of obtaining multiple user images including images corresponding to one or more users received from a server device for a video call and images corresponding to users of the electronic device; An operation of determining whether the capability for image processing of the electronic device corresponds to the plurality of user images to be decoded for the video call; An operation of acquiring a predetermined number of user images with degraded quality characteristics among the plurality of user images when it is determined that the capability for image processing of the electronic device does not correspond to the plurality of user images to be decoded for the video call; An operation of obtaining a partial screen corresponding to a predetermined number of user images by processing a predetermined number of user images with degraded quality characteristics with one decoder, and A method comprising an action of outputting a multi-view screen including the above partial screen.

11. In paragraph 10, An operation of identifying whether the number of the plurality of user images to be decoded for a video call exceeds the number of decoders of the electronic device, and A method further comprising an operation of determining that the capability for image processing of the electronic device does not correspond to the plurality of user images to be decoded for the video call, when the number of the plurality of user images is identified as exceeding the number of decoders of the electronic device.

12. In paragraph 10 or 11, A method further comprising the action of selecting a user image corresponding to a non-speaking user as a user image for which the quality characteristic is to be lowered.

13. In any one of paragraphs 10 to 12, A method further comprising an operation of identifying a user image corresponding to a user who does not speak based on at least one of information indicating whether a microphone is on / off, information indicating whether the amount of voice data exceeds a threshold, and information indicating whether the speaker is speaking.

14. In any one of paragraphs 10 to 13, A method further comprising: obtaining one or more user images with degraded quality characteristics from the server device by transmitting a signal indicating a request to degrade quality characteristics of one or more user images to the server device, and / or obtaining a predetermined number of user images with degraded quality characteristics by degrading quality characteristics of images corresponding to users of the electronic device.

15. In a non-transitory computer-readable recording medium having recorded thereon a program including one or more instructions executable by a processor 140 of an electronic device 100, by executing the one or more instructions by at least one processor 140 of the electronic device 100, the electronic device 100, Obtaining a plurality of user images including images corresponding to one or more users received from a server device for a video call and images corresponding to users of the electronic device, Determine whether the capability for image processing of the electronic device corresponds to the plurality of user images to be decoded for the video call, When it is determined that the capability for image processing of the electronic device does not correspond to the plurality of user images to be decoded for the video call, a predetermined number of user images with degraded quality characteristics among the plurality of user images are acquired, By processing a predetermined number of user images with degraded quality characteristics with one decoder, a partial screen corresponding to the predetermined number of user images is obtained. A non-transitory computer-readable recording medium that outputs a multi-view screen including the above partial screen.

Images