Video display device, video call system, and video call method

Abstract

This video display device may comprise: a memory storing user viewing patterns; a camera configured to acquire image information in front of the video display device; a microphone configured to acquire audio information from surroundings of the video display device; a communication unit connected via a network to external electronic devices capable of making video calls with the video display device; and a processor electrically connected to the memory, the camera, the microphone, and the communication unit, wherein the processor may be configured to: identify an expected viewing time period on the basis of the user viewing patterns; on the basis of the state of the video display device, the expected viewing time period, the audio information, and the image information, determine whether a user is within an accessible range to the video display device; and transmit a result of the determination to the external electronic devices.

Description

Video display device, video call system and video call method

[0001] This specification relates to a video display device, a video call system, and a video call method.

[0002] Video display devices such as general TVs, laptops, or computers can operate as video call systems by running additional applications.

[0003] Conventional video call systems using video display devices allow the caller to attempt a call without verifying whether the other party is available to answer. In this case, if the recipient is actually unable to receive the call, the caller unnecessarily remains in a connection waiting state, which degrades the user experience.

[0004]

[0005] Therefore, when a video display device, such as an actual TV, laptop, or computer, intends to additionally provide video calls, technology is required to accurately determine whether a user is present in front of the video display device and notify the caller.

[0006] The technical problems to be solved in this specification are not limited to those mentioned above, and other unmentioned technical problems will be clearly understood by those skilled in the art to which this disclosure belongs from the description below.

[0007]

[0008] An image display device according to one embodiment of the present specification may include a memory for storing a user viewing pattern, a camera configured to acquire image information in front of the image display device, a microphone configured to acquire audio information around the image display device, a communication unit connected via a network to external electronic devices capable of video calling with the image display device, and a processor electrically connected to the memory, the camera, the microphone, and the communication unit.

[0009] According to one embodiment, the processor identifies an expected viewing time based on the user viewing pattern, determines whether the user is within an accessible range of the video display device based on the state of the video display device, the expected viewing time, the audio information, and the image information, and transmits the determination result to the external electronic devices.

[0010] According to one embodiment, the processor may periodically or based on a request from the external electronic device transmit to the external electronic device the result of determining whether the user is within the range where the user can access the image display device.

[0011] According to one embodiment, the processor receives a video call request from a first external electronic device and, based on the user's selection, may accept, reject, or transfer the video call request to a designated second external electronic device.

[0012] According to one embodiment, the processor receives a video call request from a first external electronic device, and if it is determined that the user is unable to access the video display device or if the user does not provide an input responding to the video call request within a preset time, the processor may forward the video call request to a second external electronic device.

[0013] According to one embodiment, the memory includes a learning model, and the processor learns the state of the video display device, the audio information, the image information, the expected viewing time, the usage history of the video display device, and the video call reception history through the learning model to learn a user behavior pattern, and based on the user behavior pattern, can determine whether the user is within an accessible range of the video display device.

[0014] According to one embodiment, if the processor determines that the user cannot access the video display device based on the audio information and the image information, it may be configured to re-determine whether the user is within the range where the user can access the video display device by considering the user behavior pattern.

[0015] According to one embodiment, the processor can learn the user behavior pattern by updating the video call reception history based on identifying an input receiving a video call with the external electronic device from the user after transmitting a result to the external electronic device that the user is unable to access the video display device, and by learning the user behavior pattern by updating the video call reception history based on identifying an input receiving a video call with the external electronic device from the user after transmitting a result to the external electronic device that the user is able to access the video display device.

[0016] According to one embodiment, the processor can transmit a message to the external electronic device that includes information indicating whether the user can access or cannot access the image display device and an authentication code.

[0017] According to one embodiment, the processor can switch the video call with the first external electronic device to a second external electronic device based on the user's request after a video call connection with the first external electronic device has been established.

[0018] According to one embodiment, when the processor receives a request from the user for video call switching, it can disconnect the video call connected between the first external electronic device and the video display device, identify the account of the second external electronic device, and reconfigure the call connection by sending a video call request message to the identified account of the second external electronic device.

[0019] According to one embodiment, the processor can compare the expected viewing time of the video display device with the time of a video call request from the external electronic device, and if the time of the request falls within the expected viewing time, it can assign a weight to the possibility of a call.

[0020] According to one embodiment, when the processor transmits a judgment result to the external electronic device, it may transmit a reliability index indicating the reliability of the judgment result together with the judgment result.

[0021] A video call method in a video display device according to one embodiment of the present specification may include: identifying an expected viewing time based on a user viewing pattern; acquiring image information of the front of the video display device through a camera; acquiring audio information of the surroundings of the video display device through a microphone; determining whether a user is within an accessible range of the video display device based on the state of the video display device, the expected viewing time, the audio information, and the image information; and transmitting the determination result to an external electronic device.

[0022] According to one embodiment, the operation of transmitting the judgment result to an external electronic device may include the operation of transmitting the judgment result to the external electronic device periodically or based on a request from the external electronic device.

[0023] According to one embodiment, the video call method may further include the operation of receiving a video call request from a first external electronic device and the operation of forwarding the video call request to a second external electronic device when it is determined that the user is unable to access the video display device or when the user does not provide an input responding to the video call request within a preset time.

[0024] According to one embodiment, the determining operation may further include an operation to learn a user behavior pattern by learning the state of the video display device, the audio information, the image information, the expected viewing time, the usage history of the video display device, and the video call reception history through a learning model, and an operation to determine whether the user is within an accessible range of the video display device based on the user behavior pattern.

[0025] According to one embodiment, the determining operation may further include, when it is determined that the user cannot access the video display device based on the audio information and the image information, an operation to determine whether the user is within the range where the user can access the video display device by considering the user behavior pattern.

[0026] According to one embodiment, the video call method may further include the operation of performing a video call connection with a first external electronic device, the operation of receiving a request from the user for video call switching, the operation of disconnecting a video call connected between the first external electronic device and the video display device, the operation of identifying an account of the second external electronic device, and the operation of reconstructing a call connection by transmitting a video call request message to the identified account of the second external electronic device.

[0027] According to one embodiment, the operation of transmitting the judgment result to an external electronic device may include the operation of transmitting a message to the external electronic device that includes information indicating whether the user can access or cannot access the image display device and an authentication code.

[0028] According to one embodiment of the present specification, a video call system comprises an external electronic device, a call relay server connected to the external electronic device via a network, and a video display device connected to the call relay server via a network. The video display device comprises a memory for storing a user viewing pattern, a camera configured to acquire image information in front of the video display device, a microphone configured to acquire audio information around the video display device, a communication unit connected to the video display device and external electronic devices capable of video calling via a network, and a processor electrically connected to the memory, the camera, the microphone, and the communication unit. The processor can identify an expected viewing time based on the user viewing pattern, determine whether a user is within an accessible range of the video display device based on the status of the video display device, the expected viewing time, the audio information, and the image information, and transmit the determination result to the external electronic device.

[0029]

[0030] According to one embodiment, a video display device, a video call system, and a video call method can provide a device capable of accurately determining whether a recipient is capable of video calling by utilizing a sensor (camera or microphone) and a viewing pattern embedded in the video display device.

[0031] According to one embodiment, a video display device, a video call system, and a video call method can enable a predictive determination of the possibility of a user's presence beyond simple sensor detection through learning user behavior patterns.

[0032] According to one embodiment, a video display device, a video call system, and a video call method can reduce unnecessary waiting and support a smooth video call connection by allowing the caller to recognize in advance whether a call is possible.

[0033] According to one embodiment, a video display device, a video call system, and a video call method can improve user convenience and service completeness by enabling the call connection to be switched to another terminal when necessary.

[0034] The effects obtainable in this specification are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which this disclosure belongs from the description below.

[0035]

[0036] FIG. 1 is a block diagram showing a video call system according to one embodiment of the present specification.

[0037] FIG. 2 is a flowchart illustrating the operation of a video call method using a video call system according to one embodiment.

[0038] Figure 3 is a diagram showing an example of a screen displayed on an external electronic device when a user is around a video display device.

[0039] Figure 4 is a diagram showing an example of a screen displayed on an external electronic device when the user is not near the video display device.

[0040] Figure 5 is a diagram showing an example of the screens of the video display device and the external electronic device after a video call is requested to the video display device through an external electronic device.

[0041] FIG. 6 is a drawing showing an example of a screen of a video display device and an external electronic device after a user of a video display device accepts a video call, according to one embodiment of the present specification.

[0042] FIG. 7 is a diagram showing an example of a user of a video display device switching a video call to an external electronic device during a video call.

[0043] FIG. 8 is a diagram showing an example of a user switching a video call to a video display device while making a video call to an external electronic device.

[0044] FIG. 9 is a diagram illustrating the operation of requesting a video call to an external electronic device using a video display device.

[0045]

[0046] The advantages and features of this specification and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, this specification is not limited to the embodiments disclosed below but may be implemented in various different forms; these embodiments are provided merely to ensure that the disclosure of this specification is complete and to fully inform those skilled in the art of the scope of the disclosure, and this specification is defined only by the scope of the claims. Throughout the specification, the same reference numerals refer to the same components.

[0047] When one component is referred to as being "connected to" or "coupled to" another component, it includes cases where it is directly connected or coupled to the other component, or cases where another component is interposed. Conversely, when one component is referred to as being "directly connected to" or "directly coupled to" another component, it indicates that no other component is interposed. "And / or" includes each of the mentioned items and all combinations of one or more of them.

[0048] The terms used herein are for describing the embodiments and are not intended to be limiting. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. As used herein, "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components, operations, operations, and / or elements to the mentioned components, operations, and / or elements.

[0049] Although terms such as "first," "second," etc., are used to describe various components, it goes without saying that these components are not limited by these terms. These terms are used merely to distinguish one component from another.

[0050] Therefore, it is obvious that the first component mentioned below may be the second component within the technical scope of this specification. Unless otherwise defined, all terms used in this specification (including technical and scientific terms) may be used in a meaning that is commonly understood by those skilled in the art to which this specification belongs. Furthermore, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0051] The terms 'part' or 'module' used in this embodiment refer to software or hardware components such as FPGAs or ASICs, and the 'part' or 'module' performs certain roles. However, the meaning of 'part' or 'module' is not limited to software or hardware. The 'part' or 'module' may be configured to reside in an addressable storage medium or configured to run one or more processors. Thus, as an example, the 'part' or 'module' may include components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The components and functions provided within the 'part' or 'module' may be combined into a smaller number of components and 'part' or 'module', or further separated into additional components and 'part' or 'module'.

[0052] The operation of the method or algorithm described in connection with some embodiments of this specification may be directly implemented by hardware, software modules, or a combination of both, executed by a processor. Software modules may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, or any other form of recording medium known in the art. An exemplary recording medium is coupled to a processor, and the processor can read information from the recording medium and write information to the storage medium. Alternatively, the recording medium may be integral with the processor. The processor and the recording medium may reside within an Application-Specific Integrated Circuit (ASIC). The ASIC may reside within a user terminal.

[0053] Hereinafter, embodiments of this specification are described in detail with reference to the attached drawings so that those skilled in the art can easily implement them. However, this specification may be implemented in various different forms and is not limited to the embodiments described herein.

[0054] Although this specification has been described with reference to the embodiments illustrated in the drawings, this is merely illustrative, and those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom. Accordingly, the true technical scope of protection of this specification should be determined by the technical spirit of the appended claims.

[0055] FIG. 1 is a block diagram showing a video call system according to one embodiment of the present specification.

[0056] Referring to FIG. 1, the video call system (10) includes a video display device (100), a server (190), and a plurality of external electronic devices (199).

[0057] The video display device (100) can operate as a receiving terminal for video calls. For example, it may be a smart TV equipped with an operating system for the video display device. The video display device (100) may internally include a processor (110), a display (111), an input interface (116), an input unit (120), a communication unit (130), a memory (150), and a learning model (160).

[0058] The processor (110) is a central control unit that controls the overall operation of the image display device (100) and can operate by being electrically connected to the memory (150), learning model (160), input unit (120), and communication unit (130).

[0059] The processor (110) can analyze the viewing history stored in the memory (150) to derive the time period during which the user's primary video display device is used, and based on this, identify the expected viewing time period. For example, if there is a record of the user repeatedly watching during weekday evening hours, that time period can be identified as a priority viewing time period.

[0060] The processor (110) can analyze image information obtained from the camera (121) to recognize whether a user is present in front of the video display device (100). Additionally, it can analyze audio information obtained from the microphone (126) to detect ambient noise or conversational voices and to secondarily determine whether the user is active. This information can be combined with the results of the viewing pattern analysis to more accurately determine whether the user is within an accessible range of the video display device.

[0061] The processor (110) can execute a learning model (160) to utilize viewing patterns, image information, audio information, device usage history, and past video call history as learning data. Through the learning model (160), the processor (110) can predict user behavior patterns and make more precise judgments than simply relying on currently detected sensor information. For example, even if the sensor detects a temporary absence of the user, if there is a history of repeated use at the same time in the past, the processor can make a correction to positively determine whether access is possible.

[0062] The judgment result may be generated in a form such as "accessible" or "inaccessible," and, if necessary, may be provided by quantifying the reliability or transmitted to an external electronic device (199) including an authentication code. A user of the external electronic device (199) (e.g., a caller) can check the status of the other party before attempting to request a video call.

[0063] When the processor (110) receives a video call request from an external electronic device (199), it can control whether to connect the call based on the result of the above determination. After the video call is connected, it can control the server (190) to switch the call connection to another designated terminal according to the user's request. For example, it can switch a video call in progress on a video display device to a mobile terminal, or switch a call in progress on a mobile terminal to a video display device. When the processor (110) receives a call switching request, it can identify the account of the designated terminal and send a switching request message to the server (190). The server (190) can release the currently connected call and reconfigure a new call with the designated terminal, thereby enabling a smooth switching between terminals according to the user's request.

[0064] The processor (110) can be implemented in hardware such as a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), and a general-purpose microprocessor, and some functions may be implemented in software or firmware. Therefore, the implementation form of the processor (110) in the present invention is not limited to specific hardware or software.

[0065] The display (111) is embedded in the video display device (100) and can output video and graphic information. The display (111) can display various types of content, such as broadcast signals, streaming content, and / or video call footage. When a video call is connected, the display (111) displays the other party's video and simultaneously provides control options, such as a call end button, a mute button, and / or a camera switch button, through a user interface (UI). The display (111) can be implemented based on liquid crystal display (LCD), organic light emitting diode (OLED), and / or micro light emitting diode (micro LED) technology, but is not limited to a specific display technology. The display (111) can support touch input, allowing the user to directly touch the screen to provide input for accepting, rejecting, and / or switching calls.

[0066] The input interface (116) is an input device that enables a user to interact with the video display device (100) and can be implemented in various forms such as a remote control, a touch panel, a physical button, and / or a voice recognition module. The input interface (116) can be used to input commands for the user to accept or reject a video call request, and after the video call is connected, it can provide various function control inputs such as ending the call, muting, and / or switching the camera.

[0067] The input interface (116) can be connected to the video display device (100) via a wired or wireless method, and may support a non-contact input method based on gesture recognition or voice commands for user convenience. The input interface (116) is not limited to a specific hardware form and various implementations are possible to enhance the user experience.

[0068] The input unit (120) includes a camera (121) and a microphone (126). The camera (121) can detect whether a user is present in front of the device by acquiring an image in front of the video display device (100). The microphone (126) can acquire ambient audio information to secondarily determine whether the user is active.

[0069] According to one embodiment, a camera (121) is positioned on the front of an image display device (100) to acquire image information in the direction the image display device is facing. The camera (121) can be utilized as a primary sensor to detect whether a user is present in front of the image display device. The acquired image can be transmitted to a processor (110) and used for face recognition, motion detection, and / or user location determination, and can be used not only for simple presence verification but also for determining the user's activity status. The camera (121) may be a single camera module, or multiple camera modules may be positioned as needed. Additionally, the camera (121) may be implemented in combination with an optical sensor, an infrared sensor (IR, infrared), or a depth sensor, and is not limited to a specific technical implementation.

[0070] According to one embodiment, the microphone (126) can acquire audio information generated around the video display device (100). The microphone (126) can be used to capture the user's voice, living noise, and ambient sounds to assist in determining whether the user is approaching. For example, even if the video display device is turned on and the user is not detected by the camera, if the sound of conversation is detected through the microphone, it can be predicted that the user is near the device. The microphone (126) may be a single microphone or may be implemented as a multi-microphone array to be used to remove noise or analyze the direction of the sound source. Additionally, the microphone (126) may be an analog microphone or a digital microphone and is not limited to a specific hardware implementation.

[0071] According to one embodiment, the communication unit (130) may be a module that enables the video display device (100) to transmit and receive data with an external electronic device (199) and a server (190). The communication unit (130) may include a wireless communication module or a wired communication module. The wireless communication module may include a wireless fidelity (Wi-Fi) module, a Bluetooth module, and / or a mobile communication module (LTE, long term evolution or 5G), and the wired communication module may include an Ethernet port or a high definition multimedia interface (HDMI)-based communication module. The communication unit (130) may perform the reception of a video call request, the transmission of a judgment result, and the exchange of video and audio streaming data, thereby enabling a seamless connection between the transmitting terminal and the video display device.

[0072] According to one embodiment, the memory (150) can store various data necessary for the operation of the video display device (100). For example, user viewing patterns, video call history, and the usage history of the device can be stored in the memory (150). The memory (150) can store system software such as program code or an operating system, and applications executed by the processor (110). The memory (150) can be implemented in various forms such as volatile memory (RAM, random access memory), non-volatile memory (ROM, read only memory), or flash memory. The memory (150) can be utilized as a storage device for temporary storage as well as long-term storage, and is not limited to a specific implementation.

[0073] According to one embodiment, the learning model (160) may be executed by the processor (110) or implemented as program code stored in memory (150). The learning model (160) can predict user behavior patterns by utilizing user viewing patterns, image information, audio information, device usage history, and / or video call reception history as learning data. For example, if there is a history of repeated viewing during a specific time period and video calls received during that time period in the past were frequently answered, the learning model (160) may determine that there is a high probability that the user is accessible during that time period. The learning model (160) may be implemented based on machine learning algorithms or artificial intelligence (AI) models and may include various forms ranging from simple statistical analysis methods to deep learning-based neural network structures. The learning model (160) can perform the role of improving the accuracy of determining call availability by predicting user behavior.

[0074] According to one embodiment, the server (190) is a network element that relays a video call connection between a video display device (100) and an external electronic device (199), and can transmit and receive a video call request signal, a result of determining accessibility, an authentication code, and a call control signal. The server (190) can perform the role of transmitting video and audio streaming data and may include a function that ensures a stable connection between a sending terminal and a receiving terminal.

[0075] According to one embodiment, the server (190) can route a call request by selecting a designated account among a plurality of external electronic devices (199), and when a call transfer request occurs, it can perform the function of releasing the currently connected call and reconfiguring the connection to a new terminal.

[0076] According to one embodiment, the server (190) may be implemented as a cloud server or a distributed network server and is not limited to a specific server architecture. The server (190) may be utilized in various environments to ensure the stability and scalability of the video call service.

[0077] According to one embodiment, an external electronic device (199) may operate as a sending terminal for performing a video call with a video display device (100). The external electronic device (199) may be of various forms such as a smartphone, tablet, laptop computer, or desktop computer, and may generate a video call request and transmit it via a server (190) or directly to the video display device (100).

[0078] According to one embodiment, an external electronic device (199) may be equipped with a camera and a microphone to acquire video and audio data and transmit it to a video display device (100) via a network. The external electronic device (199) may display the results of the determination received from the video display device (100) on a user interface. For example, if the other party is able to receive the video call, it may be displayed as "Accessible," and if reception is not possible, a message such as "Inaccessible" may be provided. The transmitted message may include additional information such as an authentication code or a reliability indicator. After the video call connection is established, the external electronic device (199) may generate an input to terminate the call or to switch the call to the video display device (100) or another designated terminal at the user's request. In this case, the server (190) performs the switching control, and the external electronic device (199) may provide the request intuitively in terms of user experience. The external electronic device (199) can operate on a general-purpose platform such as an Android operating system, an iOS (iPhone Operating System), or a Windows operating system, and is not limited to a specific operating system. Therefore, the external electronic device (199) can be integrated with the present invention in various hardware and software environments.

[0079] According to the video call system (10) of the above-described embodiment, the video display device (100) determines whether a user is accessible based on the user's viewing pattern and sensor information, and provides the result to an external electronic device (199) through a server (190), thereby supporting the caller to attempt a video call efficiently without unnecessarily waiting.

[0080] FIG. 2 is a flowchart illustrating the operation of a video call method using a video call system according to one embodiment.

[0081] Referring to FIG. 2, the video call method can be performed with the video display device (100) and the external electronic device (199) connected via a network.

[0082] According to one embodiment, in operation S210, the video display device (100) can identify an expected viewing time based on the user's viewing pattern.

[0083] The memory (150) can store a past viewing record of the user using the video display device, and the record may include the start time and end time of the content, the duration of viewing, and information on the channel or application selected by the user.

[0084] The processor (110) can analyze the viewing records to extract time zones and usage patterns that occur repeatedly. For example, if a user frequently watches a video display device between 9 PM and 11 PM on weekdays, that time zone can be derived as a major viewing time zone. The processor (110) can statistically calculate the viewing frequency by day of the week and / or time zone to calculate the probability of viewing at a specific time zone.

[0085] The processor (110) can set a certain time interval as the expected viewing time based on the analyzed results, and this interval may be defined fixedly or may be dynamically updated according to changes in the user's viewing habits. If correction is required, the learning model (160) can correct the expected viewing time by reflecting the long-term trend of past viewing data, thereby providing higher accuracy than simple statistical analysis.

[0086] The identified estimated viewing time can be combined with sensor data obtained from the camera (121) and microphone (126) and status information of the video display device in operation S230, and used as a criterion for determining whether the user is within the range of access to the video display device.

[0087] According to one embodiment, in operation S220, the image display device (100) or the processor (110) may receive an accessibility notification request from an external electronic device (199). The request may be transmitted to the image display device (100) via a server (190) or directly from the external electronic device (199) to the image display device (100), and whether the request has been received may be recognized by the communication unit (130). The communication unit (130) may receive the request signal through a network.

[0088] A request for accessibility notification may include caller information, the time of the request, the method of connecting the call, and / or authentication information. The request message may be transmitted in an encrypted form, and the video display device (100) may decrypt and process it.

[0089] When a valid request is received, the processor (110) may proceed to operation S230 to determine whether the user is within the range where the user can access the video display device. Operation S220 may serve to provide input conditions for determining the accessibility of operation S230. However, it is not limited thereto, and operations S230 and S240 may be performed without a trigger of operation S220. For example, the processor (110) may periodically determine whether the user can access the video display device and transmit the accessibility status to all external electronic devices to which an account is registered. By receiving the accessibility status, the external electronic device (199) can identify whether a video call is possible in advance before requesting a video call. That is, the processor (110) may perform operations S230 and S240 periodically or by a trigger of operation S220. The user of the external electronic device (199) can simultaneously check the accessibility status of the video call accounts and contact the user who is able to receive first.

[0090] According to one embodiment, in operation S230, the image display device (100) or the processor (110) can determine whether the user is within the range where the user can access the image display device by combining the state of the device, the expected viewing time, image information obtained from the camera (121), and audio information obtained from the microphone (126).

[0091] According to one embodiment, the processor (110) can analyze the device operation status, such as the power on / off status, currently running application, and whether the display (111) is active, which are state information of the device. For example, if the device is turned on and a specific application is running, the processor (110) can determine that there is a high probability that a user is nearby.

[0092] The processor (110) can compare the current time with the expected viewing time derived in operation S210, and if the current time corresponds to the expected viewing time, it can assign a weight to the possibility of a call. For example, if the time of the video call request falls within the expected viewing time, the processor (110) can assign a positive weight to the determination of accessibility, and conversely, if it is inconsistent with the expected viewing time, it can assign a low weight to the possibility of a call.

[0093] The processor (110) can recognize the user's face, body parts, and movements by analyzing image information obtained through the camera (121). The processor (110) can determine that the user is in an accessible state if, as a result of image recognition, the user is located in front of the video display device.

[0094] The processor (110) can detect ambient noise or voice by analyzing audio data obtained from the microphone (126). The processor (110) can determine that a user is nearby if the sound of conversation is continuously detected in the surroundings.

[0095] Even if the processor (110) determines that the user is absent based on information obtained from the camera (121) and microphone (126), if there is a history of repeated viewing or call response from the same time period in the past, the processor (110) can re-determine whether access is possible by considering behavioral patterns. Through this re-determining procedure, misjudgments caused by momentary sensor errors or environmental factors can be reduced. If the processor (110) determines that the user is not within an accessible range based on sensor information such as the camera (121) and microphone (126), the processor (110) can additionally perform a re-determining procedure using a learning model (160).

[0096] The processor (110) can determine whether a user is accessible by identifying past behavioral patterns using a learning model (160). For example, even if the user is not visible in the camera, if there is a history of repeatedly accepting calls at the same time in the past, a positive weight can be assigned to accessibility. The processor (110) can fuse the above elements to ultimately determine whether the user is within the range of accessibility to the video display device. The result of this determination can be displayed as "accessible" or "inaccessible" and can be transmitted to an external electronic device (199) in subsequent operation S240.

[0097] According to one embodiment, in operation S240, the image display device (100) can transmit the result of determining whether a user is within the range where the image display device can be accessed to an external electronic device (199).

[0098] The processor (110) may generate a judgment result calculated in operation S230 in the form of a message, and the message may include information indicating "accessible" or "inaccessible". Additionally, if necessary, an authentication code or an indicator indicating the reliability of the judgment result may be included. The reliability indicator may be calculated based on the match rate between the viewing pattern and the time of the request, the prediction accuracy of the recent learning model (160), and / or the quality of the data collected from the camera (121) and microphone (126).

[0099] The user of the external electronic device (199) can determine whether access to the video display device (100) is possible by entering or verifying the received authentication code, and based on this result, may or may not request a video call to the video display device (100). This prevents a misconnection or incorrect message transmission between the sender and the receiver.

[0100] The video display device (100) transmits a message containing information indicating accessibility to a server (190) via a communication unit (130), and the server can transmit the message to an external electronic device (199), and the external electronic device (199) can receive it and display it on a user interface. For example, if the other party is able to receive a call, a notice such as "Accessible" may be displayed on the screen of the external electronic device (199), and conversely, if the other party is unable to receive a call, a notice such as "Inaccessible" may be displayed.

[0101] Accordingly, in operation S240, by providing the access status determined by the video display device (100) to an external electronic device (199), the caller can check the status of the other party before attempting a video call and reduce unnecessary waiting time.

[0102] According to one embodiment, when the processor (110) receives a video call request when the judgment result is "inaccessible," it can control the video call connection message to be forwarded to an account of a designated external electronic device immediately without accepting the request.

[0103] According to another embodiment, the processor (110) can control the forwarding of a video call connection message to an account of a designated external electronic device if, regardless of the result of the judgment, a video call request is received but there is no user input within a preset time. This allows the caller to attempt to connect with the recipient through another registered account even when the recipient is not near the video display device (100).

[0104] According to one embodiment, the processor (110) can compare the judgment result calculated in operation S230 with whether the user actually accepts the video call and reflect the difference between the prediction and the actual result in the learning model (160). Through this, the video display device (100) can continuously update the video call reception history, and the learning model (160) can perform increasingly accurate behavioral pattern prediction. For example, the processor (110) can learn the user behavioral pattern by updating the video call reception history based on identifying an input accepting a video call request from the external electronic device (199) after transmitting a result to the external electronic device (199) that the user is unable to access the video display device (100). Additionally, the processor (110) can be configured to learn the user behavioral pattern by updating the video call reception history based on not inputting any response to a video call request from the external electronic device (199) after transmitting a result to the external electronic device that the user is able to access the video display device.

[0105] As illustrated in FIG. 2, an external electronic device (199) can recognize the accessibility of the user's video display device (100) and attempt a video call. Details regarding this are described below.

[0106] FIG. 3 is a drawing showing an example of a screen displayed on an external electronic device when a user is around a video display device, and FIG. 4 is a drawing showing an example of a screen displayed on an external electronic device when a user is not around a video display device.

[0107] Referring to FIGS. 3 and 4, the processor (110) can transmit the result of determining whether a user can access the video display device (100) to an external electronic device (199). At this time, the result of the determination may be provided in the form of a message including an authentication code along with information indicating "accessible" or "inaccessible." A display window (320) may be displayed on the display (310) of the external electronic device (199), and the user's accessibility status may be displayed in text form on the display window (320), along with an authentication code. The caller may proceed with the video call request procedure by verifying or entering the provided authentication code, thereby ensuring the reliability of the call connection.

[0108] As illustrated in FIG. 3, when a user (301) is present around a video display device (100), the display window (320) may display text (321) and an authentication code (322) that reads "Call reception possibility: Good". The caller may request a video call using the authentication code and may select and press the "Request Video Call" button (323) to do so. If the caller wishes to cancel the requested video call, they may select the "Cancel" button (324).

[0109] Conversely, as illustrated in FIG. 4, if the user (301) is not present around the video display device (100), the display window (320) may display text (421) and an authentication code (322) stating "Call reception possibility: poor." The caller can recognize in advance that the recipient is absent and choose whether to request a video call. According to the present embodiment, the user of the external electronic device (199) can intuitively check whether the user of the video display device (100) is in a state where they can actually access the video display device (100), and can make a call connection more reliably through the authentication code.

[0110] Referring to FIG. 3, when a user (301) is positioned in front of a video display device (100) within a living space, the presence of the user can be detected through a camera (121) and a microphone (126).

[0111] Additionally, the processor (110) can determine whether the user (301) is in a state where they can access the video display device (100) by combining the viewing pattern and sensor data.

[0112] The external electronic device (199) may be a terminal such as a smartphone, and a display window (320) may be displayed on the display (310) of the terminal.

[0113] The display window (320) displays a notification via text (321) stating "Call reception possibility: Good" and may also provide an authentication code (322). Additionally, the display window (320) is provided with a "Request Video Call" button (323) and a "Cancel" button (324), allowing the caller to immediately request a video call or cancel the request. According to one embodiment, in order to attempt a video call by pressing the "Request Video Call" button (323), it may be necessary to enter the authentication code (322).

[0114] Referring to FIG. 4, when the user (301) is not present in front of the video display device (100), the camera (121) and microphone (126) can detect the absence of the user. The processor (110) can determine that the user (301) is in an inaccessible state through viewing patterns, sensor information, and a learning model (160), and the result can be transmitted to an external electronic device (199).

[0115] A display window (320) is displayed on the display (310) of the external electronic device (199), and a notification such as "Call reception possibility: Poor" may be provided via text (421). An authentication code (322) may also be displayed at this time. A "Request Video Call" button (323) and a "Cancel" button (324) are displayed on the display window (320), so that even if "Call reception possibility: Poor" is displayed, the caller can attempt to request a video call by pressing the "Request Video Call" button (323). At this time, the caller may be aware in advance that the probability of success is low or that the call may be forwarded.

[0116] According to the above-described embodiment, when the user (301) is near the video display device (100), the status of the other party and an authentication code are provided on the display (310) of the external electronic device (199), allowing the caller to smoothly request a video call. Even if the user (301) is not near the video display device (100), the caller can check the status in advance on the display (310) of the external electronic device (199), thereby reducing unnecessary waiting and selecting an efficient call attempt.

[0117] Figure 5 is a diagram showing an example of the screens of the video display device and the external electronic device after a video call is requested to the video display device through an external electronic device.

[0118] Referring to FIG. 5, a message (501) and a function selection area (510) may be displayed on the display (310) of the external electronic device (199). The caller may request a video call to the video display device (100) by selecting the "video call" button (511) included in the function selection area (510).

[0119] When a video call request is transmitted to the video display device (100) via the server (190), a notification window (520) may be displayed on the display (111) of the video display device (100). The notification window (520) may display an accept button (521), a reject button (522), and a "answer on phone" button (523) along with a message such as "BBB requests a video call." The video call screen (503) on the display (111) of the video display device (100) may remain inactive until a connection is established, and the notification window (520) may be displayed in an overlay form.

[0120] A video call connection waiting screen (502) is displayed on the display (310) of the external electronic device (199), and the caller's video (591) can be displayed together. The caller can wait for the call request to be accepted by the other party.

[0121] According to the above-described embodiment, when a video call request occurs from an external electronic device (199), the video display device (100) intuitively displays the request so that the user can quickly select whether to accept the video call.

[0122] FIG. 6 is a drawing showing an example of a screen of a video display device (100) and an external electronic device (199) after a user of a video display device (100) accepts a video call, according to one embodiment of the present specification.

[0123] Referring to FIG. 6, when a user selects an accept button (521) on a notification window (520) displayed on the display (111) of a video display device (100), a video call can be connected.

[0124] After a connection is established, the display (111) of the video display device (100) can output a video call screen (503) in which the video of the user (611) is simultaneously displayed along with the video of the other party (612). During the video call, mute, end, and / or camera off icons may be provided on the display (111).

[0125] A video call screen (502) in which the other party's video (611) and one's own video (612) are simultaneously displayed can also be output on the display (310) of the external electronic device (199). The caller can conduct a video call in real time with the user of the video display device (100) and can perform the same control functions.

[0126] According to the above-described embodiment, the video display device (100) and the external electronic device (199) provide a dual-screen interface in which the other party's video and one's own video are displayed on each other's screens, thereby enabling a smooth video call experience.

[0127] FIG. 7 is a diagram showing an example of a user of a video display device switching a video call to an external electronic device during a video call.

[0128] Referring to FIGS. 6 and 7, a video call screen (503) may be displayed on the display (111) of the video display device (100). At this time, the video display device (100) may be making a video call with a first external electronic device. After receiving a call request, the video display device (100) may provide a “Answer on Phone” button (523), and this button may be activated before or after accepting the video call.

[0129] When the user selects the “Receive on Phone” button (523), a list of registered accounts (710) may be displayed on the display (111) as illustrated in FIG. 7. The list of accounts (710) includes multiple accounts (712), and each account may be identified by an icon and a phone number. At this time, the calling account (711) that initiated the video call may be excluded from the selection items.

[0130] When a user selects a specific account (712), the processor (110) can transmit identification information of a second external electronic device (199) associated with that account to the server (190). The server (190) can disable the video call between the current video display device (100) and the first external electronic device (199) and reconfigure a new video call between the second external electronic device (199) corresponding to the selected account (712) and the first external electronic device (199).

[0131] When a user selects a registered account (712), the video display device (100) may send a video call request message to a second external electronic device (199) associated with the selected account. The request message may be transmitted via a server (190) including an authentication code and an encrypted token, and if the second external electronic device (199) associated with the selected account accepts the request, a web server-based mobile-to-mobile call connection may be established between the first external electronic device (199) and the second external electronic device (199).

[0132] When the second external electronic device (199) accepts reception, a video call between mobile terminals can be made via a web server, and the video display device (100) can return to the content screen that was previously viewed. During this process, a pop-up window indicating that the video call has been switched to the second external electronic device (199) may be displayed on the display (111).

[0133] According to the above-described embodiment, the processor (110) identifies the account to be switched and can reconfigure the call to be connected to another external electronic device through the server (190), and the user can flexibly continue the call through various registered accounts excluding the caller account. This ensures flexibility in video call connection and allows the user to continue video calls smoothly in TV and mobile terminal environments.

[0134] FIG. 8 is a diagram showing an example of a user switching a video call to a video display device while making a video call to an external electronic device.

[0135] Referring to FIG. 8, a video call screen (502) is displayed on the display (310) of an external electronic device (199), and the video of the caller (801) and the video of the receiver (802) can be displayed together. A “Receive on TV” button (803) is displayed at the bottom of the screen, allowing the receiver to switch the video call to a TV.

[0136] When the user selects the "Receive on TV" button (803), a notification window (820) may be displayed on the display (111) of the video display device (100). The notification window (820) displays an accept button (821) and a reject button (822) along with a message asking, "Would you like to continue the call on TV?"

[0137] When the user of the video display device (100) selects the accept button (821), a video call screen (503) is displayed on the display (111), and the caller's video (801) and the recipient's video (802) can be displayed simultaneously. Additionally, a "answer on phone" button (823) is provided on the screen so that the call can be switched back to the mobile terminal if necessary.

[0138] According to the above-described embodiment, the user of the external electronic device (199) can switch an ongoing video call to the TV, and can repeatedly switch between the TV and the external electronic device thereafter, thereby improving user convenience.

[0139] FIG. 9 is a diagram illustrating the operation of requesting a video call to an external electronic device using a video display device.

[0140] Referring to FIG. 9, a content sharing screen (901) may be displayed on the display (111) of the video display device (100). The content sharing screen (901) is provided with a menu (902) of shared content such as photos, videos, and YouTube, and the user can view the shared content.

[0141] A "video call" button (903) is provided at the top of the screen, and a user can make a video call request by touching the button. Here, the location of the "video call" button (903) is not limited to the top of the screen but can be at any location within the display (111). When the user touches the "video call" button (903), an account list (910) can be displayed on the display (111). The account list (910) includes multiple accounts (911), and each account is identified by an icon (912), a username, and a phone number.

[0142] When a user selects a specific account (911), the processor (110) may display a notification window (920) containing a message that reads, "Requesting a video call to AAA." The notification window (920) includes an accept button (921) and a reject button (922), allowing the other party to accept or reject the request. If the request is accepted, a video call between the video display device (100) and the external electronic device (199) may be initiated.

[0143] According to the above-described embodiment, a video call can be requested directly to an external electronic device (199) through a video display device (100), and an efficient connection is possible based on user account selection.

[0144] According to the currency switching and account selection functions according to FIGS. 7 to 9 described above, the following effects are achieved.

[0145] During a video call between a video display device (100) and an external electronic device (199), the user can immediately switch to the desired terminal environment by providing a “answer on phone” button (523) or a “answer on TV” button (803). The call can be flexibly continued according to usage situations, such as using a TV in the living room and a mobile terminal when on the move.

[0146] By excluding the outgoing account (711) from the transfer target and selecting one of the previously registered accounts (712, 911), the transfer to an incorrect account or duplicate connection can be prevented. This ensures both the stability and security of the call connection.

[0147] The processor (110) transmits selected account information to the server (190), and the server (190) disconnects the existing connection and reconstructs a new connection, thereby ensuring the continuity of the video call. Accordingly, the caller and the receiver can be provided with an experience where the call continues without being interrupted.

[0148] The video display device (100) can return to the previously viewed content when switching calls, and at the same time, display the switched call status as a pop-up window to provide intuitive guidance to the user. This allows the user to simultaneously watch content and make a video call or clearly recognize the switching situation.

[0149] A video call can be directly requested to an external electronic device (199) by selecting an account on the display (111) of the video display device (100). Thus, an intuitive and extended communication environment through a large screen can be provided.

[0150] One embodiment can simultaneously improve user convenience and service reliability by implementing a flexible switching function between terminals and an account-based selective connection function that were not provided in existing video call environments.

[0151] Hereinafter, embodiments corresponding to each claim of the appended claims are described in detail, and the effects thereof are briefly described.

[0152] According to one embodiment, the video display device (100) may include a memory (150) for storing a viewing pattern, a camera (121) for acquiring an image of the front of the device, a microphone (126) for acquiring ambient audio, a communication unit (130) connected to an external electronic device (199), and a processor (110) electrically connected to them. The processor (110) may identify an expected viewing time based on the viewing pattern, determine whether a user can access the device by combining device status, audio, and image information, and then transmit the result to the external electronic device (199). This allows the sender to check the status of the other party in advance, thereby reducing unnecessary waiting.

[0153] According to one embodiment, when the processor (110) receives a video call request from an external electronic device (199), it can determine whether access is possible and transmit the result back to the external electronic device (199). This allows the caller to check the status of the recipient simultaneously with the call request.

[0154] According to one embodiment, the processor (110) may send a video call connection message to a designated account when it is determined that the user is inaccessible. This allows the connection to be attempted through an alternative account even if the recipient is absent.

[0155] According to one embodiment, the memory (150) includes a learning model (160), and the processor (110) can predict user behavior patterns by learning device status, viewing patterns, audio and image information, usage history, and call history. In addition, even if momentary absence is detected, a re-evaluation can be made by considering the behavior pattern, and the accuracy can be continuously improved by reflecting the actual call response results in the learning model. This enables reliable accessibility determination through behavior pattern learning without relying on simple sensor information.

[0156] According to one embodiment, the processor (110) can transmit the judgment result along with an authentication code to an external electronic device (199). This allows the caller to secure a reliable call connection through an authentication process.

[0157] According to one embodiment, the processor (110) can switch the call to another terminal at the user's request after the video call connection. In this process, the account to be switched is identified, and a call request is sent to the corresponding account to reconfigure the connection through the server (190). This allows the call to continue stably through registered accounts other than the caller account, and the switching between terminals is smooth.

[0158] According to one embodiment, the processor (110) may compare the expected viewing time and the request time to assign weights, or provide the judgment result along with a reliability indicator to an external electronic device (199). This allows for a more precise determination of whether a call is possible, and enables the caller to check the status through objective indicators.

[0159] According to one embodiment, the video call method may include storing viewing patterns and identifying expected viewing times, acquiring data through a camera and microphone, determining accessibility, and transmitting the determination result. Additionally, it may receive a request from an external electronic device and respond with the determination result, send a message to a designated account if accessibility is unavailable, and improve prediction accuracy through learning behavioral patterns. Furthermore, it may include an operation to switch to a designated terminal after the call is connected. By determining user accessibility in real time and transmitting it to the caller, and switching the call when necessary, it is possible to provide an efficient service.

[0160] According to one embodiment, a video call system may include an external electronic device (199), a call relay server (190), and a video display device (100). The video display device (100) may transmit a judgment result based on a viewing pattern to the external electronic device (199) via the server (190), and the server (190) may switch the call to another terminal via connection reconfiguration. This provides stable connection management through the server and a flexible switching function between terminals.

[0161] Although this specification has been described with reference to the embodiments illustrated in the drawings, this is merely illustrative, and those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom. Accordingly, the true technical scope of protection of this specification should be determined by the technical spirit of the appended claims.

Claims

1. In a video display device, Memory that stores user viewing patterns; A camera configured to acquire image information in front of the above-mentioned image display device; A microphone configured to acquire ambient audio information of the above-mentioned video display device; A communication unit connected via a network to the above-mentioned video display device and external electronic devices capable of video calls; and It includes the memory, the camera, the microphone, and a processor electrically connected to the communication unit, The above processor is, Based on the above user viewing patterns, identify the expected viewing time slots, and Based on the status of the video display device, the expected viewing time, the audio information, and the image information, it is determined whether the user is within the range where the video display device can be accessed, and Transmitting the above judgment result to the above external electronic devices, Video display device.

2. In Paragraph 1, The above processor is, Transmitting to the external electronic device, periodically or based on a request from the external electronic device, the result of determining whether the user is within the range where the user can access the image display device. Video display device.

3. In Paragraph 1, The above processor is, Receive a video call request from a first external electronic device, and Accepting, rejecting, or transferring the video call request to a designated second external electronic device based on the above user's selection, 4. In Paragraph 1, The above processor is, Receive a video call request from a first external electronic device, and If it is determined that the user is unable to access the video display device, or if the user does not provide an input responding to the video call request within a preset time, the video call request is forwarded to a second external electronic device. Video display device.

5. In Paragraph 1, The above memory includes a learning model, and The above processor is, Through the above learning model, the status of the video display device, the audio information, the image information, the expected viewing time, the usage history of the video display device, and the video call reception history are learned to learn user behavior patterns, and Determining whether the user is within an accessible range of the image display device based on the above user behavior pattern, Video display device.

6. In Paragraph 5, The above processor is, When it is determined that the user cannot access the video display device based on the above audio information and the above image information, the system is configured to re-determine whether the user is within the range where the user can access the video display device by considering the above user behavior pattern. Video display device.

7. In Paragraph 5, The above processor is, After transmitting to the external electronic device the result that the user is determined to be unable to access the video display device, the video call reception history is updated based on identifying an input receiving a video call with the external electronic device from the user, thereby learning the user behavior pattern. After transmitting to the external electronic device the result of determining that the user can access the video display device, the video call reception history is updated based on the non-identification of an input receiving a video call with the external electronic device from the user, thereby learning the user behavior pattern. Video display device.

8. In Paragraph 1, The above processor is, Transmitting a message to an external electronic device that includes information indicating whether the user can or cannot access the image display device and an authentication code. Video display device.

9. In Paragraph 1, The above processor is, After a video call connection is established with a first external electronic device, the video call with the first external electronic device is switched to a second external electronic device based on the user's request. Video display device.

10. In Paragraph 9, The above processor is, Upon receiving a request from the user for video call switching, the video call connected between the first external electronic device and the video display device is released, and Identifying the account of the above-mentioned second external electronic device, and Reconstructing the call connection by sending a video call request message to the account of the identified second external electronic device, Video display device.

11. In Paragraph 1, The above processor is, Comparing the expected viewing time of the video display device with the time of a video call request from the external electronic device, and assigning a weight to the possibility of a call if the time of the request falls within the expected viewing time. Video display device.

12. In Paragraph 1, The above processor is, In transmitting a judgment result to the above external electronic device, a reliability index indicating the reliability of the judgment result is transmitted together with the judgment result. Video display device.

13. A method for video calling on a video display device, An action that identifies expected viewing times based on user viewing patterns; The operation of acquiring image information of the front of the image display device through a camera; The operation of acquiring audio information around a video display device through a microphone; An operation to determine whether a user is within an accessible range of the video display device based on the state of the video display device, the expected viewing time, the audio information, and the image information; and including the operation of transmitting the above judgment result to an external electronic device, How to make a video call.

14. In Paragraph 13, The operation of transmitting the above judgment result to an external electronic device is, The operation of transmitting the judgment result to the external electronic device periodically or based on a request from the external electronic device, How to make a video call.

15. In Paragraph 13, The operation of receiving a video call request from a first external electronic device; and The operation further includes forwarding the video call request to a second external electronic device when it is determined that the user is unable to access the video display device or when the user does not provide an input responding to the video call request within a preset time. How to make a video call.

16. In Paragraph 13, The above-mentioned judgment operation is, An operation to learn user behavior patterns by learning the state of the video display device, the audio information, the image information, the expected viewing time, the usage history of the video display device, and the video call reception history through a learning model; and Based on the above user behavior pattern, the operation of determining whether the user is within an accessible range of the image display device, How to make a video call.

17. In Paragraph 16, The above-mentioned judgment operation is, If it is determined that the user cannot access the video display device based on the audio information and the image information, the operation further includes determining whether the user is within the range where the user can access the video display device by considering the user behavior pattern. How to make a video call.

18. In Paragraph 12, The operation of performing a video call connection with a first external electronic device; The operation of receiving a request from the above user for video call switching; An operation to release a video call connected between the first external electronic device and the video display device; An operation to identify the account of the second external electronic device; and The operation of reconstructing a call connection by sending a video call request message to the account of the identified second external electronic device further includes How to make a video call.

19. In Paragraph 13, The operation of transmitting the above judgment result to an external electronic device is, The operation of transmitting a message to the external electronic device that includes information indicating whether the user can or cannot access the image display device and an authentication code. How to make a video call.

20. In a video call system, External electronic device; A call relay server connected to the above external electronic device via a network; and It includes a video display device connected to the above-mentioned call relay server via a network, and The above image display device is, Memory that stores user viewing patterns; A camera configured to acquire image information in front of the above-mentioned image display device; A microphone configured to acquire ambient audio information of the above-mentioned video display device; A communication unit connected via a network to the above-mentioned video display device and external electronic devices capable of video calls; and It includes the memory, the camera, the microphone, and a processor electrically connected to the communication unit, The above processor is, Based on the above user viewing patterns, identify the expected viewing time slots, and Based on the status of the video display device, the expected viewing time, the audio information, and the image information, it is determined whether the user is within the range where the video display device can be accessed, and Transmitting the above judgment result to the above external electronic device, Video call system.

Images