Image display device, system and method

Abstract

According to various embodiments of the present specification, an image display device may comprise: a display for displaying content; an input unit for acquiring an input signal of a user while the content is displayed on the display; a communication unit for transmitting / receiving information to / from an external device; and a control unit interconnected with the display, the input unit and the communication unit, wherein the control unit can: display the content on the display; receive a first input signal through the input unit; generate, in response to the first input signal, at least one piece of information from among program information, screen capture information and screen analysis information of the content displayed on the display; acquire product information on the basis of the at least one piece of information; and display the acquired product information on the display or transmit same to a first external device through the communication unit.

Description

Image display device, system and method

[0001] The present disclosure relates to an image display device, system, and method. Specifically, it relates to an image display device, system, and method for providing product information within content while viewing content.

[0002]

[0003] With the increase in the viewing of content such as OTT (over-the-top) and VOD (video on demand), interest is rising in products, travel destinations, clothing worn by celebrities, and fashion items displayed on video display devices like smart TVs, and users' desire to consume these items is growing.

[0004] Accordingly, there is an increasing demand for services that allow users to instantly search for and purchase products of interest while watching content. Although users currently have to launch separate search engines or shopping applications to find related products, this method interferes with content viewing and involves significant inconvenience in the input process.

[0005]

[0006] The objective of the image display device, system, and method according to one embodiment is to provide product information immediately when the user desires it while viewing content.

[0007] The technical problems to be solved in this specification are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this disclosure belongs from the description below.

[0008]

[0009] According to various embodiments of the present specification, a video display device comprises a display for displaying content, an input unit configured to acquire a user input signal while the content is displayed through the display, a communication unit for transmitting and receiving information with an external device, and a control unit interconnected with the display, the input unit, and the communication unit. The control unit may display content on the display, receive a first input signal through the input unit, and in response to the first input signal, generate at least one of program information, screen capture information, and screen analysis information of the content displayed on the display, acquire product information based on the at least one information, and display the acquired product information on the display or transmit it to a first external device through the communication unit.

[0010] According to one embodiment, the control unit can derive text or keywords from the at least one piece of information using OCR, an object recognition algorithm, or an artificial intelligence model, and obtain the product information based on the text or keywords.

[0011] According to one embodiment, the control unit can create a Picture-In-Picture (PIP) window on one side of the display and display the product information.

[0012] According to one embodiment, the control unit may display similar products of the same category as the product information on the display together with the product information or transmit them to the first external device.

[0013] According to one embodiment, when a user selects at least one product among the products displayed on the display, the control unit may display information about the at least one product through the display or transmit it to the first external device.

[0014] According to one embodiment, the control unit can accumulate at least one product selected by the user to generate a list, and display the list through the display or transmit it to the first external device.

[0015] According to one embodiment, the control unit automatically sets an address for ordering based on the IP address of the image display device and can display only products available for order according to the address.

[0016] According to one embodiment, the control unit transmits the text or keyword to a second external device and can obtain product information based on the text or keyword from the second external device.

[0017] According to various embodiments of the present specification, a content viewing-based product recommendation and shopping linkage system comprises a video display device configured to display content and a server that processes information obtained from the video display device, wherein the video display device displays content on a display so that a user can view it, obtains an input signal from the user, and in response to the input signal, generates at least one of program information, screen capture information, and screen analysis information of the content displayed on the display, transmits the at least one information to the server, and the server processes the received at least one information to generate product information and transmits the product information to the video display device or an external electronic device.

[0018] According to one embodiment, the server can derive text or keywords from the at least one piece of information using OCR, an object recognition algorithm, or an artificial intelligence model, and generate the product information based on the text or keywords.

[0019] According to one embodiment, the server may acquire similar products of the same category as the product information and transmit the similar products together with the product information to the image display device or the external electronic device.

[0020] According to one embodiment, when the server receives product selection information from the image display device or the external electronic device, it can accumulate products selected by the user to create a list and transmit the list to the image display device or the external electronic device.

[0021] According to one embodiment, the server can automatically set an address for ordering based on the IP address of the image display device, generate product information including only information on products available for ordering according to the address, and transmit the product information to the image display device or an external electronic device.

[0022] According to one embodiment, the server may combine the product information and content viewing information and store them in a recommendation database to build a personalized recommendation model for the user.

[0023] According to one embodiment, the server receives an order request or a payment request from the video display device or the external electronic device, and can process the order request or the payment request in conjunction with a higher-level payment server or an external affiliate shopping mall server.

[0024] According to various embodiments of the present specification, a method for product recommendation and shopping linkage based on content viewing may include: an operation in which a video display device displays content; an operation in which the video display device obtains a signal from a user viewing the content; an operation in which the video display device generates at least one piece of information among program information, screen capture information, and screen analysis information of the content in response to the signal; an operation in which the video display device transmits the at least one piece of information to a server; an operation in which the server processes the one or more pieces of information to generate product information; and an operation in which the server transmits the product information to the video display device and / or an external electronic device.

[0025] According to one embodiment, the operation of generating product information by processing one or more pieces of information may include the operation of deriving text or keywords from the at least one piece of information using an OCR, an object recognition algorithm, or an artificial intelligence model, and the operation of generating product information based on the text or keywords.

[0026] According to one embodiment, the method may further include the operation of the server receiving product selection information of a user from the image display device or the external electronic device, and the operation of the server generating a list by accumulating products selected by the user in response to the product selection information of the user, and transmitting the list to the image display device or the external electronic device.

[0027] According to one embodiment, the operation of processing one or more pieces of information to generate product information may include the operation of automatically setting an address for ordering based on the IP address of the image display device and the operation of generating product information that includes only information of products available for ordering according to the address.

[0028] According to one embodiment, the method may further include the operation of the server receiving an order request or a payment request from the image display device or the external electronic device, and the operation of the server processing the order request or the payment request in conjunction with an upper payment server or an external affiliate shopping mall server.

[0029]

[0030] According to one embodiment, a video display device, system, and method can obtain information about related products while viewing content without a separate search by the user.

[0031] According to one embodiment, a video display device, system, and method can increase the accuracy of recommendation information provided to a user by accumulating viewing information and product information.

[0032] The effects obtainable in this specification are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which this disclosure belongs from the description below.

[0033]

[0034] FIG. 1 is a block diagram schematically illustrating a system including an image display device according to one embodiment.

[0035] FIG. 2 is a flowchart illustrating the operation of a system including an image display device according to one embodiment.

[0036] FIG. 3 is a diagram illustrating the operation of a system including an image display device and an electronic device according to one embodiment.

[0037] FIG. 4 is an exemplary drawing illustrating the operation of acquiring product information within an image display device according to one embodiment.

[0038] FIG. 5 is an exemplary drawing showing a product information list displayed in an image display device according to one embodiment.

[0039] FIGS. 6 and 7 are exemplary drawings showing a list of product information provided through an image display device and another electronic device according to one embodiment.

[0040]

[0041] The advantages and features of this specification and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, this specification is not limited to the embodiments disclosed below but may be implemented in various different forms; these embodiments are provided merely to ensure that the disclosure of this specification is complete and to fully inform those skilled in the art of the scope of the disclosure, and this specification is defined only by the scope of the claims. Throughout the specification, the same reference numerals refer to the same components.

[0042] When one component is referred to as being "connected to" or "coupled to" another component, it includes cases where it is directly connected or coupled to the other component, or cases where another component is interposed. Conversely, when one component is referred to as being "directly connected to" or "directly coupled to" another component, it indicates that no other component is interposed. "And / or" includes each of the mentioned items and all combinations of one or more of them.

[0043] The terms used herein are for describing the embodiments and are not intended to be limiting. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. As used herein, "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components, steps, actions, and / or elements to the mentioned components, steps, actions, and / or elements.

[0044] Although terms such as "first," "second," etc., are used to describe various components, it goes without saying that these components are not limited by these terms. These terms are used merely to distinguish one component from another.

[0045] Therefore, it is obvious that the first component mentioned below may be the second component within the technical scope of this specification. Unless otherwise defined, all terms used in this specification (including technical and scientific terms) may be used in a meaning that is commonly understood by those skilled in the art to which this specification belongs. Furthermore, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0046] The terms 'part' or 'module' used in this embodiment refer to software or hardware components such as FPGAs or ASICs, and the 'part' or 'module' performs certain roles. However, the meaning of 'part' or 'module' is not limited to software or hardware. The 'part' or 'module' may be configured to reside in an addressable storage medium or configured to run one or more processors. Thus, as an example, the 'part' or 'module' may include components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The components and functions provided within the 'part' or 'module' may be combined into a smaller number of components and 'part' or 'module', or further separated into additional components and 'part' or 'module'.

[0047] Steps of a method or algorithm described in connection with some embodiments of this specification may be directly implemented in hardware, software modules, or a combination of both, executed by a processor. Software modules may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, or any other form of recording medium known in the art. An exemplary recording medium is coupled to a processor, and the processor can read information from the recording medium and write information to the storage medium. Alternatively, the recording medium may be integral with the processor. The processor and the recording medium may reside within an Application-Specific Integrated Circuit (ASIC). The ASIC may reside within a user terminal.

[0048] Hereinafter, embodiments of this specification are described in detail with reference to the attached drawings so that those skilled in the art can easily implement them. However, this specification may be implemented in various different forms and is not limited to the embodiments described herein.

[0049] Although this specification has been described with reference to the embodiments illustrated in the drawings, this is merely illustrative, and those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom. Accordingly, the true technical scope of protection of this specification should be determined by the technical spirit of the appended claims.

[0050] FIG. 1 is a block diagram schematically illustrating a system including a video display device according to one embodiment. The block diagram schematically illustrates the configuration of a content viewing-based product recommendation and shopping linkage system (10) according to one embodiment.

[0051] Referring to FIG. 1, according to one embodiment, a system (10) may include a video display device (100), a server (130), and an external electronic device (190). The server (130) may be implemented in a cloud environment and may be connected to the video display device (100) and the external electronic device (190) via a wired or wireless communication network.

[0052] According to one embodiment, the image display device (100) may include a control unit (110), a display (111), an input unit (116), a communication unit (121), and a memory (126).

[0053] The video display device (100) may be an electronic device that provides content. For example, the video display device (100) may be implemented as a television, smart TV, set-top box, monitor integrated device, tablet PC, and smartphone. The video display device (100) may be connected to a communication network and may exchange data with a server (130) or an external electronic device (190) through the communication network.

[0054] The control unit (110) is a processor responsible for the overall control of the image display device (100) and can be implemented in the form of a CPU, GPU, or SoC (System on Chip). The control unit (110) may include one or more processors. The processors included in the control unit (110) may each implement the functions of the image display device (100) or interact with each other to implement the functions of the image display device (100). The control unit (110) can receive and process user input signals from the input unit (116) and control the display (111) to display content and product-related information. Additionally, the control unit (110) transmits and receives data with the server (130) through the communication unit (121) and can generate product information based on screen analysis results or program information, or transmit it to the server.

[0055] The control unit (110) can provide the content to the user by decoding the received content data and converting it into a video signal, and by outputting the converted video signal through the display (111).

[0056] The display (111) is a screen on which content video is displayed and can be implemented using various panel technologies such as LCD (liquid crystal display), LED (light emitting diode), OLED (organic light emitting diode), QLED (quantum light emitting diode), and Micro LED. The resolution of the content can be supported in various ways, such as SD, HD, Full HD, 4K, and 8K. The display (111) may display not only content but also additional information related to the content (e.g., subtitles, broadcast schedule information, program guide, advertising banner). For example, the display (111) may display additional interfaces such as a PIP window, a pop-up window, a product list, and an order window together with the content. In some embodiments, the display (111) may support a multi-window function to simultaneously display other application screens, notifications, or shopping-related interfaces along with the main content video.

[0057] The input unit (116) is a device for receiving user input and may include, for example, a voice input device (microphone), a remote control receiver, a touch panel, a motion recognition sensor, an on-screen object selection module, etc. The signal obtained through the input unit (116) may be an audio signal, an external device input signal, or an object selection signal, and this may be recognized as a trigger by the control unit (110) to induce a subsequent operation.

[0058] The communication unit (121) is a module connected to an external network that transmits and receives data with a server (130) and / or an external electronic device (190). The communication unit (121) can be implemented as a wired LAN, wireless LAN (Wi-Fi), Bluetooth, 5G / 6G communication module, etc. The communication unit (121) can transmit program information, screen capture, and screen analysis information obtained from the image display device (100) to the server (130), and can transmit product information or a list of orderable products received from the server to the control unit (110).

[0059] The memory (126) is a storage device that supports the operation of the control unit (110) and may include ROM (read only memory), RAM (random access memory), flash memory, etc. The memory (126) may store program code for the operation of the control unit (110), product information data, screen analysis results, user environment setting values, etc. In some embodiments, the memory (126) may be linked with a server (130) and utilized together with a cloud storage.

[0060] According to one embodiment, the image display device (100) may be expanded to include a camera module, a near-field communication (NFC) module, a location recognition module, etc., as needed. For example, the shape of a person or product within the screen can be directly recognized through the camera module, and product information can be shared by linking with an external mobile device through the NFC module.

[0061] A server (130) according to one embodiment can process information obtained from a video display device (100), generate product information based thereon, and provide it back to the video display device (100) or an external electronic device (190). The server (130) may include a memory (131), a control unit (132), and a communication unit (133), and may be implemented as a database server or a cloud server environment as needed.

[0062] The memory (131) can be implemented as an HDD, SSD, flash memory, or cloud storage as a data storage device of the server (130). The memory (131) may store product databases, user account information, delivery area data, order history data, recommendation data, etc. The memory (131) may temporarily store original data (program information, capture frames, etc.) received from the image display device (100) and utilize it for preprocessing and analysis. The memory (131) may store program code for implementing the functions of the control unit (132).

[0063] The control unit (132) is a processor that controls the overall operation of the server (130) and can be implemented as a CPU, GPU, FPGA, or AI-dedicated processor. The control unit (132) can obtain text information by processing program information, screen capture information, and screen analysis information received from the image display device (100). The control unit (132) can analyze text or keywords derived through OCR, object recognition algorithms, etc., and search a product database based on this to generate product cards (product name, price, image, link, etc.). According to one embodiment, the control unit (132) can create a list page by collecting products selected by the user or sort and provide a list of similar products within the same category. According to one embodiment, the control unit (132) can combine product information and content viewing information from which the product information was derived, store them in a recommendation database, and utilize this to perform a personalized product recommendation service in the future.

[0064] The communication unit (133) is responsible for transmitting and receiving data between the server (130) and the video display device (100) or portable communication device. The communication unit (133) can be implemented with various network interfaces, such as a TCP / IP-based wired network, Wi-Fi, or a 5G / 6G mobile communication network. The communication unit (133) can receive program information, screen analysis results, IP address-based location information, etc. transmitted from the video display device (100), and can transmit product information, order window data, similar product lists, etc. generated based on this information to the video display device (100) or an external electronic device (190). It can receive order requests or payment requests entered by a user and process them in conjunction with an upper payment server or an external affiliate shopping mall server.

[0065] The server (130) may be a module responsible for analysis, product information generation, delivery, and order processing based on information collected from the video display device (100). The server (130) may be organically linked with a recommendation system, a database management system, and a payment system.

[0066] The video display device (100) can independently process all or part of the functions implemented through the control unit (132) and memory (131) of the server (130). For example, the control unit (110) of the video display device (100) can derive text or keywords from a content screen or a captured screen using an OCR, an object recognition algorithm, or an AI model. Based on the text or keywords, the control unit (110) of the video display device (100) can create a list page by collecting products selected by the user or a product database, provide a list of similar products, and transmit it to an external electronic device (190). The control unit (110) of the video display device (100) can perform the operations of the control unit (132) of the server (130), including the operation of combining product information and content viewing information from which the product information was derived and storing it in a recommendation database.

[0067] An external electronic device (190) according to one embodiment is a device capable of receiving product information from a server (130) or a video display device (100), providing it to a user, and transmitting the user's order input back to the server (130). The external electronic device (190) can be implemented in various forms, such as a smartphone, tablet PC, laptop, wearable terminal, etc. The external electronic device (190) includes a display module and can display product information provided from the server (130) to the user. At this time, the displayed information may consist of a product name, price, thumbnail image, purchase button, option selection window, etc.

[0068] The external electronic device (190) can be linked with the video display device (100) to simultaneously reflect and display the selected product on the screen of the video display device (100).

[0069] The external electronic device (190) can receive user input through an input unit such as a touchscreen, keyboard, voice recognition module, microphone, etc. Through the input unit, the user can perform product selection, option specification (size, color, etc.), quantity input, purchase confirmation, etc.

[0070] The external electronic device (190) can be connected to the server (130) via Wi-Fi, Bluetooth, a mobile communication network (4G / 5G / 6G), etc. Through this, product information can be received in real time, and data such as order input, payment request, and delivery address confirmation can be transmitted to the server (130).

[0071] The external electronic device (190) may include a storage module capable of storing application data, user account information, shopping cart data, a temporarily stored product list, etc. For example, products selected by the user on the video display device (100) may be linked to the shopping cart app of the external electronic device (190) to be checked and ordered later.

[0072] Although the external electronic device (190) is shown to operate as a separate device from the image display device (100), within the present disclosure, the external electronic device (190) may be the same device as the image display device (100). For example, a server (130) that has obtained product information from content provided through the image display device (100) may transmit the product information to the external electronic device (190) or the image display device (100). The external electronic device (190) or the image display device (100) may perform a product order operation based on the received product information.

[0073] A system (10) according to one embodiment processes information obtained from a video display device (100) at a server (130) and can even execute an order by linking to an external electronic device (190), thereby providing a seamless product recommendation and shopping experience even while watching content.

[0074] FIG. 2 is a flowchart illustrating the operation of a system including a video display device according to one embodiment. For example, the flowchart of FIG. 2 illustrates a method for product recommendation and shopping linkage based on content viewing.

[0075] Referring to FIG. 2, in operation S210, the control unit (110) displays content on the display (111). The video display device (100) can play and display on the display (111) content such as a broadcast signal, OTT (Over-The-Top) service streaming, VOD (Video On Demand) content, internet-based video, or content stored on a local storage medium (DVD, USB, internal memory, etc.).

[0076] In operation S220, the control unit (110) can acquire a user signal through the input unit (116) while viewing content. The user signal may be a signal through various input methods. For example, the user signal may be in various forms, such as voice input, remote control input, touch input, or on-screen object selection input. For example, the video display device (100) can receive a user's voice command through the input unit (116), such as a built-in microphone or an external microphone device. The user may provide a trigger signal to the video display device (100) through voice, such as "What is this product?" or "Buy." The video display device can acquire the input signal through a dedicated button of the remote control, a combination of general buttons, or a smart remote control that includes a voice recognition function. When the user presses a specific button, the control unit (110) may operate to collect program information and / or screen capture images of the currently viewed screen. According to one embodiment, the control unit (110) can identify that a user directly selects a specific area of ​​the screen through a display (111) implemented in the form of a touchscreen or an external mobile device. For example, when the user touches a product (clothing, home appliance, prop, or car) displayed on the screen, the location information can be transmitted to the control unit (110) as an input signal. The above operation can be similarly implemented through object selection or pointing input. For example, an object on the screen can be selected using a motion recognition sensor, camera-based gesture recognition, or a cursor movement method using a laser pointer and / or a remote control. The user can generate a signal through a motion of pointing at a specific object on the screen. Based on the received trigger signal, the control unit (110) can proceed to a process of transmitting one or more of the program information, screen capture, and screen analysis results at that time to the server (130) in subsequent operation S230.

[0077] In operation S230, the control unit (110) may transmit one or more of the content's program information, screen capture information, and screen analysis information to the server (130) based on the signal. The transmitted information may include channel information being watched, program schedule data, screen images, OCR results, object detection results, etc. For example, the control unit (110) may transmit metadata such as the program ID, channel information, program schedule data, and broadcast time of the content being displayed through the display (111) to the server (130). For example, if a broadcast drama is being watched, the control unit (110) may transmit the program code and episode number of the corresponding broadcast together.

[0078] The image display device (100) or the control unit (110) can capture the screen displayed on the display (111) at the time of the user's trigger, extract it as image data, and transmit it to the server (130). In some embodiments, for copyright protection or personal information protection, the captured image may be transmitted to the server (130) in a reduced, blurred, or watermarked state.

[0079] The image display device (100) or the control unit (110) can perform a preliminary analysis on the captured screen to derive an OCR result (subtitle or text recognition result) or an object recognition result (product, brand logo, or prop). The image display device (100) or the control unit (110) can transmit the generated text information or object label information to the server (130).

[0080] The communication unit (121) of the video display device (100) is connected to the server (130) via a communication network (e.g., wired LAN, wireless LAN (Wi-Fi), mobile communication network (5G / 6G)), and can transmit the form of information selected by the control unit (110) (program information, captured image, analysis result) to the server (130) in real time. Through operation (230), the control unit (110) extracts data included in the content at a time desired by the user, so the server (130) can search for and generate product information desired by the consumer.

[0081] According to one embodiment, the control unit (110) can synchronize program information, screen capture frames, and screen analysis results of content currently displayed on the display (111) with timestamps based on the time when a user's trigger input occurs, and transmit them to the server (130). Through this, the server (130) can search for product information that exactly corresponds to the content screen at the time when the user is actually interested.

[0082] In operation S240, the server (130) can process the received information to obtain text information and generate product information based thereon. For example, a specific brand name or item name can be derived from the OCR result or object detection result to construct a related product list. For example, if the control unit (132) of the server (130) receives screen capture information instead of text information, it can recognize subtitles or text within the screen through an OCR module to extract product-related keywords. The control unit (132) of the server (130) can recognize specific products, brand logos, clothing, home appliances, etc. within a video frame by applying an object recognition algorithm. The control unit (132) of the server (130) can also refer to a product database pre-registered by broadcaster, program, and episode in conjunction with program information.

[0083] Based on acquired character information or keywords, the server (130) can search a product database to generate related product information (product name, image, price, link, option information, etc.). According to one embodiment, the server (130) can additionally configure a list of similar products within the same category corresponding to the generated product information, or generate a list page by accumulating products selected by the user. Additionally, the server (130) can automatically set an order address based on the IP address of the video display device (100), and filter only the products that can actually be ordered according to the address to include them in the product information.

[0084] Although operation S240 was described as being performed by the control unit (132) of the server (130), it may be performed by the control unit (110) of the image display device (100). In this case, operation S250 may be omitted.

[0085] In operation S250, the server (130) can transmit the product information to a video display device (100) or an external electronic device (190).

[0086] When transmitted to a video display device (100), the control unit (110) can directly display product information on the display (111) or display it in parallel with content video in the form of a PIP window or a pop-up window.

[0087] When transmitted to an external electronic device (190), product details can be displayed through an application or browser screen of the external electronic device (190).

[0088] In one embodiment, information regarding the same product may be provided simultaneously to a video display device (100) and an external electronic device (190), so that a user can perform a search on the video display device (100) and proceed with a detailed check and payment on the external electronic device (190).

[0089] In operation S260, the video display device (100) or external electronic device (190) provides received product information to the user, and the user can perform product selection, option specification, purchase request, etc. The user's order input is transmitted to the server (130) so that payment and order procedures can proceed. The display (111) of the video display device (100) or the external electronic device (190) can display selected product information, a list of similar products, a purchase button, and a save option. When the user selects a specific product, the product can be accumulated and managed on a list page, and the list page can be accessed through the video display device (100) or the external electronic device (190). When the user confirms the order, a payment request can be entered through the input section (e.g., touchscreen, voice recognition module, or remote control) of the video display device (100) or the external electronic device (190), and the order information is transmitted to the server (130).

[0090] The server (130) can perform actual order and payment procedures by linking the received order information with the payment system and delivery management system. Automatically set address information can be used as a delivery address and can enhance user convenience.

[0091] According to the content viewing-based product recommendation and shopping linkage method of one embodiment, relevant product information can be immediately provided through simple input by the user while watching content, and ordering can be linked, thereby enabling efficient shopping while maintaining the user's viewing experience.

[0092] FIG. 3 is a diagram illustrating the operation of a system including an image display device and an electronic device according to one embodiment.

[0093] Referring to FIG. 3, a video display device (100) according to one embodiment displays content (310) on a display, and an external electronic device (190) can display product information.

[0094] As illustrated on the left side of FIG. 3, the display (111) of the video display device (100) can display content (310) being viewed. In the illustrated example, a padded jacket (311) worn by a person, a vehicle (312), and a background (313) can be displayed through the display (111).

[0095] The control unit (110) can obtain screen capture information or screen analysis information from such video scenes. According to one embodiment, the control unit (110) can recognize specific objects or screen information, such as the location of a padding (311) worn by a person, the exterior of a vehicle (312), or a background (313). According to one embodiment, the control unit (110) or the server (130) can perform OCR and object recognition algorithms in parallel on a screen capture image. OCR can extract text information within the screen, such as subtitles, signboards, product labels, and document text, and the object recognition algorithm can identify visual objects such as clothing, automobiles, home appliances, food, and brand logos. The text information and object information derived in this way can be used as keywords for searching a product database.

[0096] The control unit (110) of the image display device (100) can analyze objects included in the corresponding scene and generate related product information based on the recognized screen information. The generated product information can be provided to the user through the display (111) or transmitted to an external electronic device (190) to be provided to the user. According to one embodiment, a server, rather than the image display device (100), can generate related product information. The server (130) can analyze objects included in the corresponding scene and generate related product information based on screen information transmitted from the image display device (100). The generated product information can be transmitted from the server (130) to an external electronic device (190) to be provided to the user.

[0097] As illustrated on the right side of FIG. 3, an external electronic device (190) can provide a product guide interface (320) (or an interface within an application) on the screen (191) of the external electronic device (190) based on product information received from a server (130) (or a video display device (100)). The product guide interface (320) may include a plurality of product cards (321, 322, 323, 324), and each product card may display an image, name, price, purchase options, etc. of the corresponding product.

[0098] The user can select at least one of a plurality of product cards (321, 322, 323, 324) through touch input of an external electronic device (190), and the selected product can be added to a shopping cart or lead to an immediate purchase procedure. In addition, in some embodiments, the same product information may also be displayed in parallel on the display (111) of the video display device (100), so that product browsing is possible simultaneously through a TV screen and a mobile screen. Through this, when the user enters an order, the order information can be transmitted to a server as described in operation S260.

[0099] Accordingly, the image display device (100) and the system (10) including the same can analyze an object in the content displayed on the image display device (100) and display the resulting product information on an external electronic device (190), thereby providing the user with the opportunity to intuitively proceed with shopping.

[0100] FIG. 4 is an exemplary drawing illustrating an operation of acquiring product information within an image display device according to one embodiment. FIG. 5 is an exemplary drawing illustrating a product information list within an image display device according to one embodiment. FIG. 6 and FIG. 7 are exemplary drawings illustrating a product information list provided through an image display device and another electronic device according to one embodiment.

[0101] Referring to FIG. 4, the image display device (100) can display a Picture-In-Picture (PIP) window (400) within the display (111).

[0102] The control unit (110) of the image display device (100) can identify one or more objects by performing an object recognition algorithm on the content (310) displayed on the display (111). The identified objects may be, for example, padding (311) worn by a person, a vehicle (312) in the scene, or an object in the background (313). According to one embodiment, the image display device (100) may perform the operation of identifying objects only when there is a user input signal.

[0103] The control unit (110) can create and display a PIP window (400) on one side of the screen to provide product information corresponding to the identified object to the user.

[0104] According to another embodiment, the content (310) displayed on the display (111) may contain product information (PPL product). The control unit (110) may create and display a PIP window (400) on one side of the screen to obtain product information contained in the content (310) and provide it to the user.

[0105] As illustrated in FIG. 4, the PIP window (400) may include a product icon (410) corresponding to an object.

[0106] When only one object is identified, the control unit (110) may generate and display a single icon (410) corresponding to the object. When multiple objects are identified, the control unit (110) may generate and display multiple icons (411, 412, 413) for each object. Alternatively, one representative icon (410) may be displayed, and switching buttons (401a, 401b) may be placed around it so that the user can sequentially switch to and check other icons.

[0107] Each product icon (410, 411, 412, 413) may be a graphic image that visually simplifies a product related to the corresponding object recognition result. For example, if the control unit (110) recognizes a padding (311) within the content (310), it may display a clothing product icon (411), such as a top or a padding, and if it recognizes a vehicle (312) within the content (310), it may display a car icon (412). If the control unit (110) identifies a place through program information or an AI model, it may provide travel information about the place to the user through a travel service icon (413).

[0108] The user can select a specific product icon within the PIP window (400) via a remote control button, touch input, or voice command. When an icon is selected, the control unit (110) transmits the corresponding object information to the server (130), and the server (130) can generate a categorized recommended product list based on this.

[0109] In the illustrated example, the recommended product list consists of clothing product icons (411), automobile-related product icons (412), and travel service icons (413). The recommended product list thus generated may be displayed directly on the display (111) of the image display device (100) or transmitted to an external electronic device (190) and provided through a separate user interface (320). In the illustrated example, the recommended product list is described as an image object such as an icon, but is not limited thereto and may be displayed as text.

[0110] Accordingly, in the embodiment according to FIG. 4, the video display device (100) recognizes an object in the content (310) and generates an icon corresponding to the identified object and displays it in the PIP window (400), thereby allowing the user to intuitively search for related products without being interrupted from watching the content.

[0111] Referring to FIG. 5, the control unit (110) can recognize clothing, such as a padded jacket (311) worn by a person, as a specific object from the screen analysis result of the content (310). Product information generated by the server (130) in response to the recognized object can be transmitted to the image display device (100), and the image display device (100) can display it in a part of the screen.

[0112] The image display device (100) may provide a product list window (520) on one side of the display (111). In the product list window (520), multiple similar products corresponding to the recognized object (e.g., padded clothing) are displayed together. Each product may include information such as a product name, image, and price, and the user may select a product of interest via a remote control, touch input, or voice command.

[0113] According to one embodiment, the selected product may be added to a shopping cart or transmitted to an external electronic device (190) so that the user can specify detailed options and proceed with the purchase process. Additionally, the product list window (520) may be placed at the bottom or on one side of the screen so as not to obscure the main content video, and may be provided in a form similar to a PIP window.

[0114] Accordingly, in the embodiment according to FIG. 5, the image display device (100) provides a list of similar products within the same category associated with the recognized object to the display (111), thereby allowing the user to compare and review various products without being hindered from viewing the content.

[0115] When a user selects a specific product from the product list window (520), the video display device (100) stores the selected product information and can organize multiple selected products into a single list page (cart page).

[0116] A number of products selected by the user are displayed collectively on the list page, and detailed information for each product (product image, price, options, or quantity) may be included. The user can select the product to be finally purchased from the list page or proceed with payment by linking with an external electronic device (190).

[0117] In one embodiment, the image display device (100) provides a list of similar products associated with the recognized object, and then accumulates the products selected by the user to create a list page, thereby providing a series of user experiences leading to comparison, review, and purchase.

[0118] Referring to FIG. 6, as illustrated on the left side of FIG. 6, content (310) is displayed on the display (111) of the image display device (100). In the illustrated example, content (310) including padding (311), a vehicle (312), and a background (313) is displayed, and an icon (413) representing a travel-related object is displayed at the bottom of the screen.

[0119] In the embodiment illustrated in FIG. 4, when a user selects a travel icon (413), the image display device (100) can transmit the selection information to a server (130). The server (130) can generate travel-related product information based on content context and analysis results corresponding to the selected icon and provide it to an external electronic device (190).

[0120] As illustrated on the right side of FIG. 6, a travel product guide interface (320) may be displayed on the screen (191) of an external electronic device (190). In the illustrated example, the product guide interface (320) includes a plurality of travel product cards (601, 602, 603, 604), and each product card may display information such as specific tourist attractions, travel packages, airline tickets, and accommodation products.

[0121] The user can select a travel product card of interest through touch input of an external electronic device (190), and the selected product is connected to a detail page, which can lead to the selection of options, price check, reservation, and payment procedures.

[0122] Accordingly, the embodiment according to FIG. 6 describes a process in which travel-related product information is intuitively provided on an external electronic device (190) by the user selecting an icon (413) associated with an object detected in the image display device (100). This allows the user to easily browse and purchase products in specific areas of interest (travel, fashion, home appliances, etc.) even while watching content.

[0123] FIG. 7 is a diagram illustrating an example of providing restaurant information available for delivery based on an IP address, based on a food object recognized in a content scene according to one embodiment.

[0124] Referring to FIG. 7, as shown on the left, a content scene (710) is displayed on the display (111) of the image display device (100). In the illustrated example, a scene of a person eating ramen (711) is displayed. The control unit (110) obtains screen analysis information from the scene and can detect the ramen (711) through an object recognition algorithm.

[0125] The detected food object information is transmitted to the server (130) via the communication unit (121). The server (130) can automatically identify a delivery area corresponding to the user's location by utilizing the IP address of the video display device (100) along with the received information. Subsequently, the server (130) searches for ramen restaurants or similar delivery menus available for delivery in the area and generates related product information.

[0126] As illustrated on the right side of FIG. 7, a delivery-available ramen restaurant guide interface (720) (or application) may be displayed on the screen (191) of an external electronic device (190). In the illustrated example, the guide interface (720) includes a plurality of delivery-available ramen menu cards (721, 722, 723, 724), and each card may display information such as food images, business names, prices, and estimated delivery times.

[0127] In some embodiments, ramen icons corresponding to the recognized food object may be directly provided on the display (111) of the image display device (100) itself. The user may select the ramen icon displayed on the display (111) using a remote control, voice command, or cursor movement within the screen, and in this case, corresponding delivery restaurant information may also be provided from the server (130).

[0128] The user can select a specific menu through an interface of an external electronic device (190) or a video display device (100), and the selected order can be transmitted to a server (130) and linked to a delivery order from the corresponding restaurant. In some embodiments, the server (130) can be configured to automatically filter restaurants available for order based on IP addresses so that only menus that the user can actually order are displayed on the interface (720) or display (111). In the embodiment according to FIG. 7, food scenes in the content are recognized, and delivery-available restaurant menus are recommended using IP address-based location information, and icons corresponding to the recognized food objects are directly provided on the TV screen, thereby providing the user with an immediate and intuitive food ordering experience linked to the content.

[0129] According to one embodiment, the server (130) estimates the user's location information based on the IP address of the video display device (100) and can filter and provide only products that are actually available for delivery according to the location. For example, certain home appliances or furniture may be limited to logistics delivery areas, and certain food may be limited to stores within the delivery radius. The server (130) combines this delivery availability information with product data so that the user can check only the products that are actually available for purchase.

[0130] Referring to FIGS. 3 through 7, a system including an image display device (100) can output content (310) that a user is watching through a display (111). In the illustrated example, when a scene in which a person is wearing specific clothing is displayed, the control unit (110) can identify padding (311) through an object recognition algorithm. The identified object information can be transmitted to a server (130) through a communication unit (121). The server (130) can build a recommendation database by considering the received information, content viewing patterns, program genres, and / or broadcast times together. The recommendation database can build a personalized recommendation model by accumulating multiple content viewing information and product selection information, and matching content viewing patterns, broadcast times, and / or program genres.

[0131] The server (130) can provide a recommendation interface on the screen (191) of an external electronic device (190) or the display (111) of a video display device (100) based on information stored in the recommendation database. The personalized recommendation interface can provide a customized product list that reflects the user's past viewing history and product selection history.

[0132] If the user frequently selects a clothing category while watching past dramas, the server (130) can recommend a clothing category in advance. Within the clothing category, the server (130) can suggest new products from brands frequently selected by the user, seasonal recommended products, discount event products, or products preferred by other similar user groups.

[0133] According to one embodiment, the server (130) may consider the viewing time, region, and / or season for recommendation results. The server (130) may prioritize recommending coats or late-night snacks during the winter evening hours, and recommend air conditioners or summer seasonal foods during the summer. The server (130) may prioritize recommending travel products during the vacation season.

[0134] According to the above-described embodiment, a system (10) including a video display device (100) and a server (130) can enhance a user-customized recommendation experience by having the video display device (100) and the server (130) cooperate to store user content viewing information and product information in a recommendation database (820) and provide personalized product information to an external electronic device (190) based thereon.

[0135] According to one embodiment, the user can obtain information about products of interest through a simple input signal without going through a separate search process while watching content, thereby maintaining the continuity of the content viewing experience.

[0136] According to one embodiment, the control unit of the image display device can perform an OCR or object recognition algorithm based on screen capture, program information, or screen analysis information, so that products exposed in the content can be automatically recognized and related product information can be quickly derived.

[0137] According to one embodiment, product information and a purchase interface can be provided in the form of a PIP window or a pop-up on the display of a video display device, so that the user can immediately perform product saving or purchase operations without interrupting viewing.

[0138] According to one embodiment, a list of similar products belonging to the same category is displayed together, and a list page can be generated by accumulating the products selected by the user, so the user can compare and review various products and make purchasing decisions more easily.

[0139] According to one embodiment, an order address is automatically set based on the IP address of a video display device, and the types of products available for order can be automatically filtered according to the address. This allows the user to check only the products actually available for purchase without a complex address input process, thereby improving convenience and reliability.

[0140] According to one embodiment, since the server can combine product information and content viewing information to store and utilize them as recommendation data, the system can improve the quality of personalized recommendations over time.

[0141] According to one embodiment, through the linkage between a TV and a portable communication device, it is possible to perform content-based browsing on the TV and proceed with detailed ordering and payment on the mobile, thereby providing a multi-device shopping experience optimized for the user environment.

[0142] Accordingly, one embodiment provides a new user experience (UX) that can naturally integrate a shopping experience even while watching content, and can simultaneously secure commercial usability and service scalability.

[0143] Although this specification has been described with reference to the embodiments illustrated in the drawings, this is merely illustrative, and those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom. Accordingly, the true technical scope of protection of this specification should be determined by the technical spirit of the appended claims.

Claims

1. In a video display device, A display that displays content; An input unit configured to acquire a user's input signal while content is displayed through the above display; A communication unit that transmits and receives information with an external device; and It includes a control unit interconnected with the above-mentioned display, the above-mentioned input unit and the above-mentioned communication unit, and The above control unit is, Display content on the above display, and A first input signal is received through the above input unit, and In response to the first input signal, at least one of program information, screen capture information, and screen analysis information of the content displayed on the display is generated, and Based on at least one of the above information, product information is obtained, and Displaying acquired product information on the above display or transmitting it to a first external device through the above communication unit, Video display device.

2. In Paragraph 1, The above control unit is, Deriving text or keywords from at least one piece of information using OCR, an object recognition algorithm, or an artificial intelligence model, and obtaining product information based on the text or keywords, Video display device.

3. In Paragraph 1, The above control unit is, Creating a Picture-In-Picture (PIP) window on one side of the above display and displaying the product information, Video display device.

4. In Paragraph 1, The above control unit is, Displaying similar products of the same category as the above product information on the above display together with the above product information or transmitting them to the above first external device, Video display device.

5. In Paragraph 1, The above control unit is, When a user selects at least one product among the products displayed on the display, information regarding the at least one product is displayed through the display or transmitted to the first external device. Video display device.

6. In Paragraph 1, The above control unit is, Generating a list by accumulating at least one product selected by the user, and displaying the list through the display or transmitting it to the first external device. Video display device.

7. In Paragraph 1, The above control unit is, The address for ordering is automatically set based on the IP address of the above-mentioned video display device, and Displaying only products available for order at the above address, Video display device.

8. In Paragraph 2, The above control unit is, Transmit the above text or keyword to a second external device, and Acquiring product information based on the text or keyword from the second external device, Video display device.

9. In a content viewing-based product recommendation and shopping linkage system, A video display device configured to display content; and It includes a server that processes information obtained from the above-mentioned image display device, and The above image display device is, Display content on the display so that the user can watch it, and Acquire the input signal of the above user, and In response to the above input signal, at least one of program information, screen capture information, and screen analysis information of the content displayed on the display is generated, and Transmit at least one of the above information to the server, and The above server is, Processing at least one of the received information to generate product information, and transmitting the product information to the image display device or an external electronic device, Content viewing-based product recommendation and shopping linkage system.

10. In Paragraph 9, The above server is, Deriving text or keywords from at least one piece of information using OCR, an object recognition algorithm, or an artificial intelligence model, and generating product information based on the text or keywords, Content viewing-based product recommendation and shopping linkage system.

11. In Paragraph 9, The above server is, Acquiring similar products of the same category as the above product information, and transmitting the similar products together with the above product information to the above image display device or the above external electronic device, Content viewing-based product recommendation and shopping linkage system.

12. In Paragraph 9, The above server is, When receiving product selection information from the above-mentioned video display device or the above-mentioned external electronic device, the products selected by the user are accumulated to create a list, and the list is transmitted to the above-mentioned video display device or the above-mentioned external electronic device. Content viewing-based product recommendation and shopping linkage system.

13. In Paragraph 9, The above server is, The address for ordering is automatically set based on the IP address of the above-mentioned video display device, and Generating product information that includes only information on products available for order according to the above address, and transmitting the product information to the above image display device or external electronic device, Content viewing-based product recommendation and shopping linkage system.

14. In Paragraph 9, The above server is, To build a personalized recommendation model for the above user, the above product information and content viewing information are combined and stored in a recommendation database. Content viewing-based product recommendation and shopping linkage system.

15. In Paragraph 9, The above server is, Receiving an order request or a payment request from the above-mentioned video display device or the above-mentioned external electronic device, and processing the order request or payment request in conjunction with an upper-level payment server or an external affiliate shopping mall server, Content viewing-based product recommendation and shopping linkage system.

16. Regarding content viewing-based product recommendation and shopping linkage methods, The operation of a video display device displaying content; The above video display device performs the operation of acquiring a signal from a user who is watching the content; The above-mentioned image display device performs the operation of generating at least one of program information, screen capture information, and screen analysis information of content in response to the signal; The above image display device performs the operation of transmitting the above at least one piece of information to a server; The operation of the above server processing the above one or more pieces of information to generate product information; and The operation of transmitting the above product information to the above image display device and / or external electronic device, Content viewing-based product recommendation and shopping methods.

17. In Paragraph 16, The operation of generating product information by processing one or more of the above information is, An operation of deriving text or keywords from at least one piece of information using OCR, an object recognition algorithm, or an artificial intelligence model; and including the operation of generating the product information based on the above text or keywords, Content viewing-based product recommendation and shopping methods.

18. In Paragraph 16, The operation of the above server receiving user's product selection information from the above image display device or the above external electronic device; The above server further includes the operation of generating a list by accumulating the products selected by the user in response to the user's product selection information, and transmitting the list to the image display device or the external electronic device. Content viewing-based product recommendation and shopping methods.

19. In Paragraph 16, The operation of generating product information by processing one or more of the above information is, Operation of automatically setting an address for ordering based on the IP address of the above-mentioned video display device; and Includes an operation of generating product information that includes only information about products available for order according to the above address, Content viewing-based product recommendation and shopping methods.

20. In Paragraph 16, The operation of the above server receiving an order request or a payment request from the above image display device or the above external electronic device; and The above server further includes an operation of processing the above order request or payment request in conjunction with a higher-level payment server or an external affiliate shopping mall server, Content viewing-based product recommendation and shopping methods.

Images