Display device and window background display method

Abstract

This application provides a display device and a method for displaying a window background. The method, after a user inputs a control command to display a menu window, detects the color values ​​of multiple target points in the current user interface, selects a color sampling point from the detected color values ​​to calculate the background color, and sets the background of the menu window according to the color corresponding to the sampling point. This method can adjust the background color based on the content displayed in the bottom user interface when the menu window is displayed, making the background of the menu window more varied, improving the coordination between the menu window and the user interface, and enhancing the user experience.

Description

Technical Field

[0001] This application relates to the field of display device technology, and in particular to a display device and a method for displaying window background. Background Technology

[0002] Display devices refer to terminal devices capable of outputting specific display images, such as smart TVs, communication terminals, smart advertising screens, and projectors. Taking smart TVs as an example, smart TVs are television products based on Internet application technologies, possessing open operating systems and chips, and having open application platforms. They enable two-way human-computer interaction and integrate multiple functions such as audio-visual, entertainment, and data to meet diverse and personalized user needs.

[0003] Display devices can display various user interfaces (UIs) based on the operating system to facilitate user interaction. The user interface can present a menu window when the user inputs specific interactive actions. For example, when a user enables the image search function on the display device, a results window can be displayed in the lower area of ​​the user interface. This results window can include multiple options adapted to the image search function, such as image recognition results and a search item bar.

[0004] As a UI element, the menu window can follow the theme set in the operating system and have a specific background. For example, if the operating system's theme is blue, the menu window's background can also be blue. However, this limited background format, where the menu window's background clashes with the user interface color in some user interfaces, degrades the user experience. Summary of the Invention

[0005] This application provides a display device and a method for displaying window backgrounds to solve the problems of traditional window backgrounds being monotonous and the colors of the window backgrounds not matching those of the user interface.

[0006] On one hand, this application provides a display device, including: a display, a communicator, and a controller. The display is configured to display a user interface; the communicator is configured to establish a communication connection with a server; and the controller is configured to execute the following program steps:

[0007] Obtain user input for control commands used to display the menu window;

[0008] In response to the control command, detect the color values ​​of multiple target points in the current user interface;

[0009] Obtain the background color, which is generated by the controller or the server based on the color values ​​of color sampling points among multiple target points. The color sampling points are target points whose color difference value is less than a first color difference threshold, determined by comparing the color values ​​of multiple target points.

[0010] Set the background of the menu window according to the background color.

[0011] Optionally, the display device further includes a communicator configured to establish a communication connection with a server, and the controller is further configured to execute the following program steps:

[0012] After obtaining the background image, the background image is sent to the server to trigger the server to perform image recognition on the background image;

[0013] Receive the image recognition result fed back by the server for the background image, the image recognition result including feature targets;

[0014] Based on the image recognition result, a color sampling area is divided in the background image, and the color sampling area is the area where the feature target is located in the background image or the area outside the area where the feature target is located.

[0015] Multiple target points are added within the color sampling area, and the multiple target points are evenly distributed within the color sampling area.

[0016] On the other hand, this application also provides a window background display method, applied to the above-mentioned display device, the window background display method comprising the following steps:

[0017] Obtain user input for control commands used to display the menu window;

[0018] In response to the control command, detect the color values ​​of multiple target points in the current user interface;

[0019] Obtain a background color, which is generated based on the color values ​​of color sampling points among multiple target points. The color sampling points are target points whose color difference value is less than a first color difference threshold, determined by comparing the color values ​​of multiple target points.

[0020] Set the background of the menu window according to the background color.

[0021] As can be seen from the above technical solutions, the display device and window background display method provided in this application can detect the color values ​​of multiple target points in the current user interface after the user inputs a control command to display the menu window, and filter out the color sampling points used to calculate the background color from the detected color values, thereby setting the background of the menu window according to the color corresponding to the color sampling point. This method can adjust the background color according to the content displayed in the bottom user interface when displaying the menu window, making the background form of the menu window richer, improving the coordination between the menu window and the user interface, and enhancing the user experience. Attached Figure Description

[0022] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram illustrating the operational scenario between the display device and the control device in an embodiment of this application;

[0024] Figure 2 This is a hardware configuration block diagram of the display device in the embodiments of this application;

[0025] Figure 3 This is a schematic diagram of the display device software configuration in an embodiment of this application;

[0026] Figure 4 This is a schematic diagram showing the icon control interface of the display device application in an embodiment of this application;

[0027] Figure 5 This is a schematic diagram of the homepage in an embodiment of this application;

[0028] Figure 6 This is a schematic diagram of the image recognition interface in the embodiments of this application;

[0029] Figure 7 This is a schematic diagram of a menu window in an embodiment of this application;

[0030] Figure 8 This is a schematic diagram of another menu window in an embodiment of this application;

[0031] Figure 9 This is a schematic diagram of the window background display method in the embodiments of this application;

[0032] Figure 10 This is a schematic diagram of the target point in the embodiments of this application;

[0033] Figure 11 This is a schematic diagram of the process for selecting color sampling points in an embodiment of this application;

[0034] Figure 12 This is a schematic diagram of the process of dividing the color sampling area in an embodiment of this application;

[0035] Figure 13 This is a schematic diagram of the process of filling a color-defined area with color in an embodiment of this application. Detailed Implementation

[0036] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0037] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0038] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0039] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0040] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0041] Figure 1 This is a schematic diagram illustrating the operational scenario between the display device and the control unit according to the embodiment. Figure 1 As shown, the user can operate the display device 200 through the smart device 300 or the control device 100.

[0042] In some embodiments, the control device 100 may be a remote control. Communication between the remote control and the display device includes infrared protocol communication, Bluetooth protocol communication, and other short-range communication methods, controlling the display device 200 wirelessly or via wired means. Users can control the display device 200 by inputting user commands through buttons on the remote control, voice input, control panel input, etc.

[0043] In some embodiments, a smart device 300 (such as a communication terminal, tablet computer, computer, laptop computer, etc.) may also be used to control the display device 200. For example, an application running on the smart device may be used to control the display device 200.

[0044] In some embodiments, the display device 200 can also be controlled in ways other than the control device 100 and the smart device 300. For example, it can be controlled by directly receiving the user's voice commands through a module configured inside the display device 200 for acquiring voice commands, or it can be controlled by receiving the user's voice commands through a voice control device set outside the display device 200.

[0045] In some embodiments, the display device 200 also communicates with the server 400. The display device 200 may communicate via a local area network (LAN), wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactive features to the display device 200. The server 400 may be a cluster or multiple clusters, and may include one or more types of servers.

[0046] Figure 2 An exemplary block diagram of the configuration of the control device 100 according to an exemplary embodiment is shown. Figure 2 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive user input operation commands and convert the operation commands into commands that the display device 200 can recognize and respond to, thus acting as an intermediary for interaction between the user and the display device 200.

[0047] Figure 3 A hardware configuration block diagram of a display device 200 according to an exemplary embodiment is shown.

[0048] In some embodiments, the display device 200 includes at least one of a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0049] In some embodiments, the controller includes a processor, a video processor, an audio processor, a graphics processor, RAM, ROM, and a first interface to an nth interface for input / output.

[0050] In some embodiments, the display 260 includes a display screen component for presenting an image, a driving component for driving image display, a component for receiving image signals from the controller output, and a user control UI interface for displaying video content, image content, menu control interface, and user control UI interface.

[0051] In some embodiments, the display 260 may be a liquid crystal display, an OLED display, or a projection display, and may also be a projection device and a projection screen.

[0052] In some embodiments, the communicator 220 is a component used to communicate with external devices or servers according to various communication protocol types. For example, the communicator may include at least one of a Wi-Fi module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near-field communication protocol chips, and an infrared receiver. The display device 200 can establish the transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220.

[0053] In some embodiments, the user interface can be used to receive control signals from the control device 100 (e.g., an infrared remote control).

[0054] In some embodiments, detector 230 is used to acquire signals from the external environment or to interact with the outside world. For example, detector 230 includes a light receiver, a sensor for acquiring ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to acquire external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds.

[0055] In some embodiments, the external device interface 240 may include, but is not limited to, one or more interfaces such as: High Definition Multimedia Interface (HDMI), analog or data high-definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, etc. It may also be a composite input / output interface formed by multiple interfaces mentioned above.

[0056] In some embodiments, the tuner 210 receives broadcast television signals via wired or wireless reception and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0057] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0058] In some embodiments, the controller 250 controls the operation of the display device and responds to user operations via various software control programs stored in memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command to select a UI object to display on the display 260, the controller 250 can perform operations related to the object selected by the user command.

[0059] In some embodiments, the object can be any of the optional objects, such as a hyperlink, an icon, or other operable controls. Operations related to the selected object include: displaying links to hyperlinked pages, documents, images, etc., or performing operations corresponding to the program associated with the icon.

[0060] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), RAM (random access memory), ROM (read-only memory), a first to an nth interface for input / output, a communication bus, etc.

[0061] A CPU (CPU) processor is used to execute operating system and application instructions stored in memory, as well as various interactive instructions received from external input, to execute various applications, data, and content, ultimately for the display and playback of various audio and video content. A CPU processor can include multiple processors, such as a main processor and one or more sub-processors.

[0062] In some embodiments, a graphics processor is used to generate various graphical objects, such as icons, operation menus, and graphics displayed based on user input commands. The graphics processor includes an arithmetic logic unit (ALU) that performs calculations based on various user-input interactive commands and displays various objects according to display attributes; it also includes a renderer that renders the various objects obtained from the ALU, and the rendered objects are used to display on a monitor.

[0063] In some embodiments, the video processor is configured to receive external video signals and perform video processing such as decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, and image synthesis according to the standard encoding and decoding protocol of the input signals, so as to obtain a signal that can be directly displayed or played on the display device 200.

[0064] In some embodiments, the video processor includes a demultiplexing module, a video decoding module, an image compositing module, a frame rate conversion module, and a display formatting module. The demultiplexing module demultiplexes the input audio and video data streams. The video decoding module processes the demultiplexed video signal, including decoding and scaling. The image compositing module, such as an image synthesizer, overlays and blends a GUI signal generated by a graphics generator based on user input or its own generation with the scaled video image to generate a displayable image signal. The frame rate conversion module converts the input video frame rate. The display formatting module modifies the received frame rate-converted video output signal to conform to a display format, such as outputting RGB data signals.

[0065] In some embodiments, the audio processor is configured to receive external audio signals, and according to the standard codec protocol of the input signals, perform decompression and decoding, as well as noise reduction, digital-to-analog conversion, and amplification processing, to obtain a sound signal that can be played in a speaker.

[0066] In some embodiments, the user can input user commands through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input commands through the GUI. Alternatively, the user can input user commands by inputting specific sounds or gestures, and the user input interface receives the user input commands by recognizing the sounds or gestures through sensors.

[0067] In some embodiments, a "user interface" is the medium through which an application or operating system interacts and exchanges information with a user, converting information between its internal form and a form acceptable to the user. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be an icon, window, control, or other interface element displayed on the screen of an electronic device. Controls can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0068] In some embodiments, the display device's system may include a kernel, a command interpreter (shell), a file system, and applications. The kernel, shell, and file system together form the basic operating system structure, allowing users to manage files, run programs, and use the system. Upon power-up, the kernel starts, activates the kernel space, abstracts hardware, initializes hardware parameters, and runs and maintains virtual memory, the scheduler, signals, and inter-process communication (IPC). After the kernel starts, the shell and user applications are loaded. Applications are compiled into machine code after startup, forming a process.

[0069] See Figure 4 In some embodiments, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), the Android runtime and system library layer (referred to as the "System Runtime Layer"), and the kernel layer.

[0070] In some embodiments, at least one application runs in the application layer. These applications may be Windows programs, system settings programs, or clock programs that come with the operating system; they may also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0071] The framework layer provides application programming interfaces (APIs) and a programming framework for applications. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications within the application layer. Through the API, applications can access system resources and obtain system services during execution.

[0072] like Figure 4As shown, the application framework layer in this embodiment includes managers, content providers, etc., wherein the managers include at least one of the following modules: ActivityManager, which interacts with all activities running in the system; LocationManager, which provides access to system location services for system services or applications; PackageManager, which retrieves various information related to application packages currently installed on the device; NotificationManager, which controls the display and clearing of notification messages; and WindowManager, which manages icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0073] In some embodiments, the Activity Manager manages the lifecycle of individual applications and common navigation and back functions, such as controlling application exit, opening, and back actions. The Window Manager manages all window programs, such as obtaining the screen size, determining if a status bar is present, locking the screen, capturing the screen, and controlling display window changes (e.g., shrinking the display window, shaking the display, distorting the display, etc.).

[0074] In some embodiments, the system runtime library layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system runs the C / C++ libraries contained in the system runtime library layer to implement the functions that the framework layer needs to perform.

[0075] In some embodiments, the kernel layer is a layer between hardware and software. For example... Figure 4 As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver.

[0076] Based on the aforementioned display device 200, users can control the display device 200 to display various user interfaces during use. For example, ... Figure 5 As shown, display device 200 can display the homepage of different channels. Through operational configuration, different media assets can be displayed in the homepage sections for media recommendations. Users can select a section to expand or view the next level of the page. Users can also access the search page set on the display page to search for media assets. Users can also obtain feedback results directly through voice search.

[0077] As users perform interactive operations based on the user interface, the display device 200 can display a menu window on top of the user interface while displaying the user interface itself. The menu window is an extended interactive window that can contain multiple user interaction items.

[0078] For example, the display device 200 can perform image recognition operations on the currently displayed image or a specific image to be recognized based on user-input image recognition operations, to determine whether the image contains targets that meet specific characteristics, such as people, goods, buildings, etc. After performing the image recognition operation, the display device 200 can display a window in the user interface to present the image recognition results. That is, as... Figure 6 As shown, the menu window can include options such as links to people identified from the image and links to products for the user to choose from. When the user clicks on the person link option, the display device 200 can redirect to the person's details page or a media asset recommendation page related to the identified person.

[0079] It should be noted that, in this embodiment, the term "menu window" has a broad meaning and can refer to any UI window displayed based on the user interface. The menu window includes, but is not limited to: a recommendation bar, a results bar, a settings menu, a status bar, a quick settings menu, and a notification bar. In this embodiment, unless otherwise specified, the status bar can be any of the aforementioned UI windows.

[0080] The menu window can be displayed in different ways depending on its purpose. For example, Figure 7 As shown, the menu window for displaying image recognition results can be a bar-shaped frame displayed at the bottom of the user interface. The menu window for quick settings can be a rectangular frame displayed in the middle of the user interface, such as... Figure 8 As shown.

[0081] The items in the menu window can be displayed in a specific layout. For example, when the display device 200 displays a menu window for showing image recognition results, the product recognition results can be displayed in the left area of ​​the menu window, the recognized person image and name can be displayed in the right area, and a thumbnail of the recognized image or search terms can be displayed in the middle area of ​​the menu window, so that users can perform different interactive operations in different areas.

[0082] To highlight items in the menu window, the display device 200 can also set the background of the menu window according to a specific color or pattern. In some embodiments, the background of the menu window can be set according to the operating system theme of the display device 200. For example, when the operating system UI style of the display device 200 is blue, the display device 200 can set the menu window to a blue background when displaying the menu window.

[0083] However, when the display device 200 sets the background of the menu window based on the theme style of the operating system, due to the single display form of the theme style, when the menu window is displayed in some user interfaces, there is a large color difference between the menu window and the main color of the user interface, resulting in an uncoordinated display effect. Moreover, since the specific content displayed in the user interface changes with the user's operations, even for the same type of user interface, an uncoordinated display effect may also be caused when displaying some items. For example, when the display device 200 displays the media asset list interface, the items in the media asset list interface are generally patterns such as posters, covers, and screenshots of media asset items. And the pattern forms and color styles of different media asset items also vary greatly, further exacerbating this sense of incoordination.

[0084] In order to make the background form of the menu window more diverse and make the display effect of the menu window more coordinated, in some embodiments of the present application, a window background display method is also provided, which can be applied to the above-mentioned display device 200. Obviously, in order for the display device 200 to apply the above window background display method, in this embodiment, the display device 200 should at least include a display 260 and a controller 250. Among them, the display 260 is configured to display a user interface and display a menu window or other types of windows when the user inputs a specific interaction operation. The controller 250 can be configured to execute the window background display method to adjust the background content of the menu window according to the content of the user interface, such as Figure 9 shown, and specifically includes the following steps:

[0085] Obtain a control instruction input by the user for displaying the menu window. The display device 200 can respond to various control instructions input by the user and trigger the display of specific content according to the control instructions. Among them, some control instructions can be used to trigger the display device 200 to display the menu window. The user can input the control instruction for displaying the menu window through different interaction methods.

[0086] In some embodiments, the control instruction for displaying the menu window can be input through a button on the control device 100 supporting the display device 200. For example, the control device 100 can be provided with a screenshot key (or image recognition key). When the user presses the screenshot key, it can trigger the display device 200 to enable the image recognition function and enter the screenshot scenario. In the screenshot scenario, the image recognition application will first sense the screenshot event and then broadcast a screenshot command within the display device 200. After receiving the screenshot command, the business operating system of the display device 200 will perform screenshot and image recognition operations and trigger the display device 200 to display a menu window for presenting the image recognition result.

[0087] Obviously, for other types of menu windows, users can input information using other buttons on the control device 100. For example, the control device 100 can have a menu button directly installed; when the user presses the menu button, the display device 200 can be triggered to display the menu window. For some menu windows, they can also be triggered by specific button operations. For instance, when the focus cursor on the main control interface is at the top of the screen, the user can press the "up" arrow key on the control device 100 to trigger the display device 200 to display the status bar, which is equivalent to inputting control commands for displaying the menu window using buttons on the control device 100.

[0088] In addition to inputting control commands for displaying the menu window via buttons on the control device 100, in some embodiments, the display device 200 can also display various clickable (selectable) options in the user interface. When the user clicks on some options, the display device 200 can be triggered to display a menu window corresponding to the function of that option, i.e., inputting control commands for displaying the menu window.

[0089] In some embodiments, the display device 200 can also input control commands for displaying the menu window through a special form of interaction. For example, when the display device 200 has a built-in intelligent voice system, the user can input voice commands such as "display shortcut menu" or "image recognition" to trigger the display device 200 to perform the corresponding function and display the corresponding menu window, thus inputting control commands for displaying the menu window.

[0090] After the user inputs a control command to display the menu window, the display device 200 can respond to the input control command by detecting the color values ​​of multiple target points in the current user interface. Here, target points refer to multiple sampling points selected on the current user interface according to a specific arrangement rule.

[0091] In order to obtain color values, the display device 200 can generate a background image for extracting color information based on the currently displayed user interface. The background image can be obtained through screenshotting or parsing the displayed screen. For example, for a user interface displayed on a single layer, the display device 200 can directly extract the background image from the display screen of the layer containing the user interface.

[0092] In some embodiments, for user interfaces composed of multiple layers, in order to obtain the complete screen content, the display device 200 can perform screenshot processing on the final multi-layer screen fusion effect to generate a background image. That is, the display device 200 can perform a screenshot operation on the user interface, obtain a background image, and locate multiple target points in the background image to extract the color value of each target point. For example, after the user inputs a control command for image recognition, the display device 200 can respond to the control command by taking a screenshot of the currently displayed screen. The screenshot image can be used for image recognition and also as a background image to obtain the color values ​​of the target points.

[0093] While acquiring the background image, the display device 200 can also locate target points within the background image. Since the content displayed in the user interface can have different colors in different locations—for example, in a media asset list interface, the background area is generally a blue gradient pattern, so the main color extracted from the background area is blue—while the media asset item icons will display different colors depending on the specific pattern of the media asset cover. Therefore, to make the background of the menu window more coordinated with the display effect of the user interface, such as... Figure 10 As shown, the display device 200 needs to locate multiple target points in the background image. These target points allow for color detection in the background image at different locations.

[0094] The number of target points used to detect color values ​​can be set to a fixed value based on the screen size and resolution of the display device 200. For example, for a smart TV with a 2K display level (resolution of 1920×1080), the number of target points can be set to 20; for a smart TV with a 4K display level (resolution of 3840×2160), the number of target points can be set to 30.

[0095] The number of target points can also be adjusted in real time based on data such as the display device type, user interface type, and the content displayed in the user interface. For example, when the display device 200 is a reading device or displays a reading-type user interface, the number of target points can be reduced to decrease the amount of data processing because the content displayed in the reading interface is relatively simple and contains fewer colors.

[0096] It should be noted that, in the embodiments of this application, the color value refers to a numerical value or combination of numerical values ​​that can be used to represent a specific color. For example, the color value can be an RGB value, that is, a combination of the color values ​​corresponding to the red, green, and blue channels. However, for ease of subsequent judgment, the color value can also be a value converted from RGB values ​​using a specific weighted summation algorithm, i.e.:

[0097] C = W1 × R + W2 × G + W3 × B.

[0098] In the formula, C is the color value; W1, W2, and W3 are the calculated weights of the red, green, and blue color channels, respectively; R is the red channel value; G is the green channel value; and B is the blue channel value.

[0099] After obtaining the color value of the target point, the display device 200 can further filter out the color sampling points used to calculate the background color by comparing the color values ​​of multiple target points. To filter out the color sampling points from multiple target points, the display device 200 can pre-set filtering rules for the target points. For example, the display device 200 can run a filtering rule program to delete one or more points with the highest color value and one or more points with the lowest color value from the multiple target points, thereby using the remaining target points as color sampling points.

[0100] like Figure 11 As shown, in some embodiments, the display device 200 can also compare multiple target points to determine the color difference value between the multiple target points, and determine which target points are color sampling points based on the color difference value, that is, the color sampling points are target points whose color difference value is less than a first color difference threshold. The display device 200 can calculate the average value of the color values ​​of multiple target points; then calculate the variance of the color values ​​of each target point based on the average value to obtain the color difference value; then compare the color difference value with the first color difference threshold; if the color difference value is greater than or equal to the first color difference threshold, that is, the color corresponding to the current target point is significantly different from the color of other target points, the target point can be deleted; if the color difference value is less than the first color difference threshold, that is, the color corresponding to the current target point is less different from the color of other target points, the target point corresponding to the color difference value is marked as a color sampling point.

[0101] For example, after adding 20 target points P1, P2...P20 to a background image, you can first calculate the average color value of these 20 target points, i.e., the average value AV = (C1 + C2 + ... + C20) / 20. Then, based on the calculated average value AV, calculate the variance corresponding to each target point as the color difference judgment value, i.e., the color difference judgment value σ1 corresponding to target point P1 = (C1 - AV). 2 The color difference judgment value corresponding to the target point P2 is σ2 = (C2 - AV). 2 Finally, the color difference values ​​σ1, σ2...σ20 are compared with the first color difference threshold D0. When σ1 ≥ D0, it is determined that the color difference between point P1 and the other target points is relatively large, so target point P1 can be deleted. Similarly, following the same comparison method, target point P2 is compared. If σ2 < D0, it is determined that the color difference between point P2 and the other points is relatively small, so point P2 can be marked as a color sampling point. This process is repeated for each target point; by judging other target points, color sampling points with smaller color differences can be selected from the marked points.

[0102] After selecting the color pickers, the display device 200 can set the background of the menu window based on the color values ​​of the color pickers. When setting the menu window background, the specific color scheme can be determined based on the background format. For example, when the menu window uses a solid color background, the display device 200 can calculate the average value based on the color values ​​of the color pickers and use the color corresponding to the calculated average value as the solid color background color. When the menu window uses a gradient background, the display device 200 can set multiple area colors based on the location of the color pickers, and then render the background pattern of the entire menu window through gradient processing.

[0103] It should be noted that the process of selecting color points and setting the menu window background based on the color values ​​of the selected color points can also be completed by the server 400 connected to the display device 200. Specifically, in some embodiments, the display device 200 further includes a communicator 220, which is configured to establish a communication connection with the server 400. After the display device 200 detects the color values ​​of multiple target points in the current user interface in response to a control command, it can send the detected color values ​​to the server 400, which will then perform subsequent processing. That is, the server 400 can compare and judge the detected color values, select target points with color difference values ​​less than a first color difference threshold as color points, generate a background color based on the color values ​​of the selected color points, and send the generated background color to the display device 200, so that the display device 200 can set the background of the menu window based on the background color after obtaining it.

[0104] As can be seen from the above technical solutions, since the extracted color comes from the color sampling point with small color difference, the background set for the menu window based on the extracted color value is also consistent with the main color of the user interface. This allows the background of the menu window to change in real time according to the content displayed in the user interface, enriching the display form of the menu window and making the display effect of the menu window more coordinated with the user interface.

[0105] In some embodiments, the display device 200 can also perform image recognition on the user interface during the process of extracting the target point color value from the user interface, and set the setting position of the target point according to the image recognition result. That is, the display device 200 can also have a built-in image recognition model. After taking a screenshot of the user interface to obtain a background image, the background image can be input into the image recognition model so that the model can identify the specific content contained in the background image. For example, the image recognition model can identify whether the user interface contains characteristic targets such as people, goods, or buildings.

[0106] In some embodiments, the image recognition process can also be performed by the server 400. That is, Figure 11As shown, the server 400 can store an image recognition model. After the display device 200 obtains the background image, it can send the background image to the server 400 to trigger the server 400 to perform image recognition on the background image.

[0107] The specific method for sending the background image can be through an image recognition request. Display device 200 can generate an image recognition request based on the background image and send it to server 400. Upon receiving the request, server 400 parses it to obtain the background image to be recognized. This background image is then input into an image recognition model to determine whether it contains features such as people, goods, or buildings, and the specific locations of these features. After image recognition processing, server 400 sends the results back to display device 200.

[0108] After receiving the image recognition results from the server regarding the background image, the display device 200 divides the background image into color-selective areas based on the image recognition results. These color-selective areas are either the region containing the feature object or the region outside the feature object in the background image. The display device 200 can determine which region is the color-selective area based on user interface type, custom strategies, etc. For example, when the display device 200 displays a media asset list interface, since the cover images of the media asset items are not conducive to maintaining a harmonious effect, the location of the media asset item cover can be determined through image recognition, and the area outside the media asset item cover can be used as the color-selective area to minimize color differences within the color-selective area. When the display device 200 displays a media playback interface with buildings as the main subject, the location of the buildings can be determined through image recognition, and the location of the buildings can be used as the color-selective area to eliminate interference from content other than the buildings.

[0109] After dividing the color sampling area, the display device 200 adds multiple target points to the color sampling area. These target points can be evenly distributed within the color sampling area to obtain the dominant color tone within that area. This allows the display device 200 to render the menu window background according to the dominant color tone within the color sampling area. Therefore, by performing image recognition on the background image, the display device 200 can intelligently set the target point positions based on the content contained in the user interface, making the extracted color values ​​more consistent with the display characteristics of the user interface and resulting in a more harmonious final menu window background effect.

[0110] In some embodiments, while setting the target point position based on the image recognition result, the number of target points can also be determined based on the number of feature targets identified in the image recognition result. That is, in the step of adding multiple target points in the color sampling area, the display device 200 can iterate through the number of feature targets identified in the image recognition result, and then calculate the number of target points based on the number of feature targets, thereby adding target points in the color sampling area according to the number of target points.

[0111] The number of target points is linearly related to the number of feature targets. Clearly, when the color sampling area is the region where the feature targets are located, the number of target points is linearly positively correlated with the number of feature targets. That is, the more feature targets there are, the larger the proportion of the user interface area occupied by the feature targets, and therefore the more target points are set accordingly. Similarly, when the color sampling area is outside the region where the feature targets are located, the number of target points is linearly negatively correlated with the number of feature targets. That is, the more feature targets there are, the larger the proportion of the user interface area occupied by the feature targets, and the smaller the area outside the region where the feature targets are located, and therefore the fewer target points are set accordingly.

[0112] For example, in a media asset list interface, the display device 200 can use areas outside the media asset cover area as color sampling points. When the current user interface is identified to include n media asset items, the number of feature targets obtained is n. The display device 200 then uses the obtained number of feature targets to query the number m of target points required for the image corresponding to the current user interface, and evenly distributes m target points in the color sampling area.

[0113] In some embodiments, the number of target points added in the color sampling area can also be determined according to the resolution of the background image. That is, in the step of adding multiple target points to the color sampling area, the display device 200 can obtain the resolution of the background image, query the specification range to which the background image belongs based on the resolution, and then extract the number of target points in the specification range to which the background image belongs, thereby adding target points in the color sampling area according to the number of target points.

[0114] Obviously, when the background image has a higher display resolution, it can represent more content, thus requiring a larger number of target points to reflect the main content displayed in the user interface. Therefore, a database can be configured in the display device 200 or server 400 to store the number of target points needed for different resolutions. For example, when the background image resolution is 1920×1080, the database can be queried according to this resolution to determine that the number of target points is 20.

[0115] The two methods for setting the number of target points described above can be applied individually or in combination. That is, in some embodiments, after acquiring the background image, the display device 200 can first determine the required basic number of target points based on the resolution of the background image, and then adjust the basic number based on the image recognition results. In other words, the final number of target points to be set is determined based on the basic number.

[0116] In some embodiments, in the step of setting the background of the menu window according to the color values ​​of the color pickers, the display device 200 may first calculate the average value of the color values ​​of multiple color pickers; then render a solid color image according to the average value to add the solid color image to the display area of ​​the menu window; at the same time, the display device 200 may also calculate the foreground color according to the average value to render the items in the menu window according to the foreground color.

[0117] To facilitate clear display of items within the menu window, the foreground color should be easily distinguishable from the menu window's background color; that is, the difference between the foreground color value and the average value should be greater than a first color difference threshold. For example, if the average value calculated from multiple color sampling points is a blue-green (3, 245, 237), this color can be used as the solid background of the menu window. Simultaneously, the display device 200 can also determine a foreground color that is easily distinguishable by the user based on the color corresponding to the average value. For example, white (0, 0, 0) relative to a dark background or black (255, 255, 255) relative to a light background, and display the text and UI item graphics in the menu window according to the determined foreground color.

[0118] To determine the foreground color, the display device 200 can have at least two built-in color schemes, such as a dark scheme and a light scheme. Under each color scheme, the display device 200 can set a corresponding default foreground color scheme. After calculating the average value, the display device 200 can determine the color scheme to which the current background color belongs based on the average value and find the foreground color that matches that color scheme.

[0119] In some embodiments, the display device 200 can also add multiple colors to the background of the menu window, such as... Figure 12 As shown, in the step of setting the background of the menu window based on the color value of the color picker, the display device 200 can traverse the positions of the color-defined regions in the menu window. The color-defined region indicates the specific color displayed in a certain part of the menu window. Depending on the final effect, the display device 200 can set color-defined regions for each type of menu window; that is, a menu window can include at least one color-defined region. Furthermore, the colors displayed in different color-defined regions can be the same or different.

[0120] For example, the background color of the menu window popped up by the display device 200 can default to 10 color values. These 10 color values ​​correspond to the defined color areas of all RGB colors. The display device 200 can extract the RGB color value of each of the 20 points of the background color by taking a screenshot, and then remove values ​​that differ significantly from most colors. Then, it uses the average value to look up the corresponding relationship, thus determining the corresponding color value as the background color of the menu window.

[0121] The specific number and location of the color-defined areas can be determined based on the specific display functions of the menu window. For example, for an image recognition result display window, this window may include a product area on the left, a people area on the right, and a search area (or screenshot display area) in the middle. In this case, the display device 200 can set three color-defined areas for the menu window, respectively used to set the colors required for the left, right, and middle areas.

[0122] In some embodiments, the position of the color-fixed area can also be set according to the image recognition result, that is, the display device 200 can obtain the image recognition result of the user interface; then traverse the feature targets in the image recognition result to calculate the number of color-fixed areas according to the number of feature targets and / or the number of feature target types; and thus set the position of each color-fixed area within the menu window according to the number of color-fixed areas.

[0123] For example, when performing person recognition on a background image using an image recognition method, the display device 200 can determine the number of color-coded regions based on the number of recognized person targets after recognizing the person targets. That is, the number of color-coded regions can be set to be equal to the number of person targets or to satisfy a specific correlation.

[0124] After determining the location of the color-defined area, the display device 200 queries for associated target points among multiple target points according to the location of the color-defined area. Each color-defined area can have multiple associated target points. The specific association relationship can be determined based on the distance between the target point and the color-defined area. For example, corresponding to the left, right, and central areas on the menu window, the user interface can also be divided into regions to obtain the left associated region, the right associated region, and the central associated region. Then, the target points falling in each associated region are traversed to determine the associated target points of each color-defined area.

[0125] After retrieving the associated target points, the display device 20 can also calculate a fill color value based on the color value of the associated target points in each color-defined area, and fill the color in each color-defined area according to the fill color value. When there are multiple associated target points in a color-defined area, the fill color value can be set to the average of the color values ​​of the associated target points in each color-defined area.

[0126] In some embodiments, after determining the background color of the menu window, the display device 200 can further determine whether the determined background color is conducive to user interaction. That is, in the step of obtaining the background color, the display device 200 can compare the color values ​​of multiple target points to obtain color sampling points whose color difference value is less than a first color difference threshold, and then calculate the color to be rendered based on the color values ​​of the sampling points. The color to be rendered can be a single color or a group of colors, depending on the background rendering method. For example, for a solid color background, the color to be rendered is the single color corresponding to the average of the color values ​​of multiple sampling points; while for a background including multiple fixed-color regions, the color to be rendered represents a color corresponding to each fixed-color region.

[0127] After calculating and obtaining the color to be rendered, the display device 200 can also compare the color to be rendered with preset extreme colors. The preset extreme colors can be several colors determined based on the UI style of the display device 200's operating system, which are unfavorable for display and interactive operations, such as pure white, pure red, and pure black. The preset extreme colors can also include other colors that are the same as or too similar to the colors of content in specific menus, such as text and graphics.

[0128] By comparing the color values ​​corresponding to the color to be rendered and the preset extreme color, the display device 200 can determine whether the color to be rendered is the same as or similar to the preset extreme color, thereby determining whether the color to be rendered is suitable for display. If the color difference between the color to be rendered and the preset extreme color is less than a second color difference threshold, that is, the rendered color is the same as or similar to the preset extreme color, which is not conducive to display, then a preset selected color can be used as the background color instead of the color to be rendered. Similarly, if the color difference between the color value of the color to be rendered and the value of the preset extreme color is greater than or equal to the second color difference threshold, that is, the rendered color is not the same as or similar to the preset extreme color, which is conducive to display and user interaction, then the color to be rendered can be directly used as the background color for rendering.

[0129] In some embodiments, extreme colors can be labeled using color coordinate ranges. That is, after obtaining the color to be rendered, if the difference between the color coordinates of the color to be rendered and the color coordinates of the extreme colors meets a preset condition, then the color difference between the two is considered to meet a second color difference threshold.

[0130] In some embodiments, the color coordinates of extreme colors and the second color difference threshold can be used to define the color coordinate range. After obtaining the color to be rendered, if the color coordinates of the color to be rendered are not within the defined color coordinate range, it indicates that the color difference between the two satisfies the second color difference threshold.

[0131] In some embodiments, RGB channel data can be used to calculate color differences.

[0132] In some embodiments, when the control command for displaying the menu window triggers the display device 200 to set the background of the menu window using the method described in the above embodiments, the display device 200 can also render the foreground content of the menu window in response to the control command input by the user. For example, after the user presses the image recognition button on the control device 100, the display device 200 can receive the image recognition result of the current user interface screenshot from the server 400 in response to the control command when the image recognition button is pressed, so as to obtain the foreground image of the menu window.

[0133] The foreground image includes the image recognition results of the currently displayed screen, including feature target information identified from the user interface, such as person names, building names, and product names. The foreground image may also include related recommended items determined based on the image recognition results, including media assets, web links, and detail text associated with the feature target information.

[0134] After acquiring the foreground image, the display device 200 can also extract the background color determined in the above embodiments to render the menu window to be presented. During the rendering of the menu window, the display device 200 can first create a display layer above the layer where the user interface is located, then call the display template of the menu window, and add the foreground image and background color to the display template to generate the menu image, and display the menu image in the display layer.

[0135] For example, if the user interface presented by display device 200 can be displayed on the video layer, then the image recognition menu window can be displayed through the OSD layer located above the video layer. After the user inputs an image recognition command, display device 200 can respond to the command by calling the OSD layer to render the interface. During the rendering process, display device 200 can call the display template of the image recognition menu window and obtain the image recognition results output by the image recognition model and the related recommendation item data fed back by server 400. Finally, based on the display template and the image recognition results, the specific menu window screen is rendered and displayed on the OSD layer.

[0136] Based on the above-described window background display method, in some embodiments of this application, a display device 200 is also provided. The display device 200 includes a display 260 and a controller 250, wherein the display 260 is used to display a user interface, and the controller 250 is configured to execute the following program steps:

[0137] Obtain user input for control commands used to display the menu window;

[0138] In response to the control command, detect the color values ​​of multiple target points in the current user interface;

[0139] The color values ​​of multiple target points are compared to filter out the color sampling points used to calculate the background color. The color sampling points are the target points whose color difference value is less than a first color difference threshold.

[0140] The background of the menu window is set according to the color value of the color picker.

[0141] As can be seen from the above technical solutions, the display device and window background display method provided in the above embodiments can detect the color values ​​of multiple target points in the current user interface after the user inputs a control command to display the menu window, and filter out the color sampling points used to calculate the background color from the detected color values, thereby setting the background of the menu window according to the color corresponding to the color sampling point. The method can adjust the background color according to the content displayed in the bottom user interface when displaying the menu window, making the background form of the menu window richer, improving the coordination between the menu window and the user interface, and enhancing the user experience.

[0142] Similar parts between the embodiments provided in this application can be referred to mutually. The specific implementation methods provided above are only a few examples under the overall concept of this application and do not constitute a limitation on the scope of protection of this application. For those skilled in the art, any other implementation methods extended from the solution of this application without creative effort shall fall within the scope of protection of this application.

Claims

1. A display device, characterized in that, include: The monitor is configured to display the user interface; The communicator is configured to establish a communication connection with the server. The controller is configured as follows: Obtain user input for control commands used to display the menu window; In response to the control command, the color values ​​of multiple target points in the user interface are detected; Calculate the color difference value of the multiple target points based on their color values; Mark the target points whose color difference values ​​are less than the first color difference threshold as color sampling points; When the menu window has a solid color background, calculate the average value based on the color value of the color picker, and set the background color of the menu window to the color corresponding to the average value; When the menu window has a gradient background, colors are set for multiple areas based on the color value of the color picker and the location of the color picker. The colors corresponding to the multiple regions are subjected to gradient processing to obtain a gradient pattern; Set the background pattern of the menu window to the gradient pattern.

2. The display device according to claim 1, characterized in that, The controller is further configured to: In the step of detecting the color values ​​of multiple target points in the user interface, a screenshot operation is performed on the user interface to obtain a background image; Multiple target points are located in the background image to extract the color value of each target point.

3. The display device according to claim 2, characterized in that, The controller is further configured to: In the step of locating multiple target points in the background image, the background image is sent to the server to trigger the server to perform image recognition on the background image; Receive the image recognition result fed back by the server for the background image, the image recognition result including feature targets; Based on the image recognition result, a color sampling area is divided in the background image, and the color sampling area is the area where the feature target is located in the background image or the area outside the area where the feature target is located. Multiple target points are added within the color sampling area, and the multiple target points are evenly distributed within the color sampling area.

4. The display device according to claim 3, characterized in that, The controller is further configured to: In the step of adding multiple target points within the color sampling area, the number of feature targets identified in the image recognition result is traversed; The number of target points is calculated based on the number of feature targets, and the number of target points is linearly related to the number of feature targets; Add the target points in the color sampling area according to the stated number of target points.

5. The display device according to claim 3, characterized in that, The controller is further configured to: In the step of adding multiple target points within the color sampling area, the resolution of the background image is obtained; Based on the resolution, query the specification range to which the background image belongs; Extract the number of target points within the specification range to which the background image belongs; Add the target points in the color sampling area according to the stated number of target points.

6. The display device according to claim 1, characterized in that, The controller is further configured to: After setting the background color of the menu window to the color corresponding to the average value, the foreground color is calculated based on the average value, and the difference between the color value of the foreground color and the average value is greater than the first color difference threshold. The items in the menu window are rendered according to the foreground color.

7. The display device according to claim 1, characterized in that, The controller is further configured to: In the step of setting the background color of the menu window to the color corresponding to the average value, the image recognition result of the user interface is obtained; Based on the number of feature targets and / or the number of feature target types in the image recognition result, the number of color-defined regions is calculated, where the feature targets are used to characterize the image content in the user interface; Set the position of the color-fixed area within the menu window according to the number of color-fixed areas; Based on the location of the defined color area, query related target points among multiple target points; A fill color value is calculated based on the color value of the associated target point of each color-defined region, wherein the fill color value is the average value of the color values ​​of the associated target point of each color-defined region. Fill each of the defined color areas with color according to the fill color value.

8. The display device according to claim 1, characterized in that, The controller is further configured to: In the step of setting the background color of the menu window to the color corresponding to the average value... When the color difference between the average value and the preset extreme color is less than the second color difference threshold, the background color of the menu window is set to the preset selected color; When the color difference between the average value and the preset extreme color value is greater than or equal to the second color difference threshold, the background color of the menu window is set to the color corresponding to the average value.

9. The display device according to claim 1, characterized in that, The controller is further configured to: In response to the control command, the foreground screen of the menu window is obtained, the foreground screen including the image recognition result of the currently displayed screen and the recommended items associated with the image recognition result; The menu window's display template is invoked, and the foreground image and background color are added to the display template to generate the menu image; Create a display layer above the layer containing the user interface; The menu screen is displayed in the display layer.

10. A method for displaying a window background, characterized in that, include: Obtain user input for control commands used to display the menu window; In response to the control command, the color values ​​of multiple target points in the user interface are detected; Calculate the color difference value of the multiple target points based on their color values; Mark the target points whose color difference values ​​are less than the first color difference threshold as color sampling points; When the menu window has a solid color background, calculate the average value based on the color value of the color picker, and set the background color of the menu window to the color corresponding to the average value; When the menu window has a gradient background, colors are set for multiple areas based on the color value of the color picker and the location of the color picker. The colors corresponding to the multiple regions are subjected to gradient processing to obtain a gradient pattern; Set the background pattern of the menu window to the gradient pattern.

Images