A display device and AI enhanced display method

Abstract

The present application provides a display device and an AI enhanced display method, which can obtain the current display image after receiving a control instruction input by the user to start the AI ​​enhancement algorithm, and execute the AI ​​enhancement algorithm on the current display image to display an enhanced screen image in the display to achieve AI image quality enhancement. At the same time, the recognition result data in the AI ​​enhancement algorithm is also extracted, and a prompt item is drawn according to the recognition result data to display an AI display window with a prompt item on the upper layer of the enhanced screen image. The method can compare and display the effects of the AI ​​enhancement algorithm through the AI ​​display window to show the AI ​​enhancement algorithm to the user and improve the user experience.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and an AI enhanced display method. Background Art

[0002] Display devices refer to terminal devices that can output specific display images, such as smart TVs, mobile terminals, smart advertising screens, projectors, etc. Taking smart TVs as an example, smart TVs can be based on Internet application technology, have open operating systems and chips, have open application platforms, can realize two-way human-computer interaction functions, and are TV products that integrate multiple functions such as audio and video, entertainment, and data to meet the diverse and personalized needs of users. For example, users can select any multimedia resource through the media selection interface to control the smart TV to play the corresponding media image for users to watch.

[0003] When playing multimedia resource images, smart TVs can use artificial intelligence (AI) algorithms to perform AI enhancement algorithms on the displayed images in real time to obtain better display quality. For example, when the displayed image contains a portrait, the smart TV can identify the portrait, and then perform color optimization on the displayed image according to the portrait recognition results, so as to finally display a display image that conforms to the characteristics of the portrait. However, since the AI ​​enhancement algorithm is used to improve the image quality, but the improvement in image quality is not easy for users to distinguish, it is difficult for users to know whether the currently played image has been processed by the AI ​​enhancement algorithm. Summary of the invention

[0004] The present application provides a display device and an AI enhanced display method to solve the problem that traditional display devices cannot display the effects of AI enhanced algorithms.

[0005] In one aspect, the present application provides a display device, comprising: a display and a controller. The display is configured to display a user interface; and the controller is configured to execute the following program steps:

[0006] Receiving a control instruction input by a user for starting an AI enhancement algorithm;

[0007] In response to the control instruction, acquiring a currently displayed image;

[0008] Executing an AI enhancement algorithm on the currently displayed image to generate an enhanced screen image and recognition result data;

[0009] The display is controlled to display an enhanced screen image, and an AI display window is displayed on an upper layer of the enhanced screen image, wherein the AI ​​display window includes prompt items drawn according to the recognition result data.

[0010] On the other hand, the present application also provides an AI enhanced display method, which is applied to a display device, wherein the display device includes a display and a controller, and the AI ​​enhanced display method includes:

[0011] Receiving a control instruction input by a user for starting an AI enhancement algorithm;

[0012] In response to the control instruction, acquiring a currently displayed image;

[0013] Executing an AI enhancement algorithm on the currently displayed image to generate an enhanced screen image and recognition result data;

[0014] The display is controlled to display an enhanced screen image, and an AI display window is displayed on an upper layer of the enhanced screen image, wherein the AI ​​display window includes prompt items drawn according to the recognition result data.

[0015] It can be seen from the above technical solutions that the display device and AI enhanced display method provided by the present application can obtain the current display image after receiving the control instruction input by the user for starting the AI ​​enhancement algorithm, and execute the AI ​​enhancement algorithm on the current display image to display the enhanced picture image in the display to achieve AI picture quality enhancement. At the same time, the recognition result data in the AI ​​enhancement algorithm is also extracted, and the prompt items are drawn according to the recognition result data to display the AI ​​display window with the prompt items on the upper layer of the enhanced picture image. The method can compare and display the effects of the AI ​​enhancement algorithm through the AI ​​display window to show the AI ​​enhancement algorithm to the user and improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the embodiments are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0017] Figure 1 It is a usage scenario of the display device in the embodiment of the present application;

[0018] Figure 2 This is a hardware configuration block diagram of the control device in the embodiment of the present application;

[0019] Figure 3 A hardware configuration diagram of a display device in an embodiment of the present application;

[0020] Figure 4 A software configuration diagram of a display device in an embodiment of the present application;

[0021] Figure 5 This is a schematic diagram of the AI ​​enhancement algorithm flow in the embodiment of this application;

[0022] Figure 6 This is a schematic diagram of the demonstration interface in the embodiment of this application;

[0023] Figure 7 This is a flow chart of the AI ​​enhanced display method in the embodiment of the present application;

[0024] Figure 8 A menu diagram is provided in the embodiment of the present application;

[0025] Fig. 9 This is a schematic diagram of switching from the demonstration interface to the AI ​​display window interface in the embodiment of the present application;

[0026] Fig.10 This is a schematic diagram of the process of calling the AI ​​model in the embodiment of this application;

[0027] Fig.11 This is a schematic diagram of a process of obtaining a currently displayed image according to a signal source in an embodiment of the present application;

[0028] Fig.12 This is a schematic diagram of an AI display window of multiple portraits in an embodiment of the present application;

[0029] Fig.13 This is a schematic diagram of the AI ​​display window of a single portrait in an embodiment of the present application;

[0030] Fig.14 It is a schematic diagram of the display process based on feature targets in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0032] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0033] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0034] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0035] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0036] Figure 1 Schematic diagram of a usage scenario of a display device according to an embodiment. Figure 1 As shown, the display device 200 also performs data communication with the server 400 , and the user can operate the display device 200 through the smart device 300 or the control apparatus 100 .

[0037] In some embodiments, the control device 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and at least one of other short-range communication methods, to control the display device 200 wirelessly or wired. The user may control the display device 200 by inputting user commands through at least one of buttons on the remote controller, voice input, and control panel input.

[0038] In some embodiments, the display device 200 also communicates data with the server 400. The display device 200 may be allowed to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactions to the display device 200. The server 400 may be one cluster or multiple clusters, and may include one or more types of servers.

[0039] In some embodiments, the software steps executed by a step execution subject can be migrated to another step execution subject in data communication with it as needed for execution. Exemplarily, the software steps executed by a server can be migrated to a display device in data communication with it as needed for execution, and vice versa.

[0040] Figure 2 Schematically shows a block diagram of a configuration of the control device 100 according to an exemplary embodiment. Figure 2 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive input operation instructions from the user, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200.

[0041] Figure 3A hardware configuration block diagram of a display device 200 according to an exemplary embodiment is shown.

[0042] In some embodiments, the display apparatus 200 includes at least one of a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0043] In some embodiments, the communicator 220 is a component for communicating with an external device or server according to various communication protocol types. For example, the communicator may include at least one of a Wifi module, a Bluetooth module, a wired Ethernet module, and other network communication protocol chips or near field communication protocol chips, and an infrared receiver. The display device 200 can establish transmission and reception of control signals and data signals with the control device 100 or the server 400 through the communicator 220.

[0044] In some embodiments, the external device interface 240 may include, but is not limited to, any one or more of the following: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by the above multiple interfaces.

[0045] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0046] In some embodiments, the controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 can perform operations related to the object selected by the user command.

[0047] In some embodiments, the user may input a user command in a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI). Alternatively, the user may input a user command by inputting a specific sound or gesture, and the user input interface receives the user input command by recognizing the sound or gesture through a sensor.

[0048] In some embodiments, "user interface" is a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. A common form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be an interface element such as an icon, window, or control displayed on a display screen of an electronic device, wherein a control may include at least one of icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, and other visual interface elements.

[0049] See also Figure 4 In some embodiments, the system is divided into four layers, including, from top to bottom, the application layer (Applications layer), the application framework layer (Application Framework layer), the Android runtime and system library layer (System runtime layer), and the kernel layer.

[0050] In some embodiments, at least one application is running in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. provided by the operating system, or applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0051] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0052] like Figure 4As shown, in the embodiment of the present application, the application framework layer includes managers, content providers, etc., wherein the manager includes at least one of the following modules: an activity manager (ActivityManager) is used to interact with all activities running in the system; a location manager (Location Manager) is used to provide system services or applications with access to system location services; a package manager (Package Manager) is used to retrieve various information related to the application package currently installed on the device; a notification manager (NotificationManager) is used to control the display and clearing of notification messages; a window manager (Window Manager) is used to manage icons, windows, toolbars, wallpapers, and desktop components on the user interface.

[0053] In some embodiments, the activity manager is used to manage the life cycle of each application and the common navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes (for example, reducing the display window, shaking the display, distorting the display, etc.).

[0054] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0055] In some embodiments, the kernel layer is a layer between hardware and software. Figure 4 As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0056] When displaying part of the picture, the display device 200 may perform image quality processing on the displayed picture. For example, when the display device 200 plays a video or displays a picture, the brightness, contrast, clarity, chromaticity and other parameters of the displayed picture may be adjusted to make the displayed picture have softer brightness, clearer picture effect and more layered color perception.

[0057] In the process of performing image quality processing on the display screen, the display device 200 can adjust different image parameters according to the specific content displayed. For example, for landscape images, the chromaticity on each RGB channel can be adjusted separately to reduce the inharmonious light sources in the image, so that the effect of the landscape image is more natural. For images containing portraits, the portrait targets can be distinguished and processed according to the skin color of the portraits, thereby reflecting a better sense of color hierarchy. Therefore, when the display device 200 performs image quality processing, it is necessary to first identify the content in the displayed picture, and perform an appropriate image parameter adjustment method according to the recognition result to obtain an enhanced picture that matches the content of the picture.

[0058] like Figure 5 As shown, the display device 200 can use artificial intelligence (AI) technology to identify the content of the displayed screen, and after obtaining the recognition result, call a specific parameter value combination to adjust the image parameters according to the specific parameter value combination. Among them, AI technology refers to the establishment of a neural network model, taking the current display image as input, and outputting the classification probability of the current display image for a specific result label after multi-layer convolution operations. Among them, the higher the output classification probability, the greater the possibility that a specific target is contained in the current display image. For example, a neural network model for identifying whether an image contains a portrait can be established in advance. When a picture with a portrait is input into the neural network model, the classification result can be output, that is, the classification probability of "having a portrait" in the output result is greater than the classification probability of "not having a portrait".

[0059] The neural network model can be obtained by repeatedly training the initial model based on sample data. For example, for a neural network model for recognizing portraits, the model output result can be obtained by inputting sample images with labels into the initial model. The error is then calculated based on the model output result and the label value, and the parameters in the model are adjusted according to the error back propagation. In this way, after inputting a large number of sample images, the initial model parameters can be continuously adjusted, and finally a neural network model with a high accuracy rate can be obtained.

[0060] In order to realize image recognition in different scenarios, the display device 200 can construct multiple different neural network models according to specific application fields, so that when the displayed picture is identified to include corresponding feature targets, the image parameters are adjusted according to the scenario to obtain different image quality processing effects. Figure 6As shown, a variety of image quality processing scenarios such as "AI portrait enhancement" and "AI color balance" can be built into the display device 200. When the user selects "AI portrait enhancement", the neural network model applicable to AI portrait enhancement can be called to identify the portrait target in the image. When the user selects "AI color balance", the neural network model applicable to AI color balance is called to identify the color in the image.

[0061] After the feature target in the image is identified through the neural network model, the display device 200 then executes an image parameter adjustment method that is compatible with the feature target to achieve enhanced image quality. For ease of description, in the embodiment of the present application, the AI ​​model recognition and image parameter adjustment process are collectively referred to as an AI enhancement algorithm. For example, in the AI ​​enhancement algorithm applied to portrait enhancement, the portrait in the image is first identified through the neural network model of portrait recognition, and then color distinction and processing are performed based on the portrait recognition results in the image, so as to obtain a more layered color experience.

[0062] In some embodiments, for the same application scenario, a more detailed image quality parameter adjustment may be adopted. For example, when a portrait target is identified in the image, the display device 200 may further analyze the shape, color and other parameters of the portrait target, thereby identifying the portrait posture, part, age, gender, skin color and other information, and then adopt different image quality adjustment methods according to different recognition results.

[0063] Therefore, in order to achieve more detailed image quality parameter adjustment, the AI ​​recognition model can also identify the location, shape, color and other information of the characteristic target in the input image. This recognized information can be collectively referred to as recognition result data, which can be used to perform different image quality parameter adjustment methods for different usage scenarios.

[0064] Based on the above AI enhancement algorithm, the display device 200 can adjust the image quality of the displayed image in real time during use. However, since the AI ​​enhancement algorithm is used to improve the image quality, in some usage cases, the improvement in image quality is not easy for users to distinguish, making it difficult for users to know whether the current playback image has been processed by the AI ​​enhancement algorithm. For example, when the displayed image includes multiple portraits, and the multiple portraits have different skin colors, the user cannot know the specific effect of the current image quality processing, which reduces the user experience.

[0065] To improve user experience, Figure 7As shown, in some embodiments of the present application, a display device 200 is provided, including: a display 260 and a controller 250. Among them, the display 260 is configured to display a user interface. The user interface can be further divided into a playback interface and a UI interface according to different specific usage scenarios. The playback interface can be presented as different screen contents according to different signal sources of the display device 200. The UI interface is presented as different display screens according to different interactive operations of the user. The controller 250 is configured to execute an AI enhanced display method for displaying the AI ​​enhanced effect so that the user can perceive the actual effect of the AI ​​enhanced algorithm. The AI ​​enhanced display method specifically includes the following contents:

[0066] Receive a control instruction input by the user for starting the AI ​​enhancement algorithm. In actual use, the display device 200 can receive various control instructions input by the user in real time. Among them, some control instructions can be used to start the AI ​​enhancement algorithm. The specific control instruction can be input according to the specific application scenario of the display device 200.

[0067] like Figure 8 As shown, the control instruction for starting the AI ​​enhancement algorithm can be input through the settings menu. For example, the user can use the control device 100 to control the display device 200 to display the settings interface. The settings interface may include a settings menu, and multiple setting options are set in the settings menu. When the user selects any setting option, the submenu of the corresponding function can be called up or the specific settings interface can be jumped to. Therefore, when the control instruction for starting the AI ​​enhancement algorithm is input through the settings menu, the user can control the focus cursor through the control device 100 to select the "Settings-Screen Settings-AI Picture Quality Enhancement" options in sequence to start the AI ​​enhancement algorithm.

[0068] like Fig. 9 As shown, the control instruction for starting the AI ​​enhancement algorithm can also be input by entering a specific interface. For example, a demonstration interface for displaying AI image quality enhancement can be set in the operating system of the display device 200. When the user controls the display device 200 to display the demonstration interface, a control instruction for starting the AI ​​enhancement algorithm can be automatically generated to trigger the display device 200 to execute the AI ​​enhanced display method.

[0069] The control instruction for starting the AI ​​enhancement algorithm can also be input by the display device 200 through the judgment of the operating state. For example, in the operating system of the display device 200, the AI ​​enhancement function can be turned on by default when playing media data such as movies and television. Based on this, the display device 200 can detect whether the media data is currently being played. When it is detected that the media data is being played, a control instruction for starting the AI ​​enhancement algorithm is automatically generated to trigger the display device 200 to perform image quality enhancement processing on the picture corresponding to the media data through the AI ​​enhancement algorithm.

[0070] In addition, the user can also input a control instruction for starting the AI ​​enhancement algorithm in other ways to trigger the display device 200 to perform the AI ​​enhancement display method. For example, the input of the control instruction is completed through a dedicated AI button, an intelligent voice system, an interactive control on a touch screen, etc.

[0071] After the display device 200 receives a control instruction for starting the AI ​​enhancement algorithm, the display device 200 can respond to the control instruction, call the neural network model corresponding to the AI ​​enhancement algorithm, and obtain the currently displayed image to perform image recognition on the currently displayed image through the called neural network model.

[0072] The AI ​​enhancement algorithm model may be called to call different neural network models according to different usage scenarios. For example, when the control instruction input by the user is completed by selecting the "AI portrait enhancement" option in the demonstration interface, the display device 200 calls a neural network model suitable for portrait recognition in response to the control instruction.

[0073] In some embodiments, Fig.10 As shown, in order to be able to call neural network models suitable for different usage scenarios, the display device 200 can also maintain a communication connection with the server 400, and after the user inputs a control instruction for starting the AI ​​enhancement algorithm, the control instruction is parsed to determine the current usage scenario. According to the parsing result, the display device 200 can first detect whether the corresponding neural network model is saved in the local memory. If a suitable neural network model is saved in the local memory, the model can be called directly; if a suitable neural network model is not saved in the local memory, a call request is sent to the server 400. After receiving the call request, the server 400 can feedback the neural network model suitable for the usage scenario to the display device 200 for use by the display device 200.

[0074] Through the interaction between the server 400 and the display device 200, there is no need to store multiple neural network models in the display device 200, and the training, maintenance, and update processes of the neural network model are all completed by the server 400, which not only reduces the storage space occupied by the display device 200, but also facilitates maintaining the effectiveness of the neural network model.

[0075] The display device 200 can obtain the currently displayed image by performing a screenshot operation on the displayed screen, or by extracting key frames from the displayed video data stream. For example, after the user inputs a control instruction for starting the AI ​​enhancement algorithm, the display device 200 can perform a screenshot operation on the currently displayed screen every 2 seconds, and save the content of the currently displayed screen in the form of an image file to obtain the currently displayed image. It is also possible to extract a frame of image every 2 seconds from the currently displayed video data stream as the currently displayed image.

[0076] After acquiring the current display image, the display device 200 can perform an AI enhancement algorithm on the current display image, that is, the current display image can be input into the called neural network model. After calculation by the neural network model, the recognition result data of the current display image can be input, and the image quality parameters of the current display image can be adjusted according to the recognition result data to generate an enhanced screen image.

[0077] For example, after acquiring the current display image, the display device 200 may input the current display image into a neural network model for AI portrait enhancement. The neural network model may execute an AI algorithm for the current display image to first identify whether the current display image contains a portrait target. If the current display image contains a portrait target, data such as the position, shape, and color of the portrait target may be extracted as recognition result data.

[0078] The display device 200 then runs the image quality adjustment program to adjust the image parameters such as brightness, contrast, clarity, and chromaticity of the currently displayed image based on the position, shape, color, and other data of the portrait target, thereby obtaining an enhanced picture image that conforms to the portrait display rules.

[0079] After generating the enhanced screen image, the controller 250 of the display device 200 can control the display 260 to display the enhanced screen image. Depending on the display purpose, the display 260 displays the enhanced screen image in different ways. For example, when the AI ​​enhancement algorithm is used for normal media playback, the display device 200 needs to replace the original video screen with the data after the enhanced screen image, or set the display channel gain according to the corresponding image parameters after the enhancement algorithm to present the enhanced display screen; and when the AI ​​enhancement algorithm is used for the demonstration interface, the enhanced screen image can be overlaid on the upper layer of the original display screen.

[0080] While displaying the enhanced screen image, the display device 200 may also display an AI display window on the upper layer of the enhanced screen image, wherein the AI ​​display window is used to display the results of the AI ​​enhancement algorithm so that the user can know or obtain an intuitive comparison result. Therefore, the AI ​​display window may include a prompt item drawn according to the recognition result data.

[0081] For example, Fig. 9 As shown in the figure, during the execution of the AI ​​portrait enhancement algorithm, when displaying the display picture after the picture quality adjustment, the head portrait box and the skin color indication window can be displayed on the upper layer of the enhanced picture image. Among them, the head portrait box can be displayed at the head position of the portrait in the image to indicate the recognized portrait; the skin color indication window is displayed in the lower right area of ​​the image to indicate the skin color corresponding to each portrait.

[0082] Image parameters can also be displayed on the right side of the skin color schematic window, including parameter values ​​such as brightness, contrast, clarity, and chromaticity. These image parameters can be specifically displayed through specific text and / or graphics. For example, each image parameter can be represented by a text name and a scroll bar to represent a specific value, that is, when the brightness is 50%, the active mark on the scroll bar is at the center of the scroll bar.

[0083] It can be seen that in the above embodiment, the recognition result data in the AI ​​enhancement algorithm is extracted, and prompt items are drawn according to the recognition result data, so as to display an AI display window with prompt items on the upper layer of the enhanced screen image. The AI ​​enhancement algorithm effect is compared and displayed through the AI ​​display window, so as to show the AI ​​enhancement algorithm to the user and improve the user experience.

[0084] In some embodiments, when a control instruction for starting an AI enhancement algorithm is inputted through the display device 200 into a specific display interface, the display device 200 may detect an interactive operation for entering a demonstration interface when the control instruction for starting an AI enhancement algorithm is not received, wherein the demonstration interface includes at least one AI enhancement option. After the display device 200 detects the interactive operation, it may control the display to display the demonstration interface in response to the interactive operation.

[0085] For example, the user can perform interactive operations through the control device 100, select the "Settings-Display Settings-AI Display" option in sequence, and control the display device 200 to enter the demonstration interface. In the demonstration interface, multiple function options can be displayed, including ULED multi-level detail enhancement, full HDR, SDR extreme dynamic range, AI portrait enhancement, AI color balance, intelligent motion detection, 120Hz high refresh rate, etc. The user can select any function option to control the display device 200 to demonstrate the corresponding function.

[0086] After the demonstration interface is displayed, a control instruction for starting the AI ​​enhancement algorithm is automatically input, that is, the neural network model corresponding to the AI ​​enhancement algorithm can be called, and the current display image can be obtained. In order to be able to call the corresponding neural network model, the display device 200 can also detect the selected instruction input by the user through the demonstration interface after displaying the demonstration interface. For example, the user can select one of the multiple function options through the display processing function to be viewed to control the display device 200 to display the display effect corresponding to the function option.

[0087] After the user selects any function option, the display device 200 can respond to the selection instruction and parse the focus option specified by the selection instruction. The focus option is the AI ​​enhancement option implemented by the selection instruction, that is, the function option selected by the user is determined. For example, when the user selects the AI ​​portrait enhancement option in the demonstration interface, the focus option is AI portrait enhancement, and the display device 200 can display the image quality processing effect of AI portrait enhancement in the demonstration interface.

[0088] After the user inputs the selection instruction, the display device 200 may also call the AI ​​enhancement algorithm model corresponding to the focus option to execute the AI ​​enhancement algorithm on the current display image through the AI ​​enhancement algorithm model. Fig. 9 As shown, when the user selects the AI ​​portrait enhancement option in the demonstration interface, the display device 200 can call the portrait recognition model and perform a recognition operation on the currently displayed image through the portrait recognition model to identify whether the currently displayed image contains a portrait target.

[0089] like Fig.11 As shown, in some embodiments, in order to obtain the currently displayed image, the display device 200 can also detect the current signal source of the display device in the step of obtaining the currently displayed image. Among them, the signal source refers to the source of the image currently displayed by the display device 200, and the signal source includes an external playback source and a local file playback source. The external playback source means that the display device 200 can connect to the Internet, digital TV signals or other terminal devices through the communicator 220, the external device interface 240, etc. to present the video picture. In the process of playing the video picture, the display device 200 can continuously obtain the video data stream, and parse and display the video data stream on the display 260. The local file playback source means that the display device 200 can parse and play the locally stored video files or picture files, and display the video files or picture files on the display 260.

[0090] After the current signal source of the display device 200 is detected, different methods can be used to obtain the current display image according to the specific form of the signal source. That is, if the current signal source is an external playback source, since the video data stream of most external playback sources does not support the extraction of each frame of the image, a screenshot operation can be performed on the current display screen to obtain the current display image. For example, when the display device 200 is connected to a digital set-top box via an HDMI interface and a live TV broadcast is performed, the display device 200 cannot directly extract the current display image from the live TV signal, so a screenshot operation can be performed on the display screen presented on the display 260 at this time to save the current display screen as a picture.

[0091] In order to obtain a better demonstration effect, the display device 200 can also perform a screenshot operation at a set frequency during the process of acquiring the currently displayed image. For example, the display device 200 can perform a screenshot operation every 2 seconds to acquire the currently displayed image, thereby executing the AI ​​enhancement algorithm on multiple images separated by 2s to detect whether the image contains a portrait. When the image contains a portrait, the image parameters can be adjusted to obtain an enhanced screen image, and finally a box mark is displayed at the position of the portrait head on the enhanced screen image for 5s. By setting the frequency of the screenshot operation, the followability of the box and the head of the portrait in the display screen can be controlled, while reducing the amount of data processing.

[0092] If the current signal source is a local file playback source, since the display device 200 can directly extract a picture file or any frame image in a video from a local file, in order to reduce the consumption of the data processing capability of the display device 200 during the screenshot process, the display device 200 can extract the currently playing media file, thereby extracting the currently displayed image from the media file.

[0093] For example, when the display device 200 displays a local picture file, the display device 200 can directly extract the local picture file to be displayed and directly input the local picture file into the neural network model to obtain AI recognition result data. When the display device 200 plays a local video file, the display device 200 can directly capture any frame of the video data stream corresponding to the local video file as the current display image.

[0094] Similarly, when the signal source of the display device 200 is a local file playback source, multiple frames of images in the local picture file or video data stream can also be extracted at a set frequency to achieve the same display effect as the external playback source, that is, to control the tracking of the box and the head of the portrait in the display screen, while reducing the amount of data processing.

[0095] In some embodiments, during the process of displaying the AI ​​display window on the upper layer of the enhanced screen image, the display device 200 may also draw a prompt item according to the recognition result data. The prompt item includes a first identifier and a second identifier, wherein the first identifier is used to indicate the position corresponding to the feature target in the current display image, for example, Fig.12 As shown, the first mark is a box shape, which is used to indicate the head position of the portrait in the current display image. The second mark is used to indicate the further recognition result of the feature target, which can be specific parameter information. For example, the second mark is the outline shape of a head, and the skin color of the portrait in the current display image is indicated by filling the skin color.

[0096] In order to draw the prompt item according to the recognition result data, the display device 200 may first parse the recognition result data to obtain the recognition target position and the target color value; and then add the first mark on the enhanced screen image according to the recognition target position.

[0097] For example, after the currently displayed image is input into the neural network model, the neural network model can perform portrait position recognition on images with portrait targets, thereby identifying the positions of five portrait targets in the currently displayed image, and further representing the heads of the five portrait targets through boxes, prompting the user with the AI ​​recognition results.

[0098] The position of the characteristic target in the recognition result can be represented by the x-axis coordinate information of the upper left corner of the recognized target, the y-axis coordinate information of the upper left corner of the recognized target, the width of the recognized target in the picture, that is, the length on the x-axis, and the height of the recognized target in the picture, that is, the length on the y-axis. That is, the specific position of the recognized target can be represented by Rect. Then Rect.left represents the X coordinate of the upper left corner of the target, Rect.top represents the Y coordinate of the upper left corner of the target, Rect.width represents the width of the target, and Rect.height represents the height of the target.

[0099] After adding the first identification to the enhanced screen image, the display device 200 may also add the second identification in the AI ​​display window in the same number as the first identification, and set the color of the second identification according to the target color value.

[0100] For example, Fig.12As shown, after identifying five portraits in the image, the display device 200 can draw an AI display window in the lower right corner and add five head outline patterns in the AI ​​display window. A box with the same color as the box corresponding to the first identifier can also be provided on the outside of the head outline, which is used to indicate the association between the head outline pattern and the head of the portrait in the image. Based on the skin color recognition result in the AI ​​algorithm, the average color value of the head area can be calculated based on the main color of the portrait head, and the color of the five head outline patterns can be set based on this color value. Obviously, when the skin colors of the five portraits are different, the colors of the head outline patterns displayed in the AI ​​display window are also different, so the AI ​​recognition results can be represented for user observation. As shown Fig.13 As shown, when a portrait is recognized in the image, a head outline pattern is added to the AI ​​display window.

[0101] In some embodiments, the prompt item further includes an information bar, which can be displayed in parallel with the pattern area of ​​the head outline in the AI ​​display window to represent the image parameters corresponding to the currently displayed image. Therefore, the display device 200 can detect the image parameters of the currently displayed image.

[0102] Among them, the image parameters include the brightness, contrast, clarity and chromaticity of the currently displayed image. The image parameters can be directly extracted from the image information, or they can be calculated by the AI ​​recognition algorithm. Obviously, depending on the form of the information bar finally displayed, the image parameter content detected from the currently displayed image is also different. For example, in a partial display process, only the parameter values ​​related to brightness and contrast can be displayed, so it is not necessary to obtain the parameter values ​​corresponding to clarity and chromaticity.

[0103] After detecting the image parameters of the currently displayed image, the display device 200 sets the display content of the information bar according to the image parameters. The display content of the information bar includes graphics and / or text for displaying the brightness, contrast, clarity and chromaticity. For example, the display content in the information bar includes "brightness" representing the name and a scroll bar representing the specific value. The brightness can also be represented by a sun-shaped graphic, while the specific brightness value is represented by a number.

[0104] After setting the display content in the information bar, the display device 200 can add the information bar in the demonstration window. The information bar can display the image information processed by the current AI enhancement algorithm, which can be convenient for comparison and viewing. In addition, the information bar can also include adjustment items, and the user can adjust a certain image parameter through the adjustment item to control the display device 200 to display the image screen according to the adjusted parameter in the future. For example, the user can increase or decrease the display brightness of the current enhanced screen image by dragging the active mark on the scroll bar.

[0105] Since the AI ​​enhancement algorithm is used to adjust the current display image of the display device 200, and since the number of frames of the current display image corresponding to the video screen is large, the data processing volume is high when executing the AI ​​enhancement algorithm, which increases the processing load of the controller 250. Therefore, in order to reduce the processing load of the controller 250, in some embodiments, when the display device 200 executes the AI ​​enhancement algorithm on the current display image, the current display image can be compressed into an image to be recognized. Obviously, the resolution of the image to be recognized is lower than the resolution of the current display image. For example, when the display device 200 plays a 4k high-definition film source, the resolution of the current display image can reach 4096×2160. This large-resolution image is processed slowly in the AI ​​recognition model, so the current display image can be compressed, and its resolution can be compressed to 1280×720 and then input into the neural network model.

[0106] By inputting the image to be identified into the AI ​​enhanced algorithm model, the recognition result data output by the AI ​​enhanced algorithm model can be obtained. The display device 200 then generates a background image based on the recognition result data. Since the recognition result data includes the recognition target position, and the recognition target position is used to mark it in the AI ​​display window through the first identifier, in order to correctly display the position of the first identifier in the final display screen, the display device 200 can generate a background image based on the recognition result data. The background image can be a transparent image, or an image with a translucent menu background in a specific area such as the lower right corner. According to the recognition target position, a plurality of marking points located at the recognition target position can be added to the background image, and finally the prompt item is displayed based on the position of the marking point.

[0107] In order to display correctly, the resolution of the background image is equal to the resolution of the UI layout of the display 260. For example, the display device 200 can generate a background image with a resolution of 1920×1080 according to the processing result of the AI ​​enhancement algorithm, and the resolution of the background image is equal to the resolution of the UI layout of the display device 200. Obviously, the resolution of the image to be recognized (1280×720) is smaller than the resolution of the background image (1920×1080).

[0108] By compressing the currently displayed image and generating a background image with the same resolution as the UI layout, the AI ​​display window can be displayed at the UI layout resolution while reducing the processing load of the controller 250, so that the position relationship in the final picture presented by the display device 200 remains correct, thereby improving the user experience.

[0109] In some embodiments, Fig.13As shown, in order to provide a better display effect, the display device 200 can identify feature targets in the current image when executing the AI ​​enhancement algorithm on the current display image. If the current display image includes the feature targets, the image parameters of the current display image are adjusted according to the feature targets to generate an enhanced screen image; if the current display image does not include the feature targets, the display is controlled to display the current display image.

[0110] For example, after acquiring the current display image, the display device 200 can identify whether the current display image contains a portrait by running an AI algorithm model. When it is identified that the current display image contains a portrait target, the parameters of the current display image are adjusted according to the skin color of the portrait. For example, for a portrait image with white skin color, the brightness and contrast can be reduced to achieve effective distinction and processing of the colors in the portrait area, so that the final image presented has a more layered sense. When it is identified that the current display image does not include a portrait target, the display device 200 determines that the current display image does not include a portrait, so the image quality processing algorithm for the portrait cannot obtain a good display effect, and therefore the display 260 can continue to be controlled to display the current display image.

[0111] In some embodiments, the display device 200 may also execute the AI ​​enhancement algorithm at different cycles for different currently displayed images. That is, if the currently displayed image includes a feature target, the display 260 is controlled to display the enhanced screen image for a preset time, and then the step of acquiring the currently displayed image is performed again. For example, after displaying the enhanced screen image for 5 seconds, the currently displayed screen is captured again, and the AI ​​display window is updated every 5 seconds, so that the user can compare and watch to know the display effect of the AI ​​enhancement algorithm.

[0112] If the currently displayed image does not include a feature target, the step of acquiring the currently displayed image may be performed again after a preset time. For example, when a portrait target is not recognized in the currently displayed image, the original image may be displayed for 5 seconds, and then the portrait target in the currently displayed image may be recognized again, so that the AI ​​display window can be displayed in time.

[0113] It should be noted that in order to obtain a better display effect, animation effects can also be added to the AI ​​display window. For example, when displaying a box for marking the position of a portrait head, a movable horizontal line can be added to the box. By moving the horizontal line up and down, a scanning effect can be formed to indicate the processing method of the current AI enhancement algorithm.

[0114] By setting multiple detection cycles, not only can the impact of the picture displayed by the display device 200 on the AI ​​enhancement result be alleviated, but also the AI ​​enhancement algorithm processing results for different time periods, that is, different display pictures, can be displayed so that users can see the AI ​​enhancement effects under different picture contents.

[0115] Based on the above display device 200, some embodiments of the present application further provide an AI enhanced display method, including:

[0116] Receiving a control instruction input by a user for starting an AI enhancement algorithm;

[0117] In response to the control instruction, acquiring a currently displayed image;

[0118] Executing an AI enhancement algorithm on the currently displayed image to generate an enhanced screen image and recognition result data;

[0119] The display is controlled to display an enhanced screen image, and an AI display window is displayed on an upper layer of the enhanced screen image, wherein the AI ​​display window includes prompt items drawn according to the recognition result data.

[0120] It can be seen from the above technical solution that the AI ​​enhanced display method provided in this embodiment can obtain the current display image after receiving the control instruction input by the user for starting the AI ​​enhancement algorithm, and execute the AI ​​enhancement algorithm on the current display image to display the enhanced screen image in the display to achieve AI image quality enhancement. At the same time, the recognition result data in the AI ​​enhancement algorithm is also extracted, and the prompt item is drawn according to the recognition result data to display the AI ​​display window with the prompt item on the upper layer of the enhanced screen image. The method can compare and display the effect of the AI ​​enhancement algorithm through the AI ​​display window to show the AI ​​enhancement algorithm to the user and improve the user experience.

[0121] Similar parts between the embodiments provided in this application can be referenced to each other. The specific implementation methods provided above are only a few examples under the general concept of this application and do not constitute a limitation on the protection scope of this application. For those skilled in the art, any other implementation methods expanded based on the scheme of this application without creative work belong to the protection scope of this application.

Claims

1. A display device, characterized in that: include: a display configured to display a user interface; The controller is configured as: Receiving a control instruction input by a user for starting an AI enhancement algorithm; In response to the control instruction, acquiring a currently displayed image; Executing an AI enhancement algorithm on the currently displayed image to generate an enhanced screen image and recognition result data; Control the display to display an enhanced picture image, and display an AI display window on the upper layer of the enhanced picture image, wherein the AI ​​display window includes prompt items drawn according to the recognition result data, and the prompt items include a first identifier and a second identifier and an information bar added according to the recognition result data, and the information bar is used to display image parameters after image parameter adjustment is performed according to the recognition result data, the first identifier is used to indicate the position corresponding to the feature target in the currently displayed image, and the second identifier is used to indicate the recognition result of the feature target in the currently displayed image.

2. The display device according to claim 1, characterized in that The controller is further configured to: Detecting an interactive operation for entering a demonstration interface, wherein the demonstration interface includes at least one AI enhancement option; In response to the interactive operation, controlling the display to display the demonstration interface; After the demonstration interface is displayed, a control instruction for starting the AI ​​enhancement algorithm is automatically input.

3. The display device according to claim 2, characterized in that The controller is further configured to: Detecting a selected instruction input by a user through the demonstration interface; In response to the selection instruction, parsing a focus option specified by the selection instruction, the focus option being an AI enhancement option implemented by the selection instruction; The AI ​​enhancement algorithm model corresponding to the focus option is called to execute the AI ​​enhancement algorithm on the currently displayed image through the AI ​​enhancement algorithm model.

4. The display device according to claim 1, characterized in that The controller is further configured to: In the step of acquiring the currently displayed image, a current signal source of the display device is detected, the signal source including an external playback source and a local file playback source; If the current signal source is an external playback source, performing a screenshot operation on the current display image at a set frequency to obtain the current display image; If the current signal source is a local file playback source, the currently played media asset file is extracted to extract the currently displayed image from the media asset file.

5. The display device according to claim 1, characterized in that The controller is further configured to draw a prompt item according to the recognition result data, and drawing the prompt item according to the recognition result data includes: Parsing the recognition result data to obtain the recognition target position and target color value; Adding the first mark on the enhanced screen image according to the identified target position; Adding the same number of the second identifiers as the first identifiers in the AI ​​display window; The color of the second mark is set according to the target color value.

6. The display device according to claim 5, characterized in that The controller is further configured to: Detecting image parameters of the currently displayed image, wherein the image parameters include brightness, contrast, clarity and chromaticity of the currently displayed image; The display content of the information bar is set according to the image parameters; the display content of the information bar includes graphics and / or text for displaying the brightness, contrast, clarity and chromaticity; Adds said information bar to the presentation window.

7. The display device according to claim 1, characterized in that The controller is further configured to: In the step of executing the AI ​​enhancement algorithm on the currently displayed image, compressing the currently displayed image into an image to be recognized; Inputting the image to be recognized into the AI ​​enhanced algorithm model to obtain recognition result data output by the AI ​​enhanced algorithm model; Generate a background image according to the recognition result data, wherein the background image includes a plurality of marking points located at the recognition target position; the resolution of the background image is equal to the UI layout resolution of the display, and the resolution of the image to be recognized is smaller than the resolution of the background image; The prompt item is displayed based on the position of the mark point.

8. The display device according to claim 1, characterized in that The controller is further configured to: In the step of executing the AI ​​enhancement algorithm on the currently displayed image, identifying a characteristic target in the currently displayed image; If the currently displayed image includes the characteristic target, adjusting the image parameters of the currently displayed image according to the characteristic target to generate an enhanced picture image; If the currently displayed image does not include the characteristic object, the display is controlled to display the currently displayed image.

9. The display device according to claim 8, characterized in that The controller is further configured to: If the currently displayed image includes the characteristic target, controlling the display to display the enhanced picture image for a preset time, and then executing the step of acquiring the currently displayed image again; If the currently displayed image does not include the characteristic target, the step of acquiring the currently displayed image is performed again after the preset time.

10. An AI enhanced display method, characterized in that: Applied to a display device, the display device includes a display and a controller, and the AI ​​enhanced display method includes: Receiving a control instruction input by a user for starting an AI enhancement algorithm; In response to the control instruction, acquiring a currently displayed image; Executing an AI enhancement algorithm on the currently displayed image to generate an enhanced screen image and recognition result data; Control the display to display an enhanced picture image, and display an AI display window on the upper layer of the enhanced picture image, wherein the AI ​​display window includes prompt items drawn according to the recognition result data, and the prompt items include a first identifier and a second identifier and an information bar added according to the recognition result data, and the information bar is used to display image parameters after image parameter adjustment is performed according to the recognition result data, the first identifier is used to indicate the position corresponding to the feature target in the currently displayed image, and the second identifier is used to indicate the recognition result of the feature target in the currently displayed image.

Images