Display equipment and picture display method thereof

Abstract

The invention relates to display equipment and a picture display method thereof, and relates to the technical field of display. The display device includes a display configured to display content; the user input interface is configured to obtain a picture display instruction which is input by a user and aims at the original picture; the at least one controller is connected with the display and is configured to obtain display demand data in response to the picture display instruction; according to the display demand data, generating a derivative picture corresponding to the original picture; and controlling the display to display the derivative picture under the condition that the derivative picture meets the preset evaluation requirement. By adopting the technical scheme, the picture display effect is improved on the basis of ensuring the conformity of the picture displayed by the display and the original picture.

Description

Technical Field

[0001] The present application relates to the field of display technology, and in particular to a display device and a method for displaying images thereof. Background Art

[0002] At present, with the continuous development of technology, the functions of display devices are becoming more and more diversified. For example, today's display devices have the function of displaying local specified pictures or pictures uploaded by external devices (such as mobile phones).

[0003] When using a display device to display pictures captured by an image acquisition device, since there are certain differences between the resolution and aspect ratio of the display screen in the display device and the resolution and aspect ratio corresponding to the image acquisition device, traditional technologies usually display the pictures captured by the image acquisition device in the display device by directly stretching the pictures or directly copying the pictures multiple times for tiled display.

[0004] Since directly stretching the image will damage the image ratio, and the method of copying the image multiple times and displaying it in tiles will have problems such as a large amount of content duplication, the process of displaying the image captured by the image acquisition device through the display device will have poor display effects, affecting the user's viewing experience. Summary of the invention

[0005] The present application provides a display device and a method for displaying images thereof, so as to solve the technical problem of poor image display effect through a display device in the prior art.

[0006] In a first aspect, the present application provides a display device, including:

[0007] A display configured to display content;

[0008] A user input interface, configured to obtain a picture display instruction for an original picture inputted by a user;

[0009] At least one controller is connected to the display and is configured to:

[0010] In response to the picture display instruction, obtaining display requirement data;

[0011] Generate a derivative image corresponding to the original image according to the display requirement data;

[0012] When the derived image meets the preset evaluation requirements, the display is controlled to display the derived image.

[0013] The display device provided in the above-mentioned embodiment performs image processing by introducing a controller with an image derivation function in the display device. Specifically, in the processing process, the display requirement data corresponding to the original image is introduced to generate a derivative image corresponding to the original image. Since the derivative image is generated based on the original image, the generated image can be consistent with the original image. In addition, in the process of generating the derivative image, the display requirement data is referred to, which makes the generated derivative image look better compared with the traditional method of mechanically repeating and copying the image or adjusting the image size. Furthermore, when the derivative image meets the preset evaluation requirements, the derivative image is used to replace the original image and displayed on the display, so that the displayed image can meet the preset evaluation requirements, further improving the display effect of the displayed image.

[0014] In a second aspect, the present application also provides a method for displaying an image, comprising:

[0015] In response to a picture display instruction for an original picture, acquiring display requirement data;

[0016] Generate a derivative image corresponding to the original image according to the display requirement data;

[0017] When the derived image meets the preset evaluation requirements, the display is controlled to display the derived image.

[0018] The image display method provided in the above embodiment generates a derivative image corresponding to the original image by introducing the display requirement data corresponding to the original image. Since the derivative image is generated based on the original image, the generated image can be consistent with the original image. In addition, in the process of generating the derivative image, the display requirement data is referred to, which makes the generated derivative image look better compared with the traditional method of mechanically copying the image or adjusting the image size. Furthermore, when the derivative image meets the preset evaluation requirements, the derivative image is used to replace the original image and displayed on the display, so that the displayed image can meet the preset evaluation requirements, further improving the display effect of the displayed image.

[0019] In a third aspect, the present application further provides a picture display device, comprising:

[0020] A data acquisition module, used for acquiring display requirement data in response to a picture display instruction for an original picture;

[0021] The picture derivation module is used to generate a derivative picture corresponding to the original picture according to the display requirement data;

[0022] The display control module is used to control the display to display the derived image when the derived image meets the preset evaluation requirements.

[0023] The image display device provided in the above embodiment introduces an image derivation module to generate a derivative image corresponding to the original image according to the display requirement data corresponding to the original image. Since the derivative image is generated based on the original image, the generated image can be consistent with the original image. In addition, in the process of generating the derivative image, the display requirement data is referred to, which makes the generated derivative image look better compared with the traditional method of mechanically copying the image or adjusting the image size. Furthermore, by introducing a display control module, when the derivative image meets the preset evaluation requirements, the derivative image is used to replace the original image and displayed on the display, so that the displayed image can meet the preset evaluation requirements, further improving the display effect of the displayed image.

[0024] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

[0025] In response to a picture display instruction for an original picture, acquiring display requirement data;

[0026] Generate a derivative image corresponding to the original image according to the display requirement data;

[0027] When the derived image meets the preset evaluation requirements, the display is controlled to display the derived image.

[0028] The readable storage medium provided by the above embodiment generates a derivative image corresponding to the original image by introducing the display requirement data corresponding to the original image during the execution of the stored computer program. Since the derivative image is generated based on the original image, the generated image can be consistent with the original image. In addition, in the process of generating the derivative image, the display requirement data is referred to, which makes the generated derivative image look better compared with the traditional method of mechanically copying the image or adjusting the image size. Furthermore, when the derivative image meets the preset evaluation requirements, the derivative image is used to replace the original image and displayed on the display, so that the displayed image can meet the preset evaluation requirements, further improving the display effect of the displayed image.

[0029] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0030] In response to a picture display instruction for an original picture, acquiring display requirement data;

[0031] Generate a derivative image corresponding to the original image according to the display requirement data;

[0032] When the derived image meets the preset evaluation requirements, the display is controlled to display the derived image.

[0033] The computer program provided in the above embodiment generates a derivative image corresponding to the original image by introducing the display requirement data corresponding to the original image during the execution of the computer program. Since the derivative image is generated based on the original image, the generated image can be consistent with the original image. In addition, in the process of generating the derivative image, the display requirement data is referred to, so that the generated derivative image has a better look and feel compared with the traditional method of mechanically repeating and copying the image or adjusting the image size. Furthermore, when the derivative image meets the preset evaluation requirements, the derivative image is used to replace the original image and displayed on the display, so that the displayed image can meet the preset evaluation requirements, further improving the display effect of the displayed image. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0036] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0037] Figure 3 A schematic diagram of the hardware configuration of a control device provided in some embodiments of the present application;

[0038] Figure 4 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0039] Figure 5 A schematic diagram of a process of displaying pictures provided in some embodiments of the present application;

[0040] Figure 6 A schematic diagram of a flow chart of steps for generating a derivative image provided in some embodiments of the present application;

[0041] Figure 7 A schematic diagram of a flow chart of the updating steps of the picture generation model provided in some embodiments of the present application;

[0042] Fig. 8AA state machine diagram of a picture display process provided for some embodiments of the present application;

[0043] Figure 8B A schematic diagram of a process of displaying an image provided by some embodiments of the present application;

[0044] Figure 8C A schematic diagram of an original image provided for some embodiments of the present application;

[0045] Fig.8D A schematic diagram of a finally generated derivative image provided in some embodiments of the present application;

[0046] Fig. 9 is a structural block diagram of a picture display device in an embodiment;

[0047] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0048] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0049] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0050] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0051] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0052] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0053] In the embodiment of the present application, the display device 200 generally refers to a device with image display and data processing capabilities. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0054] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , the user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. For example, the control device 100 can be a remote controller, a stylus pen, a handle, etc.

[0055] The mobile terminal 300 can be used as a control device for performing human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device for establishing a communication connection with the display device 200 and performing data interaction. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0056] like Figure 1 It is also shown that the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN) and other networks.

[0057] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function with a computer support function, including but not limited to network television, smart television, Internet Protocol Television (IPTV), and the like.

[0058] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0059] In some embodiments, the display device 200 may include at least one of a tuner 210 , a communication device 220 , a detector 230 , a device interface 240 , a controller 250 , a display 260 , an audio output device 270 , a memory, a power supply, and a user input interface 280 .

[0060] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., which is used to receive external sounds, such as collecting user voice data.

[0061] In some embodiments, the display 260 includes a display function component for presenting a picture, and a driving component for driving an image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface and a user control UI interface (User Interface user interface).

[0062] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communication device 220 including a WiFi (Wireless Fidelity) function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0063] The communication device 220 can communicate with the display device 200 and the external device or server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 to the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 to the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0064] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0065] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0066] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0067] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0068] In some embodiments, the user input interface 280 may be used to receive instructions from user input, such as a picture display instruction for an original picture in the present application.

[0069] Figure 3 Some embodiments of the present application provide Figure 1 The hardware configuration diagram of the control device in the figure. Figure 3 As shown, the control device 100 may include: a controller 110 , a communication interface 130 , a user input / output interface 140 , a power supply 180 , and a memory 190 .

[0070] The control device 100 is configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0071] In some embodiments, the control device 100 may be a smart device, for example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0072] In some embodiments, Figure 1 As shown, the mobile terminal 300 or other intelligent electronic devices can play a similar function as the control device 100 after installing the application for controlling the display device 200 .

[0073] The controller 110 includes a processor 112, a RAM (Random Access Memory) 113, a ROM (Read-Only Memory) 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between the internal components and the external and internal data processing functions.

[0074] The communication interface 130 implements communication of control signals and data signals with the display device 200 under the control of the controller 110. The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131, a Bluetooth module 132, and an NFC (Near Field Communication) module 133.

[0075] The user input / output interface 140 , wherein the input interface includes at least one of other input interfaces such as a microphone 141 , a touch panel 142 , a sensor 143 , and a button 144 .

[0076] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with a communication interface 130, such as a WiFi, Bluetooth, NFC or other module, and can encode the user input command through the WiFi protocol, Bluetooth protocol, or NFC protocol and send it to the display device 200.

[0077] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user. Alternatively, further, the memory 190 can also be configured to locally store the original images in this application.

[0078] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0079] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system may (control the display device) provide a user interface, allow the user to interact with the display device 200, and support the running of various application programs.

[0080] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0081] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 4 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications layer) (referred to as "application layer"), the application framework layer (Application Framework layer) (referred to as "framework layer"), the system library layer and the kernel layer.

[0082] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0083] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions of applications in the application layer. Applications can access system resources and obtain system services during execution through the API interface.

[0084] like Figure 4 As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager is used to interact with all activities running in the system; a location manager is used to provide system services or applications with access to system location services; a package manager is used to retrieve various information related to the application package currently installed on the device; a notification manager is used to control the display and clearing of notification messages; and a window manager is used to manage icons, windows, toolbars, wallpapers, and desktop components on the user interface.

[0085] In some embodiments, the activity manager is used to manage the life cycle of each application and the usual navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes, for example, reducing the display window, shaking the display, distorting the display, etc.

[0086] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0087] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. Figure 4 As shown, the kernel layer may be configured with hardware drivers, and the drivers included in the kernel layer may be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB (Universal Serial Bus) driver, HDMI (High-Definition Multimedia Interface) driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0088] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0089] When the display device 200 displays pictures taken by other devices, the picture display effect is different from the display effect on the picture acquisition device due to the different resolutions and aspect ratios of the display screen. For example, when the pictures collected by a mobile terminal (such as a mobile phone) are displayed on a display device such as a TV or a computer, the corresponding resolutions are different due to the different sampling of different devices. In this scenario, the pictures collected by the mobile terminal are usually displayed by stretching, copying the original pictures, or displaying fixed content around them. However, the above solution has the problem of poor picture display effect, which seriously affects the user's viewing experience.

[0090] In order to overcome the above problems, in some optional embodiments, see Figure 5The present application provides a picture display method, which is applied to the controller 110 in the display device 200 and may include the following steps:

[0091] S510, in response to a picture display instruction, obtaining display requirement data.

[0092] The picture display instruction may be an instruction for the original picture input by the user through the user input interface 140, and the instruction is used to instruct the display device 200 to display the specified original picture. The original picture may be stored in the memory 190 of the display device 200, or in an external device connected to the display device 200, and this application does not impose any limitation on this.

[0093] The display requirement data may include at least one of display size information, derived content data, and image evaluation dimension data. The display requirement data may be obtained from the local storage 190, or input by the user through the user input interface 140 or the detector 230, or directly obtained from an external device through the communication device 200, and this application does not impose any limitation on this. The display size may be the screen size information of the display.

[0094] In some embodiments, the display device 200 may include a detector 230, wherein the detector 230 is configured to collect voice data of the user. Accordingly, the controller 110 obtains the voice data corresponding to the display demand data collected by the detector 230 in response to the picture display instruction; and converts the voice data corresponding to the display demand data into text data.

[0095] It can be understood that replacing manual text input with voice interaction to obtain display requirement data provides convenience for users to input display requirement data, thereby improving the convenience of the image display process.

[0096] S520: Generate a derivative image corresponding to the original image according to the display requirement data.

[0097] The derived image is a picture generated by resizing, expanding the content or performing derivative processing on the original image.

[0098] Optionally, the original picture may be stored in the memory 190 of the display device 200 , and accordingly, in response to a picture upload instruction input by the user through the user input interface 140 , the selected original picture is obtained from the memory 190 .

[0099] Or optionally, the controller 110 may also obtain the original image from the external device through the communication device 220 in the display device 200 in response to a picture upload instruction input by the user through the user input interface 140 .

[0100] It is understandable that by selecting original pictures from the local storage, or obtaining the data of original pictures from an external device through a communication device, the methods of obtaining original pictures are enriched, thereby improving the flexibility and convenience of picture display.

[0101] Exemplarily, based on the display requirement data, the image can be processed with the original image as a reference to generate a derivative image corresponding to the original image.

[0102] In some embodiments, the original image may be resized according to the display size information of the display in the display requirement data to obtain a derivative image corresponding to the original image.

[0103] The display size information of the display may be the display size expected in the current image display process, and the display size is not larger than the screen size of the display. It is worth noting that the display size of the derived image is the same as the display size information of the display in the display requirement data.

[0104] In a specific implementation, the display size information of the display may be the screen size information of the display. In this way, the derivative image corresponding to the original image can be adapted to the screen size information of the display, thereby improving the display effect of the image and helping to improve the user's viewing experience.

[0105] In some embodiments, the controller 110 may directly derive content from the original image based on the derived content data in the display requirement data to obtain a derived image corresponding to the original image.

[0106] The derived content data is used to represent the description data of the reference content that needs to be expanded or derived from the original image. For example, the derived content data may be "add rainbow" or the like.

[0107] Exemplarily, content derivation from the original image can be understood as taking the original image as a benchmark, generating image extension data based on the derived content data in the original image, and superimposing or combining the image extension data with the original image, thereby achieving the purpose of image expansion or content deriving from the original image, and obtaining a derivative image corresponding to the original image.

[0108] In other embodiments, the controller 110 can also communicate with the server through the communication device 220 in the display device 200, and the controller 110 sends a picture derivation request and an original picture to the server, so that the server generates a derivative picture corresponding to the original picture according to the display requirement data in the picture derivation request; and obtains the derivative picture that meets the preset evaluation requirements fed back by the server. The advantage of doing so is that it can reduce the computing power requirements of the display device 200, thereby reducing the hardware cost and service life of the display device 200.

[0109] S530: When the derived image meets the preset evaluation requirement, control the display to display the derived image.

[0110] The preset evaluation requirement is used to characterize the display requirements of the image displayed on the display 260, and may include evaluation constraints on the displayed image in at least one image evaluation dimension. Optionally, the preset evaluation requirement may include at least one of quality requirements and semantic requirements, etc., which is not limited in this application.

[0111] Optionally, the preset evaluation requirements may be pre-set and stored in the memory 190 in the display device 200, and when the image display is required, the preset evaluation requirements are obtained from the memory 190. Alternatively, the preset evaluation requirements may also be input by the user in text through the user input interface 140. Alternatively, the preset evaluation requirements may also be collected by the detector 230 for the user's voice data, and the controller 110 converts the language data corresponding to the preset evaluation requirements into text data for subsequent use.

[0112] It should be noted that when the preset evaluation requirements are input by the user through the user input interface 140 or detected by the detector 230, the preset evaluation requirements can be acquired in real time each time the image is displayed, or, after one acquisition, they can be fixedly stored in the memory 190 for subsequent use, and when the evaluation requirements need to be updated, they can be adjusted through the data input by the user input interface 140 or the data detected by the detector 230.

[0113] It is worth noting that only when the derivative image meets the preset evaluation requirements, the display is controlled to display the derivative image, so as to avoid the displayed image not meeting the preset evaluation requirements and affecting the image display experience.

[0114] In an optional embodiment, when the derived image meets the preset evaluation requirements, the controller 110 controls the display 260 to display the derived image; further, when the derived image does not meet the preset evaluation requirements, the controller 110 no longer controls the display 260 to display the derived image, or, based on the original image or the derived image, performs image expansion or derivation operations again until the derived image meets the preset evaluation requirements, and controls the display 260 to display the final generated derivative image.

[0115] Since the image derivation process requires a certain amount of time cost, in order to adapt to the image display requirements of different users, the basic display mode and the derived display mode can be pre-set in the display device. In the basic display mode, the controller 110 can use the traditional image display method to control the display to display the original image. In the derived display mode, the controller 110 then acquires the display demand data and generates the derived image, and then controls the display such as 260 to display the derived image.

[0116] The embodiment of the present application performs image processing by introducing a controller with an image derivation function into a display device. Specifically, in the processing process, display requirement data corresponding to the original image is introduced to generate a derivative image corresponding to the original image. Since the derivative image is generated based on the original image, the generated image can be consistent with the original image. In addition, in the process of generating the derivative image, the display requirement data is referred to, which makes the generated derivative image look better compared with the traditional method of mechanically copying the image or adjusting the image size. Furthermore, when the derivative image meets the preset evaluation requirements, the derivative image is used to replace the original image and displayed on the display, so that the displayed image can meet the preset evaluation requirements, further improving the display effect of the displayed image.

[0117] Based on the technical solutions of the above-mentioned embodiments, the present application also provides some optional embodiments. In these optional embodiments, the steps for generating derived images are refined to improve the convenience and versatility of image derivation.

[0118] See also Figure 6 The steps for generating the derivative image shown include:

[0119] S610, based on the image generation model and according to the display requirement data, a derivative image corresponding to the original image is generated.

[0120] Among them, the image generation model can be understood as a model dedicated to image derivation. The model can be a traditional machine learning model or a deep learning model, or it can also be a large model with a large volume (for example, billions or even hundreds of billions) of network parameters.

[0121] It is worth noting that the image generation model can directly reuse the existing model, or train a pre-built machine learning model or deep learning model through a large number of sample images and standard derivative images corresponding to the sample images. This application does not impose any restrictions on the specific model structure and training method.

[0122] In some embodiments, the image generation model may include an image encoding network, a noise network, a text encoding network, and an image decoding network. The image encoding network is used to perform latent space pixel conversion on the input image; the noise network is used to perform noise reduction on the input image when performing latent space pixel conversion; the text encoding network is used to encode the text content corresponding to the actual demand data as a basis for deriving the input image; the image decoding network is used to decode the latent space pixels generated by the image encoding network, the noise network, and the text encoding network to generate an image that meets the preset size requirements.

[0123] It is worth noting that, in the process of image derivation, a general image generation model can be used. However, since different users like different image styles or original image styles, different image generation models can be selected in a targeted manner according to the image styles liked by the users or the image styles corresponding to the original images when generating derived images.

[0124] Optionally, the derived image style can be determined according to the image style of the original image; based on the image generation model under the derived image style, the derived image corresponding to the original image is generated according to the display request data. The advantage of this is that the generated derived image can be consistent with the style of the original image, avoiding the distortion of the original image style.

[0125] Alternatively, the derived style of the image may be determined according to the user portrait data of the user corresponding to the image display instruction; and the derived image corresponding to the original image may be generated according to the display requirement data based on the image generation model under the derived style of the image. The advantage of this is that the generated derived image can better meet the image style preferences of the user.

[0126] In some optional embodiments, when the derived picture meets the preset evaluation requirements, the controller 110 directly controls the display 260 to display the derived picture.

[0127] In other optional embodiments, when the derived image does not meet the preset evaluation requirements, the controller 110 can also adjust the network parameters in the image generation model to update the image generation model; the derived image is used as a new original image, and the derived image generation operation is returned to perform until the derived image meets the preset evaluation requirements. The advantage of this is that, through the cascade derivation method, the derived image finally generated based on the original image can meet the preset evaluation requirements, thereby realizing the image display based on the original image, avoiding the direct output of the derived image that does not meet the preset evaluation requirements to bring a poor viewing experience to the user.

[0128] Exemplarily, in the process of updating the image generation model, the adjusted network parameters may be at least part of the network parameters in the image generation model, and this application does not impose any limitation on this.

[0129] In order to prevent users from waiting too long during the image derivation process, a derivation upper threshold can be pre-set for each image display process, and when the cumulative number of derivations reaches the derivation upper threshold, the target image is displayed regardless of whether the final generated derivative image meets the preset evaluation requirements. The target image can be the final generated derivative image, or the derivative image that is closest to the preset evaluation requirements among the derivative images corresponding to the original image.

[0130] The embodiment of the present application generates derivative images corresponding to the original images according to the display demand data by introducing an image generation model, and can reuse the image generation model in different image display processes without repeated training of the image generation model and repeated collection of a large amount of sample data, thereby improving the versatility and portability of the image derivation process. Furthermore, in the case where the derivative images obtained by a single derivation do not meet the preset evaluation requirements, the network parameters in the image generation model are adjusted to update the image generation model, so that the derivative images generated by the updated image generation model gradually approach the preset evaluation requirements, thereby improving the image performance of the subsequently generated derivative images.

[0131] Based on the technical solutions of the above-mentioned embodiments, the present application also provides some optional embodiments. In these optional embodiments, the updating steps of the image generation model are refined to further improve the ability of the image generation model to derive images that meet preset evaluation requirements.

[0132] See also Figure 7 The update steps of the picture generation model shown include:

[0133] S710, determining an abnormal evaluation type corresponding to a preset evaluation requirement that the derived image does not meet.

[0134] S720: Determine a network to be adjusted in the image generation model according to the abnormal evaluation type.

[0135] Among them, the preset evaluation requirements may correspond to at least one evaluation dimension, and at least some different evaluation dimensions may correspond to different evaluation types. In the picture generation model, different network compositions correspond to different performances, and the network to be adjusted in the picture generation model may be determined based on the correspondence between the evaluation dimension and the constituent network of the picture generation model. Among them, the evaluation dimension may include at least one of the evaluation dimensions such as edge smoothness, pixel gradient change, peak signal-to-noise ratio, and semantic similarity. Optionally, the evaluation dimension may correspond to evaluation types such as quality evaluation and semantic evaluation. Among them, quality evaluation is used to evaluate the quality of pixels in the picture, while semantic evaluation is used to evaluate the rationality of the semantic information carried by the picture.

[0136] In some optional embodiments, when the abnormal evaluation type includes quality evaluation, the picture coding network in the picture generation model can be used as the network to be adjusted; wherein the picture coding network is used to perform latent space pixel conversion on the input picture. It can be understood that, when the abnormal evaluation type includes quality evaluation, the picture coding network that affects the picture quality in the picture generation model is used as the network to be adjusted. Subsequently, only at least part of the network parameters in the network to be adjusted need to be adjusted, and the full amount of network parameters in the picture generation model is no longer required, that is, there is no need to adjust other networks that are not related to the picture quality, thereby reducing the amount of parameter adjustment, thereby ensuring the stability and robustness of the picture generation model, and also ensuring that the picture quality output by the adjusted picture generation model gradually improves.

[0137] In other optional embodiments, when the abnormal evaluation type includes semantic evaluation, the text encoding network in the image generation model can be used as the network to be adjusted; wherein the text encoding network is used to encode the text content corresponding to the display demand data. It can be understood that, when the abnormal evaluation type includes semantic evaluation, the text encoding network that affects the semantics of the image in the image generation model is used as the network to be adjusted. Subsequently, only at least part of the parameters in the network to be adjusted need to be adjusted, and the full amount of network parameters in the image generation model is no longer required, that is, there is no need to adjust other networks that are not related to the semantics of the image, thereby reducing the amount of parameter adjustment, thereby ensuring the stability and robustness of the image generation model, and also ensuring that the image semantics output by the adjusted image generation model gradually improves, that is, it is more consistent with the image display instructions.

[0138] S730: Adjust network parameters in the network to be adjusted to update the image generation model.

[0139] In an optional embodiment, all network parameters in the network to be adjusted may be adjusted to update the image generation model.

[0140] Optionally, the adjustment range and direction of the network parameters can be implemented based on at least one existing method, and this application does not impose any limitation on this. For example, the adjustment range and direction can be set based on a gradient descent algorithm combined with a pre-set loss function.

[0141] In order to further reduce the amount of data of the network parameters adjusted during the update process of the image generation model, thereby improving the image performance of the image generation model under the evaluation dimension corresponding to the abnormal evaluation type while taking into account the stability and robustness of the image generation model update process, in an optional embodiment, some network parameters can also be selected from the network to be adjusted to update the image generation model.

[0142] In an optional embodiment, when the abnormal evaluation type includes quality evaluation, the first network parameter in the network to be adjusted and the parameter adjustment amount of the first network parameter can be determined based on the quality evaluation index that the derived image does not meet; and the corresponding first network parameter is adjusted according to the parameter adjustment amount of the first network parameter.

[0143] Exemplarily, a first correspondence between different quality evaluation indicators and the network parameters to be adjusted can be preset; accordingly, based on the first correspondence, the network parameters to be adjusted corresponding to the quality evaluation indicators that the derived images do not meet are searched as the first network parameters.

[0144] Optionally, the default parameter adjustment value may be used as the adjustment amount of each first network parameter. Accordingly, according to the gradient descent algorithm, etc., the parameter adjustment direction of the first network parameter is determined; for each first network parameter, the default parameter adjustment value and the parameter adjustment direction of the first network parameter are combined to obtain the parameter adjustment amount corresponding to the first network parameter.

[0145] Alternatively, a second correspondence between different to-be-adjusted network parameters and to-be-adjusted quantities may be pre-established; accordingly, based on the second correspondence, the to-be-adjusted quantity corresponding to the first network parameter is searched as the parameter adjustment quantity of the first network parameter.

[0146] Or optionally, a third correspondence between different quality evaluation indicators and the parameters to be adjusted can be pre-established; accordingly, based on the third correspondence, the parameters to be adjusted corresponding to the quality evaluation indicators that the derived image does not meet are searched as parameter adjustment amounts for the first network parameters corresponding to the quality evaluation indicators.

[0147] Exemplarily, a fourth correspondence between each network parameter to be adjusted and the variable to be adjusted under different quality evaluation indicators can also be pre-established; accordingly, based on the fourth correspondence, the network parameter to be adjusted corresponding to the quality evaluation indicator that the derived image does not meet is searched as the first network parameter, and the variable to be adjusted corresponding to the first network parameter is used as the parameter adjustment amount corresponding to the first network parameter.

[0148] It is worth noting that at least one of the above-mentioned default parameter adjustment values, the first correspondence, the second correspondence, the third correspondence and the fourth correspondence, etc., can be set or adjusted by technical personnel according to needs or experience, or repeatedly determined through a large number of experiments, and this application does not impose any limitations on this.

[0149] It can be understood that by introducing the quality evaluation index to determine the first network parameter, the amount of data of the network parameter that needs to be adjusted in the update process of the image generation model is further reduced, thereby improving the robustness and robustness of the image generation model adjustment process on the basis of ensuring the image quality performance of the image output by the updated image generation model. In addition, by introducing the quality evaluation index to numerically quantify the parameter adjustment amount of the first network parameter, the first network parameter is finely adjusted, further improving the robustness and robustness of the image generation model update process.

[0150] In another optional embodiment, when the abnormal evaluation type includes semantic evaluation, the network parameters corresponding to the network layer related to the semantic evaluation in the network to be adjusted are used as the second network parameters.

[0151] Optionally, the default parameter adjustment value can be used as the adjustment amount of each second network parameter. Accordingly, according to the gradient descent algorithm, etc., the parameter adjustment direction in the second network parameter is determined; for each second network parameter, the default parameter adjustment value and the parameter adjustment direction of the second network parameter are combined to obtain the parameter adjustment amount corresponding to the second network parameter. It is worth noting that the adjustment amount of the second network parameter can be the same as or different from the aforementioned adjustment amount of the first network parameter, and this application does not impose any limitation on this.

[0152] Or optionally, based on the aforementioned pre-established second corresponding relationship, a parameter to be adjusted corresponding to the second network parameter may be searched as the parameter adjustment amount of the second network parameter.

[0153] Or optionally, the parameter to be adjusted corresponding to the semantic evaluation index may be directly used as the parameter adjustment amount of each second network parameter.

[0154] Or optionally, the semantic similarity between the semantic features of the derived image and the text content corresponding to the display requirement data can also be determined; based on the semantic similarity, the parameter adjustment amount of the second network parameter in the network to be adjusted is determined; and based on the parameter adjustment amount of the second network parameter, the second network parameter in the network to be adjusted is adjusted.

[0155] In a specific implementation, a first semantic feature of the derived image can be extracted, and a second semantic feature of the text content corresponding to the display requirement data can be extracted; based on the similarity between the first semantic feature and the second semantic feature, the semantic similarity between the semantic feature of the derived image and the text content corresponding to the display requirement data is determined. The advantage of this is that the matching degree between the derived image and the derived content in the display requirement data can be numerically quantified through the semantic similarity.

[0156] It is worth noting that the feature extraction process of the first semantic feature and the second semantic feature can be implemented by using at least one of the traditional feature extraction methods, and this application does not impose any limitation on this. In a specific implementation method, the first semantic feature can be extracted based on the existing feature extraction network, and the embedding layer in the text encoding network can be reused to feature encode the text content corresponding to the display demand data to obtain the second semantic feature.

[0157] Exemplarily, the parameters to be adjusted in different similarity intervals may be preset; and the parameters to be adjusted in the similarity interval to which the semantic similarity belongs are used as parameter adjustment amounts of the second network parameters.

[0158] It can be understood that by introducing a network layer related to semantic evaluation and determining the second network parameters, the amount of data required to adjust the network parameters in the update process of the image generation model is further reduced, thereby improving the robustness and robustness of the image generation model adjustment process on the basis of ensuring the semantic performance of the image output by the updated image generation model. In addition, by introducing semantic similarity to numerically quantify the parameter adjustment amount of the second network parameters, the second network parameters are finely adjusted, further improving the robustness and robustness of the image generation model adjustment process.

[0159] The embodiment of the present application introduces an abnormal evaluation type, selects a network to be adjusted in the image generation model, and only adjusts the network parameters in the network to be adjusted, without adjusting all the network parameters in the image generation model. Therefore, in the process of updating the image generation model, the adjustment range is minimized and controlled, and the situation where the performance of the image generation model is unstable due to a large-scale adjustment of the network parameters is avoided, thereby ensuring the stability and robustness of the image generation model during the model update process, helping to reduce the waiting time of the image display process and improving the image display efficiency.

[0160] Based on the technical solutions of the above embodiments, this application also provides an optional embodiment, in which the image display process is described in detail by taking the interaction between the display device and the server as an example. It should be noted that when the computing power of the display device is relatively strong, the technical solution executed on the server side can be at least partially migrated to the display device for implementation, and this application does not make any limitation on this.

[0161] The following will be combined Fig. 8A The picture shown shows the state machine diagram of the process. Figure 8B The picture display method shown is explained in detail.

[0162] S801, the display device obtains the original picture to be displayed in response to the picture upload instruction sent by the user;

[0163] S802, in response to the picture display instruction sent by the user, output a voice reminder instruction;

[0164] It is worth noting that the above instructions can be text instructions or voice instructions. In the case of voice instructions, the display device performs semantic understanding on the voice instructions to obtain corresponding text instructions.

[0165] S803, obtaining voice data input by the user, and performing text analysis on the voice data to obtain a derived content text;

[0166] In a specific implementation, the derived content text may be a keyword such as "increase rainbow".

[0167] S804, obtaining screen size information of the display device;

[0168] S805, sending a picture derivation request and an original picture to a server; wherein the picture derivation request includes display requirement data; wherein the display requirement data includes a screen size and a derived content text;

[0169] S806, the server determines the user portrait data of the requester corresponding to the image derivation request;

[0170] S807, calling an image generation model that matches the image derivative style corresponding to the user portrait data;

[0171] For example, the image-derived style may be a Chinese style.

[0172] The image generation model includes an image encoding network, a noise network, a text encoding network and an image decoding network. The image encoding network is used to perform latent space pixel conversion on the input image; the noise network is used to perform noise reduction on the input image when performing latent space pixel conversion; the text encoding network is used to encode the text content corresponding to the actual demand data as a basis for deriving the input image; the image decoding network is used to decode the latent space pixels generated by the image encoding network, the noise network and the text encoding network to generate an image that meets the preset size requirements.

[0173] In a specific implementation, the image encoding network can be a VAE (Variational Autoencoder) encoder, the text encoding network can be a UNET (a convolutional neural network model) network implemented based on CLIP (Contrastive Language-Image Pre-training), and the image decoding network can be a VAE decoder.

[0174] S808, obtaining a derivative image that satisfies the screen size and is generated by the image generation model based on the display requirement data;

[0175] S809, performing quality evaluation on the derived image in at least one quality evaluation dimension;

[0176] S810, determining whether the quality evaluation results under each quality evaluation dimension meet the preset quality evaluation requirements; if so, executing S811; otherwise, executing S815;

[0177] S811, calling the image understanding model to determine the semantic similarity between the derived image and the text vector corresponding to the derived content text; continue to execute S812;

[0178] Among them, the text vector corresponding to the derived content text can be the prompt obtained by embedding the derived content text.

[0179] S812, determine whether the semantic similarity meets the preset similarity requirement; if so, execute S813; otherwise, execute S816;

[0180] S813, sending the derived image to the display device; and continuing to execute S814;

[0181] S814, controlling the display to display the received derivative image.

[0182] S815, determining a first network parameter for adjusting parameters of the image coding model and a parameter adjustment amount corresponding to the first network parameter according to the quality evaluation quality that does not meet the preset quality evaluation requirement; and continuing to execute S817;

[0183] S816, using the network parameter of the preset network layer in the text encoding model as the second network parameter, and determining the parameter adjustment amount corresponding to the second network parameter according to the semantic similarity; and continuing to execute S817;

[0184] S817, adjusting the corresponding network parameters according to the parameter adjustment amount corresponding to the first / second network parameters to update the image coding model; and continuing to execute S819;

[0185] Among them, the preset network layer can be a fully connected layer in the text encoding model specifically used for semantic verification.

[0186] S818, taking the derived image as a new original image, and returning to execute S807.

[0187] See also Figure 8C The original image shown in the figure is much smaller than the screen size of 3840×2160. When the derived text content entered by the user is the keyword "add rainbow" corresponding to "add rainbow to the original image", the final generated Fig.8D Derivative images that meet the screen size and preset evaluation requirements (quality requirements and semantic requirements). Fig.8D The image content in the middle rectangular box corresponds to the original image.

[0188] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0189] Based on the same inventive concept, the embodiment of the present application also provides a picture display device for implementing the picture display method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more picture display device embodiments provided below can refer to the limitations on the picture display method above, and will not be repeated here.

[0190] In an exemplary embodiment, Fig. 9 As shown, a picture display device is provided, including: a data acquisition module 910, a picture derivation module 920 and a display control module 930. Among them,

[0191] The data acquisition module 910 is used to acquire display requirement data in response to the picture display instruction for the original picture;

[0192] The picture derivation module 920 is used to generate a derivative picture corresponding to the original picture according to the display requirement data;

[0193] The display control module 930 is used to control the display to display the derivative image when the derivative image meets the preset evaluation requirements.

[0194] In some embodiments, the image derivation module 920 includes: an image derivation unit, which is used to generate a derivative image corresponding to the original image based on the image generation model and according to display requirement data; accordingly, the image derivation module 920 also includes: a parameter updating unit, which is used to adjust the network parameters in the image generation model to update the image generation model when the derived image does not meet the preset evaluation requirements; and a return execution unit, which is used to use the derived image as a new original image and return to execute the derivative image generation operation.

[0195] In some embodiments, the parameter updating unit includes: a first determining subunit, used to determine the abnormal evaluation type corresponding to the preset evaluation requirements that the derived image does not meet; a second determining subunit, used to determine the network to be adjusted in the image generation model according to the abnormal evaluation type; and a first adjusting subunit, used to adjust the network parameters in the network to be adjusted to update the image generation model.

[0196] In some embodiments, the second determination subunit is specifically used to use the picture encoding network in the picture generation model as the network to be adjusted when the abnormal evaluation type includes quality evaluation; wherein the picture encoding network is used to perform latent space pixel conversion on the input picture.

[0197] In some embodiments, the second determination subunit is specifically used to use the text encoding network in the image generation model as the network to be adjusted when the abnormal evaluation type includes semantic evaluation; wherein the text encoding network is used to encode the text content corresponding to the display demand data.

[0198] In some embodiments, the first adjustment subunit is specifically used to determine the first network parameter in the network to be adjusted and the parameter adjustment amount of the first network parameter according to the quality evaluation index that the derived image does not meet when the abnormal evaluation type includes quality evaluation; and adjust the corresponding first network parameter according to the parameter adjustment amount of the first network parameter.

[0199] In some embodiments, the first adjustment subunit is specifically used to, when the abnormal evaluation type includes semantic evaluation, use the network parameters corresponding to the network layer related to the semantic evaluation in the network to be adjusted as the second network parameters; determine the semantic similarity between the semantic features of the derived image and the text content corresponding to the display requirement data; determine the parameter adjustment amount of the second network parameter in the network to be adjusted according to the semantic similarity; and adjust the second network parameter in the network to be adjusted according to the parameter adjustment amount of the second network parameter.

[0200] In some embodiments, the image derivation unit includes: a third determination subunit, used to determine the image derivation style according to user portrait data of the user corresponding to the image display instruction; a first generation subunit, used to generate a derivative image corresponding to the original image based on the image generation model under the image derivation style and according to display requirement data.

[0201] In some embodiments, the data acquisition module 910 includes: a first acquisition unit, used to respond to the image display instruction and acquire the voice data corresponding to the display demand data collected by the detector; and a conversion unit, used to convert the voice data corresponding to the display demand data into text data.

[0202] In some embodiments, the image display device also includes an image acquisition module, which is used to obtain the selected original image from the memory in response to an image upload instruction input through the user input interface before generating a derivative image corresponding to the original image according to display requirement data, or to obtain the original image from an external device through a communication device.

[0203] In some embodiments, the image derivation module 920 includes: a sending unit, used to send an image derivation request and an original image to a server, so that the server generates a derivative image corresponding to the original image according to display requirement data in the image derivation request; and an acquisition unit, used to acquire a derivative image that meets preset evaluation requirements and is fed back by the server.

[0204] In some embodiments, the image derivation module 920 includes: a size transformation unit, which is used to transform the size of the original image according to the screen size information of the display in the display requirement data, to obtain a derivative image corresponding to the original image; and / or a content derivation unit, which is used to derive content from the original image according to the derived content data in the display requirement data, to obtain a derivative image corresponding to the original image.

[0205] Each module in the above-mentioned image display device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module above.

[0206] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig.10 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (Near Field Communication, NFC) or other technologies. When the computer program is executed by the processor, a method for displaying a picture is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0207] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0208] In an alternative embodiment, Fig.10 The computer device shown may be the aforementioned display device.

[0209] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0210] In response to the picture display instruction, obtaining display requirement data;

[0211] Generate a derivative image corresponding to the original image according to the display requirement data;

[0212] When the derived image meets the preset evaluation requirements, the display is controlled to display the derived image.

[0213] In one embodiment, when the processor executes the computer program, the following steps are also implemented: based on the image generation model, according to the display requirement data, a derivative image corresponding to the original image is generated; when the derivative image does not meet the preset evaluation requirements, the network parameters in the image generation model are adjusted to update the image generation model; the derivative image is used as a new original image, and the derivative image generation operation is returned to execute.

[0214] In one embodiment, when the processor executes the computer program, the following steps are also implemented: determining the abnormal evaluation type corresponding to the preset evaluation requirements that the derived image does not meet; determining the network to be adjusted in the image generation model according to the abnormal evaluation type; and adjusting the network parameters in the network to be adjusted to update the image generation model.

[0215] In one embodiment, when the processor executes the computer program, the following steps are also implemented: when the abnormal evaluation type includes quality evaluation, the picture encoding network in the picture generation model is used as the network to be adjusted; wherein the picture encoding network is used to perform latent space pixel conversion on the input picture.

[0216] In one embodiment, when the processor executes the computer program, the following steps are also implemented: when the abnormal evaluation type includes semantic evaluation, the text encoding network in the image generation model is used as the network to be adjusted; wherein the text encoding network is used to encode the text content corresponding to the display demand data.

[0217] In one embodiment, when the processor executes the computer program, the following steps are also implemented: when the abnormal evaluation type includes quality evaluation, determining the first network parameter in the network to be adjusted and the parameter adjustment amount of the first network parameter according to the quality evaluation index that the derived image does not meet; and adjusting the corresponding first network parameter according to the parameter adjustment amount of the first network parameter.

[0218] In one embodiment, when the processor executes the computer program, the following steps are also implemented: when the abnormal evaluation type includes semantic evaluation, the network parameter corresponding to the network layer related to the semantic evaluation in the network to be adjusted is used as the second network parameter; the semantic similarity between the semantic features of the derived image and the text content corresponding to the display requirement data is determined; according to the semantic similarity, the parameter adjustment amount of the second network parameter in the network to be adjusted is determined; according to the parameter adjustment amount of the second network parameter, the second network parameter in the network to be adjusted is adjusted.

[0219] In one embodiment, when the processor executes the computer program, the following steps are also implemented: determining the image derivative style according to the user portrait data of the user corresponding to the image display instruction; generating a derivative image corresponding to the original image based on the image generation model under the image derivative style and according to the display requirement data.

[0220] In one embodiment, when the processor executes the computer program, the following steps are also implemented: in response to the picture display instruction, obtaining the voice data corresponding to the display demand data collected by the detector; and converting the voice data corresponding to the display demand data into text data.

[0221] In one embodiment, when the processor executes the computer program, the following steps are also implemented: in response to a picture upload instruction input through the user input interface, the selected original picture is obtained from the memory, or the original picture is obtained from an external device through a communication device.

[0222] In one embodiment, when the processor executes the computer program, the following steps are also implemented: sending a picture derivation request and an original picture to a server, so that the server generates a derivative picture corresponding to the original picture according to the display requirement data in the picture derivation request; and obtaining a derivative picture that meets preset evaluation requirements fed back by the server.

[0223] In one embodiment, when the processor executes the computer program, the following steps are also implemented: according to the screen size information of the display described in the display requirement data, the size of the original image is transformed to obtain a derivative image corresponding to the original image; and / or, according to the derivative content data in the display requirement data, the content of the original image is derived to obtain a derivative image corresponding to the original image.

[0224] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0225] In response to the picture display instruction, obtaining display requirement data;

[0226] Generate a derivative image corresponding to the original image according to the display requirement data;

[0227] When the derived image meets the preset evaluation requirements, the display is controlled to display the derived image.

[0228] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: based on the image generation model, according to the display requirement data, a derivative image corresponding to the original image is generated; when the derivative image does not meet the preset evaluation requirements, the network parameters in the image generation model are adjusted to update the image generation model; the derivative image is used as a new original image, and the derivative image generation operation is returned to execute.

[0229] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: determining the abnormal evaluation type corresponding to the preset evaluation requirements that the derived image does not meet; determining the network to be adjusted in the image generation model according to the abnormal evaluation type; and adjusting the network parameters in the network to be adjusted to update the image generation model.

[0230] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: when the abnormal evaluation type includes quality evaluation, the picture encoding network in the picture generation model is used as the network to be adjusted; wherein the picture encoding network is used to perform latent space pixel conversion on the input picture.

[0231] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the abnormal evaluation type includes semantic evaluation, the text encoding network in the image generation model is used as the network to be adjusted; wherein the text encoding network is used to encode the text content corresponding to the display demand data.

[0232] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the abnormal evaluation type includes quality evaluation, determining the first network parameter in the network to be adjusted and the parameter adjustment amount of the first network parameter according to the quality evaluation index that the derived image does not meet; and adjusting the corresponding first network parameter according to the parameter adjustment amount of the first network parameter.

[0233] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the abnormal evaluation type includes semantic evaluation, the network parameter corresponding to the network layer related to the semantic evaluation in the network to be adjusted is used as the second network parameter; the semantic similarity between the semantic features of the derived image and the text content corresponding to the display requirement data is determined; according to the semantic similarity, the parameter adjustment amount of the second network parameter in the network to be adjusted is determined; according to the parameter adjustment amount of the second network parameter, the second network parameter in the network to be adjusted is adjusted.

[0234] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: determining the image derivative style according to the user portrait data of the user corresponding to the image display instruction; generating a derivative image corresponding to the original image based on the image generation model under the image derivative style and according to the display requirement data.

[0235] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: in response to the picture display instruction, obtaining the voice data corresponding to the display demand data collected by the detector; converting the voice data corresponding to the display demand data into text data.

[0236] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: in response to a picture upload instruction input through the user input interface, the selected original picture is obtained from the memory, or the original picture is obtained from an external device through a communication device.

[0237] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: sending a picture derivation request and an original picture to a server, so that the server generates a derivative picture corresponding to the original picture according to the display requirement data in the picture derivation request; and obtaining a derivative picture that meets preset evaluation requirements fed back by the server.

[0238] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: according to the screen size information of the display described in the display requirement data, the size of the original image is transformed to obtain a derivative image corresponding to the original image; and / or, according to the derivative content data in the display requirement data, the content of the original image is derived to obtain a derivative image corresponding to the original image.

[0239] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0240] In response to the picture display instruction, obtaining display requirement data;

[0241] Generate a derivative image corresponding to the original image according to the display requirement data;

[0242] When the derived image meets the preset evaluation requirements, the display is controlled to display the derived image.

[0243] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: based on the image generation model, according to the display requirement data, a derivative image corresponding to the original image is generated; when the derivative image does not meet the preset evaluation requirements, the network parameters in the image generation model are adjusted to update the image generation model; the derivative image is used as a new original image, and the derivative image generation operation is returned to execute.

[0244] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: determining the abnormal evaluation type corresponding to the preset evaluation requirements that the derived image does not meet; determining the network to be adjusted in the image generation model according to the abnormal evaluation type; and adjusting the network parameters in the network to be adjusted to update the image generation model.

[0245] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented: when the abnormal evaluation type includes quality evaluation, the picture encoding network in the picture generation model is used as the network to be adjusted; wherein the picture encoding network is used to perform latent space pixel conversion on the input picture.

[0246] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the abnormal evaluation type includes semantic evaluation, the text encoding network in the image generation model is used as the network to be adjusted; wherein the text encoding network is used to encode the text content corresponding to the display demand data.

[0247] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: when the abnormal evaluation type includes quality evaluation, determining the first network parameter in the network to be adjusted and the parameter adjustment amount of the first network parameter according to the quality evaluation index that the derived image does not meet; and adjusting the corresponding first network parameter according to the parameter adjustment amount of the first network parameter.

[0248] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: using the network parameters corresponding to the network layer related to the semantic evaluation in the network to be adjusted as the second network parameters; determining the semantic similarity between the semantic features of the derived image and the text content corresponding to the display demand data; determining the parameter adjustment amount of the second network parameter in the network to be adjusted according to the semantic similarity; and adjusting the second network parameter in the network to be adjusted according to the parameter adjustment amount of the second network parameter.

[0249] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: determining the image derivative style according to the user portrait data of the user corresponding to the image display instruction; generating a derivative image corresponding to the original image based on the image generation model under the image derivative style and according to the display requirement data.

[0250] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: in response to the picture display instruction, obtaining the voice data corresponding to the display demand data collected by the detector; converting the voice data corresponding to the display demand data into text data.

[0251] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: in response to a picture upload instruction input through the user input interface, the selected original picture is obtained from the memory, or the original picture is obtained from an external device through a communication device.

[0252] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: sending a picture derivation request and an original picture to a server, so that the server generates a derivative picture corresponding to the original picture according to the display requirement data in the picture derivation request; and obtaining a derivative picture that meets preset evaluation requirements fed back by the server.

[0253] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: according to the screen size information of the display described in the display requirement data, the size of the original image is transformed to obtain a derivative image corresponding to the original image; and / or, according to the derivative content data in the display requirement data, the content of the original image is derived to obtain a derivative image corresponding to the original image.

[0254] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0255] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0256] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0257] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A display device, characterized in that: include: A display configured to display content; A user input interface, configured to obtain a picture display instruction for an original picture inputted by a user; At least one controller connected to the display and configured to: In response to the picture display instruction, acquiring display requirement data; Generating a derivative image corresponding to the original image according to the display requirement data; When the derivative image meets the preset evaluation requirements, the display is controlled to display the derivative image.

2. The display device according to claim 1, characterized in that When the controller generates a derivative image corresponding to the original image according to the display requirement data, the controller is configured to: Based on the image generation model, generating a derivative image corresponding to the original image according to the display requirement data; Accordingly, the controller is further configured as follows: When the derived image does not meet the preset evaluation requirement, adjusting the network parameters in the image generation model to update the image generation model; The derivative image is used as a new original image, and the derivative image generation operation is performed again.

3. The display device according to claim 2, characterized in that When the controller adjusts the network parameters in the picture generation model to update the picture generation model, the controller is configured to: Determine an abnormal evaluation type corresponding to the preset evaluation requirement that the derived image does not meet; Determining a network to be adjusted in the image generation model according to the abnormal evaluation type; Adjust the network parameters in the network to be adjusted to update the image generation model.

4. The display device according to claim 3, characterized in that When the controller determines the network to be adjusted in the image generation model according to the abnormal evaluation type, the controller is configured to: In a case where the abnormal evaluation type includes quality evaluation, using a picture encoding network in the picture generation model as the network to be adjusted; The picture encoding network is used to perform latent space pixel conversion on the input picture.

5. The display device according to claim 3, characterized in that When the controller determines the network to be adjusted in the image generation model according to the abnormal evaluation type, the controller is configured to: In a case where the abnormal evaluation type includes semantic evaluation, using a text encoding network in the image generation model as the network to be adjusted; The text encoding network is used to encode the text content corresponding to the display demand data.

6. The display device according to claim 3, characterized in that When the controller performs adjustment of the network parameters in the network to be adjusted, the controller is configured to: In a case where the abnormal evaluation type includes quality evaluation, determining a first network parameter in the network to be adjusted and a parameter adjustment amount of the first network parameter according to a quality evaluation index that the derived image does not satisfy; According to the parameter adjustment amount of the first network parameter, the corresponding first network parameter is adjusted.

7. The display device according to claim 3, characterized in that When the controller performs adjustment of the network parameters in the network to be adjusted, the controller is configured to: In the case where the abnormal evaluation type includes semantic evaluation, using a network parameter corresponding to a network layer related to the semantic evaluation in the network to be adjusted as a second network parameter; Determining the semantic similarity between the semantic features of the derived image and the text content corresponding to the display requirement data; Determining a parameter adjustment amount of a second network parameter in the network to be adjusted according to the semantic similarity; The second network parameter in the network to be adjusted is adjusted according to the parameter adjustment amount of the second network parameter.

8. The display device according to claim 2, characterized in that When the controller generates a derivative image corresponding to the original image based on the image generation model and according to the display requirement data, the controller is configured to: Determining a derived style of the picture according to the user portrait data of the user corresponding to the picture display instruction; Based on the picture generation model under the picture derivative style, a derivative picture corresponding to the original picture is generated according to the display requirement data.

9. The display device according to any one of claims 1 to 8, characterized in that: The display device further includes: A detector configured to collect voice data of a user; Accordingly, when the controller executes the image display instruction and obtains the display requirement data, it is configured to: In response to the picture display instruction, acquiring voice data corresponding to the display requirement data collected by the detector; The voice data corresponding to the display requirement data is converted into text data.

10. The display device according to any one of claims 1 to 8, characterized in that: The display device further includes: A memory, configured to locally store the original image; a communication device configured to communicate with an external device; Accordingly, before the controller generates a derivative image corresponding to the original image according to the display requirement data, the controller is further configured to: In response to a picture upload instruction input through the user input interface, the selected original picture is obtained from the memory, or the original picture is obtained from the external device through the communication device.

11. The display device according to any one of claims 1 to 8, characterized in that: The display device further includes: a communication device configured to communicate with a server; Accordingly, when the controller generates a derivative image corresponding to the original image according to the display requirement data, the controller is configured to: Sending a picture derivation request and the original picture to the server, so that the server generates a derivative picture corresponding to the original picture according to display requirement data in the picture derivation request; Obtain the derivative pictures fed back by the server that meet the preset evaluation requirements.

12. The display device according to any one of claims 1 to 8, characterized in that: When the controller generates a derivative image corresponding to the original image according to the display requirement data, the controller is configured to: According to the screen size information of the display in the display requirement data, the original image is resized to obtain a derivative image corresponding to the original image; and / or, According to the derived content data in the display requirement data, content is derived from the original image to obtain a derived image corresponding to the original image.

13. A method for displaying a picture, characterized in that: include: In response to a picture display instruction for an original picture, acquiring display requirement data; Generating a derivative image corresponding to the original image according to the display requirement data; When the derivative image meets the preset evaluation requirements, the display is controlled to display the derivative image.

Images