Display device and display method

Abstract

The invention relates to the technical field of display equipment, and provides display equipment and a display method.The display equipment comprises a displayer and a controller, and the displayer is configured to display a user interface; the controller is configured to display an image expansion interface on the user interface in response to an expansion instruction input by a user; in response to first image information input by a user in the extended input area, displaying a first image in the first display area; obtaining a second image in response to an extended parameter input by the user in the parameter setting area; according to the technical scheme, under the condition that the size parameter of the first image is adjusted, part of image information of the image can be prevented from being lost, the user experience is guaranteed, the first image and the second image are displayed on the user interface side by side, the expanded content of the second image can be conveniently checked, and the user experience is improved. And the difference between the expanded second image and the first image is visually compared, so that a user can obtain a more satisfactory second image.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a display method. Background Art

[0002] At present, with the development of the smart home industry, various functions of terminal devices are constantly being improved. For example, when using a terminal device, a user can set a personalized theme image or background image for the display interface of the terminal device. However, in actual application, the aspect ratio of the picture is often different from the aspect ratio of the display interface, which makes it impossible to directly set the picture as the theme image or background image of the display interface.

[0003] In this regard, in the prior art, the aspect ratio of the image is generally cropped to a suitable ratio on the terminal device, and then enlarged to fill the display interface. However, this cropping method will cause the image to lose some image information, resulting in a poor experience for the terminal user. Summary of the invention

[0004] The embodiments of the present application provide a display device and a display method to expand images on a terminal device and enhance the experience of the terminal user.

[0005] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0006] A first aspect provides a display device, which may include:

[0007] a display configured to display a user interface;

[0008] A controller is coupled to the display and is configured to:

[0009] In response to an expansion instruction input by a user, the display is driven to display an image expansion interface on the user interface; the image expansion interface includes a first display area, a second display area, an expansion input area and a parameter setting area;

[0010] In response to first image information input by a user in the extended input area, driving the display to display a first image in the first display area; the first image information is used to represent the first image;

[0011] In response to an expansion parameter input by a user in a parameter setting area, expanding the image size of the first image to obtain a second image;

[0012] The display is driven to display the second image in the second display area.

[0013] In this way, when adjusting the size parameters of the first image, it is possible to avoid the first image from losing part of its image information, thereby ensuring the user experience. The first image and the second image are displayed side by side on the user interface, making it easy for the user to view the expanded content of the second image and intuitively compare the difference between the expanded second image and the first image, thereby helping the user to obtain a more satisfactory second image.

[0014] In a possible implementation manner of the first aspect, after driving the display to display the first image in the first display area, the controller is further configured to:

[0015] In response to the extended parameter input by the user in the parameter setting area, the display is driven to display a prompt box corresponding to the extended parameter in the first display area, and the size of the prompt box is the size of the second image.

[0016] In this way, when the user enters the extended parameters, he can intuitively view the size of the second image after the extended parameters are expanded on the first interface, thereby providing a reference for the user to enter or modify the extended parameters, thereby making it easier for the user to obtain a satisfactory second image and improving the user experience.

[0017] In a possible implementation manner of the first aspect, after driving the display to display a prompt box corresponding to the extended parameter in the first display area, the controller is further configured to:

[0018] In response to the user dragging the border of the prompt box on the first display area, the display is driven to move the border position of the prompt box in the first display area, and the extended parameters input in the parameter setting area are updated according to the border position of the prompt box after the movement.

[0019] In this way, after generating a prompt box using extended parameters, the user can intuitively adjust the image size corresponding to the extended parameters by dragging the border, thereby realizing a customized expansion of the prompt box size, and displaying the corresponding extended parameters in real time in the parameter setting area according to the size of the prompt box when it is dragged, thereby providing users with a more convenient expansion method and improving the user experience.

[0020] In a possible implementation manner of the first aspect, the extended input area includes a text display area; after driving the display to display the first image in the first display area, the controller is further configured to:

[0021] Acquire photo editing materials and / or determine a photo editing method according to the extended information input by the user in the text display area in a voice or text manner;

[0022] According to the photo editing material and / or the photo editing method, the content of the first image is adjusted, and based on the adjusted content of the first image, the content of the second image is adjusted.

[0023] In this way, the user can adjust the first image by voice or text input, thereby facilitating user operation and improving user experience.

[0024] In a possible implementation manner of the first aspect, before adjusting the image information of the first image according to the image retouching material and / or the image retouching method, the controller is further configured to:

[0025] Acquire a circle instruction drawn by a user in the first display area, and determine a circle area of ​​the circle instruction in the first display area;

[0026] According to the photo editing material and / or photo editing method, the content of the first image in the circled area is adjusted, and based on the adjusted content of the first image, the content of the second image is adjusted.

[0027] In this way, based on the photo editing material and / or photo editing method, the user's photo editing area can be limited by applying a circle instruction, so that when the user wants to modify a partial image, it is avoided to affect the content of the non-adjustment area of ​​the first image.

[0028] In a possible implementation manner of the first aspect, after driving the display to display the first image in the first display area, the controller is further configured to:

[0029] Obtaining a line instruction drawn by the user in the first display area;

[0030] In response to the attributes of the line instruction, the content of the first image is adjusted, and based on the adjusted content of the first image, the content of the second image is adjusted; the attributes of the line instruction include one or more of line type, line color or line thickness.

[0031] In this way, more forms of image editing methods are provided to the user through line instructions, making it easier for the user to modify and adjust the image in the first display area.

[0032] In a possible implementation manner of the first aspect, after acquiring a line instruction drawn by a user in the first display area, the controller is further configured to:

[0033] In response to the line instruction, adding an instruction number to the line instruction;

[0034] In response to the user's triggering operation on the modification control in the first display area, the display is driven to display the instruction number corresponding to the line instruction in the first display area;

[0035] In response to a modification instruction input by a user, the adjustment content corresponding to the line instruction on the first image is modified, and based on the modification content corresponding to the modification instruction, the content of the second image is modified, wherein the modification instruction includes an instruction number corresponding to the line instruction and a corresponding modification method.

[0036] In this way, when the user needs to modify the adjustment content corresponding to the previous line instruction, the modification action can be implemented more conveniently, avoiding the situation where the adjustment content cannot be rolled back, causing the user to need to readjust the image.

[0037] In a possible implementation manner of the first aspect, after expanding the image size of the first image in response to the expansion parameter input by the user in the parameter setting area, the controller is further configured to:

[0038] In response to the image expansion number input by the user in the parameter setting area, a second image of the image expansion number is generated, and the second display area of ​​the display is driven to cyclically display the second image, and the contents of the multiple cyclically displayed second images are different.

[0039] In this way, in actual application, multiple second images can be generated based on the image expansion quantity input by the user, so that the user can select one from the multiple second images to use, thereby improving the user experience.

[0040] In a possible implementation manner of the first aspect, the image extension interface further includes an application extension area;

[0041] After driving the display to display the second image in the second display area, the controller is further configured to:

[0042] In response to a secondary expansion instruction input by a user in the application expansion area, the display is driven to display the second image in the first display area.

[0043] In this way, the user can, on the basis of accepting the second image, further expand the new image with the content of the second image as the original image, so as to avoid the situation in which the content of the accepted second image is lost or changed during the subsequent modification process.

[0044] In a possible implementation manner of the first aspect, the image extension interface further includes an application extension area;

[0045] After driving the display device to display the second image in the second display area, the controller is further configured to:

[0046] In response to a user triggering operation on an output control in the application extension area, the second image is output to an image receiving end corresponding to the output control.

[0047] In this way, when the user needs to use the second image in other occasions, it is convenient for the user to save or share the second image.

[0048] In a possible implementation manner of the first aspect, the controller expands an image size of the first image to obtain the second image, and is specifically configured as follows:

[0049] Get the original size of the first image;

[0050] According to the original size of the first image and the expansion parameter, the size of the first image is expanded to obtain a third image, wherein the aspect ratio of the expanded third image is equal to the aspect ratio in the expansion parameter;

[0051] According to the expansion parameter, the size of the third image is proportionally scaled to obtain the second image, and the display is driven to display the second image in the second display area.

[0052] In this way, based on the proportional relationship between the original size of the first image and the expansion parameter, the original size of the first image is expanded to obtain the third image, and then the size of the third image is proportionally scaled to obtain the second image, thereby avoiding the loss of picture information of the first image.

[0053] In a second aspect, a display method is provided, characterized in that it is applied to a display device, the display device includes a display for a user to display a user interface, and the display method includes:

[0054] In response to an expansion instruction input by a user, the display is driven to display an image expansion interface on the user interface; the image expansion interface includes a first display area, a second display area, an expansion input area and a parameter setting area;

[0055] In response to first image information input by a user in the extended input area, driving the display to display a first image in the first display area; the first image information is used to represent the first image;

[0056] In response to an expansion parameter input by a user in a parameter setting area, expanding the image size of the first image to obtain a second image;

[0057] The display is driven to display the second image in the second display area.

[0058] In this way, by applying the display method provided by the present application, first, in response to the expansion instruction input by the user, the display is driven to display the image expansion interface on the user interface; then, in response to the first image information input by the user in the expansion input area, the display is driven to display the first image in the first display area; then, in response to the expansion parameter input by the user in the parameter setting area, the image size of the first image is expanded to obtain the second image; finally, the display is driven to display the second image in the second display area.

[0059] The present application provides a display device, including a display and a controller, wherein the display is configured to display a user interface; the controller is coupled to the display, and the controller is configured to display an image expansion interface on the user interface in response to an expansion instruction input by a user; in response to first image information input by a user in an expansion input area, a first image is displayed in a first display area; in response to expansion parameters input by a user in a parameter setting area, a second image is obtained; and the display is driven to display the second image in the second display area. By applying the technical solution of the present application, when the size parameters of the first image are adjusted, the image can avoid losing part of the image information, thereby ensuring the user experience, and displaying the first image and the second image side by side on the user interface, so as to facilitate viewing of the expanded content of the second image and intuitively compare the difference between the expanded second image and the first image, thereby facilitating the user to obtain a more satisfactory second image. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0061] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0062] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0063] Figure 3 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0064] Figure 4 Schematic diagram of the image expansion process provided in some embodiments of the present application Figure 1 ;

[0065] Figure 5 A schematic diagram of a user interface layout of a display device provided in some embodiments of the present application;

[0066] Figure 6 Schematic diagram of the image expansion process provided in some embodiments of the present application Figure 2 ;

[0067] Figure 7 Schematic diagram of a closed area provided in some embodiments of the present application Figure 1 ;

[0068] Figure 8 Schematic diagram of a closed area provided in some embodiments of the present application Figure 2 ;

[0069] Fig. 9 Schematic diagram of a closed area provided in some embodiments of the present application Figure 3 ;

[0070] Fig.10 Schematic diagram of a closed area provided in some embodiments of the present application Figure 4 ;

[0071] Fig.11 Schematic diagram of the image expansion process provided in some embodiments of the present application Figure 3 ;

[0072] Fig.12 A schematic diagram of a user interface layout of a display device provided in some embodiments of the present application;

[0073] Fig.13 A schematic diagram of signal triggering state points of a diffusion model provided for some embodiments of the present application. DETAILED DESCRIPTION

[0074] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0075] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0076] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0077] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0078] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0079] In the embodiment of the present application, the display device 200 generally refers to a device with image display and data processing capabilities. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0080] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , a user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. The control device 100 is used to receive operation instructions input by the user and convert the operation instructions into control instructions that can be recognized and responded to by the display device 200. For example, the control device 100 can be a remote controller, a stylus pen, a handle, etc.

[0081] The mobile terminal 300 can be used as a control device for performing human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device for establishing a communication connection with the display device 200 and performing data interaction. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0082] In some embodiments, the mobile terminal 300 or other electronic devices may also simulate the functions of the control device 100 by running an application program for controlling the display device 200 .

[0083] like Figure 1 As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0084] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function with a computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), and the like.

[0085] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0086] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0087] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0088] In some embodiments, the display 260 includes a display function component for presenting a picture, and a driving component for driving an image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface and a user control UI interface.

[0089] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0090] The communication device 220 can enable the display device 200 to communicate with the external device or server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 with the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 with the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0091] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0092] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0093] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0094] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0095] In some embodiments, the user input interface 280 may be used to receive instructions from a user.

[0096] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system can control the display device to provide a user interface. For example, the operating system can directly control the display device to provide a user interface, or provide a user interface by running an application program. The operating system also allows the user to interact with the display device 200.

[0097] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for display devices.

[0098] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 3As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the system library layer and the kernel layer.

[0099] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on the applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0100] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0101] like Figure 3 As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to the application package currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0102] In some embodiments, the activity manager is used to manage the life cycle of each application and the usual navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes, for example, reducing the display window, shaking the display, distorting the display, etc.

[0103] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0104] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. Figure 3 As shown, the kernel layer may be configured with hardware drivers, and the kernel layer may include at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver (Universal Serial Bus: USB), HD MI driver (High Definition Multimedia Interface: HDMI), sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.) and power driver, etc.

[0105] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0106] The embodiment of the present application provides a display device, including a display and a controller, wherein the display is configured to display a user interface; the controller is coupled to the display, and Figure 4 As shown, the controller is configured to execute S410 to S440.

[0107] S410 , in response to an expansion instruction input by a user, driving a display to display an image expansion interface on a user interface.

[0108] like Figure 5 As shown, the image extension interface includes a first display area, a second display area, an extension input area and a parameter setting area; wherein the extension input area may include multiple areas, for example, Figure 5 The extended input area 1 and the extended input area 2 in the figure are both the range of the extended input area.

[0109] S420: In response to first image information input by the user in the extended input area, drive the display to display a first image in the first display area.

[0110] The first image information is used to represent the first image; specifically, Figure 5 As shown, the extended input area 1 is provided with a photo control and a code scanning control, and the first image information can be input through the photo control or the code scanning control. For example, the user triggers the photo control, the controller calls the camera of the display device, and shoots an image through the camera. After the user determines the shot image, the controller displays the shot image as the first image in the first display area. For another example, the first image information exists in the form of a QR code, the user triggers the code scanning control, the controller responds to the code scanning control, calls the camera to obtain the QR code information, or calls the QR code image in the local database and obtains the QR code information. For another example, the first image information is stored in the database of the display device in the form of an image, then the first image information can be obtained by touching the first display area and selecting the database, wherein the image stored in the database can be the first image itself, after the user triggers the control for uploading the image in the first display area, the controller calls the local database, and after the user selects the image, the selected image is displayed as the first image in the first display area. It should be noted that the first image displayed in the first display area is the image that the user needs to expand.

[0111] S430: In response to the expansion parameter input by the user in the parameter setting area, the image size of the first image is expanded to obtain a second image.

[0112] The extended parameter is used to adjust the image size of the first image. For example, the extended parameter is to set the image aspect ratio to 4:3, or the extended parameter is to set the image aspect ratio to 16:9. Alternatively, the input extended parameter is to set the image aspect size to fill the preset interface. For another example, the input extended parameter is an image of a specific form, such as the vertical screen size of a device. For another example, the input extended parameter is to set the image pixels.

[0113] It should be noted that the method of expanding the image size of the first image is not to crop, stretch or scale the first image, but to fill the image in the expanded area using a diffusion model. In this way, the missing or newly added area of ​​the image can be filled while maintaining the overall coherence of the image.

[0114] For example, AI image generation technologies represented by stability AI and MidJourney can achieve high-quality general image drawing, that is, filling or redrawing the user-specified area according to the user's intention, and the filled or redrawn image can maintain the overall continuity of the image, where the outward redrawing of the image is image expansion.

[0115] In the technical solution provided in the embodiment of the present application, 5 additional input channels are added to the U-Net input of the diffusion model. Among them, 4 input channels are used to encode the mask image and 1 input channel is used for the mask itself. Specifically, in the diffusion model, the U-Net architecture is used as a denoiser, and the performance can be improved by adding additional input channels, especially when processing tasks that require conditional input. The following is a specific method of how to add 5 additional input channels to U-Net to enhance the model's processing capabilities for mask images.

[0116] Channel design: 4 input channels are used to encode the mask image: These 4 additional input channels can encode different features of the mask image or information of different scales. For example, the mask image can be preprocessed using different filters or convolution kernels to capture its texture, boundary or shape information. 1 input channel is used for the mask itself: The mask image is directly used as one of the input channels to ensure that the model can directly obtain the original information of the mask.

[0117] Input consolidation: At the input layer of U-Net, the original image channels are merged with the additional 5 channels. For example, this can be a simple channel concatenation operation, which arranges all input channels side by side to form a larger input vector.

[0118] In this way, by adding additional input channels to enhance the processing ability of the diffusion model on the mask image, the performance of the model on specific tasks is improved, and more precise control can be achieved during image fusion, thereby improving the quality and effect of the final fused image, that is, improving the image fusion effect of the redrawn area image and the original image.

[0119] S440: Drive the display to display a second image in the second display area.

[0120] The second image displayed in the second display area is a modified image, that is, the first display area displays the original first image, and the second display area displays the expanded second image. In this way, when the size parameters of the first image are adjusted, the first image can be prevented from losing part of the image information, thereby ensuring the user experience. Furthermore, the first image and the second image are displayed side by side on the user interface, which is convenient for the user to view the expanded content of the second image, and can intuitively compare the difference between the expanded second image and the first image, thereby facilitating the user to obtain a more satisfactory second image.

[0121] In one possible implementation, after driving the display to display the first image in the first display area, the controller is further configured to: in response to the extended parameters input by the user in the parameter setting area, drive the display to display a prompt box corresponding to the extended parameters in the first display area, and the size of the prompt box is the size of the second image.

[0122] In this way, when the user enters the extended parameters, he can intuitively view the size of the second image after the extended parameters are expanded on the first interface, thereby providing a reference for the user to enter or modify the extended parameters, thereby making it easier for the user to obtain a satisfactory second image and improving the user experience.

[0123] In one possible implementation, after driving the display to display a prompt box corresponding to the extended parameters in the first display area, the controller is further configured to: in response to the user dragging the border of the prompt box on the first display area, drive the display to move the border position of the prompt box in the first display area, and update the extended parameters entered in the parameter setting area according to the border position of the prompt box after the movement.

[0124] Among them, the prompt box displayed in the first display area can be selected. For example, the user clicks and long presses the border or corner of the prompt box on the user interface. Specifically, when long pressing the border of the prompt box and dragging the border, the size of the prompt box in one direction can be adjusted; when long pressing the corner of the prompt box and dragging the corner, the size of the prompt box in two directions can be adjusted, and the extended parameters corresponding to the adjusted size are displayed in real time in the parameter setting.

[0125] In this way, after generating a prompt box using extended parameters, the user can intuitively adjust the image size corresponding to the extended parameters by dragging the border, thereby realizing a customized expansion of the prompt box size, and displaying the corresponding extended parameters in real time in the parameter setting area according to the size of the prompt box when it is dragged, thereby providing users with a more convenient expansion method and improving the user experience.

[0126] In a possible implementation, the extended input area includes a text display area; after driving the display to display the first image in the first display area, such as Figure 6 As shown, the controller is further configured to execute steps S610 to S620.

[0127] S610, obtaining photo editing materials and / or determining a photo editing method according to the extended information input by the user in the text display area in a voice or text manner.

[0128] The text display area is Figure 5 In the extended input area 1, the user can enter extended information in text form in the text display area, or, Figure 5 The extended input area 2 is provided with a voice control, through which the user can input voice, and the controller recognizes the language and converts it into text to be displayed in the extended input area 1.

[0129] The extended information may be a photo editing material, or a photo editing method, or may include both the photo editing material and the photo editing method. For example, if the extended information is a photo editing material, the extended information input may be: adding a first material to the image; for another example, if the extended information is a photo editing method, the extended information input may be: increasing the image brightness; for another example, if the extended information includes a photo editing material and a photo editing method, the extended information input may be: increasing the image brightness and adding a second material at an appropriate position.

[0130] S620: adjusting the content of the first image according to the editing material and / or the editing method, and adjusting the content of the second image based on the adjusted content of the first image.

[0131] It should be noted that adjusting the content of the first image according to the retouching material and / or retouching method is a simple adjustment on the first image. For example, if the input extended information is: adding the first material to the image, then on the first image, the first material is simply added to the first image, and the edge position of the first material is not optimized on the first image. After the second image is modified accordingly, the edge position of the first material will be optimized so that the color of the first material matches that of the surrounding image, avoiding excessive contrast between the first material and the background pattern, that is, improving the fusion effect of the first material and the original image. For another example, if the input extended information is: increasing the image brightness, then the steps or marks of the image style adjustment are recorded in the modified area of ​​the first image, and the brightness of the second image is adjusted.

[0132] In this way, the user can adjust the first image by voice or text input, thereby facilitating user operation and improving user experience.

[0133] In a possible implementation, before adjusting the image information of the first image according to the retouching material and / or the retouching method, the controller is further configured to: obtain a circle instruction drawn by the user in the first display area, and determine a circle area of ​​the circle instruction in the first display area.

[0134] Specifically, the input of the circle instruction can be that the user draws a closed figure in the first display area, and the controller automatically recognizes that the area enclosed by the closed figure is the circle area corresponding to the circle instruction; or, a circle control for inputting the circle area is arranged on the first display area, and the user can trigger the circle control and draw a figure corresponding to the circle instruction, thereby inputting the circle instruction, or the user draws a figure corresponding to the circle instruction and triggers the circle control, thereby inputting the circle instruction.

[0135] It should be noted that the closed figure includes the case where the boundary of the first image is the edge of the figure. Figure 7 As shown in FIG. 1 , the lines drawn by the user on the first image form a closed situation, thereby forming a closed figure 1. Figure 8 As shown in FIG. 1 , the line drawn by the user on the first image forms a closed state with a boundary of the first image, thereby forming a closed figure 2. For another example, Fig. 9 As shown in FIG. 1 , the line drawn by the user on the first image forms a closed state with two boundaries of the first image, thereby forming a closed figure 3. For another example, Fig.10 As shown, the line drawn by the user on the first image is closed with the three boundaries of the first image, thereby forming a closed figure 4.

[0136] It should be noted that when the drawn lines divide the first image into two areas, the position of the circled area can be determined in a preset manner. For example, it can be determined according to the area of ​​the area, and the area with the smaller area is the circled area. For another example, it can also be selected by the user. In the scene selected by the user, after the division into two areas, the user needs to select a certain area so that the circle instruction takes effect, and the selected area is determined as the circled area.

[0137] After determining the photo editing material and the photo editing method, the controller is further configured to: adjust the content of the first image in the circled area according to the photo editing material and / or the photo editing method, and adjust the content of the second image based on the adjusted content of the first image.

[0138] For example, the photo editing material is the first material, and the photo editing method is to blur the first material. In actual application, the controller now adds the first material to the circled area and preliminarily determines the blurred position. After the diffusion model processes the image, the first material is displayed at a position corresponding to the circled area on the second image, and the first material is blurred, and the color difference between the first material and the background image is excessively processed.

[0139] In this way, based on the photo editing material and / or photo editing method, the user's photo editing area can be limited by applying a circle instruction, so that when the user wants to modify a partial image, it is avoided to affect the content of the non-adjustment area of ​​the first image.

[0140] In a possible implementation, after driving the display to display the first image in the first display area, the controller is further configured to: obtain a line instruction drawn by the user in the first display area; specifically, the attributes of the line instruction include one or more of line type, line color, or line thickness, that is, line instructions with different attributes represent different operations. For example, a line control for inputting a line instruction is arranged on the first display area, and after the user triggers the line control, he can select the line color, line thickness, and / or line type, and then draw the line instruction in the first display area, thereby implementing the input of the line instruction.

[0141] After the line instruction is input, the controller adjusts the content of the first image in response to the attributes of the line instruction, and adjusts the content of the second image based on the adjusted content of the first image.

[0142] The correspondence between the attributes of the line instruction and the adjustment content can be set according to user needs. For example, a red line indicates that the image brightness needs to be increased, and a black line indicates that the image brightness needs to be reduced. For another example, a solid line indicates that the modification needs to be responded to immediately on the second image, and a dotted line indicates that the modification does not need to be responded to immediately. When other adjustment contents change, the dotted line instruction representation adjustment content can be synchronously responded to on the second image; for another example, when the line thickness is a thick line, the thick line represents that the adjustment amplitude of the corresponding instruction is A, and the thin line represents that the adjustment amplitude of the corresponding instruction is B.

[0143] In this way, more forms of image editing methods are provided to the user through line instructions, making it easier for the user to modify and adjust the image in the first display area.

[0144] In one possible implementation, Fig.11 As shown, after obtaining the line instruction drawn by the user in the first display area, the controller is further configured to perform S1110 to S1130.

[0145] S1110 , in response to a line instruction, adding an instruction number to the line instruction.

[0146] Among them, the instruction number can be arranged in chronological order without considering the type of line instruction, for example, in the order of 000, 001, 002, 003,…, 009, 010, 011; it can also be arranged on the basis of chronological order and combined with the type of line instruction, for example, the first position of the instruction number is a letter, and the subsequent positions are numbers, and the instruction numbers of the same type of line instructions have the same first letter.

[0147] S1120, in response to the user's triggering operation on the modification control in the first display area, driving the display to display the instruction number corresponding to the line instruction in the first display area.

[0148] In actual application, the instruction number of the line instruction is set to be hidden. When the user triggers the modification control, the instruction number of the line instruction is displayed in the first display area, and after the instruction number is displayed, it can be selected by the user. Selecting the instruction number means selecting the corresponding line instruction.

[0149] S1130, in response to the modification instruction input by the user, modify the adjustment content corresponding to the line instruction on the first image, and modify the content of the second image based on the modification content corresponding to the modification instruction.

[0150] The modification instruction includes the instruction number corresponding to the line instruction and the corresponding modification method. Specifically, before responding to the modification instruction input by the user, it is also necessary to determine the content to be modified selected by the user, that is, the modification instruction needs to select a specific line instruction.

[0151] In this way, by selecting the line instruction, the content that needs to be modified is determined. For example, the user selects instruction number N in the first display area, and the adjustment content corresponding to the instruction number is: increase the image brightness, that is, the content that needs to be modified by the modification instruction is the previous operation of increasing the image brightness. At this time, you can select the modification method of the modification instruction. For example, if the modification method is to delete, then after the user selects to delete and confirm, the controller withdraws the adjustment content corresponding to the instruction number N. For another example, if the modification method is to repeat execution, then after the user selects to reread and execute and confirm, the controller executes the adjustment content corresponding to the instruction number N again.

[0152] The purpose of the modification instruction is to modify the content adjusted by the line instruction. For example, if the adjustment content corresponding to the line instruction is deleted, the modification method is deletion; or if the adjustment content of the line instruction is repeatedly executed, the modification method is repetition.

[0153] In this way, when the user needs to modify the adjustment content corresponding to the previous line instruction, the modification action can be implemented more conveniently, avoiding the situation where the adjustment content cannot be rolled back, causing the user to need to readjust the image.

[0154] In one possible implementation, after expanding the image size of the first image in response to the expansion parameters entered by the user in the parameter setting area, the controller is further configured to: generate a second image with the image expansion quantity entered by the user in the parameter setting area in response to the image expansion quantity entered by the user in the parameter setting area, and drive the second display area of ​​the display to cyclically display the second image, wherein the contents of the multiple second images cyclically displayed are different.

[0155] When the diffusion model generates the second image, the image of the newly added area after expansion has a certain degree of randomness, that is, the result generated each time is not completely consistent. In the actual application process, multiple second images can be generated based on the number of image expansions input by the user. In this way, the user can choose one from the multiple second images to use, thereby improving the user experience.

[0156] Among them, when multiple second images are displayed in a loop, multiple second images can be replaced by switching or sliding. The display time of each second image can be set, and the display device automatically replaces the second image, or the user manually switches the second image. In both automatic switching and manual switching, the user can freely switch the display mode, and can select a second image to display, and exit the loop display mode, or remove a second image from the multiple second images displayed in a loop. For example, when the second image is displayed in a rotating manner, long press the second image by touch to fix the clicked second image in the second display area, and call out the control menu, so as to choose to fix the second image or delete the second image in the control menu, or determine that the second image is the final expanded image in the control menu, and choose to save, share or set it as the background image of the user interface.

[0157] In one possible implementation, the image expansion interface also includes an application expansion area; after driving the display to display the second image in the second display area, the controller is further configured to: in response to a secondary expansion instruction input by the user in the application expansion area, drive the display to display the second image in the first display area.

[0158] In this way, the user can, on the basis of accepting the second image, further expand the new image with the content of the second image as the original image, so as to avoid the situation in which the content of the accepted second image is lost or changed during the subsequent modification process.

[0159] Among them, after driving the first display area to display the second image, the display of the second image in the second display area can be cancelled, that is, the second display area is blank, or the second image in the second display area can be retained until a new second image is generated, and the original second image is updated to the new second image, or a prompt message can be displayed in the second display area, such as the prompt message: no new second image is detected, to indicate that no new second image is currently generated.

[0160] In one possible implementation, the application extension area is also provided with an output control; after driving the display device to display the second image in the second display area, the controller is further configured to: in response to a user triggering operation on the output control in the application extension area, output the second image to an image receiving end corresponding to the output control.

[0161] For example, Figure 5As shown, there are two output controls, the share control and the download control, in the application extension area. After the user triggers the share control, the controller generates a share link for the second image, and the user can choose to copy the share link. Alternatively, the share link can be sent by forwarding the application. Alternatively, the share link can be saved in the local data path; after the user triggers the download control, the controller generates the second image information and outputs the second image information to the corresponding image receiving end, or outputs the second image information to the local database.

[0162] In this way, when the user needs to use the second image in other occasions, it is convenient for the user to save or share the second image.

[0163] In one possible implementation, the controller expands the image size of the first image to obtain the second image, and is specifically configured to perform: obtaining the original size of the first image; expanding the size of the first image according to the original size of the first image and the expansion parameters to obtain a third image, the aspect ratio of the expanded third image being equal to the aspect ratio in the expansion parameters; proportionally scaling the size of the third image according to the expansion parameters to obtain the second image, and driving the display to display the second image in the second display area.

[0164] Among them, the length and width data in the extended parameters can be entered manually, or a sample image can be entered and the controller can identify the length and width data corresponding to the sample image. Alternatively, if the length and width data is the size of a user interface of the display device, the size of the user interface can be identified by the controller.

[0165] First, based on the proportional relationship between the length and width data in the extended parameters, the original size of the first image is expanded to obtain the third image, so that the length and width ratio of the third image is equal to the length and width ratio of the length and width data in the extended parameters, and then, the size of the third image is proportionally scaled to obtain the second image. In this way, the loss of picture information of the first image can be avoided.

[0166] For example, in a specific scenario, the user sets the first image taken by the mobile phone as the background image of the smart TV. At this time, the aspect ratio and the size of the first image do not match the size of the user interface of the smart TV. Compared with the existing solution of first cropping the first picture to a suitable ratio and then enlarging it to fill the user interface, the method of the present application first matches the size of the user interface and then expands the inappropriate first image to a size suitable for the user interface, and finally enlarges it to fill the user interface. This can better avoid the loss of picture information of the first image, and on this basis, it can also trigger the clarity control of the application extension area to further improve the clarity of the second image.

[0167] In order to more clearly illustrate the technical solution of this application, Fig.12Taking the extended flow chart of as an example, an embodiment of the technical solution of the present application is introduced. Fig.12 As shown, in a possible implementation, the expansion process includes S1201 to S1213.

[0168] S1201, the user opens an image expansion application on a display device.

[0169] After opening the image expansion application, the display device can accept the user's voice input, text input and control command input. In actual application, after opening the image expansion application, you can directly enter the image expansion interface, which is equivalent to entering the expansion command. Or, when the image expansion application integrates many functions, after opening the image expansion application, after triggering the expansion command, you enter the image expansion interface.

[0170] In some embodiments, the expansion instruction can also be integrated into the album, so that after the user selects an image in the album, the expansion instruction can be triggered on the image boundary page, and the controller calls the image expansion application and enters the image expansion interface of the image expansion application.

[0171] S1202: The image expansion application of the display device sends an image expansion request to the image expansion service.

[0172] Before sending the image expansion request, the user needs to input the first image and expansion parameters. The expansion parameters can be obtained by the controller. For example, if the user triggers the expansion instruction in the background image setting interface of the user interface, the default expansion parameter is the size of the display area that can be displayed on the display.

[0173] From S1203 to S1204, the image expansion service determines the size of the first image to be expanded based on the expansion parameters through the adaptive scaling module.

[0174] At this time, a prompt box corresponding to the extended parameter may be displayed in the first display area, and the user may adjust the size of the prompt box by dragging the border of the prompt box.

[0175] S1205, the image expansion service sends an extended part filling request to the filling module.

[0176] In some embodiments, the user can add a photo editing requirement based on the extension, for example, by inputting extended information including photo editing materials and / or photo editing methods in the current display area through voice or text, or by drawing line instructions in the first display area.

[0177] S1206 to S1208: When the user inputs a prompt word, the content filling module redraws the first image based on the prompt word and obtains a second image.

[0178] In some embodiments, if the user does not input a prompt word, the content filling module only needs to expand the first image according to the expansion parameters.

[0179] Since the diffusion model is a type of probabilistic generation model, in actual application, it is possible to choose to generate multiple second images, or repeat the instruction extension command to obtain more second images.

[0180] S1209: When the image needs to be adjusted in definition, the low-definition second image is sent to the pixel processing module. S1210: The pixel processing module adjusts the pixels of the low-definition second image based on the image processing model to obtain a high-definition second image.

[0181] S1211, the pixel processing module sends a high-definition second image to the image expansion service.

[0182] S1212: The image enlargement service sends a high-definition second image to the image enlargement application of the display device.

[0183] S1213, the image expansion application of the display device displays the high-definition second image on the implementation interface.

[0184] In order to more clearly illustrate the technical solution of this application, Fig.13 Taking the signal trigger state point schematic diagram as an example, an embodiment of the technical solution of the present application is introduced. In the following cases, the diffusion model will be triggered to generate a second image.

[0185] The first case: the signal monitoring module determines the signal state at a preset frequency, for example, once per second. If the signal state is updated, the diffusion model is triggered to generate a second image.

[0186] The signal status includes uploading the first image, voice input, updating the parameter setting area extension parameters or the image extension quantity.

[0187] The second case: voice signal input, including simple commands: please help me expand the picture, complex commands: the user inputs the content he wants to expand, and is triggered when the long press of the voice control ends.

[0188] The third case: the image control is triggered every time it is updated, including scanning a code to upload an image, taking a photo to upload an image, voice input, parameter content update in the parameter setting area, for example, extended parameter input, prompt word input and other operations. After obtaining the updated content, clear the update status, and determine whether all signals are None. If not, trigger the diffusion model to generate the second image. If it is None, the input or update of the extended parameter, the first image, the image extension number or the prompt word is not detected, and the second image in the second display area is cleared.

[0189] In a second aspect, a display method is provided, which is applied to a display device, wherein the display device includes a display for displaying a user interface, and the display method includes:

[0190] In response to an expansion instruction input by a user, the display is driven to display an image expansion interface on the user interface; the image expansion interface includes a first display area, a second display area, an expansion input area and a parameter setting area.

[0191] In response to first image information input by a user in the extended input area, the display is driven to display a first image in the first display area; the first image information is used to represent the first image.

[0192] In response to an expansion parameter input by a user in the parameter setting area, the image size of the first image is expanded to obtain a second image.

[0193] The display is driven to display the second image in the second display area.

[0194] A display method provided in an embodiment of the present application first drives a display to display an image expansion interface on a user interface in response to an expansion instruction input by a user; then drives the display to display a first image in a first display area in response to first image information input by the user in an expansion input area; then expands the image size of the first image to obtain a second image in response to an expansion parameter input by the user in a parameter setting area; and finally drives the display to display the second image in a second display area.

[0195] By applying the technical solution of the present application, when adjusting the size parameters of the first image, it is possible to avoid the picture from losing part of the image information, thereby ensuring the user experience, and displaying the first image and the second image side by side on the user interface, making it easy to view the expanded content of the second image and intuitively compare the difference between the expanded second image and the first image, thereby helping the user to obtain a more satisfactory second image.

Claims

1. A display device, characterized in that: include: a display configured to display a user interface; A controller is coupled to the display and is configured to: In response to an expansion instruction input by a user, the display is driven to display an image expansion interface on the user interface; the image expansion interface includes a first display area, a second display area, an expansion input area and a parameter setting area; In response to first image information input by a user in the extended input area, driving the display to display a first image in the first display area; the first image information is used to represent the first image; In response to an expansion parameter input by a user in a parameter setting area, expanding the image size of the first image to obtain a second image; The display is driven to display the second image in the second display area.

2. The display device according to claim 1, characterized in that After driving the display to display the first image in the first display area, the controller is further configured to: In response to the extended parameter input by the user in the parameter setting area, the display is driven to display a prompt box corresponding to the extended parameter in the first display area, and the size of the prompt box is the size of the second image.

3. The display device according to claim 2, characterized in that After driving the display to display a prompt box corresponding to the extended parameter in the first display area, the controller is further configured to: In response to the user dragging the border of the prompt box on the first display area, the display is driven to move the border position of the prompt box in the first display area, and the extended parameters input in the parameter setting area are updated according to the border position of the prompt box after the movement.

4. The display device according to claim 1, characterized in that The extended input area includes a text display area; after driving the display to display the first image in the first display area, the controller is further configured to: Acquire photo editing materials and / or determine a photo editing method according to the extended information input by the user in the text display area in a voice or text manner; According to the photo editing material and / or the photo editing method, the content of the first image is adjusted, and based on the adjusted content of the first image, the content of the second image is adjusted.

5. The display device according to claim 4, characterized in that Before adjusting the image information of the first image according to the image editing material and / or the image editing method, the controller is further configured to: Acquire a circle instruction drawn by a user in the first display area, and determine a circle area of ​​the circle instruction in the first display area; According to the photo editing material and / or photo editing method, the content of the first image in the circled area is adjusted, and based on the adjusted content of the first image, the content of the second image is adjusted.

6. The display device according to claim 1, characterized in that After driving the display to display the first image in the first display area, the controller is further configured to: Obtaining a line instruction drawn by the user in the first display area; In response to the attributes of the line instruction, the content of the first image is adjusted, and based on the adjusted content of the first image, the content of the second image is adjusted; the attributes of the line instruction include one or more of line type, line color or line thickness.

7. The display device according to claim 6, characterized in that After acquiring the line instruction drawn by the user in the first display area, the controller is further configured to: In response to the line instruction, adding an instruction number to the line instruction; In response to the user's triggering operation on the modification control in the first display area, the display is driven to display the instruction number corresponding to the line instruction in the first display area; In response to a modification instruction input by a user, the adjustment content corresponding to the line instruction on the first image is modified, and based on the modification content corresponding to the modification instruction, the content of the second image is modified, wherein the modification instruction includes an instruction number corresponding to the line instruction and a corresponding modification method.

8. The display device according to any one of claims 5 to 7, characterized in that: After expanding the image size of the first image in response to the expansion parameter input by the user in the parameter setting area, the controller is further configured to: In response to the image expansion number input by the user in the parameter setting area, a second image of the image expansion number is generated, and the second display area of ​​the display is driven to cyclically display the second image, and the contents of the multiple cyclically displayed second images are different.

9. The display device according to claim 1, characterized in that: The image extension interface also includes an application extension area; After driving the display to display the second image in the second display area, the controller is further configured to: In response to a secondary expansion instruction input by a user in the application expansion area, the display is driven to display the second image in the first display area.

10. The display device according to claim 1, characterized in that The image extension interface also includes an application extension area; After driving the display device to display the second image in the second display area, the controller is further configured to: In response to a triggering operation of an output control by a user in the application extension area, the second image is output to an image receiving end corresponding to the output control.

11. The display device according to claim 1, characterized in that The controller expands the image size of the first image to obtain the second image, and is specifically configured as follows: Get the original size of the first image; According to the original size of the first image and the expansion parameter, the size of the first image is expanded to obtain a third image, wherein the aspect ratio of the expanded third image is equal to the aspect ratio in the expansion parameter; According to the expansion parameter, the size of the third image is proportionally scaled to obtain the second image, and the display is driven to display the second image in the second display area.

12. A display method, characterized in that: Applied to a display device, the display device includes a display for displaying a user interface, and the display method includes: In response to an expansion instruction input by a user, the display is driven to display an image expansion interface on the user interface; the image expansion interface includes a first display area, a second display area, an expansion input area and a parameter setting area; In response to first image information input by a user in the extended input area, driving the display to display a first image in the first display area; the first image information is used to represent the first image; In response to an expansion parameter input by a user in a parameter setting area, expanding the image size of the first image to obtain a second image; The display is driven to display the second image in the second display area.

Images