Display device and image processing method

Abstract

The present application relates to a display device and an image processing method. The method comprises: receiving an image saving instruction, and in response thereto, associatively storing a first image and a second image, wherein the first image and the second image have an image-to-image relationship; displaying the first image and the second image on a user interface in a superimposed manner, wherein the second image is displayed in a first display area of the user interface, the first image is displayed in a second display area in the first display area, the second display area is smaller than the first display area, and the layer of the image displayed in the second display area is located above the layer of the image displayed in the first display area; receiving a view switching instruction, and in response thereto, switching the image displayed in the first display area to be displayed in the second display area, and switching the image displayed in the second display area to be displayed in the first display area. The present method effectively improves the user interaction experience.

Description

Display devices and image processing methods

[0001] Related applications

[0002] This application claims priority to Chinese patent applications filed on March 21, 2025, with application number 202510344974X, entitled "Image Processing Method and Display Device", and on April 30, 2025, with application number 2025105719866, entitled "Display Device and Display Content Adjustment Method", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of display device technology, and in particular to a display device and an image processing method. Background Technology

[0004] With the rapid development of display devices and the increasing diversification of user needs, people's demand for the intelligence of display devices such as smart TVs is also increasing, and the functions of display devices are becoming more and more abundant.

[0005] Currently, users can interact with display devices through various methods such as remote control and voice interaction. However, the user experience still has certain limitations. Summary of the Invention

[0006] Therefore, it is necessary to provide a display device and image processing method that can improve the user's interactive experience in response to the above-mentioned technical problems.

[0007] In a first aspect, this application provides a display device, the device comprising:

[0008] The monitor is configured to display the user interface;

[0009] and at least one controller, configured as follows:

[0010] Receive and respond to an image save command, and associate and store a first image and a second image, wherein the first image and the second image have an image-to-image relationship;

[0011] The first image and the second image are superimposed on the user interface, wherein the second image is displayed in a first display area of ​​the user interface, the first image is displayed in a second display area of ​​the first display area, the second display area is smaller than the first display area, and the layer of the image displayed in the second display area is above the layer of the image displayed in the first display area;

[0012] Upon receiving and responding to a view switching command, the image displayed in the first display area is switched to be displayed in the second display area, and the image displayed in the second display area is switched to be displayed in the first display area.

[0013] Secondly, this application also provides an image processing method, comprising:

[0014] Receive and respond to an image save command, and associate and store a first image and a second image, wherein the first image and the second image have an image-to-image relationship;

[0015] The first image and the second image are superimposed on the user interface, wherein the second image is displayed in a first display area of ​​the user interface, the first image is displayed in a second display area of ​​the first display area, the second display area is smaller than the first display area, and the layer of the image displayed in the second display area is above the layer of the image displayed in the first display area;

[0016] Upon receiving and responding to a view switching command, the image displayed in the first display area is switched to be displayed in the second display area, and the image displayed in the second display area is switched to be displayed in the first display area.

[0017] The aforementioned display device and image processing method, by associating and storing the first image and the second image, can better manage and process the first image and the second image. By superimposing the first image and the second image on display areas of different sizes in the user interface, and with the image displayed in the second display area having a higher display layer than the image displayed in the first display area, users can view the first image and the second image simultaneously on the user interface without switching pages, reducing the visual fragmentation caused by repeatedly switching views. Users can freely select and switch the display areas of the first image and the second image by sending view switching commands, allowing users to focus on a specific image according to their needs, improving the flexibility of image viewing, and effectively enhancing the overall user interaction experience. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the conventional technology, the drawings used in the description of the embodiments or the conventional technology will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the disclosed drawings without creative effort.

[0019] Figure 1 is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of this application;

[0020] Figure 2 is a schematic diagram of the hardware configuration of a display device provided in some embodiments of this application;

[0021] Figure 3 is a schematic diagram of the hardware configuration of the control device provided in some embodiments of this application;

[0022] Figure 4 is a schematic diagram of the software configuration of a display device provided in some embodiments of this application;

[0023] Figure 5 is a schematic diagram showing the position of the cursor on the remote control provided in some embodiments of this application;

[0024] Figure 6 is a schematic diagram of a pointing and pressing operation on a remote control provided in some embodiments of this application;

[0025] Figure 7 is a schematic diagram of a remote control selection operation provided in some embodiments of this application;

[0026] Figure 8 is a schematic diagram of multi-selection operation of a pointing remote control provided in some embodiments of this application;

[0027] Figure 9 is a schematic diagram of a point-and-drag operation on a remote control provided in some embodiments of this application;

[0028] Figure 10 is a schematic diagram of a pointing remote control sliding operation provided in some embodiments of this application;

[0029] Figure 11 is a schematic diagram of a continuous sliding operation of a remote control provided in some embodiments of this application;

[0030] Figure 12 is a schematic diagram of a reverse sliding operation of a remote control provided in some embodiments of this application;

[0031] Figure 13 is a schematic flowchart of the controller performing image processing steps according to some embodiments of this application;

[0032] Figure 14 is a schematic diagram showing the superimposed display of a first image and a second image provided in some embodiments of this application;

[0033] Figure 15 is a schematic diagram of the view switching process provided in some embodiments of this application;

[0034] Figure 16 is a schematic diagram of canvas adaptive landscape and portrait screens provided in some embodiments of this application;

[0035] Figure 17 is a schematic diagram of displaying drawing trajectories on a user interface according to some embodiments of this application;

[0036] Figure 18 is an interactive timing diagram of the image processing method provided in some embodiments of this application;

[0037] Figure 19 is an interactive timing diagram of the display device executing the image processing method according to some embodiments of this application;

[0038] Figure 20 is a schematic diagram of the display interface provided in some embodiments of this application;

[0039] Figure 21 is a schematic diagram of the preview image displacement process provided in some embodiments of this application;

[0040] Figure 22 is a schematic diagram of the interaction architecture provided in some embodiments of this application;

[0041] Figure 23 is a schematic diagram of the movement of the position marker box provided in some embodiments of this application;

[0042] Figure 24 is a schematic diagram of the movement of another position marker box provided in some embodiments of this application. Detailed Implementation

[0043] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0044] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0045] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0046] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0047] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0048] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0049] In this embodiment, the display device 200 generally refers to a device with screen display and data processing capabilities. For example, the display device 200 includes, but is not limited to, smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, augmented reality devices, etc.

[0050] Figure 1 is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of this application. As shown in Figure 1, a user can operate the display device 200 through touch operation, a mobile terminal 300, and a control device 100. For example, the control device 100 can be a remote control, a stylus, a gamepad, etc.

[0051] As also shown in Figure 1, the display device 200 communicates with the server 400 via various communication methods. The display device 200 can communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0052] Figure 2 is a hardware configuration block diagram of the display device 200 in Figure 1 provided in some embodiments of this application.

[0053] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface 280.

[0054] Figure 3 is a hardware configuration block diagram of the control device in Figure 1 provided in some embodiments of this application. As shown in Figure 3, the control device 100 may include: a controller 110, a communication interface 130, a user input / output interface 140, a memory 190, and a power supply 180.

[0055] The controller 110 includes a processor 112, RAM 113 and ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation of the control device 100, as well as the communication and cooperation between internal components and the external and internal data processing functions.

[0056] The communication interface 130 may include at least one of other near-field communication modules such as WiFi chip 131, Bluetooth module 132, NFC module 133, etc.

[0057] User input / output interface 140, wherein the input interface includes at least one of other input interfaces such as microphone 141, touchpad 142, sensor 143, and button 144.

[0058] In some embodiments, to enable user interaction, the display device 200 may run an operating system. The operating system is a computer program used to manage and control the hardware and software resources of the display device 200. The operating system can (control the display device) provide a user interface, allowing users to interact with the display device 200 and supporting the running of various applications.

[0059] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system that is deeply customized based on a specific operating platform, or an independent operating system specifically developed for display devices.

[0060] Operating systems can be divided into different modules or layers based on the functions they implement. For example, as shown in Figure 4, in some embodiments, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), the System Library layer, and the Kernel layer.

[0061] As shown in Figure 4, the application framework layer in this embodiment includes a ViewSystem, Managers, and ContentProviders. The ViewSystem designs and implements the application's interface and interactions, and includes lists, grids, textboxes, and buttons. The Managers include at least one of the following modules: an ActivityManager for interacting with all running activities in the system; a LocationManager for providing system services or applications with access to system location services; a PackageManager for retrieving various information related to application packages currently installed on the device; a NotificationManager for controlling the display and clearing of notification messages; and a WindowManager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0062] In some embodiments, as shown in FIG4, the kernel layer may be configured with hardware drivers. The drivers included in the kernel layer may be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0063] It should be noted that the above examples are merely a simple division of operating system functions and do not limit the specific form of the operating system of the display device 200 in this application embodiment. Depending on the function of the display device, the type of operating system, and other factors, the number of levels and the specific level type of the operating system may be expressed in other forms.

[0064] With the rapid development of display devices and the increasing diversification of user needs, people have higher and higher demands for the intelligence of display devices such as smart TVs, and the functions of display devices are becoming more and more abundant. For example, users can create picture books on display devices by interacting with them.

[0065] However, currently, if users want to view (or preview) images on a display device, they can usually only view one image at a time. If they need to view multiple images, they need to switch pages to view them separately. This creates a sense of visual fragmentation for users and reduces the interactive experience.

[0066] Accordingly, some embodiments of this application provide a display device, the display device including: a display, wherein the display is configured to: display a user interface, the user interface including at least one control; the user interface also has a cursor, the position of the cursor on the user interface being determined by the position pointed to by a remote controller in three-dimensional space.

[0067] Optionally, users can control the display device to start via voice, remote control, or the power switch of the TV. After the display device is started, the user interface corresponding to the homepage is displayed on the screen. Users can switch to other user interfaces via voice or remote control, such as the user interface corresponding to the TV series tag, the user interface corresponding to the movie tag, or the user interface corresponding to a certain page provided by a certain application. This application embodiment does not limit this.

[0068] Optionally, among all the user interfaces displayed on the screen, some user interfaces contain controls, while others may not contain controls. The solution provided in this application embodiment is for user interfaces that contain controls. The number of controls on a user interface that contains controls may be only one or more. This application embodiment does not limit this.

[0069] Optionally, the remote control used with the display device can be a remote control with pointing function, hereinafter referred to as a pointing remote control. When the pointing function is enabled, the user interface displayed on the screen also has a cursor. The position of the cursor on the user interface is determined by the position pointed to by the pointing remote control in three-dimensional space. The position pointed to by the pointing remote control in three-dimensional space in this embodiment is also called the position data of the pointing remote control in three-dimensional space. The specific implementation process of determining the position of the cursor on the user interface based on the position data of the pointing remote control is described below.

[0070] The position data of the pointing remote control in three-dimensional space can include: position coordinates and pointing direction. The position coordinates are the physical position of the pointing remote control relative to the display device (e.g., horizontal distance, vertical height, etc.). The pointing direction is the pointing angle of the pointing remote control (e.g., horizontal and vertical angles relative to the display device). The pointing remote control has a built-in gyroscope to detect the rotation angle of the remote control and an accelerometer to detect the acceleration of the remote control's movement. Through these sensors, the pointing remote control can determine its own position data in three-dimensional space in real time.

[0071] In some embodiments, after acquiring its own position data in three-dimensional space, the remote controller can send the position data to the display device. The display device can perform screen coordinate transformation based on the position data, thereby converting the position data of the remote controller in three-dimensional space to the screen coordinate system. The obtained screen coordinates can be used as the position of the cursor on the user interface, and the cursor can be drawn at that position to display the cursor.

[0072] In some embodiments, the remote controller includes a coordinate transformation module. After acquiring its own position data in three-dimensional space, the remote controller can input this position data into the coordinate transformation module. The coordinate transformation module can perform screen coordinate transformation based on the position data, thereby converting the position data into the screen coordinate system to obtain screen coordinates. These screen coordinates can be used as the cursor position on the user interface, and the cursor position on the user interface is sent to the display device. After receiving the cursor position on the user interface, the display device draws the cursor at that position, thereby displaying the cursor. For example, if the cursor position on the user interface is coordinates (a, b), the display device draws the cursor at coordinates (a, b) on the user interface, thereby displaying the cursor.

[0073] For example, referring to Figure 5, when the position data of the remote control in three-dimensional space is A, the cursor position on the user interface is coordinates (a1, b1); when the position data of the remote control in three-dimensional space is B, the cursor position on the user interface is coordinates (a2, b2). When the position data of the remote control in three-dimensional space is C, the screen coordinates calculated in the above manner exceed the screen range, the display device no longer draws the cursor, and therefore, the cursor disappears.

[0074] Referring to Figure 6, the remote control supports tapping. When the cursor hovers over a control, the user can press the touch area (confirmation button) on the remote control, and the display device will respond to the operation by executing the corresponding logic. The control can be a card, button, input box, radio button, switch, dialog box, multi-select button, slider, playback progress bar, scroll bar, drop-down menu, label, list, etc. This application embodiment does not limit this.

[0075] Referring to Figure 7, the remote control supports selection. When the user interface displays multiple options and their corresponding selection boxes, if the user wants to select the first and third options, they can first control the cursor to hover over the selection box corresponding to the first option and press the touch area on the remote control. At this time, the first option is selected. Then, the user can control the cursor to hover over the selection box corresponding to the third option and press the touch area on the remote control. At this time, the third option is selected, thus completing the selection.

[0076] As shown in Figure 8, the remote control supports drag-and-drop multi-selection. When the user interface displays multiple options and their corresponding selection boxes, if the user wants to select all options, they can control the cursor to hover over any position in the area above the first option, press the touch area on the remote control and drag it down until the cursor moves to the area below the last option. At this time, all options are selected, thus completing drag-and-drop multi-selection.

[0077] Referring to Figure 9, the remote control supports dragging. When a video is playing on the user interface and a progress bar is displayed, the user can hover the cursor over a position on the progress bar and then press the touch area on the remote control. The video playing on the user interface will then fast forward or rewind to the corresponding video frame, thus completing the dragging of the playback progress. The progress bar in this example is only one example; draggable controls also include sliders, scroll bars, labels, cards, lists, etc., and this application embodiment does not limit the scope of these controls.

[0078] As shown in Figure 10, the remote control supports swiping. When the user interface displays a cursor, the user can swipe up, down, left, or right on the touch area of ​​the remote control, and the display device will respond to these swipes.

[0079] Referring to Figure 11, the remote control supports continuous swiping. When a cursor is displayed on the user interface, the user can continuously swipe upwards, downwards, leftwards, or rightwards on the touch area of ​​the remote control. The display device will execute these continuous swipe responses and display a damped swipe effect on the user interface. Figure 11 is a schematic diagram of continuous upward swiping.

[0080] Referring to Figure 12, the remote control supports continuous reverse swipes. When a cursor is displayed on the user interface, the user can swipe in one direction and then in the opposite direction on the touch area of ​​the remote control. The display device will respond to the continuous reverse swipes and show a damped swipe effect on the user interface. Figure 12 is a schematic diagram of swiping up and then down.

[0081] In some embodiments, as shown in Figures 1 and 2, this application provides a display device 200, which may include a display 260 and a controller 250, wherein the display 260 is configured as a user interface. As shown in Figure 13, the controller 250 is configured to perform the following steps (S) 200 to S600, wherein:

[0082] S200, receive and respond to the image saving instruction, and store the first image and the second image together, wherein the first image and the second image have an image-to-image relationship.

[0083] Image saving instructions are commands used to instruct the saving of specific image data to a storage medium. The first image and the second image refer to two different images that have a graph-generated image relationship. Specifically, a graph-generated image relationship refers to a dependency relationship between two images, such as one image being the result of editing, transforming, or applying certain algorithms to another image. For example, the first image may be generated or modified based on the second image, or vice versa; this is not limited to specific cases.

[0084] In practical applications, this can refer to a situation where a first or second image is displayed on the user interface of a display device. The user, wishing to save the image, interacts with the display device and sends an image save command. Upon receiving the image save command, the display device first parses the command to determine the image to be saved and its location. Then, it reads the image data from memory and encodes it into a specified format (e.g., JPEG, PNG), subsequently performing the associated storage operation for the image.

[0085] For example, taking an image generated based on a first image as an example, the display device can generate one or more second images based on the first image. One or more second images are displayed on the user interface of the display device. The user selects a second image to save. In response to the save event, the display device generates an image generation instruction. Subsequently, in response to this instruction, it identifies the second image to be saved. Then, based on the second image, it finds the identification data of the first image with a graph-to-graph relationship, such as an image ID (identity) or number. Then, based on the ID of the first image, it associates and stores the first and second images. It is understood that, simultaneously with the generation of the second image, the display device can record the relationship (graph-to-graph relationship) between the two images in the metadata through metadata tags. In other embodiments, the first image can also be generated based on the second image. In response to the image save instruction for the first image, the display device finds the ID of the second image with a graph-to-graph relationship, and associates and stores the first and second images based on the ID of the second image. The specific method can be determined according to the actual situation and is not limited here.

[0086] S400, the first image and the second image are superimposed and displayed on the user interface, wherein the second image is displayed in the first display area of ​​the user interface, the first image is displayed in the second display area of ​​the first display area, the second display area is smaller than the first display area, and the layer of the image displayed in the second display area is above the layer of the image displayed in the first display area.

[0087] Image overlay refers to displaying multiple images on the same interface or screen in a way that follows certain rules (such as position, size, and hierarchy), rather than displaying each image individually and sequentially. Overlay displays allow users to view multiple images simultaneously on the same screen and achieve different visual effects and interactive experiences by adjusting the display attributes of these images.

[0088] The display area refers to a specific partition or display space within the user interface used to display images. The position and size of the display area can be set independently. In this embodiment, the first display area is a display space defined on the user interface, which can be the entire screen or a large window area. The second display area is a smaller display space defined within the first display area, smaller in size. Furthermore, the image displayed in the second display area has a higher display layer than the image displayed in the first display area. This means that the image displayed in the second display area will cover the image displayed in the first display area and will not be obscured, ensuring that the user can always view the images displayed in both the first and second display areas simultaneously on the user interface.

[0089] Following the previous step, after associating and storing the first and second images, the display device can set different display layers for the first and second images, and adjust the position and size of the two images according to preset rules (such as display area size and layer), placing them in different display areas of the user interface for display to achieve a superimposed effect. In this embodiment, the result image generated based on one image can be displayed in the larger second display area by default, making it easier for the user to clearly view the result image. Taking the second image as the result image generated based on the first image as an example, the second image is displayed in the larger first display area, and the first image is displayed in the smaller second display area. The first and second images can completely fill the display area, or they can not completely fill the display area, but fill the display area in an appropriate proportion. The areas that are not completely filled can be filled with the background color, or filled with a specified color such as black or white.

[0090] For example, as shown in Figure 14, the second image can be displayed in full screen as the main image of interest, and the first image can be overlaid on the second image as auxiliary information in the form of a thumbnail. In the figure, the large rectangle is the first display area and the small rectangle is the second display area.

[0091] It is understood that in other embodiments, the result image generated from one of the images may be displayed in a small display area by default, as long as the user can see the result image clearly, and no specific limitation is made here.

[0092] S600 receives and responds to a view switching command, switching the image displayed in the first display area to be displayed in the second display area, and switching the image displayed in the second display area to be displayed in the first display area.

[0093] View switching instructions are used to adjust the display positions of different images on the user interface in a multi-image display scenario. In this embodiment, the goal of the view switching instruction is to swap the display areas and / or display sizes of the first image and the second image on the user interface. In other words, it switches the image that was originally displayed in the larger first display area to the smaller second display area, and switches the image that was originally displayed in the smaller second display area to the larger second display area.

[0094] Following the previous step, let's take the initial overlay display of the first and second images as an example. The second image is displayed in full screen, while the first image is displayed as a thumbnail over the second. Users can send view switching commands to the display device via voice interaction or by clicking on an image area. After recognizing and parsing the command, the display device switches the first image to full-screen display, with the second image displayed as a thumbnail over the first. The switching process is shown in Figure 15. If the user sends another view switching command, the second image is restored to full-screen display, and the first image is displayed as a thumbnail over the second image again. In other words, each time the user sends a view switching command, the display device switches the display area and size of the two images. Thus, by dynamically adjusting the display layer and size of these two images, users can quickly switch focus as needed for more effective comparison and analysis.

[0095] The aforementioned display device facilitates subsequent image management and version control by associating and storing the first and second images. By overlaying the first and second images on display areas of different sizes in the user interface, with the image displayed in the second display area having a higher display layer than the image displayed in the first display area, users can simultaneously view the first and second images on the user interface without switching pages, reducing the visual fragmentation caused by repeatedly switching views. Users can freely select and switch the display areas of the first and second images by sending view switching commands, allowing users to focus on a specific image according to their needs, thus improving the flexibility of image viewing and effectively enhancing the overall user interaction experience.

[0096] There are no restrictions on how users send view switching commands. In some exemplary embodiments, view switching commands are triggered in any of the following ways:

[0097] The first method is triggered by detecting click operations on a screen area, which includes a first display area, a second display area, or any area of ​​the screen.

[0098] The second method is to trigger it through voice interaction.

[0099] The third method is to trigger it through gesture recognition.

[0100] The fourth method is to trigger it by tapping a directional device.

[0101] In practical applications, taking smart display devices as an example, these devices support multiple interaction methods, including but not limited to touchscreen interaction, voice interaction, gesture interaction, and interaction via directional devices. Users can send view switching commands through touchscreen, voice commands, gestures, and tapping on a remote control.

[0102] The following explanations address different interaction methods. Taking the first display area as the entire screen area and the second display area as a smaller area used to display thumbnails as an example, the second image is displayed in full screen, and the first image is displayed as a thumbnail on top of the second image.

[0103] For example, taking touchscreen interaction as an example, when a user performs a click operation on the screen of a display device, the clicked area can be a first display area, a second display area, or any other part of the screen. After receiving the click event, the device switches the view according to the click location. Specifically, this includes recalculating the new position and size of the image and then updating the user interface to reflect these changes. Specifically, if the user clicks on the first image in the thumbnail area, the display layers of the first and second images are swapped. The size of the first image is enlarged to full-screen display, and the second image is shrunk to a thumbnail overlay. Then, if the user clicks on the second image, the second image is displayed in full-screen mode, and the first image is again displayed as a thumbnail over the first image. In other embodiments, the user can click on any area of ​​the screen, in which case the display layers and display areas of the first and second images are swapped. The specific switching process is the same as described above and will not be repeated here. To prevent accidental screen touches, a "lock current screen" or "lock current layer" function can be provided. When the user enables the "lock current screen" or "lock current layer" function, the current screen or current layer is locked. In this case, clicking the screen will not trigger the view switching command. Only after the user cancels the "lock current screen" or "lock current layer" function can the view switching command be triggered normally by clicking the screen.

[0104] Taking voice interaction as an example, if a user utters a predefined command such as "switch view" or "switch image," the device's voice recognition module captures the user's voice command, parses it, and identifies the user's intent. If the user's intent is "switch view," the view switching operation is executed: the first image is enlarged to full screen, and the second image is shrunk to a thumbnail and overlaid on top of it. Then, if the user says "switch view" again, clicking the second image will display it in full screen, and the first image will again be displayed as a thumbnail on top of it. The method of overlaying the first and second images on the display interface is not limited.

[0105] Taking gesture interaction as an example, if a user makes a specific gesture (such as swiping, waving, etc.) towards the sensor devices around the screen with their finger or hand, or makes a specific gesture on the screen to request a view switch, the sensor captures the gesture, identifies the gesture type and its direction, and switches the view based on the recognition result. The specific view switching process is as described in the above embodiment and will not be repeated here.

[0106] Taking interaction via a directional device as an example, the user can use a remote control or other directional device to select an element on the screen (such as an icon or button) and confirm the selection by pressing a button. The directional device sends a wireless signal to the display device, notifying it to perform the corresponding view switching operation. After receiving the signal from the directional device, the device determines the location of the cursor and the target object selected by the user, and performs the corresponding view switching operation based on the target object selected by the user. Similarly, the specific view switching process is as described in the above embodiment, and will not be repeated here.

[0107] In this embodiment, by providing users with multiple interaction methods, users with different preferences can find the most suitable operation mode for themselves, thus improving the interactive experience.

[0108] In some exemplary embodiments, the aspect ratio and resolution of the image displayed in the first display area are matched with the aspect ratio and resolution of the first display area, and the aspect ratio and resolution of the image displayed in the second display area are matched with the aspect ratio and resolution of the second display area.

[0109] Aspect ratio refers to the ratio of width to height. For example, a screen with a resolution of 1920x1080 pixels has an aspect ratio of 16:9. Resolution refers to the total number of pixels contained in an image or display area, usually expressed as the number of pixels in width multiplied by the number of pixels in height (e.g., 1920x1080).

[0110] In practical applications, to provide a better visual experience, the aspect ratio and resolution of the image displayed in the first display area can be set to match the aspect ratio and resolution of the first display area, and the aspect ratio and resolution of the image displayed in the second display area can be set to match the aspect ratio and resolution of the second display area. In this way, the images displayed in the first and second display areas can be displayed in the correct proportions, reducing image stretching, compression or blurring caused by proportion mismatch, and improving the visual effect.

[0111] In this embodiment, the aspect ratio and resolution of the image displayed in the first display area can be set to be consistent with those of the first display area, and the aspect ratio and resolution of the image displayed in the second display area can be set to be consistent with those of the second display area. Specifically, if the aspect ratio of the image is the same as that of the display area, then the image will not be stretched or compressed when displayed. For example, if both the image and the display area are in a 16:9 ratio, the image can perfectly adapt to the display area according to its original ratio. If the resolution of the image matches the resolution of the display area, then each pixel in the image can correspond to a pixel in the display area, thereby avoiding blurring or distortion caused by scaling up or down. For example, when an image with a resolution of 1920x1080 pixels is displayed on a screen with a resolution of 1920x1080 pixels, each image pixel is directly mapped to a physical pixel on the screen, thus maximizing the preservation of image details and reducing blurring caused by scaling up or down.

[0112] The relative positions of the first image and the second image are not limited. In some embodiments, the first display area is the entire screen area, the image in the first display area is displayed in the center, and the second display area is located to the left or right of the first display area.

[0113] Assuming a screen resolution of 1920x1080 pixels, the first display area can be set to occupy the entire screen (1920x1080), ensuring its aspect ratio matches the screen (16:9). The second display area is set to a small rectangle, such as 300x200 pixels, placed to the left or right of the first display area. Then, the generated second image is loaded, its size adjusted to fit the first display area (1920x1080), and centered on the entire screen. The first image is then loaded, its size adjusted to fit the second display area (300x200), and aligned to the left or right edge of the first display area. It can be understood that the image displayed in the display area matches the aspect ratio and resolution of the display area, thus providing a better visual experience. See Figure 15, which shows the effect of the second display area being located to the left of the first display area.

[0114] In this embodiment, by setting the first display area to the entire screen area and setting the second display area to the left or right of the first display area, the main content that the user wants to view can be highlighted without hindering the effective transmission of auxiliary information, thus making full use of the screen space.

[0115] To avoid obscuring the information of the image displayed in the first display area, the transparency of the image displayed in the second display area can be set to 50%, that is, the image displayed in the second display area can be set to a semi-transparent overlay. It is understood that in other embodiments, the transparency of the image displayed in the second display area can also be 40%, 55%, or other values, depending on the actual situation, and is not limited to a single value, as long as the user can clearly view the image.

[0116] Additionally, the transparency of the image in the second display area can be set by the user. In some embodiments, input to the transparency adjustment control is received and responded to, the current transparency value is obtained, and the transparency of the image displayed in the second display area is updated based on the current transparency value.

[0117] A transparency adjustment control is a control used to dynamically adjust the transparency of an image. This control may include, but is not limited to, a slider or an input box that allows the user to input a specific transparency value. In this embodiment, the transparency adjustment control is used to control the transparency of the image in the second display area. The transparency value is typically a floating-point number between 0 and 1, where 0 represents complete transparency and 1 represents complete opacity.

[0118] For example, an opacity adjustment control, such as a slider, can be added to the user interface, and an event listener can be set for the slider. Whenever the user moves the slider, a corresponding callback function is triggered, the current opacity value is obtained from the slider, and based on the current opacity value, the opacity attribute of the image displayed in the second display area is updated, that is, the alpha channel value of the image is modified, and the image displayed in the second display area is re-rendered.

[0119] In this embodiment, by allowing users to precisely adjust the transparency according to their personal preferences, users can find the display effect that best suits their needs.

[0120] In some exemplary embodiments, the controller is also configured to perform the following steps: in response to a scaling operation of an image displayed in a first display area or a second display area, scaling the image displayed in the first display area or the second display area, and displaying the scaled image.

[0121] In this embodiment, the device also supports independent scaling of images in different display areas. Specifically, scaling gesture listeners can be added to the first and second display areas respectively. When a user's scaling gesture for a certain display area, such as two-finger scaling, two-finger pinch, or double-tap to zoom, is detected, the size of the corresponding image is adjusted according to the scaling ratio, the image is re-rendered according to the updated size, and its position on the screen is adjusted to ensure visual continuity. The scaling ratios corresponding to the scaling gestures, such as two-finger scaling, two-finger pinch, and double-tap to zoom, can be set by the user and are not limited to a single value.

[0122] In this embodiment, by providing independent scaling in different display areas, users can select the content they want to focus on based on their interests, allowing them to freely explore image details and greatly enhancing their sense of participation and satisfaction.

[0123] There are multiple ways to generate the first image. In some exemplary embodiments, prior to S200, the controller is further configured to perform the following steps: receive and respond to a drawing instruction, initialize the canvas, and acquire drawing trajectory data in the canvas. Based on the drawing trajectory data, generate the first image.

[0124] Drawing instructions are used to instruct the display device to launch the drawing board and initialize the canvas, displaying the drawing interface. Users can then begin drawing on the canvas within the drawing interface. Drawing trajectory data, also known as doodling trajectory data, is a dataset recording the position and movement path of each touch point on the canvas, or the position and movement trajectory of coordinate points on the canvas. In this embodiment, the first image is the initial image directly converted from the drawing trajectory data in the canvas.

[0125] In practical implementation, taking a smart display device as an example, users can issue drawing commands via touchscreen, voice interaction, or gestures. The display device then initializes the canvas based on screen resolution or preset information, including determining the canvas size and preparing space for user input. Subsequently, users can draw on the canvas using various data input methods. The display device captures the drawing trajectory data in real time. After the user finishes drawing, or upon receiving a save command from the user, the captured drawing trajectory data is converted into a visual first image (i.e., a doodle or sketch). Specifically, data input methods include, but are not limited to, touch input, and can also be input using contactless input tools such as remote controls. It is understood that the first image, besides being obtained based on the user's drawing trajectory data, can also be an image generated from a large model based on the user's input creative needs and corresponding prompts, or it can be an image uploaded by the user or a specified image.

[0126] In this embodiment, by capturing drawing trajectory data in the canvas and generating a first image, the user's freely drawn creative data can be transformed into an image, supporting the user's free creation.

[0127] In some exemplary embodiments, the controller is also configured to perform the following steps: receiving and responding to a drawing instruction, obtaining the screen display ratio, determining the canvas size based on the screen display ratio, and initializing the canvas based on the canvas size.

[0128] The screen aspect ratio refers to the aspect ratio of a device's screen, such as 16:9 or 4:3. The canvas size is the actual drawing area determined based on the screen aspect ratio.

[0129] In practice, the method of entering the drawing interface is not limited. It can be that after receiving and responding to a drawing command, launching a drawing application (or doodling application), and jumping to the drawing interface, the display device can obtain the screen's display width (screen_width) and display height (screen_height) through interfaces provided by the platform, such as DisplayMetrics, and then calculate the screen's aspect ratio (screen_ratio = screen_width / screen_height). Subsequently, based on the calculated aspect ratio, the display area of ​​the canvas is set, that is, the canvas is set according to the calculated aspect ratio. For example, the canvas ratio can be kept consistent with the screen ratio, or it can be set proportionally; there is no limitation here. After determining the canvas size, a blank canvas can be initialized in the drawing interface using the calculated size for the user to draw on, so that the image ratio remains consistent regardless of whether the device is displayed in landscape or portrait mode. See Figure 16 for the landscape and portrait display effects. Specifically, the drawing interface is an interface where users can write and erase their handwriting using an input device within a specific area.

[0130] In this embodiment, the canvas size is determined by the screen display ratio, which ensures that the method maintains a consistent image ratio regardless of the device on which it runs. No matter the screen size, whether it is landscape or portrait, the image will not be distorted, and the user can get a better visual experience.

[0131] As described in the above embodiments, there are various means for users to input data. In some exemplary embodiments, the controller is also configured to perform the following steps: receiving and responding to a wireless signal returned by the directional device, determining the movement trajectory of the directional device based on the wireless signal, and determining drawing trajectory data in the canvas based on the movement trajectory of the directional device.

[0132] Directional devices refer to devices such as remote controls or other devices that can send wireless signals to control a cursor.

[0133] In this embodiment, a display device (hereinafter referred to as the device) is used as an example to illustrate a scenario where a user interacts with the display device via a directional remote control (hereinafter referred to as the remote control) to achieve remote wireless painting or remote wireless drawing. The display device is typically equipped with a directional remote control (hereinafter referred to as the remote control) capable of wireless communication with it. The user interacts with the display device via the remote control, selecting relevant options and issuing control commands, enabling the display device to execute corresponding control functions based on the remote control's control signals. Based on this, the remote control's location information can be obtained by locating it via its wireless signal. Then, the remote control's movement trajectory can be obtained based on its location information. This trajectory is then converted into drawing trajectory data on the canvas, enabling remote wireless painting on the display device. Subsequently, the drawing trajectory data on the canvas is further used to generate a high-quality first image, further improving the interactive performance of the display device.

[0134] Specifically, users can issue a drawing command by moving the remote control's focus to the drawing start control and pressing a preset button, such as the OK button. The display device obtains the screen ratio, sets the canvas size based on the screen ratio, and displays a blank canvas consistent with the screen ratio for the user to draw on, ensuring the drawing area adapts to the screen size. Next, the user can simulate pen strokes by moving the remote control in the air. The remote control sends wireless signals to the display device, whose built-in sensors receive these signals and interpret them as the remote control's movement trajectory. Based on the received wireless signals, the device analyzes the remote control's direction and speed, calculates the specific movement trajectory, and then transmits the trajectory data to the canvas SDK in real time to generate a continuous drawing path. Based on the remote control's movement trajectory, the device automatically generates corresponding drawing trajectory data, displaying lines (i.e., sketches, or the first image) corresponding to the drawing trajectory on the canvas, as shown in Figure 17. When the user presses the OK button on the remote control, it indicates the completion of a drawing operation, and the drawing data is recorded. Users can also erase or undo the corresponding drawing trajectory, and select brush thickness, fill color, and other functions. After a user completes a sketch, they can choose to save the current sketch. The display device responds to the save command by saving the sketch as a permanent file; that is, the sketch is only saved when the user clicks the save button, thus helping to save system resources.

[0135] In other embodiments, if the display device is a touchscreen device, the user can also draw by touching the screen of the display device. The device directly captures the user's touch actions (including pressing, moving, and lifting), determines the position and movement trajectory of the user's touch point, and then converts it into drawing trajectory data. Alternatively, it can recognize the user's gestures and movement trajectories to determine the drawing trajectory, depending on the specific situation. For determining drawing trajectory data based on touch screen events and gesture recognition, please refer to related technologies, which will not be elaborated here.

[0136] In this embodiment, the user determines the drawing trajectory data through the wireless signal emitted by the directional device, and then generates the first image, which enables remote wireless drawing without being limited by the touch of a stylus or wired devices such as a mouse, greatly improving the user's interactive experience.

[0137] In some exemplary embodiments, the controller is further configured to perform the following steps: preprocessing a first image that meets the input requirements of a trained image generation model, obtaining style cue words, and, based on the style cue words and the preprocessed first image, calling the trained image generation model to generate a second image that matches the style cue words.

[0138] Style cues are textual descriptions of the artistic style or visual characteristics a user desires for a second image. They guide the image generation model to generate a second image that matches the style cues. Style cues can include, but are not limited to, sketching, watercolor, oil painting, comic book, hand-drawn, abstract, science fiction, retro, cyberpunk, cinematic, black and white art, natural landscape, and cartoon styles. The method of obtaining style cues is flexible; they can be input by the user, determined through content understanding of the first image, or multiple styles can be provided for the user to choose from. For example, if the device only has one pre-set image style information, the style cues can be determined directly based on that information. If the display device has multiple pre-set image style information, these can be displayed on the user interface for the user to select one or more to determine the style cues. If the user does not select a specific image style, the first displayed image style is selected by default, or the default style cues, such as comic book style, can be directly set.

[0139] Image generation models (hereinafter referred to as AI models) are algorithmic models trained using deep learning techniques. They are capable of generating new images with a specific artistic style based on a given image and style cues. The type of image generation model is not limited; it can include, but is not limited to, diffusion models or image-generated image models, as long as it can generate new images with a specific artistic style based on a given image and style cues.

[0140] In practical applications, after a user draws a sketch and generates a first image, the display device can prompt the user, or provide an interface, allowing the user to input or select style cue words to generate a second image that matches the style cue words based on the first image. After the user selects a style cue word, the selected style cue word can be parsed into a form that the system can recognize and matched with style tags in the background database. If the user clicks the control for generating the image, the first image is preprocessed, including but not limited to resizing, format conversion, color correction, noise reduction, and contrast enhancement, to meet the specific input requirements of the AI ​​model, enabling the AI ​​model to correctly understand and process the input image. Subsequently, the preprocessed first image and the corresponding style cue words are passed as input to the AI ​​model. Based on internal parameters and trained weights, the AI ​​model transforms the first image into a second image that conforms to the specified style and outputs it. The display device then displays the generated second image on the user interface. The number of second images can be one or more. If there are multiple second images, they are displayed simultaneously, and the user can select the satisfactory image from the displayed second images. It can be understood that the second image generated by the model can also be pre-stored in a buffer.

[0141] In this embodiment, by combining AI technology with user creative inspiration, the user's creation can be transformed into a high-quality image that conforms to a specified style, realizing the transformation from creation to finished product, and meeting the user's personalized needs.

[0142] In some exemplary embodiments, the controller is configured to perform the following steps: receiving and responding to an image saving instruction, storing a first image in a first storage area, storing a second image in a second storage area, and associating the first image and the second image by an identifier of the first image.

[0143] A storage area refers to a logical or physical storage region used to store data. In practical applications, when a user draws a first image, a temporary buffer is allocated for it. If the user decides to save either the first or second image, the first image is persistently stored. Similarly, when the AI ​​model outputs a second image, the device also allocates a temporary buffer for it. If the user decides to save the second image, it is persistently stored. It should be noted that the device stores the first and second images in different storage areas. For ease of distinction, in this embodiment, the two different storage areas are named "first storage area" and "second storage area," respectively. The first storage area is used to store the first image, and the second storage area is used to store the second image.

[0144] In practice, the user interface may display one or more second images for the user to preview. The user selects one second image to save. The device then stores the second image in a preset second storage area, records the file path, and establishes an association between the first and second images based on the identifier of the first image, such as its ID. In subsequent processing, the first image can be retrieved from the first storage area using its ID, and the second image can be retrieved from the second storage area. Finally, the retrieved first and second images are overlaid on the user interface for the user to preview.

[0145] In this embodiment, by storing the first image and the second image in different storage areas and associating them with the identifier of the original first image, users can easily find and re-edit their sketches, and track the drawing creation process of different versions. Users can easily view the image of a specific version and its corresponding generation result, and can even roll back to an earlier version.

[0146] To provide a clearer explanation of the above solution, a specific embodiment, along with Figures 18 and 19, will be described below. This specific embodiment includes the following:

[0147] Users launch the application by tapping the remote control icon and then switch to the interface for drawing. The user's interaction initiates the drawing process, and the device displays the drawing interface. During this process, the application loads the canvas engine, prepares basic drawing tools such as brushes, colors, and erasers, and adjusts the interface layout according to the device type. In the application initialization phase, screen parameters (such as resolution) are obtained, the aspect ratio is calculated, and the canvas size is dynamically set based on the calculated aspect ratio and screen orientation (landscape or portrait), thus initializing the canvas. Throughout this process, screen rotation events are monitored, and the canvas and image sizes are recalculated when the screen orientation changes to adapt to both landscape and portrait modes.

[0148] Users draw lines on a canvas by moving a remote control. The device receives the wireless signal from the remote control, determines its position, and simulates the position of the brushstroke based on the remote control's position. It records the starting point, movement trajectory, and ending point coordinates of the brushstroke to obtain the drawing trajectory, which is then displayed on the canvas as a sketch (first image). During the sketching process, the sketch is saved in real-time as a temporary file in a buffer. If the user completes the sketch and chooses to save it, the device responds to the save command and saves the sketch in the first storage area for later modification or editing. If the user chooses to generate an AI image, the AI ​​image generation process is triggered. Based on the input requirements of a pre-trained image generation model, the sketch is processed into a format suitable for the model's input. Subsequently, based on pre-set style prompts and the pre-processed sketch, the trained image generation model is invoked to generate multiple stylized images (second images) matching the style prompts. The system caches multiple stylized images with different styles and displays them on the user interface for the user to choose from. The image generation model can be deployed on the display device side or on the server side, depending on the actual situation.

[0149] If a user selects a stylized image and chooses to save it, the device responds to the save command by storing the stylized image in a second storage area and associating the sketch and the stylized image based on the sketch's ID. The stylized image is then displayed in full screen, with the sketch appearing as a thumbnail overlay on top and to the left of the stylized image. The sketch's transparency is set to 50% by default. If the user taps the sketch, it is displayed in full screen, with the stylized image appearing as a semi-transparent thumbnail overlay. If the user taps the stylized image again, it returns to full screen display, with the sketch appearing as a thumbnail overlay. While viewing the image, the user can adjust the thumbnail's transparency by dragging a slider, independently zoom in and out of the sketch or stylized image, and lock the current screen or layer to prevent accidental touches. Furthermore, if the user wants to edit the sketch again, they can long-press the sketch or use other interactive methods to enter edit mode, allowing them to modify the sketch. After finalizing the sketch and stylized image, users can also share the sketch or stylized image to other devices such as smartphones.

[0150] After users complete a series of operations such as associating, previewing, and editing the sketch and the stylized image, because the stylized image or hand-drawn sketch itself has rich texture and composition, users usually need to zoom in on local areas for detailed examination or secondary creation.

[0151] However, currently, image preview functions on smart display devices typically employ a "thumbnail + full-screen view" mode. When a user selects an image to enter full-screen preview, the graphics rendering engine determines an initial cropping area based on the screen resolution and a preset scaling strategy. This area defines a sub-rectangle in the original image that will be mapped to the screen's physical pixel space. If the image resolution is higher than the screen resolution, the system displays the image at 100% scale or in a way that adapts to the screen's width / height. At this point, if the user wants to view other areas of the image, they must rely on the remote's directional keys or touchpad to perform line-by-line, pixel-by-pixel, or fixed-step translation operations. This process relies on linear input, and each input only causes a localized, minute viewport movement. For ultra-widescreen or ultra-high-resolution images, multiple inputs are required, resulting in a lengthy and inefficient operation path that negatively impacts the user's interactive experience.

[0152] To address the aforementioned interaction bottlenecks and improve operational efficiency and intuitiveness in large image browsing scenarios, some embodiments of this application also provide an image partial magnification preview scheme that meets the needs of efficient interaction. Specifically, in the image magnification preview interface, in addition to the main display area, a thumbnail representing the complete original image is simultaneously presented, with a "position marker box" superimposed on it. This box precisely maps the cropped area of ​​the current main screen in the original image. When the user clicks on any target location on the thumbnail via touch or remote control pointing, the system immediately analyzes the position of the interaction point in the thumbnail coordinate system, calculates its offset from the current position marker box, and derives the corresponding new cropped area in the original image accordingly. Subsequently, the main display area seamlessly switches to the magnified view of this new area, while the marker box on the thumbnail moves synchronously to the target position. Through this mechanism, the originally cumbersome "directional key fine-tuning" is transformed into a direct point selection operation.

[0153] Specifically, in some exemplary embodiments, the controller is also configured to perform the following steps:

[0154] In response to a preview zoom-in operation on an image displayed in a first display area or a second display area, the image is identified as a preview image, and the display is controlled to display the preview image's display interface. The display interface includes a first zoomed-in view of the preview image and a thumbnail display area. The thumbnail display area includes a position marker box, which is used to indicate the position of the first zoomed-in view in the preview image.

[0155] In response to interactive operations on the target location in the thumbnail display area, the display position marker box is moved to the target location, and the display is shown as a second magnified view in the display interface.

[0156] In particular, the position of the second magnified image in the preview image matches the position of the moved position marker box in the thumbnail display area.

[0157] As an example, the interactive operation at the target location can be a touch screen operation or a remote control pointing operation at the touch screen. The response can be made by listening to the touch event corresponding to the interactive operation. The touch event can include at least one of the touch screen event or the remote control pointing event.

[0158] The target location can be within the thumbnail display area but outside the display area of ​​the location marker box. The thumbnails displayed in the thumbnail display area are the thumbnails corresponding to the preview image, and the thumbnail display area floats above the first zoomed-in view.

[0159] The first magnified image and the second magnified image can be partial magnified images of preview images that have the same display size but different display content.

[0160] Among them, the location marker box is an auxiliary element used by the display device in the "partial magnification preview of image" scenario to visually associate the "thumbnail" and the "magnified image". By marking the specific range of the current magnified image in the original image in real time in the thumbnail, it helps users quickly understand "which position in the original image the magnified detail currently seen corresponds to".

[0161] For example, in practical applications, if the first display area displays the second image (finished picture book illustration) in full screen, and the second display area displays the first image (hand-drawn sketch) in a small window overlaid, if the user needs to view the details of a certain image, they can trigger a preview magnification operation for the image in the first or second display area by pointing to the remote control and pressing and holding the confirmation button, pressing and holding the image on the touch screen, or clicking the preset magnification control on the interface. After receiving the operation command, the display device first determines the triggered image as the preview image, and then starts the preview magnification process for the preview image, controlling the display to show the preview image display interface (as shown in Figure 20). At this time, the display interface includes the first magnified screen of the preview image and the thumbnail display area. Specifically, the size of the magnified preview image is the size of image 1 in Figure 20, which is larger than the display size of the display interface. The display interface is the preview display area of ​​image 2 in Figure 20, which displays the first magnified image and the thumbnail display area (thumbnail 3 in Figure 20). The position marker box in Figure 20 is used to indicate the position of the first magnified image currently displayed on the display interface in the magnified preview image. The position marker box can also be used to describe the proportion of the preview display area of ​​image 2 in image 1.

[0162] In one example, based on the thumbnail display area in the display interface, in response to interactive operations on a target location in the thumbnail display area, such as detecting a touch screen operation or a remote control pointing at the touch screen, a location marker box is displayed in the display interface and moves to the target location, and a corresponding magnified image (i.e., a second magnified image) is displayed in the display interface according to the moved location marker box.

[0163] For example, as shown in Figure 20, in a zoomed-in preview image scenario, based on the displayed 3 thumbnails, the user can move the position marker box in the 3 thumbnail display area by pointing to the touch area. For instance, moving the position marker box to the touch location allows the user to adjust the corresponding zoomed-in content in the preview image based on the moved position of the position marker box in the 3 thumbnails. This enables quick movement of the corresponding display position in the zoomed-in preview image without requiring the user to manually adjust the image's display content.

[0164] The technical solution of this embodiment, by responding to interactive operations on the target position in the thumbnail display area, controls the display position marker box to move to the target position and controls the display to present a second magnified image in the display interface, realizes the adjustment and optimization of the display content in the magnified preview image scenario. It can efficiently and quickly move the display position corresponding to the magnified image in the preview image without the need for manual button operation by the user, effectively improving the efficiency of adjusting the image display content and helping to improve the image preview display effect.

[0165] In some embodiments, in response to an interactive operation on a target location in the thumbnail display area, the controller 250 is configured to control the display to move a location marker box toward the target location and to control the display to present a second magnified view in the display interface, wherein the controller 250 is configured to perform the following steps:

[0166] In the image preview zoom-in mode, listen for touch events corresponding to interactive operations; touch events include at least one of touch display screen events or remote control pointing to touch screen events; in response to the listened touch events, control the display to move the display position indicator box to the target position, and control the display to present a second zoomed-in image in the display interface.

[0167] In a specific implementation, as shown in Figure 21, by triggering image magnification during image preview, a magnified view of the image and an image thumbnail can be displayed. Then, based on the thumbnail, a touch event can be waited for. For example, when the thumbnail is displayed, a pointer listener can be registered with the display platform to monitor the pointed touch event. That is, in the image preview magnification mode, the touch event corresponding to the interactive operation can be listened for. Then, by responding to the listened touch event, the corresponding position of the magnified view of the preview image can be shifted based on the pointed target position.

[0168] In an optional embodiment, the interaction relationship between the pointing touch event modules, as shown in Figure 22, is implemented in the following hierarchical manner:

[0169] 1. Pointing to remote control: By providing a touch event callback interface to the business layer, it supports "press event", "move event" and "lift event", and can provide pointing touch screen events. Then the business layer can realize location confirmation or trigger confirmation events based on the relevant events listened to.

[0170] 2. Adaptation Layer: It can obtain the pointing touch screen trajectory based on the pointing remote control callback event and perform data management;

[0171] 3. Application layer: It can calculate the display position based on the determined pointing position, image size and screen display size, and thus realize the effects of image scaling and position movement.

[0172] In this embodiment, by listening to touch events corresponding to interactive operations in the image preview zoom-in mode, and then responding to the listened touch events, controlling the display to move the position marker box to the target position, and controlling the display to present a second zoomed-in image in the display interface, accurate feedback for user interactive operations can be achieved in the image preview zoom-in mode.

[0173] In some embodiments, the controller 250 is configured to perform the following steps to control the display to move the position indicator box to the target position and to control the display to present a second magnified image in the display interface:

[0174] Obtain the touch coordinates of the interactive operation in the thumbnail display area, and based on the positional relationship between the touch coordinates and the position marker box, control the display to move the position marker box to the target position, and control the display to present a second magnified image in the display interface.

[0175] Specifically, upon detecting a touch event (i.e., a touch control event), the position of the target location within the current thumbnail can be obtained, such as by acquiring the location marker (raw). x raw y This means obtaining the touch coordinates of the interactive operation in the thumbnail display area.

[0176] In one example, as shown in Figure 21, after obtaining the location marker based on the pointing touch event, the touch range can be further determined based on the location marker, such as whether it is within the visible small image range corresponding to the location marker box. That is, based on the positional relationship between the touch coordinates and the location marker box, the corresponding position of the magnified preview image is shifted.

[0177] In this embodiment, by obtaining the touch coordinates of the interactive operation in the thumbnail display area, and then controlling the display to move the position marker box to the target position based on the positional relationship between the touch coordinates and the position marker box, and controlling the display to present a second magnified image in the display interface, it is possible to dynamically adjust the position marker box to move based on the positional relationship between the user's touch position and the position marker box, and synchronously display the magnified image after the displacement, which helps to optimize the user's operating experience.

[0178] In some embodiments, the controller 250 is configured to perform the following steps.

[0179] Determine the thumbnail display parameters; the thumbnail display parameters include the area size parameters of the thumbnail display area; based on the coordinate system established by the area size parameters, obtain the touch coordinates of the interactive operation in the thumbnail display area.

[0180] For example, in the image preview zoom-in mode, the preview image size is 1920x1080, and after zooming in by 1 time, the image size is 3840x2160; regarding the image display area, the maximum displayable area on the screen can be obtained based on the screen size, such as:

[0181] Previewwidth (image display width) = screenwidth (screen display width);

[0182] Previewheight (image display height) = screenheight (screen display height).

[0183] For the thumbnail display parameters corresponding to the thumbnail display area, these can include the area size parameters of the thumbnail display area, such as defining the thumbnail display size as `Thumbnailwidth` (thumbnail display width) and `Thumbnailheight` (thumbnail display height). Then, based on the coordinate system established by the area size parameters, the touch coordinates of the interactive operation within the thumbnail display area can be obtained, for example, the location marker of the target position (raw). x raw y The corresponding coordinate system is the display range of the thumbnail from (0, 0) to (Thumbnailwidth, Thumbnailheight).

[0184] Optionally, the thumbnail display dimensions Thumbnailwidth and Thumbnailheight can be adjusted according to the display effect. For example, Thumbnailwidth = 480 and Thumbnailheight = 270 can be defined.

[0185] In this embodiment, by determining the thumbnail display parameters and then establishing a coordinate system based on the area size parameters, the touch coordinates of the interactive operation in the thumbnail display area can be obtained. This enables the accurate acquisition of touch coordinates pointing to the target location based on the touch event waiting for the thumbnail display.

[0186] In some embodiments, the thumbnail display parameters may further include the position parameters of the location identifier box. To determine the thumbnail display parameters, the controller 250 is configured to perform the following steps.

[0187] Based on the magnification factor corresponding to the first magnified image, obtain the magnified image parameters of the preview image; based on the coordinate system established by the magnified image parameters, determine the starting coordinate position of the first magnified image in the preview image; based on the screen display parameters, magnified image parameters, and area size parameters, determine the label size parameters of the position label box; the ratio between the screen display parameters and the magnified image parameters corresponds to the ratio between the label size parameters and the area size parameters; use the starting coordinate position and the label size parameters as the position parameters of the position label box.

[0188] In practical applications, if the preview image size is 1920x1080, after being magnified by 1x (i.e., the magnification factor corresponding to the first magnified image), the image size becomes 3840x2160 (i.e., the magnified image parameter), which can be represented as imagewidth (the magnified image display width) and imageheight (the magnified image display height).

[0189] In one example, the thumbnail display parameters may also include the position parameters of the location marker box, which include the starting coordinates, such as Locationlayoutleft and Locationlayouttop, to represent the starting position of the location marker box in the original image; the position parameters also include the marker size parameters of the location marker box, such as Locationwidth and Locationheight. The position parameters of the location marker box can change as the zoomed-in display position in the preview image changes.

[0190] In another example, the ratios of the parameters in the image preview zoom-in mode are as follows (i.e., the ratio between the screen display parameters and the zoomed-in image parameters corresponds to the ratio between the identifier size parameters and the area size parameters):

[0191] Previewwidth / imagewidth=Locationwidth / Thumbnailwidth;

[0192] Previewheight / imageheight=Locationheight / Thumbnailheight.

[0193] In this embodiment, the magnified image parameters of the preview image are obtained according to the magnification factor corresponding to the first magnified image. Then, the starting coordinate position of the first magnified image in the preview image is determined based on the coordinate system established by the magnified image parameters. Then, the label size parameters of the position label box are determined according to the screen display parameters of the display, the magnified image parameters, and the area size parameters. The starting coordinate position and the label size parameters are used as the position parameters of the position label box. The position parameters of the position label box can be accurately obtained based on the changes in the display position of the magnified image in the preview image.

[0194] In some embodiments, based on the positional relationship between the touch coordinates and the position marker box, the controller 250 controls the display to move the position marker box to the target position and controls the display to present a second magnified image in the display interface. The controller 250 is configured to perform the following steps.

[0195] Determine whether the touch coordinates are within the display range of the position marker box; if the touch coordinates are not within the display range, control the display to move the position marker box to the target position, and control the display to present a second magnified image on the display interface.

[0196] Specifically, as shown in Figure 21, the touch screen range can be determined based on the position marker obtained from the pointed touch screen event, that is, whether the touch coordinates are within the display range of the position marker box. Then, for cases where the touch coordinates are not within the display range, such as in Figure 21 where they are not within the visible small image range corresponding to the position marker box, the corresponding position of the magnified preview image can be shifted based on the touch coordinates.

[0197] In this embodiment, by determining whether the touch coordinates are within the display range of the position marker box, and then controlling the display to move the position marker box to the target position when the touch coordinates are not within the display range, and controlling the display to present a second magnified image in the display interface, it is possible to quickly determine whether displacement processing is triggered based on the display range of the position marker box, which helps to improve processing efficiency.

[0198] In some embodiments, the controller 250 is also configured to perform the following steps.

[0199] The display range is obtained by determining the display left, display top, display right, and display bottom boundary parameters of the position marker box based on the position size parameter of the position marker box and the starting coordinate position of the first magnified image in the preview image.

[0200] In one example, the display area of ​​the location marker box can be represented as follows:

[0201] Left boundary (i.e., the left display boundary parameter): Locationlayoutleft;

[0202] Upper boundary (i.e., the upper boundary parameter for display): Locationlayouttop;

[0203] Right boundary (i.e., the right boundary parameter for display): Locationlayoutleft + Locationwidth;

[0204] The lower boundary (i.e., the display lower boundary parameter) is Locationlayouttop+Locationheight.

[0205] In this embodiment, by determining the display left boundary parameter, display upper boundary parameter, display right boundary parameter, and display lower boundary parameter of the position marker frame based on the position parameter of the position marker frame and the starting coordinate position of the first magnified image in the preview image, the display range is obtained. This allows for accurate determination of the display range of the position marker frame and provides data support for judging whether the touch coordinates are within the display range.

[0206] In some embodiments, the controller 250 is also configured to perform the following steps.

[0207] If the horizontal coordinate of the touch coordinates is greater than or equal to the left boundary parameter of the display and less than or equal to the right boundary parameter of the display; and the vertical coordinate of the touch coordinates is greater than or equal to the upper boundary parameter of the display and less than or equal to the lower boundary parameter of the display, then the touch coordinates are determined to be within the display range.

[0208] For example, to determine whether the touch coordinates are within the display area of ​​the location marker box, the following method can be used:

[0209] If(raw x >=Locationlayoutleft&&raw x <=(Locationlayoutleft+Locationwidth))

[0210] and,

[0211] If(raw y >=Locationlayouttop&&raw y <=(Locationlayouttop+Locationheight)).

[0212] The above judgment formula can confirm that the touch coordinates are within the display range of the position indicator box, and no position movement is performed in this state.

[0213] In this embodiment, if the horizontal coordinate of the touch coordinate is greater than or equal to the display left boundary parameter and less than or equal to the display right boundary parameter, and the vertical coordinate of the touch coordinate is greater than or equal to the display upper boundary parameter and less than or equal to the display lower boundary parameter, it is determined that the touch coordinate is within the display range. This can effectively and quickly determine whether the touch coordinate is within the display range, so as to determine whether to perform position movement processing.

[0214] In some embodiments, when the touch coordinates are not within the display range, the controller 250 is configured to move the display position indicator box to the target position and to display a second magnified image in the display interface.

[0215] By combining touch coordinates, left boundary parameters, top boundary parameters, right boundary parameters, and bottom boundary parameters, position display deviation data is calculated. Using the position display deviation data, the display position of the first magnified image in the preview image is adjusted, and the display is controlled to present the second magnified image. Using the position display deviation data, the display position marker box is moved to the target position.

[0216] In the specific implementation, as shown in Figure 21, when it is determined that the touch coordinates are not within the visible small image range corresponding to the position marker box, position mapping processing can be used to calculate the position display deviation data based on the touch coordinates, the left display boundary parameter, the upper display boundary parameter, the right display boundary parameter, and the lower display boundary parameter. This can then trigger the display displacement of the magnified image in the preview image. That is, the position display deviation data is used to adjust the display position of the first magnified image in the preview image to present the second magnified image. At the same time, the position display deviation data is used to move the display position marker box to the target position.

[0217] For example, as shown in Figure 23, if the touch coordinates corresponding to the target position of the interactive operation are not within the display range of the position indicator box, the position indicator box can be moved to the left of the display area in the thumbnail display area (i.e., the position indicator box is moved to the target position) based on the calculated position display deviation data, and a magnified view after the preview image displacement processing (i.e., the second magnified view) is presented.

[0218] For example, as shown in Figure 24, if the touch coordinates corresponding to the target position of the interactive operation are not within the display range of the position indicator box, the position indicator box can be moved to the upper display area in the thumbnail display area based on the calculated position display deviation data (i.e., the position indicator box is moved to the target position), and a magnified view after the preview image displacement processing (i.e., the second magnified view) is presented.

[0219] In this embodiment, position display deviation data is calculated by combining touch coordinates, display left boundary parameter, display upper boundary parameter, display right boundary parameter and display lower boundary parameter, then the position display deviation data is used to adjust the display position of the first enlarged image in the preview picture, and the display is controlled to present the second enlarged image; the position display deviation data is used to control the display position identification frame to move towards the target position, which can realize efficient and fast movement of the display position corresponding to the enlarged image in the preview picture and present the enlarged image after displacement.

[0220] In some embodiments, position display deviation data is calculated by combining touch coordinates, display left boundary parameter, display upper boundary parameter, display right boundary parameter and display lower boundary parameter, and the controller 250 is configured to perform the following steps.

[0221] Obtain a difference between the display left boundary parameter and the abscissa in the touch coordinates, or a difference between the display right boundary parameter and the abscissa, as the abscissa deviation; obtain a difference between the display upper boundary parameter and the ordinate in the touch coordinates, or a difference between the display lower boundary parameter and the ordinate, as the ordinate deviation; take the abscissa deviation and the ordinate deviation as the position display deviation data.

[0222] In an example, the thumbnail display area can be divided into four regions: left, right, upper and lower for judgment, and the position display deviation data can be calculated in the following manner (e.g., abscissa deviation Offset x and ordinate deviation Offset y ):

[0223] If(raw x <Locationlayoutleft){ / / indicates in the X-axis direction, the displayed image moves to the right

[0224] Offset x =Locationlayoutleft-raw x}

[0225] If(raw x >Locationlayoutright){ / / indicates in the X-axis direction, the displayed image moves to the left

[0226] Offset x =Locationlayoutright-raw x}

[0227] Otherwise, Offset x = 0 and remains unchanged.

[0228] If(rawy <Locationlayouttop){ / / represents the Y-axis direction, the displayed image moves downward

[0229] Offset y =Locationlayouttop-raw y}

[0230] If(raw y >Locationlayoutbottom){ / / represents the Y-axis direction, the displayed image moves upward

[0231] Offset y =Locationlayoutbottom-raw y}

[0232] Otherwise, Offset y =0 change.

[0233] In this embodiment, by obtaining the difference between the display left boundary parameter and the abscissa in the touch coordinates, or the difference between the display right boundary parameter and the abscissa, as the abscissa deviation, and obtaining the difference between the display upper boundary parameter and the ordinate in the touch coordinates, or the difference between the display lower boundary parameter and the ordinate, as the ordinate deviation, and then taking the abscissa deviation and the ordinate deviation as the position display deviation data, the position display deviation data can be accurately calculated, which provides data support for further position movement processing.

[0234] In some embodiments, the position display deviation data is used to adjust the display position of the first enlarged picture in the preview picture, control the display to present the second enlarged picture, and the controller 250 is configured to perform the following steps.

[0235] Combining the enlarged picture parameters after the preview picture is enlarged, the position display deviation data, and the area size parameter of the thumbnail display area, determine the first display position adjustment parameter for the first enlarged picture; the ratio relationship between the first display position adjustment parameter and the enlarged picture parameter corresponds to the ratio relationship between the position display deviation data and the area size parameter; according to the first display position adjustment parameter, adjust the display position of the first enlarged picture in the preview picture, and control the display to present the second enlarged picture.

[0236] In practical applications, according to the obtained position display deviation data, such as Offset x and Offset y , by converting it into an offset (offset information) of the movement of the display position of the enlarged picture on the preview picture, the offset of the display position of the enlarged picture in the preview picture is realized, so as to present the shifted second enlarged picture.

[0237] For example, it can be handled in the following way:

[0238] Previewlayoutoffset x PreviewLayoutOffset y (i.e., the first display position adjustment parameter);

[0239] The parameters are converted as follows:

[0240] Previewlayoutoffset x / imagewidth=offset x / Thumbnailwidth;

[0241] Previewlayoutoffset x =(offset) x / Thumbnailwidth)*imagewidth;

[0242] Similarly:

[0243] Previewlayoutoffset y / imageheight=offset y / Thumbnailheight;

[0244] Preview layout offset y =(offset) y / Thumbnailheight)*imageheight.

[0245] That is, by combining the magnified image parameters after the preview image is enlarged, the position display deviation data, and the area size parameters of the thumbnail display area, the first display position adjustment parameter is determined. The ratio between the first display position adjustment parameter and the magnified image parameters corresponds to the ratio between the position display deviation data and the area size parameters.

[0246] In this embodiment, by combining the magnified image parameters after the preview image is enlarged, the position display deviation data, and the area size parameters of the thumbnail display area, the first display position adjustment parameters for the first magnified image are determined. Then, according to the first display position adjustment parameters, the display position of the first magnified image in the preview image is adjusted, and the display is controlled to present the second magnified image. This can achieve the offset of the display position of the magnified image in the preview image, thereby improving the display effect in the magnified preview image scenario.

[0247] In some embodiments, position display deviation data is used to control the display to move the position marker box to the target position, and the controller 250 is configured to perform the following steps.

[0248] Based on the position display deviation data and the starting coordinates of the first magnified image in the preview image, determine the second display position adjustment parameters of the position marker box; according to the second display position adjustment parameters, control the display to move the position marker box to the target position.

[0249] For example, offset data (such as offset) can be displayed based on the obtained position. x and Offset y Adjust the display deviation of the position marker frame to move it to the target position. The second display position adjustment parameter can be obtained as follows:

[0250] Locationlayoutleft-offset x ;

[0251] Locationlayouttop-offset y .

[0252] In this embodiment, by determining the second display position adjustment parameter of the position marker box based on the position display deviation data and the starting coordinate position of the first magnified image in the preview image, and then controlling the display to move the position marker box to the target position according to the second display position adjustment parameter, the display offset of the position marker box can be realized, thereby improving the display effect in the magnified preview image scenario.

[0253] In some exemplary embodiments, this application also provides an image processing method applied to a display device as described in any of the above embodiments, the method comprising:

[0254] Receive and respond to an image save command, associate and store the first image and the second image together, wherein the first image and the second image have an image-to-image relationship;

[0255] The first image and the second image are overlaid on the user interface, wherein the second image is displayed in the first display area of ​​the user interface, and the first image is displayed in the second display area of ​​the first display area. The second display area is smaller than the first display area, and the layer of the image displayed in the second display area is above the layer of the image displayed in the first display area.

[0256] Receive and respond to view switching instructions to switch the image displayed in the first display area to the second display area, and switch the image displayed in the second display area to the first display area.

[0257] In some exemplary embodiments, the method further includes:

[0258] In response to a preview zoom-in operation on an image displayed in a first display area or a second display area, the image is determined as a preview image, and the display is controlled to display the preview image display interface. The display interface includes a first zoomed-in view of the preview image and a thumbnail display area. The thumbnail display area includes a position marker box, which is used to indicate the position of the first zoomed-in view in the preview image.

[0259] In response to interactive operations on the target location in the thumbnail display area, control the display to move the location marker box to the target location, and control the display to present a second magnified view of the preview image in the display interface;

[0260] In particular, the position of the second magnified image in the preview image matches the position of the moved position marker box in the thumbnail display area.

[0261] In some exemplary embodiments, the method further includes:

[0262] Receive and respond to input operations from the transparency adjustment control to obtain the current transparency value;

[0263] Update the transparency of the image displayed in the second display area based on the current transparency value.

[0264] In some exemplary embodiments, the method further includes:

[0265] In response to a scaling operation on an image displayed in a first display area or a second display area, the image displayed in the first display area or the second display area is scaled, and the scaled image is displayed.

[0266] In some exemplary embodiments, before receiving and responding to an image saving instruction, the method further includes: receiving and responding to a drawing instruction to initialize a canvas; acquiring drawing trajectory data in the canvas; and generating a first image based on the drawing trajectory data.

[0267] In some exemplary embodiments, the method further includes: obtaining the screen display ratio, determining the canvas size based on the screen display ratio, and initializing the canvas based on the canvas size.

[0268] In some exemplary embodiments, acquiring drawing trajectory data in a canvas includes: receiving and responding to a wireless signal returned by a directional device; determining the movement trajectory of the directional device based on the wireless signal; and generating drawing trajectory data based on the movement trajectory of the directional device.

[0269] In some exemplary embodiments, after generating a first image based on the drawing trajectory data, the method further includes: preprocessing the first image so that the preprocessed first image meets the input requirements of the trained image generation model; obtaining style cue words; and, based on the style cue words and the preprocessed first image, calling the trained image generation model to generate a second image that matches the style cue words.

[0270] In some exemplary embodiments, storing a first image and a second image together includes: storing the first image and the second image together by means of an identifier of the first image, wherein the second image is generated based on the first image, and the first image and the second image are stored in different storage areas.

[0271] In some exemplary embodiments, in response to an interactive operation on a target location in the thumbnail display area, controlling the display to move a location marker box toward the target location and controlling the display to present a second magnified image in the display interface includes: in the image preview magnification mode, listening for touch events corresponding to the interactive operation; the touch events include at least one of a touch screen event or a remote control pointing event; in response to the listened touch events, controlling the display to move a location marker box toward the target location and controlling the display to present a second magnified image in the display interface.

[0272] In some exemplary embodiments, controlling the display to move the position indicator box to the target position and controlling the display to present a second magnified image on the display interface includes:

[0273] Obtain the touch coordinates of the interactive operation in the thumbnail display area; based on the positional relationship between the touch coordinates and the position marker box, control the display to move the position marker box to the target position, and control the display to present a second magnified image in the display interface.

[0274] In some exemplary embodiments, obtaining the touch coordinates of the interactive operation in the thumbnail display area includes:

[0275] Determine the thumbnail display parameters; these parameters include the area size of the thumbnail display region.

[0276] Based on the coordinate system established by the area size parameters, the touch coordinates of the interactive operation in the thumbnail display area are obtained.

[0277] In some exemplary embodiments, the thumbnail display parameters further include the position parameter of the location identifier box. Determining the thumbnail display parameters includes:

[0278] Based on the magnification factor corresponding to the first magnified image, obtain the magnified image parameters of the preview image; based on the coordinate system established by the magnified image parameters, determine the starting coordinate position of the first magnified image in the preview image; based on the screen display parameters, magnified image parameters, and area size parameters, determine the label size parameters of the position label box; the ratio between the screen display parameters and the magnified image parameters corresponds to the ratio between the label size parameters and the area size parameters; use the starting coordinate position and the label size parameters as the position parameters of the position label box.

[0279] In some exemplary embodiments, based on the positional relationship between the touch coordinates and the position marker frame, the display is controlled to move the position marker frame to the target position, and the display is controlled to present a second magnified image on the display interface, including:

[0280] Determine whether the touch coordinates are within the display range of the position marker box; if the touch coordinates are not within the display range, control the display to move the position marker box to the target position, and control the display to present a second magnified image on the display interface.

[0281] In some exemplary embodiments, the method further includes:

[0282] The display range is obtained by determining the display left, display top, display right, and display bottom boundary parameters of the position marker box based on the position size parameter of the position marker box and the starting coordinate position of the first magnified image in the preview image.

[0283] In some exemplary embodiments, the method further includes:

[0284] If the horizontal coordinate of the touch coordinates is greater than or equal to the left boundary parameter of the display and less than or equal to the right boundary parameter of the display; and the vertical coordinate of the touch coordinates is greater than or equal to the upper boundary parameter of the display and less than or equal to the lower boundary parameter of the display, then the touch coordinates are determined to be within the display range.

[0285] In some exemplary embodiments, when the touch coordinates are not within the display range, controlling the display to move the position marker box to the target position and controlling the display to present a second magnified image in the display interface includes: calculating position display deviation data by combining the touch coordinates, display left boundary parameters, display upper boundary parameters, display right boundary parameters, and display lower boundary parameters; using the position display deviation data, adjusting the display position of the first magnified image in the preview image, and controlling the display to present the second magnified image; and using the position display deviation data, controlling the display to move the position marker box to the target position.

[0286] In some exemplary embodiments, the position display deviation data is calculated by combining touch coordinates, display left boundary parameters, display upper boundary parameters, display right boundary parameters, and display lower boundary parameters, including:

[0287] Obtain the difference between the left boundary parameter and the horizontal coordinate in the touch coordinate system, or the difference between the right boundary parameter and the horizontal coordinate, as the horizontal coordinate deviation; obtain the difference between the upper boundary parameter and the vertical coordinate in the touch coordinate system, or the difference between the lower boundary parameter and the vertical coordinate, as the vertical coordinate deviation; use the horizontal and vertical coordinate deviations as the position display deviation data.

[0288] In some exemplary embodiments, positional display deviation data is used to adjust the display position of the first magnified image in the preview image, and the display is controlled to present the second magnified image, including:

[0289] By combining the magnified image parameters after the preview image is enlarged, the position display deviation data, and the area size parameters of the thumbnail display area, the first display position adjustment parameters for the first magnified image are determined; the ratio between the first display position adjustment parameters and the magnified image parameters corresponds to the ratio between the position display deviation data and the area size parameters; according to the first display position adjustment parameters, the display position of the first magnified image in the preview image is adjusted, and the monitor is controlled to display the second magnified image.

[0290] In some exemplary embodiments, position display deviation data is used to control the display to move the position marker box towards the target position, including:

[0291] Based on the position display deviation data and the starting coordinates of the first magnified image in the preview image, determine the second display position adjustment parameters of the position marker box; according to the second display position adjustment parameters, control the display to move the position marker box to the target position.

[0292] Specifically, the specific data processing procedures involved in the controller's execution of the steps in the above embodiments can be found in the description of the above embodiments concerning the display device, and will not be repeated here.

[0293] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the methods of the above embodiments.

[0294] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the methods of the above embodiments.

[0295] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the methods of the above embodiments.

[0296] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0297] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0298] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0299] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A display device, wherein, The device includes: The monitor is configured to display the user interface; and at least one controller, configured as follows: Receive and respond to an image save command, and associate and store a first image and a second image, wherein the first image and the second image have an image-to-image relationship; The first image and the second image are superimposed on the user interface, wherein the second image is displayed in a first display area of ​​the user interface, the first image is displayed in a second display area of ​​the first display area, the second display area is smaller than the first display area, and the layer of the image displayed in the second display area is above the layer of the image displayed in the first display area; Upon receiving and responding to a view switching command, the image displayed in the first display area is switched to be displayed in the second display area, and the image displayed in the second display area is switched to be displayed in the first display area.

2. The display device of claim 1, wherein, The controller is also configured to: Receive and respond to input from the transparency control to obtain the current transparency value; Based on the current transparency value, update the transparency of the image displayed in the second display area.

3. The display device of claim 1, wherein, The aspect ratio and resolution of the image displayed in the first display area are matched with the aspect ratio and resolution of the first display area, and the aspect ratio and resolution of the image displayed in the second display area are matched with the aspect ratio and resolution of the second display area.

4. The display device of claim 1, wherein, The controller is also configured to: In response to a preview zoom-in operation on an image displayed in the first display area or the second display area, the image is determined as a preview image, and the display is controlled to display the preview image's display interface. The display interface includes a first zoomed-in view of the preview image and a thumbnail display area. The thumbnail display area includes a position marker box, which is used to characterize the position of the first zoomed-in view in the preview image. In response to an interactive operation on a target location in the thumbnail display area, the display is controlled to move the location marker box toward the target location, and the display is controlled to present a second magnified view of the preview image in the display interface; The position of the second magnified image in the preview image matches the position of the moved position marker box in the thumbnail display area.

5. The display device of claim 4, wherein, The controller, in response to an interactive operation on a target location in the thumbnail display area, controls the display to move the location marker box toward the target location, and controls the display to present a second magnified view of the preview image in the display interface, and is configured to: In the image preview zoom-in mode, listen for touch events corresponding to the interactive operation; the touch events include at least one of the following: touching the screen of the display or pointing the remote control at the touch screen; In response to the detected touch event, the display is controlled to move the location marker box toward the target location, and the display is controlled to present the second magnified image in the first display area.

6. The display device of claim 5, wherein, The controller is configured to control the display to move the location marker box to the target location and to display a second magnified image on the display interface. Obtain the touch coordinates of the interactive operation in the thumbnail display area; Based on the positional relationship between the touch coordinates and the position marker frame, the display is controlled to show the position marker frame moving towards the target position, and the display is controlled to present a second magnified image on the display interface.

7. The display device of claim 6, wherein, The controller is configured to acquire the touch coordinates of the interactive operation in the thumbnail display area. Determine the thumbnail display parameters; the thumbnail display parameters include the area size parameters of the thumbnail display area; Based on the coordinate system established by the area size parameters, the touch coordinates of the interactive operation in the thumbnail display area are obtained.

8. The display device of claim 7, wherein, The thumbnail display parameters also include the position parameters of the location identifier box. The controller executes the determination of the thumbnail display parameters and is configured as follows: Based on the magnification factor corresponding to the first magnified image, obtain the magnified image parameters of the preview image after magnification; Based on the coordinate system established by the magnified image parameters, the starting coordinate position of the first magnified image in the preview image is determined; The identifier size parameter of the location identifier frame is determined based on the screen display parameters of the display, the magnified image parameters, and the area size parameters; the ratio between the screen display parameters and the magnified image parameters corresponds to the ratio between the identifier size parameter and the area size parameter. The starting coordinate position and the identifier size parameter are used as the position parameters of the position identifier box.

9. The display device of claim 6, wherein, The controller, based on the positional relationship between the touch coordinates and the position marker frame, controls the display to move the position marker frame towards the target position, and controls the display to present a second magnified view on the display interface, and is configured to: Determine whether the touch coordinates are within the display range of the position marker frame; If the touch coordinates are not within the display range, control the display to move the position marker box to the target position, and control the display to present a second magnified image on the display interface.

10. The display device of claim 9, wherein, The controller is also configured to: The display range is obtained by determining the display left boundary parameter, display top boundary parameter, display right boundary parameter, and display bottom boundary parameter of the location marker frame based on the marker size parameter of the location marker frame in the location parameters of the location marker frame and the starting coordinate position of the first magnified image in the preview image.

11. The display device of claim 10, wherein, The controller is also configured to: If the horizontal coordinate in the touch coordinates is greater than or equal to the left display boundary parameter, and the horizontal coordinate is less than or equal to the right display boundary parameter; The vertical coordinate in the touch coordinates is greater than or equal to the upper boundary parameter of the display, and the vertical coordinate is less than or equal to the lower boundary parameter of the display; Determine that the touch coordinates are within the display range.

12. The display device of claim 10, wherein, The controller, when the touch coordinates are not within the display range, controls the display to move the position marker box towards the target position, and controls the display to present a second magnified image on the display interface, and is configured to: By combining the touch coordinates, the left boundary parameter of the display, the upper boundary parameter of the display, the right boundary parameter of the display, and the lower boundary parameter of the display, the position display deviation data is calculated; Using the positional display deviation data, the display position of the first magnified image in the preview image is adjusted, and the display is controlled to show the second magnified image. Using the position display deviation data, the display is controlled to move the position marker box toward the target position.

13. The display device of claim 12, wherein, The controller calculates the position display deviation data by combining the touch coordinates, the left display boundary parameter, the upper display boundary parameter, the right display boundary parameter, and the lower display boundary parameter, and is configured as follows: The difference between the left boundary parameter of the display and the horizontal coordinate in the touch coordinates, or the difference between the right boundary parameter of the display and the horizontal coordinate, is obtained as the horizontal coordinate deviation; The difference between the upper boundary parameter of the display and the vertical coordinate in the touch coordinate system, or the difference between the lower boundary parameter of the display and the vertical coordinate, is obtained as the vertical coordinate deviation. The horizontal axis deviation and the vertical axis deviation are used as the position display deviation data.

14. The display device of claim 12, wherein, The controller, using the positional display deviation data, adjusts the display position of the first magnified image in the preview image, and controls the display to present the second magnified image, as configured to: Combining the magnified image parameters of the preview image, the position display deviation data, and the area size parameters of the thumbnail display area, a first display position adjustment parameter is determined for the first magnified image; the ratio between the first display position adjustment parameter and the magnified image parameters corresponds to the ratio between the position display deviation data and the area size parameters. Adjust the display position of the first magnified image in the preview image according to the first display position adjustment parameters, and control the display to present the second magnified image.

15. The display device of claim 12, wherein, The controller, using the position display deviation data, controls the display to move the position marker box toward the target position, and is configured to: Based on the position display deviation data and the starting coordinate position of the first magnified image in the preview image, the second display position adjustment parameter of the position marker box is determined; According to the second display position adjustment parameter, control the display to move the position marker box to the target position.

16. The display device according to any one of claims 1 to 15, wherein, The first display area is the entire screen area, and the image in the first display area is displayed in the center.

17. An image processing method, wherein, Applied to a display device as described in any one of claims 1 to 16, the method comprises: Receive and respond to an image save command, and associate and store a first image and a second image, wherein the first image and the second image have an image-to-image relationship; The first image and the second image are superimposed on the user interface, wherein the second image is displayed in a first display area of ​​the user interface, the first image is displayed in a second display area of ​​the first display area, the second display area is smaller than the first display area, and the layer of the image displayed in the second display area is above the layer of the image displayed in the first display area; Upon receiving and responding to a view switching command, the image displayed in the first display area is switched to be displayed in the second display area, and the image displayed in the second display area is switched to be displayed in the first display area.

18. The image processing method of claim 17, wherein, The method further includes: In response to a preview zoom-in operation on an image displayed in the first display area or the second display area, the image is determined as a preview image, and the display is controlled to display the preview image's display interface. The display interface includes a first zoomed-in view of the preview image and a thumbnail display area. The thumbnail display area includes a position marker box, which is used to characterize the position of the first zoomed-in view in the preview image. In response to an interactive operation on a target location in the thumbnail display area, the display is controlled to move the location marker box toward the target location, and the display is controlled to present a second magnified view of the preview image in the display interface; The position of the second magnified image in the preview image matches the position of the moved position marker box in the thumbnail display area.

19. A computer readable storage medium having stored thereon a computer program, wherein, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 17 to 18.

20. A computer program product comprising a computer program, wherein, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 17 to 18.

Images