Image processing method and display device

Abstract

The invention relates to an image processing method and display equipment. The method comprises the steps that an image saving instruction is received and responded to, a first image and a second image are stored in an associated mode, the first image and the second image have an image generation relation, the first image and the second image are displayed on a user interface in an overlapped mode, the second image is displayed in a first display area of the user interface, and the image is displayed in a second display area of the user interface. The first image is displayed in a second display area in the first display area, the second display area is smaller than the first display area, a layer of the image displayed in the second display area is located above a layer of the image displayed in the first display area, and a view switching instruction is received and responded to, and switching the image displayed in the first display area to be displayed in the second display area, and switching the image displayed in the second display area to be displayed in the first display area. According to the scheme, the first image and the second image are displayed on the user interface at the same time, the display levels of the images can be switched, and the interaction experience is improved.

Description

Technical Field

[0001] This application relates to the technical field of display devices, and particularly to an image processing method and a display device. Background Art

[0002] With the rapid development of display devices and the increasing diversification of user needs, people's intelligent requirements for display devices such as smart TVs are also getting higher and higher, and the functions of display devices are becoming more and more abundant.

[0003] Currently, users can interact with display devices in a series of ways through remote control interaction and voice interaction. However, there are still certain limitations in the user interaction experience. Summary of the Invention

[0004] This application provides an image processing method, a display device, and a server to solve the problem of poor user interaction experience.

[0005] In a first aspect, some embodiments provide an image processing method, including:

[0006] Receiving and responding to an image saving instruction, and associatively storing a first image and a second image, where the first image and the second image have a relationship of generating an image from an image;

[0007] Overlaying and displaying the first image and the second image on a user interface, where the second image is displayed in a first display area of the user interface, the first image is displayed in a second display area within the first display area, the second display area is smaller than the first display area, and the layer of the image displayed in the second display area is above the layer of the image displayed in the first display area;

[0008] Receiving and responding to a view switching instruction, switching the image displayed in the first display area to be displayed in the second display area, and switching the image displayed in the second display area to be displayed in the first display area.

[0009] The solutions of the above embodiments have the following advantages or beneficial effects:

[0010] By associatively storing the first image and the second image, it facilitates the subsequent management and version control of the images. By superimposing and displaying the first image and the second image in display areas of different sizes in the user interface, and the display level of the image displayed in the second display area is higher than that of the image displayed in the first display area, the user can view the first image and the second image simultaneously on the user interface without switching pages, reducing the visual fragmentation caused by repeated view switching. By sending a view switching instruction, the user can arbitrarily select to switch the display areas of the first image and the second image, enabling the user to focus on a certain image according to their own needs, improving the flexibility of image viewing, and overall effectively enhancing the user interaction experience.

[0011] In a second aspect, some embodiments further provide a display device, including: a display, and at least one processor, where the processor is configured to execute the steps in the above-mentioned embodiment of the image processing method.

[0012] The solutions of the above embodiments have the following advantages or beneficial effects:

[0013] For the above display device, by associatively storing the first image and the second image, it can better manage and process the first image and the second image. By superimposing and displaying the first image and the second image in display areas of different sizes in the user interface, and the display level of the image displayed in the second display area is higher than that of the image displayed in the first display area, the user can view the first image and the second image simultaneously on the user interface without switching pages, reducing the visual fragmentation caused by repeated view switching. By sending a view switching instruction, the user can arbitrarily select to switch the display areas of the first image and the second image, enabling the user to focus on a certain image according to their own needs, improving the flexibility of image viewing, and overall effectively enhancing the user interaction experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0015] Figure 1 It is a schematic diagram of the operation scenario between the display device and the control device provided by some embodiments of the present application;

[0016] Figure 2 It is a schematic diagram of the hardware configuration of the display device provided by some embodiments of the present application;

[0017] Figure 3 It is a schematic diagram of the hardware configuration of the control device provided by some embodiments of the present application;

[0018] Figure 4 Schematic diagram of the software configuration of the display device provided for some embodiments of the present application;

[0019] Figure 5 Schematic flow chart of the image processing method provided for some embodiments of the present application;

[0020] Figure 6 Schematic diagram of the superimposed display of the first image and the second image provided for some embodiments of the present application;

[0021] Figure 7 Schematic diagram of the process of switching views provided for some embodiments of the present application;

[0022] Figure 8 Schematic flow chart of the steps for generating the first image provided for some embodiments of the present application;

[0023] Figure 9 Schematic flow chart of the steps for generating the first image provided for some other embodiments of the present application;

[0024] Figure 10 Schematic diagram of the canvas adapting to portrait and landscape screens provided for some embodiments of the present application;

[0025] Figure 11 Schematic diagram of displaying a drawing trajectory on the user interface provided for some embodiments of the present application;

[0026] Figure 12 Schematic flow chart of the steps for generating the first image and the second image provided for some embodiments of the present application;

[0027] Figure 13 Schematic flow chart of the image processing method provided for some other embodiments of the present application;

[0028] Figure 14 Interaction timing diagram of the image processing method provided for some embodiments of the present application;

[0029] Figure 15 Interaction timing diagram when the image processing method is executed on the display device side provided for some embodiments of the present application. Detailed implementation manners

[0030] The embodiments will be described in detail below, and the examples are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following embodiments do not represent all implementation manners consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application described in detail in the claims.

[0031] It should be noted that the brief description of terms in this application is only for the convenience of understanding the following described embodiments, rather than intending to limit the embodiments of this application. Unless otherwise specified, these terms should be understood in their ordinary and common meanings.

[0032] In this application, terms such as "first", "second", "third", etc. in the specification, claims and the above-mentioned drawings are used to distinguish similar or like objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that such terms can be interchanged under appropriate circumstances.

[0033] The terms "comprising" and "having" and any variations thereof are intended to cover but not exclude inclusion. For example, a product or device comprising a series of components does not have to be limited to all the components clearly listed, but may include other components not clearly listed or inherent to these products or devices.

[0034] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or a combination of hardware or / and software code that can perform functions related to the element.

[0035] In the embodiments of this application, the display device 200 generally refers to a device with the ability to display images and process data. For example, the display device 200 includes but is not limited to smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, augmented reality devices, etc.

[0036] Figure 1 It is a schematic diagram of the operation scenario between the display device and the control device provided for some embodiments of this application. As Figure 1 shown, the user can operate the display device 200 through touch operations, the mobile terminal 300 and the control device 100. For example, the control device 100 can be a remote control, a stylus, a handle, etc.

[0037] The mobile terminal 300 can be used as a control device to perform the human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device to establish a communication connection with the display device 200 for data interaction. In some embodiments, the mobile terminal 300 and the display device 200 can install software applications, and achieve connection communication through network communication protocols to achieve the purpose of one-to-one control operations and data communication. It is also possible to transmit the audio and video content displayed on the mobile terminal 300 to the display device 200 to achieve the synchronous display function.

[0038] As Figure 1It is also shown that the display device 200 also communicates with the server 400 through various communication methods. The display device 200 is allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0039] The display device 200 can provide a broadcast receiving television function, and can also additionally provide an intelligent network television function with computer support functions, including but not limited to, network television, smart television, Internet Protocol Television (IPTV), etc.

[0040] Figure 2 For some embodiments of this application Figure 1 The hardware configuration block diagram of the display device 200 in

[0041] In some embodiments, the display device 200 may include at least one of a tuner demodulator 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0042] In some embodiments, the detector 230 is used to collect signals from the external environment or interact with the outside. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of environmental light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environmental scenes, user attributes, or user interaction gestures. Or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0043] In some embodiments, the display 260 includes a display function component for presenting a picture, a touch component for receiving user touch operations, and a driving component for driving image display. The display 260 is used to receive the image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, components of a menu control interface, and a user control UI interface, etc.

[0044] In some embodiments, the communication device 220 is a component for communicating with external devices or the server 400 according to various communication protocol types. The display device 200 can be provided with a plurality of communication devices 220 according to different supported communication methods. For example, when the display device 200 supports wireless network communication, the display device 200 can be provided with a communication device 220 including a WiFi function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0045] The communication device 220 can enable the display device 200 to communicate with an external device or a server 400 through a wireless or wired connection. Among them, the wired connection can connect the display device 200 to the external device through components such as data lines and interfaces. The wireless connection can connect the display device 200 to the external device through wireless signals or a wireless network. The display device 200 can directly establish a connection relationship with the external device or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0046] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and first to n interfaces for input / output. The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0047] In some embodiments, the controller 250 and the tuner demodulator 210 may be located in different split devices, that is, the tuner demodulator 210 may also be in an external device of the main device where the controller 250 is located, such as an external set-top box, etc.

[0048] In some embodiments, if a user inputs a user command on the graphical user interface (GUI) displayed on the display 260, the user input interface receives the user input command through the graphical user interface (GUI).

[0049] In some embodiments, the audio output device 270 may be the native speaker of the display device 200 or an external audio output device connected to the display device 200. Among them, for the external audio output device connected to the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device can be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0050] In some embodiments, the user input interface 280 can be used to receive instructions from user input.

[0051] Figure 3 For some embodiments provided in this application Figure 1 The hardware configuration block diagram of the control device. As Figure 3 shown, the control device 100 may include: a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0052] The control device 100 is configured to control the display device 200, receive input operation instructions from the user, and convert the operation instructions into instructions recognizable and responsive by the display device 200, acting as an interaction intermediary between the user and the display device 200.

[0053] In some embodiments, the control device 100 can be an intelligent device. For example, the control device 100 can install various applications for controlling the display device 200 according to user needs.

[0054] In some embodiments, as Figure 1 shown, the mobile terminal 300 or other intelligent electronic devices, after installing the application for controlling the display device 200, can perform functions similar to those of the control device 100.

[0055] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between internal components and the data processing functions between the external and internal.

[0056] Under the control of the controller 110, the communication interface 130 realizes the communication of control signals and data signals with the display device 200. The communication interface 130 can include at least one of a WiFi chip 131, a Bluetooth module 132, an NFC module 133, and other near - field communication modules.

[0057] The user input / output interface 140, where the input interface includes at least one of a microphone 141, a touchpad 142, a sensor 143, a button 144, and other input interfaces.

[0058] In some embodiments, the control device 100 includes at least one of the communication interface 130 and the input / output interface 140. When the communication interface 130 is configured in the control device 100, such as modules like WiFi, Bluetooth, and NFC, the user input instructions can be encoded through the WiFi protocol, or the Bluetooth protocol, or the NFC protocol and sent to the display device 200.

[0059] The memory 190 is used to store various operating programs, data, and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0060] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0061] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system may provide a user interface (control the display device), allow the user to interact with the display device 200, and support the running of various application programs.

[0062] It should be noted that the operating system may be a native operating system based on a specific operating platform, or a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for a display device.

[0063] The operating system can be divided into different modules or layers according to the functions implemented, for example Figure 4 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the system library layer and the kernel layer.

[0064] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on the applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0065] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0066] like Figure 4As shown in the figure, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc. Among them, the view system can design and implement the interface and interaction of the application. The view system includes lists, grids, text boxes, buttons, etc. The managers include at least one of the following modules: The Activity Manager is used to interact with all the activities running in the system; the Location Manager is used to provide access to the system location service for system services or applications; the Package Manager is used to retrieve various information related to the application packages currently installed on the device; the Notification Manager is used to control the display and clearing of notification messages; the Window Manager is used to manage the icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0067] In some embodiments, the Activity Manager is used to manage the life cycle of each application and the general navigation back function, such as controlling the exit, opening, and backward of the application. The Window Manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, taking screenshots, and controlling the display window changes. For example, shrinking the display window, jittering display, distorting display, etc.

[0068] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction libraries included in the system runtime layer, such as C / C++ instruction libraries, to implement the functions that the framework layer needs to achieve.

[0069] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. For example, as Figure 4 shown, hardware drivers can be configured in the kernel layer. The drivers included in the kernel layer can be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor drivers (such as fingerprint sensors, temperature sensors, pressure sensors, etc.), and power drivers, etc.

[0070] It should be noted that the above examples are only simple divisions of the functions of the operating system, and do not limit the specific form of the operating system of the display device 200 in the embodiments of the present application. According to factors such as the functions of the display device and the type of the operating system, the number of levels and the specific types of levels included in the operating system may be in other forms.

[0071] With the rapid development of display devices and the increasing diversification of user needs, people's demand for the intelligence of display devices such as smart TVs is also getting higher and higher, and the functions of display devices are becoming more and more abundant. For example, users can interact with the display device to create picture books on the display device.

[0072] However, currently, if a user wants to view (or preview) an image through a display device, usually only one image can be viewed singly. If multiple images need to be viewed, the user needs to switch pages to view them separately. In this way, it causes a sense of visual fragmentation for the user and reduces the interaction experience.

[0073] To improve the interaction experience, it is found that different display levels can be set for two images or the display areas of the images, and the images can be superimposed and displayed on the same user interface, so that the user can view two images simultaneously on one user interface and can dynamically switch the display levels of the images. Specifically, taking the first image and the second image as examples, different display levels can be set for the first image and the second image respectively, and then they are superimposed and displayed in different-sized display areas of the user interface, and the image with a higher level covers the image with a lower level. For example, one of the images can be displayed full screen, and the other image is covered on the full-screen displayed image in the form of a thumbnail. The user can dynamically switch the display levels and display sizes of the two images, so that the user can view the images at any time according to needs.

[0074] Accordingly, some embodiments of the present application provide an image processing method, as Figure 5 shown. Taking the method applied to a user terminal (hereinafter simply referred to as the terminal) as an example for illustration, the user terminal may include, but is not limited to, a display device, a computer, etc. The method includes the following steps (hereinafter simply referred to as S) 202 to S206, where:

[0075] S200, receive and respond to an image saving instruction, and store the first image and the second image in an associated manner. The first image and the second image have a relationship of image generation from another image.

[0076] An image saving instruction refers to an instruction used to indicate saving specific image data to a storage medium. The first image and the second image refer to two different images, and the first image and the second image have a relationship of generating an image from another image. Specifically, the relationship of generating an image from another image refers to a dependency relationship existing between two images. For example, one image is a result image obtained by editing, transforming, or applying certain algorithms to another image. It is not limited here whether the first image is generated or modified based on the second image, or the second image is generated or modified based on the first image.

[0077] In practical applications, it can mean that the generated first image or second image is displayed on the user interface of the terminal. When the user wants to save the image, by interacting with the terminal, an image saving instruction is sent to the terminal. When the terminal receives the image saving instruction, it will first parse the instruction to determine the image to be saved and the saving location. Then, it reads the image data in the memory and encodes it into a specified format (such as JPEG, PNG, etc.). Subsequently, it performs an associated storage operation of the image.

[0078] Exemplarily, taking the case where the second image is an image generated based on the first image as an example, it can be that the terminal generates one or more second images based on the first image. One or more second images are displayed on the user interface of the terminal. The user selects one second image to save and clicks the "Save" button. At this time, the terminal generates an image generation instruction in response to the button event. Subsequently, in response to this instruction, it identifies the second image to be saved. Then, based on the second image, it finds the identification data of the first image that has a relationship of generating an image from another image with the second image, such as an image ID (identity) or number. Then, based on the ID of the first image, the first image and the second image are stored associatively. It can be understood that while the second image is being generated, the terminal can record the relationship (the relationship of generating an image from another image) between these two images in the metadata in the form of metadata tags. In other embodiments, it can also be that the first image is generated based on the second image. The terminal, in response to the image saving instruction for the first image, finds the ID of the second image that has a relationship of generating an image from another image with the first image, and stores the first image and the second image associatively based on the ID of the second image. It can be determined specifically according to the actual situation and is not limited here.

[0079] S400, superimpose and display the first image and the second image on the user interface, where the second image is displayed in the first display area of the user interface, the first image is displayed in the second display area within the first display area, the second display area is smaller than the first display area, and the layer of the image displayed in the second display area is above the layer of the image displayed in the first display area.

[0080] An overlay display image refers to a way of overlapping multiple images on the same interface or screen according to certain rules (such as position, size, and layer), rather than separately and sequentially displaying each image. The overlay display method allows users to view multiple image contents simultaneously on the same interface and enables different visual effects and interaction experiences by adjusting the display attributes of these images.

[0081] The display area refers to the specific partition or display space in the user interface for displaying images. Attributes such as the position and size of the display area can be set by itself. In this embodiment, the first display area is a display space set on the user interface, which can be the entire screen or a relatively large window area. The second display area is a smaller display space set within the first display area with a size smaller than that of the first display area, and the display layer of the image displayed in the second display area is higher than that of the image displayed in the first display area, which means that the image displayed in the second display area will cover the image displayed in the first display area and will not be blocked by it, enabling users to always view the images displayed in the first display area and the second display area simultaneously on the user interface.

[0082] Following the previous step, after the terminal associates and stores the first image and the second image, it can set different display layers for the first image and the second image, and adjust the position and size of the two images according to preset rules (such as the size and layer of the display area), and place them in different display areas of the user interface for display to achieve an overlay effect. In this embodiment, it can be default to display the result image generated based on one image in the larger second display area, so that users can clearly view the result image. Taking the second image as the result image generated based on the first image as an example, that is, the second image is displayed in the larger first display area, and the first image is displayed in the smaller second display area. The first image and the second image can completely fill the display area or not completely fill the display area, but fill the display area in a suitable proportion. Among them, the unfilled area can be filled with the background color or filled with a specified color such as (black or white).

[0083] Exemplarily, as Figure 6 shown, the second image can be displayed full screen as the main image of concern, and the first image can be overlaid on the second image in the form of a thumbnail as auxiliary information. The large rectangular frame in the figure is the first display area, and the small rectangular frame is the second display area.

[0084] It can be understood that in some other embodiments, it can also be default to display the result image generated from one of the images in the smaller display area, as long as users can clearly see the result image, and no specific limitation is made here.

[0085] The S600 receives and responds to a view switching instruction, switches the image displayed in the first display area to be displayed in the second display area, and switches the image displayed in the second display area to be displayed in the first display area.

[0086] The view switching instruction refers to an instruction used to adjust the display positions of different images on the user interface in a multi-image display scenario. In this embodiment, the target of the view switching instruction is to exchange the display areas and / or display sizes of the first image and the second image on the user interface. That is, the image originally displayed in the larger first display area is switched to the smaller second display area, and the image originally displayed in the smaller second display area is switched to be displayed in the larger second display area.

[0087] Continuing from the previous step, taking the case where the first image and the second image are first superimposed and displayed with the second image being displayed full screen and the first image being overlaid on the second image in thumbnail form as an example, the user can send a view switching instruction to the terminal through voice interaction or other means such as clicking on the image area. After the terminal recognizes and parses the instruction, it can switch the first image to be displayed full screen and the second image to be overlaid on the first image in thumbnail form. The switching process can be referred to Figure 7 . If the user sends a view switching instruction again, the full-screen display of the second image is restored, and the first image continues to be overlaid on the second image in thumbnail form. That is to say, every time the user sends a view switching instruction, the terminal switches the display areas and display sizes of the two images once. In this way, by dynamically adjusting the display levels and sizes of these two images, the user can quickly switch the focus according to needs for more effective comparison and analysis.

[0088] The above image processing method associates and stores the first image and the second image, facilitating subsequent image management and version control. By superimposing and displaying the first image and the second image in display areas with different sizes on the user interface, and the display level of the image displayed in the second display area is higher than that of the image displayed in the first display area, the user can view the first image and the second image on the user interface at the same time without switching pages, reducing the visual fragmentation of repeatedly switching views. By sending a view switching instruction, the user can freely choose to switch the display areas of the first image and the second image, enabling the user to focus on a certain image according to their own needs, improving the flexibility of image viewing, and overall effectively enhancing the user interaction experience.

[0089] The way for the user to send a view switching instruction is not limited. In some exemplary embodiments, the view switching instruction is triggered by any of the following methods:

[0090] The first type is triggered by detecting a click operation on the screen area, where the screen area includes the first display area, the second display area, or any area of the screen.

[0091] The second type is triggered by means of voice interaction.

[0092] The third type is triggered by means of gesture recognition.

[0093] The fourth type is triggered by pressing a pointing device.

[0094] In practical applications, taking the terminal as an intelligent display device as an example, the intelligent display device supports multiple interaction methods, including but not limited to touch screen interaction, voice interaction, gesture interaction, and interaction through a pointing device, etc. Users can send view switching instructions by means of touching the screen, sending voice, making gestures, and pressing the remote control, etc.

[0095] The following will separately describe different interaction methods. Taking the first display area as the entire screen area and the second display area as a smaller area for displaying thumbnails as an example, where the second image is displayed full screen and the first image is displayed on top of the second image in the form of a thumbnail.

[0096] Exemplarily, taking touch screen interaction as an example, the user performs a click operation on the screen of the display device. The click area can be the first display area, the second display area, or any other part of the screen. After the device receives the click event, it performs view switching according to the click position, specifically including recalculating the new position and size of the image, and then updating the user interface to reflect these changes. Specifically, if the user clicks on the first image in the thumbnail area, the display levels of the first image and the second image are swapped, the size of the first image is enlarged to full screen display, and the second image is reduced to a thumbnail and covered on the second image. After that, if the user clicks on the second image, the second image is displayed full screen, and the first image is again displayed on top of the first image in the form of a thumbnail. In some other embodiments, it can also be that the user clicks on any area of the screen, and the display levels and display areas of the first image and the second image are swapped. The specific switching process is the same as the above process and will not be elaborated here. To prevent the user from accidentally touching the screen, a function of "locking the current screen" or "locking the current layer" can be provided. After the user turns on the function of "locking the current screen" or "locking the current layer", the current screen or the current layer is locked. At this time, the user's click on the screen will not trigger a view switching instruction. Only after the user cancels the function of "locking the current screen" or "locking the current layer" can the view switching instruction be normally triggered by clicking on the screen.

[0097] Taking voice interaction as an example, if the user utters predefined command words such as "switch view" or "switch image", the voice recognition module on the device captures the user's voice command, parses the voice command to recognize the user's intention. If the recognized user intention is "switch view", then the view switching operation is executed, the size of the first image is enlarged to full screen display, and the second image is reduced to a thumbnail and covered on the second image. After that, if the user utters "switch view" again and clicks on the second image, then the second image is displayed in full screen, and the first image is again displayed as a thumbnail on top of the first image. The way the first image and the second image are superimposed and displayed on the display interface is not limited.

[0098] Taking gesture interaction as an example, if the user makes specific gestures (such as swiping, waving, etc.) with a finger or by holding another device towards the sensor device around the screen, or makes specific gestures on the screen to request a view switch, the sensor captures the gesture action, recognizes the gesture type and its direction, and switches the view according to the recognition result. The specific view switching process is as described in the above embodiment and will not be elaborated here.

[0099] Taking interaction through a pointing device as an example, it can be that the user uses a remote control or other pointing device to select an element (such as an icon or a button) on the screen and confirms the selection by pressing a button. The pointing device sends a wireless signal to the display device to notify it to perform the corresponding view switching operation. After receiving the signal from the pointing device, the device determines the position of the cursor and the target object selected by the user, and performs the corresponding view switching operation according to the target object selected by the user. Similarly, the specific view switching process is as described in the above embodiment and will not be elaborated here.

[0100] In this embodiment, by providing the user with multiple interaction methods, users with different preferences can find the most suitable operation mode for themselves, improving the interaction experience.

[0101] In some exemplary embodiments, the aspect ratio and resolution of the image displayed in the first display area match the aspect ratio and resolution of the first display area, and the aspect ratio and resolution of the image displayed in the second display area match the aspect ratio and resolution of the second display area.

[0102] The aspect ratio refers to the ratio of the width to the height. For example, a screen with a resolution of 1920x1080 pixels has an aspect ratio of 16:9. The resolution refers to the total number of pixels contained in an image or a display area, usually expressed as the number of width pixels multiplied by the number of height pixels (such as 1920x1080).

[0103] In practical applications, in order to provide a better visual experience, the aspect ratio and resolution of the image displayed in the first display area can be set to match those of the first display area, and the aspect ratio and resolution of the image displayed in the second display area can be set to match those of the second display area. In this way, the images displayed in the first display area and the second display area can be presented in the correct proportion, reducing the cases of image stretching, compression, or blurring caused by mismatched proportions and improving the visual effect.

[0104] In this embodiment, the aspect ratio and resolution of the image displayed in the first display area can be set to be the same as those of the first display area, and the aspect ratio and resolution of the image displayed in the second display area can be set to be the same as those of the second display area. Specifically, if the aspect ratio of the image (is the same as that of the display area, there will be no stretching or compression of the image when displaying the image. For example, if both the image and the display area have an aspect ratio of 16:9, the image can perfectly fit the display area in its original proportion. If the resolution of the image matches that of the display area, each pixel in the image can correspond to a pixel in the display area, thus avoiding blurring or distortion problems caused by magnification or reduction. For example, when an image with a resolution of 1920x1080 pixels is displayed on a screen with a resolution of 1920x1080 pixels, each image pixel is directly mapped to a physical pixel on the screen, so that the image details can be maximally retained and the blurring problem caused by magnification or reduction can be reduced.

[0105] The relative positions of the first image and the second image are not limited. In some embodiments, the first display area is the entire screen area, the image in the first display area is centered, and the second display area is located on the left or right side of the first display area.

[0106] Assume the screen resolution is 1920x1080 pixels. It can be set that the first display area occupies the entire screen (1920x1080) and ensure that its aspect ratio is the same as that of the screen (16:9). Set the second display area as a small rectangle, such as 300x200 pixels, and place it on the left or right side of the first display area. Subsequently, load the generated second image, adjust its size to fit the first display area (1920x1080), and center it on the entire screen. Load the first image, adjust its size to fit the second display area (300x200), and align it to the left or right boundary of the first display area. It can be understood that the images displayed in the display area are consistent with the aspect ratio and resolution of the display area, so that a better visual experience can be provided. See Figure 6 which shows the effect of the second display area being located on the left side of the first display area.

[0107] In this embodiment, by setting the first display area as the entire screen area and setting the second display area on the left or right side of the first display area, it can ensure the prominent display of the main content that the user wants to view, and at the same time, it does not prevent the effective transmission of auxiliary information, making full use of the screen space.

[0108] In order not to block the information of the image displayed in the first display area, the transparency of the image displayed in the second display area can also be set to 50%, that is, the image displayed in the second display area is set to be semi-transparent overlay. It can be understood that in other embodiments, the transparency of the image displayed in the second display area can also be other values such as 40% or 55%, which can be determined according to the actual situation and is not uniquely limited, as long as the user can clearly view the image.

[0109] In addition, the user can also set the transparency of the image in the second display area by himself. In some embodiments, in response to an input operation of a transparency adjustment control, the current transparency value is obtained, and based on the current transparency value, the transparency of the image displayed in the second display area is updated.

[0110] The transparency adjustment control refers to a control used to dynamically adjust the transparency of an image. The transparency adjustment control can include, but is not limited to, a slider, or an input box that supports the user to input a specific transparency value, etc. In this embodiment, the transparency adjustment control is used to control the transparency of the image in the second display area. The transparency value is usually a floating point number between 0 and 1, where 0 represents completely transparent and 1 represents completely opaque.

[0111] Exemplarily, a transparency adjustment control such as a slider can be added to the user interface, and an event listener is set for the slider. Whenever the user moves the slider, a corresponding callback function is triggered, the current transparency value is obtained from the slider, and based on the current transparency value, the transparency attribute of the image displayed in the second display area is updated, that is, the alpha channel value of the image is modified, and the image displayed in the second display area is re-rendered.

[0112] In this embodiment, by allowing the user to precisely adjust the transparency according to personal preferences, the user can find the display effect that best suits their needs.

[0113] In some embodiments, the method further includes: in response to a zoom operation on the image displayed in the first display area or the second display area, performing a zoom process on the image displayed in the first display area or the second display area, and displaying the zoomed image.

[0114] In this embodiment, the device also supports independent scaling of images in different display areas. Specifically, pinch gesture listeners can be added to the first display area and the second display area respectively. When a user's pinch gesture for a certain display area, such as two-finger scaling, two-finger pinching, double-tap to zoom in, etc., is detected, the size of the corresponding image is adjusted according to the scaling ratio, and then the image is re-rendered according to the updated size, and its position on the screen is adjusted to ensure visual coherence. Among them, the scaling ratios corresponding to pinch gestures such as two-finger scaling, two-finger pinching, double-tap to zoom in, etc. can be set by oneself and are not uniquely limited.

[0115] In this embodiment, by providing independent scaling within different display areas, users can select the content they are interested in according to their own points of interest, enabling users to freely explore image details, which greatly enhances the user's sense of participation and satisfaction.

[0116] There are multiple ways to generate the first image. For example Figure 8 As shown, in some exemplary embodiments, before S200, the method further includes:

[0117] S100, receiving and responding to a drawing instruction to initialize a canvas.

[0118] S120, obtaining drawing trajectory data in the canvas.

[0119] S140, generating a first image based on the drawing trajectory data.

[0120] The drawing instruction is an instruction used to instruct the terminal to start a drawing board and initialize the canvas, and display a drawing interface. The user can start drawing on the canvas of the drawing interface. The drawing trajectory data can also be understood as graffiti trajectory data, which is a data set recording the positions and movement paths of each touch point of the user on the canvas, or the positions and movement trajectories of coordinate points on the canvas, etc. In this embodiment, the first image is an initial image directly converted from the drawing trajectory data in the canvas.

[0121] In specific implementation, taking the terminal as an intelligent terminal as an example, the user can issue a drawing instruction through touch screen, voice interaction or gestures. Subsequently, the terminal can initialize the canvas according to the screen resolution or preset information, including determining the size of the canvas and preparing the space for receiving user input. Subsequently, the user can draw on the canvas through diverse data input means, and the terminal captures the drawing trajectory data in the canvas in real time. After the user finishes drawing or receives a save instruction sent by the client, the captured drawing trajectory data is converted into a visual first image (i.e., a doodle or a sketch). Specifically, the data input means include but are not limited to touch input, and can also be input through non-contact input tools such as a remote control. It can be understood that the first image can be obtained not only based on the user's drawing trajectory data, but also can be an image obtained by an image generation large model based on the user's input creation requirements and corresponding prompt words, or can be an image uploaded or specified by the user.

[0122] In this embodiment, by capturing the drawing trajectory data in the canvas and generating the first image, the creative data freely drawn by the user can be converted into an image, supporting the user's free creation.

[0123] As Figure 9 shown, in some exemplary embodiments, S100 includes: S102, receiving and responding to a drawing instruction, obtaining the screen display ratio, determining the size of the canvas based on the screen display ratio, and initializing the canvas based on the size of the canvas.

[0124] The screen display ratio refers to the aspect ratio of the device screen, such as 16:9, 4:3, etc. The canvas size is the actual drawing area size determined according to the screen display ratio.

[0125] In specific implementation, the way to enter the drawing interface is not limited. It can be that after receiving and responding to the drawing instruction, starting the drawing application (or doodle application) and jumping to the drawing interface, the terminal can obtain the display width screen_width and display height screen_height of the screen through an interface provided by the platform such as DisplayMetrics, and then calculate the aspect ratio of the screen screen_ratio = screen_width / screen_height. Subsequently, according to the calculated aspect ratio, set the display area of the canvas, that is, set the canvas with the corresponding ratio according to the calculated aspect ratio. Exemplarily, the ratio of the canvas can be kept consistent with the screen ratio, or can be set in a proportional relationship, which is not limited here. After determining the size of the canvas, the calculated size can be used to initialize a blank canvas in the drawing interface for the user to draw, so that the image ratio can be kept consistent regardless of whether the device is in landscape or portrait display. For details, please refer to Figure 10The horizontal screen display and vertical screen display effects. Specifically, the painting interface is an interface where handwriting and erasing can be performed within a specific area using an input device.

[0126] In this embodiment, by determining the size of the canvas through the screen display ratio, it can ensure that the method maintains the consistency of the generated image ratio regardless of the device on which it runs. Whether it is a screen of any size, horizontal or vertical, the image will not be distorted, and users can obtain a better visual experience.

[0127] As described in the above embodiments, there are various means for users to input data. Such as Figure 9 As shown, in some exemplary embodiments, S120 includes: S122, receiving and responding to the wireless signal returned by the pointing device, determining the movement trajectory of the pointing device based on the wireless signal, and determining the drawing trajectory data in the canvas based on the movement trajectory of the pointing device.

[0128] The pointing device refers to a device such as a remote control or other device that can send a wireless signal to control the cursor.

[0129] In this embodiment, taking the terminal as the display device (hereinafter simply referred to as the device) as an example, a scenario where the user interacts with the display device through a pointing remote control (hereinafter simply referred to as the remote control) to achieve remote wireless painting or remote wireless graffiti is described. The display device is usually equipped with a pointing remote control (hereinafter simply referred to as the remote control) that can communicate wirelessly with the display device. The user can interact with the display device through the remote control to select relevant options of the display device to issue control commands, so that the display device can execute corresponding control functions according to the control signal of the remote control. Based on this, the remote control can be located by its wireless signal to obtain the position information of the remote control, and then the movement trajectory of the remote control can be obtained based on the position information of the remote control. Then, by converting the obtained movement trajectory of the remote control into the drawing trajectory data on the canvas, remote wireless painting for the display device can be achieved. Subsequently, by further generating a high-quality first image from the drawing trajectory data on the canvas, the interaction performance of the display device can be further improved.

[0130] Specifically, after the user moves the focus of the remote control to the position of the "Start Painting" control, the user can press a preset button such as the "OK" button to issue a painting instruction. The display device obtains the screen display ratio, sets the size of the canvas based on the screen display ratio, and displays a blank canvas that is consistent with the screen display ratio for the user to draw, ensuring that the drawing area adapts to the screen size. Then, the user can operate the remote control to move the remote control in the air to simulate the brush stroke action. The remote control sends a wireless signal to the display device. The built-in sensor of the device receives these signals and parses them into the movement trajectory of the remote control. The device analyzes information such as the movement direction and speed of the remote control based on the received wireless signal, calculates the specific movement trajectory, and then transmits the movement trajectory data to the drawing board SDK in real time for generating a continuous drawing path. Then, based on the movement trajectory of the remote control, corresponding drawing trajectory data is automatically generated, that is, a line corresponding to the drawing trajectory is displayed on the canvas (i.e., a sketch, which is also the first image). For details, please refer to Figure 11 . It can be that when the user presses the confirmation button on the remote control, it indicates that a drawing operation is completed, and the data of this drawing is recorded. It can be understood that the user can also erase or withdraw the corresponding drawing trajectory, and can also select the brush thickness, filling color, and other functions. After the user completes the drawing of the sketch, the user can choose to save the current sketch. In response to the save instruction, the terminal can save the sketch as a permanent file, that is, the sketch is stored only when the user clicks the save button. In this way, it helps to save system resources.

[0131] In some other embodiments, if the display device is a touch screen device, the user can also draw by touching the screen of the display device. The device directly captures the user's touch actions (including pressing, moving, and lifting), determines the position and movement trajectory of the user's touch point, and then converts it into drawing trajectory data. It can also be to recognize the user's gestures and movement trajectory, and then determine the drawing trajectory, which can be determined according to the specific situation. Refer to the related technology for determining the drawing trajectory data based on touch screen events and gesture recognition, which will not be elaborated here.

[0132] In this embodiment, the user determines the drawing trajectory data through the wireless signal sent by the pointing device, and then generates the first image, which can realize remote wireless drawing without being limited by the touch of a stylus and devices such as a wired mouse, greatly improving the user interaction experience.

[0133] As Figure 12 shown, in some exemplary embodiments, after S140, the method further includes:

[0134] S160. Preprocess the first image so that the preprocessed first image meets the input requirements of the trained image generation model. Obtain a style prompt. Based on the style prompt and the preprocessed first image, call the trained image generation model to generate a second image that matches the style prompt.

[0135] The style prompt is a textual description of the artistic style or visual features that the user hopes the second image will possess, and is used to guide the image generation model to generate a second image that matches the style prompt. The style prompt can include, but is not limited to, sketch style, watercolor style, oil painting style, comic style, hand-drawn style, abstract style, science fiction style, retro style, cyberpunk style, cinematic feeling, black and white art style, natural scenery style, cartoon style, etc. The method of obtaining the style prompt is not limited. It can be input by the user, determined by understanding the content of the first image, or multiple styles can be provided for the user to choose from. For example, if the device only pre-sets information about one image style, the style prompt can be directly determined based on the information about the image style. If multiple image style information is pre-set in the display device, the multiple image style information can be displayed on the user interface for the user to select one or more, and then the style prompt is determined. If the user does not select the corresponding image style information, it is default that the user selects the first image style shown, or the style prompt is directly determined as the default style prompt such as the comic style.

[0136] The image generation model (hereinafter referred to as the AI model) is an algorithm model trained using deep learning technology, which can generate a new image with a specific artistic style based on the given image and style prompt. The type of the image generation model is not limited, and can include, but is not limited to, diffusion models or image-to-image models, etc., as long as it can generate a new image with a specific artistic style based on the given image and style prompt.

[0137] In practical applications, after the user draws a sketch and generates a first image, the user can be prompted, or an interface can be provided for the user to input or select style prompt words to generate a second image that matches the style prompt words based on the first image. After the user selects the style prompt words, the selected style prompt words can be parsed into a form recognizable by the system and matched with the style tags in the background database. If the user clicks the "Generate Image" control, the first image is preprocessed, including but not limited to operations such as size adjustment, format conversion, color correction, denoising, and contrast enhancement, to meet the specific input requirements of the AI model, enabling the AI model to correctly understand and process the input image. Subsequently, the preprocessed first image and the corresponding style prompt words are passed as inputs to the AI model. Based on its internal parameters and trained weights, the AI model transforms the first image into a second image that conforms to the specified style and outputs it. The display device presents the generated second image on the user interface. The number of second images can be one or multiple. If there are multiple, multiple second images are presented simultaneously, and the user can select a satisfactory image from the presented second images. It can be understood that the second images generated by the model can also be pre-stored in the buffer,

[0138] In this embodiment, by combining AI technology with the user's creative inspiration, the user's creation can be transformed into high-quality images that conform to the specified style, realizing the transformation from creation to finished products and meeting the user's personalized needs.

[0139] As Figure 13 shown, in some exemplary embodiments, S200 includes: S202, receiving and responding to an image save instruction, storing the first image in a first storage area, storing the second image in a second storage area, and associating the first image and the second image through the identifier of the first image.

[0140] The storage area refers to a logical or physical storage area for storing data. In practical applications, when the user draws the first image, a temporary buffer is allocated for the first image. If the user decides to save the first image or the second image, the first image is persistently stored. Similarly, when the AI model outputs the second image, the device also allocates a temporary buffer for the second image. If the user decides to save the second image, the second image is persistently stored. It should be noted that the device stores the first image and the second image in different storage areas respectively. For the sake of distinction, in this embodiment, the two different storage areas are named "first storage area" and "second storage area" respectively. Among them, the first storage area is used to store the first image, and the second storage area is used to store the second image.

[0141] In specific implementation, one or more second images may be displayed on the user interface for the user to preview. The user selects one second image and clicks "Save to Home Screen". At this time, the device stores the second image in a preset second storage area, records the file path, and establishes an association between the first image and the second image according to the identifier of the first image, such as ID. In subsequent processing, the first image can be obtained from the first storage area respectively through the ID of the first image, and the second image can be obtained from the second storage area. Subsequently, the obtained first image and second image are superimposed and displayed on the user interface for the user to preview.

[0142] In this embodiment, by storing the first image and the second image in different storage areas respectively and performing associated storage through the identifier of the original first image, it can help the user conveniently find and re-edit their sketches, and can track the drawing creation process of different versions. The user can easily view a specific version of the image and its corresponding generation results, and can even roll back to an earlier version.

[0143] To make a clearer description of the image processing method provided in this application, the following combines a specific embodiment, as well as Figure 14 and Figure 15 are described. The specific embodiment includes the following content:

[0144] The user selects the application icon to request to start the application by pressing the remote control, and switches to the "Magic Doodle" interface through the remote control. The user selects the "Start Doodling" button, and the device opens the drawing interface. During this process, the application loads the drawing board engine, prepares basic drawing tools such as brushes, colors, and erasers, and adjusts the interface layout according to the device type. In the application initialization stage, the screen parameters of the device (such as resolution) are obtained, the aspect ratio is calculated, and based on the calculated aspect ratio, combined with the screen orientation (landscape or portrait), the canvas size is dynamically set and the canvas is initialized. Throughout the process, the screen rotation event is monitored, and the canvas and image sizes are recalculated when the screen orientation changes to adapt to the landscape and portrait modes.

[0145] The user draws a line on the canvas by moving the remote control. The device receives the wireless signal sent by the remote control, determines the position of the remote control, and at the same time simulates the position of the pen tip based on the position of the remote control, records the starting point of the pen tip, the moving trajectory, and the end coordinates to obtain the drawing trajectory, and displays the user's drawing trajectory on the canvas to obtain a sketch (the first image). During the process of the user drawing the sketch, the sketch will be saved in the buffer in the form of a temporary file in real time. If the user clicks the "Save" button after completing the sketch, the device responds to the save instruction and saves the sketch in the first storage area for the user to modify or edit the sketch later. If the user clicks the "Generate AI Image" button, the AI image generation process will be triggered. According to the input requirements of the pre-trained image generation model, the sketch is processed into a format suitable for input to the image generation model. Subsequently, based on the pre-set style prompt words and the pre-processed sketch, the trained image generation model is called to generate multiple stylized images (the second images) that match the style prompt words. The system caches and generates multiple stylized images with different styles and displays them on the user interface for the user to select. Among them, the image generation model can be deployed on the display device side or on the server side, which can be determined according to the actual situation.

[0146] If the user selects one of the stylized images and clicks the "Save" button, at this time, the device responds to the save instruction, stores the stylized image in the second storage area, and associates the sketch and the stylized image based on the ID of the sketch. Subsequently, the stylized image is displayed full screen, and the sketch covers the stylized image in the form of a thumbnail and is located on the left side of the stylized image. Among them, the transparency of the sketch is defaulted to 50%. If the user clicks on the sketch, the sketch will be displayed full screen, and the stylized image will cover the sketch in the form of a semi-transparent thumbnail. If the user clicks on the stylized image again, the full-screen display of the stylized image will be restored, and the sketch will cover the stylized image in the form of a thumbnail. During the process of the user viewing the image, the user can set the transparency of the thumbnail by dragging the slider, independently scale the sketch or the stylized image, and can also lock the current screen or layer to prevent accidental touch. In addition, if the user wants to edit the sketch again, the user can also enter the editing mode by long-pressing the sketch or clicking the "Edit Sketch" button, allowing the user to modify the sketch. After determining the sketch and the stylized image, the user can also share the sketch or the stylized image to other devices such as a smart phone.

[0147] To solve the above technical problems, an embodiment of the present application provides a display device, which includes a display and at least one processor.

[0148] Among them, the display is configured to display a user interface, and the processor is configured to execute the following steps:

[0149] Receive and in response to an image saving instruction, store the first image and the second image associatively, where the first image and the second image have a relation of image generation from another image.

[0150] Overlay and display the first image and the second image on the user interface. Among them, the second image is displayed in a first display area of the user interface, the first image is displayed in a second display area within the first display area. The second display area is smaller than the first display area, and the layer of the image displayed in the second display area is above the layer of the image displayed in the first display area.

[0151] Receive and in response to a view switching instruction, switch the image displayed in the first display area to be displayed in the second display area, and switch the image displayed in the second display area to be displayed in the first display area.

[0152] The above display device, by storing the first image and the second image associatively, can better manage and process the first image and the second image. By overlaying and displaying the first image and the second image in display areas of different sizes in the user interface, and the display level of the image displayed in the second display area is higher than that of the image displayed in the first display area, enabling the user to view the first image and the second image simultaneously on the user interface without switching pages, reducing the visual fragmentation caused by repeated view switching. The user can freely select to switch the display areas of the first image and the second image by sending a view switching instruction, enabling the user to focus on a certain image according to their own needs, improving the flexibility of image viewing, and overall effectively enhancing the user interaction experience.

[0153] In some exemplary embodiments, the processor is further configured to: receive and in response to an input operation of a transparency adjustment control, obtain the current transparency value, and based on the current transparency value, update the transparency of the image displayed in the second display area.

[0154] In some exemplary embodiments, the processor is further configured to: in response to a zoom operation on the image displayed in the first display area or the second display area, perform a zoom process on the image displayed in the first display area or the second display area, and display the zoomed image.

[0155] In some exemplary embodiments, the processor is further configured to: receive and in response to a drawing instruction, initialize a canvas, obtain drawing trajectory data in the canvas, and based on the drawing trajectory data, generate the first image.

[0156] In some exemplary embodiments, the processor is further configured to: obtain the screen display ratio, based on the screen display ratio, determine the size of the canvas, and based on the size of the canvas, initialize the canvas.

[0157] In some exemplary embodiments, the processor is further configured to: receive and respond to a wireless signal returned by a pointing device, determine a movement trajectory of the pointing device based on the wireless signal, and generate drawing trajectory data based on the movement trajectory of the pointing device.

[0158] In some exemplary embodiments, the processor is further configured to: preprocess a first image, where the preprocessed first image meets the input requirements of a trained image generation model, obtain a style prompt, and call the trained image generation model based on the style prompt and the preprocessed first image to generate a second image that matches the style prompt.

[0159] In some exemplary embodiments, the processor is further configured to: associate and store the first image and the second image through an identifier of the first image, where the second image is generated based on the first image, and the first image and the second image are respectively stored in different storage areas.

[0160] Specifically, for the specific data processing process involved when the display device executes the steps in the above embodiments, reference may be made to the description in the embodiments of the above image processing method, which will not be elaborated here.

[0161] Based on the same inventive concept, an embodiment of the present application provides a server, which includes a communication module and at least one processor.

[0162] Among them, the communication module is configured to communicate with the display device, and the processor is configured to execute the following steps:

[0163] Receive and respond to an image saving instruction, and associate and store the first image and the second image, where the first image and the second image have a relationship of image-to-image generation.

[0164] Overlay and display the first image and the second image on the user interface, where the second image is displayed in a first display area of the user interface, the first image is displayed in a second display area within the first display area, the second display area is smaller than the first display area, and the layer of the image displayed in the second display area is above the layer of the image displayed in the first display area.

[0165] Receive and respond to a view switching instruction, switch the image displayed in the first display area to be displayed in the second display area, and switch the image displayed in the second display area to be displayed in the first display area.

[0166] The above-mentioned server can better manage and process the first image and the second image by associatively storing them. By superimposing and displaying the first image and the second image in display areas with different sizes in the user interface, and the display level of the image displayed in the second display area is higher than that of the image displayed in the first display area, the user can view the first image and the second image simultaneously on the user interface without switching pages, reducing the visual fragmentation caused by repeatedly switching views. By sending a view switching instruction, the user can arbitrarily select to switch the display areas of the first image and the second image, enabling the user to focus on a certain image according to their own needs, enhancing the flexibility of image viewing, and overall effectively improving the user interaction experience.

[0167] In some exemplary embodiments, the processor is further configured to: receive and respond to an input operation of a transparency adjustment control, obtain the current transparency value, and update the transparency of the image displayed in the second display area based on the current transparency value.

[0168] In some exemplary embodiments, the processor is further configured to: respond to a zoom operation on the image displayed in the first display area or the second display area, perform zoom processing on the image displayed in the first display area or the second display area, and display the zoomed image.

[0169] In some exemplary embodiments, the processor is further configured to: receive and respond to a drawing instruction, initialize a canvas, obtain drawing trajectory data in the canvas, and generate a first image based on the drawing trajectory data.

[0170] In some exemplary embodiments, the processor is further configured to: obtain the screen display ratio, determine the size of the canvas based on the screen display ratio, and initialize the canvas based on the size of the canvas.

[0171] In some exemplary embodiments, the processor is further configured to: receive and respond to a wireless signal returned by a pointing device, determine the movement trajectory of the pointing device based on the wireless signal, and generate drawing trajectory data based on the movement trajectory of the pointing device.

[0172] In some exemplary embodiments, the processor is further configured to: preprocess the first image, where the preprocessed first image meets the input requirements of a trained image generation model, obtain a style prompt, and call the trained image generation model based on the style prompt and the preprocessed first image to generate a second image that matches the style prompt.

[0173] In some exemplary embodiments, the processor is further configured to: associate and store a first image and a second image through an identifier of the first image, where the second image is generated based on the first image, and the first image and the second image are respectively stored in different storage areas.

[0174] Specifically, for the specific data processing process involved when the server executes the steps in the above embodiments, reference may be made to the description in the above embodiments of the image processing method, which will not be elaborated here.

[0175] In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the methods of the above embodiments are implemented.

[0176] In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the methods of the above embodiments are implemented.

[0177] In an embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the methods of the above embodiments are implemented.

[0178] It should be noted that the user information (including but not limited to user device information such as resolution, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data such as the first image and the second image, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0179] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in this application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, artificial intelligence (AI) processors, etc., without limitation.

[0180] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this application.

[0181] The above embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. An image processing method, characterized in that, The method includes: Receiving and in response to an image saving instruction, storing the first image and the second image in an associated manner, where the first image and the second image have a relationship of image generation from another image; Overlaying and displaying the first image and the second image on a user interface, where the second image is displayed in a first display area of the user interface, the first image is displayed in a second display area within the first display area, the second display area is smaller than the first display area, and the layer of the image displayed in the second display area is above the layer of the image displayed in the first display area; Receiving and in response to a view switching instruction, switching the image displayed in the first display area to be displayed in the second display area, and switching the image displayed in the second display area to be displayed in the first display area.

2. The method according to claim 1, wherein The method further includes: Receiving and in response to an input operation on a transparency adjustment control, obtaining a current transparency value; Based on the current transparency value, updating the transparency of the image displayed in the second display area.

3. The method according to claim 1, wherein The aspect ratio and resolution of the image displayed in the first display area match the aspect ratio and resolution of the first display area, and the aspect ratio and resolution of the image displayed in the second display area match the aspect ratio and resolution of the second display area.

4. The method according to claim 1, wherein The method further includes: In response to a zoom operation on the image displayed in the first display area or the second display area, performing a zoom process on the image displayed in the first display area or the second display area, and displaying the zoomed image.

5. The method according to any one of claims 1 to 4, characterized in that The first display area is the entire screen area, the second display area is located on the left or right side within the first display area, and the image in the first display area is centered.

6. The method according to any one of claims 1 to 4, characterized in that Before receiving and in response to the image saving instruction, the method further includes: Receiving and in response to a drawing instruction, initializing a canvas; Obtaining drawing trajectory data in the canvas; Based on the drawing trajectory data, generating the first image.

7. The method according to claim 6, characterized in that, The obtaining of the drawing trajectory data in the canvas includes: Receiving and in response to a wireless signal returned by a pointing device; Based on the wireless signal, determining the movement trajectory of the pointing device; Based on the movement trajectory of the pointing device, generating the drawing trajectory data.

8. The method according to claim 6, characterized in that, After generating the first image based on the drawing trajectory data, the method further includes: Performing preprocessing on the first image, and the preprocessed first image meets the input requirements of a trained image generation model; Obtaining a style prompt; Based on the style prompt and the preprocessed first image, invoking the trained image generation model to generate the second image that matches the style prompt.

9. The method according to any one of claims 1 to 4, characterized in that, The storing of the first image and the second image in an associated manner includes: Associatively storing the first image and the second image through the identifier of the first image, where the second image is generated based on the first image, and the first image and the second image are stored in different storage areas respectively.

10. A display device, characterized in that, The device includes: A display configured to display a user interface; and at least one processor configured to perform the steps of the method according to any one of claims 1 to 9.

Images