Image display method and device

Abstract

The invention provides an image display method and device, relates to the field of image processing, and can improve the visual experience of a user. According to the specific technical scheme, an original image is received and decoded, and a first image is obtained; when the resolution negotiation event is detected, stopping sending the mouse event information, the keyboard event information and the coordinate information; identifying a state of the resolution negotiation event; when the state of the resolution negotiation event is identified to be ongoing or ended, acquiring a second image, and acquiring a similarity curve according to the first image and the second image; when a first preset condition is met, the second image is displayed, the first preset condition is that the similarity curve conforms to the similarity characteristics or the timer is overtime, and the similarity characteristics are that the change process of the similarity curve is high, low and high. According to the invention, when an unpredictable image frequent change process is finished can be accurately judged by judging the change curve of the similarity, so that the phenomenon that the image displayed by the terminal has a frequent change process in most scenes can be avoided.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to an image display method and device. Background Art

[0002] In the field of cloud desktop image display, in order to ensure a good user visual experience when the image resolution changes, before the terminal display module sends the image captured by the cloud server image acquisition module (referred to as: cloud server acquisition module, the same below) to the terminal display, the terminal display module and the cloud server acquisition module must complete the resolution and other parameter negotiation (referred to as resolution negotiation, the same below) process.

[0003] Currently, if the image resolution changes, the graphics card must reset the image being rendered and pause the current image output, and then re-render and output the new image based on the target resolution. The image sequence collected by the server acquisition module in the above process has an unpredictable and frequent change process, and this process takes a long time. At the same time, due to the unpredictable and frequent change process of the collected image sequence, the image received and displayed by the terminal must also have the same phenomenon, which brings a very bad visual experience to the user, which is an urgent problem to be solved.

[0004] Industry analysis has found that the above problems are caused by the inherent characteristics of graphics cards, and the weaker the graphics card performance, the longer the unpredictable frequent image changes will take. For example, the above problems are very obvious in software-simulated GPU and virtual graphics card (Virtual GPU, vGPU) scenarios.

[0005] Currently, although efforts have been made to reduce the time of unpredictable and frequent changes in image sequences caused by image resolution negotiation, in actual applications, limited by hardware cost control, people generally use software-simulated GPUs and low-performance vGPUs. However, due to the low performance of the CPU and graphics card, it is difficult to achieve good optimization results, and the user experience is poor. Summary of the invention

[0006] The embodiments of the present disclosure provide an image display method and device, which can avoid frequent changes in the image displayed by the terminal in most scenarios, thereby improving the user's visual experience. The technical solution is as follows:

[0007] According to a first aspect of an embodiment of the present disclosure, there is provided an image display method, the method comprising: receiving and decoding an original image to obtain a first image; when a resolution negotiation event is detected, suspending sending mouse event information, keyboard event information, and coordinate information; identifying the status of the resolution negotiation event; when the status of the resolution negotiation event is identified as being in progress or completed, obtaining a second image, and obtaining a similarity curve based on the first image and the second image, the second image being a new decoded image; when a first preset condition is met, displaying the second image, the first preset condition being that the similarity curve meets a similarity feature or a timer times out, the similarity feature being that the change process of the similarity curve is high, low, and high.

[0008] For example, after the terminal subsystem and the cloud server subsystem establish a session connection, the transmission module can continuously receive the image sequence (ie, the original image) sent by the cloud server subsystem, and forward the image sequence to the decoding module, and then the decoding module can decode the image frame to obtain the first image.

[0009] When the detection module detects a resolution negotiation event, the mouse event information, keyboard event information, and coordinate information may be sent through the reverse control module.

[0010] The identification module can identify or determine the value of the state S of the resolution negotiation event. If S=0, it means that the state of the resolution negotiation event is not started or the resolution negotiation process is not initiated; if S=1, it means that the state of the resolution negotiation event is in progress or has ended.

[0011] When the state of the resolution negotiation event is identified as being in progress or completed, the second image Pn (n>1) may be received by the identification module, which is also a new decoded image (ie, a new image frame).

[0012] When the first preset condition is met, the second image (ie, the latest image frame) can be displayed by the display module.

[0013] Specifically, the similarity feature refers to a process of changing from high to very low, and then from very low to high, where high and very low correspond to two predetermined similarity threshold parameters respectively; the timer timeout can be determined by T t >T1 indicates that T1 is a predetermined time parameter.

[0014] Based on the above scheme, it is possible to accurately determine when the unpredictable frequent image change process ends by judging the similarity change curve, thereby avoiding frequent changes in the image displayed on the terminal in most scenarios, thereby achieving the purpose of improving the user's visual experience.

[0015] In some embodiments, when the status of the resolution negotiation event is not started, the method also includes: caching or updating the first image; initiating a resolution negotiation process, setting the status of the resolution negotiation event to the ongoing or ended; starting the timer and setting the recognition status to recognizing; displaying a preset image; clearing the second image; returning to execute the step of receiving and decoding the original image to obtain the first image.

[0016] Specifically, when the state of the resolution negotiation event is not started, the first image P1 (i.e., the current image frame) is cached. If the first image P1 already exists, the first image P1 is updated, and the session module is notified to initiate the resolution negotiation process. Afterwards, the session module initiates the resolution negotiation process and sets the state of the resolution negotiation event to ongoing or ended (i.e., sets the state of the resolution negotiation event S=1). Then, the timer T can be started by the identification module. t The timing is performed, and the recognition state is set to be in recognition (ie, the recognition state R is set to 1). Finally, the preset image P0 can be displayed by the display module, and the second image P0 in the display image frame buffer is cleared. n , and returns to execute S201, receiving and decoding the original image, and obtaining the first image. The preset image P0 is a specific image, which usually displays a prompt like "image loading" or a dynamic image similar to displaying progress.

[0017] In some embodiments, when the first preset condition is not met, the method further includes: returning to execute the step of clearing the second image.

[0018] In some embodiments, when the resolution negotiation event is not detected, the method further includes: determining the recognition state; when the recognition state is not in recognition, displaying the first image; returning to execute the step of receiving and decoding the original image to obtain the first image; when the recognition state is in recognition, returning to execute the step of identifying the state of the resolution negotiation event.

[0019] For example, when a resolution negotiation event is not detected, on the one hand, the recognition module can determine the recognition state R. When the recognition state is non-recognition (for example, recognition state R=0), the decoding module forwards the decoded image (i.e., the first image) to the display module, which renders and displays the first image through the display module, and returns to execute the step of receiving and decoding the original image to obtain the first image. On the other hand, when the recognition state is recognition (for example, R=1), the following steps of identifying the state of the resolution negotiation event can be executed.

[0020] In some embodiments, the method further includes: clearing the first image and the second image; turning off the timer; setting the recognition state to non-recognition; and resuming sending the mouse event information, the keyboard event information, and the coordinate information.

[0021] Specifically, after the second image is displayed by the display module, the image cache (ie, the first image P1 and the second image P2) can be cleared by the recognition module. n ), and turn off the timer T t Afterwards, the recognition state is set to non-recognition (ie, R=0), and the reverse control module is notified to resume sending mouse event information, keyboard event information, and coordinate information.

[0022] In some embodiments, the method further comprises: continuously detecting the resolution negotiation event.

[0023] For example, the resolution negotiation event may be continuously detected by the detection module.

[0024] In some embodiments, obtaining a similarity curve based on the first image and the second image includes: obtaining the first image and the second image of the same size through an image scaling algorithm; calculating the image content similarity between the first image and the second image of the same size; and recording and obtaining a similarity curve based on the image content similarity.

[0025] For example, the first image P1 and the second image P n If the two images are of different sizes, the image frames of the two images are scaled to the same size through an image scaling algorithm to obtain a first image P1 and a second image P of the same size. n , and calculate the first image P1 and the second image P through the image similarity algorithm n The image content similarity is recorded and a similarity curve is formed.

[0026] According to a second aspect of an embodiment of the present disclosure, an image display device is provided, comprising: a first acquisition module, configured to receive and decode an original image to acquire a first image; a pause module, configured to pause sending mouse event information, keyboard event information, and coordinate information when a resolution negotiation event is detected; a first identification module, configured to identify the state of the resolution negotiation event; the first acquisition module, further configured to acquire a second image when identifying the state of the resolution negotiation event as being in progress or completed, and to acquire a similarity curve based on the first image and the second image, the second image being a new decoded image; and a first display module, configured to display the second image when a first preset condition is met, the first preset condition being that the similarity curve meets a similarity feature or a timer times out, the similarity feature being that the change process of the similarity curve is high, low, and high.

[0027] In some embodiments, the device also includes: an update module, used to cache or update the first image; an initiation module, used to initiate a resolution negotiation process, and set the status of the resolution negotiation event to the ongoing or ended; a start module, used to start the timer and set the recognition status to identifying; a second display module, used to display a preset image; a first clearing module, used to clear the second image; a second acquisition module, used to return to execute the steps of receiving and decoding the original image and acquiring the first image.

[0028] In some embodiments, when the first preset condition is not met, the device further includes: a second clearing module, configured to return to the step of clearing the second image.

[0029] In some embodiments, when the resolution negotiation event is not detected, the device also includes: a judgment module for judging the recognition state; a third display module for displaying the first image when the recognition state is not in recognition; a third acquisition module for returning to execute the step of receiving and decoding the original image to acquire the first image; and a second recognition module for returning to execute the step of identifying the state of the resolution negotiation event when the recognition state is in recognition.

[0030] In some embodiments, the device also includes: a third clearing module, used to clear the first image and the second image; a closing module, used to close the timer; a setting module, used to set the recognition state to non-recognition; and a sending module, used to resume sending the mouse event information, the keyboard event information, and the coordinate information.

[0031] In some embodiments, the apparatus further comprises: a detection module, configured to continuously detect the resolution negotiation event.

[0032] In some embodiments, the first acquisition module specifically includes: an acquisition submodule, used to obtain the first image and the second image of the same size through an image scaling algorithm; a calculation submodule, used to calculate the image content similarity between the first image and the second image of the same size; an acquisition submodule, used to record and obtain a similarity curve based on the image content similarity.

[0033] According to a third aspect of an embodiment of the present disclosure, a device for image display is provided. The image display processing device includes a processor and a memory. The memory stores at least one computer instruction. The instruction is loaded and executed by the processor to implement the steps performed in the image display method described in the first aspect and any embodiment of the first aspect.

[0034] Based on the above device, it is possible to accurately determine when the unpredictable frequent image change process ends by judging the similarity change curve, thereby avoiding frequent changes in the image displayed on the terminal in most scenarios, thereby achieving the purpose of improving the user's visual experience.

[0035] According to a fourth aspect of an embodiment of the present disclosure, a computer program storage medium is provided, characterized in that the computer program storage medium has program instructions, and when the program instructions are executed by a processor, the processor executes the method described in the first aspect.

[0036] According to a fifth aspect of an embodiment of the present disclosure, a chip system is provided, characterized in that the chip system includes at least one processor, and when a program instruction is executed in the at least one processor, the at least one processor executes the method described in the first aspect.

[0037] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0039] Figure 1 is a structural diagram of a cloud desktop image display system 100 provided in an embodiment of the present disclosure;

[0040] Figure 2 is a schematic diagram of an image display method 200 provided by an embodiment of the present disclosure;

[0041] Figure 3 is a flow chart of an image display method 200 provided by an embodiment of the present disclosure;

[0042] Figure 4is a structural diagram of an image display device 400 provided in an embodiment of the present disclosure;

[0043] Figure 5 is a structural diagram of an image display device 500 provided in an embodiment of the present disclosure;

[0044] Figure 6 is another structural diagram of an image display device 500 provided in an embodiment of the present disclosure;

[0045] Figure 7 is another structural diagram of an image display device 500 provided in an embodiment of the present disclosure;

[0046] Figure 8 is another structural diagram of an image display device 500 provided in an embodiment of the present disclosure.

[0047] Fig. 9 is another structural diagram of an image display device 500 provided in an embodiment of the present disclosure;

[0048] Fig.10 is another structural diagram of an image display device 500 provided in an embodiment of the present disclosure;

[0049] Fig.11 is another structural diagram of an image display device 500 provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0050] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0051] Cloud desktop technology is a virtualization technology based on cloud computing. It transfers the user's desktop environment and applications from the local computer to the cloud server for operation and management. Users can access their personal desktop from any terminal device (such as a personal computer, laptop, tablet or smartphone) through a network connection and enjoy a similar experience to that of a local computer. The cloud desktop experience is mainly affected by factors such as the performance of the cloud server and terminal device, the performance of the graphics card or virtual graphics card (referred to as graphics card, the same below), network bandwidth, network latency, etc., which are directly related to the hardware cost. With less hardware cost investment, how to improve the user experience through technology is a key research direction in the industry.

[0052] In the field of cloud desktop image display, in the case of changes in image resolution, to ensure a good user visual experience, before the terminal display module sends the image collected by the cloud server image acquisition module (referred to as: cloud server acquisition module, the same below) to the terminal display, the terminal display module and the cloud server acquisition module must complete the resolution and other parameter negotiation (referred to as resolution negotiation, the same below) process. There are many scenarios that trigger the terminal display module and the cloud server acquisition module to initiate the resolution negotiation process, including but not limited to the following five: 1) When the terminal screen resolution changes; 2) When the terminal session window resolution changes; 3) When the terminal user plugs in, unplugs, increases or decreases the terminal display; 4) When the terminal user connects to the target cloud server, the cloud desktop operating system of the target cloud server switches from the original virtual display to the target virtual display; 5) When the terminal user modifies the resolution and layout parameters in the display settings of the cloud desktop operating system through the client software.

[0053] Currently, in the prior art, if the image resolution changes, the graphics card must reset the image being rendered and pause the current image output, and then re-render and output the new image based on the target resolution. The image sequence collected by the server acquisition module in the above process has an unpredictable and frequent change process, which can be as short as 2 to 4 seconds or as long as 6 to 8 seconds. At the same time, since the collected image sequence has an unpredictable and frequent change process, the image received and displayed by the terminal must also have the same phenomenon, which brings a very bad visual experience to the user, which is an urgent problem to be solved.

[0054] Industry analysis has found that the above problems are caused by the inherent characteristics of graphics cards, and the weaker the graphics card performance, the longer the unpredictable frequent image changes will take. For example, the above problems are very obvious in software-simulated GPU and virtual graphics card (vGPU) scenarios. Among them, virtual graphics card is a technology that virtualizes physical GPUs. It is achieved by creating a virtual GPU on the host machine, allocating one or more virtual GPU instances to each virtual machine, thereby providing high-performance graphics processing capabilities for virtual machines or users, and realizing resource sharing and dynamic allocation.

[0055] Although the industry has done some optimization work to address the above issues, for example, efforts have been made to reduce the time of unpredictable and frequent changes in image sequences caused by image resolution negotiation. In actual applications, limited by hardware cost control, people generally use software-simulated GPUs (this technology essentially simulates GPUs through CPUs) and vGPUs with relatively low performance. However, due to the low performance of CPUs and graphics cards, it is difficult to achieve good optimization results, and the user experience is poor.

[0056] In view of this, the present disclosure provides an image display method, which can improve the above problems and enhance user experience.

[0057] Figure 1 1 is a structural diagram of a cloud desktop image display system 100 provided in an embodiment of the present disclosure, which can be applied to a (cloud desktop) image display method proposed in an embodiment of the present disclosure. Figure 1 As shown, the cloud desktop image display method and system include two main subsystems, namely the cloud server subsystem and the terminal subsystem. The cloud server subsystem is composed of an image rendering module, an acquisition module, an encoding module, a transmission module, and a reverse control module; the terminal subsystem is composed of a session module, a transmission module, a decoding module, a detection module, an identification module, a display module, and a reverse control module.

[0058] The following is a detailed introduction to the main functions of each module of the cloud server subsystem. The rendering module is mainly responsible for cloud server image rendering and generating desktop images, and its output is the input of the acquisition module; the acquisition module is mainly responsible for acquiring desktop images from the graphics card driver or the operating system of the cloud server subsystem. Its acquisition method and virtual display working mode are determined by whether there is a graphics card and the type and model of the graphics card; the encoding module is mainly responsible for image compression and encoding, and its input is the output of the acquisition module; the reverse control module is mainly responsible for receiving keyboard, mouse events and coordinate information sent by the terminal subsystem; the transmission module is mainly responsible for sending cloud desktop images and cursor data and receiving reverse control data.

[0059] The following is a detailed introduction to the main functions of each module of the terminal subsystem. The main functions of the session module include two parts. On the one hand, it is responsible for session management such as establishing, maintaining and disconnecting the session between the terminal subsystem and the cloud server subsystem; on the other hand, it is responsible for resolution negotiation, etc. The transmission module is mainly responsible for receiving the desktop image and cursor data sent by the cloud server subsystem, and sending the reverse control data of the terminal subsystem to the cloud server subsystem. The decoding module is mainly responsible for decoding the image data received by the transmission module, and its output is the input of the recognition module; the recognition module is mainly responsible for calculating the similarity between the received image sequence and the image frame P1 frame by frame, and judging whether to end the recognition process based on the recognition result and the timer measurement time, and notifying the display module to display the corresponding image frame according to the recognition result. The output of the recognition module is one of the inputs of the display module. The display module is mainly responsible for receiving the desktop image data and recognition results of the recognition module, and rendering and displaying the corresponding image frame. The reverse control module is mainly responsible for collecting the keyboard, mouse events and coordinate information of the terminal subsystem, and deciding whether to send the above information to the cloud server subsystem based on the detection results of the detection module and the recognition results of the recognition module.

[0060] It should be understood that the client software runs on the terminal, and its function is to receive the image sequence sent by the cloud server and send it to the display module for display, receive user operation behavior and forward the operation event to the cloud server operating system.

[0061] The present disclosure provides an image display method 200, such as Figure 2 As shown, the image display method comprises the following steps:

[0062] S201, receiving and decoding an original image to obtain a first image.

[0063] For example, after the terminal subsystem and the cloud server subsystem establish a session connection, the transmission module can continuously receive the image sequence (ie, the original image) sent by the cloud server subsystem, and forward the image sequence to the decoding module, and then the decoding module can decode the image frame to obtain the first image.

[0064] S202: When a resolution negotiation event is detected, suspend sending mouse event information, keyboard event information, and coordinate information.

[0065] For example, when the detection module detects a resolution negotiation event, the mouse event information, keyboard event information, and coordinate information may be sent through the reverse control module.

[0066] It should be understood that when this step is performed for the first time, the method 200 further includes:

[0067] The session module sets the resolution negotiation state to indicate that the resolution negotiation event has not started, for example, setting the resolution negotiation state S=0.

[0068] As another example, when no resolution negotiation event is detected, the method 200 may further include:

[0069] Determine the recognition status;

[0070] When the recognition state is not in recognition, displaying the first image;

[0071] Return to S201 to receive and decode the original image to obtain the first image;

[0072] When the identification status is in the process of identification, the following step S204 is executed to identify the status of the resolution negotiation event.

[0073] For example, when a resolution negotiation event is not detected, on the one hand, the recognition state R can be determined by the recognition module. When the recognition state is non-recognition (for example, recognition state R=0), the decoding module forwards the decoded image (i.e., the first image) to the display module, and the display module renders and displays the first image, and returns to execute the step of receiving and decoding the original image to obtain the first image; on the other hand, when the recognition state is recognition (for example, R=1), the following S204 can be executed to identify the state of the resolution negotiation event.

[0074] Optionally, before S202, the method 200 may further include:

[0075] Continuously detect resolution negotiation events.

[0076] For example, the resolution negotiation event may be continuously detected by the detection module.

[0077] S203: Identify the status of the resolution negotiation event.

[0078] For example, the identification module can identify or determine the value of the state S of the resolution negotiation event. If S=0, it means that the state of the resolution negotiation event is not started or the resolution negotiation process is not initiated; if S=1, it means that the state of the resolution negotiation event is in progress or has ended.

[0079] S204: When it is identified that the state of the resolution negotiation event is in progress or completed, a second image is acquired, and a similarity curve is acquired according to the first image and the second image.

[0080] The second image is a new decoded image.

[0081] For example, when the state of the resolution negotiation event is identified as being in progress or completed, the second image P may be received by the recognition module. n (n>1), which is also a new decoded image (ie, a new image frame).

[0082] Specifically, the similarity curve can be obtained according to the first image and the second image through the following steps:

[0083] Obtaining a first image and a second image of the same size by an image scaling algorithm;

[0084] Calculating image content similarity between a first image and a second image of the same size;

[0085] Record and obtain a similarity curve based on the image content similarity.

[0086] For example, the first image P1 and the second image P n If the two images are of different sizes, the image frames of the two images are scaled to the same size through an image scaling algorithm to obtain a first image P1 and a second image P of the same size. n , and calculate the first image P1 and the second image P through the image similarity algorithm n The image content similarity is recorded and a similarity curve is formed.

[0087] As another example, when the status of the resolution negotiation event is not started, the method 200 may further include:

[0088] caching or updating the first image;

[0089] Initiate the resolution negotiation process and set the status of the resolution negotiation event to ongoing or completed;

[0090] Start the timer and set the recognition status to identifying;

[0091] Display preset images;

[0092] clear the second image;

[0093] Return to S201 to receive and decode the original image to obtain the first image.

[0094] Specifically, when the state of the resolution negotiation event is not started, the first image P1 (i.e., the current image frame) is cached. If the first image P1 already exists, the first image P1 is updated, and the session module is notified to initiate the resolution negotiation process. Afterwards, the session module initiates the resolution negotiation process and sets the state of the resolution negotiation event to ongoing or ended (i.e., sets the state of the resolution negotiation event S=1). Then, the timer T can be started by the identification module. t The timing is performed, and the recognition state is set to be in recognition (ie, the recognition state R is set to 1). Finally, the preset image P0 can be displayed by the display module, and the second image P0 in the display image frame buffer is cleared. n , and returns to execute S201, receiving and decoding the original image, and obtaining the first image. The preset image P0 is a specific image, which usually displays a prompt like "image loading" or a dynamic image similar to displaying progress.

[0095] S205: When the first preset condition is met, display the second image.

[0096] The first preset condition is that the similarity curve meets the similarity feature or the timer times out, and the similarity feature is that the change process of the similarity curve is high, low, and high.

[0097] For example, when the first preset condition is met, the second image (ie, the latest image frame) may be displayed by the display module.

[0098] Specifically, the similarity feature refers to a process of changing from high to very low, and then from very low to high, where high and very low correspond to two predetermined similarity threshold parameters respectively; the timer timeout can be determined by T t >T1 indicates that T1 is a predetermined time parameter.

[0099] As another example, when the first preset condition is not met, the method 200 may further include:

[0100] Return to execute the above step of clearing the second image.

[0101] For example, when the first preset condition is not met, the process directly returns to executing the above step of clearing the second image.

[0102] Optionally, the method 200 may further include:

[0103] Clear the first image and the second image;

[0104] Turn off the timer;

[0105] Set the recognition status to non-recognition;

[0106] Resume sending mouse event information, keyboard event information, and coordinate information.

[0107] Specifically, after the second image is displayed by the display module, the image cache (ie, the first image P1 and the second image P2) can be cleared by the recognition module. n ), and turn off the timer T t Afterwards, the recognition state is set to non-recognition (ie, R=0), and the reverse control module is notified to resume sending mouse event information, keyboard event information, and coordinate information.

[0108] As an example, Figure 3 It is a flowchart of an image display method 200 provided in an embodiment of the present disclosure.

[0109] Based on the above scheme, on the one hand, the terminal subsystem detection module can detect resolution negotiation events, and by detecting these events, it can accurately determine when to initiate the resolution negotiation process; on the other hand, after the current detection module detects the resolution negotiation event, the terminal subsystem reverse control module immediately stops sending mouse event information, keyboard event information, and coordinate information to the cloud server operating system, avoiding the reverse control from affecting the image sequence to cause unexpected changes; on the other hand, the terminal subsystem identification module is responsible for three main functions. First, when the identification process is started, the display module is notified to display a specific image; second, the new image frame is received and the similarity between the new image frame and the original image frame is calculated, the similarity is recorded and a similarity curve is formed, and it is determined whether the similarity curve meets the change characteristics from high to extremely low and then from extremely low to high, or whether the identification timer has timed out. When the characteristics are met or the identification timer has timed out, the display module is notified to display the new image frame. Third, if the identification process ends, the reverse control module is notified to resume sending mouse event information, keyboard event information, and coordinate information to the cloud server operating system. Furthermore, the method of the embodiment of the present disclosure can avoid the unpredictable frequent image change process caused by resolution negotiation, or significantly shorten the above-mentioned unpredictable frequent image change process, thereby improving the user's visual experience.

[0110] Specifically, the disclosed embodiment makes full use of the three characteristics of the desktop image before and after the resolution negotiation: the first characteristic is that the desktop image is stably output before and after the resolution negotiation, and the stability means that the desktop image is free of tearing or incompleteness; the second characteristic is that the desktop image output during the resolution negotiation has tearing or incompleteness; the third characteristic is that in most application scenarios, the similarity of the desktop images stably output before and after the resolution negotiation is high, and the similarity of the desktop image output during the resolution negotiation is extremely low with the desktop image output before the resolution negotiation. Analysis of the above three characteristics shows that the similarity between the desktop image output during the entire process before, during and after the resolution negotiation and the desktop image before the resolution negotiation changes from high to extremely low, and then from extremely low to high. Further, the method provided by the disclosed embodiment can accurately determine when the unpredictable frequent image change process ends by judging the similarity change curve, thereby avoiding the frequent change process of the image displayed on the terminal in most scenarios, thereby achieving the purpose of improving the user's visual experience.

[0111] Based on the above Figure 2 The image display method described in the corresponding embodiment is as follows: an embodiment of the device disclosed herein, which can be used to execute the embodiment of the method disclosed herein.

[0112] The present disclosure provides an image display device, such as Figure 4 The image display device 400 includes: a memory 401 and a processor 402 .

[0113] The memory 401 is used to store programs.

[0114] When the program is executed in the processor 402 , the processor 402 is used to execute the above-mentioned image display method.

[0115] Processor 403 is used to: receive and decode the original image to obtain the first image; when a resolution negotiation event is detected, suspend sending mouse event information, keyboard event information, and coordinate information; identify the status of the resolution negotiation event; when the status of the resolution negotiation event is identified as being in progress or completed, obtain the second image, and obtain a similarity curve based on the first image and the second image, the second image being a new decoded image; when a first preset condition is met, display the second image, the first preset condition being that the similarity curve meets the similarity feature or the timer times out, and the similarity feature being that the change process of the similarity curve is high, low, and high.

[0116] Optionally, when the status of the resolution negotiation event is not started, the processor 403 is also used to: cache or update the first image; initiate a resolution negotiation process, set the status of the resolution negotiation event to in progress or ended; start the timer and set the recognition status to recognizing; display a preset image; clear the second image; return to execute the step of receiving and decoding the original image to obtain the first image.

[0117] Optionally, when the first preset condition is not met, the processor 403 is further configured to: return to execute the step of clearing the second image.

[0118] Optionally, when no resolution negotiation event is detected, the processor 403 is also used to: determine the recognition status; when the recognition status is not in recognition, display the first image; return to execute the step of receiving and decoding the original image to obtain the first image; when the recognition status is in recognition, return to execute the step of identifying the status of the resolution negotiation event.

[0119] Optionally, the processor 403 is further configured to: clear the first image and the second image; turn off the timer; set the recognition state to non-recognition; and resume sending mouse event information, keyboard event information, and coordinate information.

[0120] Optionally, the processor 403 is further configured to: continuously detect resolution negotiation events.

[0121] Optionally, the processor 403 is specifically used to: obtain the first image and the second image of the same size by using an image scaling algorithm; calculate the image content similarity between the first image and the second image of the same size; and record and obtain a similarity curve according to the image content similarity.

[0122] Based on the above device, it is possible to accurately determine when the unpredictable frequent image change process ends by judging the similarity change curve, thereby avoiding the frequent changes of the image displayed on the terminal in most scenarios, thereby achieving the purpose of improving the user's visual experience.

[0123] Based on the above Figure 2 The image display method described in the corresponding embodiment is as follows: an embodiment of the device disclosed herein, which can be used to execute the embodiment of the method disclosed herein.

[0124] Based on the above Figure 2 Corresponding to the image display method described in the embodiment, the embodiment of the present disclosure also provides an image display device 500, such as Figure 5 shown.

[0125] The image display device 500 includes: a first acquisition module 501, which is used to receive and decode the original image to obtain the first image; a pause module 502, which is used to pause sending mouse event information, keyboard event information, and coordinate information when a resolution negotiation event is detected; a first identification module 503, which is used to identify the status of the resolution negotiation event; the first acquisition module 501 is also used to obtain a second image when the status of the resolution negotiation event is identified as being in progress or completed, and obtain a similarity curve based on the first image and the second image, and the second image is a new decoded image; a first display module 504, which is used to display the second image when a first preset condition is met, the first preset condition is that the similarity curve meets the similarity feature or the timer times out, and the similarity feature is that the change process of the similarity curve is high, low, and high.

[0126] Alternatively, if Figure 6 As shown, the device 500 also includes: an update module 505, which is used to cache or update the first image; an initiation module 506, which is used to initiate a resolution negotiation process and set the status of the resolution negotiation event to ongoing or ended; a start module 507, which is used to start the timer and set the recognition status to identifying; a second display module 508, which is used to display a preset image; a first clearing module 509, which is used to clear the second image; and a second acquisition module 510, which is used to return to the step of receiving and decoding the original image and acquiring the first image.

[0127] Alternatively, if Figure 7 As shown, when the first preset condition is not met, the device 500 further includes: a second clearing module 511, which is used to return to the step of clearing the second image.

[0128] Alternatively, if Figure 8 As shown, when no resolution negotiation event is detected, the device 500 also includes: a judgment module 512, used to judge the recognition state; a third display module 513, used to display the first image when the recognition state is not in recognition; a third acquisition module 514, used to return to the step of receiving and decoding the original image and acquiring the first image; a second recognition module 515, used to return to the step of executing the state of recognizing the resolution negotiation event when the recognition state is in recognition.

[0129] Alternatively, if Fig. 9 As shown, the device 500 also includes: a third clearing module 516, used to clear the first image and the second image; a closing module 517, used to close the timer; a setting module 518, used to set the recognition state to non-recognition; a sending module 519, used to resume sending mouse event information, keyboard event information, and coordinate information.

[0130] Alternatively, if Fig.10As shown, the apparatus 500 further includes: a detection module 520, configured to continuously detect resolution negotiation events.

[0131] Alternatively, if Fig.11 As shown, the first acquisition module 501 specifically includes: an acquisition submodule 5011, used to obtain the first image and the second image of the same size through an image scaling algorithm; a calculation submodule 5012, used to calculate the image content similarity between the first image and the second image of the same size; an acquisition submodule 5013, used to record and obtain a similarity curve based on the image content similarity.

[0132] Based on the above Figure 2 In accordance with the image display method described in the embodiment, the embodiment of the present disclosure further provides a computer-readable storage medium. For example, the non-temporary computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, or an optical data storage device. The storage medium stores computer instructions for executing the above Figure 2 The image display method described in the corresponding embodiment will not be repeated here.

[0133] Based on the above Figure 2 Corresponding to the image display method described in the embodiment, the embodiment of the present disclosure also provides a chip system, the chip system includes at least one processor, when the program instruction is executed in the at least one processor, the at least one processor executes the above Figure 2 The image display method described in the corresponding embodiment will not be repeated here.

[0134] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0135] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An image display method, characterized in that: include: Receiving and decoding the original image to obtain a first image; When a resolution negotiation event is detected, the sending of mouse event information, keyboard event information, and coordinate information is suspended; identifying a status of the resolution negotiation event; When it is identified that the state of the resolution negotiation event is in progress or completed, a second image is acquired, and a similarity curve is acquired according to the first image and the second image, where the second image is a new decoded image; When a first preset condition is met, the second image is displayed, the first preset condition being that the similarity curve meets a similarity feature or a timer times out, the similarity feature being that a change process of the similarity curve is high, low, and high.

2. The method according to claim 1, characterized in that: When the status of the resolution negotiation event is not started, the method further includes: caching or updating the first image; Initiate a resolution negotiation process, and set the status of the resolution negotiation event to be in progress or ended; Starting the timer and setting the recognition state to being recognized; Display preset images; clearing the second image; Return to the step of receiving and decoding the original image to obtain the first image.

3. The method according to claim 2, characterized in that When the first preset condition is not met, the method further includes: Return to the step of clearing the second image.

4. The method according to claim 1, characterized in that When the resolution negotiation event is not detected, the method further includes: Determine the recognition status; When the recognition state is not in recognition, displaying the first image; Return to the step of receiving and decoding the original image to obtain the first image; When the identification status is identifying, the process returns to the step of identifying the status of the resolution negotiation event.

5. The method according to claim 1, characterized in that The method further comprises: clearing the first image and the second image; Turning off the timer; Set the recognition status to non-recognition; Resume sending the mouse event information, the keyboard event information, and the coordinate information.

6. The method according to claim 1, characterized in that The method further comprises: The resolution negotiation event is continuously detected.

7. The method according to claim 1, characterized in that The obtaining a similarity curve according to the first image and the second image includes: Obtaining the first image and the second image of the same size by an image scaling algorithm; Calculating the image content similarity between the first image and the second image of the same size; Record and obtain a similarity curve based on the image content similarity.

8. An image display device, characterized in that: including memory and processor; The memory has a stored program; When the program is executed in the processor, the processor is configured to execute the method according to any one of claims 1 to 7.

9. An image display device, characterized in that: include: A first acquisition module, used for receiving and decoding the original image to acquire a first image; A pause module, used to pause sending mouse event information, keyboard event information, and coordinate information when a resolution negotiation event is detected; A first identification module, configured to identify a state of the resolution negotiation event; The acquisition module is further configured to acquire a second image when recognizing that the state of the resolution negotiation event is in progress or completed, and acquire a similarity curve according to the first image and the second image, wherein the second image is a new decoded image; The first display module is used to display the second image when a first preset condition is met, the first preset condition being that the similarity curve meets the similarity feature or the timer times out, and the similarity feature being that the similarity curve changes from high to low to high.

10. A computer-readable storage medium, characterized in that: The computer program storage medium has program instructions, and when the program instructions are executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 7.

Images