Data processing method, device, computer equipment and computer program

Abstract

The present invention discloses a data processing method, a computer device and a storage medium. The method includes the steps of: when a first screen frame of an application client is obtained by encoding, detecting a data amount of the first screen frame (step S101); when the data amount of the first screen frame is equal to or greater than a data amount threshold, performing screen reduction adjustment on the first screen frame to obtain a first adjusted screen frame, where the resolution of the first screen frame is a first resolution, and the resolution of the first adjusted screen frame is a second resolution, and the second resolution is lower than the first resolution (step S102); encoding the first adjusted screen frame and obtaining a screen frame code stream of the first adjusted screen frame (step S103); and sending the screen frame code stream to the application client, so that the application client displays a second screen frame based on the screen frame code stream, where the second screen frame is obtained by adjusting the resolution of the first adjusted screen frame back to the first resolution after the application client decodes the screen frame code stream to obtain the first adjusted screen frame (step S104).

Description

[Technical Field]

[0001] The present invention claims priority from a Chinese patent application filed on August 24, 2021, bearing application number 202110974007.3 and entitled "Data Processing Method, Computer Device, and Storage Medium," the entire contents of which are incorporated herein by reference.

[0002] The present invention relates to the technical field of data processing, and in particular to a data processing method, apparatus, computer equipment, storage medium and computer program product. [Background technology]

[0003]

[0003] A screen frame displayed on an application client is usually obtained by encoding in an application server of the application client. When a scene change occurs in the application client (e.g., switching from one application scene to another), the data volume of the screen frame after the scene change obtained by encoding in the application server is usually very large. In this case, if the application server directly transmits the code stream of the screen frame to the application client, the transmission speed of the code stream will be slow, resulting in a time lag when the application client displays the screen frame. Therefore, how to increase the transmission speed of the code stream and improve the smoothness of the application client displaying the screen frame has become a hot topic. Summary of the Invention

[0004] Embodiments of the present invention provide a data processing method, apparatus, computer device, storage medium and computer program product.

[0005] In one aspect of the present invention, there is provided a data processing method executed by an application server, the method including the steps of: when a first screen frame of an application client is obtained by encoding, detecting a data amount of the first screen frame; and if the data amount of the first screen frame is equal to or greater than a data amount threshold, performing screen reduction adjustment on the first screen frame to obtain a first adjusted screen frame, wherein a resolution of the first screen frame is a first resolution and a resolution of the first adjusted screen frame is a second resolution, and the second resolution is lower than the first resolution; encoding the first adjusted screen frame to obtain a screen frame code stream of the first adjusted screen frame; and sending the screen frame code stream to the application client so that the application client displays a second screen frame based on the screen frame code stream, wherein the second screen frame is obtained by adjusting the resolution of the first adjusted screen frame back to the first resolution after the application client decodes the screen frame code stream to obtain the first adjusted screen frame.

[0006] In one aspect of the present invention, there is provided a data processing method executed by an application client, the method including the steps of: acquiring a screen frame code stream transmitted by an application server, the screen frame code stream being obtained by encoding a first adjusted screen frame, the first adjusted screen frame being obtained by performing screen reduction adjustment on a first screen frame of the application client, the data volume of the first screen frame being equal to or greater than a data volume threshold, the first screen frame having a first resolution, and the resolution of the first adjusted screen frame being a second resolution, the second resolution being lower than the first resolution; decoding the screen frame code stream to acquire the first adjusted screen frame; adjusting the resolution of the first adjusted screen frame back to the first resolution, acquiring a second screen frame, and displaying the second screen frame.

[0007] In one aspect of the present invention, there is provided a data processing device including: a screen encoding module that, when a first screen frame of an application client is obtained by encoding, detects a data amount of the first screen frame; a reduction module that, when the data amount of the first screen frame is equal to or greater than a data amount threshold, performs screen reduction adjustment on the first screen frame and obtains a first adjusted screen frame, wherein the resolution of the first screen frame is a first resolution and the resolution of the first adjusted screen frame is a second resolution that is lower than the first resolution; a code stream encoding module that encodes the first adjusted screen frame and obtains a screen frame code stream of the first adjusted screen frame; and a transmission module that transmits the screen frame code stream to the application client so that the application client displays a second screen frame based on the screen frame code stream, wherein the second screen frame is obtained by adjusting the resolution of the first adjusted screen frame back to the first resolution after the application client decodes the screen frame code stream to obtain the first adjusted screen frame.

[0008] In one aspect of the present invention, a data processing device is provided, the device including: a code stream acquisition module that acquires a screen frame code stream transmitted by an application server, the screen frame code stream being obtained by encoding a first adjusted screen frame, the first adjusted screen frame being obtained by performing screen reduction adjustment on a first screen frame of an application client, the data volume of the first screen frame being equal to or greater than a data volume threshold, the first screen frame having a first resolution, and the resolution of the first adjusted screen frame being a second resolution that is lower than the first resolution; a code stream decoding module that decodes the screen frame code stream and acquires the first adjusted screen frame; and a resolution adjustment module that adjusts the resolution of the first adjusted screen frame back to the first resolution, acquires a second screen frame, and displays the second screen frame.

[0009] In one aspect of the present invention, there is provided a computing device including a memory having computer-readable instructions stored therein and one or more processors, the computer-readable instructions, when executed by the one or more processors, causing the one or more processors to perform the steps of the data processing method described in any of the above aspects.

[0010] In one aspect of the present invention, there is provided a computer-readable storage medium having stored thereon a computer program, the computer program including program instructions which, when executed by a processor, cause the processor to perform a data processing method according to any of the above aspects.

[0011] In one aspect of the present invention, there is provided a computer program product or computer program comprising computer-readable instructions, the computer-readable instructions being stored on a computer-readable storage medium, and wherein one or more processors of a computing device read and execute the computer-readable instructions from the computer-readable storage medium so as to perform the data processing method according to any of the above aspects.

[0012] The details of one or more embodiments of the invention are set forth in the drawings and description which follow. Other features, objects, and advantages of the invention will become apparent from the description, drawings, and claims. [Brief explanation of the drawings]

[0013] In order to more clearly explain the technology according to the embodiments of the present invention, the following briefly introduces drawings necessary for the description of the embodiments, and it should be noted that the drawings in the following description are merely some embodiments of the present invention, and those skilled in the art can derive other drawings based on these drawings without creative work. [Figure 1] 1 is a schematic diagram of a network architecture configuration according to an embodiment of the present invention; [Figure 2] 1 is a schematic diagram of an example of a scenario for encoding a screen frame according to the present invention; [Figure 3] 1 is a schematic diagram of an example of the flow of a data processing method according to the present invention. [Figure 4] FIG. 2 is a schematic diagram of an example of a scene change screen according to the present invention. [Figure 5] 1 is a schematic diagram of an example of the principle of encoding a screen frame according to the present invention; [Figure 6] 1 is a schematic diagram of an example of a scenario for encoding a screen frame according to the present invention; [Figure 7] 1 is a schematic diagram of an example of a scenario for encoding a screen frame according to the present invention; [Figure 8]1 is a schematic diagram of an example of a flow of encoding a screen frame according to the present invention; [Figure 9] 1 is a schematic diagram of an example of the flow of a data processing method according to the present invention. [Figure 10] 1 is a schematic diagram of an example of a screen frame shrinking scenario according to the present invention; [Figure 11] 1 is a schematic diagram of an example of the flow of a data processing method according to the present invention. [Figure 12] FIG. 2 is a schematic diagram of an example of a scenario for decoding a screen frame according to the present invention; [Figure 13] FIG. 2 is a schematic diagram of an example of a flow of decoding a screen frame according to the present invention; [Figure 14] 1 is a schematic diagram of an example of the configuration of a data processing device according to the present invention; [Figure 15] 1 is a schematic diagram of an example of the configuration of a data processing device according to the present invention; [Figure 16] FIG. 1 is a schematic diagram of an example of the configuration of a computer device according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0014] The following will clearly and completely describe the technology in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention, but the described embodiments are only a part of the embodiments of the present invention, and are not all of the embodiments. All other embodiments that a person skilled in the art can obtain based on the embodiments of the present invention without performing creative work are within the scope of the present invention.

[0015] 1 is a schematic diagram of a network architecture configuration according to an embodiment of the present invention. As shown in FIG. 1, the network architecture may include a server 200 and a terminal device cluster 100. The terminal device cluster 100 may include one or more terminal devices, where the number of terminal devices is not limited. As shown in FIG. 1, each terminal device in the terminal device cluster 100 may be connected to the server 200 via a network so that each terminal device can exchange data with the server 200 via the network connection.

[0016] The server 200 shown in FIG. 1 may be an independent physical server, a server cluster or a distributed system consisting of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDNs, and big data and artificial intelligence platforms. The terminal devices may be desktop computers, notebook computers, smartphones, tablet computers, IoT devices, and portable wearable devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car devices, etc. Portable wearable devices may be smart watches, smart bands, headsets, etc.

[0017] Here, the server 200 may be a background server for a cloud game client, and therefore may be referred to as a cloud game server. A terminal device (which may be any one of a terminal device cluster) may include a cloud game client. The server 200 may render a game in an operating system container to obtain game screen frames, and then perform video encoding on the game screen frames to obtain a code stream (binary code) of the game screen frames. The server 200 may then transmit the code stream obtained by encoding to the terminal device via a network. The terminal device may decode the code stream via the cloud game client to obtain game screen frames and display the decoded game screen frames on a screen. Here, a game screen frame refers to each screen frame in a game video, and the encoding and decoding of the game screen frames refers to the encoding and decoding of the game video.

[0018] 2 is a schematic diagram of an example of a scenario for encoding a screen frame according to the present invention. As shown in FIG. 2, when a cloud game server (e.g., the above-mentioned server 200) acquires a first screen frame of a cloud game client through encoding, the cloud game server may detect the amount of data of the first screen frame. The first screen frame may be any one frame screen of the cloud game client. In other words, each time the cloud game server acquires a screen frame of a cloud game client through encoding, the cloud game server may detect the amount of data of the screen frame acquired through encoding.

[0019] If the data amount of the first screen frame detected by the cloud game server is smaller than a data amount threshold (which may be set according to actual application scenarios), the cloud game server may directly encode the first screen frame and obtain an original code stream of the first screen frame (shown in box 100a). The cloud game server may send the original code stream of the first screen frame to a terminal device (which may be any one of the terminal devices in the above terminal device cluster 100). Then, the terminal device may decode the code stream to obtain the first screen frame, and then display the first screen frame on the front end (shown in box 101a).

[0020] When the data amount of the first screen frame detected by the cloud game server is equal to or greater than a data amount threshold, the first screen frame is typically a screen frame for a scene change, and the cloud game server may perform screen reduction adjustment on the first screen frame (indicated by frame 102a). The screen reduction adjustment on the screen frame may mean reducing the resolution of the screen frame. The cloud game server may then encode the first screen frame for which screen reduction adjustment has been performed and obtain a corresponding code stream (indicated by frame 103a). The data amount of the code stream is smaller than the data amount of the code stream obtained by directly encoding the first screen frame. The cloud game server may transmit the code stream obtained by encoding the first screen frame for which screen reduction adjustment has been performed to the terminal device so that the terminal device can decode the obtained code stream and obtain the first screen frame for which screen reduction adjustment has been performed. The terminal device may then perform super-resolution adjustment on the first screen frame for which screen reduction adjustment has been performed, obtain a screen frame having the same resolution as the first screen frame, and display the screen frame on the front end. Since the resolution of this screen frame is the same as that of the first screen frame, the image quality of the displayed screen frame does not change significantly compared with that of the first screen frame.

[0021] According to the method of the present invention, since the data volume of a screen frame for a scene change is usually relatively large, a screen frame with a relatively large data volume for a scene change may first be subjected to screen reduction adjustment before encoding. In this case, the data volume of the code stream obtained by encoding is much smaller than the data volume of the code stream obtained by encoding the original screen frame. As a result, the speed at which the cloud game server transmits the code stream obtained by encoding at this time to the cloud game client also increases, allowing the cloud game client to display the corresponding screen frame (e.g., a screen frame obtained by performing super-resolution adjustment on the first screen frame that has been subjected to screen reduction adjustment) at a faster speed, and the user does not experience a time lag in the screen frame display.

[0022] 3 is a schematic diagram of an example of the flow of a data processing method according to the present invention. As shown in FIG. 3, the method may include the following steps:

[0023] Step S101: When the first screen frame of the application client is acquired by encoding, the data amount of the first screen frame is detected.

[0024] Preferably, the execution entity according to the embodiment of the present invention may be an application server (e.g., like the above-mentioned server 200, the application server may be the above-mentioned cloud game server). The application server may also be a background server of an application client. The application client may exist in a terminal device (e.g., like the terminal devices in the above-mentioned terminal device cluster 100, the application client may be the above-mentioned cloud game client). The application client may be an APP (application program) installed in the terminal device, or may be a web client in the terminal device, but is not limited to these. Note that the operations performed by the application client below may be executed by the terminal device on which it is located. The number of the above-mentioned application servers may be one or more. If there are more than one, the multiple application servers may form a distributed data processing system to execute the method according to the embodiment of the present invention in a distributed manner.

[0025] Each screen frame displayed on the application client may be encoded by the application server, and the first screen frame may be any one of the screen frames encoded by the application server. When the application server acquires the first screen frame through encoding, the application server may detect the amount of data (which may be in bytes) of the first screen frame. In other words, each time the application server acquires one screen frame of the application client through encoding, the application server may detect the amount of data of the screen frame acquired through encoding. One screen frame may be one image frame.

[0026] Note that each screen frame displayed on the application client may be displayed in response to a user operation by the user. For example, the user may click a button on the application client to switch to and display the corresponding screen frame. Alternatively, the application client may be a cloud game client (which may also be referred to as a game client) that supports a user playing a cloud game on the cloud game client. Thus, each screen frame on the application client may be a game screen displayed in response to a game operation related to the user when the user plays a cloud game, and the first screen frame may be a game screen frame on the cloud game client. The application server may then acquire the user operation on the user's application client, encode the screen frame requested for display by the user operation, and provide the encoded screen frame to the application client in the form of a codestream. The application client may then decode the acquired codestream to display the corresponding screen frame.

[0027] Step S102: If the data amount of the first screen frame is equal to or greater than the data amount threshold, perform screen reduction adjustment on the first screen frame to obtain a first adjusted screen frame, where the resolution of the first screen frame is a first resolution and the resolution of the first adjusted screen frame is a second resolution, which is lower than the first resolution.

[0028] Preferably, the application server may perform screen reduction adjustment on the first screen frame if the detected data amount of the first screen frame is equal to or greater than the data amount threshold. For some scene-switching screen frames of the encoded application client, the data amount of the screen frame is usually greater than or equal to the data amount threshold. Therefore, by performing screen reduction adjustment on the screen frame greater than or equal to the data amount threshold, the data amount of the code stream corresponding to the screen frame acquired later can be reduced. This can accelerate the transmission speed when the application server transmits the code stream corresponding to the screen frame to the application client, thereby enabling the application client to display the scene-switching screen frame with less delay (specifically, display without time lag). The screen frame may be the first screen frame, and displaying the screen frame may mean displaying a second screen frame corresponding to the first screen frame. The second screen frame has the same screen content as the first screen frame, but is obtained by decoding the code stream after the data amount corresponding to the first frame has been reduced. For details, see the related description of step S103 below.

[0029] Preferably, the application client may be a game client. When the detected data amount of the first screen frame is equal to or greater than a data amount threshold and the first screen frame is a frame at the time of scene change in the game client, the application server may perform screen reduction adjustment on the first screen frame.

[0030] Here, when switching from screen frame a of an application client to screen frame b of the application client, if the screen content of screen frame a and the screen content of screen frame b are significantly different (for example, screen frame a belongs to a game battle screen in a game, and screen frame b belongs to a game settlement screen after the game ends), screen frame b may be determined to be the screen frame when the application client performs a scene change. In other words, a screen frame of a scene change may mean that the content type (or content scene) to which the screen content of the screen frame belongs has changed compared to the screen frame immediately before that screen frame. Also, for example, performing a scene change in a game client may mean switching from a scene of game login of the game client to a scene after game login of the game client. In this case, because the difference between the screen frame after the change and the screen frame before the change is extremely large, the screen frame after the change may be the first screen frame described above, and when a scene change occurs, the data amount of the first screen frame after the scene change is usually large (for example, larger than a data amount threshold).

[0031] 4 is a schematic diagram of an example of a scene switching screen according to the present invention. As shown in FIG. 4, the screen frame 100b is the screen frame of the scan login interface, and the screen frame 101b is the screen frame of the game login interface. When the screen frame 100b jumps to the screen frame 101b, the screen frame 101b belongs to the screen frame when the scene is switched.

[0032] In this way, when the application client is a game client, the first screen frame belongs to the game screen frames when the game client switches scenes, and the data volume of the first screen frame is equal to or greater than the data volume threshold, the first screen frame is subjected to screen reduction adjustment to obtain the first adjusted screen frame. When the game client switches scenes, if the data volume of the first screen frame is too large, the first screen frame may be subjected to screen reduction adjustment to obtain the first adjusted screen frame, and then encoded to obtain a screen frame code stream of the first adjusted screen frame. Because the data volume of the screen frame code stream is significantly smaller than the data volume of the code stream obtained by directly encoding the first screen frame, the application server can transmit the screen frame code stream to the game client at a faster speed, and the game client can display the second screen frame used to replace the first screen frame at a faster speed, thereby making the display of screen frames smoother and more accelerated and preventing the user from experiencing a time lag in displaying the screen frames.

[0033] The data volume threshold may be set according to an actual application scenario, and is not limited to this. An example of a method for obtaining the data volume threshold will be described below.

[0034] The application server may acquire a unit transmission data amount and a unit transmission frame number for the application client. The unit transmission data amount represents the number of transmission bits per second for the application client, and may be expressed as Mbps. The unit transmission frame number represents the number of transmission frames per second for the application client, and may be expressed as fps.

[0035] Therefore, the application server may obtain an average frame data amount based on the unit transmission data amount and the unit transmission frame number. The average frame data amount may generally represent the amount of data per screen frame transmitted, and may be obtained by, for example, dividing the unit transmission data amount by the unit transmission frame number. Note that while a typical user does not notice much jitter of one or two frames, users will visually notice a relatively large time lag if the number of frames exceeds three. Therefore, the data amount threshold may be set to three times the average frame data amount.

[0036] For example, the unit transmission data amount may typically be equal to 3 Mbps, and the unit transmission frame rate may typically be equal to 30 fps. Therefore, the average frame data amount may typically be equal to 3 Mbps / 30 fps = 100 kbits (100 kbits) = 12.5 kbytes (i.e., 12.5 kbytes). Here, 3M represents 3000 k, and k represents one thousand (1000). Therefore, the data amount threshold may be equal to 12.5 kbytes * 3 = 37.5 kbytes (i.e., 37.5 kbytes).

[0037] In this way, the unit transmission data volume and the number of unit transmission frames for the screen frames of the application client are obtained, the average frame data volume is determined based on the unit transmission data volume and the number of unit transmission frames, and the data volume threshold is determined based on the average frame data volume. Note that different application clients each have a corresponding unit transmission data volume and unit transmission frame volume, and accordingly, different application clients each have a corresponding average frame data volume and therefore a corresponding data volume threshold. In this way, the screen frames that need to be subjected to screen reduction adjustment for each different application client are determined based on the data volume thresholds corresponding to the different application clients, thereby achieving differentiated screen frame coding between different application clients and achieving screen frame coding that is adaptive to the application clients, thereby improving the smoothness and speed of the display screen of the application client and eliminating the time lag felt by the user when displaying screen frames.

[0038] Therefore, if the data volume of the first screen frame is equal to or greater than the data volume threshold, the application server may perform screen reduction adjustment on the first screen frame. The first screen frame subjected to screen reduction adjustment may be referred to as a first adjusted screen frame, and the reduction adjustment on the screen frame may mean reducing the resolution of the screen frame. For example, the resolution of the first screen frame may be a first resolution, and the resolution of the first adjusted screen frame may be a second resolution, which is lower than the first resolution. For example, the first resolution may be 1280*720 and the second resolution may be 640*360, where the first resolution is four times the second resolution and the second resolution is one-quarter of the first resolution. Here, for a specific process of screen reduction adjustment on the first screen frame, please refer to the description in the corresponding embodiment of FIG. 9 described later.

[0039] On the other hand, if the data amount of the first screen frame is smaller than the data amount threshold, it is not necessary to perform screen reduction adjustment on the first screen frame.

[0040] Step S103: Encode the first adjusted screen frame to obtain a screen frame codestream of the first adjusted screen frame.

[0041] Preferably, the application server may encode the first adjusted screen frame to obtain a screen frame code stream (may be abbreviated as code stream) of the first adjusted screen frame, where the unit of the code stream is a byte or a bit, and therefore the code stream may be called a byte stream or a bit stream.

[0042] Here, the screen reduction adjustment for the first screen frame is usually performed based on a horizontal reduction ratio and a vertical reduction ratio for the first screen frame (specifically, see the description of the corresponding embodiment in FIG. 9 below). The screen frame code stream of the first adjusted screen frame may further include the horizontal reduction ratio and the vertical reduction ratio. The screen frame code stream may also include a reduction flag (i.e., a reduction flag), which indicates that the first adjusted screen frame has been screen reduced. A code stream of a screen frame that has not been screen reduced may not include the reduction flag. The horizontal reduction ratio, vertical reduction ratio, and reduction flag included in the screen frame code stream of the first adjusted screen frame may all be referred to as SEI information (supplementary extension information).

[0043] Here, the specific process of the application server encoding the first adjusted screen frame may be as follows:

[0044] The application server may obtain a blank screen frame with the above-mentioned first resolution (e.g., 1280*720). The application server may copy the first adjusted screen frame to the blank screen frame, for example, to the upper left corner of the blank screen frame, and the blank screen frame to which the first adjusted screen frame is copied may be referred to as a first copy screen frame. Because the second resolution of the first adjusted screen frame (e.g., 640*360) is lower than the first resolution, after the first adjusted screen frame is copied to the blank screen frame, the first copy screen frame may further include a blank area, and the pixel values ​​of the blank area may all be equal to 0. Meanwhile, the pixel values ​​of the non-blank area of ​​the first copy screen frame may be the pixel values ​​included in the first adjusted screen frame.

[0045] In the present invention, the reason why the first adjusted screen frame is encoded by copying the first adjusted screen frame to a blank screen frame, obtaining the first copied screen frame, and then encoding the first copied screen frame is as follows: the application server can encode the screen frame using an encoder to obtain the code stream of the screen frame, and does not need to perform screen reduction adjustment for the screen frame whose data amount is smaller than the data amount threshold (hereinafter may be referred to as a normal screen frame), so that the normal screen frame whose data amount is smaller than the data amount threshold can be directly encoded by the encoder to obtain the code stream corresponding to the normal screen frame.

[0046] On the other hand, a screen frame (e.g., the above-mentioned first screen frame) whose data volume is equal to or greater than the data volume threshold value requires screen reduction adjustment, and the size (e.g., the size of the second resolution) of the screen frame (e.g., the above-mentioned first adjusted screen frame) obtained at this time after screen reduction adjustment does not conform to the encoding standard of the encoder (the encoding standard may mean the standard for encoding a screen frame of the first resolution size). Therefore, the first copy screen frame obtained by copying the first adjusted screen frame to a blank screen frame may be understood as a screen frame adjusted to return the size of the first adjusted screen frame to its original size (i.e., the size of the first resolution). In this case, the size of the first copy screen frame is the same as the size of a normal screen frame and conforms to the encoding standard of the encoder, so the first copy screen frame can be directly encoded by the encoder to achieve encoding of the first adjusted screen frame. In this way, by copying the first adjusted screen frame to a blank screen frame and then encoding it, there is no need to reconfigure the encoder, and the technical problems and resource occupation caused by reconfiguring the encoder can be avoided.

[0047] Then, the application server may encode the first copy screen frame to obtain a screen frame code stream of the first adjusted screen frame.

[0048] Note that the first copy screen frame obtained by performing screen reduction adjustment on the above-mentioned first screen frame has the same size as the first screen frame (it has the first resolution), but the amount of data is significantly reduced compared to the first screen frame. For example, if the second resolution is 1 / 4 of the first resolution, only 1 / 4 of the data in the first copy screen frame is valid, and the other 3 / 4 of the data is invalid data (for example, pixel values ​​equal to 0). In other words, the amount of data in the first copy screen frame is 1 / 4 of the amount of data in the first screen frame.

[0049] After obtaining the first copy screen frame, the application server may obtain a screen frame immediately preceding the first screen frame of the application client. The immediately preceding screen frame is obtained by encoding by the application server, and the application server obtains the immediately preceding screen frame from the background, not from the application client. The immediately preceding screen frame is the last screen frame displayed to the application client before displaying the first screen frame, and the screen frame immediately preceding the first screen frame may be referred to as the reference screen frame of the first screen frame. The application server may perform screen reduction adjustment on the reference screen frame of the first screen frame using a principle similar to that of the first screen frame, and the reference screen frame of the first screen frame after screen reduction adjustment may be referred to as the second adjusted screen frame.

[0050] Similarly, the application server may copy the second adjusted screen frame to the blank screen frame, and the blank screen frame to which the second adjusted screen frame is copied may be referred to as a second copy screen frame. The application server may use the second copy screen frame as a reference screen frame for the first copy screen frame, and further encode the first copy screen frame based on the second copy screen frame to obtain a screen frame code stream for the first adjusted screen frame.

[0051] Note that, when encoding one screen frame, encoding is usually performed with reference to the screen frame immediately preceding the screen frame (i.e., the reference frame). Therefore, the application server inputs the first copy screen frame and the second copy screen frame to an encoder (the encoder may select to apply intra-frame coding to some image blocks in the first copy screen frame using the second copy screen frame or to apply inter-frame coding to some image blocks according to its encoding policy), thereby encoding the first copy screen frame using the second copy screen frame to obtain a screen frame code stream of the first adjusted screen frame (which may also be referred to as a screen frame code stream of the first copy screen frame). Note that the screen frame code stream may be used instead of the code stream for encoding the first screen frame, and the data amount of the screen frame code stream is much smaller than the data amount of the code stream for directly encoding the first screen frame. In the present invention, the screen frame code stream may be obtained by encoding the screen frame using H264 (encoding standard) or H265 (encoding standard), and the code stream is binary data obtained by encoding.

[0052] In this way, the screen frame immediately preceding the first screen frame of the application client is obtained as the reference screen frame of the first screen frame, screen reduction adjustment is performed on the reference screen frame of the first screen frame, a second adjusted screen frame is obtained, the second adjusted screen frame is copied to the blank screen frame, a second copy screen frame is obtained, and the first copy screen frame is encoded based on the second copy screen frame to obtain a screen frame code stream. Note that the first copy screen frame may be encoded using an inter-frame coding method. When performing inter-frame coding, the screen frame immediately preceding the first screen frame is used as the reference screen frame of the first screen frame, and a method of generating the first copy screen frame based on the first screen frame is referenced, and the second copy screen frame is generated based on the reference screen frame, so that the first copy screen frame can be encoded based on the second copy screen frame to obtain a screen frame code stream.

[0053] On the other hand, if the data amount of the first screen frame is lower than the data amount threshold, the first screen frame may be directly encoded to obtain the codestream of the first screen frame.

[0054] Fig. 5 is a schematic diagram of an example of the principle of encoding a screen frame according to the present invention. As shown in Fig. 5, in a reference frame (the second copy screen frame) of a current frame (for example, the first copy screen frame), a macroblock that matches each macroblock (i.e., image block) in the current frame may be searched for, and a predicted block of the macroblock in the current frame may be obtained.

[0055] Then, the application server may obtain a corresponding residual (which may be obtained by subtracting a macroblock from its predicted block) from each macroblock in the current frame and its predicted block, and then perform transformation (transformation of the signal into the frequency domain) → quantization (discarding unimportant information) → reordering (concentrating low-frequency parts and concentrating high-frequency parts) → entropy coding (encoding into a binary stream) on the residual to obtain a coded code stream (e.g., a screen frame code stream of the above-mentioned first adjusted screen frame).

[0056] The application server may then reconstruct the current frame from the above predicted block, residual, and quantization result, and use the reconstructed current frame (reconstructed screen frame) as a reference frame (reference screen frame) for the immediately following frame.

[0057] Step S104: Send the screen frame codestream to the application client, so that the application client displays a second screen frame based on the screen frame codestream, where the second screen frame is obtained by the application client decoding the screen frame codestream to obtain a first adjusted screen frame, and then adjusting the resolution of the first adjusted screen frame back to the first resolution.

[0058] Preferably, the application server may transmit the screen frame code stream of the first adjusted screen frame obtained by the encoding to the application client. Then, the application client may display the second screen frame based on the screen frame code stream. For example, the application client may decode the code stream of the screen frame using a decoder to obtain the first copy screen frame. The application client may extract the first adjusted screen frame from the first copy screen frame. Then, the application client may perform super-resolution processing on the first adjusted screen frame using a super-resolution technique, i.e., adjust the resolution of the first adjusted screen frame from the second resolution back to the first resolution. The first adjusted screen frame after the super-resolution adjustment may be referred to as a second screen frame. The application client may display the second screen frame on an application interface. Note that the second screen frame displayed by the application client is a frame displayed in place of the first screen frame, and the second screen frame and the first screen frame have the same screen content and resolution.

[0059] FIG. 6 is a schematic diagram of an example of a scenario for encoding screen frames according to the present invention. As shown in FIG. 6, the current frame 102c may be the first screen frame, and the immediately preceding frame (i.e., the reference frame 100c) may be the reference screen frame of the first screen frame. If the data amount of the current frame 102c is equal to or greater than a data amount threshold, the application server may reduce the reference frame 100c and copy it to a blank screen frame to obtain the second copy screen frame 101c. The application server may then reduce the current frame 102c and copy it to a blank screen frame to obtain the first copy screen frame 103c. The application server may then encode the first copy screen frame 103c using the second copy screen frame 101c and obtain a codestream of the first copy screen frame 103c (the codestream may be referred to as a codestream of the first adjusted screen frame). The application server may then transmit the codestream of the first copy screen frame 103c obtained by encoding to the application client via a network. The application client may display the second screen frame based on the retrieved codestream.

[0060] Further, the application server may use a super-resolution technique to perform super-resolution adjustment on the first adjusted screen frame to obtain a second screen frame. The application server may use the second screen frame as a reference screen frame for a screen frame immediately following the first screen frame. The reference screen frame for the screen frame immediately following the first screen frame is used to encode the screen frame immediately following the first screen frame. To match the encoding and decoding, since the application client can only obtain the second screen frame and cannot obtain the original first screen frame, the application server needs to encode the screen frame immediately following the first screen frame with the second screen frame, and then the application client can decode the codestream of the screen frame immediately following the first screen frame with the second screen frame.

[0061] 7 is a schematic diagram of an example of a scenario for encoding screen frames according to the present invention. As shown in FIG. 7, the application server may perform reconstruction on the current frame (i.e., may reconstruct a first adjusted screen frame 100d). The reconstruction may be performed by performing super-resolution processing on the first adjusted screen frame 100d to obtain a second screen frame 101d whose resolution is restored to the first resolution. Then, the application server may encode the screen frame 102d immediately following the first screen frame using the second screen frame 101d, so that the application client can decode the obtained codestream and display the screen frame 102d, and send the encoded codestream to the application client via a network.

[0062] FIG. 8 is a schematic diagram of an example of a flow of encoding a screen frame according to the present invention. As shown in FIG. 8, at step s1, the application server may use a video encoder to encode a current frame screen (i.e., a current frame screen, a current screen frame), which may be the first screen frame. At step s2, if the data volume size of the encoded current frame screen is greater than (or equal to) three times the average frame size (three times the average frame size may refer to the data volume threshold), step s4 is executed; and if the data volume size of the encoded current frame screen is less than (or equal to) three times the average frame size, step s3 is executed. At step s3, the current frame screen is directly encoded to obtain a codestream of the current frame screen, and the codestream of the current frame screen is transmitted to the application client (i.e., the client) via a network. At step s4, the current screen frame is reduced in resolution (i.e., screen reduction adjustment is performed) and re-encoded to obtain a corresponding codestream (e.g., the codestream of the first adjusted screen frame), and the codestream obtained by encoding is transmitted to the application client.

[0063] By adopting the method of the present invention, if the data volume of the first screen frame is too large (greater than the data volume threshold), the first screen frame can be subjected to screen reduction adjustment to obtain the first adjusted screen frame, which is then encoded to obtain a screen frame code stream of the first adjusted screen frame. Since the data volume of the screen frame code stream can be significantly reduced compared with the code stream obtained by directly encoding the first screen frame, the application server can transmit the screen frame code stream to the application client at a faster speed, and the application client can display the second screen frame to replace the first screen frame at a faster speed, making the display of the screen frames smoother and faster and preventing the user from feeling a time lag in displaying the screen frames.

[0064] In the present invention, when an application server code acquires a first screen frame from an application client, it detects the data volume of the first screen frame. If the data volume of the first screen frame is equal to or greater than a data volume threshold, it performs screen reduction adjustment on the first screen frame to acquire a first adjusted screen frame. The first screen frame has a first resolution, and the resolution of the first adjusted screen frame is a second resolution, which is lower than the first resolution. The application server code then encodes the first adjusted screen frame to acquire a screen frame code stream of the first adjusted screen frame, and sends the screen frame code stream to the application client so that the application client displays a second screen frame based on the screen frame code stream. The second screen frame is acquired by the application client decoding the screen frame code stream to acquire the first adjusted screen frame, and then adjusting the resolution of the first adjusted screen frame back to the first resolution. Thus, according to the method of the present invention, the application server may perform screen reduction adjustment on a screen frame of a scene change (i.e., a first screen frame whose data amount is equal to or greater than a data amount threshold), encode the screen frame, obtain a corresponding code stream, and transmit the code stream to the application client. The application client may decode the obtained code stream, adjust the resolution of the decoded screen frame back to the original first resolution, and display the decoded screen frame again. This ensures the image quality of the displayed screen frame (e.g., a second screen frame), reduces the data amount of the transmitted screen frame code stream, and improves the transmission speed of the screen frame code stream, thereby improving the smoothness of the application client's display of the screen frame of a scene change.

[0065] 9 is a schematic diagram of an example of the flow of a data processing method according to the present invention. The content of the embodiment of the present invention may be combined with the content described in the embodiment corresponding to FIG. 3 above. The execution entity of the embodiment of the present invention may be the application server described above. As shown in FIG. 9, the method may include the following steps:

[0066] Step S201: When the first screen frame of the application client is acquired by encoding, the data amount of the first screen frame is detected.

[0067] Step S202: The horizontal and vertical reduction ratios for the first screen frame are obtained.

[0068] Preferably, the application server may obtain a horizontal reduction ratio and a vertical reduction ratio for performing a screen reduction adjustment on the first screen frame. The horizontal reduction ratio is used to reduce the length of the first screen frame, and the vertical reduction ratio is used to reduce the width of the first screen frame. The horizontal reduction ratio and the vertical reduction ratio may be determined according to an actual application scenario, but are not limited thereto. For example, the horizontal reduction ratio may be equal to 1 / 2, and the vertical reduction ratio may be equal to 1 / 2.

[0069] Step S203: Perform screen reduction adjustment on the first screen frame based on the horizontal reduction ratio and the vertical reduction ratio to obtain a first adjusted screen frame.

[0070] Preferably, the application server performs screen reduction adjustment on the first screen frame based on a horizontal reduction ratio and a vertical reduction ratio, and the step of obtaining the first adjusted screen frame may specifically include the following steps:

[0071] The application server may sample horizontal pixel values ​​(e.g., pixel values ​​in the length direction) in the first screen frame at a horizontal reduction ratio. Pixel values ​​obtained by sampling horizontal pixel values ​​in the first screen frame may be referred to as horizontally sampled pixel values. For example, if the horizontal reduction ratio is equal to 1 / 2, the application server may sample horizontal pixel values ​​in the first screen frame by 1 / 2. For example, the application server may scan horizontal pixel values ​​in the first screen frame and sample and obtain pixel values ​​in one column from pixel values ​​in two adjacent columns obtained by each scan. For example, if pixel values ​​in the first column, the second column, the third column, and the fourth column are obtained by scanning, a column of horizontally sampled pixel values ​​may be obtained by sampling from the pixel values ​​in the first column and the second column, and a column of horizontally sampled pixel values ​​may be obtained by sampling from the pixel values ​​in the third column and the fourth column.

[0072] The application server may sample pixel values ​​in the vertical direction (e.g., pixel values ​​in the length direction) in the first screen frame at a vertical reduction ratio. Pixel values ​​obtained by sampling pixel values ​​in the vertical direction in the first screen frame may be referred to as vertically sampled pixel values. For example, if the vertical reduction ratio is equal to 1 / 2, the application server may sample pixel values ​​in the vertical direction in the first screen frame by 1 / 2. For example, the application server may scan pixel values ​​in the vertical direction in the first screen frame and sample and obtain pixel values ​​in one row from pixel values ​​in two adjacent rows obtained by each scan. For example, if pixel values ​​in the first row, second row, third row, and fourth row are obtained by scanning, a row of vertically sampled pixel values ​​may be obtained by sampling from pixel values ​​in the first row and second row, and a row of vertically sampled pixel values ​​may be obtained by sampling from pixel values ​​in the third row and fourth row.

[0073] The application server may then generate the first adjusted screen frame from the horizontally sampled pixel values ​​and the vertically sampled pixel values ​​obtained by the sampling, where the number of pixel values ​​in the first adjusted screen frame is smaller than the number of pixel values ​​in the first screen frame, i.e., the second resolution of the first adjusted screen frame is lower than the first resolution of the first screen frame.

[0074] In this way, horizontal pixel values ​​in the first screen frame are sampled based on the horizontal reduction ratio to obtain horizontal sampled pixel values, vertical pixel values ​​in the first screen frame are sampled based on the vertical reduction ratio to obtain vertical sampled pixel values, and the first adjusted screen frame is generated based on the horizontal sampled pixel values ​​and the vertical sampled pixel values. In this way, by sampling the horizontal pixel values ​​and vertical pixel values ​​in the first screen frame based on the horizontal reduction ratio and the vertical reduction ratio, respectively, the first adjusted screen frame can be generated quickly.

[0075] Step S204: Encode the first adjusted screen frame to obtain a screen frame codestream of the first adjusted screen frame.

[0076] Preferably, the application server may encode the first adjusted screen frame. For example, the application server may encode the first copy screen frame corresponding to the first adjusted screen frame using the second copy screen frame. The code stream obtained by this encoding may be referred to as an initial screen frame code stream. The application server may then add the horizontal reduction ratio, vertical reduction ratio, and reduction flag to the initial screen frame code stream, and use the initial screen frame code stream with the horizontal reduction ratio, vertical reduction ratio, and reduction flag added as the final screen frame code stream of the first adjusted screen frame. In this manner, the application server may encode the first adjusted screen frame to obtain the initial screen frame code stream of the first adjusted screen frame, and then add the horizontal reduction ratio and vertical reduction ratio to the initial screen frame code stream to obtain the screen frame code stream. Therefore, the application client may receive this screen frame code stream and quickly and accurately decode the screen frame code stream based on the horizontal reduction ratio and vertical reduction ratio in the screen frame code stream to obtain the first adjusted screen frame.

[0077] 10 is a schematic diagram of an example of a screen frame downsizing scenario according to the present invention. As shown in FIG. 10, a first screen frame may include 8 rows and 8 columns of pixel values. The 8 rows and 8 columns of pixel values ​​are specifically: pixel value 11, pixel value 12, pixel value 13, pixel value 14, pixel value 15, pixel value 16, pixel value 17, pixel value 18, pixel value 21, pixel value 22, pixel value 23, pixel value 24, pixel value 25, pixel value 26, pixel value 27, pixel value 28, pixel value 31, pixel value 32, pixel value 33, pixel value 34, pixel value 35, pixel value 36, pixel value 37, pixel value 38, pixel value 41, pixel value 42, pixel value 43, pixel value 44, pixel value 45, pixel value 46, pixel value 47, pixel value 48, pixel value 49, pixel value 50, pixel value 51, pixel value 52, pixel value 53, pixel value 54, pixel value 55, pixel value 56, pixel value 57, pixel value 58, pixel value 59, pixel value 60, pixel value 61, pixel value 62, pixel value 63, pixel value 64, pixel value 65, pixel value 66, pixel value 67, pixel value 68, pixel value 69, pixel value 70, pixel value 71, pixel value 72, pixel value 73, pixel value 74, pixel value 75, pixel value 76, pixel value 77, pixel value 78, pixel value 79, pixel value 80, pixel value 81, pixel value 82, pixel value 83, pixel value 84, pixel value 85, pixel value 86, pixel value 87, pixel value 88, pixel value 89, pixel value 90, pixel value 7, pixel value 48, pixel value 51, pixel value 52, pixel value 53, pixel value 54, pixel value 55, pixel value 56, pixel value 57, pixel value 58, pixel value 61, pixel value 62, pixel value 63, pixel value 64, pixel value 65, pixel value 66, pixel value 67, pixel value 68, pixel value 71, pixel value 72, pixel value 73, pixel value 74, pixel value 75, pixel value 76, pixel value 77, pixel value 78, pixel value 81, pixel value 82, pixel value 83, pixel value 84, pixel value 85, pixel value 86, pixel value 87, pixel value 88.

[0078] When both the horizontal and vertical reduction ratios for the first screen frame are equal to 1 / 2, both the horizontal and vertical pixel values ​​of the first screen frame may be sampled by 1 / 2. Here, the horizontally sampled pixel values ​​obtained by sampling pixel values ​​in the horizontal direction of the first screen frame are pixel values ​​in the first column of the first screen frame (specifically, including pixel value 11, pixel value 21, pixel value 31, pixel value 41, pixel value 51, pixel value 61, pixel value 71, and pixel value 81), pixel values ​​in the third column (specifically, including pixel value 13, pixel value 23, pixel value 33, pixel value 43, pixel value 53, pixel value 63, pixel value 73, and pixel value 83), pixel values ​​in the fifth column (specifically, including pixel value 15, pixel value 25, pixel value 35, pixel value 45, pixel value 55, pixel value 65, pixel value 75, and pixel value 85), and pixel values ​​in the seventh column (specifically, including pixel value 17, pixel value 27, pixel value 37, pixel value 47, pixel value 57, pixel value 67, pixel value 77, and pixel value 87), It is assumed that the vertically sampled pixel values ​​obtained by sampling pixel values ​​in the vertical direction of the first screen frame are pixel values ​​of the first row in the first screen frame (specifically, including pixel value 11, pixel value 12, pixel value 13, pixel value 14, pixel value 15, pixel value 16, pixel value 17, and pixel value 18), pixel values ​​of the third row (specifically, including pixel value 31, pixel value 32, pixel value 33, pixel value 34, pixel value 35, pixel value 36, pixel value 37, and pixel value 38), pixel values ​​of the fifth row (specifically, including pixel value 51, pixel value 52, pixel value 53, pixel value 54, pixel value 55, pixel value 56, pixel value 57, and pixel value 58), and pixel values ​​of the seventh row (specifically, including pixel value 71, pixel value 72, pixel value 73, pixel value 74, pixel value 75, pixel value 76, pixel value 77, and pixel value 78).

[0079] In this case, a first adjusted screen frame may be generated by the intersection of the horizontally sampled pixel values ​​and the vertically sampled pixel values ​​obtained by the above sampling. The first adjusted screen frame may include pixel values ​​11, 13, 15, 17, 31, 33, 35, 37, 51, 53, 55, 57, 71, 73, 75, and 77. The first adjusted screen frame may then be copied onto a blank screen frame having the same size as the first screen frame to obtain a first copy screen frame. In this case, only 1 / 4 of the first copy screen frame contains valid data and 3 / 4 contains invalid data (i.e., 0 data).

[0080] Step S205: Send the screen frame codestream to the application client, so that the application client displays a second screen frame based on the screen frame codestream, where the second screen frame is obtained by the application client decoding the screen frame codestream to obtain a first adjusted screen frame, and then adjusting the resolution of the first adjusted screen frame back to the first resolution.

[0081] In the above embodiment, the horizontal and vertical reduction ratios for the first screen frame are acquired, and the screen reduction adjustment is performed on the first screen frame based on the horizontal and vertical reduction ratios to acquire the first adjusted screen frame. In this way, the screen reduction adjustment is performed on the first screen frame based on the horizontal and vertical reduction ratios, and the first adjusted screen frame can be quickly acquired.

[0082] 11 is a schematic diagram of an example of the flow of a data processing method according to the present invention. The content of the embodiment of the present invention may be combined with the content described in the embodiment corresponding to FIG. 3 above. As shown in FIG. 11, the method may include the following steps:

[0083] Step S301: Obtain a screen frame code stream sent by an application server. The screen frame code stream is obtained by encoding a first adjusted screen frame, where the first adjusted screen frame is obtained by performing screen reduction adjustment on the first screen frame of the application client, where the data amount of the first screen frame is equal to or greater than a data amount threshold, and the resolution of the first screen frame is a first resolution, and the resolution of the first adjusted screen frame is a second resolution, which is lower than the first resolution.

[0084] Preferably, the execution entity in the embodiment of the present invention may be an application client. The application client may obtain the screen frame code stream of the first adjusted screen frame sent by the application server. The screen frame code stream is obtained by encoding the first copy screen frame corresponding to the first adjusted screen frame with the second copy screen frame. For a specific method of obtaining the screen frame code stream, please refer to the description of the embodiment corresponding to Figure 3 above.

[0085] Step S302: Decode the screen frame codestream to obtain a first adjusted screen frame.

[0086] Preferably, the screen frame codestream acquired by the application client may include a horizontal reduction ratio and a vertical reduction ratio for performing screen reduction adjustment on the first screen frame. The application client may extract the horizontal reduction ratio and the vertical reduction ratio from the screen frame codestream. The application client may also acquire a screen frame immediately preceding the first screen frame, which may exist within the application client. The immediately preceding screen frame may be a screen frame previously decoded and displayed by the application client. The application client may use the immediately preceding screen frame as a reference screen frame for the first screen frame.

[0087] The application client may perform screen reduction adjustment on the reference screen frame of the first screen frame in the same principle as the screen reduction adjustment on the first screen frame (the ratio of the reduction adjustment is the same as that of the first screen frame) to obtain the above-mentioned second adjusted screen frame. The application client may copy the second adjusted screen frame to a blank screen frame having a resolution equal to the first resolution to obtain the second copied screen frame.

[0088] The application client may also input the second copy screen frame and the acquired screen frame code stream to a decoder, decode the screen frame code stream using the second copy screen frame, and acquire the first copy screen frame.

[0089] Furthermore, the application client may extract the first adjusted screen frame from the decoded first copy screen frame using the horizontal and vertical reduction ratios extracted from the codestream of the screen frame.

[0090] In this way, the horizontal and vertical reduction ratios are analyzed from the screen frame code stream, and the screen frame code stream is decoded to obtain a first copy screen frame. The first copy screen frame is obtained by the application server copying the first adjusted screen frame to a blank screen frame, where the resolution of the blank screen frame is a first resolution. The first adjusted screen frame is extracted from the first copy screen frame based on the horizontal and vertical reduction ratios. Note that, corresponding to the encoding process, the first copy screen frame having the first resolution can be obtained by decoding the screen frame code stream. Then, based on the horizontal and vertical reduction ratios analyzed from the screen frame code stream, the first adjusted screen frame having the exact resolution of the second resolution can be extracted from the first copy screen frame.

[0091] The method includes: obtaining a screen frame immediately preceding a first screen frame in the application client as a reference screen frame for the first screen frame; performing screen reduction adjustment on the reference screen frame of the first screen frame; obtaining a second adjusted screen frame; copying the second adjusted screen frame to a blank screen frame; obtaining a second copy screen frame; and decoding the screen frame code stream using the second copy screen frame to obtain the first copy screen frame. The method may also use an inter-frame decoding method to decode the screen frame code stream. When performing inter-frame decoding, the method may use a screen frame immediately preceding the first screen frame in the application client as a reference screen frame for the first screen frame; refer to the method of generating the first copy screen frame based on the first screen frame; and generate the second copy screen frame based on the reference screen frame. Because the first copy screen frame and the second copy screen frame have the same resolution, the screen frame code stream can be successfully decoded based on the second copy screen frame to obtain the first copy screen frame.

[0092] Step S303: Adjust the resolution of the first adjusted screen frame back to the first resolution, obtain a second screen frame, and display the second screen frame.

[0093] Preferably, the application client may perform super-resolution adjustment on the first adjusted screen frame based on a horizontal reduction ratio and a vertical reduction ratio to obtain a second screen frame. The resolution of the second screen frame is the first resolution of the first screen frame. In this way, the application client can perform super-resolution adjustment on the first adjusted screen frame based on a horizontal reduction ratio and a vertical reduction ratio so that the resolution of the second screen frame obtained by adjustment is the first resolution of the first screen frame, thereby ensuring that the resolutions of the screen frames displayed by the application client are all the same and are all the original resolution.

[0094] 12 is a schematic diagram of an example of a scenario for decoding a screen frame according to the present invention. As shown in FIG. 12, an application client may receive a codestream of a current frame (e.g., the screen frame codestream of the above-mentioned first adjusted screen frame) sent by an application server via a network. The application client may obtain a frame immediately preceding the current frame (i.e., the frame immediately preceding the first screen frame) as a reference frame 102e (i.e., a reference screen frame). In this case, the resolution of the reference frame 102e is the first resolution. The application client may downscale the reference frame and copy it onto a blank screen frame to obtain the above-mentioned second copy screen frame 103e. Then, the application client may obtain the above-mentioned first copy screen frame 100e by decoding the codestream of the current frame using the second copy screen frame 103e. The application client may obtain a second screen frame 101e having a first resolution by performing super-resolution processing on an image block (i.e., the first adjusted screen frame) of the first copy screen frame 100e that corresponds to the first adjusted screen frame, and display the second screen frame 101e on the client interface of the application client.

[0095] 13 is a schematic diagram of an example of a flow of decoding an image frame according to the present invention. As shown in FIG. 13, at a1, the application client may analyze SEI information (i.e., supplemental extension information) in the acquired codestream and determine whether a reduction flag is set in the analyzed SEI information, i.e., whether the SEI information includes a flag indicating that the image frame has been reduced. At a2, if the SEI information includes a flag indicating that the image frame has been reduced, it means that the image frame has been reduced, and step a4 is executed. If the SEI information does not include a flag indicating that the image frame has been reduced, it means that the image frame has not been reduced, and step a3 is executed. At a3, the application client may directly perform normal decoding on the acquired codestream and display the image frame obtained by the decoding. At a4, the application client may decode the codestream using a reference image frame (e.g., a second adjusted image frame) whose resolution has been similarly reduced, restore the resolution of the decoded image frame (e.g., the first adjusted image frame) to the first resolution, and display the corresponding image frame (e.g., the above-mentioned second image frame).

[0096] By adopting the method of the present invention, the application server can transmit the codestream of the first adjusted screen frame to the application client as the codestream of the first screen frame, thereby reducing the transmission data amount of the codestream for the first screen frame and improving the transmission speed of the codestream for the first screen frame, so that the application client can output the second screen frame to replace the first screen frame with less delay, and ensure that the resolution of the second screen frame is the same as that of the first screen frame, that is, ensure the display effect (e.g., clarity) of the screen.

[0097] In the present invention, when an application server obtains a first screen frame for an application client through encoding, it detects the data volume of the first screen frame. If the data volume of the first screen frame is equal to or greater than a data volume threshold, it performs screen reduction adjustment on the first screen frame to obtain a first adjusted screen frame. The resolution of the first screen frame is a first resolution, and the resolution of the first adjusted screen frame is a second resolution, which is lower than the first resolution. The application server then encodes the first adjusted screen frame to obtain a screen frame code stream for the first adjusted screen frame, and sends the screen frame code stream to the application client so that the application client displays a second screen frame based on the screen frame code stream. The second screen frame is obtained by the application client decoding the screen frame code stream to obtain the first adjusted screen frame, and then adjusting the resolution of the first adjusted screen frame back to the first resolution. As described above, according to the method of the present invention, the application server may perform screen reduction adjustment on a screen frame of a scene change (i.e., a first screen frame whose data amount is equal to or greater than a data amount threshold), encode the screen frame, obtain a corresponding code stream, and transmit the code stream to the application client. The application client may decode the obtained code stream, adjust the resolution of the decoded screen frame to the original first resolution, and then display the screen frame again. This ensures the image quality of the displayed screen frame (e.g., a second screen frame), reduces the data amount of the transmitted screen frame code stream, and improves the transmission speed of the screen frame code stream, thereby improving the smoothness of the application client's display of the screen frame of a scene change.

[0098] 14 is a schematic diagram of an example of the configuration of a data processing device according to the present invention. As shown in FIG. 14, the data processing device may be computer-readable instructions (including program code) executed in a computer device, for example, the data processing device may be application software, which may be used to perform corresponding steps in a method according to an embodiment of the present invention. As shown in FIG. 14, the data processing device 1 may include a screen encoding module 11, a downscaling module 12, a codestream encoding module 13, and a transmitting module 14.

[0099] When the screen encoding module 11 obtains the first screen frame of the application client through encoding, it detects the data amount of the first screen frame.

[0100] When the data amount of the first screen frame is equal to or greater than a data amount threshold, the reduction module 12 performs screen reduction adjustment on the first screen frame to obtain a first adjusted screen frame, where the resolution of the first screen frame is a first resolution and the resolution of the first adjusted screen frame is a second resolution, which is lower than the first resolution.

[0101] The codestream encoding module 13 encodes the first adjusted screen frame to obtain a screen frame codestream of the first adjusted screen frame.

[0102] The transmitting module 14 transmits the screen frame code stream to the application client so that the application client displays a second screen frame based on the screen frame code stream, where the second screen frame is obtained by the application client decoding the screen frame code stream to obtain a first adjusted screen frame, and then adjusting the resolution of the first adjusted screen frame back to the first resolution.

[0103] Preferably, when the reduction module 12 performs screen reduction adjustment on the first screen frame and obtains the first adjusted screen frame, it obtains a horizontal reduction ratio and a vertical reduction ratio for the first screen frame, and performs screen reduction adjustment on the first screen frame based on the horizontal reduction ratio and the vertical reduction ratio, and obtains the first adjusted screen frame.

[0104] Preferably, the reduction module 12 performs screen reduction adjustment on the first screen frame based on a horizontal reduction ratio and a vertical reduction ratio, and when obtaining the first adjusted screen frame, samples horizontal pixel values ​​in the first screen frame based on the horizontal reduction ratio to obtain the horizontal sampled pixel values, samples vertical pixel values ​​in the first screen frame based on the vertical reduction ratio to obtain the vertical sampled pixel values, and generates the first adjusted screen frame based on the horizontal sampled pixel values ​​and the vertical sampled pixel values.

[0105] Preferably, when the code stream encoding module 13 encodes the first adjusted screen frame and obtains the screen frame code stream of the first adjusted screen frame, it encodes the first adjusted screen frame and obtains the initial screen frame code stream of the first adjusted screen frame, and adds the horizontal reduction ratio and the vertical reduction ratio to the initial screen frame code stream to obtain the screen frame code stream.

[0106] Preferably, when encoding the first adjusted screen frame and obtaining a screen frame code stream of the first adjusted screen frame, the code stream encoding module 13 obtains a blank screen frame having a first resolution, copies the first adjusted screen frame to the blank screen frame, obtains a first copy screen frame, encodes the first copy screen frame, and obtains a screen frame code stream.

[0107] Preferably, when encoding the first copy screen frame and obtaining the screen frame code stream, the code stream encoding module 13 obtains the screen frame of the application client immediately before the first screen frame as a reference screen frame of the first screen frame, performs screen reduction adjustment on the reference screen frame of the first screen frame, obtains a second adjusted screen frame, copies the second adjusted screen frame to a blank screen frame, obtains a second copy screen frame, encodes the first copy screen frame based on the second copy screen frame, and obtains a screen frame code stream.

[0108] Preferably, the application client is a game client. When the data amount of the first screen frame is equal to or greater than a data amount threshold, the reduction module 12 performs screen reduction adjustment on the first screen frame. When acquiring the first adjusted screen frame, if the first screen frame belongs to a game screen frame at the time of scene switching of the game client and the data amount of the first screen frame is equal to or greater than a data amount threshold, the reduction module 12 performs screen reduction adjustment on the first screen frame and acquires the first adjusted screen frame.

[0109] Preferably, the data processing device 1 acquires the unit transmission data amount and the number of unit transmission frames for the screen frames of the application client, determines the average frame data amount based on the unit transmission data amount and the number of unit transmission frames, and determines the data amount threshold based on the average frame data amount.

[0110] Preferably, the data processing device 1 performs super-resolution adjustment on the first adjusted screen frame to obtain a second screen frame, and uses the second screen frame as a reference screen frame of a screen frame immediately after the first screen frame, which is used to encode the screen frame immediately after the first screen frame.

[0111] According to one embodiment of the present invention, the steps in the data processing method shown in Fig. 3 may be performed by each module in the data processing apparatus 1 shown in Fig. 14. For example, step S101 shown in Fig. 3 may be performed by the screen encoding module 11 in Fig. 14, step S102 shown in Fig. 3 may be performed by the downsizing module 12 in Fig. 14, step S103 shown in Fig. 3 may be performed by the codestream encoding module 13 in Fig. 14, and step S104 shown in Fig. 3 may be performed by the sending module 14 in Fig. 14.

[0112] In the present invention, when an application server obtains a first screen frame for an application client through encoding, it detects the data volume of the first screen frame. If the data volume of the first screen frame is equal to or greater than a data volume threshold, it performs screen reduction adjustment on the first screen frame to obtain a first adjusted screen frame. The resolution of the first screen frame is a first resolution, and the resolution of the first adjusted screen frame is a second resolution, which is lower than the first resolution. The application server then encodes the first adjusted screen frame to obtain a screen frame code stream for the first adjusted screen frame, and sends the screen frame code stream to the application client so that the application client displays a second screen frame based on the screen frame code stream. The second screen frame is obtained by the application client decoding the screen frame code stream to obtain the first adjusted screen frame, and then adjusting the resolution of the first adjusted screen frame back to the first resolution. As described above, according to the device of the present invention, the application server may perform screen reduction adjustment on a screen frame of a scene change (i.e., a first screen frame whose data amount is equal to or greater than a data amount threshold), encode the screen frame, obtain a corresponding code stream, and transmit the code stream to the application client. The application client may decode the obtained code stream, adjust the resolution of the decoded screen frame to the original first resolution, and then display the screen frame again. This ensures the image quality of the displayed screen frame (e.g., a second screen frame), reduces the data amount of the transmitted screen frame code stream, and improves the transmission speed of the screen frame code stream, thereby improving the smoothness of the application client's display of the screen frame of a scene change.

[0113] According to one embodiment of the present invention, each module of the data processing device 1 shown in FIG. 14 may be integrated into one or more units, or one of the units may be functionally subdivided into multiple subunits that perform similar operations without affecting the technical effect of the embodiment of the present invention. The modules are divided based on logical functions, and in actual applications, the functions of one module may be realized by multiple units, or the functions of multiple modules may be realized by one unit. In other embodiments of the present invention, the data processing device 1 may include other units, and in actual applications, these functions may be realized by other units or multiple units working together.

[0114] According to one embodiment of the present invention, the data processing device 1 shown in Fig. 14 may be configured by a general-purpose computing device such as a computer including a processing unit and a storage unit, such as a central processing unit (CPU), a random access memory (RAM), and a read-only memory (ROM), executing computer-readable instructions (including program code) capable of executing the steps of the methods shown in Fig. 3, thereby realizing the data processing method of one embodiment of the present invention. The computer-readable instructions may, for example, be recorded on a computer-readable recording medium and loaded into the computing device via the computer-readable recording medium for execution.

[0115] 15 is a schematic diagram of an example of the configuration of a data processing device according to the present invention. As shown in FIG. 15, the data processing device may be computer-readable instructions (including program code) executed in a computer device, for example, the data processing device may be application software, which may be used to perform corresponding steps in a method according to an embodiment of the present invention. As shown in FIG. 15, the data processing device 2 may include a codestream acquisition module 21, a codestream decoding module 22, and a resolution adjustment module 23.

[0116] The code stream acquisition module 21 acquires a screen frame code stream sent by the application server. The screen frame code stream is obtained by encoding a first adjusted screen frame, and the first adjusted screen frame is obtained by performing screen reduction adjustment on the first screen frame of the application client, where the data amount of the first screen frame is equal to or greater than a data amount threshold, and the resolution of the first screen frame is a first resolution, and the resolution of the first adjusted screen frame is a second resolution, and the second resolution is lower than the first resolution.

[0117] The codestream decoding module 22 decodes the screen frame codestream to obtain the first adjusted screen frame.

[0118] The resolution adjustment module 23 adjusts the resolution of the first adjusted screen frame back to the first resolution, obtains a second screen frame, and displays the second screen frame.

[0119] Preferably, the screen frame code stream includes a horizontal reduction ratio and a vertical reduction ratio for the first screen frame when performing screen reduction adjustment on the first screen frame. When decoding the screen frame code stream to obtain the first adjusted screen frame, the code stream decoding module 22 analyzes the horizontal reduction ratio and the vertical reduction ratio from the screen frame code stream, decodes the screen frame code stream, and obtains a first copy screen frame, the first copy screen frame being obtained by the application server copying the first adjusted screen frame to a blank screen frame, the resolution of the blank screen frame being a first resolution, and extracts the first adjusted screen frame from the first copy screen frame based on the horizontal reduction ratio and the vertical reduction ratio.

[0120] Preferably, the resolution adjustment module 23 adjusts the resolution of the first adjusted screen frame back to the first resolution, and when acquiring the second screen frame, performs super-resolution adjustment on the first adjusted screen frame based on the horizontal reduction ratio and the vertical reduction ratio, and determines the first adjusted screen frame that has undergone super-resolution adjustment as the second screen frame.

[0121] When decoding the screen frame code stream and obtaining the first copy screen frame, the code stream decoding module 22 obtains the screen frame immediately preceding the first screen frame in the application client as a reference screen frame for the first screen frame, performs screen reduction adjustment on the reference screen frame of the first screen frame, obtains a second adjusted screen frame, copies the second adjusted screen frame to a blank screen frame, obtains the second copy screen frame, decodes the screen frame code stream based on the second copy screen frame, and obtains the first copy screen frame.

[0122] According to one embodiment of the present invention, the steps in the data processing method shown in Fig. 11 are performed by each module in the data processing device 2 shown in Fig. 15. For example, step S301 shown in Fig. 11 may be performed by the codestream acquisition module 21 in Fig. 11, step S302 shown in Fig. 11 may be performed by the codestream decoding module 22 in Fig. 11, and step S303 shown in Fig. 11 may be performed by the resolution adjustment module 23 in Fig. 11.

[0123] In the present invention, when an application server obtains a first screen frame for an application client through encoding, it detects the data volume of the first screen frame. If the data volume of the first screen frame is equal to or greater than a data volume threshold, it performs screen reduction adjustment on the first screen frame to obtain a first adjusted screen frame. The resolution of the first screen frame is a first resolution, and the resolution of the first adjusted screen frame is a second resolution, which is lower than the first resolution. The application server then encodes the first adjusted screen frame to obtain a screen frame code stream for the first adjusted screen frame, and sends the screen frame code stream to the application client so that the application client displays a second screen frame based on the screen frame code stream. The second screen frame is obtained by the application client decoding the screen frame code stream to obtain the first adjusted screen frame, and then adjusting the resolution of the first adjusted screen frame back to the first resolution. As described above, according to the device of the present invention, the application server may perform screen reduction adjustment on a screen frame of a scene change (i.e., a first screen frame whose data amount is equal to or greater than a data amount threshold), encode the screen frame, obtain a corresponding code stream, and transmit the code stream to the application client. The application client may decode the obtained code stream, adjust the resolution of the decoded screen frame to the original first resolution, and then display the screen frame again. This ensures the image quality of the displayed screen frame (e.g., a second screen frame), reduces the data amount of the transmitted screen frame code stream, and improves the transmission speed of the screen frame code stream, thereby improving the smoothness of the application client's display of the screen frame of a scene change.

[0124] According to one embodiment of the present invention, each module of the data processing device 2 shown in FIG. 15 may be integrated into one or more units, or one of the units may be functionally subdivided into multiple subunits that perform similar operations without affecting the technical effect of the embodiment of the present invention. The modules are divided based on logical functions, and in actual applications, the functions of one module may be realized by multiple units, or the functions of multiple modules may be realized by one unit. In other embodiments of the present invention, the data processing device 1 may include other units, and in actual applications, these functions may be realized by other units or multiple units working together.

[0125] According to one embodiment of the present invention, the data processing device 2 shown in Fig. 15 may be configured by a general-purpose computing device such as a computer including a processing unit and a storage unit, such as a central processing unit (CPU), a random access memory (RAM), and a read-only memory (ROM), executing computer-readable instructions (including program code) that can execute the steps of the methods shown in Fig. 11, thereby realizing the data processing method of one embodiment of the present invention. The computer-readable instructions may, for example, be recorded on a computer-readable recording medium and loaded into the computing device via the computer-readable recording medium for execution.

[0126] FIG. 16 is a schematic diagram of an example of a configuration of a computer device according to the present invention. As shown in FIG. 16, the computer device 1000 may include one or more processors 1001, a network interface 1004, and a memory 1005. The computer device 1000 may also include a user interface 1003 and at least one communication bus 1002. The communication bus 1002 is used to realize communication between these components. The user interface 1003 may include a display and a keyboard, and preferably, the user interface 1003 may further include a standard wired interface and a wireless interface. The network interface 1004 may preferably include a standard wired interface and a wireless interface (e.g., a Wi-Fi interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The memory 1005 may preferably be at least one storage device located remotely from the one or more processors 1001. As shown in FIG. 16, memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a device control application program.

[0127] 16, the network interface 1004 can provide a network communication function. The user interface 1003 is an interface that is primarily used to provide input to a user. The one or more processors 1001 may be used to invoke a device control application stored in the memory 1005 to implement the following steps: when a first screen frame of an application client is obtained by encoding, detecting a data amount of the first screen frame; if the data amount of the first screen frame is equal to or greater than a data amount threshold, performing screen reduction adjustment on the first screen frame to obtain a first adjusted screen frame, wherein a resolution of the first screen frame is a first resolution and a resolution of the first adjusted screen frame is a second resolution, the second resolution being lower than the first resolution; encoding the first adjusted screen frame and obtaining a screen frame code stream of the first adjusted screen frame; and sending the screen frame code stream to the application client, so that the application client displays a second screen frame based on the screen frame code stream, wherein the second screen frame is obtained by adjusting the resolution of the first adjusted screen frame back to the first resolution after the application client decodes the screen frame code stream to obtain the first adjusted screen frame.

[0128] Preferably, the one or more processors 1001 may be used to invoke a device control application stored in the memory 1005 to implement the steps of: obtaining a screen frame code stream sent by an application server, wherein the screen frame code stream is obtained by encoding a first adjusted screen frame, and the first adjusted screen frame is obtained by performing screen reduction adjustment on a first screen frame of the application client, wherein the data amount of the first screen frame is equal to or greater than a data amount threshold, and the resolution of the first screen frame is a first resolution, and the resolution of the first adjusted screen frame is a second resolution, and the second resolution is lower than the first resolution; decoding the screen frame code stream to obtain the first adjusted screen frame; adjusting the resolution of the first adjusted screen frame back to the first resolution, obtaining a second screen frame, and displaying the second screen frame.

[0129] The computer device 1000 described in the embodiment of the present invention may execute the data processing method described in the corresponding embodiment of Fig. 3 or Fig. 11, or may realize the data processing device 1 described in the corresponding embodiment of Fig. 14, or may realize the data processing device 2 described in the corresponding embodiment of Fig. 15. Also, the description of the beneficial effects of adopting similar methods will be omitted.

[0130] Furthermore, a computer-readable storage medium is provided herein that stores a computer program executed by the data processing device 1 and the data processing device 2. The computer program includes computer-readable program instructions. When one or more processors execute the computer-readable program instructions, the data processing method described in the embodiment corresponding to FIG. 3 or FIG. 11 above can be executed, and therefore a description thereof will be omitted here. Furthermore, a description of the beneficial effects of employing a similar method will also be omitted. For technical details not disclosed in the embodiment of the computer storage medium according to the present invention, reference may be made to the description of the embodiment of the method according to the present invention.

[0131] As an example, the program instructions may be executed on a single computer device, multiple computer devices located at a single location, and / or multiple computer devices located at multiple locations and interconnected via a communications network, which may constitute a blockchain network.

[0132] The computer-readable storage medium may be a data processing device according to any of the above-described embodiments, or may be an internal storage device of a computer device, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of a computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, or a flash memory card. Furthermore, the computer-readable storage medium may include both an internal storage unit and an external storage device of a computer device. The computer-readable storage medium stores computer-readable instructions and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store output data or data to be output.

[0133] The present invention provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. One or more processors of a computer device can read the computer-readable instructions from the computer-readable storage medium, and the one or more processors can execute the computer-readable instructions to perform the above-described data processing method in the corresponding embodiment of FIG. 3 or FIG. 11 above, so a description thereof will be omitted here. Also, a description of the beneficial effects of adopting a similar method will be omitted. For technical details not disclosed in the embodiment of the computer-readable storage medium of the present invention, please refer to the description of the embodiment of the method of the present invention.

[0134] Terms such as "first," "second," and the like in the description, claims, and drawings of the embodiments of the present invention are used to distinguish different objects, not to describe a particular order. Also, the term "comprises" and any variations thereof are intended to cover an exclusive inclusion. For example, a process, method, apparatus, product, or device comprising a series of steps or units is not limited to the enumerated steps or modules, and may include unenumerated steps or modules, or other step units inherent to the process, method, apparatus, product, or device.

[0135] Those skilled in the art will appreciate that the various example elements and algorithm steps described in connection with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both, and that the above description has generally described the various example configurations and steps functionally to clearly illustrate the interchangeability of hardware and software. Whether these functions are performed in hardware or software depends on the specific application and design constraints. Those skilled in the art may use different methods to achieve the described functions for each specific application, but such implementations should not be considered beyond the scope of the present invention.

[0136] Methods and related apparatus according to embodiments of the present invention are described with reference to flowcharts and / or structural diagrams of the methods according to embodiments of the present invention. In particular, each flow and / or block of the flowcharts and / or structural diagrams, as well as combinations of the flows and / or blocks of the flowcharts and / or block diagrams, may be implemented by computer-readable instructions. These computer-readable instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate an apparatus such that the instructions, executed by the processor of the computer or other programmable data processing device, generate means for implementing the functions specified in one or more flow and / or structural diagram blocks. These computer-readable instructions may also be stored in a computer-readable memory that can cause the computer or other programmable data processing device to operate in a specific manner, such that the instructions stored in the computer-readable memory generate an article of manufacture including instruction means for implementing the functions specified in one or more flow and / or structural diagram blocks. These computer-readable instructions may be loaded into a computer or other programmable data processing apparatus such that a series of operational steps are performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions executing on the computer or other programmable apparatus provide steps for implementing the functions specified in one or more flows of the flowcharts and / or one or more blocks of the structural schematics.

[0137] The above disclosed contents are merely preferred embodiments of the present invention, and the scope of the present invention cannot be limited thereby, so that equivalent modifications made according to the claims of the present invention are within the scope of the present invention.

Claims

1. A data processing method executed by an application server, comprising: obtaining a unit transmission data amount and a unit transmission frame number for a screen frame of an application client; determining an average frame data amount based on the unit transmission data amount and the unit transmission frame number; determining a data amount threshold based on the average frame data amount; When a first screen frame to be transmitted to the application client is obtained by encoding, detecting a data amount of the first screen frame; When the data amount of the first screen frame is equal to or greater than the data amount threshold, performing screen reduction adjustment on the first screen frame to obtain a first adjusted screen frame, wherein a resolution of the first screen frame is a first resolution, a resolution of the first adjusted screen frame is a second resolution, and the second resolution is lower than the first resolution; encoding the first calibrated screen frame to obtain a screen frame codestream of the first calibrated screen frame; sending the screen frame code stream to the application client so that the application client displays a second screen frame based on the screen frame code stream, the second screen frame being obtained by the application client decoding the screen frame code stream to obtain the first adjusted screen frame, and then adjusting a resolution of the first adjusted screen frame back to the first resolution.

2. The step of performing a screen reduction adjustment on the first screen frame and obtaining a first adjusted screen frame includes: obtaining a horizontal reduction ratio and a vertical reduction ratio for the first screen frame; performing a screen reduction adjustment on the first screen frame based on the horizontal reduction ratio and the vertical reduction ratio to obtain the first adjusted screen frame.

3. performing screen reduction adjustment on the first screen frame based on the horizontal reduction ratio and the vertical reduction ratio to obtain the first adjusted screen frame, sampling pixel values ​​in the horizontal direction in the first screen frame based on the horizontal reduction ratio to obtain sampled pixel values ​​in the horizontal direction; sampling pixel values ​​in the vertical direction in the first screen frame based on the vertical reduction ratio to obtain sampled pixel values ​​in the vertical direction; and generating the first adjusted screen frame based on the horizontally sampled pixel values ​​and the vertically sampled pixel values.

4. The step of encoding the first adjusted screen frame and obtaining a screen frame codestream of the first adjusted screen frame includes: encoding the first calibrated screen frame to obtain an initial screen frame codestream of the first calibrated screen frame; and adding the horizontal reduction ratio and the vertical reduction ratio to the initial screen frame code stream to obtain the screen frame code stream.

5. The step of encoding the first adjusted screen frame and obtaining a screen frame codestream of the first adjusted screen frame includes: obtaining a blank screen frame having a resolution of the first resolution; copying the first adjusted screen frame onto the blank screen frame to obtain a first copied screen frame; encoding the first copy screen frame to obtain the screen frame codestream.

6. The step of encoding the first copy screen frame and obtaining the screen frame codestream includes: obtaining a screen frame of the application client immediately before the first screen frame as a reference screen frame for the first screen frame; performing a screen reduction adjustment on a reference screen frame of the first screen frame to obtain a second adjusted screen frame; copying the second adjusted screen frame onto the blank screen frame to obtain a second copied screen frame; and encoding the first copy screen frame based on the second copy screen frame to obtain the screen frame codestream.

7. the application client is a game client; When the data amount of the first screen frame is equal to or greater than a data amount threshold, performing screen reduction adjustment on the first screen frame and obtaining a first adjusted screen frame includes:

2. The method of claim 1, further comprising: when the first screen frame belongs to a game screen frame at a scene change of the game client and the data amount of the first screen frame is equal to or greater than the data amount threshold, performing a screen reduction adjustment on the first screen frame to obtain the first adjusted screen frame.

8. The method of claim 1 , wherein the data amount threshold is determined to be three times the average frame data amount.

9. performing super-resolution adjustment on the first adjustment screen frame to obtain the second screen frame; 2. The method of claim 1, further comprising: using the second screen frame as a reference screen frame of a screen frame immediately following the first screen frame, wherein the reference screen frame of the screen frame immediately following the first screen frame is used to encode the screen frame immediately following the first screen frame.

10. A data processing method executed by an application client and an application server, comprising: The application server obtaining a unit transmission data amount and a unit transmission frame number for a screen frame of the application client; determining an average frame data amount based on the unit transmission data amount and the unit transmission frame number; determining a data amount threshold based on the average frame data amount; When a first screen frame to be transmitted to the application client is obtained by encoding, detecting a data amount of the first screen frame; When the data amount of the first screen frame is equal to or greater than the data amount threshold, performing screen reduction adjustment on the first screen frame to obtain a first adjusted screen frame, wherein a resolution of the first screen frame is a first resolution, a resolution of the first adjusted screen frame is a second resolution, and the second resolution is lower than the first resolution; encoding the first calibrated screen frame to obtain a screen frame codestream of the first calibrated screen frame; sending the screen frame code stream to the application client so that the application client displays a second screen frame based on the screen frame code stream, the second screen frame being obtained by the application client decoding the screen frame code stream to obtain the first adjusted screen frame and then adjusting a resolution of the first adjusted screen frame back to the first resolution; The application client: obtaining a screen frame codestream sent by the application server; decoding the screen frame codestream to obtain the first adjusted screen frame; adjusting a resolution of the first adjusted screen frame back to the first resolution, obtaining a second screen frame, and displaying the second screen frame.

11. the screen frame code stream includes a horizontal reduction ratio and a vertical reduction ratio for the first screen frame when performing screen reduction adjustment on the first screen frame, The step of decoding the screen frame codestream to obtain the first adjusted screen frame includes: analyzing the horizontal reduction ratio and the vertical reduction ratio from the screen frame codestream; decoding the screen frame codestream to obtain a first copy screen frame, the first copy screen frame being obtained by the application server copying the first adjusted screen frame to a blank screen frame, and a resolution of the blank screen frame being the first resolution; and extracting the first adjusted screen frame from the first copy screen frame based on the horizontal reduction ratio and the vertical reduction ratio.

12. adjusting the resolution of the first adjusted screen frame back to the first resolution and obtaining a second screen frame; performing super-resolution adjustment on the first adjustment screen frame based on the horizontal reduction ratio and the vertical reduction ratio; and determining the first adjusted screen frame, after super-resolution adjustment, as the second screen frame.

13. The step of decoding the screen frame codestream to obtain a first copy screen frame includes: obtaining a screen frame immediately before the first screen frame in the application client as a reference screen frame for the first screen frame; performing a screen reduction adjustment on a reference screen frame of the first screen frame to obtain a second adjusted screen frame; copying the second adjusted screen frame onto the blank screen frame to obtain a second copied screen frame; and decoding the screen frame codestream based on the second copy screen frame to obtain the first copy screen frame.

14. 1. A data processing device, comprising: a screen encoding module for detecting a data amount of a first screen frame when the first screen frame is obtained by encoding and to be transmitted to the application client; a reduction module that, when the data amount of the first screen frame is equal to or greater than a data amount threshold, performs screen reduction adjustment on the first screen frame and obtains a first adjusted screen frame, wherein a resolution of the first screen frame is a first resolution, a resolution of the first adjusted screen frame is a second resolution, and the second resolution is lower than the first resolution; a codestream encoding module for encoding the first adjusted screen frame to obtain a screen frame codestream of the first adjusted screen frame; a transmitting module that transmits the screen frame code stream to the application client so that the application client displays a second screen frame based on the screen frame code stream, the second screen frame being obtained by the application client decoding the screen frame code stream to obtain the first adjusted screen frame and then adjusting a resolution of the first adjusted screen frame back to the first resolution; Acquire a unit transmission data amount and a unit transmission frame number for a screen frame of the application client; determining an average frame data amount based on the unit transmission data amount and the unit transmission frame number; The apparatus determines the data volume threshold based on the average frame data volume.

15. A data processing system including an application server and an application client, The application server a screen encoding module for detecting a data amount of a first screen frame when the first screen frame to be transmitted to the application client is obtained through encoding; a reduction module that, when the data amount of the first screen frame is equal to or greater than a data amount threshold, performs screen reduction adjustment on the first screen frame and obtains a first adjusted screen frame, wherein a resolution of the first screen frame is a first resolution, a resolution of the first adjusted screen frame is a second resolution, and the second resolution is lower than the first resolution; a codestream encoding module for encoding the first adjusted screen frame to obtain a screen frame codestream of the first adjusted screen frame; a transmitting module that transmits the screen frame code stream to the application client so that the application client displays a second screen frame based on the screen frame code stream, the second screen frame being obtained by the application client decoding the screen frame code stream to obtain the first adjusted screen frame and then adjusting a resolution of the first adjusted screen frame back to the first resolution; The application server Acquire a unit transmission data amount and a unit transmission frame number for a screen frame of the application client; determining an average frame data amount based on the unit transmission data amount and the unit transmission frame number; determining the data amount threshold based on the average frame data amount; The application client: a codestream acquisition module for acquiring the screen frame codestream sent by the application server; a codestream decoding module for decoding the screen frame codestream to obtain the first adjusted screen frame; a resolution adjustment module that adjusts a resolution of the first adjusted screen frame back to the first resolution, obtains a second screen frame, and displays the second screen frame.

16. 10. A computing device comprising a memory having computer-readable instructions stored thereon and one or more processors, the computer-readable instructions, when executed by the one or more processors, causing the one or more processors to perform the steps of the method of any one of claims 1 to 9.

17. A computer program comprising computer readable instructions which, when executed by one or more processors, implement the steps of the method of any of claims 1 to 9.

Images