Image processing method, apparatus and device

Abstract

The application discloses an image processing method, device and equipment, which can be applied to various scenes such as image processing, games and game basic technology. The method comprises the following steps: performing one-time encoding on a current frame image to obtain an encoding parameter, wherein the encoding parameter indicates whether there is scene switching in the current frame image; determining whether to re-encode the current frame image according to the encoding parameter; if yes, re-encoding the current frame image, and the data amount of the current frame image after re-encoding is less than that of the current frame image after one-time encoding. In the embodiment of the application, whether to re-encode the current frame image is determined through the encoding parameter of the current frame image. In the embodiment of the application, whether to re-encode the current frame image is determined through the encoding parameter, for example, when the encoding parameter indicates that there is scene switching after encoding, it is determined to re-encode the current frame image, so as to reduce the data amount of the current frame image, and the frame with large data amount can be effectively avoided, and the stability of the code rate is maintained.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to an image processing method, apparatus, and device. Background Technology

[0002] Currently, the terminal receives the bitstream (i.e., the encoded data stream) sent from the cloud in real time. However, due to the significant differences in the content of different frames, the amount of data in the bitstream of different frames may fluctuate greatly, thus affecting the stability of the bitrate. Summary of the Invention

[0003] This application provides an image processing method, apparatus, and device that can effectively avoid frames with large data volumes and maintain stable bitrate.

[0004] On one hand, an image processing method is provided, the method comprising encoding a current frame image once to obtain encoding parameters, the encoding parameters indicating whether the current frame image has a scene change; determining whether to re-encode the current frame image according to the encoding parameters; if so, re-encoding the current frame image, wherein the data volume of the re-encoded current frame image is less than the data volume of the current frame image after the first encoding.

[0005] On the other hand, an image processing apparatus is provided, the apparatus including a first encoding module, a first determining module, and a second encoding module. The first encoding module is used to encode a current frame image once to obtain encoding parameters, the encoding parameters indicating whether a scene change exists in the current frame image; the first determining module is used to determine whether to re-encode the current frame image based on the encoding parameters; the second encoding module is used to re-encode the current frame image when it is determined that re-encoding is necessary, wherein the data size of the re-encoded current frame image is less than the data size of the current frame image after the first encoding.

[0006] On the other hand, a computer-readable storage medium is provided that stores a computer program adapted for loading by a processor to perform the steps in the image processing method as described in any of the above embodiments.

[0007] On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory storing a computer program, the processor executing the steps of the image processing method as described in any of the above embodiments by calling the computer program stored in the memory.

[0008] This application embodiment determines whether to re-encode the current frame image by obtaining the encoding parameters of the current frame image, thereby reducing the data volume of the current frame image through re-encoding. This application embodiment determines whether to re-encode the current frame image by using encoding parameters. For example, if the encoding parameters indicate that there is a scene change after encoding, it determines to re-encode the current frame image to reduce the data volume of the current frame image, which can effectively avoid frames with large data volumes and maintain stable bitrate. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic diagram illustrating an application scenario provided in an embodiment of this application.

[0011] Figure 2 This is a schematic diagram of the structure of the image processing system provided in an embodiment of this application.

[0012] Figures 3 to 13 This is a schematic flowchart of the image processing method provided in an embodiment of this application.

[0013] Figure 14 This is a schematic diagram of the structure of the image processing apparatus provided in the embodiments of this application.

[0014] Figure 15 A schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0015] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0016] This application provides an image processing method, apparatus, computer device, and storage medium. Specifically, the image processing method of this application can be executed by a computer device, which can be a terminal or a server. The terminal can be a smartphone, tablet, laptop, desktop computer, smart TV, smart speaker, wearable smart device, smart vehicle terminal, etc. The terminal can also include a client, which can be a cloud gaming client, a client applet, a video client, a browser client, or an instant messaging client, etc. The server can be an independent physical server, a server cluster composed of multiple physical servers, or a distributed system. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0017] The embodiments of this application can be applied to various scenarios such as image processing, games, and game-related technologies.

[0018] For example, when this method runs on a server, it can be cloud gaming. Cloud gaming refers to a gaming method based on cloud computing. In the cloud gaming operating mode, the main body running the game application and the main body displaying the game screen are separated. The storage and execution of the method are completed on the cloud gaming server. The display of the game screen is completed on the cloud gaming client. The cloud gaming client is mainly used for receiving and sending game data and displaying the game screen. For example, the cloud gaming client can be a display device with data transmission capabilities close to the user, such as a mobile terminal, television, computer, PDA, personal digital assistant, etc., but the terminal device for processing game data is the cloud gaming server in the cloud. When playing the game, the user operates the cloud gaming client to send operation commands to the cloud gaming server. The cloud gaming server runs the game according to the operation commands, encodes and compresses the game screen and other data, returns it to the cloud gaming client through the network, and finally, the cloud gaming client decodes and outputs the game screen.

[0019] First, some of the nouns or terms that appear in the description of the embodiments of this application are explained as follows:

[0020] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Based on the cloud computing business model, cloud technology encompasses network technology, information technology, integration technology, management platform technology, and application technology. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0021] Cloud gaming, also known as gaming on demand, is an online gaming technology based on cloud computing. It enables thin clients with relatively limited graphics processing and data processing capabilities to run high-quality games. In cloud gaming, the game does not reside on the player's terminal but runs on a cloud server. The cloud server renders the game scene as a video and audio stream, which is then transmitted to the player's terminal via the network. The player's terminal does not need powerful graphics processing and data processing capabilities; it only needs basic streaming media playback capabilities and the ability to receive player input commands and send them to the cloud server.

[0022] Video encoding: The method of converting a file in an original video format into a file in another video format through compression technology. The converted data can be called a bitstream.

[0023] Video decoding: the reverse process of video encoding.

[0024] IDR frame: Instantaneous Decoding Refresh Frame (IDR) is a type of encoded frame defined in video coding technology. IDR frames use only intra-frame predictive coding, and the decoder can independently decode the content of the IDR frame without information from other frames. IDR frames are generally used as reference frames for subsequent frames and also as entry points for bitstream switching.

[0025] Intra-frame prediction: Predicts the current pixel using neighboring encoded pixels within the same frame, without referencing the encoded image.

[0026] Inter-frame prediction: Predicts the current pixel using neighboring encoded image pixels, requiring reference to the encoded image.

[0027] Quantization parameters: Parameters used in the quantization step of video encoding. The higher the quantization level, the higher the compression ratio, but the image quality will decrease, and vice versa.

[0028] A cloud server is a server that runs games in the cloud and has functions such as video enhancement (pre-encoding processing) and video encoding.

[0029] A terminal refers to a type of device that has rich human-computer interaction methods, internet access capabilities, typically runs various operating systems, and possesses strong processing power. Terminals include smartphones, living room TVs, tablets, in-vehicle terminals, handheld game consoles, etc.

[0030] To enable better collaboration and joint optimization between cloud servers and terminals, video encoding configurations and computational tasks such as video image processing and video content analysis can be rationally allocated based on the terminal's hardware capabilities and real-time performance during cloud gaming. This further enhances the cloud gaming visual experience within limited cloud server resources.

[0031] Video encoding collaboration: Based on the terminal's encoding and decoding capabilities, and in combination with the game type and user network type, select the optimal encoding and decoding configuration and strategy.

[0032] Video rendering collaboration: Based on the terminal's graphics processing capabilities, video rendering tasks are rationally divided to enable effective collaboration between the cloud server and the terminal, thereby improving video quality. This includes collaboration in rendering regions, rendering tasks, and video analysis and processing.

[0033] Terminal status coordination: Based on the real-time performance of the terminal, dynamically adjust the encoding coordination tasks and rendering coordination strategies of the cloud server and the terminal to ensure the best user experience in real time.

[0034] The cloud server and terminal collaborative architecture mainly includes: cloud server, terminal-cloud collaborative strategy, terminal-cloud collaborative protocol, terminal collaborative interface, and software and hardware collaborative module.

[0035] The edge-cloud collaboration strategy includes: video encoding collaboration strategy, video rendering collaboration strategy, and terminal status collaboration strategy. The cloud server formulates the optimal video encoding and rendering collaboration strategy based on the device capabilities reported by the terminal, combined with the game type and the user's network environment. Simultaneously, the cloud server dynamically adjusts the edge-cloud collaboration strategy by acquiring real-time terminal performance data through terminal status collaboration.

[0036] The edge-cloud collaboration protocol refers to a unified protocol for data interaction between cloud servers and terminals.

[0037] The terminal collaboration interface refers to the interface between the terminal's software and hardware modules. Through this interface, one can effectively interact with the terminal, configure video encoding and rendering parameters, and obtain real-time hardware performance.

[0038] The decoding protocols include video codec protocols such as H.264, H.265, and AV1, as well as the Profile and Level supported by the terminal under each codec protocol. Decoding performance refers to the highest supported decoding frame rate and single-frame decoding latency for a given video size under a specific decoding protocol.

[0039] The video sizes are defined as follows: 360p, 576p, 720p, 1080p, 2k, 4k. The video frame rates are defined as follows: 30fps, 40fps, 50fps, 60fps, 90fps, 120fps.

[0040] The decoding performance supported by the terminal is given in the form of a triple. The first element is an enumerated definition of the video resolution, the second element is an enumerated definition of the video frame rate, and the third element is the single-frame decoding latency under the video resolution and video frame rate. For example, the single-frame decoding latency of device A's H264 decoding at 720p@60fps is 10ms.

[0041] The cloud server determines the set of encoding functions that need to be enabled based on the game type and network conditions, and then determines the optimal encoding configuration for the current device based on the device type and encoding capabilities reported by the terminal.

[0042] The data structure requirements for terminal decoding capabilities are shown in Table 1:

[0043] Table 1. Data Structure Requirements for Terminal Decoding Capability

[0044]

[0045] Based on the terminal's decoding capabilities, and combined with the game type and network communication information, the cloud server determines the optimal decoding protocol, decoding resolution, video frame rate, and other encoding and decoding configurations for the current device, as well as the number of video encoding reference frames and SVC enabling encoding and decoding strategies.

[0046] like Figure 1 The diagram shows a collaborative optimization connection process for video rendering.

[0047] 1. The cloud server initiates a request to the terminal through the START client (i.e., the cloud gaming client) to obtain the terminal's decoding capability information. The request protocol fields include the protocol version number and the specific encoding / decoding protocol query.

[0048] 2. When the terminal receives a capability information retrieval request, it returns a status flag (0 for success, a specific error code for failure), the supported protocol version number, and the terminal device capability information. If the terminal only supports some decoding protocols, it returns the supported decoding protocol information; if the terminal does not support any decoding protocols, it returns codecs=0; if the terminal capability information request fails, it returns the specific error code.

[0049] 3. After receiving the terminal's decoding capability information, the cloud server, in conjunction with the game type and network communication information, determines the optimal decoding protocol, decoding resolution, video frame rate, and other encoding and decoding configurations for the current terminal device, as well as the number of video encoding reference frames and SVC enabling encoding and decoding strategies.

[0050] 4. After receiving the optimal decoding configuration, the terminal decodes the video stream.

[0051] Among them, the START client is a cloud gaming client that is installed on the terminal.

[0052] The following is an example of a frequency coding collaborative optimization connection protocol:

[0053] 1. Video decoding capability request:

[0054]

[0055] 2. Video decoding capability response (supports all query decoding protocols):

[0056]

[0057] 3. Video decoding capability response (only supports partial query decoding protocols):

[0058]

[0059]

[0060] 4. Video decoding capability request (decoding protocol not supported):

[0061]

[0062] 5. Video decoding capability request (protocol request failed):

[0063]

[0064] Currently, cloud gaming providers primarily rely on hardware encoders to significantly reduce encoding time before transmitting the data to the terminal for playback. While hardware encoders offer extremely high encoding speeds, they currently lack scene transition detection. Furthermore, when scene transitions occur, hardware encoders typically insert a large frame, leading to a decrease in the stability of the transmission bitrate.

[0065] This application proposes an image processing method that can determine whether to re-encode the current frame image based on encoding parameters. For example, when a scene change occurs and the data volume of the current frame image becomes large, it can be determined to re-encode the current frame image to reduce the data volume of the current frame image. This can effectively avoid frames with large data volumes and maintain the stability of the bit rate.

[0066] Please refer to Figure 2 , Figure 2 This is a schematic diagram of the structure of an image processing system provided in an embodiment of this application. The image processing system includes a terminal 10 and a server 20, etc.; the terminal 10 and the server 20 are connected via a network, such as a wired or wireless network.

[0067] Terminal 10 can be used to display a graphical user interface (GUI). Terminal 10 is used to interact with the user through the GUI, such as downloading and installing a client, running a mini-program, or accessing a website. In this embodiment, terminal 10 can be a device used by the user to receive the bitstream transmitted from server 20 to display game footage. Server 20 transmits the encoded bitstream to terminal 10, which receives and decodes the bitstream, then plays video, displays game footage, etc., based on the decoded video data.

[0068] In this embodiment of the application, when encoding, the server 20 may specifically be used to: encode the current frame image to obtain encoding parameters; determine whether to re-encode the current frame image based on the encoding parameters; if so, re-encode the current frame image to reduce the data volume of the current frame image.

[0069] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the priority of the embodiments.

[0070] This application provides an image processing method, which can be executed by terminal 10 or server 20, or by both terminal 10 and server 20. This application uses the example of the image processing method being executed by server 20 for illustration.

[0071] Please see Figure 3 Image processing methods include:

[0072] Step 210: Encode the current frame image once to obtain encoding parameters. The encoding parameters indicate whether there is a scene change in the current frame image.

[0073] Server 20 can acquire multiple consecutive frames of raw images, where raw images refer to unencoded images. Server 20 can then encode the raw images, and the raw image currently being encoded becomes the current frame image, thereby obtaining the encoding parameters of the current frame image. The encoding parameters can be used to indicate whether there is a scene transition in the current frame image.

[0074] Optionally, the encoding parameters include the image size, which represents the amount of data in the bitstream after encoding the current frame image. The larger the image size, the larger the amount of data after encoding the current frame image.

[0075] Optionally, the encoding parameters may also include the proportion of intra-frame predicted pixels. It can be understood that the larger the proportion of intra-frame predicted pixels, the greater the probability of scene switching in the current frame image.

[0076] Step 220: Determine whether to re-encode the current frame image based on the encoding parameters.

[0077] After obtaining the encoding parameters of the current frame image, it can be determined whether to re-encode the current frame image based on the encoding parameters of the current frame image.

[0078] Optionally, in order to ensure the stability of the bit rate (i.e., the amount of data used per unit time), if the image size after encoding the current frame image is too large, it will lead to an increase in the bit rate required for transmission, affecting the stability of the bit rate.

[0079] Therefore, server 20 can determine whether the size of the encoded image of the current frame is greater than a size threshold. If it is, it indicates that re-encoding is required to reduce the image size, thereby reducing the amount of data in the encoded image of the current frame. The encoding parameters may also include the current encoding frame rate and the current encoding bitrate. The size threshold can be determined based on the current encoding frame rate and the current encoding bitrate, such as size threshold = current encoding bitrate / current encoding frame rate. This involves comparing the size threshold with the average amount of data allocated per frame. Therefore, if the size of the encoded image of the current frame is greater than the size threshold, it is determined that the current frame needs to be re-encoded to reduce the amount of data occupied by the current frame and ensure a stable bitrate.

[0080] Optionally, when there is a scene change, the number of encoded images of the current frame is generally large. Therefore, when there is a scene change, it can be determined that the current frame image needs to be re-encoded.

[0081] Server 20 can determine whether the proportion of intra-frame predicted pixels after encoding the current frame image is greater than a preset proportion, such as 70%, 80%, 85%, 89%, 95%, etc. If the proportion of intra-frame predicted pixels is greater than the preset proportion, it can be determined that there is a scene change in the current frame image, and server 20 determines that the current frame image needs to be re-encoded to reduce the amount of data after encoding the current frame image.

[0082] Optionally, such as Figure 4 and 5 As shown, step 220 can also be: when the current frame image is not decoded and refreshed as an IDR frame in real time, determine whether to re-encode the current frame image based on the encoding parameters.

[0083] When the current frame is not an IDR frame, it can be disregarded as a reference image for subsequent frames, and re-encoding it will not affect the encoding of subsequent frames. Therefore, re-encoding of the current frame is only determined based on the encoding parameters when the current frame is not an IDR frame that is decoded and refreshed in real time.

[0084] Step 230: If it is determined that the current frame image needs to be re-encoded, then the current frame image is re-encoded. The amount of data in the re-encoded current frame image is less than the amount of data in the current frame image after the first encoding.

[0085] After determining that the current frame image needs to be re-encoded, the server 20 can re-encode the current frame image. The amount of data in the re-encoded current frame image is less than the amount of data in the current frame image before re-encoding, thereby reducing the amount of data required to transmit the current frame image.

[0086] Optionally, the server 20 can re-encode the current frame image by adjusting the average quantization parameter in the encoding parameters. The larger the average quantization parameter is set, the smaller the amount of data after re-encoding the current frame image. Therefore, in situations where the current frame image may have a large amount of data after encoding due to scene switching or a large image size, the current frame image can be re-encoded with a larger average quantization parameter (such as adding a predetermined value greater than 0 to the average quantization parameter of the current frame image's encoding parameters). This reduces the amount of data after encoding the current frame image, prevents bitrate fluctuations, and ensures bitrate stability. For cloud gaming scenarios, stable bitrate is beneficial to improving the gaming experience.

[0087] Optionally, such as Figure 4 As shown, the image processing method also includes:

[0088] Step 240: When the current frame image is an Instantaneous Decoding Refresh Frame (IDR), determine the encoding parameters of the Instantaneous Decoding Refresh Frame as reference encoding parameters, and transmit the Instantaneous Decoding Refresh Frame.

[0089] When the current frame is an IDR frame, it may be the first frame of a Group of Pictures (GOP), and its encoding information will be used as reference information for determining subsequent IDR frames, so there is no need to re-encode it. Therefore, even if the current frame image is large or there is a scene change, the IDR frame will not be re-encoded when it is an IDR frame; instead, it will be transmitted directly to ensure encoding efficiency.

[0090] Optionally, such as Figure 5 As shown, the image processing method also includes:

[0091] Step 250: The re-encoded current frame image is determined as an instant-decoded refresh IDR frame and transmitted.

[0092] It's understandable that re-encoding the current frame indicates a potential scene change, and subsequent images may contain content from that changed scene. Therefore, using the re-encoded current frame as an IDR frame, and then using it as a reference frame for encoding subsequent images, can improve the encoding accuracy of those images.

[0093] To better illustrate the image processing method provided in the embodiments of this application, please refer again. Figures 3 to 6 The image processing method provided in this application embodiment can be summarized into the following steps:

[0094] Step 510: Encode the current frame image to obtain encoding parameters.

[0095] Step 520: Determine whether the current frame image is an instant-decoded refresh IDR frame.

[0096] Step 530: If the current frame image is not decoded and refreshed as an IDR frame in real time, determine whether to re-encode the current frame image.

[0097] Step 540: When the current frame image is an IDR frame that is decoded and refreshed in real time, the average quantization parameter of the IDR frame is determined as the reference quantization parameter, and the image size of the IDR frame is determined as the reference image size.

[0098] Step 550: Reset the average quantization parameters, re-encode the current frame image to obtain the bitstream, and determine the current frame image as an instantaneous decoded refresh IDR frame.

[0099] Step 560: Transmit the bitstream.

[0100] Wherein, step 210 can be referred to in the description of step 210, steps 520 and 530 can be referred to in the description of steps 220 and 240, step 540 can be referred to in the description of steps 230 and 250, step 550 can be referred to in the description of step 240, and step 560 can be referred to in steps 240 and 250.

[0101] Optional, such as Figure 7 As shown, step 220 can be achieved through steps 221 to 222, specifically as follows:

[0102] Step 221: When the proportion of predicted pixels within a frame is greater than a preset proportion, it is determined that there is a scene change in the current frame image.

[0103] Optionally, the encoding parameters include the proportion of intra-frame predicted pixels. After the server 20 encodes the current frame image, the proportion of intra-frame predicted pixels in the current frame image can be obtained. Then, the server 20 compares the proportion of intra-frame predicted pixels with a preset proportion. If the proportion of intra-frame predicted pixels is greater than the preset proportion, it means that most of the pixels in the current frame image are obtained through intra-frame prediction. This is likely because a scene switch has caused the difference between the current frame image and the previous frame or the previous IDR frame to be too large, making accurate inter-frame prediction impossible. Therefore, it is necessary to fill pixels based on intra-frame prediction.

[0104] Therefore, when the proportion of predicted pixels within a frame is greater than a preset proportion, server 20 can determine that there is a scene change in the current frame image; conversely, when the proportion of predicted pixels within a frame is less than or equal to the preset proportion, server 20 can determine that there is no scene change in the current frame image. Alternatively, when the proportion of predicted pixels within a frame is greater than or equal to the preset proportion, server 20 can determine that there is a scene change in the current frame image; or, when the proportion of predicted pixels within a frame is less than the preset proportion, server 20 can determine that there is no scene change in the current frame image.

[0105] Step 222: When there is a scene change in the current frame image, determine to re-encode the current frame image.

[0106] When a scene change occurs in the current frame image, current hardware encoders typically generate a large frame after encoding the current frame image, leading to a decrease in bitrate stability. Therefore, when a scene change is determined to occur in the current frame image, it is necessary to re-encode the current frame image to reduce the amount of data after encoding.

[0107] Optionally, such as Figure 8 As shown, step 220 can be achieved through steps 223 to 226, specifically as follows:

[0108] Step 223: When the proportion of predicted pixels within a frame is greater than a preset proportion, it is determined that there is a scene change in the current frame image.

[0109] The implementation of step 223 is basically the same as that of step 221. Please refer to the description of step 221, and it will not be repeated here.

[0110] Step 224: Determine the size threshold based on the current encoding frame rate and the current encoding bit rate.

[0111] When it is determined that there is a scene change in the current frame image, it is necessary to further determine whether the size of the encoded image of the current frame image is too large.

[0112] Encoding parameters may also include the proportion of intra-frame predicted pixels, image size, current encoding frame rate, and current encoding bit rate.

[0113] Server 20 can first determine a size threshold corresponding to the current frame image by using the current encoding frame rate and the current encoding bitrate, such as size threshold = current encoding bitrate / current encoding frame rate.

[0114] Of course, the size threshold calculated by dividing the current encoding bitrate by the current encoding frame rate is basically the same as the average image size per frame. Therefore, the size threshold can be increased, such as size threshold = 1.5 * current encoding bitrate / current encoding frame rate. If the encoded image size of the current frame is greater than 1.5 times the average image size (i.e., current encoding bitrate / current encoding frame rate), the encoded image size of the current frame is determined to be too large; or, if the encoded image size of the current frame is greater than or equal to 1.5 times the average image size, the encoded image size of the current frame is determined to be too large. The size threshold can also be other multiples of the average image size, such as 1.2 times, 1.3 times, 1.4 times, etc., which can be set according to the requirements for the stability of bitrate control.

[0115] In this way, by comparing the size threshold corresponding to the current frame image with the size of the encoded image of the current frame image, it is possible to more accurately determine whether the size of the encoded image of the current frame image is too large.

[0116] Step 225: When there is a scene change in the current frame image and the image size is greater than the size threshold, determine to re-encode the current frame image.

[0117] When the current frame image undergoes a scene change and the image size is greater than (or greater than or equal to) the size threshold, it indicates that the current frame image has not only undergone a scene change, but the scene change has also caused the encoded image size to become larger. It can be determined that the current frame image needs to be re-encoded to reduce the amount of data after encoding.

[0118] Step 226: If there is no scene change in the current frame image, or if there is a scene change in the current frame image and the image size of the current frame image is less than the size threshold, determine that the current frame image will not be re-encoded.

[0119] If there is no scene change in the current frame image, or if there is a scene change in the current frame image but the size of the encoded image of the current frame image is smaller than (or less than or equal to) the size threshold, it means that the amount of data after encoding the current frame image is not large, and there is no need to re-encode to ensure the stability of the bit rate.

[0120] To better illustrate the image processing method provided in the embodiments of this application, please refer again. Figures 7 to 9 The image processing method provided in this application embodiment can be summarized into the following steps:

[0121] Step 810: Encode the current frame image to obtain encoding parameters.

[0122] Step 820: Is the proportion of intra-frame predicted pixels greater than a preset proportion?

[0123] Step 830: Is the image size greater than the size threshold?

[0124] Step 840: When the proportion of predicted pixels within the frame is greater than a preset proportion and the image size is greater than a size threshold, it is determined that re-encoding is required.

[0125] Step 850: When the proportion of predicted pixels within the frame is less than a preset proportion, or when the proportion of predicted pixels within the frame is greater than a preset proportion and the image size of the current frame image is less than a size threshold, no re-encoding is required.

[0126] In this context, step 810 can refer to the description of step 210, step 820 can refer to the description of step 223, step 830 can refer to the descriptions of steps 224 and 225, step 840 can refer to the description of step 225, and step 850 can refer to the description of step 226.

[0127] Optionally, such as Figure 10 As shown, step 230 can be implemented through steps 231 to 233, specifically as follows:

[0128] Step 231: When the average quantization parameter is greater than the reference average quantization parameter, adjust the average quantization parameter according to the first adjustment value.

[0129] The reference average quantization parameter is determined based on the average quantization parameter of the IDR frames that were instantaneously decoded and refreshed before the current frame image.

[0130] For example, the encoding parameters also include the average quantization parameter. When re-encoding the current frame image, it can be first determined whether the average quantization parameter of the current frame image after encoding is greater than (or greater than or equal to) the reference average quantization parameter.

[0131] For example, if the average quantization parameter of the encoded current frame image is greater than (or greater than or equal to) the reference average quantization parameter, the average quantization parameter of the encoded current frame image can be adjusted according to the first adjustment value. After adjustment according to the first adjustment value, the average quantization parameter of the encoded current frame image increases by the first adjustment value, resulting in a higher compression ratio for the encoded current frame image, thereby reducing the amount of data after encoding the current frame image.

[0132] The reference average quantization parameter is determined based on the average quantization parameter of the previously encoded, real-time decoded, refreshed IDR frames. For example, the reference average quantization parameter is equal to the average quantization parameter of the previous IDR frame. Determining the reference average quantization parameter based on the average quantization parameter of the previous IDR frame improves the accuracy of its determination.

[0133] Since a large average quantization parameter has already been used when encoding the current frame image, in order to prevent the compression rate of the current frame image from being too large after re-encoding, which would cause the current frame image to be distorted, the first adjustment value can be set to a small value, such as 1 or 2.

[0134] Step 232: When the average quantization parameter is less than the reference average quantization parameter, adjust the average quantization parameter according to the second adjustment value, where the second adjustment value is greater than the first adjustment value.

[0135] For example, if the average quantization parameter of the encoded current frame image is less than (or less than or equal to) the reference average quantization parameter, then the average quantization parameter of the encoded current frame image can be adjusted according to the second adjustment value. After adjustment according to the second adjustment value, the average quantization parameter of the encoded current frame image increases by the second adjustment value, resulting in a higher compression ratio for the encoded current frame image, thereby reducing the amount of data in the encoded current frame image.

[0136] Since a relatively small average quantization parameter was used when encoding the current frame image, in order to prevent the data volume of the current frame image from still being too large after re-encoding, the second adjustment value can be set to a larger value, such as 3, 4, 5, etc., so as to ensure that the data volume of the current frame image after re-encoding is not too large, thus ensuring the stability of the bit rate.

[0137] Step 233: Re-encode the current frame image according to the adjusted average quantization parameters.

[0138] Finally, server 20 re-encodes the current frame image based on the adjusted average quantization parameters, thereby minimizing the amount of data in the re-encoded current frame image while ensuring that the re-encoded current frame image is not distorted, thus ensuring the stability of the bitrate.

[0139] Optionally, such as Figure 11 As shown, step 230 can also be implemented through steps 234 to 236, specifically as follows:

[0140] Step 234: When the image size is smaller than the reference image size, adjust the average quantization parameter according to the first adjustment value.

[0141] The reference image size is determined based on the image size of the IDR frame that was instantaneously decoded and refreshed before the current frame image.

[0142] For example, the encoding parameters also include the image size. When re-encoding the current frame image, it can be first determined whether the encoded image size of the current frame image is smaller than (or, smaller than or equal to) the reference image size.

[0143] For example, if the size of the encoded image of the current frame is smaller than (or less than or equal to) the size of the reference image, the average quantization parameter of the encoded image of the current frame can be adjusted according to the first adjustment value. After adjustment according to the first adjustment value, the average quantization parameter of the encoded image of the current frame increases by the first adjustment value, resulting in a higher compression ratio of the encoded image of the current frame, thereby reducing the amount of data after encoding the image of the current frame.

[0144] The reference image size is determined based on the image size of the IDR frame that was previously encoded and then immediately decoded and refreshed before the current frame. For example, the reference image size is equal to the image size of the previous IDR frame preceding the current frame. Determining the reference image size based on the image size of the previous IDR frame improves the accuracy of the reference image size determination.

[0145] Since the image size after encoding the current frame is already small, in order to prevent the compression rate of the current frame image from being too high after re-encoding, which would cause the current frame image to be distorted, the first adjustment value can be set to a small value, such as 1 or 2.

[0146] Step 235: When the image size is larger than the reference image size, adjust the average quantization parameter according to the second adjustment value, which is greater than the first adjustment value.

[0147] For example, if the size of the encoded image of the current frame is greater than (or greater than or equal to) the size of the reference image, then the average quantization parameter of the encoded image of the current frame can be adjusted according to the second adjustment value. After adjustment according to the second adjustment value, the average quantization parameter of the encoded image of the current frame increases by the second adjustment value, resulting in a higher compression ratio of the encoded image of the current frame, thereby reducing the amount of data in the encoded image of the current frame.

[0148] Since the image size after encoding the current frame is large, in order to prevent the data volume of the current frame image from still being large after re-encoding, the second adjustment value can be set to a large value, such as 3, 4, 5, etc., so as to ensure that the data volume of the current frame image after re-encoding is not too large, thus ensuring the stability of the bit rate.

[0149] Step 236: Re-encode the current frame image according to the adjusted average quantization parameters.

[0150] Finally, server 20 re-encodes the current frame image based on the adjusted average quantization parameters, thereby minimizing the amount of data in the re-encoded current frame image while ensuring that the re-encoded current frame image is not distorted, thus ensuring the stability of the bitrate.

[0151] Optionally, such as Figure 12 As shown, step 230 can also be implemented through steps 237 to 239, specifically as follows:

[0152] Step 237: When the average quantization parameter is greater than the reference average quantization parameter and the image size is smaller than the reference image size, adjust the average quantization parameter according to the first adjustment value.

[0153] The reference average quantization parameter is determined based on the average quantization parameter of the instantaneous decoding refreshed IDR frame preceding the current frame image, and the reference image size is determined based on the image size of the instantaneous decoding refreshed IDR frame preceding the current frame image.

[0154] When re-encoding the current frame image, it can be first determined whether the average quantization parameter of the current frame image after encoding is greater than (or greater than or equal to) the reference average quantization parameter, and whether the image size of the current frame image after encoding is less than (or less than or equal to) the reference image size.

[0155] If the average quantization parameter of the encoded current frame image is greater than (or greater than or equal to) the reference average quantization parameter and the size of the encoded current frame image is less than (or less than or equal to) the reference image size, then the average quantization parameter of the encoded current frame image can be adjusted according to the first adjustment value. After adjustment according to the first adjustment value, the average quantization parameter of the encoded current frame image increases by the first adjustment value, resulting in a higher compression ratio for the encoded current frame image, thereby reducing the amount of data in the encoded current frame image.

[0156] For details on determining the reference average quantization parameter and the reference image size, please refer to the descriptions of steps 231 and 234, respectively. They will not be repeated here.

[0157] Since a large average quantization parameter has already been used when encoding the current frame image, and the size of the encoded current frame image is small, in order to prevent the compression rate of the current frame image from being too large after re-encoding, which would cause the current frame image to be distorted, the first adjustment value can be set to a small value, such as 1 or 2.

[0158] Step 238: When the average quantization parameter is less than the reference average quantization parameter, or when the average quantization parameter is greater than the reference average quantization parameter and the image size is greater than the reference image size, adjust the average quantization parameter according to the second adjustment value.

[0159] The second adjustment value is greater than the first adjustment value.

[0160] For example, when the average quantization parameter after encoding the current frame image is less than (or less than or equal to) the reference average quantization parameter, or when the average quantization parameter after encoding the current frame image is greater than (or greater than or equal to) the reference average quantization parameter and the image size after encoding the current frame image is greater than (or greater than or equal to) the reference image size, the average quantization parameter after encoding the current frame image can be adjusted according to the second adjustment value. After adjustment according to the second adjustment value, the average quantization parameter after encoding the current frame image increases by the second adjustment value, resulting in a higher compression ratio for the current frame image encoding, thereby reducing the amount of data after encoding the current frame image.

[0161] Since the current frame image uses a small average quantization parameter, or a large average quantization parameter but the image size after encoding is still large, in order to prevent the data volume of the current frame image from still being large after re-encoding, the second adjustment value can be set to a large value, such as 3, 4, 5, etc., so as to ensure that the data volume of the current frame image after re-encoding is not too large, thus ensuring the stability of the bit rate.

[0162] Step 239: Re-encode the current frame image according to the adjusted average quantization parameters.

[0163] Finally, server 20 re-encodes the current frame image based on the adjusted average quantization parameters, thereby minimizing the amount of data in the re-encoded current frame image while ensuring that the re-encoded current frame image is not distorted, thus ensuring the stability of the bitrate.

[0164] Optionally, the method further includes:

[0165] The cloud gaming client sends a request to the terminal to obtain information about the terminal's encoding and decoding capabilities.

[0166] Receive encoding / decoding capability information uploaded by the terminal;

[0167] The target encoding / decoding configuration and target encoding / decoding strategy are determined based on encoding / decoding capability information, the type of game running on the terminal, and network communication information.

[0168] The current frame image is encoded once to obtain encoding parameters, including:

[0169] The current frame image is encoded once according to the target encoding / decoding configuration and target encoding / decoding strategy to obtain encoding parameters.

[0170] For example, please combine Figure 1 The flowchart shown illustrates that the cloud (cloud server) initiates a request to the terminal (smart terminal) to obtain the encoding and decoding capability information through a cloud gaming client (such as the START client).

[0171] The request protocol field may include information such as the protocol version number and specific codec protocol query. When the terminal receives a capability information retrieval request, it uploads the codec capability information to the cloud gaming client, which then returns the terminal's codec capability information to the cloud. This codec capability information may include a status flag, supported protocol version numbers, and terminal device capability information, which may include terminal encoding capability information and terminal decoding capability information.

[0172] The capability information acquisition request may include a decoding capability request and / or an encoding capability request. After receiving the encoding / decoding capability information, the cloud can determine the target encoding / decoding configuration and target encoding / decoding strategy based on the encoding / decoding capability information, game type, and network communication information. The target encoding / decoding configuration may include the optimal encoding configuration and the optimal decoding configuration. The optimal decoding configuration may include the optimal decoding protocol, decoding resolution, video frame rate, etc. for the current terminal device. The target encoding / decoding strategy may include the number of video encoding reference frames, SVC enabling, etc.

[0173] The cloud can determine the set of encoding functions to be enabled based on the game type and network conditions, and then determine the optimal encoding configuration for the current terminal based on the device type and encoding capability information reported by the terminal. The cloud can also determine the optimal decoding configuration for the current terminal, such as the optimal decoding protocol, decoding resolution, video frame rate, and encoding / decoding strategies, such as the number of video encoding reference frames and SVC enabling, based on the terminal's decoding capability information and in combination with the game type and network conditions.

[0174] The cloud can encode the current frame image once according to the target encoding / decoding configuration (specifically the optimal encoding configuration) and the target encoding / decoding strategy to obtain the encoding parameters.

[0175] To better illustrate the image processing method provided in the embodiments of this application, please refer again. Figures 10 to 13 The image processing method provided in this application embodiment can be summarized into the following steps:

[0176] Step 1210: Encode the current frame image to obtain encoding parameters.

[0177] Step 1220: Is the average quantization parameter greater than the reference average quantization parameter?

[0178] Step 1230: Is the image size greater than the size threshold?

[0179] Step 1240: When the average quantization parameter is greater than the reference average quantization parameter and the image size is smaller than the reference image size, adjust the average quantization parameter according to the first adjustment value.

[0180] Step 1250: When the average quantization parameter is less than the reference average quantization parameter, or when the average quantization parameter is greater than the reference average quantization parameter and the image size is greater than the reference image size, adjust the average quantization parameter according to the second adjustment value.

[0181] In this context, step 1210 can be referred to in the description of step 210, steps 1220 and 1230 can be referred to in the description of steps 237 and 238, step 1240 can be referred to in the description of step 237, and step 1250 can be referred to in the description of step 238.

[0182] To better illustrate the image processing method provided in the embodiments of this application, please refer again. Figures 3 to 13 The image processing method provided in this application embodiment can be summarized into the following steps:

[0183] Step 210: Encode the current frame image once to obtain the encoding parameters.

[0184] Step 220: Determine whether to re-encode the current frame image based on the encoding parameters.

[0185] Specifically, step 220 can be implemented through steps 221 and 222; or, step 220 can be implemented through steps 223 to 226.

[0186] Step 230: If it is determined that the current frame image needs to be re-encoded, then the current frame image is re-encoded. The amount of data in the re-encoded current frame image is less than the amount of data in the current frame image after the first encoding.

[0187] Specifically, step 220 can be implemented through steps 231 to 223, or steps 234 to 226, or steps 237 to 229.

[0188] All of the above technical solutions can be combined in any way to form optional embodiments of this application, and will not be described in detail here.

[0189] This application embodiment determines whether to re-encode the current frame image by obtaining the encoding parameters of the current frame image, thereby reducing the data volume of the current frame image through re-encoding. This application embodiment determines whether to re-encode the current frame image by using encoding parameters. For example, if the encoding parameters indicate that there is a scene change after encoding, it determines to re-encode the current frame image to reduce the data volume of the current frame image, which can effectively avoid frames with large data volumes and maintain stable bitrate.

[0190] In addition, the embodiments of this application can more accurately determine whether re-encoding is needed by judging the scene switching and image size coordination, and determine the adjustment range of the average quantization parameter of the current frame image by the average quantization parameter and image size of the current frame image, thereby minimizing the amount of data after re-encoding of the current frame image while ensuring that the current frame image is not distorted, and ensuring the stability of the bit rate.

[0191] To facilitate better implementation of the image processing method of this application embodiment, this application embodiment also provides an image processing apparatus. Please refer to... Figure 14 , Figure 14 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application. The image processing apparatus 1000 may include:

[0192] The first encoding module 1010 is used to encode the current frame image once to obtain encoding parameters;

[0193] The first determining module 1020 is used to determine whether to re-encode the current frame image based on the encoding parameters;

[0194] The second encoding module 1030 is used to re-encode the current frame image when it is determined that the current frame image needs to be re-encoded, wherein the amount of data of the current frame image after re-encoding is less than the amount of data of the current frame image after the first encoding.

[0195] Optionally, the first determining module 1020 can also be used for:

[0196] When the proportion of predicted pixels within a frame is greater than a preset proportion, it is determined that there is a scene change in the current frame image;

[0197] When a scene change occurs in the current frame image, the current frame image is re-encoded.

[0198] Optionally, the first determining module 1020 can also be used for:

[0199] When the proportion of predicted pixels within a frame is greater than a preset proportion, it is determined that there is a scene change in the current frame image;

[0200] Determine the size threshold based on the current encoding frame rate and the current encoding bit rate;

[0201] If there is a scene change in the current frame image and the image size is larger than the size threshold, it is determined to re-encode the current frame image;

[0202] If there is no scene change in the current frame image, or if there is a scene change in the current frame image and the image size of the current frame image is less than the size threshold, it is determined that the current frame image will not be re-encoded.

[0203] Optionally, the second encoding module 1030 may also be used for:

[0204] When the average quantization parameter is greater than the reference average quantization parameter, the average quantization parameter is adjusted according to the first adjustment value. The reference average quantization parameter is determined based on the average quantization parameter of the IDR frame that was decoded and refreshed before the current frame image.

[0205] When the average quantization parameter is less than the reference average quantization parameter, the average quantization parameter is adjusted according to the second adjustment value, which is greater than the first adjustment value.

[0206] The current frame image is re-encoded based on the adjusted average quantization parameters.

[0207] Optionally, the second encoding module 1030 may also be used for:

[0208] When the image size is smaller than the reference image size, the average quantization parameter is adjusted according to the first adjustment value. The reference image size is determined based on the image size of the IDR frame that was instantaneously decoded and refreshed before the current frame image.

[0209] When the image size is larger than the reference image size, the average quantization parameter is adjusted according to the second adjustment value, which is greater than the first adjustment value.

[0210] The current frame image is re-encoded based on the adjusted average quantization parameters.

[0211] Optionally, the second encoding module 1030 may also be used for:

[0212] When the average quantization parameter is greater than the reference average quantization parameter and the image size is smaller than the reference image size, the average quantization parameter is adjusted according to the first adjustment value. The reference average quantization parameter is determined based on the average quantization parameter of the instantaneous decoding refreshed IDR frame before the current frame image, and the reference image size is determined based on the image size of the instantaneous decoding refreshed IDR frame before the current frame image.

[0213] When the average quantization parameter is less than the reference average quantization parameter, or when the average quantization parameter is greater than the reference average quantization parameter and the image size is greater than the reference image size, the average quantization parameter is adjusted according to the second adjustment value, which is greater than the first adjustment value.

[0214] The current frame image is re-encoded based on the adjusted average quantization parameters.

[0215] Optionally, the image processing device 1000 may further include a second determining module 1040, which is used to determine the encoding parameters of the instant decoding refresh IDR frame as reference encoding parameters when the current frame image is an instant decoding refresh IDR frame, and transmit the instant decoding refresh IDR frame.

[0216] Optionally, the image processing apparatus 1000 may further include a third determining module 1050, which is used to determine the re-encoded current frame image as an instant-decoded refresh IDR frame and transmit it.

[0217] Each module in the aforementioned image processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0218] The image processing device 1000 can be integrated into a terminal 10 or a server 20 that has storage and a processor and thus computing power, or the image processing device 1000 can be the terminal 10 or the server 20.

[0219] Optionally, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0220] Figure 15 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. The computer device may be... Figure 2 The terminal 10 or server 20 shown. For example... Figure 15As shown, the computer device 1200 may include: a communication interface 2010, a memory 2020, a processor 2030, and a communication bus 2040. The communication interface 2010, memory 2020, and processor 2030 communicate with each other via the communication bus 2040. The communication interface 2010 is used for data communication between the device 1000 and external devices. The memory 2020 can be used to store software programs and modules, and the processor 2030 runs the software programs and modules stored in the memory 2020, such as the software programs for corresponding operations in the foregoing method embodiments.

[0221] Optionally, the processor 2030 may call the software program and module stored in the memory 2020 to perform the following operations: encode the current frame image once to obtain the encoding parameters; determine whether to re-encode the current frame image according to the encoding parameters; if so, re-encode the current frame image, and the data volume of the re-encoded current frame image is less than the data volume of the current frame image after the first encoding.

[0222] This application also provides a computer-readable storage medium for storing a computer program. This computer-readable storage medium can be applied to a computer device, and the computer program causes the computer device to execute the corresponding processes in the image processing methods of the embodiments of this application; for the sake of brevity, further details are omitted here.

[0223] This application also provides a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the corresponding process in the image processing method of this application embodiment. For brevity, further details are omitted here.

[0224] This application also provides a computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the corresponding process in the image processing method of this application embodiment. For brevity, further details are omitted here.

[0225] It should be understood that the processor in the embodiments of this application may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0226] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0227] It should be understood that the above-described memory is exemplary and not a limiting description. For example, the memory in the embodiments of this application may also be static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DR RAM), etc. That is to say, the memory in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0228] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0229] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0230] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0231] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0232] In addition, the functional modules in the embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0233] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer or server 20) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0234] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, The method includes: The current frame image is encoded once to obtain encoding parameters, which indicate whether there is a scene change in the current frame image; Determine whether to re-encode the current frame image based on the encoding parameters; If so, the current frame image is re-encoded, and the data size of the re-encoded current frame image is less than the data size of the current frame image after the first encoding; The encoding parameters include the proportion of intra-frame predicted pixels, image size, current encoding frame rate, and current encoding bit rate. Determining whether to re-encode the current frame image based on the encoding parameters includes: When the proportion of predicted pixels within the frame is greater than a preset proportion, it is determined that there is a scene change in the current frame image; Determine the size threshold based on the current encoding frame rate and the current encoding bit rate; When the current frame image undergoes a scene change and the image size is greater than the size threshold, it is determined that the current frame image should be re-encoded.

2. The image processing method as described in claim 1, characterized in that, Also includes: When the current frame image is an Instant Decoding Refresh (IDR) frame, the encoding parameters of the IDR frame are determined as reference encoding parameters, and the IDR frame is transmitted.

3. The image processing method as described in claim 1, characterized in that, Also includes: The re-encoded current frame image is determined as an instant-decoded refresh IDR frame and transmitted.

4. The image processing method as described in claim 1, characterized in that, The encoding parameters include the proportion of intra-frame predicted pixels, and determining whether to re-encode the current frame image based on the encoding parameters includes: When the proportion of predicted pixels within the frame is greater than a preset proportion, it is determined that there is a scene change in the current frame image; When a scene change occurs in the current frame image, it is determined that the current frame image should be re-encoded.

5. The image processing method as described in claim 1, characterized in that, The step of determining whether to re-encode the current frame image based on the encoding parameters further includes: When there is no scene change in the current frame image, or when there is a scene change in the current frame image and the image size of the current frame image is smaller than the size threshold, it is determined that the current frame image will not be re-encoded.

6. The image processing method as described in claim 1, characterized in that, The encoding parameters also include an average quantization parameter, and the re-encoding of the current frame image includes: When the average quantization parameter is greater than the reference average quantization parameter, the average quantization parameter is adjusted according to the first adjustment value. The reference average quantization parameter is determined based on the average quantization parameter of the instantaneous decoded refresh IDR frame preceding the current frame image. When the average quantization parameter is less than the reference average quantization parameter, the average quantization parameter is adjusted according to a second adjustment value, wherein the second adjustment value is greater than the first adjustment value. The current frame image is re-encoded based on the adjusted average quantization parameters.

7. The image processing method as described in claim 1, characterized in that, The encoding parameters also include image size and average quantization parameters, and the re-encoding of the current frame image includes: When the image size is smaller than the reference image size, the average quantization parameter is adjusted according to a first adjustment value. The reference image size is determined based on the image size of the instantaneous decoded refresh IDR frame preceding the current frame image. When the image size is larger than the reference image size, the average quantization parameter is adjusted according to a second adjustment value, where the second adjustment value is greater than the first adjustment value. The current frame image is re-encoded based on the adjusted average quantization parameters.

8. The image processing method as described in claim 1, characterized in that, The encoding parameters also include average quantization parameters and image size. The re-encoding of the current frame image includes: When the average quantization parameter is greater than the reference average quantization parameter and the image size is smaller than the reference image size, the average quantization parameter is adjusted according to the first adjustment value. The reference average quantization parameter is determined based on the average quantization parameter of the instantaneous decoded refresh IDR frame preceding the current frame image, and the reference image size is determined based on the image size of the instantaneous decoded refresh IDR frame preceding the current frame image. When the average quantization parameter is less than the reference average quantization parameter, or when the average quantization parameter is greater than the reference average quantization parameter and the image size is greater than the reference image size, the average quantization parameter is adjusted according to a second adjustment value, wherein the second adjustment value is greater than the first adjustment value. The current frame image is re-encoded based on the adjusted average quantization parameters.

9. The image processing method as described in claim 1, characterized in that, Also includes: The cloud gaming client sends a request to the terminal to obtain the terminal's encoding and decoding capability information; Receive the encoding / decoding capability information uploaded by the terminal; The target encoding / decoding configuration and target encoding / decoding strategy are determined based on the encoding / decoding capability information, the type of game running on the terminal, and network communication information. The process of encoding the current frame image once to obtain encoding parameters includes: The current frame image is encoded once according to the target encoding / decoding configuration and target encoding / decoding strategy to obtain the encoding parameters.

10. An image processing apparatus, characterized in that, The device includes: The first encoding module is used to encode the current frame image once to obtain encoding parameters, wherein the encoding parameters indicate whether the current frame image has a scene switch; The first determining module is used to determine whether to re-encode the current frame image based on the encoding parameters; The second encoding module is used to re-encode the current frame image when it is determined that the current frame image needs to be re-encoded, wherein the data volume of the re-encoded current frame image is less than the data volume of the current frame image after the first encoding. The encoding parameters include the proportion of intra-frame predicted pixels, image size, current encoding frame rate, and current encoding bitrate. The first determining module is further configured to: determine that the current frame image has a scene change when the proportion of intra-frame predicted pixels is greater than a preset proportion; determine a size threshold based on the current encoding frame rate and the current encoding bitrate; and determine to re-encode the current frame image when the current frame image has a scene change and the image size is greater than the size threshold.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted for loading by a processor to perform the steps of the image processing method as described in any one of claims 1-9.

12. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing a computer program, and the processor executes the steps of the image processing method according to any one of claims 1-9 by calling the computer program stored in the memory.

13. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the steps of the image processing method according to any one of claims 1-9.

Images