A coding processing method, device, server and medium

Abstract

An embodiment of the present application discloses an encoding processing method, device, server, and medium. The method includes: when detecting a decoding exception situation existing in a terminal device, according to the corresponding relationship between different decoding exception situations and encoding strategies, taking the encoding strategy corresponding to the decoding exception situation as a recommended encoding strategy; obtaining the reference decoding capabilities that can be provided when the terminal device decodes the encoded data generated by the recommended encoding strategy, and obtaining the expected decoding capabilities that the terminal device needs to possess when enabling the terminal device to normally decode the encoded data generated by the recommended encoding strategy; performing a matching process on the reference decoding capabilities and the expected decoding capabilities to obtain a matching result, and when the matching result indicates that the reference decoding capabilities and the expected decoding capabilities are successfully matched, using the recommended encoding strategy to perform encoding processing on the to-be-processed game frames in the target cloud game, which can ensure the smoothness of decoding of the terminal device and the stable output of the screen.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular, to an encoding processing method, apparatus, server, and medium. Background Art

[0002] With the development of video encoding and decoding technology, in some encoding and decoding scenarios (such as scenarios of collaboration between the cloud and terminals), it is usually possible for a video content producer (such as a server) to transmit compressed video data to a video content consumer (such as a terminal device). In this process, the output of the picture is often unstable due to decoding jams of the video content consumer, for example, phenomena such as picture jams occur. The traditional processing method usually directly discards frames with transmission failures or frames that are too late to be decoded. However, practice has shown that using the traditional frame processing method cannot effectively avoid the problem of jams during the decoding process of terminal devices. Therefore, how to ensure the decoding fluency of terminal devices has become a current research hotspot. Summary of the Invention

[0003] Embodiments of the present application provide an encoding processing method, apparatus, server, and medium, which can perform encoding processing by effectively using the decoding capabilities provided by terminal devices based on an encoding strategy that is fully adapted to terminal devices, can better avoid decoding anomalies, and ensure the decoding fluency of terminal devices and the stable output of pictures.

[0004] On the one hand, embodiments of the present application provide an encoding processing method, including:

[0005] When a decoding anomaly situation existing in a terminal device is detected, according to the correspondence between different decoding anomaly situations and encoding strategies, the encoding strategy corresponding to the decoding anomaly situation is used as a recommended encoding strategy; wherein, there are at least two encoding strategies, and the recommended encoding strategies corresponding to different decoding anomaly situations are different;

[0006] Obtain the reference decoding capabilities that can be provided when the terminal device decodes the encoded data generated by the recommended encoding strategy, and obtain the expected decoding capabilities that the terminal device needs to possess when enabling the terminal device to normally decode the encoded data generated by the recommended encoding strategy;

[0007] Perform a matching process on the reference decoding capabilities and the expected decoding capabilities to obtain a matching result, and when the matching result indicates that the reference decoding capabilities match the expected decoding capabilities successfully, use the recommended encoding strategy to perform encoding processing on the to-be-processed game frames in the target cloud game.

[0008] On the one hand, embodiments of the present application provide an encoding processing apparatus, including:

[0009] A determination module, configured to, when detecting a decoding anomaly in the terminal device, based on the correspondence between different decoding anomalies and encoding strategies, use the encoding strategy corresponding to the decoding anomaly as the recommended encoding strategy; wherein, there are at least two encoding strategies, and the recommended encoding strategies corresponding to different decoding anomalies are different;

[0010] An acquisition module, configured to acquire the reference decoding ability that can be provided when the terminal device decodes the encoded data generated by the recommended encoding strategy, and acquire the expected decoding ability that the terminal device needs to possess to correctly decode the encoded data generated by the recommended encoding strategy;

[0011] A matching module, configured to perform a matching process on the reference decoding ability and the expected decoding ability to obtain a matching result;

[0012] An encoding module, configured to, when the matching result indicates that the reference decoding ability matches the expected decoding ability successfully, use the recommended encoding strategy to encode the game frames to be processed in the target cloud game.

[0013] Correspondingly, an embodiment of the present application provides a server, including: a processor, a memory, and a network interface; the processor is connected to the memory and the network interface, wherein the network interface is used to provide network communication functions, the memory is used to store program codes, and the processor is used to call the program codes to execute the encoding processing method in the embodiment of the present application.

[0014] Correspondingly, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and the computer program includes program instructions that, when executed by a processor, execute the encoding processing method in the embodiment of the present application.

[0015] Correspondingly, an embodiment of the present application provides a computer program product, which includes a computer program or computer instructions that, when executed by a processor, implement the encoding processing method in the embodiment of the present application.

[0016] In an embodiment of the present application, a decoding abnormality of a terminal device can be detected. Through the correspondence between different decoding abnormalities and encoding strategies, the encoding strategy corresponding to the detected decoding abnormality of the terminal device can be used as a recommended encoding strategy. In this way, a corresponding encoding strategy can be adaptively recommended for the decoding abnormality, and then the reference decoding capability provided by the terminal device when decoding the encoded data generated using the recommended encoding strategy can be obtained, and the expected decoding capability required for the terminal device to normally decode the encoded data generated by the recommended encoding strategy can be obtained, and then based on the obtained reference decoding capability and expected decoding capability, it is determined whether the recommended encoding strategy can be adopted. Specifically, the reference decoding capability is matched with the expected decoding capability to make a judgment. When the matching result indicates that the two match, it means that the terminal device supports normal decoding of the encoded data generated using the recommended encoding strategy, thereby overcoming the decoding abnormality. The recommended encoding strategy is to fully adapt to the decoding capability provided by the terminal device. The recommended encoding strategy is used to encode the pending game frames of the target cloud game. The decoding capability of the terminal device can also be fully utilized, which can better avoid decoding abnormalities, so that the terminal device can normally decode the received encoded data, ensuring the smoothness of the decoding of the terminal device and the stable output of the picture. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is an architecture diagram of a coding processing system provided by an exemplary embodiment of the present application;

[0018] Figure 2 It is a flowchart of a coding processing method provided by an exemplary embodiment of the present application;

[0019] Figure 3 is a flowchart of another encoding processing method provided by an exemplary embodiment of the present application;

[0020] Figure 4a It is a schematic diagram of sending network detection data within a time window provided by an exemplary embodiment of the present application;

[0021] Figure 4b is a schematic diagram of a scenario in which a terminal device is overloaded, provided by an exemplary embodiment of the present application;

[0022] Figure 5a It is a schematic diagram of the effect of shielding a reference frame provided by an exemplary embodiment of the present application;

[0023] Figure 5b It is a schematic diagram of a coding process flow provided by an exemplary embodiment of the present application;

[0024] Figure 6is a structural diagram of a coding processing device provided in an embodiment of the present application;

[0025] Figure 7 It is a structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The present application provides an encoding processing method. Before encoding the to-be-processed game frames of the target cloud game, the server as the producer of content (such as video content) can detect the decoding anomaly of the terminal device (the consumer of content (such as video content)). After detecting the decoding anomaly of the terminal device, the corresponding recommended encoding strategy is obtained through the correspondence between different decoding anomalies and encoding strategies. After obtaining the recommended encoding strategy, the server can match the decoding capability of the terminal device with the expected decoding capability when the encoded data generated using the recommended encoding strategy is to be normally decoded, and the reference decoding capability that can be provided by the terminal device when decoding the encoded data generated using the recommended encoding strategy. The computer device can determine whether the terminal device can normally decode the coded data obtained by using the recommended coding strategy based on the matching result. Then, when the matching result indicates that the expected decoding capability matches the reference decoding capability that the terminal device can provide, it means that the terminal device supports normal decoding of the coded data generated using the recommended coding strategy, which means that the recommended coding strategy is fully adapted to the decoding capability provided by the terminal device. In this case, the server can also use the recommended coding strategy to encode the pending game frames of the target cloud game, and the decoding capability of the terminal device can also be fully utilized, so that the terminal device can normally decode the received coded data, ensuring the smoothness of the decoding of the terminal device and the stable output of the picture. In the case where the matching result indicates that the expected decoding capability does not match the reference decoding capability that the terminal device can provide, the server can select a new coding reference frame and use the new coding reference frame as the immediate refresh frame of the pending game frame of the target cloud game to encode the pending game frame of the target cloud game, so that the server can realize the adaptive and flexible adjustment of the coding processing method of the pending game frame of the target cloud game based on the effective evaluation of the decoding capability of the terminal device, thereby improving the flexibility of the server in the encoding process.

[0027] In one embodiment, when the server can be a cloud server, then the effective encoding process of cloud games based on the reference implementation of the decoding ability of the terminal device by the cloud server is the end-cloud collaborative encoding process of cloud games between the cloud server and the terminal device. A cloud game refers to an online game technology based on cloud computing technology. Cloud computing technology belongs to a type of cloud technology. The so-called cloud technology (Cloud technology) refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or a local area network to achieve data computing, storage, processing, and sharing. Among them, cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computing devices, enabling various application systems to obtain computing power, storage space, and information services as needed. The network that provides resources is called the "cloud". The resources in the "cloud" seem to be infinitely expandable to users, and can be obtained at any time, used on demand, expanded at any time, and paid according to usage. As a basic capability provider of cloud computing, a cloud computing resource pool (referred to as a cloud platform, generally called an IaaS (Infrastructure as a Service) platform) will be established, and various types of virtual resources will be deployed in the resource pool for external customers to choose and use. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices. Among them, cloud servers have powerful computing resources, are highly virtualized and highly distributed, and have relatively strong computing capabilities. As a specific device facing the object, the terminal device has different types (such as smart phones, computers), and the corresponding computing capabilities are uneven, and the computing capabilities are relatively weak. Since there are differences in the computing capabilities of the content producer (cloud side) and the content consumer (terminal), by collaborating between the two ends to complete a relatively complex compression and encoding task, it is possible to utilize the resources and powerful computing capabilities (such as encoding capabilities) of the cloud side to reduce the amount of data transmitted over the network, and also effectively utilize the computing capabilities (such as decoding capabilities) of the terminal device.

[0028] The encoding process method proposed in the embodiments of the present application can be applied in, for example, Figure 1 the encoding process system shown. This encoding process system is an encoding process system composed of the above-mentioned terminal device and cloud server. As shown in Figure 1 the terminal device can be any one marked by 11 in Figure 1 and the cloud server (or server) can be, for example, Figure 1Any one or more of them are marked by 12 in the server cluster. In the embodiments of the present application, mainly taking the cloud server as any one server in the server cluster as an example for illustration. In one embodiment, the decoders included in the terminal device 11 are different, and the corresponding decoding capabilities also have certain differences. For example, when the included decoder is a decoder supporting H264 (a video coding standard), or when the included decoder is a decoder supporting HEVC (High Efficiency Video Coding), the video coding and decoding standards adopted by the corresponding decoder (i.e., the decoding chip) are different, resulting in certain differences in the decoding capabilities of the terminal device 11. That is to say, when the server 12 determines the decoding capabilities that the terminal device 11 can provide, it can be determined based on the type of decoder included in the terminal device 11.

[0029] In addition to the influence of the type of decoder on the decoding capabilities of the terminal device 11, the decoding capabilities of the terminal device 11 may also be limited due to the poor network condition (such as network fluctuations) between the terminal device 11 and the server 12. In this case, it is not only reflected in the poor decoding capabilities of the terminal device, but also leads to abnormal decoding of the terminal device, such as packet loss and jamming of the terminal device. Based on the connection relationship between the terminal device 11 and the server 12, the server 12 can timely detect the abnormal decoding situation of the terminal device. When the abnormal decoding situation occurs, the server 12 can adjust the local encoding strategy according to the actual decoding capabilities that the terminal device 11 can provide, so as to ensure that the terminal device 11 can normally decode the encoded data sent by the server 12, which effectively avoids the abnormal decoding of the terminal device 11 due to network fluctuations, and further ensures the smoothness of the presentation of the corresponding picture data on the terminal device side. It can be understood that the terminal device 11 includes but is not limited to: smart phones, tablet computers, smart wearable devices, intelligent voice interaction devices, smart home appliances, personal computers, vehicle-mounted terminals, and other devices. The present application does not make any restrictions on this. Regarding the number of terminal devices, the present application does not make any restrictions. The server 12 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Deliver Network), and big data and artificial intelligence platforms, but is not limited thereto.

[0030] Please refer to Figure 2, which is an encoding processing method provided by an exemplary embodiment of the present application. This method can be executed by the above-mentioned server. The encoding processing method includes the following steps S201 - S203:

[0031] S201, when a decoding anomaly of the terminal device is detected, according to the correspondence between different decoding anomalies and encoding strategies, the encoding strategy corresponding to the decoding anomaly is used as the recommended encoding strategy.

[0032] The decoding anomaly refers to a situation where an anomaly occurs during the decoding process of the terminal device for the encoded data, which can be used to reflect the actual decoding situation of the terminal device. For example, the terminal device experiences lags during decoding. The decoding anomaly existing in the terminal device may be caused by unstable network transmission or insufficient hardware capabilities of the terminal device. That is, network fluctuations or fluctuations in the real-time decoding ability of the terminal device may both result in decoding anomalies. Therefore, different decoding anomalies may include anomalies due to network fluctuations and anomalies due to overload (i.e., an excessive amount of encoded data waiting to be decoded by the terminal device). When the decoding anomaly existing in the terminal device is an anomaly due to network fluctuations or an anomaly due to overload, it is usually manifested as the picture presented by the terminal device being stuck. In one implementation, the anomaly due to network fluctuations includes either a frequent fluctuation anomaly (i.e., an anomaly where the network fluctuates frequently) or an occasional fluctuation anomaly (i.e., an anomaly where the network fluctuates instantaneously), and the anomaly due to overload includes either a frequent overload or an occasional overload. At this time, the decoding anomaly can be a frequent fluctuation anomaly or an occasional fluctuation anomaly, or it can be a frequent overload or an occasional overload.

[0033] The correspondence between different decoding anomalies and encoding strategies is used to indicate an encoding strategy adapted to a decoding anomaly. One decoding anomaly can correspond to one encoding strategy, and different decoding anomalies can correspond to different encoding strategies. Furthermore, for different decoding anomalies, the recommended encoding strategies determined based on the correspondence between the decoding anomaly and the encoding strategy are also different. For example, the encoding strategies include encoding strategy A and encoding strategy B. When the decoding anomaly is an occasional overload, it can correspond to encoding strategy A, and encoding strategy A can be used as the recommended encoding strategy. When the decoding anomaly is a frequent fluctuation anomaly, it can correspond to encoding strategy B, and encoding strategy B can be used as the recommended encoding strategy.

[0034] Different decoding exception cases can also correspond to the same recommended encoding strategy. For example, if the decoding exception case is occasional overload or occasional fluctuation anomaly, the corresponding encoding strategy A can be used as the recommended encoding strategy. In addition, different network fluctuation anomaly cases (including frequent network fluctuations and occasional network fluctuations), or different overload anomaly cases (including frequent overload and occasional overload) can also correspond to different recommended encoding strategies. For example, if the detected decoding exception case of the terminal device is occasional overload, then the determined encoding strategy A can be used as the recommended encoding strategy; if the detected decoding exception case of the terminal device is frequent overload, then the determined encoding strategy B can be used as the recommended encoding strategy.

[0035] The encoding strategies include at least two kinds, and the encoding strategies are used to indicate the way of encoding the data to be encoded. In this application, the encoding reference frames can be managed through the encoding strategies, specifically, the dependency relationship between the encoding reference frames and the frames to be encoded is managed. Among them, the encoding reference frame refers to a frame that the data to be encoded refers to. Usually, after a frame that the data to be encoded refers to is decoded in the decoding stage, it can also be used as a decoding reference frame. The dependency relationship is used to indicate which encoding reference frames the frames to be encoded can refer to for encoding. In one implementation, the encoding strategy can be a hierarchical encoding strategy or a frame skipping encoding strategy, and the decoding exception case can correspond to either the hierarchical encoding strategy or the frame skipping encoding strategy. The hierarchical encoding strategy refers to the strategy of using hierarchical encoding method for encoding. Under the hierarchical encoding method, it can be divided into an enhancement layer and a base layer, and discarding of the frames in the enhancement layer is supported. The frame skipping encoding strategy refers to the strategy of encoding by masking specific reference frames or designated reference frames. The specific reference frames or designated reference frames to be masked can be flexibly set based on the actual situation. The reference frames mentioned in the above two encoding strategies are all encoding reference frames.

[0036] When the server detects a decoding anomaly in the terminal device, it can select, based on the pre-set correspondence between decoding anomalies and encoding strategies, an encoding strategy corresponding to the detected decoding anomaly from at least two encoding strategies, and then use the encoding strategy corresponding to the detected decoding anomaly as the recommended encoding strategy. In this way, an encoding strategy adapted to the detected decoding anomaly is adaptively recommended based on the correspondence between decoding anomalies and encoding strategies. For different terminal devices with decoding anomalies, the server can recommend corresponding encoding strategies. Through the encoding strategies, the reference frames required for the frames to be encoded can be managed, such as discarding some reference frames, so that neither the encoding nor the decoding process needs to refer to the discarded frames. For example, if the encoding strategy indicates that one of the three originally referenced frames needs to be discarded during encoding, then the frames to be encoded can refer to the non-discarded frames during actual encoding. If a decoding anomaly occurs when decoding a certain reference frame, discarding the reference frame can achieve normal decoding, the decoding anomaly can be alleviated, and the probability of decoding anomalies can be reduced.

[0037] S202. Obtain the reference decoding ability that the terminal device can provide when decoding the encoded data generated by the recommended encoding strategy, and obtain the expected decoding ability that the terminal device needs to possess when enabling the terminal device to normally decode the encoded data generated by the recommended encoding strategy.

[0038] Before detecting the decoding anomaly of the terminal device, the server can detect the terminal device to obtain the reference decoding ability that the terminal device can provide, and obtain it from the server's database when needed. The reference decoding ability is used to reflect the actual decoding ability of the terminal device when decoding the encoded data generated by the recommended encoding strategy. The reference decoding ability can include any one or more of static decoding ability and dynamic decoding ability. The static decoding ability is detected in a static manner. For example, query information can be sent to the hardware interface of the terminal device (specifically, the communication structure of the decoder) to obtain it. For example, the chip type supported by the terminal device, which is divided according to the supported video coding and decoding standards, such as H264 chips, HEVC chips, whether the chip of the terminal device supports hierarchical coding and decoding, and the threshold value of the number of reference frames used by the chip of the terminal device for decoding one frame. The dynamic decoding ability is detected in a dynamic manner. For example, bitstreams generated by different encoding strategies are sent, and it is judged whether the terminal device can normally decode (that is, whether decoding fails or freezes). The dynamic decoding ability and the static decoding ability are determined by the hardware device (such as the decoder) of the terminal device itself and usually will not change due to other factors (such as network fluctuations).

[0039] The server can use a recommended encoding strategy to encode the data to be encoded, obtaining encoded data. When the terminal device performs normal decoding on the encoded data generated by the recommended encoding strategy, the decoding anomaly can be overcome. If it is necessary for the terminal device to perform normal decoding on the encoded data generated by the recommended encoding strategy (i.e., there is no lag or error prompt during decoding by the terminal device), then the terminal device needs to have a decoding ability that can support normal decoding, that is, the expected decoding ability. Therefore, it is possible to obtain the expected decoding ability that the terminal device needs to have when performing normal decoding on the encoded data generated by the recommended encoding strategy. This expected decoding ability is the ability that the terminal device is expected to achieve for normal decoding of the encoded data based on the recommended encoding strategy to overcome the decoding anomaly. The expected decoding ability can be reflected by the number of reference frames supported by the terminal device for decoding one frame, or by the decoding situation of the encoded data generated by an encoding strategy supported by the terminal device for decoding. Subsequently, it can be further determined whether the recommended encoding strategy is available based on the matching result between the actual reference decoding ability of the terminal device and the expected decoding ability expected for the terminal device.

[0040] S203. Perform a matching process on the reference decoding ability and the expected decoding ability to obtain a matching result. When the matching result indicates that the reference decoding ability and the expected decoding ability match successfully, use the recommended encoding strategy to encode the game frame to be processed in the target cloud game.

[0041] The server can perform a matching process on the reference decoding ability and the expected decoding ability. Specifically, it can compare the information contained in the reference decoding ability and the expected decoding ability (such as the maximum number of reference frames supported for actually decoding one frame and the maximum number of reference frames expected to be achieved for decoding one frame). The matching result obtained from the matching between the reference decoding ability and the expected decoding ability can indicate whether the two decoding abilities match successfully, that is, whether the decoding ability supported by the terminal device reaches the decoding ability expected by the server for the terminal device. Different processes can be executed based on the different contents indicated by the matching result.

[0042] When the matching result indicates that the reference decoding ability matches the expected decoding ability successfully, it means that the decoding ability provided by the terminal device meets the requirements of the decoding ability for expected normal decoding. Then the server can use the recommended encoding strategy to encode the data to be encoded. Here, the data to be encoded is the game frame to be processed in the target cloud game, where the target cloud game is any one of one or more cloud games running in the server, and the game frame to be processed is a frame of game screen waiting to be encoded in the target cloud game. Since whether the server uses the recommended encoding strategy is determined by comparing the reference decoding ability provided by the terminal device with the expected decoding ability required for the terminal device, when it is determined to use the recommended encoding strategy to encode the game frame to be processed and send the encoded game frame to the terminal device, the terminal device can normally decode the game frame processed based on this recommended encoding strategy without decoding anomalies, realizing the stable and smooth output of the game screen of the cloud game on the terminal device.

[0043] In one implementation, when the matching result indicates that the reference decoding ability does not match the expected decoding ability, it means that the decoding ability provided by the terminal device does not meet the requirements of the decoding ability for expected normal decoding. At this time, the server can use other encoding strategies to encode the data to be encoded. For example, when the matching result indicates that the two do not match, the game frame to be processed in the target cloud game is encoded by selecting a new encoding reference frame. In this way, the recommended encoding strategy can also be adaptively adjusted based on the decoding ability of the terminal device, and the flexibility of using the encoding strategy is high.

[0044] It should be noted that for different terminal devices, the encoding processing method provided by this application can be used. Based on the decoding ability of the terminal device itself, it can be intelligently judged whether to support the use of the recommended encoding strategy. For terminal devices that support the use of the recommended encoding strategy, the recommended encoding strategy can be used for reference frame management, and the original management method of the encoding reference frame can be modified for encoding processing. Even if the decoding abilities of different terminal devices are different, the original management method of the encoding reference frame can be adaptively modified for encoding processing according to the decoding ability of the terminal device itself and combined with the decoding anomaly situation. In this way, different decoding anomaly situations and the decoding ability of the terminal itself are comprehensively considered, and decoding anomalies can be effectively avoided.

[0045] In the encoding processing method provided by the embodiment of the present application, the server can detect the decoding anomaly of the terminal device. According to the corresponding relationship between different decoding anomalies and encoding strategies, the encoding strategy corresponding to the detected decoding anomaly of the terminal device can be used as the recommended encoding strategy. Different decoding anomalies correspond to different recommended encoding strategies. In this way, an encoding strategy can be adaptively recommended for the decoding anomaly, and the reference decoding ability provided by the terminal device when decoding the encoded data generated using the recommended encoding strategy can be obtained, as well as the expected decoding ability required for the terminal device to normally decode the encoded data generated using the recommended encoding strategy. Furthermore, it can be determined whether the recommended encoding strategy can be adopted based on the reference decoding ability and the expected decoding ability. Specifically, by matching the reference decoding ability with the expected decoding ability and comparing the difference between the two decoding abilities, a matching result can be obtained. When the matching result indicates that the two match, it means that the terminal device supports normal decoding of the encoded data generated using the recommended encoding strategy, thereby overcoming the decoding anomaly. The recommended encoding strategy is fully adapted to the decoding ability provided by the terminal device. By using the recommended encoding strategy to encode the to-be-processed game frames of the target cloud game, the decoding ability of the terminal device can also be fully utilized, enabling the terminal device to normally decode the encoded data sent by the server and ensuring the smoothness of the terminal device's decoding and the stable output of the screen.

[0046] Please refer to Figure 3 , Figure 3 which is an encoding processing method provided by an exemplary embodiment of the present application. This method can be executed by the server. The encoding processing method includes the following steps S301 - S305:

[0047] S301. When detecting the decoding anomaly existing in the terminal device, according to the corresponding relationship between different decoding anomalies and encoding strategies, use the encoding strategy corresponding to the decoding anomaly as the recommended encoding strategy.

[0048] In one embodiment, the determination method for the decoding anomaly existing in the terminal device can be the content introduced in (1) - (3) as follows:

[0049] (1) Send network probe data to the terminal device within a time window. Specifically, the server can send network probe data to the terminal device within the time window. The network probe data can be the encoded data of the target cloud game, specifically one or more encoded game frames; the network probe data can also be encoded data unrelated to the target cloud game, such as any encoded video frame. The time window can be a sliding window with a time length of T. Among them, the time window contains multiple time points, and the network probe data sent to the terminal device at one time point is one or more. Continuous detection can be carried out by sending network probe data at different time points within the time window. The multiple time points can be periodic time points or non-periodic time points, that is, corresponding to periodic detection or non-periodic detection. For example, within a time window with a time length of 1 second, an encoded game frame can be sent to the terminal device every 100 milliseconds (ms). In this way, 10 detections can be carried out within the time window, as shown in Figure 4a . Through the detection within the time window, information about the network probe data can be collected in units of the time window, and the detection of whether there is a decoding anomaly in the terminal device within the time window can be realized.

[0050] (2) Obtain the detection feedback data corresponding to the network probe data sent at each time point of the time window, and generate the network detection result at the corresponding time point according to the detection feedback data.

[0051] The server sends network probe data to the terminal device at each time point within the time window, and the detection feedback data corresponding to the network probe data at each time point can be obtained. The detection feedback data can be used to feedback the transmission situation of the network probe data. The detection feedback data corresponding to one time point can generate a network detection result corresponding to that time point. The network detection result corresponding to any time point is used to indicate whether there is a network fluctuation at any time point.

[0052] In one implementation, taking any time point as the target time point as an example, what can generate the network detection result of the target time point based on the detection feedback data of the target time point is as follows: Compare the detection feedback data of the target time point with the preset fluctuation threshold. When the detection feedback data of the target time point reaches the preset fluctuation threshold, generate a network detection result for indicating that network fluctuation occurs at the target time point; when the detection feedback data of the target time point does not reach the preset fluctuation threshold, generate a network detection result for indicating that no network fluctuation occurs at the target time point. Optionally, the detection feedback data includes any one or more of the packet loss count and the network transmission delay. The packet loss count refers to the number of times the network detection data is lost when transmitted to the terminal device, and the network transmission delay refers to the time delay when the network detection data is transmitted to the terminal device through the network. For example, the packet loss count is 3 and the network transmission delay is 200 milliseconds. The preset fluctuation threshold includes a packet loss count threshold and a transmission delay threshold. The detection feedback data reaching the preset fluctuation threshold means that: the packet loss count in the detection feedback data is greater than or equal to the packet loss count threshold, and / or the network transmission delay is greater than or equal to the transmission delay threshold. When the detection feedback data of the target time point reaches the preset fluctuation threshold, the server can determine that a network fluctuation occurs at the target time point for the current network, and the generated detection result is used to indicate that network fluctuation occurs at the target time point. On the contrary, it is determined that no network fluctuation occurs at the target time point for the current network, and the generated detection result is used to indicate that no network fluctuation occurs at the target time point.

[0053] The network fluctuation occurring at any time point may be in the scenario of long-term network instability or in the scenario of instantaneous network fluctuation. Therefore, the network detection results of each time point within the time window can be comprehensively used to determine in which scenario the network fluctuation occurs, and further determine the decoding anomaly situation of the terminal device.

[0054] (3) According to the network detection results respectively corresponding to each time point in the time window, count the number of times of network fluctuation in the time window, and determine the decoding anomaly situation of the terminal device according to the counted number.

[0055] One time point within the time window corresponds to one network detection result. The network detection result is used to indicate whether network fluctuation occurs at the corresponding time point, and one network fluctuation corresponds to one time point. Network fluctuation may occur at multiple time points within the time window, or may not occur at each time point. The number of times of network fluctuation within the time window is counted through the network detection results, and the counted number of times of network fluctuation can be used to determine the decoding anomaly situation of the terminal device.

[0056] In one implementation, the decoding anomaly existing in the terminal device is determined based on the number of statistics, including: obtaining a fluctuation index value; when the number of statistics is greater than or equal to the fluctuation index value, determining that the decoding anomaly existing in the terminal device is a frequent fluctuation anomaly; when the number of statistics is greater than or equal to 1 and less than the fluctuation index value, determining that the decoding anomaly existing in the terminal device is an occasional fluctuation anomaly.

[0057] The fluctuation index value is a preset threshold value for judging whether the network fluctuation is frequent. The number of statistics is the number of times the network fluctuation occurs. When the number of statistics is greater than or equal to the fluctuation index value, it means that the number of network fluctuations in the time window exceeds the preset fluctuation threshold value, and then it can be determined that the decoding abnormality of the terminal device is a frequent fluctuation abnormality, that is, the abnormality of frequent network fluctuations. For example, the fluctuation index value is 2, and the number of network fluctuations in the time window T1 is 3. The number of network fluctuations is greater than the fluctuation index value, then it can be determined that the current network fluctuations are frequent. It can be known that the network environment in which the terminal device is located may be an environment in which the network is unstable for a long time. On the contrary, when the number of statistics is greater than or equal to 1 and less than the fluctuation index value, it means that the network fluctuation occurs in the time window, but the number of network fluctuations does not exceed the preset fluctuation threshold value, and then it can be determined that the decoding abnormality of the terminal device is an occasional fluctuation abnormality, that is, an abnormality of occasional network fluctuations. When an occasional fluctuation abnormality occurs, the network environment in which the terminal device is located may be an environment in which instantaneous network fluctuations or freezes occur. The above-mentioned occasional fluctuation anomaly and frequent fluctuation anomaly are both decoding anomalies, and these two decoding anomalies can be collectively referred to as network fluctuation anomalies. When the decoding anomaly existing in the terminal device includes the network fluctuation anomaly, it usually refers to occasional fluctuation anomaly or frequent fluctuation anomaly.

[0058] It can be understood that when the number of statistics is equal to 0, it means that there is no network fluctuation in the time window, and it can be determined that there is no decoding anomaly in the terminal device. Furthermore, when there is no network fluctuation in a time window, the time window can be updated and the network environment of the terminal device can continue to be detected. When it is detected that the terminal device has a decoding anomaly, the current decoding anomaly is overcome based on this solution and the subsequent decoding anomaly is avoided. In this way, during the entire encoding process, it is possible to follow up whether there is a decoding anomaly in the terminal device, and perform encoding processing by adopting the corresponding encoding strategy to enable smooth decoding of the terminal device and achieve stable output of the picture.

[0059] In another embodiment, the method for determining the decoding anomaly situation of the terminal device may also include the following: (1) Obtain the load information of the terminal device at different time points within the time window. (2) Determine the number of overloads of the terminal device within the time window according to the load information corresponding to different time points of the terminal device within the time window. (3) Determine the decoding anomaly situation of the terminal device according to the number of overloads.

[0060] The different time points within the time window are the time points when the server probes the terminal device. The time window includes at least two time points. The different time points can be periodic probing time points or aperiodic probing time points. In one way, each time point can be a historical time point before the time point of encoding a frame. For example, within a 1s time window, the current frame is the 4th frame encoded at the 400ms, then each time point can be the 100ms, the 200ms, and the 300ms. In this way, the decoding load situation of the terminal device before encoding a frame can be obtained in real time, so as to provide a reference for the use of the encoding strategy. It can be understood that the time point for sending the above network probing data can also be a historical time point before encoding a frame. By obtaining the network fluctuation situation (which may be occasional network fluctuations or frequent network fluctuations) before encoding a frame, and providing corresponding encoding strategies to adapt to different network fluctuation situations, intelligent reference frame management can be performed.

[0061] The server can obtain the load information of the terminal device at each time point within the time window. The load information is the amount of data to be decoded of the terminal device at the corresponding time point. The amount of data to be decoded refers to the amount of encoded data waiting for the terminal device to decode, such as the number of encoded frames waiting for the terminal device to decode. One time point corresponds to one load information, and also corresponds to one probe of the server for the terminal device.

[0062] When the amount of data to be decoded of the terminal device at any time point is greater than or equal to the data volume threshold, the terminal device is overloaded once at any time point. If the amount of data to be decoded is the number of encoded frames waiting for the terminal device to decode, the data volume threshold is the threshold value of the number of frames waiting for the terminal device to decode. When the number of encoded frames waiting for the terminal device to decode is greater than or equal to this threshold value, it indicates that the terminal device has a temporarily high decoding load, that is, an overload occurs. It can be understood that when the amount of data to be decoded of the terminal device at any time point is less than the data volume threshold, the terminal device is not overloaded at any time point.

[0063] Overload of the terminal device may be caused by unstable network or the limitation of the hardware capabilities of the terminal device itself. The real-time decoding ability of the terminal device may fluctuate due to unstable network transmission or the impact of the hardware device of the terminal device, resulting in too high decoding load and overload. On the terminal device, it is manifested as decoding stuttering and picture stuttering. For example, due to network stuttering, the transmission of the encoded second frame to the terminal device fails. The server will try to retransmit the second frame multiple times. During the retry process of the server, the third frame, whose encoding order is after the second frame, is also encoded and waiting to be sent to the terminal device. When the network becomes stable at a certain moment, the terminal device may receive the encoded data of the second frame and the third frame simultaneously, thus causing overload, as Figure 4b shown. In addition, it is also possible that the decoding speed of the terminal device cannot keep up with the encoding speed of the server. The encoded data sent by the server to the terminal device within a unit time cannot be decoded in time by the terminal device due to the limitation of the hardware computing power, thus causing overload.

[0064] For the load information corresponding to each time point within the time window, it can be judged in the above way to determine whether each time point is overloaded, and finally the overload times of the terminal device within the time window can be obtained. For example, the terminal device is detected at 10 time points within a 1-second time window, and the amount of data to be decoded at the 10 time points is obtained. By comparing the amount of data to be decoded with the data volume threshold, it is obtained that the terminal device is overloaded at the first time point, the third time point, and the fifth time point within the time window, and not overloaded at other times, that is, it is determined that the terminal device is overloaded 3 times within the time window.

[0065] An overload index value can be set, and the decoding abnormality of the terminal device can be determined by judging whether the overload times within the time window reach the overload index value. In one implementation, when the overload times reach the overload index value, it is determined that the decoding abnormality of the terminal device is frequent overload. When the overload times are greater than 1 and do not reach the overload index value, it is determined that the decoding abnormality of the terminal device is occasional overload.

[0066] When the number of overloads reaches the overload index value, that is, when the number of overloads is greater than or equal to the overload index value, it indicates that the terminal device has experienced high decoding loads multiple times within the time window. It can be determined that the decoding anomaly existing in the terminal device is frequent overload, and the current decoding pressure of the terminal device is too high. When the number of overloads does not reach the overload index value, that is, when the number of overloads is less than the overload index value and greater than 1, it indicates that the terminal device has experienced overload within the time window (for example, only one overload), but the high decoding load is sporadic. It can be determined that the decoding anomaly existing in the terminal device is sporadic overload, and the terminal device has sporadic excessive decoding pressure. It can be understood that when the number of overloads is equal to 0, it is determined that the terminal device does not have a decoding anomaly, that is, the terminal device does not have an overload situation. The above-mentioned frequent overload and sporadic overload both belong to decoding anomaly situations, and these two decoding anomaly situations can be collectively referred to as overload anomaly situations. When the decoding anomaly situation existing in the terminal device includes the overload anomaly situation, it usually refers to frequent overload or sporadic overload.

[0067] It can be understood that the detection methods involved in detecting the decoding anomaly situation existing in the terminal device described above can also be applied in the terminal device to detect the decoding load situation and network fluctuation situation of the terminal device, and then obtain the decoding anomaly situation existing in the terminal device. The decoding anomaly situation existing in the terminal device can be directly sent by the terminal device to the server, or the terminal device can periodically send relevant information (such as load information) to the server, and the server determines the decoding anomaly situation existing in the terminal device. For example, when the terminal device detects that the load information exceeds the load threshold, it can be determined that the terminal device has an overload, and the counted number of overloads is sent to the server after the end of a time window, or when an overload occurs once within a time window, an information indicating that the terminal device has an overload is sent to the server, and the server counts the number of overloads within the time window.

[0068] In one implementation, the encoding strategy includes a hierarchical encoding strategy and a frame skipping encoding strategy; when the decoding anomaly situation is a network fluctuation anomaly situation, the corresponding recommended encoding strategy includes the hierarchical encoding strategy; when the decoding anomaly situation is an overload anomaly situation, the corresponding recommended encoding strategy includes the frame skipping encoding strategy.

[0069] Specifically, when the decoding anomaly is a network fluctuation anomaly, it can specifically be a frequent fluctuation anomaly or an occasional fluctuation anomaly. When the decoding anomaly is an overload anomaly, it can specifically be frequent overload or occasional overload. The correspondence between different decoding anomalies and encoding strategies can be set as follows: The encoding strategy corresponding to the decoding anomaly of frequent fluctuation anomaly is the hierarchical encoding strategy, and the encoding strategy corresponding to the decoding anomaly of occasional overload is the frame skipping encoding strategy; the encoding strategy corresponding to the decoding anomaly of occasional fluctuation anomaly is the frame skipping encoding strategy, and the encoding strategy corresponding to the decoding anomaly of frequent overload is the hierarchical encoding strategy.

[0070] It can be seen that the encoding strategies corresponding to different decoding anomalies are different. In this way, when there is a decoding anomaly in the terminal device, based on the above correspondence, the corresponding encoding strategy can be used as the recommended encoding strategy in a targeted manner to overcome different decoding anomalies. For example, when the decoding anomaly is a frequent fluctuation anomaly included in the network fluctuation anomaly, the corresponding recommended encoding strategy includes the hierarchical encoding strategy. When the decoding anomaly is an occasional overload included in the overload anomaly, the corresponding recommended encoding strategy is the frame skipping encoding strategy.

[0071] Among them, the hierarchical encoding strategy refers to the strategy of encoding in a hierarchical encoding manner. Hierarchical coding (Scalable Video Coding, SVC) is a video coding technology. Hierarchical coding encodes the video signal into a hierarchical form. When the bandwidth is insufficient, only the bitstream of the base layer is transmitted and decoded, but the decoded video quality is not high at this time. When the bandwidth gradually increases, the bitstream of the enhancement layer can be transmitted and decoded to improve the decoding quality of the video. Among them, the video frames in the base layer are the video frames that cannot be discarded in hierarchical coding and are used as references for subsequent video frames. If discarded, it will cause decoding failure or screen freeze; the video frames in the enhancement layer are the video frames that can be discarded at will in hierarchical coding and are not used as references for subsequent video frames. If discarded, it has no impact on the decoder that supports SVC. When the hierarchical encoding strategy is adopted, the video frames in the base layer and the enhancement layer can be transmitted through different channels to reduce the network transmission pressure.

[0072] The frame skipping encoding strategy refers to the strategy of encoding in a frame skipping encoding manner. Frame skipping encoding means that before encoding a certain frame, by specifying a specific reference frame or masking a specific reference frame, the reference frame dependency of the current frame is temporarily modified. For example, if the current frame defaults to referring to the previous 2 frames, flexible frame skipping can modify the current frame to refer to the third and second previous frames, or by masking the previous 1 frame of the current frame, so that the current frame refers to the third and second previous frames. The above different encoding strategies can reflect different management methods of reference frames and specifically correspond to different frame dropping schemes.

[0073] In the scenario of unstable network, the configuration of hierarchical coding can be enabled, and the hierarchical coding strategy can be used for coding. Some reference frames in the enhancement layer can be discarded during transmission to reduce the pressure of network transmission. Since enabling hierarchical coding will lead to a decrease in coding quality during the enabling period, for instantaneous network fluctuations or occasional overloads, enabling hierarchical coding will cause a decrease in coding quality for extra time. That is, the use of the hierarchical coding strategy is more applicable to scenarios with long-term unstable network or frequent overloads. Therefore, the coding strategy corresponding to frequent fluctuation anomalies or frequent overloads can be set as the hierarchical coding strategy, while for occasional decoding load being too high or instantaneous network fluctuations, some reference frames can be discarded through flexible frame skipping coding. In dealing with occasional decoding freezes, when using frame skipping coding, only the coding quality will slightly decrease instantaneously, without affecting the coding quality at other times, thus ensuring the overall picture quality of the terminal device.

[0074] The coding strategy corresponding to the decoding anomaly being frequent fluctuation anomaly or frequent overload is the hierarchical coding strategy, and the coding strategy corresponding to the decoding anomaly being occasional fluctuation anomaly or occasional overload is the frame skipping coding strategy. It can be seen that the coding strategies corresponding to different decoding anomalies can also be the same, that is, both frequent fluctuation anomalies and frequent overloads can correspond to the hierarchical coding strategy, while both occasional fluctuation anomalies and occasional overloads can correspond to the frame skipping coding strategy. When it is detected that the decoding anomaly existing in the terminal device is frequent network anomaly or frequent overload, the corresponding hierarchical coding strategy can be used as the recommended coding strategy. When it is detected that the decoding anomaly existing in the terminal device is occasional fluctuation anomaly or occasional overload, the corresponding frame skipping coding strategy can be used as the recommended coding strategy. The recommended coding strategy is the coding strategy that is recommended to be tried first, and whether the recommended coding strategy can be finally used still needs to be further determined based on the matching result between the reference decoding ability and the expected decoding ability of the terminal device. In the above method, the decoding anomaly can reflect the terminal decoding speed (which can be represented by the decoding load situation) and the current network situation (such as the network fluctuation situation). For different decoding anomalies (including the anomaly of overload and the anomaly of network fluctuation), the server can automatically determine the coding strategy corresponding to the decoding anomaly according to the corresponding relationship between different decoding anomalies and coding strategies, avoiding jams caused by network fluctuations or the ability limitations of the terminal device. It can be understood that during the entire coding process of the target cloud game, the coding strategy is not fixed, but can be flexibly adjusted based on the currently detected decoding anomaly existing in the terminal device. For example, if the frame skipping coding strategy is used for coding the 4th frame, and frequent overload of the terminal device is detected before coding the 7th frame, the frame skipping coding strategy can be changed to the hierarchical coding strategy.

[0075] In a feasible manner, the abnormal situation of network fluctuation and the abnormal situation of overload can also exist simultaneously. That is, when the detected decoding abnormal situation of the terminal device includes frequent fluctuation abnormality and occasional overload, or includes frequent overload and occasional fluctuation abnormality, the hierarchical coding strategy can be used as the recommended coding strategy. That is, when comprehensively considering the network fluctuation situation and the decoding load of the terminal device, no matter what kind of decoding abnormal situation occurs, once the decoding abnormal situation corresponding to the hierarchical coding strategy appears, it is determined that the hierarchical coding strategy is used as the recommended coding strategy. This can preferentially solve the situation of long-term network instability or too high decoding load to avoid unpredictable decoding risks.

[0076] S302, obtain the reference decoding ability that can be provided when the terminal device decodes the encoded data generated by the recommended coding strategy, and obtain the expected decoding ability that the terminal device needs to possess when enabling the terminal device to normally decode the encoded data generated by the recommended coding strategy.

[0077] In one embodiment, the reference decoding ability that the terminal device can provide includes static decoding ability. For the detection process of the static decoding ability, it can include the following contents: (1) Determine the communication interface corresponding to the decoder in the terminal device, and obtain the description information of the decoder through the communication interface; (2) When the description information indicates that the decoder supports decoding the encoded data generated by the recommended coding strategy, use the number of decoding reference frames indicated in the description information as the static decoding ability that the terminal device can provide.

[0078] Specifically, the decoder in the terminal device can be a hardware chip that supports decoding encoded data. Due to different video coding and decoding standards, the types of decoders supported by the terminal device can also be different. For example, the decoder can be an H264 chip or an HEVC chip, etc. Under different video coding and decoding standards, the supported coding strategies also have differences. In this way, when the terminal device supports different types of decoders, the decoding abilities that can be provided also have certain differences. For example, the H264 chip supports hierarchical coding and decoding, while other chips do not. The server can send a acquisition request to the communication interface through the determined communication interface corresponding to the decoder, and the terminal device can respond to the acquisition request and feedback the description information about the decoder to the server. The description information includes: information indicating whether the decoder supports decoding the encoded data generated by the recommended coding strategy, and information indicating the number of decoding reference frames provided when supported.

[0079] Based on the description information of the decoder, it can be known whether the decoder supports decoding the encoded data generated by the recommended encoding strategy, and the number of decoding reference frames provided by the decoder when it supports decoding the encoded data generated by the recommended encoding strategy. Among them, the recommended encoding strategy can be one or more of the hierarchical encoding strategy and the frame skipping encoding strategy. The number of decoding reference frames refers to the number of reference frames required to decode one frame. Usually, the number of reference frames required to decode one frame can take values from 0 to 8, or 0 to 16. The number of decoding reference frames provided when the terminal device supports can include the maximum number of decoding reference frames provided when the terminal device supports.

[0080] When the description information indicates that the decoder supports decoding the encoded data generated by the recommended encoding strategy, it shows that the decoder in the terminal device has the ability to decode the encoded data generated by the recommended encoding strategy. Further, to more accurately determine whether the recommended encoding strategy is available, the server can use the number of decoding reference frames provided when the terminal device supports as the static decoding ability that the terminal device can provide. When the description information indicates that the decoder does not support decoding the encoded data generated by the recommended encoding strategy, it shows that the decoder in the terminal device does not have the ability to decode the encoded data generated by the recommended encoding strategy. At this time, the dynamic decoding ability of the terminal device can be obtained, and whether the recommended encoding strategy is available can be further determined through the dynamic decoding ability. For the acquisition of the dynamic decoding ability, refer to the following embodiments.

[0081] In another embodiment, the reference decoding ability that the terminal device can provide includes the dynamic decoding ability. The process of detecting the dynamic decoding ability can include the following: First, the server can send the first probe bitstream generated by the hierarchical encoding strategy and the second probe bitstream generated by the frame skipping encoding strategy to the terminal device; then, the server can obtain from the terminal device the first decoding result of decoding each probe segment of the first probe bitstream and the second decoding result of decoding each probe segment of the second probe bitstream; finally, the server can generate the dynamic decoding ability that the terminal device can provide according to the first decoding result and the second decoding result.

[0082] The server can send probe bitstreams generated by different encoding strategies to the terminal device. The probe bitstream can be a video bitstream for detecting the dynamic decoding ability of the terminal device and can be obtained by encoding and compressing any video data. Both the first probe bitstream and the second probe bitstream contain multiple probe segments, and different probe segments are encoded using different numbers of encoding reference frames. The number of probe segments included in the first probe bitstream and the number of probe segments included in the second bitstream can be the same or different. The numbers of encoding reference frames for different probe segments in the same probe bitstream are different. The encoding reference frame refers to the reference frame required for encoding one frame, and the encoding reference frame can be an I frame or a P frame. For example, the first probe bitstream includes 2 probe segments generated by hierarchical encoding, and the numbers of encoding reference frames used for each probe segment are 4 and 8 respectively; the second probe bitstream includes 3 probe segments generated by frame skipping encoding, and the numbers of encoding reference frames used for each probe segment are 4, 8, and 16 respectively.

[0083] After receiving each probe segment of the first probe bitstream and each probe segment of the second probe bitstream, the terminal device can perform decoding processing on each probe segment included in different probe bitstreams to obtain a decoding result. The decoding result is used to indicate whether the corresponding probe segment is decoded normally (i.e., whether there is an error prompt or stuttering). The server can obtain the first decoding result and the second decoding result from the terminal device. The first decoding result includes the decoding results obtained by the terminal device for decoding each probe segment of the first probe bitstream, and the second decoding result includes the decoding results obtained by the terminal device for decoding each probe segment of the second probe bitstream. Based on the decoding results obtained by performing decoding processing on each probe segment of the acquired first probe bitstream and the decoding results obtained by performing decoding processing on each probe segment of the second probe bitstream, the server can determine whether the terminal device supports normal decoding of the encoded data generated by the recommended encoding strategy, and then generate the dynamic decoding ability that the terminal device can provide.

[0084] In one implementation, according to the first decoding result and the second decoding result, the dynamic decoding capabilities that can be provided by the terminal device are generated, including: associating and storing the decoding description information of each detection segment of the first detection stream by the terminal device recorded in the first decoding result, and the number of coding reference frames corresponding to each detection segment of the first detection stream, as the dynamic decoding capabilities that the terminal device can provide when supporting the hierarchical coding strategy; associating and storing the decoding description information of each detection segment of the second detection stream by the terminal device recorded in the second decoding result, and the number of coding reference frames corresponding to each detection segment of the second detection stream, as the dynamic decoding capabilities that the terminal device can provide when supporting the frame skipping coding strategy. Among them, associative storage means associating and storing the decoding description information indicating whether a detection segment is successfully decoded and the number of coding reference frames corresponding to the detection segment. By associative storage, it is convenient to obtain all information about a detection segment in one stop, which is beneficial to judging whether the recommended coding strategy is available. The dynamic decoding capabilities that can be provided include one or more of the decoding capabilities that the terminal device can provide when supporting the hierarchical coding strategy and the decoding capabilities that the terminal device can provide when supporting the frame skipping coding strategy. Therefore, the dynamic decoding capabilities that can be provided include one or more of the following: ① the decoding description information of each detection segment in the first detection stream, and the number of coding reference frames of each detection segment ② the decoding description information of each detection segment in the second detection stream, and the number of coding reference frames of each detection segment. Among them, the decoding description information is used to indicate whether the terminal device successfully decodes the corresponding detection segment.

[0085] The dynamic decoding capability can indicate that the decoder in the terminal device can support decoding the encoded data of the hierarchical coding strategy, and the maximum number of encoded reference frames when the decoding is successful (i.e., no stuttering occurs). For example, the provided dynamic decoding capability can be used to indicate that the terminal device successfully decodes the probe segment D generated by the hierarchical coding strategy, and the number of encoded reference frames corresponding to the probe segment D is 8. If the decoding description information recorded in the first decoding result includes the information of successfully decoding at least one probe segment of the first probe bitstream, it indicates that the terminal device supports the hierarchical coding strategy, that is, it supports decoding the encoded data obtained by using the hierarchical coding strategy. However, the maximum limit of the number of encoded reference frames that the terminal device can support for decoding is determined by the probe segment corresponding to the maximum number of encoded reference frames in at least one successfully decoded probe segment. That is, the maximum number of decoded reference frames supported by the decoder is equal to the maximum value of the number of encoded reference frames of the stored successfully decoded probe segments. That is to say, for the same recommended coding strategy (such as the hierarchical coding strategy), in the dynamic decoding capability provided by the terminal device when supporting the recommended coding strategy (such as the hierarchical coding strategy), when all probe segments are successfully decoded or in the case where some probe segments are successfully decoded, due to different successfully decoded probe segments, the maximum number of decoded reference frames supported by the terminal device may be the same or different. For example, the first probe bitstream encoded with different numbers of encoded reference frames using the hierarchical coding strategy includes 3 probe segments, namely probe segment D1, probe segment D2, and probe segment D3, and the corresponding numbers of encoded reference frames are 8, 6, and 4 respectively. The decoding description information of probe segment D2 and probe segment D3, the decoding description information of the failed-to-decode probe segment D1, and the number of encoded reference frames corresponding to each probe segment are stored as the dynamic decoding capability that the terminal device can provide when supporting the hierarchical coding strategy. If only probe segment D2 and probe segment D3 are successfully decoded, the maximum number of decoded reference frames supported by the decoder is 6. If all the above 3 probe segments can be successfully decoded, the maximum number of decoded reference frames supported by the decoder is 8.

[0086] It can be understood that if the decoding description information recorded in the first decoding result is the information of decoding failure for each probe segment of the first probe bitstream, it indicates that the terminal device does not support the hierarchical coding strategy. At this time, only the decoding description information in the first decoding result can be stored as the dynamic decoding capability provided by the terminal device, and this dynamic decoding capability indicates that the decoder does not support the hierarchical coding strategy. The same applies to the second decoding result. When all probe segments are successfully decoded or in the case where some probe segments are successfully decoded, the number of encoded reference frames stored in the dynamic decoding capability provided by the terminal device when supporting the frame skipping coding strategy is different, and the maximum number of decoded reference frames supported may be the same or different.

[0087] It should be noted that the detection of the static decoding ability and the dynamic decoding ability of the terminal device can be performed before encoding. The relevant information obtained from the detection is stored in the database of the server and retrieved from the corresponding database when needed. Before processing the frame to be encoded, it is necessary to detect the decoding anomalies existing in the terminal device. Since the static decoding ability and the dynamic decoding ability are the inherent abilities of the terminal device itself, they can be reused after one detection.

[0088] In addition to obtaining the reference decoding ability of the terminal device, the server can also obtain the expected decoding ability that the terminal device needs to possess when the terminal device performs normal decoding processing on the encoded data generated by the recommended encoding strategy. Since when the terminal device performs normal decoding processing on the encoded data generated by the recommended encoding strategy, there will be no decoding stuttering or error prompt during the decoding process, the decoding anomalies existing in the terminal device can be overcome. The expected decoding ability is the basic decoding ability required for the terminal device to overcome the decoding anomalies. The expected decoding ability can be set by the server based on the corresponding recommended encoding strategy. For example, when the recommended encoding strategy is the hierarchical encoding strategy, the server requires a corresponding number of reference frames to enable the hierarchical encoding strategy. Then, the maximum number of decoding reference frames supported by the terminal device needs to reach at least the required number of reference frames in order to perform normal decoding on the encoded data obtained using the hierarchical encoding strategy.

[0089] In one embodiment, before performing the following step S303, that is, matching the reference decoding ability and the expected decoding ability to obtain a matching result, it is also possible to first determine whether the recommended encoding strategy can be supported by the server according to the relevant information of the recommended encoding strategy.

[0090] (1) If the recommended encoding strategy is the hierarchical encoding strategy, it is possible to: obtain the support information regarding the hierarchical encoding strategy, and when the support information regarding the hierarchical encoding strategy indicates support for the hierarchical encoding strategy, trigger the execution of the step of matching the reference decoding ability and the expected decoding ability. Specifically, the support information regarding the hierarchical encoding strategy is used to characterize whether the encoder in the server supports the hierarchical encoding strategy. When the support information indicates that the server supports the hierarchical encoding strategy, the step of matching the reference decoding ability and the expected decoding ability can be executed, and the availability of the hierarchical encoding strategy can be determined by comparing the actual decoding ability of the terminal device with the expected decoding ability that the terminal device is required to achieve.

[0091] In one implementation, when the support information regarding the hierarchical encoding strategy indicates that the hierarchical encoding strategy is not supported, the frame skipping encoding strategy is used as the new recommended encoding strategy. That is to say, when the recommended encoding strategy is the hierarchical encoding strategy but the hierarchical encoding strategy is not supported for use, to solve the decoding anomaly of the terminal device and ensure the stable output of the picture in the terminal device, a new encoding strategy can be adopted, which here refers to the frame skipping encoding strategy. After that, the determination of whether the frame skipping encoding strategy can ultimately be used for encoding processing is the same as that of the hierarchical encoding strategy, that is: when the support information regarding the frame skipping encoding strategy indicates support for frame skipping, the step of matching the reference decoding capability and the expected decoding capability can be triggered for execution. Conversely, other solutions can be adopted for encoding processing. For example, the server can apply for an IDR frame (Instantaneous Decoding Refresh) to be inserted before the game frame to be processed, and the game frame to be processed is encoded with reference to only this IDR frame. It should be noted that since the IDR frame is usually significantly larger than the P frame, re-applying for an IDR frame may cause further network lag. Therefore, when the network is lagging or the decoding is lagging, the recommended encoding strategy (either the hierarchical encoding strategy or the adjusted encoding strategy) is usually preferred to replace the IDR frame with a P frame, avoiding the possible further network lag caused by the IDR frame and allowing some of the already encoded frames to be discarded. However, if the server does not support the recommended encoding strategy, to advance the decoding progress, an IDR frame can also be used to resolve the lag problem as much as possible.

[0092] (2) If the recommended encoding strategy is the frame skipping encoding strategy, the following steps ① to ③ may further be included:

[0093] ① Determine the reference quantity corresponding to the reference frame to be masked and the reference frame list corresponding to the target cloud game according to the current network condition of the terminal device.

[0094] The server can judge the current network condition of the network environment where the terminal device is located. The current network condition includes either the network fluctuation condition or the network stable condition. The network stable condition indicates that the network environment is stable, and the network fluctuation condition indicates that the network may have experienced lag. For each encoded reference frame transmitted to the terminal device, it can be determined whether to mask (i.e., discard) it based on the number of lost packets or the transmission delay, thereby determining the reference frame to be masked (i.e., the reference frame that needs to be discarded) and the reference quantity (i.e., the number of reference frames that need to be discarded). The reference frame list corresponding to the target cloud game includes the reference frames required for the game frame to be processed. For example, Figure 5aAs shown in the figure, before encoding the 6th frame, the reference frame list contains 4 encoded reference frames required for the 6th frame (i.e., the 2nd to 5th frames). Due to the influence of the current network condition (such as network lag), the 5th frame is lost when transmitted to the terminal device, and after attempting to retransmit, it is lost 3 times, reaching the packet loss threshold. Then, it can be determined that the 5th frame is the reference frame to be masked. Each frame is judged in turn, and it is determined that the 5th frame and the 4th frame need to be masked. The 3rd frame and the 2nd frame are normally transmitted to the terminal device, that is, the reference quantity corresponding to the reference frame to be masked is 2. After the reference frame to be masked is masked, the encoding of the frame to be encoded does not depend on the masked encoded reference, that is, the 5th frame depends on the 2nd frame and the 3rd frame for encoding. Among them, masking the reference frame to be masked means not using it as a reference frame during both encoding and decoding. The masked frame can be transmitted to the terminal device and not used as a reference frame during decoding, or it can directly not be transmitted to the terminal device.

[0095] ② When determining to support the frame skipping encoding strategy, based on the total number of reference frames included in the reference frame list and the reference quantity, determine whether to support masking the reference frames with the reference quantity in the target cloud game.

[0096] The support information indicating whether the encoder supports the frame skipping encoding strategy can be obtained through the communication interface of the encoder in the server. When determining to support the frame skipping encoding strategy, it indicates that the server has the ability to support specifying specific reference frames or masking reference frames. It can be judged whether the reference frame list supports implementing the masking of the reference frames with the reference quantity. Specifically, it can be determined by comparing the total number of reference frames included in the reference frame list and the reference quantity corresponding to the reference frame to be masked. If the reference quantity corresponding to the reference frame to be masked is greater than or equal to the total number of reference frames included in the reference frame list, that is, after masking the reference frames with the reference quantity, there are no reference frames in the reference frame list. Since the normal encoding of the frame to be encoded needs to be ensured, at this time, the server can apply for an IDR frame to be used as the reference frame of the frame to be encoded to ensure the normal output of the picture. Therefore, in order to ensure the use of the frame skipping encoding strategy, usually after masking the reference frames with the reference quantity, it is necessary to ensure that there are still reference frames in the reference frame list and no IDR frame appears, corresponding to the requirement that the reference quantity corresponding to the reference frame to be masked is less than the total number of reference frames in the reference frame list.

[0097] The server supporting the frame skipping encoding strategy correspondingly supports masking a preset quantity of reference frames. The preset quantities of reference frames that different servers support masking may vary. For any server, if the reference quantity is less than the total number of reference frames in the reference frame list, and the server has a limited number of reference frames that can be masked, even if the reference quantity is less than the total number of reference frames included in the reference frame list, since the server does not support masking more reference frames than its own limit, that is, the server does not support masking the reference frames with the reference quantity. Specifically, the reference frame list does not support implementing the masking of the reference frames with the reference quantity, and the frame skipping encoding strategy is unavailable.

[0098] ③When it is determined to support masking the reference frames with the reference quantity in the target cloud game, trigger the execution of matching the reference decoding capability and the expected decoding capability. Specifically, when the server determines that it can support masking the reference frames with the reference quantity in the target cloud game, it indicates that the server supports the frame skipping encoding strategy, and thus the reference decoding capability and the expected decoding capability can be matched. Specifically, it can be the matching between any one or more of the dynamic decoding capability and the static decoding capability and the expected decoding capability to obtain a matching result.

[0099] Whether it is the hierarchical encoding strategy or the frame skipping encoding strategy, it is possible to discard some of the already encoded frames, so that when encoding P frames, they no longer depend on the frames to be discarded. At the same time, when either of the two encoding strategies can be used, the scheme of inserting IDR frames can be avoided. Since the size of the IDR frame is significantly larger than that of the P frame, the re - application of the IDR frame may cause further network jamming. When neither of the two encoding strategies can be used, in order to prevent the terminal device from jamming in the decoding of a certain frame and to advance the decoding progress of the terminal device, therefore, the scheme of inserting IDR frames can be adopted when neither the hierarchical encoding strategy nor the frame skipping encoding strategy can be used.

[0100] In one implementation, when it is determined not to support masking the reference frames with the reference quantity in the target cloud game, it is possible to: select a new encoded reference frame from the processed game frames of the target cloud game, and use the selected new encoded reference frame as the immediate refresh frame of the game frames to be processed in the target cloud game, so as to perform encoding processing on the game frames to be processed based on the immediate refresh frame.

[0101] Among them, the method of selecting a new encoded reference frame from the processed game frames of the target cloud game can be: determine the group of frame sequences where the processed game frames of the target cloud game are located, and determine the first intra - frame encoded reference frame in the group of frame sequences as the new encoded reference frame. The group of frame sequences contains multiple frames of images. During the encoding process, the processed game frames are usually divided into multiple groups of frame sequences for transmission. The group of frame sequences can also be called a group of pictures. Exemplarily, if a group of frame sequences contains IPPPP, and the currently encoded 4th frame references the previous 3 P frames, when decoding jamming occurs, the first I frame can be used as the immediate refresh frame (IDR frame). Starting from the IDR frame, a new sequence can be recalculated for encoding. By using the selected new encoded reference frame as the immediate refresh frame, the immediate refresh frame can be inserted before the game frames to be processed in the target cloud game, so that the game frames to be processed no longer reference the original reference frames, but only reference this IDR frame for encoding processing. Through the setting of the IDR, the encoded reference frames before the insertion position of the IDR frame can be discarded at will.

[0102] S303, match the reference decoding capability and the expected decoding capability to obtain a matching result.

[0103] In one embodiment, the reference decoding capabilities that the terminal device can provide include: static decoding capabilities and the dynamic decoding capabilities that can be provided. The server can match the static decoding capabilities with the expected decoding capabilities, and match the dynamic decoding capabilities with the expected decoding capabilities, and obtain the final matching result by synthesizing the respective matching results of the static decoding capabilities and the dynamic decoding capabilities. The dynamic decoding capabilities that can be provided can include the dynamic decoding capabilities provided under the support of the hierarchical coding strategy, and the dynamic decoding capabilities provided under the support of the frame skipping coding strategy.

[0104] In the case where the reference decoding capabilities include static decoding capabilities and dynamic decoding capabilities, the method of matching the reference decoding capabilities with the expected decoding capabilities to obtain the matching result may include the following:

[0105] First, obtain the number of decoding reference frames indicated by the expected decoding capabilities, and obtain the number of decoding reference frames that the terminal device can provide from the static decoding capabilities. Specifically, the number of decoding reference frames indicated by the expected decoding capabilities is a condition for ensuring the normal decoding of the encoded data generated by the recommended coding strategy by the terminal device. The matching between the expected decoding capabilities and the static decoding capabilities specifically means that the number of decoding reference frames that the terminal device can provide reaches the expected number of decoding reference frames. The expected number of decoding reference frames can be determined by the number of encoding reference frames required under the coding strategy supported by the server. For example, under the support of the frame skipping coding strategy, the server supports masking A1 number of encoding reference frames in the reference frame list, then the expected number of decoding reference frames is A1, and under the support of the hierarchical coding strategy, the server sets A2 number of encoding reference frames required for hierarchical coding, then the expected number of decoding reference frames is A2.

[0106] Then, use the quantity matching result obtained by matching the number of decoding reference frames that the terminal device can provide with the expected number of decoding reference frames as the matching result of the static decoding capabilities and the expected decoding capabilities. Specifically, the server can compare the magnitudes of the number of decoding reference frames that can be provided and the expected number of decoding reference frames to obtain the quantity matching result, and this quantity matching result is used to indicate whether the number of decoding reference frames that can be provided reaches the expected number of decoding reference frames. Reaching the expected number of decoding reference frames means that the number of decoding reference frames that can be provided is greater than or equal to the expected number of decoding reference frames, and not reaching the expected number of decoding reference frames means that the number of decoding reference frames that can be provided is less than the expected number of decoding reference frames. The quantity matching result can be used as the matching result obtained by matching the static decoding capabilities and the expected decoding capabilities, so as to know whether the static decoding capabilities and the dynamic decoding capabilities are successfully matched.

[0107] Finally, from the available dynamic decoding capabilities, a target detection segment with the number of encoded reference frames greater than or equal to the expected number of decoding reference frames, and the decoding description information of the target detection segment are matched, and the matched decoding description information is used as the matching result of the dynamic decoding capability and the expected decoding capability. Specifically, the server can match the target detection segment from each detection segment included in the dynamic decoding capabilities that the terminal device can provide. The number of encoded reference frames included in the target detection segment is greater than or equal to the number of decoding reference frames expected by the expected decoding capability. If the recommended encoding strategy is the hierarchical encoding strategy, the matched target detection segment is the detection segment in the first detection bitstream generated by the hierarchical encoding strategy. If the recommended encoding strategy is the frame skipping encoding strategy, the matched target detection segment is the detection segment in the second detection bitstream generated by the frame skipping encoding strategy. The decoding description information of the matched target detection segment is used to indicate whether the target detection segment is decoded successfully, and can be used as the matching result between the dynamic decoding capability and the expected decoding capability to indicate whether the dynamic decoding capability and the expected decoding capability match successfully. When the decoding description information of the matched target detection segment indicates that the target detection segment is successfully decoded, the matching result indicates that the dynamic decoding capability and the expected decoding capability match successfully; otherwise, the matching result indicates that the dynamic decoding capability and the expected decoding capability match fails.

[0108] In a feasible implementation manner, when the quantity matching result indicates that the number of decoding reference frames that the terminal device can provide is greater than or equal to the expected number of decoding reference frames, and the matched decoding description information indicates that the terminal device has successfully decoded the target detection segment, it indicates that the terminal device can at least successfully decode the encoded data including the expected number of decoding reference frames under the recommended encoding strategy, and it is determined that the matching result indicates that the reference decoding capability and the expected decoding capability match successfully.

[0109] When the quantity matching result indicates that the number of decoding reference frames that the terminal device can provide is greater than or equal to the expected number of decoding reference frames, it indicates that the static decoding capability and the expected decoding capability match successfully, and when the matched decoding description information indicates that the terminal device has successfully decoded the target detection segment, it indicates that the dynamic decoding capability and the expected decoding capability match successfully. When both the static decoding capability and the dynamic decoding capability included in the reference decoding capability match successfully with the expected decoding capability, it is determined that the reference decoding capability and the expected decoding capability match successfully. Otherwise, when any one of the static decoding capability and the dynamic decoding capability included in the reference decoding capability fails to match with the expected decoding capability, it is determined that the reference decoding capability and the expected decoding capability match fails.

[0110] S304. When the matching result indicates that the reference decoding capability and the expected decoding capability match successfully, the recommended encoding strategy is used to encode the game frames to be processed in the target cloud game.

[0111] When the recommended encoding strategy is the hierarchical encoding strategy, the encoding process for the game frame to be processed in the target cloud game can be to appropriately discard the encoding reference frames belonging to the enhancement layer that the game frame to be processed refers to, and transmit the encoding reference frames of different layers through different channels to reduce the network transmission pressure. When the recommended encoding strategy is the frame skipping encoding strategy, the encoding reference frames referred to by the game frame to be processed in the target cloud game can be appropriately adjusted. For example, one of the four encoding reference frames referred to by the game frame to be processed is masked, and then the game frame to be processed can be encoded with reference to the remaining three frames.

[0112] S305. When the matching result indicates that the reference decoding ability fails to match the expected decoding ability, select a new encoding reference frame from the processed game frames of the target cloud game, and use the selected new encoding reference frame as the immediate refresh frame of the game frame to be processed in the target cloud game, so as to perform the encoding process on the game frame to be processed based on the immediate refresh frame.

[0113] When the matching result indicates that the reference decoding ability fails to match the expected decoding ability, it means that the server can neither adopt the hierarchical encoding strategy nor the frame skipping encoding strategy. When neither of these two encoding strategies can be adopted, the game frame to be processed can be encoded with the immediate refresh frame. The specific processing principle can refer to the content introduced above and will not be elaborated here. For the above content, refer to Figure 5b the schematic flowchart of the encoding process shown in

[0114] The encoding processing solution provided by the embodiments of this application can detect decoding anomalies existing in a terminal device. Specifically, it can determine the decoding anomaly based on the corresponding number of network fluctuations or overload occurrences within a time window. This decoding anomaly can reflect the real-time decoding state of the terminal device. Based on the real-time decoding state, an encoding strategy can be flexibly recommended for encoding processing, realizing intelligent management of reference frames and ensuring smooth decoding and stable output of the screen on the terminal device. Among them, for different network fluctuation anomalies (such as frequent fluctuations and occasional fluctuations) or different overload anomalies (including frequent overloads and occasional overloads), different encoding strategies can be provided for frame dropping to adapt to different network fluctuation conditions and real-time decoding capabilities. In addition, static and dynamic capabilities of the decoding ability of the terminal device can be detected to obtain the static decoding ability and the dynamic decoding ability. The server can adaptively determine the availability of the recommended encoding strategy based on the decoding ability and network conditions provided by the terminal device, thereby making full use of the decoding ability of the terminal device and avoiding decoding anomalies. When the matching result indicates that the reference decoding ability does not match the expected decoding ability, other encoding strategies can be used to encode the game frames to be processed in the target cloud game. In this way, the recommended encoding strategy can also be adaptively adjusted based on the decoding ability of the terminal device, and the flexibility of using the encoding strategy is high.

[0115] Please refer to Figure 6 , Figure 6 FIG. is a schematic structural diagram of an encoding processing device provided by an embodiment of this application. The above encoding processing device can be a computer program (including program code) running on a server. For example, the encoding processing device is an application software. The encoding processing device can be used to execute the corresponding steps in the method provided by the embodiments of this application. As Figure 6 shown, the encoding processing device 600 can include at least one of the following: a determination module 601, an acquisition module 602, a matching module 603, and an encoding module 604.

[0116] The determination module 601 is configured to, when detecting a decoding anomaly existing in the terminal device, based on the corresponding relationship between different decoding anomalies and encoding strategies, use the encoding strategy corresponding to the decoding anomaly as the recommended encoding strategy. Among them, there are at least two encoding strategies, and the recommended encoding strategies corresponding to different decoding anomalies are different.

[0117] The acquisition module 602 is configured to acquire the reference decoding ability that the terminal device can provide when decoding the encoding data generated by the recommended encoding strategy, and acquire the expected decoding ability that the terminal device needs to possess to normally decode the encoding data generated by the recommended encoding strategy.

[0118] A matching module 603, configured to perform a matching process on the reference decoding capability and the expected decoding capability to obtain a matching result;

[0119] An encoding module 604, configured to, when the matching result indicates that the reference decoding capability matches the expected decoding capability successfully, perform an encoding process on the to-be-processed game frames in the target cloud game by using the recommended encoding strategy.

[0120] In one embodiment, a determining module 601 is configured to: send network detection data to a terminal device within a time window, where the time window includes multiple time points, and the network detection data sent to the terminal device at one time point is one or more; obtain detection feedback data corresponding to the network detection data sent at each time point in the time window, and generate a network detection result at the corresponding time point according to the detection feedback data; the network detection result corresponding to any time point is used to indicate whether there is a network fluctuation at the any time point; according to the network detection results corresponding to the respective time points in the time window, count the number of times of network fluctuations in the time window, and determine the decoding anomaly situation of the terminal device according to the counted number of times.

[0121] In one embodiment, the determining module 601 is configured to: obtain a fluctuation index value; when the counted number of times is greater than or equal to the fluctuation index value, determine that the decoding anomaly situation of the terminal device is a frequent fluctuation anomaly; when the counted number of times is greater than or equal to 1 and less than the fluctuation index value, determine that the decoding anomaly situation of the terminal device is an occasional fluctuation anomaly.

[0122] In one embodiment, the determining module 601 is configured to: obtain the load information of the terminal device at different time points in the time window, where the load information is the amount of data to be decoded by the terminal device at the corresponding time point; determine the number of overload times of the terminal device in the time window according to the load information corresponding to the different time points of the terminal device in the time window; when the amount of data to be decoded by the terminal device at any time point is greater than or equal to a data amount threshold, the terminal device is overloaded once at the any time point; determine the decoding anomaly situation of the terminal device according to the number of overload times; where, when the number of overload times reaches an overload index value, determine that the decoding anomaly situation of the terminal device is a frequent overload, and when the number of overload times is greater than 1 and does not reach the overload index value, determine that the decoding anomaly situation of the terminal device is an occasional overload.

[0123] In one embodiment, the encoding strategy includes a hierarchical encoding strategy and a frame skipping encoding strategy; when the decoding exception is a network fluctuation exception, the corresponding recommended encoding strategy includes the hierarchical encoding strategy; when the decoding exception is an overload exception, the corresponding recommended encoding strategy includes the frame skipping encoding strategy.

[0124] In one embodiment, the obtaining module 602 is configured to: determine a communication interface corresponding to the decoder from the terminal device, and obtain description information of the decoder through the communication interface, where the description information includes: information indicating whether the decoder supports decoding the encoded data generated by the recommended encoding strategy, and information indicating the number of decoding reference frames provided when supported; when the description information indicates that the decoder supports decoding the encoded data generated by the recommended encoding strategy, use the number of decoding reference frames indicated in the description information as the static decoding capability that the terminal device can provide.

[0125] In one embodiment, the obtaining module 602 is configured to: send a first probe bitstream generated by using a hierarchical encoding strategy and a second probe bitstream generated by using a frame skipping encoding strategy to the terminal device, where both the first probe bitstream and the second probe bitstream include a plurality of probe segments, and different probe segments are encoded with different numbers of encoding reference frames; obtain a first decoding result of decoding each probe segment of the first probe bitstream and a second decoding result of decoding each probe segment of the second probe bitstream from the terminal device; and generate the dynamic decoding capability that the terminal device can provide according to the first decoding result and the second decoding result.

[0126] In one embodiment, the obtaining module 602 is specifically configured to: associatively store the decoding description information of each probe segment of the first probe bitstream recorded in the first decoding result and the number of encoding reference frames corresponding to each probe segment of the first probe bitstream as the dynamic decoding capability that the terminal device can provide when supporting the hierarchical encoding strategy; associatively store the decoding description information of each probe segment of the second probe bitstream recorded in the second decoding result and the number of encoding reference frames corresponding to each probe segment of the second probe bitstream as the dynamic decoding capability that the terminal device can provide when supporting the frame skipping encoding strategy; where the decoding description information is used to indicate whether the terminal device successfully decodes the corresponding probe segment.

[0127] In one embodiment, if the recommended encoding strategy is a hierarchical encoding strategy, the obtaining module 602 is configured to: obtain support information regarding the hierarchical encoding strategy, and trigger the execution of matching the reference decoding capability and the expected decoding capability when the support information regarding the hierarchical encoding strategy indicates support for the hierarchical encoding strategy; the determining module 601 is configured to: when the support information regarding the hierarchical encoding strategy indicates non - support for the hierarchical encoding strategy, use the frame - skipping encoding strategy as the new recommended encoding strategy.

[0128] In one embodiment, if the recommended encoding strategy is a frame - skipping encoding strategy, the determining module 601 is configured to: determine the reference quantity corresponding to the reference frame to be masked and the reference frame list corresponding to the target cloud game according to the current network condition of the terminal device; when it is determined that the frame - skipping encoding strategy is supported, determine whether to support masking the reference frames with the reference quantity in the target cloud game according to the total number of reference frames included in the reference frame list and the reference quantity; when it is determined that the reference frames with the reference quantity in the target cloud game can be masked, trigger the execution of matching the reference decoding capability and the expected decoding capability; otherwise, select new encoding reference frames from the processed game frames of the target cloud game, and use the selected new encoding reference frames as the immediate refresh frames of the game frames to be processed in the target cloud game, so as to perform encoding processing on the game frames to be processed based on the immediate refresh frames.

[0129] In one embodiment, the reference decoding capabilities that the terminal device can provide include: static decoding capabilities and available dynamic decoding capabilities; the matching module 603 is configured to: obtain the number of decoding reference frames expected by the expected decoding capability, and obtain the number of decoding reference frames that the terminal device can provide from the static decoding capabilities; use the quantity matching result obtained by matching the number of decoding reference frames that the terminal device can provide with the expected number of decoding reference frames as the matching result between the static decoding capabilities and the expected decoding capabilities; from the available dynamic decoding capabilities, match the target detection segments whose number of encoding reference frames is greater than or equal to the expected number of decoding reference frames and the decoding description information of the target detection segments, and use the matched decoding description information as the matching result between the dynamic decoding capabilities and the expected decoding capabilities.

[0130] In one embodiment, the determining module 601 is further configured to: when the quantity matching result indicates that the number of decoding reference frames that the terminal device can provide is greater than or equal to the expected number of decoding reference frames, and the matched decoding description information indicates that the terminal device has successfully decoded the target detection segment, determine that the matching result indicates that the reference decoding capability matches the expected decoding capability successfully.

[0131] In one embodiment, the encoding module 604 is further configured to: when the matching result indicates that the matching between the reference decoding capability and the expected decoding capability fails, select a new encoded reference frame from the processed game frames of the target cloud game, and use the selected new encoded reference frame as an immediate refresh frame for the game frames to be processed of the target cloud game, so as to perform encoding processing on the game frames to be processed based on the immediate refresh frame.

[0132] It can be understood that the functions of the functional modules of the encoding processing device described in the embodiments of the present application can be specifically implemented according to the methods in the above method embodiments, and the specific implementation process can refer to the relevant descriptions of the above method embodiments, which will not be elaborated here. In addition, the beneficial effects of using the same method will not be elaborated either.

[0133] Please refer to Figure 7 , Figure 7 FIG. is a schematic structural diagram of a server provided by an embodiment of the present application. The server 700 may include an independent device (such as one or more of a server, a node, a terminal, etc.), or may include components inside an independent device (such as a chip, a software module, or a hardware module, etc.). The server 700 may include at least one processor 701 and a communication interface 702. Further optionally, the server 700 may further include at least one memory 703 and a bus 704. Among them, the processor 701, the communication interface 702, and the memory 703 are connected through the bus 704.

[0134] Among them, the processor 701 is a module for performing arithmetic operations and / or logical operations, and may specifically be a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor unit (MPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a complex programmable logic device (CPLD), a coprocessor (assisting the central processing unit to complete corresponding processing and applications), a microcontroller unit (MCU), or a combination of one or more of such processing modules.

[0135] The communication interface 702 can be used to provide information input or output for at least one processor. And / or, the communication interface 702 can be used to receive externally transmitted data and / or transmit data externally. It can be a wired link interface including, for example, an Ethernet cable, or a wireless link (Wi-Fi, Bluetooth, Universal Wireless Transmission, Vehicular Short Range Communication Technology, and other short-range wireless communication technologies, etc.) interface. The communication interface 702 can serve as a network interface.

[0136] The memory 703 is used to provide storage space, and data such as an operating system and computer programs can be stored in the storage space. The memory 703 can be one or a combination of multiple types such as random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM), etc.

[0137] At least one processor 701 in the server 700 is used to call the computer program stored in at least one memory 703 to execute the encoding processing method described in the embodiments shown in this application.

[0138] In a possible implementation manner, the processor 701 in the server 700 is used to call the computer program stored in at least one memory 703 to perform the following operations: when a decoding anomaly situation of the terminal device is detected, according to the correspondence between different decoding anomaly situations and encoding strategies, use the encoding strategy corresponding to the decoding anomaly situation as the recommended encoding strategy; where the encoding strategies include at least two types, and the recommended encoding strategies corresponding to different decoding anomaly situations are different; obtain the reference decoding ability that can be provided when the terminal device decodes the encoded data generated by the recommended encoding strategy, and obtain the expected decoding ability that the terminal device needs to possess when enabling the terminal device to normally decode the encoded data generated by the recommended encoding strategy; perform a matching process on the reference decoding ability and the expected decoding ability to obtain a matching result; when the matching result indicates that the reference decoding ability matches the expected decoding ability successfully, use the recommended encoding strategy to perform encoding processing on the game frame to be processed in the target cloud game.

[0139] In one embodiment, the processor 701 is configured to: send network probe data to the terminal device within a time window, where the time window includes multiple time points, and one or more network probe data are sent to the terminal device at one time point; obtain probe feedback data corresponding to the network probe data sent at each time point within the time window, and generate a network probe result at the corresponding time point according to the probe feedback data; the network probe result corresponding to any time point is used to indicate whether network fluctuations occur at the any time point; according to the network probe results respectively corresponding to each time point within the time window, count the number of times of network fluctuations within the time window, and determine the decoding exception situation existing in the terminal device according to the counted number of times.

[0140] In one embodiment, the processor 701 is configured to: obtain a fluctuation index value; when the counted number of times is greater than or equal to the fluctuation index value, determine that the decoding exception situation existing in the terminal device is frequent fluctuation exception; when the counted number of times is greater than or equal to 1 and less than the fluctuation index value, determine that the decoding exception situation existing in the terminal device is occasional fluctuation exception.

[0141] In one embodiment, the processor 701 is configured to: obtain the load information of the terminal device at different time points within the time window, where the load information is the amount of data to be decoded by the terminal device at the corresponding time point; determine the number of overload times of the terminal device within the time window according to the load information respectively corresponding to different time points within the time window; when the amount of data to be decoded by the terminal device at any time point is greater than or equal to the data amount threshold, the terminal device is overloaded once at the any time point; determine the decoding exception situation existing in the terminal device according to the number of overload times; where, when the number of overload times reaches the overload index value, determine that the decoding exception situation existing in the terminal device is frequent overload, and when the number of overload times is greater than 1 and does not reach the overload index value, determine that the decoding exception situation existing in the terminal device is occasional overload.

[0142] In one embodiment, the encoding strategy includes a hierarchical encoding strategy and a frame skipping encoding strategy; when the decoding exception situation is an exception situation of network fluctuations, the corresponding recommended encoding strategy includes the hierarchical encoding strategy; when the decoding exception situation is an exception situation of overload, the corresponding recommended encoding strategy includes the frame skipping encoding strategy.

[0143] In one embodiment, the processor 701 is configured to: determine a communication interface corresponding to the decoder from the terminal device, and obtain description information of the decoder through the communication interface, where the description information includes: information indicating whether the decoder supports decoding the encoded data generated by the recommended encoding strategy, and information indicating the number of decoding reference frames provided when supported; when the description information indicates that the decoder supports decoding the encoded data generated by the recommended encoding strategy, use the number of decoding reference frames indicated in the description information as the static decoding capability that the terminal device can provide.

[0144] In one embodiment, the processor 701 is configured to: send a first probe bitstream generated using a hierarchical encoding strategy and a second probe bitstream generated using a frame skipping encoding strategy to the terminal device, where both the first probe bitstream and the second probe bitstream include multiple probe segments, and different probe segments are encoded using different numbers of encoding reference frames; obtain a first decoding result of decoding each probe segment of the first probe bitstream and a second decoding result of decoding each probe segment of the second probe bitstream from the terminal device; generate the dynamic decoding capability that the terminal device can provide according to the first decoding result and the second decoding result.

[0145] In one embodiment, the processor 701 is specifically configured to: associatively store the decoding description information of each probe segment of the first probe bitstream recorded in the first decoding result and the number of encoding reference frames corresponding to each probe segment of the first probe bitstream as the dynamic decoding capability that the terminal device can provide when supporting the hierarchical encoding strategy; associatively store the decoding description information of each probe segment of the second probe bitstream recorded in the second decoding result and the number of encoding reference frames corresponding to each probe segment of the second probe bitstream as the dynamic decoding capability that the terminal device can provide when supporting the frame skipping encoding strategy; where the decoding description information is used to indicate whether the terminal device successfully decodes the corresponding probe segment.

[0146] In one embodiment, if the recommended encoding strategy is a hierarchical encoding strategy, the processor 701 is configured to: obtain support information about the hierarchical encoding strategy, and when the support information of the hierarchical encoding strategy indicates support for the hierarchical encoding strategy, trigger the execution of matching the reference decoding capability and the expected decoding capability; the processor 701 is configured to: when the support information about the hierarchical encoding strategy indicates that the hierarchical encoding strategy is not supported, use the frame skipping encoding strategy as a new recommended encoding strategy.

[0147] In one embodiment, if the recommended encoding strategy is a frame skipping encoding strategy, the processor 701 is configured to: determine the reference quantity corresponding to the reference frame to be masked and the reference frame list corresponding to the target cloud game according to the current network condition of the terminal device; when it is determined that the frame skipping encoding strategy is supported, determine whether to support masking the reference frames with the reference quantity in the target cloud game according to the total number of reference frames included in the reference frame list and the reference quantity; when it is determined that masking the reference frames with the reference quantity in the target cloud game is supported, trigger and execute the matching process of the reference decoding capability and the expected decoding capability; otherwise, select new encoding reference frames from the processed game frames of the target cloud game, and use the selected new encoding reference frames as the immediate refresh frames of the game frames to be processed of the target cloud game, so as to perform encoding processing on the game frames to be processed based on the immediate refresh frames.

[0148] In one embodiment, the reference decoding capabilities that the terminal device can provide include: static decoding capabilities and available dynamic decoding capabilities; the processor 701 is configured to: obtain the number of decoding reference frames expected by the expected decoding capability, and obtain the number of decoding reference frames that the terminal device can provide from the static decoding capabilities; use the quantity matching result obtained by matching the number of decoding reference frames that the terminal device can provide with the number of expected decoding reference frames as the matching result of the static decoding capabilities and the expected decoding capabilities; from the available dynamic decoding capabilities, match the target detection segments whose number of encoding reference frames is greater than or equal to the number of expected decoding reference frames, and the decoding description information of the target detection segments, and use the matched decoding description information as the matching result of the dynamic decoding capabilities and the expected decoding capabilities.

[0149] In one embodiment, the processor 701 is further configured to: when the quantity matching result indicates that the number of decoding reference frames that the terminal device can provide is greater than or equal to the number of expected decoding reference frames, and the matched decoding description information indicates that the terminal device has successfully decoded the target detection segment, determine that the matching result indicates that the reference decoding capability matches the expected decoding capability successfully.

[0150] In one embodiment, the processor 701 is further configured to: when the matching result indicates that the reference decoding capability fails to match the expected decoding capability, select new encoding reference frames from the processed game frames of the target cloud game, and use the selected new encoding reference frames as the immediate refresh frames of the game frames to be processed of the target cloud game, so as to perform encoding processing on the game frames to be processed based on the immediate refresh frames.

[0151] It should be understood that the server 700 described in the embodiments of the present application can execute the description of the encoding processing method in the corresponding foregoing embodiments, and can also execute the description of the encoding processing device 600 in the corresponding foregoing Figure 6 embodiments, which will not be elaborated herein. In addition, the description of the beneficial effects of adopting the same method will not be elaborated either.

[0152] In addition, it should be pointed out that an exemplary embodiment of the present application further provides a storage medium, in which a computer program of the foregoing encoding processing method is stored. The computer program includes program instructions. When one or more processors load and execute the program instructions, the description of the encoding processing method in the embodiments can be implemented, which will not be elaborated herein, and the description of the beneficial effects of adopting the same method will not be elaborated herein either. It can be understood that the program instructions can be deployed to be executed on one or multiple servers capable of communicating with each other.

[0153] The above computer-readable storage medium can be the internal storage unit of the encoding processing device or the above server provided in any foregoing embodiment, such as the hard disk or memory of the server. The computer-readable storage medium can also be an external storage device of the server, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the server. Further, the computer-readable storage medium can also include both the internal storage unit and the external storage device of the server. The computer-readable storage medium is used to store the computer program and other programs and data required by the server. The computer-readable storage medium can also be used to temporarily store the data that has been output or will be output.

[0154] In one aspect of the present application, a computer program product or a computer program is provided. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the server reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the server executes the method provided in one aspect of the embodiments of the present application.

[0155] In one aspect of the present application, another computer program product is provided. The computer program product includes a computer program or computer instructions, and when the computer program or computer instructions are executed by a processor, the steps of the encoding processing method provided in the embodiments of the present application are implemented.

[0156] The steps in the method of the embodiments of the present application can be adjusted, combined, and deleted according to actual needs.

[0157] The modules in the device according to the embodiments of the present application can be combined, divided, and deleted according to actual needs.

[0158] The above-disclosed are only some embodiments of the present application. Of course, the scope of rights of the present application cannot be limited thereby. Those of ordinary skill in the art can understand all or part of the processes of implementing the above embodiments, and the equivalent changes made according to the claims of the present application still fall within the scope covered by the invention.

Claims

1. A coding processing method, characterized in that, it includes: When a decoding anomaly in the terminal device is detected, according to the correspondence between different decoding anomalies and coding strategies, the coding strategy corresponding to the decoding anomaly is used as the recommended coding strategy; wherein, there are at least two coding strategies, and the recommended coding strategies corresponding to different decoding anomalies are different; the decoding anomalies include one or more of the following: anomalies of network fluctuations and anomalies of overload; when the decoding anomaly is an anomaly of network fluctuations, the corresponding recommended coding strategy includes a hierarchical coding strategy; when the decoding anomaly is an anomaly of overload, the corresponding recommended coding strategy includes a frame skipping coding strategy; Obtain the reference decoding ability that can be provided when the terminal device decodes the coded data generated by the recommended coding strategy, and obtain the expected decoding ability that the terminal device needs to possess when enabling the terminal device to normally decode the coded data generated by the recommended coding strategy; the reference decoding ability is used to indicate: the actual decoding ability of the terminal device when decoding the coded data generated by the recommended coding strategy, and the expected decoding ability is used to indicate: the decoding ability required for the terminal device to normally decode the coded data generated by the recommended coding strategy and overcome the decoding anomaly; Match the reference decoding ability and the expected decoding ability to obtain a matching result, and when the matching result indicates that the reference decoding ability and the expected decoding ability are successfully matched, use the recommended coding strategy to code the game frame to be processed in the target cloud game; when the matching result indicates that the reference decoding ability and the expected decoding ability fail to match, select a new coding reference frame from the processed game frames of the target cloud game, and use the selected new coding reference frame as the immediate refresh frame of the game frame to be processed in the target cloud game, so as to code the game frame to be processed based on the immediate refresh frame.

2. The method according to claim 1, characterized in that, the detection method of the decoding anomaly existing in the terminal device includes: Send network probe data to the terminal device within a time window, where the time window contains multiple time points, and the network probe data sent to the terminal device at one time point is one or more; Obtain the probe feedback data corresponding to the network probe data sent at each time point in the time window, and generate a network probe result at the corresponding time point according to the probe feedback data; the network probe result corresponding to any time point is used to indicate whether network fluctuations occur at the any time point; According to the network probe results corresponding to each time point in the time window, count the number of times of network fluctuations in the time window, and determine the decoding anomaly existing in the terminal device according to the counted number of times.

3. The method according to claim 2, characterized in that, the determining the decoding anomaly existing in the terminal device according to the counted number of times includes: Obtain the fluctuation index value; When the counted number of times is greater than or equal to the fluctuation index value, determine that the decoding anomaly situation existing in the terminal device is a frequent fluctuation anomaly; When the counted number of times is greater than or equal to 1 and less than the fluctuation index value, determine that the decoding anomaly situation existing in the terminal device is an occasional fluctuation anomaly.

4. The method according to claim 1, wherein, the detection method for the decoding anomaly situation existing in the terminal device includes: Obtain the load information of the terminal device at different time points within a time window, where the load information is the amount of data to be decoded by the terminal device at the corresponding time point; Determine the number of overload times of the terminal device within the time window according to the load information corresponding to different time points of the terminal device within the time window; when the amount of data to be decoded by the terminal device at any time point is greater than or equal to the data volume threshold, the terminal device is overloaded once at the any time point; Determine the decoding anomaly situation existing in the terminal device according to the number of overload times; wherein, when the number of overload times reaches the overload index value, determine that the decoding anomaly situation existing in the terminal device is a frequent overload, and when the number of overload times is greater than 1 and does not reach the overload index value, determine that the decoding anomaly situation existing in the terminal device is an occasional overload.

5. The method according to claim 1, wherein, the obtaining of the reference decoding ability that the terminal device can provide when decoding the encoded data generated by the recommended encoding strategy includes: Determine the communication interface corresponding to the decoder from the terminal device, and obtain the description information of the decoder through the communication interface, where the description information includes: information indicating whether the decoder supports decoding the encoded data generated by the recommended encoding strategy, and information indicating the number of decoding reference frames provided when supported; When the description information indicates that the decoder supports decoding the encoded data generated by the recommended encoding strategy, use the number of decoding reference frames indicated in the description information as the static decoding ability that the terminal device can provide.

6. The method according to claim 1, wherein, the obtaining of the reference decoding ability that the terminal device can provide when decoding the encoded data generated by the recommended encoding strategy includes: Send a first probing bitstream generated using a hierarchical encoding strategy and a second probing bitstream generated using a frame skipping encoding strategy to the terminal device, where both the first probing bitstream and the second probing bitstream include multiple probing segments, and different probing segments are encoded using different numbers of encoding reference frames; Obtain a first decoding result of decoding each probing segment of the first probing bitstream and a second decoding result of decoding each probing segment of the second probing bitstream from the terminal device; Generate the dynamic decoding ability that the terminal device can provide according to the first decoding result and the second decoding result.

7. The method according to claim 6, wherein, Generating the dynamic decoding capability that can be provided by the terminal device according to the first decoding result and the second decoding result includes: Associatively storing the decoding description information of each detection segment of the first detection code stream recorded in the first decoding result and the number of coding reference frames corresponding to each detection segment of the first detection code stream as the dynamic decoding capability that can be provided by the terminal device when supporting the hierarchical coding strategy; Associatively storing the decoding description information of each detection segment of the second detection code stream recorded in the second decoding result and the number of coding reference frames corresponding to each detection segment of the second detection code stream as the dynamic decoding capability that can be provided by the terminal device when supporting the frame skipping coding strategy; wherein, the decoding description information is used to indicate whether the terminal device successfully decodes the corresponding detection segment.

8. The method according to claim 1, wherein, if the recommended coding strategy is the hierarchical coding strategy, the method further includes: Obtaining support information about the hierarchical coding strategy, and triggering the matching process of the reference decoding capability and the expected decoding capability when the support information of the hierarchical coding strategy indicates support for the hierarchical coding strategy; When the support information about the hierarchical coding strategy indicates non - support for the hierarchical coding strategy, taking the frame skipping coding strategy as the new recommended coding strategy.

9. The method according to claim 1, wherein, if the recommended coding strategy is the frame skipping coding strategy, the method further includes: Determining the reference number corresponding to the reference frame to be masked and the reference frame list corresponding to the target cloud game according to the current network condition of the terminal device; When it is determined that the frame skipping coding strategy is supported, determining whether to support masking the reference frames with the reference number in the target cloud game according to the total number of reference frames included in the reference frame list and the reference number; When it is determined that the reference frames with the reference number in the target cloud game can be masked, triggering the matching process of the reference decoding capability and the expected decoding capability; Otherwise, selecting new coding reference frames from the processed game frames of the target cloud game and using the selected new coding reference frames as the immediate refresh frames of the to - be - processed game frames of the target cloud game to perform coding processing on the to - be - processed game frames based on the immediate refresh frames.

10. The method according to claim 1, wherein, The reference decoding capability that can be provided by the terminal device includes: static decoding capability and the dynamic decoding capability that can be provided; the matching process of the reference decoding capability and the expected decoding capability to obtain a matching result includes: Obtaining the expected decoding reference frame number indicated by the expected decoding capability, and obtaining the decoding reference frame number that can be provided by the terminal device from the static decoding capability; Taking the quantity matching result obtained by matching the decoding reference frame number that can be provided by the terminal device with the expected decoding reference frame number as the matching result of the static decoding capability and the expected decoding capability; Match a target detection segment with the number of encoded reference frames greater than or equal to the desired number of decoding reference frames from the available dynamic decoding capabilities, and the decoding description information of the target detection segment, and use the matched decoding description information as the matching result of the dynamic decoding capabilities and the desired decoding capabilities.

11. The method according to claim 10, wherein, the method further includes: When the quantity matching result indicates that the number of decoding reference frames that the terminal device can provide is greater than or equal to the desired number of decoding reference frames, and the matched decoding description information indicates that the terminal device successfully decodes the target detection segment, determine that the matching result indicates that the reference decoding capability matches the desired decoding capability successfully.

12. An encoding processing device, wherein, it includes: A determination module, configured to, when detecting a decoding anomaly situation of a terminal device, according to the corresponding relationship between different decoding anomaly situations and encoding strategies, use the encoding strategy corresponding to the decoding anomaly situation as the recommended encoding strategy; wherein, there are at least two encoding strategies, and the recommended encoding strategies corresponding to different decoding anomaly situations are different; the decoding anomaly situation includes one or more of the following: an anomaly situation of network fluctuation and an anomaly situation of overload; when the decoding anomaly situation is an anomaly situation of network fluctuation, the corresponding recommended encoding strategy includes a hierarchical encoding strategy; when the decoding anomaly situation is an anomaly situation of overload, the corresponding recommended encoding strategy includes a frame skipping encoding strategy; An acquisition module, configured to acquire the reference decoding capability that the terminal device can provide when decoding the encoded data generated by the recommended encoding strategy, and acquire the desired decoding capability that the terminal device needs to possess when enabling the terminal device to normally decode the encoded data generated by the recommended encoding strategy; the reference decoding capability is used to indicate: the actual decoding capability of the terminal device when decoding the encoded data generated by the recommended encoding strategy, and the desired decoding capability is used to indicate: the decoding capability that the terminal device needs to possess when normally decoding the encoded data generated by the recommended encoding strategy and overcoming the decoding anomaly situation; A matching module, configured to perform a matching process on the reference decoding capability and the desired decoding capability to obtain a matching result; An encoding module, configured to, when the matching result indicates that the reference decoding capability matches the desired decoding capability successfully, use the recommended encoding strategy to encode the to-be-processed game frames in the target cloud game; when the matching result indicates that the reference decoding capability does not match the desired decoding capability, select new encoded reference frames from the already processed game frames of the target cloud game, and use the selected new encoded reference frames as the immediate refresh frames of the to-be-processed game frames of the target cloud game, so as to encode the to-be-processed game frames based on the immediate refresh frames.

13. A server, wherein, it includes: A processor, a memory, and a network interface; The processor is connected to the memory and the network interface, wherein the network interface is used to provide network communication functions, the memory is used to store program codes, and the processor is used to call the program codes to execute the encoding processing method according to any one of claims 1-11.

14. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by a processor, the encoding processing method according to any one of claims 1-11 is executed.

15. A computer program product, characterized in that the computer program product includes a computer program, the computer program includes program instructions, and when the program instructions are called by a processor, the processor is caused to execute the method according to any one of claims 1-11.

Images