FEC strategy fusion method and device, electronic device and storage medium

By dynamically adjusting the rs-fec and xor-fec strategies based on the terminal video packet processing delay and packet loss number, and generating adaptive fec packets, the cloud gaming experience problem caused by the single fec strategy in the existing technology is solved, and the user's cloud play is improved Experience.

CN118764137BActive Publication Date: 2024-11-29HAIMA CLOUD TIANJIN INFORMATION TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411243266.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-05
Publication Date
2024-11-29
Estimated Expiration
2044-09-05

AI Technical Summary

Technical Problem

In the prior art, the user terminal device has only used one fec strategy (rs-fec or xor-fec) in cloud games, resulting in lag in screen display or poor picture quality, and the overall cloud gaming experience is poor.

Method used

The protection rate is calculated based on the video packet processing delay and historical continuous packet loss number received by the terminal, determine the protection rate of the rs-fec and xor-fec policies, and generate corresponding types of fec packets, dynamically adjust the fec policy to adapt to the terminal capabilities and network environment. .

Benefits of technology

By adapting to different network environments and terminal performance, users' cloud gaming experience can be improved, screen lag and poor picture quality can be reduced, and overall cloud gaming experience can be improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118764137B_ABST
    Figure CN118764137B_ABST
Patent Text Reader

Abstract

The present application provides an FEC strategy fusion method and device, an electronic device and a storage medium, the method comprising: calculating a protection rate according to a processing delay and / or a number of historical continuous packet losses of a video packet received by a terminal, and determining a protection rate of an RS-FEC strategy and a protection rate of an XOR-FEC strategy according to the protection rate, wherein the protection rate is used to characterize the proportion of video packets adopting the RS-FEC strategy or the XOR-FEC strategy; packaging a cloud application screen into a video packet, and generating an RS-type FEC packet and an XOR-type FEC packet according to the protection rate of the RS-FEC strategy and the protection rate of the XOR-FEC strategy, respectively, and sending the video packet together with the RS-type FEC packet and the XOR-type FEC packet to the terminal, wherein the number of RS-type FEC packets is the number of video packets × the protection rate of the RS-FEC strategy, and the number of XOR-type FEC packets is the number of video packets × the protection rate of the XOR-FEC strategy, which can improve the overall cloud gaming experience of users.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to an FEC strategy fusion method and device, an electronic device and a storage medium. Background Art

[0002] Cloud applications (such as cloud games) are an application model based on cloud computing technology. Users connect to cloud servers through the network. The cloud servers run applications in real time and transmit application screens to the user's terminal device through video streaming. The user's input instructions are also transmitted to the cloud server through the network for real-time interaction. This model greatly reduces the hardware requirements for user terminal devices, so that they only need basic video decoding capabilities and network connection capabilities to experience high-quality applications. In the existing solution, forward error correction FEC technology is used to recover lost video packets. There are two FEC strategies: RS-FEC and XOR-FEC. The RS-FEC strategy consumes relatively more CPU resources and has a higher computational complexity. It has a better recovery effect for sudden packet loss; while the XOR-FEC strategy consumes relatively less CPU resources and has a lower computational complexity, but has a poor recovery effect for sudden packet loss. The existing technology usually uses only one of the FEC strategies of RS-FEC and XOR-FEC for packet loss recovery for all user terminals, which will cause the screen display of some user terminals to be relatively stuck or the picture quality to be poor, that is, the overall cloud gaming experience of users is poor.

[0003] In view of this, how to provide a FEC strategy to improve users' overall cloud gaming experience has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] In view of the technical problems existing in the prior art, the embodiments of the present application provide a FEC strategy fusion method and device, an electronic device and a storage medium.

[0005] In a first aspect, an embodiment of the present application provides a FEC strategy fusion method, including:

[0006] Calculate the protection rate according to the processing delay of the video packets received by the terminal and / or the number of historical continuous packet losses, and determine the protection rate of the RS-FEC strategy and the protection rate of the XOR-FEC strategy according to the protection rate, wherein the protection rate is used to characterize the proportion of video packets adopting the RS-FEC strategy or the XOR-FEC strategy;

[0007] The cloud application screen is packaged into video packets, and RS type FEC packets and XOR type FEC packets are generated according to the protection rate of the RS-FEC policy and the protection rate of the XOR-FEC policy respectively, and the video packets are sent to the terminal together with the RS type FEC packets and the XOR type FEC packets, wherein the number of RS type FEC packets is the number of video packets × the protection rate of the RS-FEC policy, and the number of XOR type FEC packets is the number of video packets × the protection rate of the XOR-FEC policy.

[0008] In a second aspect, an embodiment of the present application further provides an FEC strategy fusion device, including:

[0009] A calculation unit, configured to calculate a protection rate according to a processing delay of a video packet received by a terminal and / or a number of historical continuous packet losses, and determine a protection rate of an RS-FEC strategy and a protection rate of an XOR-FEC strategy according to the protection rate, wherein the protection rate is used to characterize a proportion of video packets adopting an RS-FEC strategy or an XOR-FEC strategy;

[0010] The sending unit is used to pack the cloud application screen into video packets, generate RS type FEC packets and XOR type FEC packets according to the protection rate of RS-FEC policy and the protection rate of XOR-FEC policy respectively, and send the video packets together with RS type FEC packets and XOR type FEC packets to the terminal, wherein the number of RS type FEC packets is the number of video packets × the protection rate of RS-FEC policy, and the number of XOR type FEC packets is the number of video packets × the protection rate of XOR-FEC policy.

[0011] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the FEC strategy fusion method described in the first aspect are executed.

[0012] In a fourth aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the FEC strategy fusion method as described in the first aspect.

[0013] In the above scheme, the protection rate is calculated according to the processing delay of the video packet received by the terminal and / or the number of historical continuous packet losses, the protection rate of the RS-FEC strategy and the protection rate of the XOR-FEC strategy are determined according to the protection rate, and the RS-type FEC packet and the XOR-type FEC packet of the video packet are generated according to the protection rate of the RS-FEC strategy and the protection rate of the XOR-FEC strategy respectively, and the video packet is sent to the terminal together with the RS-type FEC packet and the XOR-type FEC packet. That is to say, the RS-FEC and XOR-FEC strategies are integrated according to the capabilities of the terminal and the actual network environment. Compared with the existing technology of using one FEC strategy for all terminals, suitable FEC strategies can be provided for terminals with different performance in different network environments, thereby improving the overall cloud gaming experience of users. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A flowchart of an embodiment of an FEC strategy fusion method provided in an embodiment of the present application;

[0015] Figure 2 A structural diagram of an embodiment of an FEC strategy fusion device provided in an embodiment of the present application;

[0016] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of order, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0018] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0019] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0020] Reference Figure 1 As shown, it is a flow chart of a FEC strategy fusion method provided in an embodiment of the present application, and the method includes:

[0021] S10, calculating a protection rate according to a processing delay of a video packet received by the terminal and / or a number of historical continuous packet losses, and determining a protection rate of an RS-FEC strategy and a protection rate of an XOR-FEC strategy according to the protection rate, wherein the protection rate is used to characterize a proportion of video packets using an RS-FEC strategy or an XOR-FEC strategy;

[0022] In this embodiment, it can be understood that the processing delay of the video packets received by the terminal is used to reflect the capability of the terminal, and the historical number of continuous packet losses is used to reflect the network environment of the terminal. The protection rate adapted to the terminal capability and / or the current network environment can be accurately calculated based on the processing delay of the video packets received by the terminal and / or the historical number of continuous packet losses.

[0023] S11. Pack the cloud application screen into video packets, and generate RS type FEC packets and XOR type FEC packets according to the protection rate of the RS-FEC policy and the protection rate of the XOR-FEC policy respectively, and send the video packets together with the RS type FEC packets and the XOR type FEC packets to the terminal, wherein the number of RS type FEC packets is the number of video packets × the protection rate of the RS-FEC policy, and the number of XOR type FEC packets is the number of video packets × the protection rate of the XOR-FEC policy.

[0024] In the present embodiment, it can be understood that when generating FEC packets for video packets, RS-type FEC packets are generated according to the protection rate of the RS-FEC policy, and XOR-type FEC packets are generated according to the protection rate of the XOR-FEC policy, that is, the RS-FEC policy and the XOR-FEC policy are integrated according to the capabilities of the terminal and the network environment. In this way, the FEC policy will not be unsuitable for the terminal, and the occurrence of terminal display screen freezes and poor terminal display quality can be reduced, thereby maximizing the user's cloud gaming experience.

[0025] The FEC strategy fusion method provided in the embodiment of the present application calculates a protection rate according to the processing delay of the video packet received by the terminal and / or the number of historical continuous packet losses, determines the protection rate of the RS-FEC strategy and the protection rate of the XOR-FEC strategy according to the protection rate, generates RS-type FEC packets and XOR-type FEC packets of the video packet according to the protection rate of the RS-FEC strategy and the protection rate of the XOR-FEC strategy, respectively, and sends the video packet together with the RS-type FEC packet and the XOR-type FEC packet to the terminal, that is, the RS-FEC and XOR-FEC strategies are fused according to the capabilities of the terminal and the actual network environment. Compared with the prior art of using one FEC strategy for all terminals, suitable FEC strategies can be provided for terminals with different performances in different network environments, thereby improving the overall cloud gaming experience of users.

[0026] Based on the foregoing method embodiment, the calculation of the protection rate according to the processing delay of the video packets received by the terminal and / or the number of historical continuous packet losses may include:

[0027] Calculate a first protection rate according to a processing delay of a video packet received by the terminal, and calculate a second protection rate according to a historical number of continuous packet losses, wherein the first protection rate and the second protection rate are used to characterize a proportion of video packets using an RS-FEC strategy or an XOR-FEC strategy;

[0028] Determine a comprehensive protection rate according to a preset strategy, the first protection rate and the second protection rate;

[0029] The step of determining the protection rate of the RS-FEC strategy and the protection rate of the XOR-FEC strategy according to the protection rate includes:

[0030] The protection rate of the RS-FEC strategy and the protection rate of the XOR-FEC strategy are determined according to the comprehensive protection rate.

[0031] In this embodiment, it can be understood that the sum of the proportion of video packets using the RS-FEC strategy and the proportion of video packets using the XOR-FEC strategy is 1, that is, if the first protection rate calculated according to the processing delay of the video packets received by the terminal represents the proportion of video packets using the RS-FEC strategy (assuming a%), then the first protection rate for representing the proportion of video packets using the XOR-FEC strategy calculated according to the processing delay of the video packets received by the terminal is 1-a%; or if the first protection rate calculated according to the processing delay of the video packets received by the terminal represents the proportion of video packets using the XOR-FEC strategy (assuming b%), then the first protection rate for representing the proportion of video packets using the RS-FEC strategy calculated according to the processing delay of the video packets received by the terminal is 1-b%. The second protection rate has the same meaning as the first protection rate, except that the second protection rate is calculated based on the number of historical continuous packet losses, while the first protection rate is calculated based on the processing delay of the video packets received by the terminal.

[0032] The preset strategy may be a process such as weighted summing of the first protection rate and the second protection rate, and the first protection rate and the second protection rate may be processed using the preset strategy to obtain a comprehensive protection rate. Finally, the protection rate of the RS-FEC strategy and the protection rate of the XOR-FEC strategy are determined based on the comprehensive protection rate.

[0033] Based on the foregoing method embodiment, the calculation formula of the first protection ratio rs_protection_ratio_perf may be:

[0034] rs_protection_ratio_perf2=rs_protection_ratio_perf1×alpha1+(1-alpha1)×new_rs_protection_ratio;

[0035] Among them, rs_protection_ratio_perf2 and rs_protection_ratio_perf1 represent the first protection ratio rs_protection_ratio_perf to be calculated this time and the first protection ratio rs_protection_ratio_perf calculated last time, respectively. When calculating for the first time, rs_protection_ratio_perf1 can be set as needed, for example, to 50%. alpha1 represents the smoothing factor, which can be set to a value between 0 and 1 as needed. new_rs_protection_ratio=[(max_rs_protection_ratio1-min_rs_protection_ratio1) / (max_delay-min_delay)]×(max_delay-process_delay), max_rs_protection_ratio1 and min_rs_protection_ratio1 represent the maximum value and minimum value of the first protection ratio, respectively, which can be set as needed, for example, to 60% and 10%, respectively. max_delay and min_delay represent the maximum processing delay and the minimum processing delay allowed, respectively, which can be set as needed, for example, to 30 and 0, respectively. process_delay represents the packet processing delay of the terminal, and the calculation formula can be: process_delay2=process_delay1×alpha3+(1-alpha3)×t, where process_delay2 is the processing delay corresponding to the last frame of the cloud application screen in the time window, that is, process_delay, process_delay1 is the processing delay corresponding to the previous frame of the cloud application screen in the time window, alpha3 is a smoothing factor, which can be set to a value between 0 and 1 as needed, and t is the interval between the framing completion time of the last frame of the cloud application screen in the time window and the reception time of the first video packet of the last frame of the cloud application screen received by the terminal. The calculation formula of process_delay1 is consistent with the calculation formula of process_delay2, and will not be repeated here. When calculating the processing delay corresponding to the first frame of the cloud application screen in the time window, the value at the position of process_delay1 can be 0.

[0036] In this embodiment, it can be understood that the greater the packet processing delay of the terminal, the smaller the first protection ratio rs_protection_ratio_perf. In this embodiment, the first protection ratio rs_protection_ratio_perf represents the proportion of video packets using the rs-fec strategy; if the first protection ratio represents the proportion of video packets using the xor-fec strategy, the first protection ratio = 1-rs_protection_ratio_perf.

[0037] Based on the foregoing method embodiment, the calculation formula of the second protection ratio rs_protection_ratio_lost may be:

[0038] rs_protection_ratio_lost2=rs_protection_ratio_lost1×alpha2+(1-alpha2)×new_lost_rs_protection_ratio;

[0039] Among them, rs_protection_ratio_lost2 and rs_protection_ratio_lost1 represent the second protection ratio rs_protection_ratio_lost to be calculated this time and the second protection ratio rs_protection_ratio_lost calculated last time, respectively. When calculating for the first time, rs_protection_ratio_lost1 can be set as needed, for example, to 0. alpha2 represents the smoothing factor, which can be set to a value between 0 and 1 as needed. new_lost_rs_protection_ratio=(max_rs_protection_ratio2-min_rs_protection_ratio2) / MAX_LOST×min(MAX_LOST, number), max_rs_protection_ratio2 and min_rs_protection_ratio2 represent the maximum and minimum values ​​of the second protection ratio, respectively, which can be set as needed, for example, to 70% and 10%, respectively. MAX_LOST represents the maximum number of continuous packet losses allowed, which can be set as needed, for example, to 5. number represents the maximum number of continuous packet losses in the time window, and min(x, y) represents the smaller value of x and y.

[0040] In this embodiment, it can be understood that when number is less than MAX_LOST, as number increases, the second protection ratio rs_protection_ratio_lost increases. In this embodiment, the second protection ratio rs_protection_ratio_lost represents the proportion of video packets using the rs-fec strategy; if the second protection ratio represents the proportion of video packets using the xor-fec strategy, the second protection ratio = 1-rs_protection_ratio_lost.

[0041] Based on the above method embodiment,

[0042] The calculation formula of the protection ratio rs_protection_ratio of the rs-fec policy can be: rs_protection_ratio=protection_ratio×total_protection_ratio;

[0043] The calculation formula of the protection ratio xor_protection_ratio of the xor-fec strategy can be: xor_protection_ratio = (1-protection_ratio) × total_protection_ratio;

[0044] Among them, protection_ratio represents the comprehensive protection ratio, and total_protection_ratio represents the protection ratio calculated based on the packet loss rate.

[0045] In this embodiment, it should be noted that the protection rate calculated based on the packet loss rate is the sum of the protection rate of the RS-FEC strategy and the protection rate of the XOR-FEC strategy. The protection rate calculated based on the packet loss rate is calculated according to the packet loss rate. For example, a corresponding relationship between the packet loss rate and the protection rate can be set, and the protection rate corresponding to a certain packet loss rate can be obtained by searching the corresponding relationship.

[0046] Reference Figure 2 As shown, it is a structural schematic diagram of an FEC strategy fusion device provided in an embodiment of the present application, and the device includes:

[0047] A calculation unit 20 is used to calculate a protection rate according to a processing delay of a video packet received by the terminal and / or a number of historical continuous packet losses, and determine a protection rate of an RS-FEC strategy and a protection rate of an XOR-FEC strategy according to the protection rate, wherein the protection rate is used to characterize a proportion of video packets using an RS-FEC strategy or an XOR-FEC strategy;

[0048] The sending unit 21 is used to pack the cloud application screen into video packets, and generate RS type FEC packets and XOR type FEC packets according to the protection rate of the RS-FEC policy and the protection rate of the XOR-FEC policy respectively, and send the video packets together with the RS type FEC packets and the XOR type FEC packets to the terminal, wherein the number of RS type FEC packets is the number of video packets×the protection rate of the RS-FEC policy, and the number of XOR type FEC packets is the number of video packets×the protection rate of the XOR-FEC policy.

[0049] The FEC strategy fusion device provided in the embodiment of the present application calculates a protection rate according to the processing delay of the video packet received by the terminal and / or the number of historical continuous packet losses, determines the protection rate of the RS-FEC strategy and the protection rate of the XOR-FEC strategy according to the protection rate, generates RS-type FEC packets and XOR-type FEC packets of the video packet according to the protection rate of the RS-FEC strategy and the protection rate of the XOR-FEC strategy, respectively, and sends the video packet together with the RS-type FEC packet and the XOR-type FEC packet to the terminal, that is, the RS-FEC and XOR-FEC strategies are fused according to the capabilities of the terminal and the actual network environment. Compared with the prior art of using one FEC strategy for all terminals, suitable FEC strategies can be provided for terminals with different performances in different network environments, thereby improving the overall cloud gaming experience of users.

[0050] Based on the above device embodiment, the computing unit may be used for:

[0051] Calculate a first protection rate according to a processing delay of a video packet received by the terminal, and calculate a second protection rate according to a historical number of continuous packet losses, wherein the first protection rate and the second protection rate are used to characterize a proportion of video packets using an RS-FEC strategy or an XOR-FEC strategy;

[0052] Determine a comprehensive protection rate according to a preset strategy, the first protection rate and the second protection rate;

[0053] The protection rate of the RS-FEC strategy and the protection rate of the XOR-FEC strategy are determined according to the comprehensive protection rate.

[0054] Based on the above device embodiment, the calculation formula of the first protection ratio rs_protection_ratio_perf may be:

[0055] rs_protection_ratio_perf2=rs_protection_ratio_perf1×alpha1+(1-alpha1)×new_rs_protection_ratio;

[0056] Among them, rs_protection_ratio_perf2 and rs_protection_ratio_perf1 respectively represent the first protection rate rs_protection_ratio_perf to be calculated this time and the first protection rate rs_protection_ratio_perf calculated last time, alpha1 represents the smoothing factor, new_rs_protection_ratio=[(max_rs_protection_ratio1-min_rs_protection_ratio1) / (max_delay-min_delay)]×(max_delay-process_delay), max_rs_protection_ratio1 and min_rs_protection_ratio1 respectively represent the maximum and minimum values ​​of the first protection rate, max_delay and min_delay respectively represent the maximum allowed processing delay and the minimum allowed processing delay, and process_delay represents the packet processing delay of the terminal.

[0057] The implementation process of the FEC policy fusion device provided in the embodiment of the present application is consistent with the FEC policy fusion method provided in the embodiment of the present application, and the effect that can be achieved is also the same as the FEC policy fusion method provided in the embodiment of the present application, which will not be repeated here.

[0058] like Figure 3 As shown, an electronic device provided by an embodiment of the present application includes: a processor 30, a memory 31 and a bus 32, wherein the memory 31 stores machine-readable instructions executable by the processor 30, and when the electronic device is running, the processor 30 communicates with the memory 31 through the bus 32, and the processor 30 executes the machine-readable instructions to perform the steps of the above-mentioned FEC strategy fusion method.

[0059] Specifically, the above-mentioned memory 31 and processor 30 can be general-purpose memories and processors, which are not specifically limited here. When the processor 30 runs the computer program stored in the memory 31, the above-mentioned FEC strategy fusion method can be executed.

[0060] Corresponding to the above-mentioned FEC strategy fusion method, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned FEC strategy fusion method are executed.

[0061] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A fec strategy fusion method, characterized in that: include: Calculate a protection rate according to a processing delay and / or a number of historical continuous packet losses of video packets received by the terminal, and determine a protection rate of an RS-FEC strategy and a protection rate of an XOR-FEC strategy according to the protection rate, wherein the protection rate is used to characterize a proportion of video packets adopting an RS-FEC strategy or an XOR-FEC strategy, and the sum of a proportion of video packets adopting an RS-FEC strategy and a proportion of video packets adopting an XOR-FEC strategy is 1; The cloud application screen is packaged into video packets, and RS-type FEC packets and XOR-type FEC packets are generated according to the protection rate of the RS-FEC policy and the protection rate of the XOR-FEC policy respectively, and the video packets are sent to the terminal together with the RS-type FEC packets and the XOR-type FEC packets, wherein the number of RS-type FEC packets is the number of video packets × the protection rate of the RS-FEC policy, and the number of XOR-type FEC packets is the number of video packets × the protection rate of the XOR-FEC policy; Calculating a protection rate according to a processing delay of a video packet received by a terminal and / or a number of historical continuous packet losses, and determining a protection rate of an RS-FEC strategy and a protection rate of an XOR-FEC strategy according to the protection rate, including: Calculating a first protection ratio according to a processing delay of a video packet received by the terminal, and determining a protection ratio of an RS-FEC strategy and a protection ratio of an XOR-FEC strategy according to the first protection ratio; or Calculate a second protection rate according to the number of historical continuous packet losses, and determine the protection rate of the RS-FEC strategy and the protection rate of the XOR-FEC strategy according to the second protection rate; or A first protection rate is calculated according to a processing delay of a video packet received by a terminal, and a second protection rate is calculated according to a number of historical continuous packet losses; a comprehensive protection rate is determined according to a preset strategy, the first protection rate, and the second protection rate; a protection rate of an RS-FEC strategy and a protection rate of an XOR-FEC strategy are determined according to the comprehensive protection rate, wherein the first protection rate and the second protection rate are used to characterize a proportion of video packets adopting an RS-FEC strategy or an XOR-FEC strategy.

2. The method according to claim 1, characterized in that The calculation formula of the first protection ratio rs_protection_ratio_perf is: rs_protection_ratio_perf2=rs_protection_ratio_perf1×alpha1+(1-alpha1)×new_rs_protection_ratio; Among them, rs_protection_ratio_perf2 and rs_protection_ratio_perf1 respectively represent the first protection rate rs_protection_ratio_perf to be calculated this time and the first protection rate rs_protection_ratio_perf calculated last time, alpha1 represents the smoothing factor, new_rs_protection_ratio=[(max_rs_protection_ratio1-min_rs_protection_ratio1) / (max_delay-min_delay)]×(max_delay-process_delay), max_rs_protection_ratio1 and min_rs_protection_ratio1 respectively represent the maximum and minimum values ​​of the first protection rate, max_delay and min_delay respectively represent the maximum allowed processing delay and the minimum allowed processing delay, and process_delay represents the packet processing delay of the terminal.

3. The method according to claim 1, characterized in that The calculation formula of the second protection ratio rs_protection_ratio_lost is: rs_protection_ratio_lost2=rs_protection_ratio_lost1×alpha2+(1-alpha2)×new_lost_rs_protection_ratio; Among them, rs_protection_ratio_lost2 and rs_protection_ratio_lost1 represent the second protection ratio rs_protection_ratio_lost to be calculated this time and the second protection ratio rs_protection_ratio_lost calculated last time, respectively, alpha2 represents the smoothing factor, new_lost_rs_protection_ratio=(max_rs_protection_ratio2-min_rs_protection_ratio2) / MAX_LOST×min(MAX_LOST, number), max_rs_protection_ratio2 and min_rs_protection_ratio2 represent the maximum and minimum values ​​of the second protection ratio, respectively, MAX_LOST represents the maximum number of consecutive packet losses allowed, number represents the maximum number of consecutive packet losses in the time window, and min(x, y) represents the smaller value of x and y.

4. The method according to claim 1, characterized in that The calculation formula of the protection ratio rs_protection_ratio of the rs-fec strategy is: rs_protection_ratio=protection_ratio×total_protection_ratio; The calculation formula of the protection ratio xor_protection_ratio of the xor-fec strategy is: xor_protection_ratio = (1-protection_ratio) × total_protection_ratio; Among them, protection_ratio represents the comprehensive protection ratio, and total_protection_ratio represents the overall protection ratio calculated based on the packet loss rate.

5. A fec strategy fusion device, characterized in that: include: a calculation unit, configured to calculate a protection rate according to a processing delay of a video packet received by a terminal and / or a number of historical continuous packet losses, and determine a protection rate of an RS-FEC strategy and a protection rate of an XOR-FEC strategy according to the protection rate, wherein the protection rate is used to characterize a proportion of video packets adopting an RS-FEC strategy or an XOR-FEC strategy, and the sum of a proportion of video packets adopting an RS-FEC strategy and a proportion of video packets adopting an XOR-FEC strategy is 1; A sending unit, used for packaging the cloud application screen into video packets, generating RS type FEC packets and XOR type FEC packets according to the protection rate of the RS-FEC policy and the protection rate of the XOR-FEC policy respectively, and sending the video packets together with the RS type FEC packets and the XOR type FEC packets to the terminal, wherein the number of RS type FEC packets is the number of video packets × the protection rate of the RS-FEC policy, and the number of XOR type FEC packets is the number of video packets × the protection rate of the XOR-FEC policy; The computing unit is specifically used for: Calculating a first protection ratio according to a processing delay of a video packet received by the terminal, and determining a protection ratio of an RS-FEC strategy and a protection ratio of an XOR-FEC strategy according to the first protection ratio; or Calculate a second protection rate according to the number of historical continuous packet losses, and determine the protection rate of the RS-FEC strategy and the protection rate of the XOR-FEC strategy according to the second protection rate; or A first protection rate is calculated according to a processing delay of a video packet received by a terminal, and a second protection rate is calculated according to a number of historical continuous packet losses; a comprehensive protection rate is determined according to a preset strategy, the first protection rate, and the second protection rate; a protection rate of an RS-FEC strategy and a protection rate of an XOR-FEC strategy are determined according to the comprehensive protection rate, wherein the first protection rate and the second protection rate are used to characterize a proportion of video packets adopting an RS-FEC strategy or an XOR-FEC strategy.

6. The device according to claim 5, characterized in that The calculation formula of the first protection ratio rs_protection_ratio_perf is: rs_protection_ratio_perf2=rs_protection_ratio_perf1×alpha1+(1-alpha1)×new_rs_protection_ratio; Among them, rs_protection_ratio_perf2 and rs_protection_ratio_perf1 respectively represent the first protection rate rs_protection_ratio_perf to be calculated this time and the first protection rate rs_protection_ratio_perf calculated last time, alpha1 represents the smoothing factor, new_rs_protection_ratio=[(max_rs_protection_ratio1-min_rs_protection_ratio1) / (max_delay-min_delay)]×(max_delay-process_delay), max_rs_protection_ratio1 and min_rs_protection_ratio1 respectively represent the maximum and minimum values ​​of the first protection rate, max_delay and min_delay respectively represent the maximum allowed processing delay and the minimum allowed processing delay, and process_delay represents the packet processing delay of the terminal.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the FEC strategy fusion method according to any one of claims 1 to 4 are executed.

8. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the FEC strategy fusion method according to any one of claims 1 to 4.

Citation Information

Patent Citations

  • Audio and video transmission forward error correction method and device, computer equipment and storage medium

    CN109862440A

  • Audio and video transmission packet loss recovery method and device, equipment and storage medium

    CN117675118A