Cloud game video stream processing method and device, medium and product

Abstract

The invention provides a cloud game video stream processing method and device, a medium and a product, and the method comprises the steps: carrying out the coding of static data and dynamic data in to-be-transmitted video stream data of a cloud game, and obtaining binary video data; performing standard transformation on the binary video data to obtain a first transformed video stream; carrying out carry transformation on the binary video data to obtain a second transformed video stream; and performing correction bit transformation on the first transformed video stream to obtain a third transformed video stream, and pushing the third transformed video stream to a user terminal.

Description

Technical Field

[0001] The present disclosure relates to the field of video processing technology, and in particular to a method, device, medium and product for processing cloud game video streams. Background Art

[0002] In the cloud gaming mode, the game calculation and rendering are performed on the server side, and the rendered game screen is pushed to the user terminal for display through video streaming. Cloud gaming is a highly interactive video streaming service, and players are sensitive to the game delay.

[0003] In the prior art, the real-time transport protocol (RTP) data packets are usually used to transmit the video stream of cloud games. However, in the case of weak network, the process of data packaging may cause errors in the video stream in the RTP data packet, resulting in delays and failures in video frame decoding and rendering, causing problems such as blurred image quality, choppy video, and high control latency. In view of this situation, it is necessary to ensure that the transmitted video stream data will not have the problem of video stream data packet loss in the case of weak network, so as to improve the user experience. Summary of the invention

[0004] The embodiments of the present disclosure at least provide a method, device, equipment, medium and product for processing cloud game video streams.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for processing a cloud game video stream, including:

[0006] Encode the static data and dynamic data in the video stream data to be transmitted of the cloud game respectively to obtain binary video data;

[0007] Performing local transformation on the binary video data to obtain a first transformed video stream;

[0008] Performing a carry transformation on the binary video data to obtain a second transformed video stream; performing a modified bit transformation on the first transformed video stream to obtain a third transformed video stream, and pushing the third transformed video stream to the user terminal.

[0009] In an optional implementation, the encoding of the static data and the dynamic data in the video stream data to be transmitted of the cloud game to obtain the binary first video data and the multi-value binary second video data includes:

[0010] Binary-encode the static data to obtain first video data;

[0011] Perform modified symbol MSD encoding on each dynamic data to obtain second video data.

[0012] The binary video data is determined based on the first video data and the second video data.

[0013] In an optional implementation manner, performing digital transformation in an MSD addition operation on the first video data and the second video data to obtain a first transformed video stream includes:

[0014] Performing local transformation in an MSD addition operation on corresponding video data in the first video data and the second video data to obtain first transformed data;

[0015] Performing a carry transformation in an MSD addition operation on the corresponding video data in the first video data and the third video data to obtain second transformed data; wherein the third video data is obtained by performing a carry processing on the second video data;

[0016] The first transformed data and the second transformed data are determined as the first transformed video stream.

[0017] In an optional implementation manner, the pushing the third transformed video stream to the user terminal includes:

[0018] When it is determined that the third transformed video stream meets the data fragmentation condition, fragmenting the third transformed video stream to obtain a fragmentation result;

[0019] The segmentation result is pushed to the user terminal for display.

[0020] In an optional implementation manner, the performing slicing processing on the third transformed video stream to obtain a slicing result includes:

[0021] Using the difference between the first preset data length and the length of the third transformed video stream as the slice data length; wherein the first preset data length is used to indicate the maximum data transmission length;

[0022] The third transformed video stream is sliced ​​according to the slice data length to obtain the slice result.

[0023] In an optional implementation manner, the pushing the third transformed video stream to the user terminal includes:

[0024] In the case where it is determined that the length of the third transformed video stream is less than the second preset data length, the third transformed video stream is spliced ​​with other video streams to be transmitted to the user terminal to obtain a fourth transformed video stream; the length of the other video streams to be transmitted to the user terminal is less than the second preset data length, and the second preset data length is less than or equal to half of the maximum data transmission length;

[0025] Push the fourth transformed video stream to the user terminal.

[0026] In a second aspect, the embodiment of the present disclosure further provides a cloud game video stream processing device, including:

[0027] An encoding module, used for respectively encoding static data and dynamic data in the video stream data to be transmitted of the cloud game to obtain binary first video data and multi-value binary second video data;

[0028] A first conversion module, used for performing a digital conversion in an MSD addition operation on the first video data and the second video data to obtain a first converted video stream; wherein the digital conversion at least includes: a local conversion and / or a carry conversion;

[0029] The second transformation module is used to perform a modified bit transformation on the first transformed video stream to obtain a third transformed video stream, and push the third transformed video stream to the user terminal.

[0030] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect are performed.

[0031] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned first aspect or any possible implementation of the first aspect are executed.

[0032] In an embodiment of the present disclosure, first, the static data and dynamic data in the video stream data to be transmitted of the cloud game are encoded respectively to obtain binary video data; wherein the binary video data at least includes multi-value binary video data; secondly, the binary video data is digitally transformed to obtain a first transformed video stream; wherein the digital transformation at least includes: a base transformation and / or a carry transformation; finally, the first transformed video stream is corrected by a bit transformation to obtain a third transformed video stream, and the third transformed video stream is pushed to the user terminal.

[0033] In the above implementation, by encoding the static data and dynamic data in the video stream data to be transmitted and digitally transforming the binary video data obtained after encoding, the superposition of the dynamic data and static data in the video stream data to be transmitted can be achieved, thereby achieving efficient processing of the video stream and low-latency transmission of the video stream of the cloud game under weak network conditions. Under weak network conditions, the first transformed video stream obtained by the above-mentioned processing may have problems such as screen distortion, picture loss, or poor superposition effect. Therefore, by performing a corrected bit transformation on the first transformed video stream, the above problems can be avoided, thereby achieving high-quality transmission of the video stream of the cloud game.

[0034] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0036] Figure 1 A flowchart of a method for processing a cloud game video stream provided by an embodiment of the present disclosure is shown;

[0037] Figure 2 A schematic diagram of slicing processing of a method for processing a cloud game video stream provided by an embodiment of the present disclosure is shown;

[0038] Figure 3 A schematic diagram of video stream splicing processing of a cloud game video stream processing method provided in an embodiment of the present disclosure is shown;

[0039] Figure 4 A specific flow chart of the method for processing a cloud game video stream provided by an embodiment of the present disclosure is shown;

[0040] Figure 5 A schematic diagram of a cloud game video stream processing device provided by an embodiment of the present disclosure is shown;

[0041] Figure 6 A schematic diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0043] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0044] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.

[0045] Research has found that in the cloud gaming mode, the game's calculation and rendering are run on the server side, and then the rendered game screen is pushed to the user terminal for display via video streaming.

[0046] Therefore, cloud gaming is a highly interactive video streaming service, and players are more sensitive to game latency. In the prior art, real-time transport protocol (RTP) packets are usually used to transmit video stream data. However, in weak network conditions, the process of data packaging may cause video stream data errors in the RTP packets, causing delays and failures in video frame decoding and rendering, resulting in blurred image quality, choppy video, high control latency, and other issues. In response to this situation, it is necessary to ensure that the transmitted video stream data will not have the problem of video stream data packet loss in weak network conditions, thereby improving the user experience.

[0047] Based on the above research, the present disclosure provides a method for processing cloud game video streams. By encoding the static data and dynamic data in the video stream data to be transmitted, and digitally transforming the binary video data obtained after encoding, the superposition of dynamic data and static data in the video stream data to be transmitted can be achieved, thereby achieving efficient processing of the video stream and low-latency transmission of the video stream of the cloud game under weak network conditions. Under weak network conditions, the first transformed video stream obtained by the above method may have problems such as screen distortion, picture loss, or poor superposition effect. Therefore, by performing a corrected bit transformation on the first transformed video stream, the above problems can be avoided, thereby achieving high-quality transmission of the video stream of the cloud game.

[0048] To facilitate understanding of this embodiment, a method for processing a cloud game video stream disclosed in an embodiment of the present disclosure is first introduced in detail. The execution subject of the method for processing a cloud game video stream provided in the embodiment of the present disclosure is generally an electronic device with certain computing capabilities. In some possible implementations, the method for processing a cloud game video stream can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0049] See also Figure 1 As shown, it is a flowchart of a method for processing a cloud game video stream provided by an embodiment of the present disclosure, the method comprising steps S101 to S104, wherein:

[0050] S101. Encode the static data and dynamic data in the video stream data to be transmitted of the cloud game respectively to obtain binary first video data and multi-value binary second video data.

[0051] In the embodiment of the present disclosure, the video stream data to be transmitted is encoded data, and the video stream data to be transmitted needs to be re-encoded to obtain binary video data.

[0052] Here, first, the static data and dynamic data in the video stream data to be transmitted can be determined. Then, the static data can be binary coded or modified symbol MSD (i.e., multi-level binary) coded to obtain a binary number or MSD (Modified Signed-Digit, modified symbol) number. Then, the dynamic data can be binary coded or multi-level binary coded to obtain a binary number or MSD (Modified Signed-Digit, modified symbol) number. Among them, the encoding method of the static data is different from the encoding method of the dynamic data.

[0053] The static data and the dynamic data are encoded in different ways, for example, the static data is encoded in binary to obtain a binary number, and the dynamic data is encoded in three-value binary to obtain an MSD number.

[0054] Here, static data may be data in a relatively static state in the video stream data to be transmitted, for example, static data includes but is not limited to: trees, flowers, houses, ground surface and text data. Static data may be data in a relatively moving state in the video stream data to be transmitted, for example, dynamic data includes but is not limited to: a started car, a moving task and an action performed by a person.

[0055] S102: Perform local transformation on the binary video data to obtain a first transformed video stream.

[0056] In the embodiment of the present disclosure, the first transformed video stream data can be obtained by performing local transformation on the obtained binary code and the ternary binary code (ie, the binary video data) in parallel.

[0057] Here, the binary video data may be subjected to a local transformation (ie, a P transformation), and the video stream obtained after the local transformation may be used as the first transformed video stream.

[0058] S103, performing a carry transformation on the binary video data to obtain a second transformed video stream.

[0059] In an embodiment of the present disclosure, the second transformed video stream data may be obtained by performing carry transformation on the obtained binary code and the ternary binary code (ie, the binary video data) in parallel.

[0060] Here, the binary video data may be subjected to a carry transformation (ie, C transformation), and the video stream obtained after the carry transformation may be used as the second transformed video stream.

[0061] Here, the binary video data includes a plurality of binary data and a plurality of ternary binary data. For each binary data, the ternary binary data to be calculated can be determined from the plurality of ternary binary data for the binary data, and then the ternary binary data and the binary data are subjected to a local transformation, and the binary data and the ternary binary data are subjected to a carry transformation. For the next binary data, the ternary binary data to be calculated can be determined from the remaining ternary binary data, and then the ternary binary data and the binary data are subjected to a local transformation, and the binary data and the ternary binary data are subjected to a carry transformation. Here, for other binary data, the above-described method is used for digital transformation. Finally, after all binary data and all ternary binary data are subjected to digital transformation, binary video data is obtained. Among them, the binary data and the ternary binary data for each local transformation or carry transformation are different.

[0062] S104: Perform a modified bit transformation on the first transformed video stream and the second transformed video stream to obtain a third transformed video stream, and push the third transformed video stream to the user terminal.

[0063] In the embodiment of the present disclosure, after the binary video data is subjected to the local transformation, the local transformation video stream data can be obtained, and after the binary video data is subjected to the carry transformation, the carry transformation video stream data can be obtained. At this time, the local transformation video stream data and the carry transformation video stream data are the first transformation video stream described above.

[0064] Here, the modified bit transformation (ie, R transformation) can be performed on the carry-transformed video stream data corresponding to the original bit transformation video stream data to obtain a third transformed video stream.

[0065] Here, the binary data and three-value binary data corresponding to the original-bit transformed video stream data that are simultaneously subjected to the modified bit transformation are the same as the binary data and three-value binary data corresponding to the carry-bit transformed video stream data.

[0066] Here, based on the length of the second changed video stream, a push mode for pushing the third transformed video stream to the user terminal may be selected, wherein the push mode includes but is not limited to splicing and segmentation.

[0067] In an embodiment of the present disclosure, first, the static data and dynamic data in the video stream data to be transmitted of the cloud game are respectively encoded to obtain binary video data; secondly, the binary video data is subjected to a base-level transformation to obtain a first transformed video stream; secondly, the binary video data is subjected to a carry-level transformation to obtain a second transformed video stream; finally, the first transformed video stream and the second transformed video stream are subjected to a modified-bit transformation to obtain a third transformed video stream, and the third transformed video stream is pushed to the user terminal.

[0068] In the above implementation, by encoding the static data and dynamic data in the video stream data to be transmitted and digitally transforming the binary video data obtained after encoding, the superposition of the dynamic data and static data in the video stream data to be transmitted can be achieved, thereby achieving efficient processing of the video stream and low-latency transmission of the video stream of the cloud game under weak network conditions. Under weak network conditions, the first transformed video stream obtained by the above-mentioned processing may have problems such as screen distortion, picture loss, or poor superposition effect. Therefore, by performing a corrected bit transformation on the first transformed video stream, the above problems can be avoided, thereby achieving high-quality transmission of the video stream of the cloud game.

[0069] In an optional implementation, the above steps encode the static data and dynamic data in the video stream data to be transmitted of the cloud game respectively to obtain binary video data, which specifically includes the following steps:

[0070] First, binary encoding is performed on each static data to obtain first video data;

[0071] Then, each dynamic data is encoded using a modified symbol MSD to obtain second video data.

[0072] Finally, binary video data is determined based on the first video data and the second video data.

[0073] In the embodiment of the present disclosure, first, static data in the video data to be transmitted may be determined, and then each static data may be binary-encoded to obtain 01-type binary data, and the 01-type binary data may be used as the first video data.

[0074] For example, the binary number B corresponding to a static data after encoding 1 It can be expressed as:

[0075] B 1 =b 1 n b 1 n-1 …b 1 i …b 1 2b 1 1; where b 1 i is the i-th bit of the binary data of the static data, i ranges from 1 to n, and n is the number of bits of the binary number of the static data. 1 i Can be one of {0,1}.

[0076] Here, the second video data can be determined by the following steps: First, the dynamic data in the video data to be transmitted can be determined. Then, each dynamic data can be multi-level binary coded to obtain an MSD number, and the MSD number is used as the second video data.

[0077] The number of digits of the MSD number corresponding to the dynamic data is the same as the number of digits of the binary number corresponding to the static data.

[0078] For example, the MSD number M corresponding to a dynamic data after encoding 1 It can be expressed as:

[0079] M 1 =m 1 n m 1 n-1 …m 1 i …m 1 2m 1 1; where m1 i The i-th bit data of the binary data of the static data. 1 i Can be one of {0, 1, -1}.

[0080] In an optional implementation manner, the above step performs local transformation on the binary video data to obtain a first transformed video stream, which specifically includes the following steps:

[0081] First, the corresponding video data in the first video data and the second video data are transformed to obtain a first transformed data stream.

[0082] In an embodiment of the present disclosure, corresponding video data in the first video data and the second video data may be subjected to local transformation according to a local transformation truth table to obtain a first transformed data stream.

[0083] Here, in the case where the binary number in the first video data is less than the MSD number in the second video data, the remaining MSD number may be subjected to local transformation alone.

[0084] For example, the binary number B corresponding to the static data can be 1 MSD number M corresponding to dynamic data 1 Calculation is performed based on the native transformation truth table to obtain the first transformed data stream (ie, native transformed video stream data).

[0085] Among them, the local transformation truth table meets the following conditions:

[0086]

[0087] Here, u is represented by -1.

[0088] Here, the local transformation can be understood as processing the video stream based on static data to render a dynamic and static picture in the video to be processed. First, the binary number corresponding to the static data and the MSD number corresponding to the dynamic data are localized based on the local transformation truth table. Then, a zero is added to the front of the local transformation result and the result after the zero addition is used as the first transformation data stream.

[0089] Here, the jth first transformed data p j Meet the following conditions:

[0090] p j =0p j n p j n-1 …p j i …p j 2p j 1;

[0091] Among them, p j i is the i+1th bit of the jth first transformed data, the value of j is 1 to m, and m is determined according to the number of static data and the number of dynamic data.

[0092] Wherein, when the number of static data is greater than the number of dynamic data, m is the number of static data, otherwise m is the number of dynamic data.

[0093] In an optional implementation manner, the above step performs a carry transformation on the binary video data to obtain a second transformed video stream, which specifically includes the following steps:

[0094] The corresponding video data in the first video data and the third video data are subjected to carry transformation to obtain a second transformed data stream; wherein the third video data is obtained by carrying out the second video data.

[0095] Here, the corresponding video data in the first video data and the third video data may be carry-transformed according to a carry-transformation truth table to obtain a second transformed data stream (ie, carry-transformed video stream data).

[0096] For example, the binary number B corresponding to the static data can be 1 MSD number M corresponding to dynamic data 1 Calculation is performed based on the carry transformation truth table to obtain a second transformed data stream.

[0097] Among them, the local transformation truth table meets the following conditions:

[0098]

[0099] Here, the carry transformation can be understood as processing the video stream based on the dynamic data, which is used to render the dynamic and static images in the video to be processed. First, the binary number corresponding to the static data and the MSD number corresponding to the dynamic data are carried out based on the carry transformation truth table. Then, a 0 is added to the end of the carry transformation result and the result after the 0 is used as the second transformation data.

[0100] Here, the jth second transformed data c j Meet the following conditions:

[0101] c j =c j n c j n-1 …c j i …c j 2c j 10;

[0102] Among them, c j i is the i-th bit of the j-th second transformed data.

[0103] In an embodiment of the present disclosure, after obtaining the first transformed data and the second transformed data, a modified bit transformation may be performed on the first transformed data and the second transformed data according to a modified bit transformation truth table to obtain a second transformed video stream.

[0104] For example, the jth first transformed data p j and the jth second transformed data c j Perform a modified bit transformation based on the modified bit transformation truth table to obtain the j-th third transformed data stream.

[0105] Among them, the modified bit transformation truth table meets the following conditions:

[0106]

[0107] Here, after the mth first transformed data and the mth second transformed data are subjected to modified bit transformation based only on the modified bit transformation truth table, the obtained mth third transformed data is used as the third transformed video stream.

[0108] Here, the correction bit transformation can convert dynamic data into relatively unchanged static data for each frame for processing, that is, the video stream is processed based on the static data. First, the first transformation data and the second transformation data are subjected to correction bit transformation based on the correction bit transformation truth table. Then, a zero is added to the front of the correction bit transformation result and the result after the zero addition is used as the third transformed video stream.

[0109] Here, the jth third transformed data r j Meet the following conditions:

[0110] r j =r j n+1 r j n …r j i …r j 2r j 1;

[0111] Among them, r j i is the i-th bit of the j-th third transformed data.

[0112] In an embodiment of the present disclosure, when the network status of the server is not a weak network status, the transformation method can be determined according to the amount of static data and dynamic data in the video to be transmitted.

[0113] Here, when there is more static data than dynamic data in the video to be transmitted, only the local transformation based on the static data for static and dynamic image processing can be used to process the video stream data to be transmitted, obtain a fourth transformed video stream, and send the fourth transformed video stream to the user terminal.

[0114] Among them, the local transformation based on static data for dynamic and static image processing is used to process the video stream data to be transmitted. Since it can be compressed and reduced by one bit, that is, a 0 is added in front of the data after the local transformation processing, it can save bandwidth and improve processing efficiency.

[0115] Here, when there is more dynamic data than static data in the video data to be transmitted, only the carry transformation based on the dynamic data can be used to process the dynamic and static images to process the video stream data to obtain a fourth transformed video stream, and the fourth transformed video stream is sent to the user terminal.

[0116] Among them, the local transformation that uses dynamic data as a reference to process dynamic and static images is used to process the video stream data to be transmitted. A preset video can be generated in advance based on the dynamic video. Therefore, a redundant video frame is required, that is, one bit needs to be added after the data processed by the carry transformation. This can increase the processing speed of the video stream that needs to be preset.

[0117] In the embodiments of the present disclosure, when the server's network status is a weak network status, the dynamic data has been subjected to relatively unchanged static processing for each frame. After video processing is performed based on the static data, that is, only the local transformation (i.e., P transformation) is performed on the transmitted video stream data, there may still be screen distortion and picture loss.

[0118] Alternatively, after video processing is performed based on dynamic data, that is, only the carry transformation (i.e., C transformation) is performed on the video stream data to be transmitted, but there is a problem of poor network status during the processing, which will also have a negative impact on the superposition effect of the video stream data to be transmitted. Therefore, it is necessary to perform a correction bit transformation on the transformation result obtained after the video stream data to be transmitted is subjected to the standard transformation and carry transformation, so as to avoid the problems of screen distortion, picture loss, or poor superposition effect that may exist in a weak network state.

[0119] In an optional implementation manner, the above step of pushing the third transformed video stream to the user terminal specifically includes the following steps:

[0120] First, when it is determined that the third transformed video stream meets the data fragmentation condition, fragmentation processing is performed on the third transformed video stream to obtain a fragmentation result;

[0121] Then, the sharding results are pushed to the user terminal for display.

[0122] In the embodiment of the present disclosure, the length of the third transformed video stream can be obtained. After the length of the third transformed video stream is obtained, a comparator can be used to determine whether the third transformed video stream meets the data segmentation condition.

[0123] Here, it can be determined by the following steps whether the length of the third transformed video stream meets the data fragmentation condition. First, a video stream fragmentation transmission threshold (i.e., a first preset data length) can be set in a comparator. Then, the comparator can determine whether the second video stream meets the data fragmentation condition based on the video stream fragmentation transmission threshold and the length of the third transformed video stream.

[0124] Here, when the length of the third transformed video stream is greater than the video stream fragmentation transmission threshold, it is determined that the length of the third transformed video stream meets the data fragmentation condition.

[0125] The video stream segment transmission threshold can be automatically adjusted according to the network status and the length of the video stream that needs the third transformation.

[0126] Here, when it is determined that the length of the third transformed video stream meets the data fragmentation condition, the fragmented data length can be determined, and the third transformed video stream is fragmented based on the fragmented data length to obtain a fragmentation result.

[0127] Here, after the fragmentation result is sent to the user terminal, the user terminal may decode the fragmentation result and then display it.

[0128] In an optional implementation manner, the above step performs slice processing on the third transformed video stream to obtain a slice result, which specifically includes the following steps:

[0129] First, based on the first preset length, determining the length of the sliced ​​data; wherein the first preset data length is used to indicate the maximum data transmission length, and the sliced ​​data length is less than the first preset length;

[0130] Then, the third transformed video stream is sliced ​​according to the slice data length to obtain a slice result.

[0131] In an embodiment of the present disclosure, the third transformed video stream can be sliced ​​by an M-1-B type adder. First, the slice data length is input to the M-1-B type adder. Second, the third transformed video stream is input to the M-1-B type adder. Finally, the output of the M-1-B type adder is used as the slice result.

[0132] Here, when there are multiple fragmentation results of the third transformed video stream, each fragmentation result is right-aligned to maintain the order in the third transformed video stream. If any fragmentation result does not meet the fragmentation data length, the fragmentation result is padded with zeros on the left.

[0133] Here, when the difference between the first preset data length and the length of the third transformed video stream is smaller than the first preset data length and larger than the second preset data length, the following processing method is used for segmentation processing.

[0134] See also Figure 2 , which is a schematic diagram of the slicing processing of the cloud game video stream processing method provided by the embodiment of the present disclosure, wherein:

[0135] The fragment data length w meets the following conditions:

[0136] w = xy;

[0137] Wherein, x is the length of the third transformed video stream, and y is the first preset data length.

[0138] Here, the length of each slice data is w, and the first slice result is from the 0th bit of the second transformed data stream to the (w-1)th bit of the third transformed video stream. The second slice result is from the (w-1)th bit of the third transformed video stream to the 2(w-1)th bit of the third transformed video stream. The tth slice result is from the (w-1)(t-1)th bit of the third transformed video stream to the t(w-1)th bit of the third transformed video stream. The t+1th slice result is from the t(w-1)th bit of the third transformed video stream to the last bit of the third transformed video stream.

[0139] Here, t+1 is the last segmentation result of the third transformed video stream, and t meets the following conditions:

[0140]

[0141] Here, when the number of bits of the fragmentation result of the third transformed video stream of the t+1th segment is not w, the fragmentation result is padded with zeros on the left until the number of bits of the fragmentation result of the t+1th segment is w.

[0142] In the above implementation, in a weak network state, if the length of the second transformed data stream exceeds the first preset data length, data loss may occur during the transmission process. The present application performs slice processing on the second transformed data stream and transmits the slice processing result to the user terminal. Through reasonable slice processing, the problem of data loss that may occur during the transmission process is avoided.

[0143] In an optional manner, the above step of pushing the third transformed video stream to the user terminal specifically includes the following steps:

[0144] First, when it is determined that the length of the third transformed video stream is less than the second preset data length, the third transformed video stream is spliced ​​with other video streams to be transmitted to the user terminal to obtain a fourth transformed video stream; the lengths of the other video streams to be transmitted to the user terminal are less than the second preset data length, and the second preset data length is less than or equal to half of the maximum data transmission length;

[0145] Then, the fourth transformed video stream is pushed to the user terminal.

[0146] In an embodiment of the present disclosure, first, when it is determined that the length of the third transformed video stream is less than the second preset data length, it is determined whether the user terminal corresponding to the third transformed video stream has other video streams to be transmitted.

[0147] Secondly, in case of other video streams to be transmitted by the user terminal, it is determined whether the length of the other video streams to be transmitted is less than the second preset data length.

[0148] Finally, when there is any video stream to be transmitted that is shorter than the second preset data length among other video streams to be transmitted corresponding to the user terminal, the data stream to be transmitted is spliced ​​with the third transformed video stream to obtain a fourth transformed video stream.

[0149] Here, it can be determined whether the length of the spliced ​​fourth transformed video stream is less than the second preset data length. In the case where the length of the fourth transformed video stream is less than the second preset data length, the fourth transformed video stream can be spliced ​​with other video streams to be transmitted to the user terminal until the length of the obtained fourth transformed video stream is greater than or equal to the second preset data length. The fourth transformed video stream is sent to the user terminal.

[0150] Here, the user terminal may perform separation processing on the third changed video stream or the fourth changed video stream to restore the third transformed video stream and other video streams of the user terminal.

[0151] See also Figure 3 , which is a schematic diagram of video stream splicing processing of a cloud game video stream processing method provided by an embodiment of the present disclosure, wherein:

[0152] Here, first, a maximum splicing length h can be set. Secondly, a length x of the third transformed video stream can be obtained. Secondly, a length k of the video stream to be spliced ​​with the third transformed video stream is determined. Finally, when the length x+k of the spliced ​​fourth transformed video stream is ≥ the maximum splicing length h, the fourth transformed video stream is transmitted to the user terminal.

[0153] See also Figure 4 As shown, it is a specific flow chart of the method for processing cloud game video streams provided by an embodiment of the present disclosure, wherein:

[0154] S10. Determine static data and dynamic data in the video stream data to be transmitted of the cloud game.

[0155] Here, static data includes but is not limited to: trees, flowers, houses, ground surface and text data. Dynamic data includes but is not limited to: started cars, tasks in progress and actions performed by characters.

[0156] S20, performing binary encoding on each static data to obtain first video data, and performing multi-level binary encoding on each dynamic data to obtain second video data; wherein the second video data is the multi-value binary video data.

[0157] Here, first, the static data and dynamic data in the video stream data to be transmitted can be determined, and then the static data and the dynamic data are binary coded and multi-level binary coded respectively to obtain a binary number and an MSD (Modified Signed-Digit) number respectively.

[0158] The static data and the dynamic data are encoded in different ways, for example, the static data is encoded in binary to obtain a binary number, and the dynamic data is encoded in three-value binary to obtain an MSD number.

[0159] S30. Determine the binary video data based on the first video data and the second video data.

[0160] Here, the first video data and the second video data may be determined as binary video data.

[0161] S40: Perform local transformation on the corresponding video data in the first video data and the second video data to obtain a first transformed video stream.

[0162] Here, the corresponding video data in the first video data and the second video data may be subjected to local transformation according to a local transformation truth table to obtain a first transformed video stream.

[0163] Here, the local transformation can be understood as processing the video stream based on static data to render a dynamic and static picture in the video to be processed. First, the binary number corresponding to the static data and the MSD number corresponding to the dynamic data are localized based on the local transformation truth table. Then, a zero is added to the front of the local transformation result and the result after the zero addition is used as the first transformed video stream.

[0164] S50, performing a carry transformation on the corresponding video data in the first video data and the third video data to obtain a second transformed video stream.

[0165] Here, corresponding video data in the first video data and the second video data may be subjected to a bit-wise transformation according to a bit-wise transformation truth table to obtain a second transformed video stream.

[0166] Here, the carry transformation can be understood as processing the video stream based on the dynamic data, which is used to render the dynamic and static images in the video to be processed. First, the binary number corresponding to the static data and the MSD number corresponding to the dynamic data are carried out based on the carry transformation truth table. Then, the end of the carry transformation result is padded with a 0 and the result after the 0 is used as the second transformed video stream.

[0167] S60: Perform a modified bit transformation on the first transformed video stream and the second transformed video stream to obtain a third transformed video stream, and push the third transformed video stream to a user terminal.

[0168] Here, the correction bit transformation can convert dynamic data into relatively unchanged static data for each frame for processing, that is, the video stream is processed based on the static data. First, the first transformation data and the second transformation data are subjected to correction bit transformation based on the correction bit transformation truth table. Then, a zero is added to the front of the correction bit transformation result and the result after the zero addition is used as the third transformed video stream.

[0169] S70: When it is determined that the third transformed video stream meets the data fragmentation condition, fragment the third transformed video stream to obtain a fragmentation result, and push the fragmentation result to the user terminal for display.

[0170] Here, it can be determined by the following steps whether the length of the third transformed video stream meets the data fragmentation condition. First, a video stream fragmentation transmission threshold (i.e., a first preset data length) can be set in a comparator. Then, the comparator can determine whether the second video stream meets the data fragmentation condition based on the video stream fragmentation transmission threshold and the length of the third transformed video stream.

[0171] Here, when the length of the third transformed video stream is greater than the video stream fragmentation transmission threshold, it is determined that the length of the third transformed video stream meets the data fragmentation condition.

[0172] The video stream segment transmission threshold can be automatically adjusted according to the network status and the length of the video stream that needs the third transformation.

[0173] Here, when it is determined that the length of the third transformed video stream meets the data fragmentation condition, the fragmented data length can be determined, and the third transformed video stream is fragmented based on the fragmented data length to obtain a fragmentation result.

[0174] Here, after the fragmentation result is sent to the user terminal, the user terminal may decode the fragmentation result and then display it.

[0175] S80. When it is determined that the length of the third transformed video stream is less than the second preset data length, concatenate the third transformed video stream with other video streams to be transmitted to the user terminal to obtain a fourth transformed video stream, and push the fourth transformed video stream to the user terminal.

[0176] In an embodiment of the present disclosure, first, when it is determined that the length of the third transformed video stream is less than the second preset data length, it is determined whether the user terminal corresponding to the third transformed video stream has other video streams to be transmitted.

[0177] Secondly, in case of other video streams to be transmitted by the user terminal, it is determined whether the length of the other video streams to be transmitted is less than the second preset data length.

[0178] Finally, when there is any video stream to be transmitted that is shorter than the second preset data length among other video streams to be transmitted corresponding to the user terminal, the data stream to be transmitted is spliced ​​with the third transformed video stream to obtain a fourth transformed video stream.

[0179] In the above implementation, by encoding the static data and dynamic data in the video stream data to be transmitted and digitally transforming the binary video data obtained after encoding, the superposition of the dynamic data and static data in the video stream data to be transmitted can be achieved, thereby achieving efficient processing of the video stream and low-latency transmission of the video stream of the cloud game under weak network conditions. Under weak network conditions, the first transformed video stream obtained by the above-mentioned processing may have problems such as screen distortion, picture loss, or poor superposition effect. Therefore, by performing a corrected bit transformation on the first transformed video stream, the above problems can be avoided, thereby achieving high-quality transmission of the video stream of the cloud game.

[0180] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0181] Based on the same inventive concept, the embodiments of the present disclosure also provide a cloud game video stream processing device corresponding to the cloud game video stream processing method. Since the principle of solving the problem by the device in the embodiments of the present disclosure is similar to the above-mentioned cloud game video stream processing method in the embodiments of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0182] Reference Figure 5 As shown, it is a schematic diagram of a cloud game video stream processing device provided by an embodiment of the present disclosure, the device includes: an encoding module 11, a first transformation module 12, a second transformation module 13, and a third transformation module 14; wherein,

[0183] The encoding module is used to encode the static data and dynamic data in the video stream data to be transmitted of the cloud game respectively to obtain binary video data;

[0184] A first transformation module, configured to perform local transformation on the binary video data to obtain a first transformed video stream;

[0185] A second transformation module, used for performing a carry transformation on the binary video data to obtain a second transformed video stream;

[0186] The third transformation module is used to perform a modified bit transformation on the first transformed video stream and the second transformed video stream to obtain a third transformed video stream, and push the third transformed video stream to the user terminal.

[0187] The disclosed embodiment can achieve the superposition of dynamic data and static data in the video stream data to be transmitted by encoding the static data and dynamic data in the video stream data to be transmitted and digitally transforming the binary video data obtained after encoding, thereby achieving efficient processing of the video stream and low-latency transmission of the video stream of the cloud game under weak network conditions. Under weak network conditions, the first transformed video stream obtained by the above-mentioned processing method may have problems such as screen distortion, picture loss or poor superposition effect. Therefore, the above-mentioned problems can be avoided by performing a corrected bit transformation on the first transformed video stream, thereby achieving high-quality transmission of the video stream of the cloud game.

[0188] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0189] Corresponds to Figure 1 The present disclosure also provides an electronic device 600, such as Figure 6 FIG. 6 is a schematic diagram of the structure of an electronic device 600 provided in an embodiment of the present disclosure, including:

[0190] Processor 61, memory 62, and bus 63; memory 62 is used to store execution instructions, including internal memory 621 and external memory 622; the internal memory 621 is also called internal memory, which is used to temporarily store the operation data in the processor 61 and the data exchanged with the external memory 622 such as a hard disk. The processor 61 exchanges data with the external memory 622 through the internal memory 621. When the electronic device 600 is running, the processor 61 communicates with the memory 62 through the bus 63, so that the processor 61 executes the following instructions:

[0191] Encode the static data and dynamic data in the video stream data to be transmitted of the cloud game respectively to obtain binary video data;

[0192] Performing local transformation on the binary video data to obtain a first transformed video stream;

[0193] Performing a carry transformation on the binary video data to obtain a second transformed video stream;

[0194] Performing modified bit transformation on the first transformed video stream and the second transformed video stream to obtain a third transformed video stream, and pushing the third transformed video stream to the user terminal.

[0195] The embodiment of the present disclosure also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method for processing a cloud game video stream described in the above method embodiment are executed. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0196] The embodiments of the present disclosure also provide a computer program product, which carries a program code. The instructions included in the program code can be used to execute the steps of the cloud game video stream processing method described in the above method embodiment. Please refer to the above method embodiment for details, which will not be repeated here.

[0197] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).

[0198] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0199] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0200] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0201] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0202] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A method for processing a cloud game video stream, characterized in that: include: Encode the static data and dynamic data in the video stream data to be transmitted of the cloud game respectively to obtain binary video data; Performing local transformation on the binary video data to obtain a first transformed video stream; Performing a carry transformation on the binary video data to obtain a second transformed video stream; Performing modified bit transformation on the first transformed video stream and the second transformed video stream to obtain a third transformed video stream, and pushing the third transformed video stream to the user terminal.

2. The method according to claim 1, characterized in that The step of encoding the static data and the dynamic data in the video stream data to be transmitted of the cloud game to obtain binary video data includes: Binary-encode each of the static data to obtain first video data; Performing modified symbol MSD encoding on each of the dynamic data to obtain second video data; The binary video data is determined based on the first video data and the second video data.

3. The method according to claim 2, characterized in that The step of performing local transformation on the binary video data to obtain a first transformed video stream includes: The corresponding video data in the first video data and the second video data are transformed to obtain a first transformed video stream.

4. The method according to claim 2, characterized in that: The step of performing a carry transformation on the binary video data to obtain a second transformed video stream comprises: The corresponding video data in the first video data and the third video data are subjected to a carry transformation to obtain a second transformed data stream; wherein the third video data is obtained by carrying out a carry processing on the second video data.

5. The method according to claim 1, characterized in that The pushing the third transformed video stream to the user terminal comprises: When it is determined that the third transformed video stream meets the data fragmentation condition, fragmenting the third transformed video stream to obtain a fragmentation result; The segmentation result is pushed to the user terminal for display.

6. The method according to claim 5, characterized in that The performing slicing processing on the third transformed video stream to obtain a slicing result includes: Determine the length of the fragmented data based on a first preset length; wherein the first preset data length is used to indicate a maximum data transmission length, and the fragmented data length is less than the first preset length; The third transformed video stream is sliced ​​according to the slice data length to obtain the slice result.

7. The method according to claim 1, characterized in that The pushing the third transformed video stream to the user terminal comprises: In the case where it is determined that the length of the third transformed video stream is less than the second preset data length, the third transformed video stream is spliced ​​with other video streams to be transmitted to the user terminal to obtain a fourth transformed video stream; the length of the other video streams to be transmitted to the user terminal is less than the second preset data length, and the second preset data length is less than or equal to half of the maximum data transmission length; Push the fourth transformed video stream to the user terminal.

8. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the method for processing a cloud game video stream as described in any one of claims 1 to 6 are performed.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the method for processing a cloud gaming video stream as described in any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product is stored in a storage medium, and the program product is executed by at least one processor to implement the steps of the method for processing a cloud game video stream as described in any one of claims 1 to 6.

Images