Coding and decoding method, device and system

Abstract

The present disclosure provides an encoding and decoding method, device, and system, relating to the field of image technologies. The method includes obtaining the current packet loss rate of a decoding device; when it is determined that the current packet loss rate is greater than a preset packet loss threshold, dividing a to-be-encoded image frame into multiple target to-be-encoded regions; encoding a current target to-be-encoded region to obtain bitstream data; while sending the bitstream data to the decoding device, encoding the next target to-be-encoded region, so that the decoding device decodes each bitstream data in sequence. The present disclosure divides the to-be-encoded image frame into multiple target to-be-encoded regions for encoding. The encoded bitstream obtained for each target to-be-encoded region is smaller than the entire-frame bitstream of the to-be-encoded image frame. Sending each encoded bitstream separately to the network brings less pressure than the entire-frame bitstream, thereby reducing the packet loss rate of the transmitted bitstream.

Description

Technical Field

[0001] The present disclosure relates to the field of image technology, and in particular to encoding and decoding methods, devices, and systems. Background Art

[0002] Computer synthesized images are artificial images obtained by using computer graphics technology and calculated by a graphics card on a computer. When transmitting an image, it is usually necessary to encode and decode the image.

[0003] In the related art, when an encoding device encodes an image, it encodes an entire frame of the image into an entire bitstream, and then sends the entire bitstream to a decoding device together. The decoding device decodes the entire bitstream and then displays it.

[0004] However, in the above technology, sending the entire bitstream together will generate a peak in a low-bandwidth network, resulting in a relatively high packet loss rate. Summary of the Invention

[0005] Embodiments of the present disclosure provide an encoding and decoding method, device, and system, which can solve the problem of relatively high packet loss rate in the prior art. The technical solutions are as follows:

[0006] According to a first aspect of embodiments of the present disclosure, an encoding and decoding method is provided, which is applied to an encoding device. The method includes:

[0007] Obtain the current packet loss rate of the decoding device;

[0008] When it is determined that the current packet loss rate is greater than a preset packet loss threshold, divide the image frame to be encoded into multiple target regions to be encoded;

[0009] Encode the current target region to be encoded to obtain bitstream data;

[0010] While sending the bitstream data to the decoding device, encode the next target region to be encoded, so that the decoding device decodes each bitstream data in sequence.

[0011] Embodiments of the present disclosure provide an encoding and decoding method. When an encoding device determines that the current packet loss rate of a decoding device is greater than a preset packet loss threshold, it divides the image frame to be encoded into multiple target regions to be encoded, then encodes the current target region to be encoded to obtain bitstream data, and while sending the bitstream data to the decoding device, encodes the next target region to be encoded, so that the decoding device decodes each bitstream data in sequence. In this way, by dividing the image frame to be encoded into multiple target regions to be encoded for encoding, the encoded bitstream obtained for each target region to be encoded is smaller than the entire-frame bitstream of the image frame to be encoded, and the pressure on the network caused by separately sending each encoded bitstream is smaller than that of the entire-frame bitstream, thereby reducing the packet loss rate of the transmitted bitstream.

[0012] In one embodiment, after sending the bitstream data to the decoding end device, the step of obtaining the current packet loss rate of the decoding end device is returned, and when it is determined that the current packet loss rate is greater than the preset packet loss threshold, the size of the target area to be encoded is reduced until the current packet loss rate is less than the preset packet loss threshold.

[0013] In one embodiment, it further includes:

[0014] Obtain the type of the current application program;

[0015] When it is determined that the current application program is an instant messaging application, obtain the current delay information of the decoding end device;

[0016] When it is determined that the current delay information is greater than the preset delay threshold, the step of dividing the image frame to be encoded into multiple target areas to be encoded is returned, and the size of the target area to be encoded is increased until the current delay information is less than the preset delay threshold.

[0017] In one embodiment, it further includes:

[0018] When it is determined that the current delay information is less than the preset delay threshold, the step of encoding the current target area to be encoded is returned.

[0019] In one embodiment, it further includes:

[0020] When it is determined that the current application program is not the instant messaging application, the step of obtaining the current packet loss rate of the decoding end device is returned.

[0021] In one embodiment, before obtaining the current packet loss rate of the decoding end device, it further includes:

[0022] Obtain encoding parameters; the encoding parameters at least include image resolution, image division rule, encoding method, and quantization parameter;

[0023] Encode the encoding parameters to obtain an encoding parameter bitstream;

[0024] Send the encoding parameter bitstream to the decoding end device.

[0025] In one embodiment, the step of dividing the image frame to be encoded into multiple target areas to be encoded includes:

[0026] Divide the image frame to be encoded into multiple target areas to be encoded according to the macroblock row division rule.

[0027] In one embodiment, it further includes:

[0028] Determine whether the encoding of all target regions to be encoded is completed;

[0029] When it is determined that the encoding of all target regions to be encoded is not completed, sequentially obtain the target regions to be encoded that have not been encoded and perform encoding until the encoding of all target regions to be encoded is completed.

[0030] According to the second aspect of the embodiments of the present disclosure, there is provided an encoding and decoding method applied to a decoding end device, and the method includes:

[0031] Receive each stream of coded data sent by an encoding end device; the stream of coded data is data obtained by encoding a current target region to be encoded; the current target region to be encoded is a region obtained by dividing an image frame to be encoded;

[0032] Decode the received stream of coded data sequentially to obtain decoded data.

[0033] The embodiments of the present disclosure provide an encoding and decoding method. When an encoding end device determines that the current packet loss rate of a decoding end device is greater than a preset packet loss threshold, it divides an image frame to be encoded into multiple target regions to be encoded, then encodes the current target region to be encoded to obtain a stream of coded data, and while sending the stream of coded data to the decoding end device, encodes the next target region to be encoded, so that the decoding end device decodes each stream of coded data sequentially. In this way, the image frame to be encoded is divided into multiple target regions to be encoded for encoding. The coded stream obtained for each target region to be encoded is smaller than the entire-frame coded stream of the image frame to be encoded. The pressure on the network caused by separately sending each coded stream is smaller than that of the entire-frame coded stream, thereby reducing the packet loss rate of the transmitted coded stream.

[0034] In one embodiment, before receiving each stream of coded data sent by the encoding end device, it further includes:

[0035] Receive a coded stream of encoding parameters sent by the encoding end device;

[0036] Decode the coded stream of encoding parameters to obtain encoding parameters; the encoding parameters at least include image resolution, image division rule, encoding method, and quantization parameter;

[0037] The sequentially decoding the received stream of coded data includes:

[0038] Configure decoding parameters according to the encoding parameters;

[0039] Decode the received stream of coded data according to the decoding parameters.

[0040] In one embodiment, after sequentially decoding the received stream of coded data, it further includes:

[0041] Determine whether the decoding of the bitstream data for all target regions to be encoded corresponding to the image frame to be encoded is completed;

[0042] When it is determined that the decoding of the bitstream data for all target regions to be encoded corresponding to the image frame to be encoded is completed, save or display all the decoded data.

[0043] In one embodiment, the saving or displaying all the decoded data includes:

[0044] Combine all the decoded data according to the image division rule and the image resolution;

[0045] Save or display all the combined decoded data.

[0046] According to the third aspect of the embodiments of the present disclosure, there is provided an encoding end device, including:

[0047] A first processing module, configured to obtain the current packet loss rate of the decoding end device;

[0048] The first processing module is further configured to divide the image frame to be encoded into multiple target regions to be encoded when it is determined that the current packet loss rate is greater than a preset packet loss threshold;

[0049] The first processing module is further configured to encode the current target region to be encoded to obtain bitstream data;

[0050] A sending module, configured to encode the next target region to be encoded while sending the bitstream data to the decoding end device, so that the decoding end device decodes each bitstream data in sequence.

[0051] In one embodiment, the first processing module is further configured to, after the sending module sends the bitstream data to the decoding end device, return to execute the step of obtaining the current packet loss rate of the decoding end device, and when it is determined that the current packet loss rate is greater than the preset packet loss threshold, reduce the size of the target region to be encoded until the current packet loss rate is less than the preset packet loss threshold.

[0052] In one embodiment, the first processing module is further configured to obtain the type of the current application program;

[0053] The first processing module is further configured to obtain the current delay information of the decoding end device when it is determined that the current application program is an instant messaging application;

[0054] The first processing module is further configured to, when determining that the current delay information is greater than a preset delay threshold, return to execute the step of dividing the image frame to be encoded into multiple target regions to be encoded, and increase the size of the target regions to be encoded until the current delay information is less than the preset delay threshold.

[0055] In one embodiment, the first processing module is further configured to, when determining that the current delay information is less than the preset delay threshold, return to execute the step of encoding the current target region to be encoded.

[0056] In one embodiment, the first processing module is further configured to, when determining that the current application is not the instant messaging application, return to execute the step of obtaining the current packet loss rate of the decoding end device.

[0057] In one embodiment, the encoding end device further includes a second sending module;

[0058] The first processing module is further configured to obtain encoding parameters; the encoding parameters at least include image resolution, image division rule, encoding method, and quantization parameters;

[0059] The first processing module is further configured to encode the encoding parameters to obtain an encoding parameter bitstream;

[0060] The second sending module is configured to send the encoding parameter bitstream to the decoding end device.

[0061] In one embodiment, the first processing module includes a division sub-module;

[0062] The division sub-module is configured to divide the image frame to be encoded into multiple target regions to be encoded according to the macroblock row division rule.

[0063] In one embodiment, the first processing module is further configured to determine whether the encoding of all target regions to be encoded is completed. When it is determined that the encoding of all target regions to be encoded is not completed, the unencoded target regions to be encoded are sequentially obtained for encoding until the encoding of all target regions to be encoded is completed.

[0064] According to a fourth aspect of the embodiments of the present disclosure, a decoding end device is provided, including:

[0065] A first receiving module, configured to receive bitstream data sent by an encoding end device; the bitstream data is data obtained by encoding a current target region to be encoded; the current target region to be encoded is a region obtained by dividing an image frame to be encoded;

[0066] A second processing module, configured to decode the received bitstream data to obtain decoded data.

[0067] In one embodiment, the decoding end device further includes a second receiving module, and the second processing module includes a configuration sub-module and a decoding sub-module;

[0068] The second receiving module is configured to receive the encoded parameter bitstream sent by the encoding end device;

[0069] The second processing module is configured to decode the encoded parameter bitstream to obtain encoded parameters; the encoded parameters at least include image resolution, image partitioning rule, encoding method, and quantization parameter;

[0070] The configuration sub-module is configured to configure decoding parameters according to the encoded parameters;

[0071] The decoding sub-module is configured to decode the received bitstream data according to the decoding parameters.

[0072] In one embodiment, the second processing module is further configured to determine whether the decoding of the bitstream data of all target regions to be encoded corresponding to the image frame to be encoded is completed; when it is determined that the decoding of the bitstream data of all target regions to be encoded corresponding to the image frame to be encoded is completed, all the decoded data is saved or displayed.

[0073] In one embodiment, the second processing module further includes a combination sub-module and a display sub-module;

[0074] The combination sub-module is configured to combine all the decoded data according to the image partitioning rule and the image resolution;

[0075] The display sub-module is configured to save or display all the combined decoded data.

[0076] According to a fifth aspect of the embodiments of the present disclosure, there is provided an encoding and decoding system, including the encoding end device described in any of the above embodiments and the decoding end device described in any of the above embodiments.

[0077] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure.

[0079] Figure 1 is a flowchart of an encoding and decoding method provided by an embodiment of the present disclosure;

[0080] Figure 2It is a schematic structural diagram of the region division of an image frame to be encoded provided by an embodiment of the present disclosure;

[0081] Figure 3 It is a schematic structural diagram of the region division of an image frame to be encoded provided by an embodiment of the present disclosure;

[0082] Figure 4 It is a schematic flowchart of an encoding and decoding process provided by an embodiment of the present disclosure;

[0083] Figure 5 It is a schematic flowchart of an encoding and decoding process provided by an embodiment of the present disclosure;

[0084] Figure 6 It is a flowchart of an encoding and decoding method provided by an embodiment of the present disclosure;

[0085] Figure 7 It is a flowchart of an encoding and decoding method provided by an embodiment of the present disclosure;

[0086] Figure 8 It is a flowchart of an encoding and decoding method provided by an embodiment of the present disclosure;

[0087] Figure 9 It is a schematic structural diagram of the region division of an image frame to be encoded provided by an embodiment of the present disclosure;

[0088] Figure 10 It is a flowchart of an encoding and decoding method provided by an embodiment of the present disclosure;

[0089] Figure 11 It is a flowchart of an encoding and decoding method provided by an embodiment of the present disclosure;

[0090] Figure 12a It is a structural diagram of an encoding end device provided by an embodiment of the present disclosure;

[0091] Figure 12b It is a structural diagram of an encoding end device provided by an embodiment of the present disclosure;

[0092] Figure 12c It is a structural diagram of an encoding end device provided by an embodiment of the present disclosure;

[0093] Figure 13a It is a structural diagram of a decoding end device provided by an embodiment of the present disclosure;

[0094] Figure 13b It is a structural diagram of a decoding end device provided by an embodiment of the present disclosure;

[0095] Figure 13c It is a structural diagram of a decoding end device provided by an embodiment of the present disclosure;

[0096] Figure 14It is a block diagram of an encoding and decoding system provided by an embodiment of the present disclosure. Detailed implementation manners

[0097] Here, exemplary embodiments will be described in detail, and examples thereof are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0098] An embodiment of the present disclosure provides an encoding and decoding method, which is applied to an encoding end device, such as Figure 1 As shown, the method includes the following steps:

[0099] Step 101: Obtain the current packet loss rate of the decoding end device.

[0100] Exemplarily, the encoding end device sends a packet loss rate acquisition request to the decoding end device. When the decoding end device receives the packet loss rate acquisition request, it sends the calculated current packet loss rate to the encoding end device, and then the encoding end device obtains the current packet loss rate of the decoding end device; or when the decoding end device calculates the current packet loss rate, it actively sends the current packet loss rate to the encoding end device, so that the encoding end device obtains the current packet loss rate of the decoding end device.

[0101] Step 102: When it is determined that the current packet loss rate is greater than a preset packet loss threshold, divide the image frame to be encoded into multiple target regions to be encoded.

[0102] Exemplarily, when the encoding end device obtains the current packet loss rate of the decoding end device, it compares the current packet loss rate with the preset packet loss threshold. When it is determined that the current packet loss rate is greater than the preset packet loss threshold, it indicates that the current network bandwidth is small, resulting in serious packet loss. At this time, the image frame to be encoded is divided into multiple target regions to be encoded; when it is determined that the current packet loss rate is less than the preset packet loss threshold, it indicates that the current network bandwidth is large and can meet the transmission of the entire frame bitstream. At this time, the image frame to be encoded is encoded as a whole frame, maintaining the non-split encoding method.

[0103] Optionally, divide the image frame to be encoded into multiple target regions to be encoded according to the macroblock row division rule.

[0104] Exemplarily, the image frame to be encoded includes multiple macroblocks, and each macroblock is composed of multiple pixel points. The image frame to be encoded is divided according to the macroblock row division rule. The macroblock row division rule means that one row of macroblocks or multiple rows of macroblocks are used as a target region to be encoded. For example Figure 2 As shown, the image frame to be encoded is divided with every two rows of macroblocks as a target region to be encoded. InFigure 2 The image frame to be encoded is split into multiple non-overlapping rectangular regions, each rectangular region contains multiple complete macroblock rows, and the areas of each rectangular region are equal or close. Here, "close" means dividing the target region to be encoded according to a unified macroblock row division rule. In this way, the last target region to be encoded may be different from the previous target regions to be encoded. For example, if the macroblock rows are taken as two rows as a target region to be encoded, and the image frame to be encoded has an odd number of rows, then the last target region to be encoded is 3 rows or 1 row of macroblocks, and the area is different from the previous target regions to be encoded, but it does not affect the change of the encoding rule.

[0105] Exemplarily, the image frame to be encoded can also be evenly divided. For example, the image frame to be encoded is divided into M macroblock regions of the same size, and each macroblock region is a target region to be encoded. Even division can ensure that after multiple threads are started, the encoding time for each thread for macroblock regions of the same size is the same, giving full play to the advantages of multi-core. As Figure 3 shown, the size of each target region to be encoded is the same, that is, each target region to be encoded is a macroblock region of 4*2 size.

[0106] It should be noted that if the split target regions to be encoded are smaller, the overall encoded bitstream may increase; if the split target regions to be encoded are larger, the overall encoding and decoding time will be longer. The specific region splitting can be preset according to requirements or automatically adjusted based on certain conditions.

[0107] It should be noted that no matter which rule is used to split the image frame to be encoded into multiple target regions to be encoded, the encoding is performed according to the rows composed of macroblocks, and all the target regions to be encoded do not overlap.

[0108] Step 103: Encode the current target region to be encoded to obtain bitstream data.

[0109] Exemplarily, when the image frame to be encoded is divided into multiple target regions to be encoded, each target region to be encoded is encoded in sequence to obtain the bitstream data corresponding to each target region to be encoded.

[0110] It should be noted that an encoding process usually includes processes such as intra-frame prediction, pixel translation, pixel conversion, quantization, and encoding. For the specific encoding method of each target region to be encoded, reference can be made to the prior art, and details are not elaborated herein.

[0111] Step 104: While sending the bitstream data to the decoding device, encode the next target region to be encoded, so that the decoding device decodes each bitstream data in sequence.

[0112] Exemplarily, asFigure 4 As shown, when both the encoding device and the decoding device operate in a single-threaded manner, the encoding device encodes the first target area to be encoded. After the encoding is completed, the obtained bitstream data is sent to the decoding device, and immediately starts encoding the second target area to be encoded. When the decoding device receives each bitstream data, it immediately starts decoding. In this way, the encoding process and the decoding process are parallel, and most of the encoding and decoding times overlap. The total encoding and decoding duration is equal to the total encoding duration plus the duration of decoding the bitstream data corresponding to the last target area to be encoded. As Figure 5 shown, when both the encoding device and the decoding device operate in a multi-threaded manner, using the encoding and decoding method of the present disclosure can further shorten the encoding and decoding time. Figure 5 Both the encoding device and the decoding device adopt three threads.

[0113] It should be noted that it is preferably to select multi-threading for encoding and transmission, which can reduce the encoding and transmission time of the overall bitstream; if the services that cannot be split into parallel are involved, single-threading is selected for processing.

[0114] Further, after step 104 is executed, return to execute step 101 to obtain the current packet loss rate of the decoding device, and when it is determined that the current packet loss rate is greater than the preset packet loss threshold, reduce the size of the target area to be encoded until the current packet loss rate is less than the preset packet loss threshold.

[0115] Exemplarily, when sending the bitstream data corresponding to the current area to be encoded to the decoding device, the decoding device can calculate the packet loss rate corresponding to the bitstream data, which is the current packet loss rate. The encoding device also needs to obtain the current packet loss rate of the decoding device, and when it is determined that the current packet loss rate is still greater than the preset packet loss threshold, continue to divide the target area to be encoded to reduce the size of the target area to be encoded, and encode and transmit the reduced target area to be encoded. In this way, the size of the target area to be encoded is gradually reduced until the current packet loss rate is less than the preset packet loss threshold.

[0116] Further, as Figure 6 shown, the method further includes the following steps 105 to 109:

[0117] Step 105: Obtain the type of the current application program.

[0118] Exemplarily, the encoding device pre-stores the correspondence between the identifier and type of each application program. According to the identifier of the current application program, the type of the current application program can be found. The types of application programs include instant messaging applications, game applications, office applications, entertainment applications, etc.

[0119] Step 106: When it is determined that the current application is an instant messaging application, obtain the current latency information of the decoding device.

[0120] Exemplarily, when it is found that the type of the current application is an instant messaging application, a latency acquisition request is sent to the decoding device. When the decoding device receives the latency acquisition request, it sends the calculated current latency information to the encoding device, and then the encoding device obtains the current latency information of the decoding device; or when the decoding device calculates the current latency information, it actively sends the current latency information to the encoding device, so that the encoding device obtains the current latency information of the decoding device.

[0121] Step 107: When it is determined that the current latency information is greater than the preset latency threshold, return to execute Step 102, and increase the size of the target area to be encoded until the current latency information is less than the preset latency threshold.

[0122] Exemplarily, when the encoding device obtains the current latency information of the decoding device, it compares the current latency information with the preset latency threshold. When it is determined that the current latency information is greater than the preset latency threshold, it means that the target area to be encoded divided at this time is too small and the bitstream is large. It is necessary to re-divide the image frame to be encoded to increase the size of the target area to be encoded, and encode and transmit the increased target area to be encoded. In this way, the size of the target area to be encoded is gradually increased until the current latency information is less than the preset latency threshold.

[0123] It should be noted that when the current latency information cannot be less than the preset latency threshold all the time, the image frame to be encoded is not split, and the whole frame of the image frame to be encoded is encoded.

[0124] Step 108: When it is determined that the current latency information is less than the preset latency threshold, return to execute Step 101 to encode the current target area to be encoded.

[0125] Exemplarily, when it is determined that the current latency information is less than the preset latency threshold, it means that the area to be encoded is reasonably divided, and encoding can be performed according to the currently split target area to be encoded.

[0126] Step 109: When it is determined that the current application is not the instant messaging application, return to execute Step 101 to obtain the current packet loss rate of the decoding device.

[0127] Exemplarily, when it is found that the current application is not an instant messaging application, it means that the current application has low requirements for latency. At this time, it can directly return to execute Step 101 to obtain the current packet loss rate of the decoding device, and split and encode the image frame to be encoded according to the size of the current packet loss rate, etc., without considering the latency information.

[0128] Further, as Figure 7 shown, before performing step 101, the following steps 110 to 112 are further included.

[0129] Step 110: Obtain encoding parameters.

[0130] Among them, the encoding parameters at least include image resolution, image partitioning rule, encoding method, and quantization parameter; the image resolution is used to identify the fineness of image details. Generally, the higher the resolution of the image, the more pixel points it contains, the clearer the image, the better the printing quality, and at the same time, it will also increase the storage space occupied by the file; the image partitioning rule is a specific method for partitioning the image frame to be encoded into multiple target regions to be encoded; the encoding method is the encoding rule adopted during encoding.

[0131] Exemplarily, the user can input encoding parameters such as image resolution, image partitioning rule, encoding method, and quantization parameter in the corresponding decoder in advance, so that the encoding device obtains the encoding parameters.

[0132] It should be noted that the encoding parameters may also include other parameters required during encoding. For details, reference can be made to the prior art, and the present disclosure will not elaborate herein.

[0133] Step 111: Encode the encoding parameters to obtain an encoding parameter bitstream.

[0134] Exemplarily, before encoding the image frame to be encoded, the encoding device needs to first encode the obtained encoding parameters to obtain an encoding parameter bitstream.

[0135] Step 112: Send the encoding parameter bitstream to the decoding device.

[0136] Exemplarily, when the encoding device obtains the encoding parameter bitstream, it sends the encoding parameter bitstream to the decoding device, so that when the decoding device receives the encoding reference bitstream, it decodes the encoding reference bitstream to obtain the encoding parameters, configures the decoding parameters according to the encoding parameters, and then decodes according to the decoding parameters.

[0137] Further, as Figure 8 shown, after performing step 104, the method further includes the following steps 113 and 114:

[0138] Step 113: Determine whether the encoding of all target regions to be encoded is completed.

[0139] Exemplarily, when the encoding end device finishes encoding each target region to be encoded, it needs to determine whether it has completed encoding each target region to be encoded in the division of the image frame to be encoded. When it determines that it has completed encoding each target region to be encoded in the division of the image frame to be encoded, the encoding ends.

[0140] Step 114, when it is determined that not all target regions to be encoded have been encoded, sequentially obtain the target regions to be encoded that have not been encoded for encoding until all target regions to be encoded have been encoded.

[0141] Exemplarily, when it is determined that not all target regions to be encoded in the division of the image frame to be encoded have been encoded, sequentially obtain the target regions to be encoded that have not been encoded for encoding until all target regions to be encoded in the division of the image frame to be encoded have been encoded, and at this time the encoding ends.

[0142] It should be noted that when dividing the image frame to be encoded into multiple target regions to be encoded, each target region to be encoded can also be divided into multiple target sub-regions to be encoded, and then each target sub-region to be encoded is sequentially encoded, and then the bitstreams of each obtained target sub-region to be encoded are merged to obtain the bitstream data of the target region to be encoded, and then the bitstream data is sent to the decoding end device for decoding together. The bitstream structure of the bitstream data of each target region to be encoded is as Figure 9 shown. The bitstream structure contains multiple image encoding data to facilitate the re-splitting of the encoded region. The specific bitstream structure includes a start flag, a region number, encoding parameters, and multiple groups of encoding data. Each group of encoding data includes an image data encoding start flag, image encoding data, and an image data encoding end flag. Among them, the start flag is used to identify the start of encoding a new target region to be encoded; the region number is used to identify the absolute position of the image of the target region to be encoded in the entire image frame to be encoded. For example, as Figure 9 shown, the image to be encoded is split into four regions, and the position identifier of each region can be 00 (upper left), 01 (upper right), 10 (lower left), 11 (lower right). The encoding parameters are the above-mentioned encoding parameters; the image data encoding start flag is used to identify the start of encoding the image data of the macroblock region at this position; the image data encoding end flag is used to identify that the encoding of the image data of the macroblock region at this position has been completed; the image encoding data is the bitstream obtained by encoding each target sub-region to be encoded.

[0143] An embodiment of the present disclosure provides an encoding and decoding method. When the encoding end device determines that the current packet loss rate of the decoding end device is greater than a preset packet loss threshold, it divides the image frame to be encoded into multiple target regions to be encoded, then encodes the current target region to be encoded to obtain bitstream data, and while sending the bitstream data to the decoding end device, encodes the next target region to be encoded, so that the decoding end device decodes each bitstream data in sequence. In this way, by dividing the image frame to be encoded into multiple target regions to be encoded, the encoded bitstream obtained for each target region to be encoded is smaller than the entire-frame bitstream of the image frame to be encoded, and the pressure on the network caused by separately sending each encoded bitstream is smaller than that of the entire-frame bitstream, thereby reducing the packet loss rate of the transmitted bitstream.

[0144] An embodiment of the present disclosure provides an encoding and decoding method, which is applied to a decoding end device. As Figure 10 shown, the method includes the following steps:

[0145] Step 1001: Receive each bitstream data sent by the encoding end device.

[0146] Wherein, the bitstream data is data obtained by encoding the current target region to be encoded; the current target region to be encoded is a region obtained by dividing the image frame to be encoded.

[0147] Step 1002: Decode the received bitstream data in sequence to obtain decoded data.

[0148] An embodiment of the present disclosure provides an encoding and decoding method. When the encoding end device determines that the current packet loss rate of the decoding end device is greater than a preset packet loss threshold, it divides the image frame to be encoded into multiple target regions to be encoded, then encodes the current target region to be encoded to obtain bitstream data, and while sending the bitstream data to the decoding end device, encodes the next target region to be encoded, so that the decoding end device decodes each bitstream data in sequence. In this way, by dividing the image frame to be encoded into multiple target regions to be encoded, the encoded bitstream obtained for each target region to be encoded is smaller than the entire-frame bitstream of the image frame to be encoded, and the pressure on the network caused by separately sending each encoded bitstream is smaller than that of the entire-frame bitstream, thereby reducing the packet loss rate of the transmitted bitstream.

[0149] An embodiment of the present disclosure provides an encoding and decoding method, which is applied to a decoding end device. As Figure 11 shown, the method includes the following steps:

[0150] Step 1101: Receive the encoded parameter bitstream sent by the encoding end device.

[0151] Step 1102: Decode the encoded parameter bitstream to obtain encoded parameters.

[0152] Among them, the encoding parameters at least include image resolution, image partitioning rule, encoding method, and quantization parameters.

[0153] Exemplarily, when the decoding device receives the encoding parameter bitstream, it decodes the encoding parameter bitstream to obtain encoding parameters such as image resolution, image partitioning rule, encoding method, and quantization parameters.

[0154] Step 1103: Configure decoding parameters according to the encoding parameters.

[0155] Exemplarily, when the decoding device obtains the encoding parameters, it configures the decoding parameters according to the corresponding encoding parameters. The specific configuration method can refer to the prior art, and details are not elaborated herein.

[0156] Step 1104: Receive each bitstream data sent by the encoding device.

[0157] Step 1105: Decode the received bitstream data according to the decoding parameters.

[0158] Exemplarily, when the decoding device receives the bitstream data, it immediately decodes the bitstream data according to the decoding parameters to obtain decoded data.

[0159] Step 1106: Determine whether the decoding of the bitstream data for all target regions to be encoded corresponding to the image frame to be encoded is completed.

[0160] Exemplarily, when the decoding device finishes decoding each bitstream data, it needs to determine whether new bitstream data is received. If no new bitstream data is received, it waits; if new bitstream data is received, it determines whether the new bitstream data is the bitstream data of the image frame to be encoded. If it is determined that the new bitstream data is the bitstream data of the image frame to be encoded, it continues to decode the new bitstream data. In this way, it can be determined whether the decoding of the bitstream data for all target regions to be encoded corresponding to the image frame to be encoded is completed.

[0161] Step 1107: When it is determined that the decoding of the bitstream data for all target regions to be encoded corresponding to the image frame to be encoded is completed, save or display all the decoded data.

[0162] Optionally, combine all the decoded data according to the image partitioning rule and the image resolution; save or display all the combined decoded data.

[0163] Among them, the bitstream data is the data obtained by encoding the current target region to be encoded; the current target region to be encoded is the region obtained by partitioning the image frame to be encoded.

[0164] Exemplarily, when it is determined that the decoding of the bitstream data of all target regions to be encoded corresponding to the image frame to be encoded is completed, the obtained decoding data for the image frame to be encoded is combined according to the image division rule and the image resolution in the encoding parameters, and finally, all the decoding data of the image frame to be encoded is obtained, and all the decoding data of the image frame to be encoded is displayed or stored in the storage module to complete the entire decoding and display process of the image frame to be encoded.

[0165] An embodiment of the present disclosure provides an encoding and decoding method. When the encoding end device determines that the current packet loss rate of the decoding end device is greater than a preset packet loss threshold, the image frame to be encoded is divided into multiple target regions to be encoded, and then the current target region to be encoded is encoded to obtain bitstream data. At the same time as sending the bitstream data to the decoding end device, the next target region to be encoded is encoded, so that the decoding end device decodes each bitstream data in sequence. In this way, the image frame to be encoded is divided into multiple target regions to be encoded for encoding. The encoded bitstream obtained for each target region to be encoded is smaller than the entire-frame bitstream of the image frame to be encoded. The pressure brought by separately sending each encoded bitstream to the network is smaller than that of the entire-frame bitstream, thereby reducing the packet loss rate of the transmitted bitstream.

[0166] Based on the encoding and decoding method described in the above embodiments, the following is an embodiment of the device of the present disclosure, which can be used to execute the method embodiment of the present disclosure.

[0167] An embodiment of the present disclosure provides an encoding end device, as Figure 12a shown. The encoding end device 120 includes a first processing module 1201 and a first sending module 1202.

[0168] Among them, the first processing module 1201 is used to obtain the current packet loss rate of the decoding end device;

[0169] The first processing module 1201 is further configured to divide the image frame to be encoded into multiple target regions to be encoded when it is determined that the current packet loss rate is greater than a preset packet loss threshold.

[0170] The first processing module 1201 is further configured to encode the current target region to be encoded to obtain bitstream data.

[0171] The sending module 1202 is configured to encode the next target region to be encoded while sending the bitstream data to the decoding end device, so that the decoding end device decodes each bitstream data in sequence.

[0172] In one embodiment, the first processing module 1201 is further configured to, after the sending module sends the bitstream data to the decoding end device, return to execute the step of obtaining the current packet loss rate of the decoding end device, and when it is determined that the current packet loss rate is greater than the preset packet loss threshold, reduce the size of the target area to be encoded until the current packet loss rate is less than the preset packet loss threshold.

[0173] In one embodiment, the first processing module 1201 is further configured to obtain the type of the current application program.

[0174] The first processing module 1201 is further configured to, when it is determined that the current application program is an instant messaging application, obtain the current latency information of the decoding end device.

[0175] The first processing module 1201 is further configured to, when it is determined that the current latency information is greater than the preset latency threshold, return to execute the step of dividing the image frame to be encoded into multiple target areas to be encoded, and increase the size of the target area to be encoded until the current latency information is less than the preset latency threshold.

[0176] In one embodiment, the first processing module 1201 is further configured to, when it is determined that the current latency information is less than the preset latency threshold, return to execute the step of encoding the current target area to be encoded.

[0177] In one embodiment, the first processing module 1201 is further configured to, when it is determined that the current application program is not the instant messaging application, return to execute the step of obtaining the current packet loss rate of the decoding end device.

[0178] In one embodiment, as Figure 12b shown, the encoding end device further includes a second sending module 1203.

[0179] Wherein, the first processing module 1201 is further configured to obtain encoding parameters; the encoding parameters at least include image resolution, image division rule, encoding method, and quantization parameter.

[0180] The first processing module 1201 is further configured to encode the encoding parameters to obtain an encoding parameter bitstream.

[0181] The second sending module 1203 is configured to send the encoding parameter bitstream to the decoding end device.

[0182] In one embodiment, as Figure 12c shown, the first processing module 1201 includes a division sub-module 12011.

[0183] Among them, the sub-module 12011 for division is configured to divide the image frame to be encoded into a plurality of the target regions to be encoded according to the macroblock row division rule.

[0184] In one embodiment, the first processing module 1201 is further configured to determine whether the encoding of all the target regions to be encoded is completed. When it is determined that the encoding of all the target regions to be encoded is not completed, the target regions to be encoded that have not been encoded are sequentially obtained for encoding until the encoding of all the target regions to be encoded is completed.

[0185] An embodiment of the present disclosure provides an encoding device. When the encoding device determines that the current packet loss rate of the decoding device is greater than a preset packet loss threshold, the image frame to be encoded is divided into a plurality of target regions to be encoded, and then the current target region to be encoded is encoded to obtain bitstream data. While sending the bitstream data to the decoding device, the next target region to be encoded is encoded, so that the decoding device decodes each bitstream data in sequence. In this way, the image frame to be encoded is divided into a plurality of target regions to be encoded for encoding. The encoded bitstream obtained for each target region to be encoded is smaller than the entire-frame bitstream of the image frame to be encoded. The pressure brought by separately sending each encoded bitstream to the network is smaller than that of the entire-frame bitstream, thereby reducing the packet loss rate of the transmitted bitstream.

[0186] An embodiment of the present disclosure provides a decoding device, as Figure 13a shown. The decoding device 130 includes a first receiving module 1301 and a second processing module 1302.

[0187] Among them, the first receiving module 1301 is configured to receive the bitstream data sent by the encoding device; the bitstream data is data obtained by encoding the current target region to be encoded; the current target region to be encoded is a region obtained by dividing the image frame to be encoded.

[0188] The second processing module 1302 is configured to decode the received bitstream data to obtain decoded data.

[0189] In one embodiment, as Figure 13b shown, the decoding device further includes a second receiving module 1303, and the second processing module 1302 includes a configuration sub-module 13021 and a decoding sub-module 13022.

[0190] Among them, the second receiving module 1303 is configured to receive the encoded parameter bitstream sent by the encoding device.

[0191] The second processing module 1302 is configured to decode the encoded parameter bitstream to obtain encoded parameters; the encoded parameters at least include image resolution, image division rule, encoding method, and quantization parameter.

[0192] The configuration sub-module 13021 is configured to configure decoding parameters according to the encoding parameters.

[0193] The decoding sub-module 13022 is configured to decode the received bitstream data according to the decoding parameters.

[0194] In one embodiment, the second processing module 1302 is further configured to determine whether the decoding of the bitstream data for all target regions to be encoded corresponding to the image frame to be encoded is completed; when it is determined that the decoding of the bitstream data for all target regions to be encoded corresponding to the image frame to be encoded is completed, save or display all the decoded data.

[0195] In one embodiment, as Figure 13c shown, the second processing module 1032 further includes a combining sub-module 13023 and a display sub-module 13024.

[0196] Among them, the combining sub-module 13023 is configured to combine all the decoded data according to the image division rule and the image resolution.

[0197] The display sub-module 13024 is configured to save or display all the combined decoded data.

[0198] The embodiments of the present disclosure provide a decoding-end device. When the encoding-end device determines that the current packet loss rate of the decoding-end device is greater than a preset packet loss threshold, the image frame to be encoded is divided into multiple target regions to be encoded, and then the current target region to be encoded is encoded to obtain bitstream data. While sending the bitstream data to the decoding-end device, the next target region to be encoded is encoded, so that the decoding-end device decodes each bitstream data in sequence. In this way, the image frame to be encoded is divided into multiple target regions to be encoded for encoding. The encoded bitstream obtained for each target region to be encoded is smaller than the entire-frame bitstream of the image frame to be encoded. The pressure brought by separately sending each encoded bitstream to the network is smaller than that of the entire-frame bitstream, thereby reducing the packet loss rate of the transmitted bitstream.

[0199] Figure 14 is a framework diagram of an encoding and decoding system provided by the embodiments of the present disclosure. As Figure 14 shown, the encoding and decoding system includes the encoding-end device 1401 described in any of the above embodiments and the decoding-end device 1402 described in any of the above embodiments.

[0200] Among them, the decoding device 1402 is a cloud terminal device or a computer device. A decoding module is provided on the cloud terminal device or the computer device. The encoding device 1401 is a virtual machine corresponding to the cloud terminal device or the computer device running on a cloud server. An encoding module is provided on the virtual machine. The virtual machine is correspondingly connected to the cloud terminal device or the computer device. This disclosure is applied to scenarios of cloud terminal devices such as cloud phones, bracelets, watches, glasses, and finger rings, or scenarios of computer devices such as computers and tablets. Based on VDI (Virtual Desktop Infrastructure) technology, the VDI architecture includes a cloud server and at least one cloud terminal device or computer device. At least one virtual machine runs on the cloud server. By running an operating system on the cloud server in the data center, the user's desktop is virtualized. The user accesses the corresponding virtual machine through the cloud terminal device or the computer device, thereby obtaining a desktop image and controlling the obtained desktop image in a reverse control manner.

[0201] Based on the above Figure 1 corresponding encoding and decoding method described in the embodiment, the embodiment of the present disclosure also provides a computer-readable storage medium. For example, a non-transitory computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc. Computer instructions are stored on this storage medium for executing the above Figure 1 corresponding encoding and decoding method described in the embodiment, which will not be elaborated here.

[0202] Based on the above Figure 10 corresponding encoding and decoding method described in the embodiment, the embodiment of the present disclosure also provides a computer-readable storage medium. For example, a non-transitory computer-readable storage medium may be a read-only memory, a random access memory, a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc. Computer instructions are stored on this storage medium for executing the above Figure 10 corresponding encoding and decoding method described in the embodiment, which will not be elaborated here.

[0203] Those of ordinary skill in the art can understand that all or part of the steps to implement the above embodiments can be completed by hardware, or can be completed by a program instructing relevant hardware. The program can be stored in a computer-readable storage medium. The above-mentioned storage medium can be a read-only memory, a magnetic disk, or an optical disc, etc.

[0204] Other embodiments of the present disclosure will be readily apparent to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common general knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and examples are only to be considered as exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

Claims

1. A coding and decoding method, characterized in that, Applied to the encoding end device, the method includes: Obtain the current packet loss rate of the decoding end device; When it is determined that the current packet loss rate is greater than a preset packet loss threshold, divide the image frame to be encoded into multiple target regions to be encoded; Encode the current target region to be encoded to obtain bitstream data; While sending the bitstream data to the decoding end device, encode the next target region to be encoded, so that the decoding end device decodes each bitstream data in sequence; After sending the bitstream data to the decoding end device, return to execute the step of obtaining the current packet loss rate of the decoding end device, and when it is determined that the current packet loss rate is greater than the preset packet loss threshold, reduce the size of the target region to be encoded until the current packet loss rate is less than the preset packet loss threshold.

2. The method according to claim 1, characterized in that It further includes: Obtain the type of the current application; When it is determined that the current application is an instant messaging application, obtain the current delay information of the decoding end device; When it is determined that the current delay information is greater than a preset delay threshold, return to execute the step of dividing the image frame to be encoded into multiple target regions to be encoded, and increase the size of the target region to be encoded until the current delay information is less than the preset delay threshold.

3. The method according to claim 2, characterized in that, It further includes: When it is determined that the current delay information is less than the preset delay threshold, return to execute the step of encoding the current target region to be encoded.

4. The method according to claim 2, wherein It further includes: When it is determined that the current application is not the instant messaging application, return to execute the step of obtaining the current packet loss rate of the decoding end device.

5. The method according to claim 1, characterized in that, Before obtaining the current packet loss rate of the decoding end device, it further includes: Obtain encoding parameters; the encoding parameters at least include image resolution, image division rule, encoding method, and quantization parameter; Encode the encoding parameters to obtain an encoding parameter bitstream; Send the encoding parameter bitstream to the decoding end device.

6. The method according to claim 1, characterized in that, The step of dividing the image frame to be encoded into multiple target regions to be encoded includes: Divide the image frame to be encoded into multiple target regions to be encoded according to the macroblock row division rule.

7. The method according to claim 1, characterized in that, It further includes: Determine whether the encoding of all target regions to be encoded is completed; When it is determined that the encoding of all target regions to be encoded is not completed, sequentially obtain the target regions to be encoded that have not been encoded for encoding until the encoding of all target regions to be encoded is completed.

8. A coding and decoding method, applied to a decoding end device, characterized in that, It includes: Receive each bitstream data sent by the encoding end device; The bitstream data is data obtained by encoding the current target region to be encoded; The current target region to be encoded is a region obtained by dividing the image frame to be encoded; Decode the received bitstream data in sequence to obtain decoded data; The bitstream data is encoded according to the encoding and decoding method according to any one of claims 1-7.

9. The method according to claim 8, wherein Before receiving each bitstream data sent by the encoding end device, it further includes: Receive the encoding parameter bitstream sent by the encoding end device; Decode the encoding parameter bitstream to obtain encoding parameters; the encoding parameters at least include image resolution, image division rule, encoding method, and quantization parameter; The step of decoding the received bitstream data in sequence includes: Configure decoding parameters according to the encoding parameters; Decode the received bitstream data according to the decoding parameters.

10. The method according to claim 9, characterized in that, After sequentially decoding the received bitstream data, it further includes: Determine whether the decoding of the bitstream data for all target regions to be encoded corresponding to the image frame to be encoded is completed; When it is determined that the decoding of the bitstream data for all target regions to be encoded corresponding to the image frame to be encoded is completed, save or display all the decoded data.

11. The method according to claim 10, wherein The saving or displaying all the decoded data includes: Combine all the decoded data according to the image partitioning rule and the image resolution; Save or display all the combined decoded data.

12. An encoding terminal device, characterized in that, It includes: A first processing module for obtaining the current packet loss rate of the decoding device; The first processing module is further configured to divide the image frame to be encoded into multiple target regions to be encoded when it is determined that the current packet loss rate is greater than a preset packet loss threshold; The first processing module is further configured to encode the current target region to be encoded to obtain bitstream data; A first sending module for, while sending the bitstream data to the decoding device, encoding the next target region to be encoded, so that the decoding device sequentially decodes each bitstream data; The first processing module is further configured to obtain the type of the current application program; The first processing module is further configured to obtain the current delay information of the decoding device when it is determined that the current application program is an instant messaging application; The first processing module is further configured to, when it is determined that the current delay information is greater than a preset delay threshold, return to execute the step of dividing the image frame to be encoded into multiple target regions to be encoded, and increase the size of the target region to be encoded until the current delay information is less than the preset delay threshold.

13. A decoding end device, characterized in that, It includes: A first receiving module for receiving the bitstream data sent by the encoding device; The bitstream data is data obtained by encoding the current target region to be encoded; the current target region to be encoded is a region obtained by dividing the image frame to be encoded; A second processing module for decoding the received bitstream data to obtain decoded data; The bitstream data is encoded by the encoding device according to claim 12.

14. A codec system, characterized in that, It includes the encoding device according to claim 12 and the decoding device according to claim 13.

Images