Data encapsulation method and related apparatus, device, medium

Abstract

The application discloses a data encapsulation method and related device, equipment and medium, wherein the data encapsulation method comprises: obtaining first encoding data and second encoding data of a to-be-encoded image; determining a first load required for carrying a transparent channel image by using a preset encapsulation mechanism based on a first data amount of the first encoding data; determining a first number of data packets required for carrying the to-be-encoded image based on the first load and a second data amount of the second encoding data; determining a third data amount of the first encoding data respectively allocated to each data packet based on the first data amount and the first number; for each data packet, determining a second load required for carrying the third data amount by using the preset encapsulation mechanism, and allocating an unallocated part of the second encoding data to a remaining load of the data packet based on the second load and a payload of the data packet. The above scheme can be beneficial to adapt to network transmission and improve the convenience of backend processing.

Description

[0001] This application is a divisional application of the applicant's patent application filed on December 30, 2021, entitled "Compression Encoding Method and Data Encapsulation Method and Related Apparatus, Devices and Media", with application number 202111651192.9. Technical Field

[0002] This application relates to the field of image processing technology, and in particular to a data encapsulation method and related apparatus, devices, and media. Background Technology

[0003] The widespread adoption of smart mobile devices and the continuous development of network technology have made video a viable medium for information transmission, leading to its increasingly widespread use. The alpha channel (hereinafter referred to as the transparency channel) is typically used to represent the transparency of corresponding pixels in an image. The transparency channel plays a crucial role in image processing software, animation creation, and video editing software, among other applications.

[0004] Therefore, how to encapsulate the first encoded data of the transparent channel image after compression encoding to make it suitable for network transmission and improve the convenience of back-end processing has become an urgent problem to be solved. Summary of the Invention

[0005] The main technical problem addressed by this application is to provide a data encapsulation method and related devices, equipment, and media that can encapsulate the first encoded data of a transparent channel image after compression and encoding through a packaging mechanism, which is beneficial for adapting to network transmission and improving the convenience of backend processing.

[0006] To address the aforementioned technical problems, a first aspect of this application provides a data encapsulation method, comprising: acquiring first encoded data and second encoded data of an image to be encoded; wherein the first encoded data is encoded data of the transparency channel image of the image to be encoded, and the second encoded data is encoded data of the chroma and / or luminance of the image to be encoded; determining a first payload required to carry the transparency channel image using a preset encapsulation mechanism based on a first data volume of the first encoded data; determining a first number of data packets required to carry the image to be encoded based on the first payload and the second data volume of the second encoded data; determining a third data volume of the first encoded data allocated to each data packet based on the first data volume and the first number; and for each data packet, determining a second payload required to carry the third data volume using the preset encapsulation mechanism, and allocating the unallocated portion of the second encoded data to the remaining payload of the data packet based on the second payload and the payload of the data packet.

[0007] To address the aforementioned technical problems, a second aspect of this application provides a data encapsulation device, comprising: a data acquisition module, a payload determination module, a quantity determination module, a first allocation module, and a second allocation module. The data acquisition module is used to acquire first encoded data and second encoded data of an image to be encoded; wherein the first encoded data is encoded data of the transparency channel image of the image to be encoded, and the second encoded data is encoded data of the chroma and / or luminance of the image to be encoded. The payload determination module is used to determine, based on a first data volume of the first encoded data, a first payload required to carry the transparency channel image using a preset encapsulation mechanism. The quantity determination module is used to determine, based on the first payload and the second data volume of the second encoded data, a first quantity of data packets required to carry the image to be encoded. The first allocation module is used to determine, based on the first data volume and the first quantity, a third data volume of the first encoded data allocated to each data packet. The second allocation module is used to determine, for each data packet, a second payload required to carry the third data volume using the preset encapsulation mechanism, and, based on the second payload and the effective payload of the data packet, allocate the unallocated portion of the second encoded data to the remaining payload of the data packet.

[0008] To address the aforementioned technical problems, a third aspect of this application provides an electronic device including a memory and a processor coupled to each other. The memory stores program instructions, and the processor executes the program instructions to implement the data encapsulation method of the first aspect described above.

[0009] To address the aforementioned technical problems, a fourth aspect of this application provides a computer-readable storage medium storing program instructions executable by a processor, the program instructions being used to implement the data encapsulation method of the first aspect described above.

[0010] The above scheme obtains first and second encoded data of the image to be encoded, where the first encoded data is the encoded data of the transparency channel image of the image to be encoded, and the second encoded data is the encoded data of the chroma and / or luminance of the image to be encoded. Based on the first data volume of the first encoded data, a first payload required to carry the transparency channel image using a preset packetization mechanism is determined. Based on the first payload and the second data volume of the second encoded data, a first number of data packets required to carry the image to be encoded is determined. Based on this, a third data volume of the first encoded data allocated to each data packet is determined based on the first data volume and the first number. For each data packet, a second payload required to carry the third data volume using the preset packetization mechanism is determined. Based on the second payload and the effective payload of the data packet, the unallocated portion of the second encoded data is allocated to the remaining payload of the data packet. Therefore, the first encoded data of the transparency channel image after compression encoding can be encapsulated through a packetization mechanism, which is beneficial for adapting to network transmission and improving the convenience of backend processing. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating an embodiment of the compression encoding method of this application;

[0012] Figure 2 This is an application diagram illustrating an embodiment of a transparency channel image;

[0013] Figure 3 This is a schematic diagram illustrating the division of an embodiment of a transparency channel image;

[0014] Figure 4 This is a schematic diagram of an embodiment of sub-block partitioning with a side length of 1;

[0015] Figure 5 This is a schematic diagram of an embodiment of sub-block partitioning with a side length of 2;

[0016] Figure 6 This is a schematic diagram of an embodiment of sub-block partitioning with a side length of 3;

[0017] Figure 7 yes Figure 1 A flowchart illustrating an embodiment of step S12;

[0018] Figure 8 This is a schematic diagram illustrating the division of another embodiment of a transparency channel image;

[0019] Figure 9 This is a schematic diagram of an embodiment of pixel expansion;

[0020] Figure 10 This is a flowchart illustrating an embodiment of the data encapsulation method of this application;

[0021] Figure 11 This is a schematic diagram of the structure of an embodiment of the RTP protocol packet;

[0022] Figure 12 This is a schematic diagram of a one-byte header extension implementation;

[0023] Figure 13 This is a schematic diagram of an embodiment of a two-byte header extension;

[0024] Figure 14 This is a schematic diagram of the structure of an embodiment of an IP packet;

[0025] Figure 15 This is a schematic diagram of the framework of an embodiment of the compression encoding device of this application;

[0026] Figure 16 This is a schematic diagram of the framework of an embodiment of the data encapsulation device of this application;

[0027] Figure 17 This is a schematic diagram of the framework of an embodiment of the electronic device of this application;

[0028] Figure 18 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0029] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0030] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0031] In this paper, the terms "system" and "network" are often used interchangeably. The term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this paper means two or more.

[0032] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the compression encoding method of this application.

[0033] Specifically, this may include the following steps:

[0034] Step S11: Obtain the alpha channel image based on the transparency of each pixel in the image to be encoded.

[0035] In one implementation scenario, the image to be encoded can be an image captured solely by a camera device. For example, the image to be encoded can be an image captured by a user using a camera such as a mobile phone, tablet, or camera. Alternatively, the image to be encoded can also be at least one frame from video data captured by a camera device. For example, since mobile phones, tablets, cameras, and other devices can capture video data, each frame from the video data can be used as the image to be encoded, thereby allowing for compression encoding of the alpha channel image of each frame from the video data. Of course, at least one image can also be selected from the video data as the image to be encoded; this is not limited here.

[0036] In one implementation scenario, the alpha channel image and the image to be encoded have the same resolution, and the pixel value of a pixel in the alpha channel image is the transparency of the corresponding pixel in the image to be encoded. For example, for ease of description, the resolution of both the image to be encoded and the alpha channel image can be denoted as W*H, where W represents the image width and H represents the image height. Then, the pixel value of the pixel located in the i-th row and j-th column of the alpha channel image is the transparency of the pixel located in the i-th row and j-th column of the image to be encoded. Other cases can be deduced similarly, and will not be exemplified here.

[0037] In one implementation scenario, transparency can be represented by a value within a preset range. For example, the preset range can be 0 to 1. Of course, other ranges are also possible and are not limited here. Furthermore, the value and transparency can be positively correlated; that is, the larger the value, the higher the transparency, and vice versa. For example, with a preset range of 0 to 1, a transparency of 0 can represent complete opacity, and a transparency of 1 can represent complete transparency. Conversely, the value and transparency can also be negatively correlated; that is, the smaller the value, the higher the transparency, and vice versa. For example, with a preset range of 0 to 1, a transparency of 0 can represent complete transparency, and a transparency of 1 can represent complete opacity. The above examples are merely a few possible implementation methods in practical applications and do not limit the specific methods used in actual applications.

[0038] In one implementation scenario, please refer to the following: Figure 2 , Figure 2 This is an application illustration of an embodiment of a transparency channel image. For example... Figure 2 As shown, image C consists of the foreground of image A and image B, which can be simply represented as C = A*α + B*(1-), where 0 < α < 1. Figure 2 As shown, the presence of the alpha channel in image A allows objects in image A to be separated from the background and added to the new image (i.e., image B). Just like images A, B, and C, the alpha channel α is also a spatially varying signal component. In digital video, the alpha channel also changes over time, behaving similarly to the RGB channels. Figure 2 The alpha channel can represent both shape and pixel transparency. A key characteristic of the alpha channel is that the background and foreground have sharp edges, and typically, the background and foreground are concentrated in separate areas, unlike the relatively dispersed luminance and chrominance components. The technical solution disclosed in this application effectively preserves this edge information, thereby minimizing the possibility of artifacts during backend processing and improving encoding quality. For details, please refer to the following description.

[0039] Step S12: Divide the transparency channel image into several sub-blocks.

[0040] In this embodiment, the pixel values ​​of all pixels in a sub-block are the same, and the set of several sub-blocks can cover the alpha channel image. It should be noted that the sizes of the sub-blocks can be the same or different; that is, any two sub-blocks can have the same or different sizes, and this is not limited here. Furthermore, the specific method of dividing the alpha channel image can be referred to the relevant description below, and will not be repeated here.

[0041] In one implementation scenario, each sub-block can be rectangular, with different lengths and widths. Alternatively, each sub-block can be rectangular, with the same length and width; in this case, the sub-block shape is square. Or, each sub-block can be rectangular, with some sub-blocks having the same length and width, while others have different lengths and widths. It should be noted that the above scenarios are merely possible implementations in practical applications and do not limit the possibility of using other shapes for the sub-blocks.

[0042] In one implementation scenario, the sub-blocks can be mutually exclusive. This means that any pixel in the alpha channel image exists only in one sub-block, and not simultaneously in two or more sub-blocks. This method, which divides the alpha channel image into mutually exclusive sub-blocks, helps to minimize data redundancy and improve data compression.

[0043] In one implementation scenario, for any given sub-block, there is at least one adjacent sub-block surrounding it, and there is at least one pair of adjacent corner points between the sub-block and its adjacent sub-block. For example, if the sub-block is rectangular, the vertices of the rectangle can be considered corner points of the sub-block, thus at least one corner point can be found within the sub-block that is adjacent to the corner point of its adjacent sub-block. It should be noted that for a pixel in the i-th row and j-th column of the alpha channel image, for ease of description, it can be simply denoted as pixel (i,j). Pixels at other positions can be deduced similarly. Pixels adjacent to pixel (i,j) can include: pixels (i+1,j), (i-1,j), (i,j+1), and (i,j-1). This method, where at least one adjacent sub-block surrounds a sub-block and there is at least one pair of adjacent corner points between the sub-block and its adjacent sub-block, improves the orderliness of the sub-block's positional distribution, which is beneficial for improving the efficiency of subsequent data decoding.

[0044] Step S13: Encode the alpha channel image based on the attribute information of each sub-block to obtain the encoded data.

[0045] In this embodiment of the disclosure, the attribute information includes at least one of the following: sub-block position, sub-block size, and sub-block amplitude, wherein the sub-block amplitude represents the pixel value of each pixel in the sub-block. It should be noted that, since the pixel values ​​of each pixel in the sub-block are the same, the sub-block amplitude can be regarded as the pixel value of any pixel in the sub-block.

[0046] In one implementation scenario, as mentioned earlier, the shape of the sub-block can be rectangular. In this case, the sub-block position can represent the coordinates of the rectangle's vertices, and the sub-block size can include the rectangle's length and width. This approach, which sets the sub-blocks to rectangles, with the sub-block position representing the rectangle's vertices and the sub-block size including the rectangle's length and width, helps reduce the difficulty of sub-block division and improves compression coding efficiency.

[0047] In a specific implementation scenario, the coordinates representing the sub-block positions in the attribute information of each sub-block are all the coordinates of the vertex of the rectangle at the first position. That is, when the sub-block shape is rectangular, the coordinates of the vertex at the same position of the rectangle can be used to represent it. For example, the first position can be any one of the top-left, bottom-left, top-right, and bottom-right corners. For instance, the coordinates of the top-left corner vertex of each sub-block can be chosen to represent the sub-block position in the attribute information of each sub-block. Of course, other positions can also be chosen, but not all will be listed here. In this method, the coordinates representing the sub-block positions in the attribute information of each sub-block are all the coordinates of the vertex at the first position of the rectangle, which can be any one of the top-left, bottom-left, top-right, and bottom-right corners. That is, using the vertex at the same position to represent the sub-block position can minimize coding complexity and improve compression coding efficiency.

[0048] In a specific implementation scenario, as mentioned earlier, the length and width of the rectangle can also be the same. In this case, the shape of the sub-block is a square, and correspondingly, the size of the sub-block can be the side length of the square. The above method, setting the sub-block to a square and its size to the side length of the square, further reduces the difficulty of sub-block division, which is beneficial for further improving compression coding efficiency. Furthermore, since setting the sub-block shape to a square means that the sub-block size only needs to encode the side length of the square, it can further reduce the amount of data, which is beneficial for further improving the compression level.

[0049] In one implementation scenario, please refer to the following: Figure 3 , Figure 3 This is a schematic diagram illustrating the division of an embodiment of a transparency channel image. For example... Figure 3 As shown, to maximize compression, the sub-blocks can all be squares. Therefore, the attribute information of each sub-block can be represented by a quadruple (X, Y, L, A). Here, X and Y represent the sub-block positions, such as... Figure 3As shown, for the sub-block indicated by the white line in the lower left corner, X and Y can represent the x and y coordinates of the upper left vertex of the sub-block; L represents the size of the sub-block, such as... Figure 3 As shown, for the sub-block indicated by the white line in the lower left corner, L can represent the side length of the sub-block; A can represent the amplitude of the sub-block, such as... Figure 3 As shown, for the sub-block indicated by the white line in the lower left corner, A can represent the pixel value of each pixel in that sub-block. Therefore, when the shape of the sub-block is set to a square, using a quadruple can represent the encoding of each sub-block, which helps improve the compression ratio.

[0050] In one implementation scenario, for example, each element in the attribute information of a sub-block can be represented using one byte. For instance, if the sub-block's shape is set to square, one byte can represent the horizontal coordinate X, one byte can represent the vertical coordinate Y, one byte can represent the side length L, and one byte can represent the sub-block amplitude A. That is, when the sub-block's shape is square, the encoded data size of each sub-block is four bytes. This principle applies to other sub-block shapes as well, and will not be elaborated further here. Based on this, the set of encoded bytes for all sub-blocks can be used as the encoded data for the alpha channel image.

[0051] In one implementation scenario, to illustrate the effectiveness of this application's sub-block partitioning-based compression encoding of alpha channel images in improving compression levels, please refer to [the relevant documentation / reference]. Figure 4 , Figure 5 and Figure 6 , Figure 4 This is a schematic diagram of an embodiment of sub-block partitioning with a side length of 1. Figure 5 This is a schematic diagram of an embodiment of sub-block partitioning with a side length of 2. Figure 6 This is a schematic diagram of an embodiment of sub-block partitioning with a side length of 3. For example... Figure 4 As shown, when the side length L is 1, the sub-block ( Figure 3 The white block (shown here) contains one pixel of the alpha channel image. A sub-block can be represented by a quadruple (2, 3, 1, 1). Before encoding, each element in the sub-block can be represented by one byte; after encoding, the quadruple requires four bytes. At this point, the compression ratio R = 4 / 1 = 400%. Figure 5 As shown, when the side length L is 2, the sub-block ( Figure 4 The white block (shown here) contains four pixels of the alpha channel image. A sub-block can be represented by a quadruple (2, 3, 2, 1). Before encoding, each element in the sub-block can be represented using 4 bytes; after encoding, the quadruple requires 4 bytes. At this point, the compression ratio R = 4 / 4 = 100%. Figure 6 As shown, when the side length L is 3, the sub-block ( Figure 4The white block (shown here) contains nine pixels of the alpha channel image. Each sub-block can be represented by a quadruple (2, 3, 3, 1). Before encoding, each element in the sub-block uses 9 bytes; after encoding, the quadruple requires 4 bytes. In this case, the compression ratio R = 4 / 9 ≈ 44.4%. Therefore, the relationship between the compression ratio R and L can be expressed as: R = 4 / L 2 In other words, the larger the size of the sub-block, the lower the compression ratio and the higher the compression degree. Therefore, this application uses sub-block division for compression encoding, which can make full use of the data concentration characteristics of the transparency channel image and is conducive to maximizing the compression degree of the transparency channel image.

[0052] The above scheme obtains a transparency channel image based on the transparency of each pixel in the image to be encoded. This transparency channel image is then divided into several sub-blocks, each containing pixels with the same pixel value. The set of sub-blocks covers the transparency channel image. Based on this, encoding is performed using the attribute information of each sub-block to obtain the encoded data of the transparency channel image. The attribute information includes at least one of the following: sub-block position, sub-block size, and sub-block amplitude. The sub-block amplitude represents the pixel value of each pixel within the sub-block. On the one hand, during the compression encoding process, the scheme fully utilizes the characteristic that the foreground and background of the transparency channel typically have sharp edges and are usually concentrated in a single region. Dividing the transparency channel image into several sub-blocks with the same pixel value within their respective regions achieves compression encoding, preserving edge information as much as possible and improving the quality of compression encoding. On the other hand, since compression encoding can be achieved simply by dividing the image into sub-blocks without other complex operations, it reduces computational complexity, thereby improving the efficiency and power consumption of compression encoding.

[0053] Please see Figure 7 , Figure 7 yes Figure 1 A flowchart illustrating an embodiment of step S12. Specifically, it may include the following steps:

[0054] Step S121: Use the preset position of the transparency channel image as a reference point.

[0055] In one implementation scenario, the preset position is located at the corner of the alpha channel image in the second direction. Specifically, the second direction can be any one of the top left, bottom left, top right, or bottom right corners. That is, during the initial segmentation, any corner of the alpha channel image can be selected as a reference point. For example, please refer to [reference needed]. Figure 8 , Figure 8 This is a schematic diagram illustrating the division of another embodiment of a transparency channel image. For example... Figure 8 As shown, during the initial segmentation, the top left corner point D1 of the alpha channel image can be selected as a reference point.

[0056] Step S122: In the transparent channel image, expand from the reference point until the preset conditions are no longer met to obtain a sub-block.

[0057] In this embodiment of the disclosure, the corner point of the sub-block is the reference point; that is, one of the corner points of the expanded sub-block is the reference point (i.e., the starting point). For example, as shown... Figure 8 As shown, the first expansion yields... Figure 8 The sub-block shown in the white box is marked with the number "1" in the white box for easy distinction, indicating that it was obtained during the initial expansion. The top-left corner point D1 of sub-block number 1 is the reference point.

[0058] In one implementation scenario, the specific expansion method can be set according to the predefined shape of the sub-block. For example, if the predefined shape of the sub-block is rectangular, it can expand by a certain number of pixels in both the horizontal and vertical directions each time; or, if the predefined shape of the sub-block is square, it can expand by the same number of pixels in both the horizontal and vertical directions each time. Other cases can be deduced similarly, and will not be listed here.

[0059] In one implementation scenario, as mentioned earlier, the preset position is located at the corner of the alpha channel image in the second direction. Based on this, the first pixel width and the second pixel width can be initialized. Starting from the reference point, the first pixel width is expanded in a first direction, and the second pixel width is expanded in a second direction to obtain candidate expansion regions. The first direction is horizontal and away from the starting point, and the second direction is vertical and away from the starting point. Based on this, if the current expansion meets the preset conditions, the latest candidate expansion region can be taken as the target expansion region. The first pixel width is updated based on the first pixel step size, and the second pixel width is updated based on the second pixel step size. The steps starting from the reference point and subsequent steps are repeated until the preset conditions are no longer met, and the latest target expansion region is taken as a sub-block. The above method, by taking the reference point as the starting point and expanding pixels in both the first horizontal and second vertical directions away from the starting point, and by taking the latest candidate expansion region as the target expansion region based on the current expansion meeting the preset conditions, and by updating the pixel width during expansion based on the pixel step size, can simplify the sub-block division process as much as possible, which is beneficial to improving the compression coding efficiency.

[0060] In a specific implementation scenario, taking a predefined square shape for the sub-block as an example, the first pixel width and the second pixel width can be initialized to 1. Other cases can be deduced similarly, and will not be listed here. It should be noted that, to improve the accuracy of compression encoding, the first pixel width and the second pixel width can be set to 1 by default.

[0061] In a specific implementation scenario, when the second position is the upper left corner, the first direction is horizontal to the right and the second direction is vertically downward; or, when the second position is the lower left corner, the first direction is horizontal to the right and the second direction is vertically upward; or, when the second position is the upper right corner, the first direction is horizontal to the left and the second direction is vertically downward; or, when the second position is the lower right corner, the first direction is horizontal to the left and the second direction is vertically upward. This approach, by selecting a corner point of the transparency channel image as the starting point and expanding both horizontally and vertically away from the starting point, improves expansion efficiency.

[0062] In a specific implementation scenario, taking a predefined square shape for the sub-block as an example, the first pixel step size and the second pixel step size can both be 1, meaning that the width expands by 1 pixel in both the first and second directions each time. Other cases can be deduced similarly, and will not be listed here. It should be noted that, to improve the accuracy of the compression encoding, the first pixel step size and the second pixel step size can be set to 1 by default.

[0063] In a specific implementation scenario, if the current expansion does not meet the preset conditions during the initial expansion, the reference point can be considered an isolated pixel, thus allowing for reference... Figure 4 The relevant descriptions are encoded to achieve lossless compression. Of course, if lossy compression is acceptable in practical applications, considering that a single pixel has negligible impact on subsequent decoding, it may not need to be encoded. In subsequent decoding, interpolation or other algorithms can be used to predict the pixel value. Furthermore, if the current expansion does not meet the preset conditions at the i-th (i > 1) expansion, the target expansion region obtained at the (i-1)-th expansion can be used as a sub-block.

[0064] In one implementation scenario, the preset conditions are set to include: the pixel values ​​of the extended pixels are all the same as the pixel values ​​of the starting point, the extended pixels do not belong to the sub-blocks that have been divided, and the extended pixels belong to the alpha channel image. The above method judges from three aspects: the pixel value of the extended pixels, whether they have been occupied, and whether they belong to the alpha channel image, which helps to improve the accuracy of compression coding.

[0065] In one implementation scenario, please refer to the following: Figure 9 , Figure 9 This is a schematic diagram of one embodiment of pixel expansion. For example... Figure 9As shown, the top left corner of the alpha channel image can be used as a reference point, and this reference point can be used as the starting point for the first expansion. Specifically, the first and second pixel widths can be initialized to 1 each, and then expanded by 1 pixel width horizontally to the right and vertically downwards respectively. The resulting candidate expansion area is the pixel area of ​​the top-left corner pixel. Since the current expansion meets the preset conditions, this pixel area can be used as the target expansion area for the first expansion. The first and second pixel widths are updated according to the first and second pixel step sizes of 1, respectively, resulting in an updated first and second pixel width of 2. In the second expansion, the widths can be expanded by 2 pixels horizontally to the right and vertically downwards respectively. The resulting candidate expansion area is the pixel area represented by the dotted dashed arrows in the top-left region. Since the current expansion meets the preset conditions, this pixel area can be used as the target expansion area for the second expansion. The first and second pixel widths are updated according to the first and second pixel step sizes of 1, respectively, resulting in an updated first and second pixel width of 3. In the third expansion, the widths can be expanded by 3 pixels horizontally to the right and vertically downwards respectively. The resulting candidate expansion area is the top-left region. The pixel region represented by the crossed dashed arrows can be used as the target expansion region for the third expansion because the current expansion meets the preset conditions. The first pixel width and the second pixel width are updated according to the first pixel step size 1 and the second pixel step size 1, respectively, and the updated first pixel width and the second pixel width are 4. In the fourth expansion, the pixel region can be expanded by 4 pixels in the horizontal rightward direction and vertical downward direction, respectively. The resulting candidate expansion region is the pixel region represented by the crossed solid arrows in the upper left region. Since the current expansion meets the preset conditions, the pixel region can be used as the target expansion region for the fourth expansion. The first pixel width and the second pixel width are updated according to the first pixel step size 1 and the second pixel step size 1, respectively, and the updated first pixel width and the second pixel width are 5. In the fifth expansion, the pixel region can be expanded by 5 pixels in the horizontal rightward direction and vertical downward direction, respectively. Since the current expansion does not meet the preset conditions (i.e., the pixel value of the expanded pixel is no longer the same as the pixel value of the starting point), the expansion can be ended at this time, and the latest target expansion region (i.e., the target expansion region corresponding to the fourth expansion) is used as a sub-block. Other cases can be deduced similarly, and will not be listed here one by one.

[0066] Step S123: Based on the remaining corner points of the sub-block, obtain new reference points, and re-execute the steps of expanding the alpha channel image with the reference points as the starting points, as well as subsequent steps, until all reference points have been used as starting points.

[0067] In this embodiment of the disclosure, the remaining corner points are corner points other than the starting point. Still using... Figure 8 Taking the alpha channel image shown as an example, after initially expanding to obtain sub-block number 1, new reference points can be obtained based on the remaining corner points of sub-block number 1. The remaining corner points can include the three corner points other than corner point D1, and other cases can be deduced similarly, without further examples here. It should be noted that when performing pixel expansion again, the aforementioned first pixel width and second pixel width need to be reinitialized. For example, in the previous example, during the last pixel expansion, the first pixel width and second pixel width were both 5. Therefore, when obtaining a sub-block and performing pixel expansion again to find a new sub-block, the first pixel width and second pixel width are reinitialized to 1. Other cases can be deduced similarly, without further examples here.

[0068] In one implementation scenario, after expanding to obtain sub-blocks, pixels in the alpha channel image that are adjacent to each of the remaining corner points of the sub-block but do not belong to that sub-block can be used as new reference points. This method, by using pixels in the alpha channel image that are adjacent to each of the remaining corner points but do not belong to the sub-block as new reference points, ensures that any two sub-blocks do not intersect, which is beneficial for further improving the compression level.

[0069] In one implementation scenario, after obtaining new reference points, the aforementioned pixel expansion steps can be performed based on each new reference point to obtain several sub-blocks of type 2. After obtaining several sub-blocks of type 2, new reference points can be obtained based on the remaining corner points of each sub-block of type 2, and the aforementioned pixel expansion steps can be performed based on each new reference point to obtain several sub-blocks of type 3, and so on. Examples are not provided here. As the sub-block division continues, the number of reference points decreases. When no more reference points are available, the division can be terminated; that is, the alpha channel image division can be considered complete at this point.

[0070] The above scheme uses a preset position of the alpha channel image as a reference point. The alpha channel image is expanded from the reference point until the preset conditions are no longer met, resulting in a sub-block. The corner points of the sub-block are used as reference points. Based on this, new reference points are obtained based on the remaining corner points of the sub-block, and the above steps of expanding the alpha channel image from the reference point are repeated until all reference points have been used as starting points and the remaining corner points are corner points other than the starting points. Therefore, it can achieve sub-block division in an orderly and efficient manner through pixel expansion, which is beneficial to improving the efficiency and quality of compression coding.

[0071] Please see Figure 10 , Figure 10 This is a flowchart illustrating an embodiment of the data encapsulation method of this application.

[0072] Specifically, this may include the following steps:

[0073] Step S101: Obtain the first and second encoded data of the image to be encoded.

[0074] In this embodiment of the disclosure, the first encoded data is the encoded data of the transparency channel image of the image to be encoded, and the second encoded data is the encoded data of the chroma and / or luminance of the image to be encoded.

[0075] In one implementation scenario, the first encoded data can be obtained through the steps in any of the aforementioned compression encoding method embodiments. For the specific encoding process, please refer to the aforementioned disclosed embodiments, which will not be repeated here. Of course, the first encoded data may also be obtained by encoding using other compression encoding methods, which are not limited here.

[0076] In one implementation scenario, the second encoded data can be obtained by encoding using standard encoding methods such as H.264, which is not limited here.

[0077] In one implementation scenario, as mentioned above, the image to be encoded can be an image captured by the camera device alone, or it can be at least one frame of video data captured by the camera device. For details, please refer to the relevant descriptions in the aforementioned public embodiments, which will not be repeated here.

[0078] Step S102: Based on the first data volume of the first encoded data, determine the first payload required to carry the transparent channel image using a preset packetization mechanism.

[0079] In one implementation scenario, the default packet encapsulation mechanism may include encapsulation using RTP (Real-time Transport Protocol) header extensions. For better understanding, please refer to [reference needed]. Figure 11 , Figure 11 This is a structural diagram of an embodiment of the RTP protocol packet. For example... Figure 11As shown, each RTP packet always contains the first 12 bytes (i.e., the RTP header). It's important to note that the CSRC identifier (Contributor Source Identifier) ​​is only present when the Mixer is inserted. V represents the version (2 bits), indicating the RTP version. P represents padding (1 bit); setting padding means the message end will contain one or more padding bytes, but these are not part of the payload. X represents extension (1 bit); setting extension adds a header extension after the RTP header. If the CSRC exists, this header extension can follow it. Note that X can be set to 1 if a header extension is required, otherwise it can be set to 0. Additionally, M represents the marker (1 bit), used to delineate frame boundaries in the message stream, and PT represents the payload type (7 bits), specifying the encoding standard used for the payload. `Sequencenum` represents the sequence number, occupying 16 bits. Each time an RTP packet is sent, the sequence number is incremented by 1. The receiver can use the sequence number to detect packet loss or reconstruct the sequence. `Timestamp` represents the timestamp, occupying 32 bits. The timestamp reflects the sampling time of the first byte of the RTP packet. The receiver can use the timestamp to calculate latency and jitter. `SSRCidentifier` represents the synchronization source identifier, occupying 32 bits. It is used to identify the synchronization source. This identifier is randomly selected, and two synchronization sources participating in the same video conference cannot have the same synchronization source identifier. `CSRCidentifier` represents the contributor source identifier, occupying 32 bits. The number can be 0 to 15. Each contributor source identifier identifies all contributor sources included in the payload of the RTP packet.

[0080] In one implementation scenario, the default packet encapsulation mechanism may include encapsulation using RTP (Real-time Transport Protocol) header extensions. Please refer to [reference needed]. Figure 12 and Figure 13 , Figure 12 This is a schematic diagram of a one-byte header extension implementation. Figure 13This is a schematic diagram of an embodiment of a two-byte header extension. It should be noted that the main difference between a one-byte (single-byte) header extension and a two-byte (double-byte) header extension lies in the byte length occupied by the ID and L fields. In a one-byte header extension, each field occupies one byte, while in a two-byte header extension, each field occupies two bytes. The first two bytes of the header extension indicate the type of header extension: 0xBEDE indicates a one-byte header extension, and 0x1000 indicates a two-byte header extension. The following two bytes indicate the length (i.e., the length after removing the 4-byte header). The header extension consists of multiple extension elements, each divided into a header and data parts. The header of each extension element consists of the ID and L fields. The ID field identifies different types of extension elements, and the L field indicates the byte length occupied by the extension element (i.e., the bytes occupied after removing the ID and L fields). Furthermore, as... Figure 12 or Figure 13 As shown, the header extension needs to be aligned to 4 bytes; therefore, if it does not meet the 4-byte alignment requirement, zero padding will be used for alignment. It should be noted that, to improve data transmission efficiency, a two-byte header extension can be used.

[0081] In one implementation scenario, RTP protocol packets can be encapsulated within IP data packets for transmission. Please refer to [the relevant documentation / reference]. Figure 14 , Figure 14 This is a schematic diagram of the structure of an embodiment of an IP packet. For example... Figure 14 As shown, IP packets typically do not exceed 1500 bytes. After removing the IP header, UDP (User Datagram Protocol) header, and RTP header (40 bytes in total), 1460 bytes remain. Furthermore, since the first encoded data of the transparent channel image needs to be added to the RTP header extension, an additional 4 bytes of extension header must be removed. This means that 1456 bytes of payload remain to carry the encoded data. It should be noted that when using packets from other protocols to encapsulate RTP protocol packets, the payload can be calculated similarly; examples of this will not be provided here.

[0082] In one implementation scenario, a second number of extended elements required to carry the data to be allocated can be determined based on the amount of data to be allocated and the maximum data capacity of the extended elements. Furthermore, based on the amount of data to be allocated, the second number, and the fourth data volume occupied by the element headers within the extended elements, the actual payload consumed by carrying the data to be allocated using a preset packetization mechanism can be determined. It should be noted that the amount of data to be allocated can be the first amount of data, in which case the actual payload can be the first payload. For ease of description, the first data volume can be denoted as X bytes, meaning the amount of data to be allocated is X bytes. Taking a two-byte header extension as an example, each extension element can carry a maximum of 255 bytes, meaning the maximum data volume that an extension element can carry is 255 bytes. Therefore, the second number of extension elements required to carry the amount of data to be allocated is X / 255. Considering that the fourth data volume occupied by the header element in each two-byte header extension is 2 bytes, based on the amount of data to be allocated X, the second number X / 255, and the fourth data volume occupied by the header element (i.e., 2 bytes), the required payload can be determined to be X + (X / 255) * 2. For ease of description, the above required payload X + (X / 255) * 2 can be denoted as T. Of course, considering that the header extension needs to be padded with zeros every 4 bytes... In other words, when T is divisible by 4, no padding with zeros is needed, meaning the actual payload Q is simply T. Conversely, when T is not divisible by 4, padding with zeros is required, meaning the actual payload Q can be expressed as (T / 4+1)*4, where () indicates rounding down the result. For example, when T is 21 bytes, the actual payload Q is 24 bytes, requiring 3 bytes of padding. Other cases follow the same logic and will not be listed here. The above method uses a pre-defined packetization mechanism that includes encapsulation via RTP header extensions. These RTP header extensions contain several extension elements. Based on the amount of data to be allocated and the maximum data capacity of each extension element, a second number of extension elements is determined to carry the allocated data. Then, based on the allocated data, the second number, and the fourth data volume occupied by the header elements in the extension elements, the actual payload consumed by the pre-defined packetization mechanism to carry the allocated data is determined. Since the allocated data is the first data volume and the actual payload is the first payload, the accuracy of determining the first payload is improved.

[0083] Step S103: Based on the second data volume of the first payload and the second encoded data, determine the first number of data packets required to carry the image to be encoded.

[0084] Specifically, for ease of description, the first payload can be denoted as Q, and the second data volume of the second encoded data can be denoted as Y. Based on the first payload Q and the second data volume Y, the consumed payload Q+Y of the image to be encoded can be obtained. Furthermore, based on the consumed payload Q+Y and the effective payload of the data packet (e.g., an IP data packet), the first quantity can be determined. For example, taking an IP data packet as an example, as mentioned earlier, the effective payload of an IP data packet is 1456 bytes. It can be determined whether the consumed payload Q+Y is not greater than the effective payload. If so, it is determined that only one frame of data packet is needed for encoding. Conversely, if the consumed payload Q+Y is greater than the effective payload, it is determined that multiple frames of data packet are needed for encoding. For example, the quotient of the consumed payload Q+Y divided by the effective payload can be incremented by 1. Taking an IP data packet as an example, the first quantity N can be expressed as (Q+Y) / 1456+1. Other cases can be deduced similarly, and will not be exemplified here. The above method, based on the first payload and the second quantity, obtains the consumed payload of the image to be encoded, and determines the first quantity based on the consumed payload and the effective payload of the data packet, which helps to reduce the complexity of determining the first quantity.

[0085] Step S104: Based on the first data volume and the first quantity, determine the third data volume of the first encoded data allocated to each data packet.

[0086] In one implementation scenario, the first encoded data can be randomly allocated to each data packet, and the remaining payload of each data packet can then be allocated to the second encoded data. In other words, the amount of third data allocated to each data packet can be randomly distributed.

[0087] In one implementation scenario, the ratio between the first data volume and the first quantity can be used as the third data volume, and the remaining payload of each data packet is then redistributed as second encoded data. That is, the third data volume allocated to each data packet is evenly distributed. For example, as mentioned earlier, the first data volume can be denoted as X, the first quantity as N, and the third data volume can be expressed as X / N. This method, by using the ratio between the first data volume and the first quantity as the third data volume, allows the first encoded data to be evenly distributed across the payload of each data packet. This enables the decoding of a portion of the image to be encoded for each received data packet during the backend decoding process, thus improving backend processing efficiency.

[0088] Step S105: For each data packet, determine the second payload required to carry the third data volume using the preset packetization mechanism, and based on the second payload and the effective payload of the data packet, allocate the unallocated portion of the second encoded data to the remaining payload of the data packet.

[0089] In one implementation scenario, based on the amount of data to be allocated and the maximum data capacity of the extended elements, a second number of extended elements required to carry the amount of data to be allocated can be determined. Then, based on the amount of data to be allocated, the second number, and the fourth data volume occupied by the element headers in the extended elements, the actual payload consumed by the preset packetization mechanism to carry the amount of data to be allocated can be determined. At this point, the amount of data to be allocated is the third data volume, and the actual payload is the second payload. For example, taking average allocation as an example, the third data volume is represented as X / N. Referring to the calculation process of the first payload mentioned above, the payload consumed by each RTP can be calculated as X / N + (X / N / 255)*2. Based on this, whether zero padding is needed can be determined based on whether the consumed payload X / N + (X / N / 255)*2 is divisible by 4, so as to obtain the second payload actually required for each data packet to carry the third data volume. For details, please refer to the relevant description of the calculation of the first payload mentioned above, which will not be repeated here. That is to say, the first payload and the second payload can be calculated using similar calculation steps, which can help to further reduce the complexity of data encapsulation and improve the efficiency of data encapsulation.

[0090] In one implementation scenario, after determining the second payload required to carry the third data volume for each data packet, for each data packet, based on the second payload and the packet's effective payload, the unallocated portion of the second encoded data is allocated to the remaining payload of the data packet. For example, for data packet number 1, based on the second payload and the packet's effective payload (e.g., for an IP data packet, its effective payload is 1456 bytes), the effective remaining payload of the data packet can be calculated. Then, the second encoded data is packetized and added to the remaining payload using a mechanism such as H.264NAL (Network Abstraction Layer) cell segmentation. After packetization, the sequence number of each RTP protocol packet (i.e., the aforementioned sequence number) increments by 1 sequentially, and the RTP protocol packets have identical and unique timestamps.

[0091] The above scheme obtains first and second encoded data of the image to be encoded, where the first encoded data is the encoded data of the transparency channel image of the image to be encoded, and the second encoded data is the encoded data of the chroma and / or luminance of the image to be encoded. Based on the first data volume of the first encoded data, a first payload required to carry the transparency channel image using a preset packetization mechanism is determined. Based on the first payload and the second data volume of the second encoded data, a first number of data packets required to carry the image to be encoded is determined. Based on this, a third data volume of the first encoded data allocated to each data packet is determined based on the first data volume and the first number. For each data packet, a second payload required to carry the third data volume using the preset packetization mechanism is determined. Based on the second payload and the effective payload of the data packet, the unallocated portion of the second encoded data is allocated to the remaining payload of the data packet. Therefore, the first encoded data of the transparency channel image after compression encoding can be encapsulated through a packetization mechanism, which is beneficial for adapting to network transmission and improving the convenience of backend processing.

[0092] Please see Figure 15 , Figure 15 This is a schematic diagram of a framework of an embodiment of the compression encoding apparatus 150 of this application. The compression encoding apparatus 150 includes: a channel acquisition module 151, a sub-block division module 152, and an attribute encoding module 153. The channel acquisition module 151 is used to obtain a transparent channel image based on the transparency of each pixel in the image to be encoded. The sub-block division module 152 is used to divide the transparent channel image into several sub-blocks. The pixel values ​​of each pixel in the sub-block are the same, and the set of several sub-blocks covers the transparent channel image. The attribute encoding module 153 is used to encode the encoded data of the transparent channel image based on the attribute information of each sub-block. The attribute information includes at least one of the following: sub-block position, sub-block size, and sub-block amplitude, and the sub-block amplitude represents the pixel value of each pixel in the sub-block.

[0093] The above scheme, on the one hand, can make full use of the characteristics of the transparent channel, where the foreground and background usually have sharp edges and are usually concentrated in a certain area, in the compression encoding process. By dividing the transparent channel image into several sub-blocks with the same pixel value in each area, compression encoding can be achieved, which can preserve edge information as much as possible and improve the quality of compression encoding. On the other hand, since compression encoding can be achieved simply by dividing the sub-blocks, no other complex operations are required, which helps to reduce computational complexity, thereby improving the efficiency of compression encoding and reducing the power consumption of compression encoding.

[0094] In some disclosed embodiments, the sub-block is a rectangle, and the position of the sub-block represents the coordinates of the vertices of the rectangle. The size of the sub-block includes the length and width of the rectangle.

[0095] Therefore, setting the sub-block as a rectangle, with the sub-block position representing the coordinates of the rectangle's vertices, and the sub-block size including the rectangle's length and width, helps reduce the difficulty of sub-block division and improves compression coding efficiency.

[0096] In some disclosed embodiments, the coordinates of the sub-block positions in the attribute information of each sub-block are the coordinates of the vertex of the rectangle at the first position, which is any one of the following: top left corner, bottom left corner, top right corner, or bottom right corner.

[0097] Therefore, the coordinates of the sub-block positions in the attribute information of each sub-block are the coordinates of the vertex of the rectangle at the first position. The first position can be any one of the top left, bottom left, top right, or bottom right corners. That is, using the same vertex to represent the sub-block position can reduce the encoding complexity as much as possible and improve the compression encoding efficiency.

[0098] In some disclosed embodiments, the sub-block is a square, and the size of the sub-block is the side length of the square.

[0099] Therefore, setting the sub-block to a square, and the size of the sub-block being the side length of the square, can further reduce the difficulty of sub-block division, which is conducive to further improving the compression coding efficiency. In addition, since the sub-block shape is set to a square, the sub-block size only needs to encode the side length of the square, which can further reduce the amount of data, which is conducive to further improving the compression degree.

[0100] In some publicly available embodiments, the sub-blocks do not intersect.

[0101] Therefore, dividing the image into transparent channels results in non-intersecting sub-blocks, which helps to minimize data redundancy and improve data compression.

[0102] In some disclosed embodiments, there is at least one adjacent sub-block around the sub-block, and there is at least one pair of adjacent corner points between the sub-block and its adjacent sub-block.

[0103] Therefore, the existence of at least one adjacent sub-block around a sub-block, and the existence of at least one pair of adjacent corner points between the sub-block and its adjacent sub-block, can improve the orderliness of the positional distribution of each sub-block, which is beneficial to improving the efficiency of subsequent data decoding.

[0104] In some disclosed embodiments, the sub-block division module 152 includes a start determination sub-module, used to use a preset position of the alpha channel image as a reference point; the sub-block division module 152 includes a pixel expansion sub-module, used to expand the alpha channel image with the reference point as the starting point until the preset condition is no longer met, thus obtaining a sub-block; wherein, the corner points of the sub-block are reference points; the sub-block division module 152 includes a reference acquisition sub-module, used to obtain new reference points based on the remaining corner points of the sub-block; the sub-block division module 152 includes a repetition execution sub-module, used to combine the pixel expansion sub-module and the reference acquisition sub-module to re-execute the step of expanding the alpha channel image with the reference point as the starting point and subsequent steps until all reference points have been used as starting points; wherein, the remaining corner points are corner points other than the starting points.

[0105] Therefore, by using a preset position of the alpha channel image as a reference point, the alpha channel image is expanded starting from the reference point until the preset conditions are no longer met, resulting in sub-blocks. The corner points of the sub-blocks are then used as reference points. Based on this, new reference points are obtained based on the remaining corner points of the sub-blocks, and the above steps of expanding the alpha channel image starting from the reference points are repeated until all reference points have been used as starting points, and the remaining corner points are corner points other than the starting points. Thus, sub-block division can be achieved in an orderly and efficient manner through pixel expansion, which is beneficial to improving the efficiency and quality of compression coding.

[0106] In some disclosed embodiments, the preset position is located at the corner of the transparency channel image in the second direction; the pixel expansion submodule includes an initialization unit for initializing a first pixel width and a second pixel width; the pixel expansion submodule includes a candidate expansion unit for expanding the first pixel width in a first direction and expanding the second pixel width in a second direction, starting from a reference point, to obtain a candidate expansion region; wherein, the first direction is a horizontal direction away from the starting point, and the second direction is a vertical direction away from the starting point; the pixel expansion submodule includes a target expansion unit for using the latest candidate expansion region as the target expansion region based on the current expansion meeting preset conditions; the pixel expansion submodule includes a width update unit for updating the first pixel width based on a first pixel step size and updating the second pixel width based on a second pixel step size; the pixel expansion submodule includes a loop unit for re-executing the steps starting from the reference point and subsequent steps in combination with the candidate expansion unit, the target expansion unit, and the width update unit until the preset conditions are no longer met, and using the latest target expansion region as a sub-block.

[0107] Therefore, by taking the reference point as the starting point and expanding pixels in a first direction that is horizontal and far away from the starting point and a second direction that is vertical and far away from the starting point, the latest candidate expansion area is taken as the target expansion area according to the current expansion meeting the preset conditions, and the pixel width during expansion is updated based on the pixel step size to expand again. This simplifies the sub-block division process as much as possible and helps to improve the compression coding efficiency.

[0108] In some disclosed embodiments, the second orientation is the upper left corner, the first direction is horizontal to the right, and the second direction is vertically downward; or, the second orientation is the lower left corner, the first direction is horizontal to the right, and the second direction is vertically upward; or, the second orientation is the upper right corner, the first direction is horizontal to the left, and the second direction is vertically downward; or, the second orientation is the lower right corner, the first direction is horizontal to the left, and the second direction is vertically upward.

[0109] Therefore, by selecting the corner point of the alpha channel image as the starting point and expanding it horizontally away from the starting point and vertically away from the starting point respectively, it is beneficial to improve the expansion efficiency.

[0110] In some disclosed embodiments, the reference acquisition submodule is specifically used to take pixels in the transparency channel image that are adjacent to each of the other corner points and do not belong to the sub-block as new reference points.

[0111] Therefore, by using the pixels in the alpha channel image that are adjacent to each of the other corner points and do not belong to the sub-blocks as new reference points, it is possible to ensure that any two sub-blocks do not intersect, which is beneficial to further improve the compression level.

[0112] In some disclosed embodiments, the preset conditions include: the pixel values ​​of the extended pixels are all the same as the pixel values ​​of the starting point, the extended pixels do not belong to the sub-blocks that have been divided, and the extended pixels all belong to the alpha channel image.

[0113] Therefore, judging from three aspects—the pixel value of the extended pixel, whether it has been occupied, and whether it belongs to the transparent channel image—is helpful to improve the accuracy of compression coding.

[0114] Please see Figure 16 , Figure 16This is a schematic diagram of a framework of an embodiment of the data encapsulation device 160 of this application. The data encapsulation device 160 includes: a data acquisition module 161, a payload determination module 162, a quantity determination module 163, a first allocation module 164, and a second allocation module 165. The data acquisition module 161 is used to acquire first encoded data and second encoded data of an image to be encoded; wherein the first encoded data is encoded data of the transparency channel image of the image to be encoded, and the second encoded data is encoded data of the chroma and / or luminance of the image to be encoded. The payload determination module 162 is used to determine, based on the first data volume of the first encoded data, the first payload required to carry the transparency channel image using a preset encapsulation mechanism. The quantity determination module 163 is used to determine, based on the first payload and the second data volume of the second encoded data, the first quantity of data packets required to carry the image to be encoded. The first allocation module 164 is used to determine, based on the first data volume and the first quantity, the third data volume of the first encoded data allocated to each data packet. The second allocation module 165 is used to determine, for each data packet, the second payload required to carry the third data volume using the preset encapsulation mechanism, and, based on the second payload and the effective payload of the data packet, allocate the unallocated portion of the second encoded data to the remaining payload of the data packet.

[0115] The above scheme obtains first and second encoded data of the image to be encoded, where the first encoded data is the encoded data of the transparency channel image of the image to be encoded, and the second encoded data is the encoded data of the chroma and / or luminance of the image to be encoded. Based on the first data volume of the first encoded data, a first payload required to carry the transparency channel image using a preset packetization mechanism is determined. Based on the first payload and the second data volume of the second encoded data, a first number of data packets required to carry the image to be encoded is determined. Based on this, a third data volume of the first encoded data allocated to each data packet is determined based on the first data volume and the first number. For each data packet, a second payload required to carry the third data volume using the preset packetization mechanism is determined. Based on the second payload and the effective payload of the data packet, the unallocated portion of the second encoded data is allocated to the remaining payload of the data packet. Therefore, the first encoded data of the transparency channel image after compression encoding can be encapsulated through a packetization mechanism, which is beneficial for adapting to network transmission and improving the convenience of backend processing.

[0116] In some disclosed embodiments, the first encoded data is obtained using the compression encoding apparatus in any of the above-described compression encoding apparatuses disclosed embodiments.

[0117] Therefore, using any of the above compression encoding devices to encode the first encoded data is beneficial to improving data encapsulation efficiency and the quality of the final encapsulated data packet.

[0118] In some disclosed embodiments, the preset packetization mechanism includes encapsulation by RTP header extension, and the RTP header extension includes several extension elements; the load determination module 162 or the second allocation module 165 includes an element quantity determination submodule and an actual load calculation submodule. The element quantity determination submodule is used to determine a second number of extension elements required to carry the data to be allocated based on the data to be allocated and the maximum data capacity of the extension elements; the actual load calculation submodule is used to determine the actual load consumed by carrying the data to be allocated using the preset packetization mechanism based on the data to be allocated, the second number, and the fourth data capacity occupied by the element headers in the extension elements; wherein, when the data to be allocated is a first data volume, the actual load is a first load, and when the data to be allocated is a third data volume, the actual load is a second load.

[0119] Therefore, the preset encapsulation mechanism includes encapsulation by RTP header extension, and the RTP header extension contains several extension elements. Based on this, the second number of extension elements required to carry the data to be allocated is determined based on the amount of data to be allocated and the maximum data capacity of the extension elements. Then, based on the amount of data to be allocated, the second number, and the fourth data capacity occupied by the element headers in the extension elements, the actual payload consumed by the preset encapsulation mechanism to carry the data to be allocated is determined. This can improve the accuracy of determining the first and second payloads. Since the first and second payloads are calculated using similar calculation steps, it can further reduce the complexity of data encapsulation and improve the efficiency of data encapsulation.

[0120] In some disclosed embodiments, the quantity determination module 163 includes a consumption load calculation submodule for obtaining the consumption load of the image to be encoded based on a first load and a second data volume; the quantity determination module 163 includes a first quantity determination submodule for determining a first quantity based on the consumption load and the effective load of the data packet.

[0121] Therefore, obtaining the consumed payload of the image to be encoded based on the first payload and the second quantity, and determining the first quantity based on the consumed payload and the effective payload of the data packet, helps to reduce the complexity of determining the first quantity.

[0122] In some disclosed embodiments, the first allocation module 164 is specifically used to use the ratio between the first data volume and the first quantity as the third data volume.

[0123] Therefore, by using the ratio between the first data volume and the first quantity as the third data volume, the first encoded data can be evenly distributed into the payload of each data packet. This allows for the decoding of a portion of the image to be encoded for each data packet received during the backend decoding process, which is beneficial for improving backend processing efficiency.

[0124] Please see Figure 17 , Figure 17 This is a schematic diagram of a framework of an embodiment of the electronic device 170 of this application. The electronic device 170 includes a memory 171 and a processor 172 coupled to each other. The memory 171 stores program instructions, and the processor 172 executes the program instructions to implement the steps in any of the above-described compression encoding method embodiments, or to implement the steps in any of the above-described data encapsulation method embodiments. Specifically, the electronic device 170 may include, but is not limited to, desktop computers, laptops, servers, mobile phones, tablet computers, etc., and is not limited thereto.

[0125] Specifically, processor 172 controls itself and memory 171 to implement the steps in any of the above-described compression encoding method embodiments, or to implement the steps in any of the above-described data encapsulation method embodiments. Processor 172 may also be referred to as a CPU (Central Processing Unit). Processor 172 may be an integrated circuit chip with signal processing capabilities. Processor 172 may also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor. Furthermore, processor 172 may be implemented using integrated circuit chips.

[0126] The above scheme, on the one hand, can make full use of the characteristics of the transparent channel, where the foreground and background usually have sharp edges and are usually concentrated in a certain area, in the compression encoding process. By dividing the transparent channel image into several sub-blocks with the same pixel value in each area, compression encoding can be achieved, which can preserve edge information as much as possible and improve the quality of compression encoding. On the other hand, since compression encoding can be achieved simply by dividing the sub-blocks, no other complex operations are required, which helps to reduce computational complexity, thereby improving the efficiency of compression encoding and reducing the power consumption of compression encoding.

[0127] Please see Figure 18 , Figure 8 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium 180 of this application. The computer-readable storage medium 180 stores program instructions 181 that can be executed by a processor. The program instructions 181 are used to implement the steps in any of the above-described compression encoding method embodiments, or to implement the steps in any of the above-described data encapsulation method embodiments.

[0128] The above scheme, on the one hand, can make full use of the characteristics of the transparent channel, where the foreground and background usually have sharp edges and are usually concentrated in a certain area, in the compression encoding process. By dividing the transparent channel image into several sub-blocks with the same pixel value in each area, compression encoding can be achieved, which can preserve edge information as much as possible and improve the quality of compression encoding. On the other hand, since compression encoding can be achieved simply by dividing the sub-blocks, no other complex operations are required, which helps to reduce computational complexity, thereby improving the efficiency of compression encoding and reducing the power consumption of compression encoding.

[0129] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0130] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0131] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0132] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0133] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0134] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A data encapsulation method, characterized in that, include: Obtain first encoded data and second encoded data of the image to be encoded; wherein, the first encoded data is the encoded data of the transparency channel image of the image to be encoded, and the second encoded data is the encoded data of the chroma and / or luminance of the image to be encoded; Based on the first data volume of the first encoded data, determine the first payload required to carry the transparent channel image using a preset packetization mechanism; Based on the first payload and the second data volume of the second encoded data, a first number of data packets required to carry the image to be encoded is determined; Based on the first data volume and the first quantity, determine the third data volume of the first encoded data allocated to each of the data packets; For each of the data packets, a second payload required to carry the third data volume using the preset packetization mechanism is determined, and based on the second payload and the payload of the data packet, the unallocated portion of the second encoded data is allocated to the remaining payload of the data packet.

2. The method according to claim 1, characterized in that, The acquisition of the first encoded data of the image to be encoded includes: Based on the transparency of each pixel in the image to be encoded, a transparency channel image is obtained; The alpha channel image is divided into several sub-blocks; wherein, the pixel values ​​of each pixel in the same sub-block are the same, and the set of several sub-blocks covers the alpha channel image; The first encoded data is obtained by encoding based on the attribute information of each sub-block; wherein the attribute information includes at least one of the following: sub-block position, sub-block size, sub-block amplitude, and the sub-block amplitude represents the pixel value of each pixel point in the sub-block.

3. The method according to claim 2, characterized in that, The sub-block is a rectangle, and the position of the sub-block represents the coordinates of the vertex of the rectangle. The size of the sub-block includes the length and width of the rectangle.

4. The method according to claim 2, characterized in that, The sub-blocks do not intersect.

5. The method according to claim 2, characterized in that, The process of dividing the transparency channel image into several sub-blocks includes: The preset position of the transparency channel image is used as a reference point; In the transparent channel image, the expansion is performed starting from the reference point until the preset condition is no longer met, thus obtaining the sub-block; wherein, the corner point of the sub-block is the reference point; Based on the remaining corner points of the sub-block, a new reference point is obtained, and the steps of expanding the transparency channel image with the reference point as the starting point and subsequent steps are re-executed until all the reference points have been used as the starting point. The remaining corner points are those outside the starting point.

6. The method according to claim 5, characterized in that, The preset position is located at the corner of the transparency channel image in the second direction; the sub-block is obtained by expanding from the reference point until the preset condition is no longer met, including: Initialize the first pixel width and the second pixel width; Using the reference point as the starting point, the width of the first pixel is expanded in a first direction, and the width of the second pixel is expanded in a second direction to obtain a candidate expansion region; wherein, the first direction is horizontal and away from the starting point, and the second direction is vertical and away from the starting point; Based on the fact that the current expansion meets the preset conditions, the latest candidate expansion region is taken as the target expansion region; The first pixel width is updated based on the first pixel step size, and the second pixel width is updated based on the second pixel step size. The steps starting from the reference point and subsequent steps are re-executed until the preset condition is not met, and the latest target expansion area is taken as the sub-block.

7. The method according to claim 5, characterized in that, The process of obtaining a new reference point based on the remaining corner points of the sub-block includes: The pixels in the transparent channel image that are adjacent to each of the remaining corner points and do not belong to the sub-block are used as the new reference points.

8. The method according to claim 5, characterized in that, The preset conditions include: the pixel values ​​of the extended pixels are all the same as the pixel values ​​of the starting point, the extended pixels do not belong to the already divided sub-blocks, and the extended pixels all belong to the transparency channel image.

9. The method according to claim 1, characterized in that, The preset packetization mechanism includes encapsulation by RTP header extension, and the RTP header extension contains several extension elements. The steps for determining the first load or the second load include: Based on the amount of data to be allocated and the maximum amount of data that the extended element can carry, determine a second number of extended elements required to carry the amount of data to be allocated; Based on the amount of data to be allocated, the second quantity, and the fourth data amount occupied by the element header in the extended element, the actual load consumed by using the preset packetization mechanism to carry the amount of data to be allocated is determined. Wherein, when the amount of data to be allocated is the first amount of data, the actual load is the first load; and when the amount of data to be allocated is the third amount of data, the actual load is the second load.

10. The method according to claim 1, characterized in that, The process of obtaining the first number of data packets required to carry the image to be encoded, based on the second data volume of the first payload and the second encoded data, includes: Based on the first load and the second data volume, the consumed load of the image to be encoded is obtained; The first quantity is determined based on the consumed payload and the payload of the data packet.

11. The method according to claim 1, characterized in that, The step of determining the third data volume of the first encoded data carried by each of the data packets based on the first data volume and the first quantity includes: The ratio between the first data volume and the first quantity is taken as the third data volume.

12. A data encapsulation device, characterized in that, include: The data acquisition module is used to acquire first encoded data and second encoded data of the image to be encoded; wherein, the first encoded data is the encoded data of the transparency channel image of the image to be encoded, and the second encoded data is the encoded data of the chroma and / or luminance of the image to be encoded; The load determination module is used to determine, based on the first data volume of the first encoded data, the first load required to carry the transparent channel image using a preset packetization mechanism; The quantity determination module is used to determine a first number of data packets required to carry the image to be encoded based on the first payload and the second data volume of the second encoded data; The first allocation module is used to determine the third data amount of the first encoded data allocated to each of the data packets based on the first data amount and the first quantity; The second allocation module is used to determine, for each of the data packets, the second payload required to carry the third data volume using the preset packetization mechanism, and based on the second payload and the payload of the data packet, allocate the unallocated portion of the second encoded data to the remaining payload of the data packet.

13. An electronic device, characterized in that, The method includes a memory and a processor coupled to each other, the memory storing program instructions, and the processor executing the program instructions to implement the data encapsulation method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that, The device stores program instructions that can be executed by a processor, the program instructions being used to implement the data encapsulation method according to any one of claims 1 to 11.

Images