A video encoding method, apparatus, electronic device and storage medium

Abstract

This application provides a video encoding method, apparatus, electronic device, and storage medium. The method includes: segmenting multiple pixels in each frame of video image data to be encoded to obtain multiple image blocks corresponding to the frame; encoding each pixel in each image block using multiple encoding methods to generate multiple candidate image block bitstreams for the image block under different encoding methods; determining the bitstream length of each candidate image block bitstream corresponding to the image block, and selecting the candidate image block bitstream with the smallest bitstream length as the target image block bitstream for the image block; and packaging the target image block bitstream corresponding to the image block to generate an image block data packet corresponding to the image block. By adopting the above-mentioned video encoding method, apparatus, electronic device, and storage medium, the problem of not being able to simultaneously balance hardware implementation difficulty and compression rate in KVM application scenarios is solved.

Description

Technical Field

[0001] This application relates to the field of video transmission technology, and more specifically, to a video encoding method, apparatus, electronic device, and storage medium. Background Technology

[0002] Video coding technology is a technique that uses compression to convert raw video data into another video format. The purpose of video coding is to reduce transmission bandwidth or decrease the hardware space required to store video files. Video coding can be categorized into intra-frame coding and inter-frame coding based on coding type, and into shallow compression and deep compression based on compression ratio. Shallow compression is easier to implement in hardware, but its compression ratio is also relatively low, ranging from 1 to 30 times, with 4 to 10 times being commonly used, such as JPEG, JPEG-XS, and VC-2 coding standards. Deep compression is more difficult to implement in hardware, but its compression ratio is much higher, reaching several thousand times, such as H.264 and H.265.

[0003] However, in KVM (Keyboard Video Mouse) application scenarios, the video screen does not switch frequently. In most scenarios, only the mouse and a few areas of the screen change. If the above video encoding method is still used for encoding, it will lead to the inability to simultaneously take into account the hardware implementation difficulty and compression rate in KVM application scenarios. It will also cause the problem that the generated encoded result is too long to transmit. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a video encoding method, apparatus, electronic device and storage medium to solve the problems of not being able to simultaneously consider hardware implementation difficulty and compression rate in KVM application scenarios, as well as the problem of the long generated encoding results being difficult to transmit.

[0005] In a first aspect, embodiments of this application provide a video encoding method, including:

[0006] Segment the multiple pixels in each frame of the video image in the video data to be encoded to obtain multiple image blocks corresponding to that frame of video image;

[0007] For each image block, each pixel in the image block is encoded using multiple encoding methods to generate multiple candidate image block bitstreams under different encoding methods;

[0008] Determine the bitstream length of each candidate image block bitstream corresponding to the image block, and select the candidate image block bitstream with the smallest bitstream length as the target image block bitstream of the image block;

[0009] Pack the target image block bitstream corresponding to the image block to generate the image block data packet corresponding to the image block.

[0010] Optionally, multiple encoding methods include direct transmission, direct encoding, and differential encoding. Each pixel in the image block is encoded using these methods to generate multiple candidate image block bitstreams under different encoding methods. This includes: for the direct transmission method, obtaining the color value of each pixel in the image block and concatenating the color values ​​sequentially to generate the first candidate image block bitstream under the direct transmission method, where each color channel corresponds to one component; for the direct encoding method, performing an XYZ color space transformation on each pixel in the image block to generate XYZ space data corresponding to each pixel, where the XYZ space data includes X component data, Y component data, and Z component data; for each component, subtracting the component data between adjacent pixels in the image block under that component to generate entropy-determining encoded data for that component. The entropy-coding data under different components is used to generate the second candidate image block bitstream under the direct encoding method for the image block. For the differential encoding method, the XYZ spatial data of the target image block in the target encoded frame video image is obtained. The target encoded frame video image is the frame video data that is closest to the frame video data corresponding to the image block and the target image block is the image block in the target encoded frame video data that corresponds to the image block. The XYZ spatial data of each pixel in the image block is subtracted from the XYZ spatial data of the corresponding pixel in the target image block to obtain the component difference data of each pixel in the image block. For each component, the component difference data between adjacent pixels in the image block is subtracted to generate the entropy-coding data under that component. The third candidate image block bitstream under the differential encoding method for the image block is generated based on the entropy-coding data under different components.

[0011] Optionally, the process of segmenting multiple pixels in each frame of video image in the video data to be encoded to obtain multiple image blocks corresponding to the frame includes: determining the image block length of the image block; connecting the pixels in different rows of the frame of video image line by line to form a pixel sequence; and selecting a number of pixels corresponding to the image block length from the pixel sequence in a preset order to form an image block, thereby forming multiple image blocks from all the pixels selected in the frame of video image.

[0012] Optionally, the method further includes: determining a first interval number and a second interval number, wherein the first interval number is used to determine image blocks within a single frame of video data where differential encoding is prohibited, and the second interval number is used to determine image blocks between different frames of video data where differential encoding is prohibited; for each frame of video data in the video data to be encoded, selecting a first prohibited image block and a second prohibited image block according to the first interval number and the second interval number respectively; determining whether the image block is the first prohibited image block or the second prohibited image block; if it is the first prohibited image block or the second prohibited image block, prohibiting the use of differential encoding for encoding the image block.

[0013] Optionally, after generating entropy-encoded data for each component by subtracting the component difference data between adjacent pixels in the image block under that component, the method further includes: if the entropy-encoded data for each pixel in the image block under different components is all 0, then the image block does not need to be encoded.

[0014] Optionally, the target image block bitstream corresponding to the image block is packaged to generate an image block data packet corresponding to the image block, including: for each component, dividing the target image block bitstream under that component into multiple bitstream blocks of a set length; according to a preset component order, sequentially combining the bitstream length under each component and the multiple bitstream blocks corresponding to different components to generate a bitstream data block for the image block; and combining the bitstream data block with the bitstream information to generate an image block data packet for the image block.

[0015] Optionally, according to a preset component order, the bitstream length of each component and multiple bitstream blocks corresponding to different components are sequentially combined to generate the bitstream data block of the image block, including: sequentially combining the bitstream lengths of different components according to a preset component order to generate a first bitstream data block; sequentially combining the bitstream blocks at corresponding positions under different components according to a preset component order to generate a second bitstream data sub-block; combining multiple second bitstream data sub-blocks together to generate a second bitstream data block; and the bitstream data block is composed of the first bitstream data block and the second bitstream data block.

[0016] Secondly, embodiments of this application also provide a video encoding apparatus, the apparatus comprising:

[0017] The image segmentation module is used to segment multiple pixels in each frame of video image in the video data to be encoded, and obtain multiple image blocks corresponding to that frame of video image;

[0018] The image block encoding module is used to encode each pixel in each image block using multiple encoding methods, generating multiple candidate image block bitstreams for the image block under different encoding methods;

[0019] The bitstream selection module is used to determine the bitstream length of each candidate image block bitstream corresponding to the image block, and select the candidate image block bitstream with the smallest bitstream length as the target image block bitstream of the image block;

[0020] The data packet generation module is used to package the target image block bitstream corresponding to the image block and generate the image block data packet corresponding to the image block.

[0021] Thirdly, embodiments of this application also provide an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the video encoding method described above are performed.

[0022] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the video encoding method described above.

[0023] The embodiments of this application bring the following beneficial effects:

[0024] This application provides a video encoding method, apparatus, electronic device, and storage medium that can segment each frame of video image into multiple image blocks and encode each image block using various encoding methods. Simultaneously, it selects the candidate image block bitstream with the shortest bitstream length from multiple candidate image block bitstreams as the target image block bitstream. The target image block bitstream is then used to obtain the corresponding image block data packet. This achieves the technical effect of selecting the most suitable encoding method based on the actual situation of each frame of video image, shortening the length of the encoded result and achieving the goal of fast video encoding transmission. Compared with existing video encoding methods, it solves the problem of simultaneously balancing hardware implementation difficulty and compression rate, and also solves the problem of long encoded results being difficult to transmit. Furthermore, since the entire encoding process processes individual image blocks, it also solves the problem of high hardware implementation difficulty due to excessive hardware resource consumption.

[0025] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A flowchart of the video encoding method provided in an embodiment of this application is shown;

[0028] Figure 2 A schematic diagram of a bitstream data block provided in an embodiment of this application is shown;

[0029] Figure 3 A schematic diagram of the structure of the video encoding apparatus provided in an embodiment of this application is shown;

[0030] Figure 4 A schematic diagram of the structure of the electronic device provided in the embodiments of this application is shown. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0032] It is worth noting that prior to this application, video encoding technology was a technique that used compression to convert data in one video format into another. The purpose of video encoding is to reduce transmission bandwidth or decrease the hardware space required to store video files. Video encoding can be categorized into intra-frame encoding and inter-frame encoding based on encoding type, and into shallow compression and deep compression based on compression ratio. Shallow compression has low hardware implementation complexity but also low compression ratios, ranging from 1 to 30 times, with typical applications using 4 to 10 times, such as JPEG, JPEG-XS, and VC-2 encoding standards. Deep compression has high hardware implementation complexity but high compression ratios, reaching several thousand times, such as H.264 and H.265. However, in KVM (Keyboard Video Mouse) applications, the video feed does not switch frequently; in most scenarios, only the mouse cursor and a few other areas of the screen change. If the aforementioned video encoding methods are still used, it will be impossible to simultaneously address both hardware implementation complexity and compression ratio issues in KVM applications, and it will also cause problems with the resulting encoded results being too long to transmit easily.

[0033] Based on this, embodiments of this application provide a video encoding method to shorten the length of the encoding result and improve the compression rate of video encoding while ensuring that it is easy to implement in hardware.

[0034] Please see Figure 1 , Figure 1 This is a flowchart illustrating a video encoding method provided in an embodiment of this application. Figure 1 As shown, the video encoding method provided in this application includes:

[0035] Step S101: Segment multiple pixels in each frame of video image in the video data to be encoded to obtain multiple image blocks corresponding to that frame of video image.

[0036] In this step, the video data to be encoded can refer to the video image data to be encoded.

[0037] The video data to be encoded includes multiple temporally consecutive video frames, each of which contains multiple pixels that can form a two-dimensional dot matrix.

[0038] An image block can refer to a region block in a frame video image, and a frame video image is composed of multiple region blocks.

[0039] In this embodiment, the video encoding method is applied to a video encoder. The video data to be encoded is input into the video encoder, which then encodes the video data. First, each frame of the video data to be encoded is segmented into multiple image blocks. Each image block is a basic encoding unit, and a single image block is called a slice. A single slice is a set of pixels of size 1×N, containing N pixels. Typically, N is 256.

[0040] In one optional embodiment, the process of segmenting multiple pixels in each frame of video image in the video data to be encoded to obtain multiple image blocks corresponding to the frame of video image includes: determining the image block length of the image block; connecting the pixels in different rows of the frame of video image line by line to form a pixel sequence; and selecting a number of pixels corresponding to the image block length from the pixel sequence in a preset order to form an image block, and using all the pixels selected in the frame of video image to form multiple image blocks.

[0041] Specifically, first, the image block length is set. Taking the example above, the image block length is N = 256. After determining the image block length, the single-frame video image can be segmented according to this length. Assuming the single-frame video image is 1920×1080, each row in the single-frame video image has 1920 pixels. The 1920 pixels of the first row are connected to the 1920 pixels of each subsequent row in ascending order of row number to form the pixel sequence corresponding to the single-frame video image. The pixel sequence is then segmented into multiple image blocks in a forward-to-back order, with each consecutive 256 pixels forming an image block. In this way, the first row of the single-frame video image is divided into 7 image blocks and 128 pixels. These 128 pixels will be combined with the first 128 pixels of the second row to form the second image block, and so on, to obtain multiple image blocks. In addition, if there are fewer than 256 remaining pixels in a single frame of video image, the remaining pixels are treated as an image block, and each image block has an image block identifier, which is used to determine the specific position of the current image block in the single frame of video image.

[0042] Step S102: For each image block, each pixel in the image block is encoded using multiple encoding methods to generate multiple candidate image block bitstreams under different encoding methods.

[0043] In this step, "multiple encoding methods" can refer to different video encoding methods.

[0044] The number of candidate image blocks is equal to the number of encoding methods. If there are three encoding methods, then the number of candidate image block bitstreams is three.

[0045] In this embodiment of the application, it is assumed that a single frame video image is divided into three image blocks A, B, and C, and the multiple encoding methods are encoding method a, encoding method b, and encoding method c. Then, three candidate image block bitstreams are generated for each image block, and a total of nine candidate image block bitstreams are generated for the three image blocks corresponding to the single frame video image.

[0046] In one optional embodiment, multiple encoding methods include direct transmission, direct encoding, and differential encoding. Each pixel in the image block is encoded using these methods to generate multiple candidate image block bitstreams under different encoding methods. This includes: for the direct transmission method, obtaining the color value of each pixel in the image block and concatenating the color values ​​of each pixel sequentially to generate the first candidate image block bitstream under the direct transmission method, where each color channel corresponds to one component; for the direct encoding method, performing an XYZ color space transformation on each pixel in the image block to generate XYZ space data corresponding to each pixel, where the XYZ space data includes X component data, Y component data, and Z component data; and for each component, subtracting the component data between adjacent pixels within the image block under that component to generate entropy-determining encoded data for that component. Based on the entropy-to-entropy encoded data under different components, a second candidate image block bitstream under the direct encoding method of the image block is generated. For the differential encoding method, the XYZ spatial data of the target image block in the target encoded frame video image is obtained. The target encoded frame video image is the frame video data that is closest to the frame video data corresponding to the image block and the target image block is the image block in the target encoded frame video data that corresponds to the image block. The XYZ spatial data of each pixel in the image block is subtracted from the XYZ spatial data of the corresponding pixel in the target image block to obtain the component difference data of each pixel in the image block. For each component, the component difference data between adjacent pixels in the image block under that component is subtracted to generate the entropy-to-entropy encoded data under that component. Based on the entropy-to-entropy encoded data under different components, a third candidate image block bitstream under the differential encoding method of the image block is generated.

[0047] Here, direct transmission can refer to an encoding method that directly uses the color value of a pixel as the bitstream.

[0048] Direct encoding is an intra-frame encoding method, which refers to the method of encoding the component values ​​of each pixel in a single image block within a frame.

[0049] Differential coding is an inter-frame coding method. Differential coding can be used to encode the component differences of each pixel in two different image blocks at the same position in an inter-frame frame.

[0050] The candidate image block bitstream can be the first candidate image block bitstream, the second candidate image block bitstream, or the third candidate image block bitstream.

[0051] The first candidate image block bitstream can refer to the image block bitstream corresponding to the direct transmission method.

[0052] The second candidate image block bitstream can refer to the image block bitstream corresponding to the direct encoding method.

[0053] The third candidate image block bitstream can refer to the image block bitstream corresponding to the differential encoding method.

[0054] The target encoded frame video image can refer to the single frame of video data that has been encoded and is closest to the current frame video image containing the current image block. "Closest to" can refer to the closest time or frame number.

[0055] Taking the case of frame loss as an example, when frames 2 and 4 are lost, the decoding end only receives the bitstream of frames 0, 1, 3, 5, and 6. Therefore, for frame 5, the closest frame is frame 3.

[0056] The target image block can refer to the image block in the target encoded frame video image corresponding to the current image block position.

[0057] Suppose that the current image block has the index 3 in the 5th frame of the video image, then the image block with the index 3 in the 3rd frame of the video image that is closest to the 5th frame of the video image is the target image block.

[0058] Specifically, for the direct transmission method, the RGB values ​​of each pixel in the image block are determined, and the RGB values ​​of each pixel are concatenated sequentially according to the order of the pixels to generate the first candidate image block bitstream corresponding to that image block. Each color channel corresponds to one component: the R color channel corresponds to the first component, the G color channel corresponds to the second component, and the B color channel corresponds to the third component.

[0059] For direct encoding, before encoding, the RGB values ​​of all pixels need to be converted to the XYZ color space to obtain the XYZ space data of each pixel. The XYZ space data includes X component data, Y component data, and Z component data. The X component is the first component, the Y component is the second component, and the Z component is the third component. For ease of description, the XYZ space data of an image patch is denoted as: [(X1,Y1,Z1),(X2,Y2,Z2),(X3,Y3,Z3),...,(X... N ,Y N Z NDuring encoding, the corresponding components between adjacent pixels within an image block are subtracted to obtain the entropy-encoded data under the direct encoding method. That is, the first component of the entropy-encoded data is obtained by subtracting the X components, which is [X1, X2-X1, X3-X2, ..., X]. N -X N-1 The second component, obtained by subtracting the Y components, is entropy-encoded data as [Y1, Y2-Y1, Y3-Y2, ..., Y]. N -Y N -1], subtracting the Z components yields the third component's entropy-decoding data as [Z1,Z2-Z1,Z3-Z2,…,Z N -Z N-1 The advantage of this approach is that if the component data of adjacent pixels are the same, the entropy-coding data of the corresponding component is 0, which reduces the number of bitstreams. Then, Golomb coding is performed on the entropy-coding data of different components to generate the second image block bitstream.

[0060] For the difference encoding method, before encoding, it is also necessary to convert the RGB values ​​of all pixels to the XYZ color space to obtain the XYZ space data of each pixel in the XYZ color space. The XYZ space data includes X component data, Y component data, and Z component data. Among them, the X component is the first component, the Y component is the second component, and the Z component is the third component. Then, the difference between the XYZ space data of the image patch and the XYZ space data of the target image patch is calculated to obtain the component difference data as [(dX1,dY1,dZ1),(dX2,dY2,dZ2),(dX3,dY3,dZ2),...,(dX... N ,dY N ,dZ N Then, during encoding, the component differences between corresponding pixels within an image block are subtracted to obtain the entropy-to-entropy encoded data under the difference encoding method. The entropy-to-entropy encoded data includes the first entropy-to-entropy encoded data corresponding to the X component, the second entropy-to-entropy encoded data corresponding to the Y component, and the third entropy-to-entropy encoded data corresponding to the Z component. The first entropy-to-entropy encoded data is [dX1, dX2-dX1, dX3-dX2, ..., dX]. N -dX N-1 The second component, the data to be encoded by entropy, is [dY1, dY2-dY1, dY3-dY2, ..., dY]. N -dY N-1 The third component, entropy-encoded data, is [dZ1, dZ2-dZ1, dZ3-dZ2, ..., dZ]. N -dZ N-1Finally, Columbus coding is performed on the entropy-coding data of different components to generate the third image block bitstream. It should be noted that differential coding cannot be performed on the first frame of the video data to be encoded, because no other frames have been differentially processed with the first frame.

[0061] In an optional embodiment, after generating entropy-encoded data for each component by subtracting the component difference data between adjacent pixels in the image block under that component, the method further includes: if the entropy-encoded data for each pixel in the image block under different components is all 0, then the image block does not need to be encoded.

[0062] Specifically, for the differential coding method, if all the entropy-to-entropy encoded data under different components in the image block are 0, it indicates that the current frame video image and the previous frame video image have a lot of the same content in the current image block position entropy. In order to reduce the number of bitstreams, there is no need to repeatedly encode the image block. Therefore, when all the entropy-to-entropy encoded data are 0, the image block is not encoded.

[0063] In an optional embodiment, the method further includes: determining a first interval number and a second interval number, wherein the first interval number is used to determine image blocks within a single frame of video data where differential encoding is prohibited, and the second interval number is used to determine image blocks between different frames of video data where differential encoding is prohibited; for each frame of video data in the video data to be encoded, selecting a first prohibited image block and a second prohibited image block according to the first interval number and the second interval number respectively; determining whether the image block is the first prohibited image block or the second prohibited image block; if it is the first prohibited image block or the second prohibited image block, prohibiting the use of differential encoding for encoding the image block.

[0064] Specifically, since differential encoding may result in situations where encoding is unnecessary, it is necessary to prohibit the use of differential encoding in specific image blocks to ensure image accuracy. This is to guarantee that the image can be recovered correctly within a few frames after network packet loss, thus requiring data packet loss recovery processing.

[0065] Here, we first set the values ​​for the first and second interval counts. For example, the first interval count is 59 and the second interval count is 56. This way, even if one frame of video is lost, it can be recovered correctly after 60 frames. When encoding the video data to be encoded, we first determine which image blocks in each frame of video are prohibited from using differential encoding based on the first and second interval counts, and must be encoded using direct transmission or direct encoding.

[0066] Taking the above example, the first interval number is 59, the second interval number is 56, and assuming each frame of video image is divided into 100 image blocks, then the first frame of video image cannot use differential encoding. For the second frame of video image, since the first image block cannot use differential encoding, after the first interval number, the 61st image block also cannot use differential encoding. Therefore, the first and 61st image blocks in the first frame of video image cannot use differential encoding. For the third frame of video image, after the second interval number of image blocks, 61 + 56 + 1 - 100 = 18, so the 18th image block cannot use differential encoding. After the first interval number in the third frame of video image, 18 + 59 + 1 = 78, so the 78th image block cannot use differential encoding. That is, the 18th and 78th image blocks in the third frame of video image cannot use differential encoding. Similarly, for the fourth frame of the video image, 78+56+1-100=35, 35+59+1=95, so the 35th and 95th image blocks in the fourth frame of the video image cannot use differential encoding. However, it is not mandatory that the first image block cannot use differential encoding; those skilled in the art can choose whether or not to use differential encoding for the first image block based on the actual situation.

[0067] Step S103: Determine the bitstream length of each candidate image block bitstream corresponding to the image block, and select the candidate image block bitstream with the smallest bitstream length as the target image block bitstream of the image block.

[0068] In this step, after each image block is encoded, the bitstream length of that image block under the corresponding encoding method needs to be recorded.

[0069] In this embodiment, taking image block A as an example, the bitstream length of the first candidate image block bitstream a obtained by direct transmission is L1, the bitstream length of the second candidate image block bitstream b obtained by direct encoding is L2, and the bitstream length of the third candidate image block bitstream c obtained by differential encoding is L3. The result of min(L1, L2, L3) is then calculated. Assuming the minimum bitstream length is L3, the third candidate image block bitstream c obtained by differential encoding is taken as the target image block bitstream.

[0070] The target image block bitstream includes a first component bitstream, a second component bitstream, and a third component bitstream. The first component bitstream is the bitstream corresponding to the X component or the R component, the second component bitstream is the bitstream corresponding to the Y component or the G component, and the third component bitstream is the bitstream corresponding to the Z component or the B component.

[0071] After obtaining the bitstreams of different components, the individual component bitstreams are concatenated to obtain three bitstream segments, which constitute the target image block bitstream. If the number of bytes in the bitstream is not a multiple of 8, it is padded with 0s in the high-order bits.

[0072] Taking the four data points (pixels) after the first component is encoded as an example, we can introduce the concatenation process of a single component. These four data points are code0 = 01101, code1 = 001010101, code2 = 0, and code3 = 00101. After concatenating the four data points code3, code2, code1, and code0, we get: 0010 1000 1010 1010 1101. After padding with 0s, it becomes: 0000001010001010 10101101. The final output bitstream is: [10101101,10001010,00000010], which is [173,138,2].

[0073] Step S104: Pack the target image block bitstream corresponding to the image block to generate the image block data packet corresponding to the image block.

[0074] In this step, one image block data packet corresponds to one image block, that is, one image block data packet transmits an image block of 1×N pixels.

[0075] The image block data packet includes, but is not limited to: image block header marker, bitstream length, encoding method, image block identifier, start and end decoding markers, and bitstream data block.

[0076] Image block header markers are used to determine the starting position of image blocks during decoding.

[0077] The bitstream length is used to record the bitstream size of each component, and together with the bitstream data block, it is used to determine the specific data of each component bitstream in the current image block.

[0078] The encoding method is used to determine which corresponding decoding method to use to process the entropy-decoded data.

[0079] Image block identifiers are used to determine the specific location of the current image block within a single frame of video image.

[0080] Start and end decoding markers are used to determine whether decoding of a single frame of video has begun and whether decoding of a single frame of video has been completed.

[0081] In one optional embodiment, the target image block bitstream corresponding to the image block is packaged to generate an image block data packet corresponding to the image block, including: for each component, dividing the target image block bitstream under that component into multiple bitstream blocks of a set length; according to a preset component order, combining the bitstream length under each component and the multiple bitstream blocks corresponding to different components in sequence to generate a bitstream data block of the image block; and combining the bitstream data block with the bitstream information to generate an image block data packet of the image block.

[0082] Specifically, after determining the three segments of the target image block bitstream, the sizes of the first component bitstream, the second component bitstream, and the third component bitstream can be determined respectively. The first component bitstream is divided into size_x 2-byte blocks, the second component bitstream is divided into size_y 2-byte blocks, and the third component bitstream is divided into size_z 2-byte blocks. The specific content of the first bitstream block is denoted as code_x[(size_nx-1)×2:size_nx×2-1], the specific content of the second bitstream block is denoted as code_y[(size_ny-1)×2:size_ny×2-1], and the specific content of the third bitstream block is denoted as code_z[(size_nz-1)×2:size_nz×2-1]. The values ​​of size_nx, size_ny, and size_nz are [1, size_x], [1, size_y], and [1, size_z], respectively. Their values ​​change with the number of bitstream blocks. For example, size_nx, size_ny, and size_nz are all 1 in the first bitstream block, and size_nx, size_ny, and size_nz are all 2 in the second bitstream block.

[0083] According to the component order, size_x, size_y, size_z and each bitstream block are first combined together to obtain a bitstream data block. Then, the bitstream data block is combined with the image block packet header marker, bitstream length, encoding method, image block identifier, start and end decoding markers to obtain an image block data packet.

[0084] In one optional embodiment, the bitstream length of each component and multiple bitstream blocks corresponding to different components are sequentially combined to generate the bitstream data block of the image block according to a preset component order, including: sequentially combining the bitstream lengths of different components to generate a first bitstream data block according to a preset component order; sequentially combining the bitstream blocks at corresponding positions under different components to generate a second bitstream data sub-block according to a preset component order; combining multiple second bitstream data sub-blocks to generate a second bitstream data block; and the bitstream data block is composed of the first bitstream data block and the second bitstream data block.

[0085] Specifically, following the order of the first component, the second component, and the third component, size_x, size_y, and size_z are first combined to obtain the first bitstream data block. Then, similarly following the order of the first component, the corresponding bitstream blocks under different components are combined to generate the second bitstream data sub-block. For example, code_x[0:1], code_y[0:1], and code_z[0:1] are combined to obtain the first second bitstream data sub-block, and code_x[2:3], code_y[2:3], and code_z[2:3] are combined to obtain the second second bitstream data sub-block. Multiple second bitstream data sub-blocks are combined to generate the second bitstream data block, which is composed of the first bitstream data block and the second bitstream data block.

[0086] The following reference Figure 2 Let's introduce the structure of the bitstream data blocks.

[0087] Figure 2 A schematic diagram of a bitstream data block provided in an embodiment of this application is shown.

[0088] like Figure 2 As shown, the bitstream data block is divided into two parts: a first bitstream data block 210 and a second bitstream data block 220. The first bitstream data block 210 is used to store the bitstream length, and the second bitstream data block is used to store the specific bitstream data.

[0089] Compared with existing video coding methods, this application can segment each video frame into multiple image blocks and encode each image block separately using various coding methods. Simultaneously, it selects the candidate image block bitstream with the shortest bitstream length from multiple candidate image block bitstreams as the target image block bitstream. The target image block bitstream is then used to obtain the corresponding image block data packet. This achieves the technical effect of selecting the most suitable coding method based on the actual situation of each video frame, shortening the length of the encoded result and achieving the goal of fast video encoding transmission. It solves the problem of simultaneously balancing hardware implementation difficulty and compression rate, and also addresses the issue of long encoded results being difficult to transmit. Furthermore, since the entire encoding process processes individual image blocks, it also solves the problem of high hardware implementation difficulty due to excessive hardware resource consumption.

[0090] Based on the same inventive concept, this application also provides a video encoding device corresponding to the video encoding method. Since the principle of the device in this application is similar to that of the video encoding method described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0091] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of a video encoding device provided in an embodiment of this application. Figure 3 As shown, the video encoding device 300 includes:

[0092] The image segmentation module 301 is used to segment multiple pixels in each frame of video image in the video data to be encoded, and obtain multiple image blocks corresponding to the frame of video image.

[0093] The image block encoding module 302 is used to encode each pixel in each image block using multiple encoding methods to generate multiple candidate image block bitstreams for the image block under different encoding methods.

[0094] The bitstream selection module 303 is used to determine the bitstream length of each candidate image block bitstream corresponding to the image block, and select the candidate image block bitstream with the smallest bitstream length as the target image block bitstream of the image block;

[0095] The data packet generation module 304 is used to package the target image block bitstream corresponding to the image block to generate the image block data packet corresponding to the image block.

[0096] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 As shown, the electronic device 400 includes a processor 410, a memory 420, and a bus 430.

[0097] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, they can perform the operations described above. Figure 1 The steps of the video encoding method in the illustrated method embodiment can be found in the method embodiment for specific implementation, and will not be repeated here.

[0098] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 The steps of the video encoding method in the illustrated method embodiment can be found in the method embodiment for specific implementation, and will not be repeated here.

[0099] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0100] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0101] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0102] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0103] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0104] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A video encoding method, characterized in that, include: Segment the multiple pixels in each frame of the video image in the video data to be encoded to obtain multiple image blocks corresponding to that frame of video image; For each image block, each pixel in the image block is encoded using multiple encoding methods to generate multiple candidate image block bitstreams under different encoding methods; Determine the bitstream length of each candidate image block bitstream corresponding to the image block, and select the candidate image block bitstream with the smallest bitstream length as the target image block bitstream of the image block; Pack the target image block bitstream corresponding to the image block to generate the image block data packet corresponding to the image block; The various encoding methods include direct transmission, direct encoding, and differential encoding. The process of encoding each pixel in the image block using multiple encoding methods to generate multiple candidate image block bitstreams under different encoding methods includes: For the direct transmission method, the color value of each pixel in the image block is obtained, and the color values ​​of each pixel are directly concatenated to generate the first candidate image block bitstream under the direct transmission method. Each color channel corresponds to a component. For the direct encoding method, each pixel in the image block is transformed by XYZ color space to generate XYZ space data corresponding to each pixel. The XYZ space data includes X component data, Y component data and Z component data. For each component, the component data between adjacent pixels in the image block are subtracted under that component to generate entropy-encoded data under that component. Based on the entropy-encoded data under different components, a second candidate image block bitstream under the direct encoding method of the image block is generated. For the differential coding method, the XYZ spatial data of the target image block in the target encoded frame video image is obtained. The target encoded frame video image is the frame data that has been encoded and is closest to the frame video data corresponding to the image block. The target image block is the image block in the target encoded frame video data that corresponds to the image block. Subtract the XYZ spatial data of each pixel in the image block from the XYZ spatial data of the corresponding pixel in the target image block to obtain the component difference data of each pixel in the image block; For each component, the component difference data between adjacent pixels in the image block is subtracted to generate the entropy-to-entropy encoded data for that component. Based on the entropy-to-entropy encoded data for different components, the third candidate image block bitstream under the difference encoding method is generated for that image block.

2. The method according to claim 1, characterized in that, The step of segmenting multiple pixels in each frame of video image in the video data to be encoded to obtain multiple image blocks corresponding to that frame of video image includes: Determine the length of the image block; The pixels in different rows of the video frame are connected end to end to form a pixel sequence; According to a preset order, a number of pixels corresponding to the length of the image block are selected sequentially from the pixel sequence to form an image block, and multiple image blocks are formed by all the pixels selected in the video frame.

3. The method according to claim 1, characterized in that, The method further includes: A first interval number and a second interval number are determined. The first interval number is used to determine the image blocks in a single frame of video data that are prohibited from using differential encoding. The second interval number is used to determine the image blocks in different frames of video data that are prohibited from using differential encoding. For each frame of video data in the video data to be encoded, a first disabled image block and a second disabled image block are selected according to the first interval number and the second interval number, respectively. Determine whether the image block is the first disabled image block or the second disabled image block; If it is the first or second disabled image block, the differential encoding method is prohibited from being used to encode the image block.

4. The method according to claim 1, characterized in that, After subtracting the component difference data between adjacent pixels within the image block under each component to generate the entropy-encoded data for that component, the method further includes: If the entropy encoding data of each pixel in the image block is all 0 under different weights, then the image block does not need to be encoded.

5. The method according to claim 1, characterized in that, The step of packaging the target image block bitstream corresponding to the image block to generate the image block data packet corresponding to the image block includes: For each component, the target image block bitstream under that component is divided into multiple bitstream blocks of a set length; According to the preset component order, the bitstream length of each component and the multiple bitstream blocks corresponding to different components are combined together to generate the bitstream data block of the image block. The bitstream data block is combined with the bitstream information to generate the image block data packet for that image block.

6. The method according to claim 5, characterized in that, The step of generating the bitstream data block of the image block by sequentially combining the bitstream length of each component and multiple bitstream blocks corresponding to different components according to a preset component order includes: The first bitstream data block is generated by combining the bitstream lengths of different components in a preset component order. According to the preset component order, the corresponding positions of the bitstream blocks under different components are combined together to generate the second bitstream data sub-block, and multiple second bitstream data sub-blocks are combined together to generate the second bitstream data block. A stream data block is formed by the first stream data block and the second stream data block.

7. A video encoding device, characterized in that, include: The image segmentation module is used to segment multiple pixels in each frame of video image in the video data to be encoded, and obtain multiple image blocks corresponding to that frame of video image; The image block encoding module is used to encode each pixel in each image block using multiple encoding methods, generating multiple candidate image block bitstreams for the image block under different encoding methods; The bitstream selection module is used to determine the bitstream length of each candidate image block bitstream corresponding to the image block, and select the candidate image block bitstream with the smallest bitstream length as the target image block bitstream of the image block; The data packet generation module is used to package the target image block bitstream corresponding to the image block and generate the image block data packet corresponding to the image block. The various encoding methods include direct transmission, direct encoding, and differential encoding. The image block encoding module is specifically used for: For the direct transmission method, the color value of each pixel in the image block is obtained, and the color values ​​of each pixel are directly concatenated to generate the first candidate image block bitstream under the direct transmission method. Each color channel corresponds to a component. For the direct encoding method, each pixel in the image block is transformed by XYZ color space to generate XYZ space data corresponding to each pixel. The XYZ space data includes X component data, Y component data and Z component data. For each component, the component data between adjacent pixels in the image block are subtracted under that component to generate entropy-encoded data under that component. Based on the entropy-encoded data under different components, a second candidate image block bitstream under the direct encoding method of the image block is generated. For the differential coding method, the XYZ spatial data of the target image block in the target encoded frame video image is obtained. The target encoded frame video image is the frame data that has been encoded and is closest to the frame video data corresponding to the image block. The target image block is the image block in the target encoded frame video data that corresponds to the image block. Subtract the XYZ spatial data of each pixel in the image block from the XYZ spatial data of the corresponding pixel in the target image block to obtain the component difference data of each pixel in the image block; For each component, the component difference data between adjacent pixels in the image block is subtracted to generate the entropy-to-entropy encoded data for that component. Based on the entropy-to-entropy encoded data for different components, the third candidate image block bitstream under the difference encoding method is generated for that image block.

8. An electronic device, characterized in that, include: The device includes a processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of the video encoding method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the video encoding method as described in any one of claims 1 to 6.

Images