Methods for processing video data, encoding methods, electronic devices, and storage media
By selecting a suitable scanning order and encoding method in the wavelet transform subgraph, the problem of low encoding efficiency in the prior art is solved, and efficient video image storage and reconstruction are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- PEKING UNIV
- Filing Date
- 2023-03-24
- Publication Date
- 2026-06-02
AI Technical Summary
Existing entropy coding algorithms fail to effectively utilize the frequency domain characteristics of wavelet transform in their scanning method within the wavelet transform subgraph, resulting in uneven distribution of non-zero transform coefficients and affecting coding efficiency.
Based on the video bitstream information, a suitable scanning order is selected, and coefficients are scanned for different types of wavelet transform subgraphs, including zigzag order and explicit/implicit selection methods, and encoding is performed in combination with context-adaptive coding methods.
It improves encoding efficiency, saves bitrate on video image storage, reconstructs visually lossless video image content, and has extremely low encoding latency and high parallelism, making it suitable for high-resolution video image storage.
Smart Images
Figure CN116489357B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of video data processing technology, specifically to a method for processing video data, an encoding method, an electronic device, and a storage medium. Background Technology
[0002] Entropy coding algorithms are a crucial component of video compression algorithms. With the iteration and innovation of video compression algorithms, a series of entropy coding algorithms have been developed for different scenarios. Discrete Wavelet Transform (DWT), due to its simplicity, high compression efficiency, and support for scalable decoding, is used in various image and video compression standards such as JPEG-2000 and JPEG-xs. A WxH resolution image, after undergoing one DWT, yields four WxH resolution images. The subgraphs are categorized into LL, HL, LH, and HH based on their corresponding frequency bands in the original graph. Several methods exist for encoding wavelet transform subgraphs, such as further dividing the subgraph into blocks for Discrete Cosine Transform (DCT) to concentrate energy, followed by encoding the transform coefficients using Context Adaptive Variable-Length Code (CAVLC). Encoding the transform coefficients with fewer bits remains a technical challenge.
[0003] In related technologies, DCT transform coefficients are usually scanned in a Zigzag manner. This method does not take into account the characteristics of wavelet transform sub-images. Taking the HL sub-image as an example, the horizontal direction represents the high-frequency part of the original image, and the vertical direction represents the low-frequency part of the original image. Usually, the energy of the low-frequency part in the image is higher than that of the high-frequency part, which causes the non-zero transform coefficients in the HL sub-image to be mainly concentrated in the first row. Similarly, in the LH sub-image, the non-zero transform coefficients are mainly concentrated in the first column.
[0004] In addition, due to the frequency domain characteristics of wavelet transform, the spectrum of high-frequency subgraphs will be flipped. Taking the HL subgraph as an example, the lowest frequency in the horizontal direction of the subgraph corresponds to the highest frequency in the horizontal direction of the original image, and the highest frequency in the horizontal direction of the subgraph corresponds to the middle frequency of the original image. This characteristic causes the non-zero coefficients of the HL subgraph to be concentrated in the upper right corner after DCT transformation, while the non-zero coefficients in the image are usually concentrated in the upper left corner. This makes the existing coefficient scanning method unsuitable for wavelet transform subgraphs. Summary of the Invention
[0005] The purpose of this application is to provide a method for processing video data, an encoding method, an electronic device, and a storage medium. To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general description, nor is it intended to identify key / important components or describe the scope of protection of these embodiments. Its sole purpose is to present some concepts in a simple form as a prelude to the detailed description that follows.
[0006] According to one aspect of the embodiments of this application, a method for processing video data is provided, comprising:
[0007] The decision is made based on information related to a coding block in the video bitstream, and a selection is made from a variety of predefined scan orders to perform mutual conversion between the coefficient scan sequence of the current coding block and the transform coefficient matrix of the current coding block.
[0008] In some embodiments of this application, for a coding block, one of a variety of preset scanning methods is implicitly selected according to different subgraph types to perform mutual conversion between the coefficient scan sequence of the current coding block and the transformation coefficient matrix of the current coding block.
[0009] In some embodiments of this application, the scanning method includes:
[0010] The zigzag sequence is defined as starting from a corner of the region and scanning along the 45-degree diagonal of the isosceles right triangle formed by that corner, then scanning along the next adjacent diagonal in the opposite direction, and repeating this process until all coefficients have been scanned.
[0011] The transform blocks in the LL subgraph are scanned in zigzag order starting from the top left corner;
[0012] The transform block in the HL subgraph first scans the first row of coefficients from right to left, and the remaining area is scanned in zigzag order starting from the top right corner;
[0013] In the LH subgraph, the transform block first scans the coefficients of the first row from bottom to top, and the remaining area is scanned in zigzag order starting from the bottom left corner;
[0014] The transform blocks in the HH subgraph are scanned in zigzag order starting from the bottom right corner.
[0015] In some embodiments of this application, for a coding block, one of a variety of preset scanning methods is explicitly selected based on the marking information in the video bitstream, wherein the marking information is located in the bitstream corresponding to the coding block, or in the bitstream defined in a higher-level bitstream structure to which the current coding block belongs.
[0016] In some embodiments of this application, the bitstream defined in the higher-level bitstream structure to which the current coding block belongs includes bitstreams at the subgraph level, stripe level, and image level.
[0017] In some embodiments of this application, the coefficient scan sequence is derived from its representation in the video bitstream.
[0018] In some embodiments of this application, the representation is defined by context-adaptive coding CAVLC, or the representation is defined by context-adaptive binary arithmetic coding CABAC.
[0019] According to another aspect of the embodiments of this application, an encoding method is provided, including performing DWT transformation on an image to obtain several sub-images, dividing each sub-image into an encoding block for prediction, transformation, and quantization, and then converting the quantized coefficient matrix into a bitstream representation according to any one of the above methods.
[0020] According to another aspect of the embodiments of this application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in any of the above embodiments.
[0021] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided having a computer program stored thereon, the computer program being executed by a processor to implement the method described in any of the preceding claims.
[0022] One aspect of the technical solution provided in this application embodiment may include the following beneficial effects:
[0023] The video data processing method provided in this application makes a decision based on information related to a coding block in the video bitstream, selecting one from a variety of predefined scanning sequences for mutual conversion between the coefficient scanning sequence and the transform coefficient matrix of the current coding block. This can further save the bitrate of video image storage, can be integrated into a video image coding system, reconstruct visually lossless video image content, and has extremely low coding latency and high parallelism, which is beneficial for the storage of high-resolution video images. Attached Figure Description
[0024] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0025] Figure 1 A schematic flowchart of a method for processing video data according to an embodiment of this application is shown.
[0026] Figure 2 This illustration shows a transformation block in an LL subgraph of this application scanned in a zigzag order starting from the top left corner.
[0027] Figure 3 The diagram illustrates how, in an example of this application, the transform block in the HL subgraph first scans the first row of coefficients from right to left, and the remaining area is scanned in a zigzag order starting from the top right corner.
[0028] Figure 4 The diagram illustrates how, in an example of this application, the transform block in the LH subgraph first scans the first row of coefficients from bottom to top, and the remaining area is scanned in zigzag order starting from the bottom left corner.
[0029] Figure 5 This illustration shows a transformation block in the HH subgraph of an example of this application, scanned in zigzag order starting from the lower right corner.
[0030] Figure 6 A block diagram of an electronic device structure according to an embodiment of this application is shown.
[0031] Figure 7 A schematic diagram of a computer-readable storage medium according to an embodiment of this application is shown. Detailed Implementation
[0032] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.
[0033] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have a meaning consistent with their meaning in the context of the relevant art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.
[0034] Video image content creation is a crucial industrial application area. Unlike broadcast television video image coding methods, content creation coding places extremely high demands on the quality of encoding and reconstruction. With the increase in video image resolution, the amount of data that needs to be stored increases exponentially across multiple dimensions, posing challenges to existing coding methods in terms of compression efficiency, real-time performance, and parallelism. Therefore, there is an urgent need for efficient, real-time, and high-quality video image coding methods.
[0035] refer to Figure 1As shown, one embodiment of this application provides a method for processing video data, including:
[0036] The decision is made based on information related to a coding block in the video bitstream, and a selection is made from a variety of predefined scan orders to perform mutual conversion between the coefficient scan sequence of the current coding block and the transform coefficient matrix of the current coding block.
[0037] Specifically, for a coding block, one of a variety of preset scanning methods is implicitly selected based on different subgraph types to perform mutual conversion between the coefficient scan sequence of the current coding block and the transformation coefficient matrix of the current coding block.
[0038] In one implementation, the scanning method may include:
[0039] a) Define the zigzag order as starting from a corner of the region and scanning along the 45-degree diagonal of the isosceles right triangle formed by that corner, then scanning along the next adjacent diagonal in the opposite direction, repeating this process until all coefficients have been scanned.
[0040] b) The transform blocks in the LL subgraph are scanned in zigzag order starting from the top left corner; see reference for details. Figure 2 As shown;
[0041] c) In the HL subgraph, the transform block first scans the coefficients of the first row from right to left, and then scans the remaining regions in zigzag order starting from the top right corner; see reference for details. Figure 3 As shown;
[0042] d) In the LH subgraph, the transform block first scans the coefficients of the first row from bottom to top, and then scans the remaining regions in zigzag order starting from the bottom left corner; for details, please refer to [reference needed]. Figure 4 As shown;
[0043] e) The transform blocks in the HH subgraph are scanned in zigzag order starting from the bottom right corner; see reference for details. Figure 5 As shown.
[0044] In one implementation, for a coding block, one of a variety of preset scanning methods is explicitly selected based on the marking information in the video bitstream, wherein the marking information is located in the bitstream corresponding to the coding block, or in the bitstream defined in a higher-level bitstream structure to which the current coding block belongs.
[0045] In one implementation, the bitstream defined in the higher-level bitstream structure to which the current coding block belongs includes bitstreams at the subgraph level, stripe level, and image level.
[0046] For example, the coefficient scan sequence is derived from its representation in the video bitstream.
[0047] For example, the representation is the representation defined by Context Adaptive Coding (CAVLC).
[0048] For example, the representation may be a representation defined by the context-adaptive binary arithmetic code CABAC.
[0049] In one embodiment, the method further includes: inverse quantization of the result obtained by conversion through implicitly selected scanning methods.
[0050] In one embodiment, the method further includes: performing an inverse transformation on the result obtained from the transformation.
[0051] For example, the inverse transform includes the discrete cosine transform and the discrete sine transform.
[0052] In one embodiment, the method further includes: performing inverse prediction on the converted result according to a method defined in the bitstream to obtain the reconstructed pixel matrix of the current coding block.
[0053] In one embodiment, the method further includes performing an inverse DWT transform after all coded blocks of the current image have been processed.
[0054] In one implementation, the decision-making method can be explicit, such as selecting a first scanning order when the scanning mode indicator bit is a first value, selecting a second scanning order when the scanning mode indicator bit is a second value, and so on; it can also be implicit, for example, implicitly selecting the scanning order based on the type of the sub-image to which the coded block belongs, such as using the first scanning order when the coded block belongs to an LL sub-image, and using the second scanning order when the coded block belongs to an HL sub-image, and so on. The scanning mode used in this embodiment is a coefficient scanning mode suitable for coding wavelet transform sub-images.
[0055] The video data processing method provided in this application makes a decision based on information related to a coding block in the video bitstream, selecting one from a variety of predefined scanning sequences for mutual conversion between the coefficient scanning sequence and the transform coefficient matrix of the current coding block. This can further save the bitrate of video image storage, can be integrated into a video image coding system, reconstruct visually lossless video image content, and has extremely low coding latency and high parallelism, which is beneficial for the storage of high-resolution video images.
[0056] Another embodiment of this application provides an encoding method, which includes performing DWT transformation on an image to obtain several sub-images, dividing each sub-image into an encoding block for prediction, transformation, and quantization, and then converting the quantized coefficient matrix into a bitstream representation according to any of the above embodiments.
[0057] The encoding method of this application embodiment can further save the bit rate of video image storage, can be integrated into the video image encoding system, reconstruct visually lossless video image content, and has extremely low encoding latency and high parallelism, which is beneficial for the storage of high-resolution video images.
[0058] Another embodiment of this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in any of the above embodiments.
[0059] refer to Figure 6 As shown, the electronic device 10 may include: a processor 100, a memory 101, a bus 102 and a communication interface 103. The processor 100, the communication interface 103 and the memory 101 are connected through the bus 102. The memory 101 stores a computer program that can run on the processor 100. When the processor 100 runs the computer program, it executes the method provided in any of the foregoing embodiments of this application.
[0060] The memory 101 may include high-speed random access memory (RAM), and may also include non-volatile memory, such as at least one disk storage device. Communication between this device network element and at least one other network element is achieved through at least one communication interface 103 (which may be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.
[0061] Bus 102 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. Memory 101 is used to store programs. After receiving an execution instruction, processor 100 executes the program. The methods disclosed in any of the foregoing embodiments of this application can be applied to processor 100, or implemented by processor 100.
[0062] Processor 100 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 100 or by instructions in software form. The processor 100 may be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it may also be a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), an Off-the-shelf Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 101. The processor 100 reads the information in memory 101 and, in conjunction with its hardware, completes the steps of the above method.
[0063] The electronic device provided in this application embodiment and the method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement. They can further save the bit rate of video image storage, can be integrated into the video image encoding system, reconstruct visually lossless video image content, and have extremely low encoding latency and high parallelism, which is beneficial for the storage of high-resolution video images.
[0064] Another embodiment of this application provides a computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the methods described in any of the above embodiments. Reference Figure 7 As shown, the computer-readable storage medium is an optical disc 20, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it executes the methods provided in any of the aforementioned embodiments.
[0065] It should be noted that examples of computer-readable storage media may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.
[0066] The computer-readable storage medium provided in the above embodiments of this application and the method provided in the embodiments of this application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application stored therein. They can further save the bit rate of video image storage, can be integrated into the video image encoding system, reconstruct visually lossless video image content, and have extremely low encoding latency and high parallelism, which is beneficial for the storage of high-resolution video images.
[0067] It should be noted that:
[0068] The algorithms and displays provided herein are not inherently related to any particular computer, virtual device, or other equipment. Various general-purpose devices can also be used with the examples based on this. The required structure for constructing such devices is obvious from the above description. Furthermore, this application is not directed to any particular programming language. It should be understood that the content of this application described herein can be implemented using various programming languages, and the above description of specific languages is for the purpose of disclosing the best mode of implementation of this application.
[0069] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.
[0070] The above embodiments merely illustrate the implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A method for processing video data, characterized in that, include: Decisions are made based on information related to a coding block in the video bitstream, and one of several predefined scanning orders is selected to perform mutual conversion between the coefficient scan sequence of the current coding block and the transform coefficient matrix of the current coding block; The step of selecting one from multiple predefined scanning orders includes: for a coding block, implicitly selecting one from multiple preset scanning methods according to different subgraph types; The scanning method includes: in the HL subgraph, the transform block first scans the first row of coefficients from right to left, and the remaining area is scanned in a zigzag order starting from the upper right corner; in the LH subgraph, the transform block first scans the first row of coefficients from bottom to top, and the remaining area is scanned in a zigzag order starting from the lower left corner; wherein, the zigzag order is defined as starting from a corner of the area and scanning along the 45-degree diagonal of the isosceles right triangle formed by that corner, and then scanning along the next adjacent diagonal in the opposite direction, repeating this process until all coefficients have been scanned.
2. The method for processing video data according to claim 1, characterized in that, The scanning methods include: The transform blocks in the LL subgraph are scanned in zigzag order starting from the top left corner; The transform blocks in the HH subgraph are scanned in zigzag order starting from the bottom right corner.
3. The method for processing video data according to claim 1, characterized in that, For a given coding block, one of several preset scanning methods is explicitly selected based on the marker information in the video bitstream, wherein the marker information is located in the bitstream corresponding to the coding block, or in the bitstream defined in a higher-level bitstream structure to which the current coding block belongs.
4. The method for processing video data according to claim 3, characterized in that, The bitstreams defined in the higher-level bitstream structure to which the current coding block belongs include bitstreams at the subgraph level, stripe level, and image level.
5. The method for processing video data according to claim 1, characterized in that, The coefficient scan sequence is derived from its representation in the video bitstream.
6. The method for processing video data according to claim 5, characterized in that, The representation is defined by context-adaptive coding CAVLC, or the representation is defined by context-adaptive binary arithmetic coding CABAC.
7. An encoding method, characterized in that, The method includes performing DWT transformation on an image to obtain several sub-images, dividing each sub-image into a coding block for prediction, transformation, and quantization, and then converting the quantized coefficient matrix into a bitstream representation according to any one of claims 1-6.
8. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to implement the method as described in any one of claims 1-7.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is executed by a processor to implement the method as described in any one of claims 1-7.