Intra prediction mode decision apparatus and method for avs3

Abstract

The application discloses an intra prediction mode decision device and method for AVS3, which comprises a mode coarse selection module, a mode decision module and a pipeline control module. The mode coarse selection module acquires block division information of a maximum coding unit, and removes 64xN and Nx64 block types from the block division information. For 4x4 block types in the remaining block types, a prediction mode is determined. For non-4x4 block types in the remaining block types, the prediction mode is adaptively screened by using image region consistency. The mode obtained above is taken as a coarse selection result. The mode decision module constructs an initial mode list according to the coarse selection result, calculates rate-distortion costs of each prediction mode in the list, and takes a mode corresponding to the minimum rate-distortion cost as an optimal intra prediction mode. The pipeline control module controls seamless switching of the intra prediction mode. The application can effectively solve the problem of AVS3 hardware implementation difficulty, reduce the operation complexity of intra mode decision and accelerate the application of AVS3 in real-time scenarios.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video coding, in particular to an intra prediction mode decision device and method for AVS3. BACKGROUND

[0002] In the modern video communication and entertainment industry, there is an increasing demand for high-efficiency video coding technology. In particular, in the transmission and storage of 4K and 8K ultra-high-definition video content, traditional video coding methods face severe challenges. These application scenarios require the encoder to work at a higher compression rate while maintaining video quality to adapt to the network environment and storage media with limited bandwidth.

[0003] At present, a variety of video coding standards are widely used in the field of video coding, such as high-level video coding (H.264 / AVC) and high-efficiency video coding (H.265 / HEVC). These standards have significantly improved compression efficiency by introducing more complex coding tools and techniques, such as multi-prediction mode, transform quantization, loop filtering, etc. In addition, these standards also support scalable coding unit size to adapt to different resolution and coding needs.

[0004] Although existing video coding standards have made significant progress in compression efficiency, they still have some defects in handling ultra-high-resolution videos. First, as the coding unit size increases, the computational complexity of the coding process significantly increases, which not only increases the computational burden of the codec, but also prolongs the encoding and decoding time, especially on devices with limited hardware resources. Second, in order to handle high-resolution videos, existing technologies often require higher memory bandwidth and processing power, which limits their application in low-power and mobile devices. In addition, existing coding standards may have inaccurate motion estimation when dealing with fast motion scenes, resulting in a decrease in the quality of the encoded video. SUMMARY

[0005] The embodiments of the present application provide an intra prediction mode decision device and method for AVS3. To have a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not a general review, nor does it determine the key / important components or delineate the protection scope of these embodiments. Its only purpose is to present some concepts in a simple form as a prelude to the detailed description that follows.

[0006] In a first aspect, the embodiments of the present application provide an intra prediction mode decision device for AVS3, the device comprising:

[0007] a mode coarse selection module, a mode decision module, and a pipeline control module; wherein,

[0008] The mode coarse selection module, the mode decision module and the pipeline control module are electrically connected.

[0009] The mode coarse selection module is configured to obtain block division information of a maximum coding unit, and filter block types supported by the AVS3 standard from the block division information to eliminate 64xN and Nx64 block types, so as to obtain a plurality of target block types; for a 4x4 block type included in the plurality of target block types, filter an initial prediction mode of the 4x4 block; for a non-4x4 block type included in the plurality of target block types, adaptively filter an initial prediction mode of the non-4x4 block by using image region consistency; and take the initial prediction mode of the 4x4 block and the initial prediction mode of the non-4x4 block as a coarse selection result.

[0010] The mode decision module is configured to construct an initial mode list of a mode decision stage according to the coarse selection result, calculate rate-distortion costs of each prediction mode in the initial mode list, and take a prediction mode corresponding to a minimum rate-distortion cost as an optimal intra prediction mode.

[0011] The pipeline control module is configured to control seamless switching of the intra prediction mode, so as to ensure that a calculation result is correct and one calculation result is generated in each clock cycle.

[0012] Optionally, the mode coarse selection module comprises a first intra prediction circuit, a Hadamard transform cost calculation circuit and a cost sorting circuit; the first intra prediction circuit, the Hadamard transform cost calculation circuit and the cost sorting circuit are electrically connected; the first intra prediction circuit adopts 64-pixel parallel calculation, and different prediction modes are calculated in a pipeline manner.

[0013] The first intra prediction circuit is configured to, for a 4x4 block type included in the plurality of target block types, traverse prediction mode 0 to prediction mode 32 in a preset intra prediction mode set supported by the AVS3 standard, so as to obtain 33 intra prediction results.

[0014] The Hadamard transform cost calculation circuit is configured to calculate Hadamard transform costs by using the 33 intra prediction results.

[0015] The cost sorting circuit is configured to sort all the calculated Hadamard transform costs, and take an initial prediction mode of a 4x4 block corresponding to N minimum values in the sorting as the initial prediction mode.

[0016] Optionally, the mode decision module comprises a mode initial list construction circuit, an intra prediction circuit and a rate-distortion optimization circuit; the mode initial list construction circuit, the second intra prediction circuit and the rate-distortion optimization circuit are electrically connected; wherein,

[0017] The mode initial list construction circuit is configured to construct an initial mode list of a mode decision stage according to the coarse selection result.

[0018] a second intra prediction circuit configured to perform intra prediction on each mode in the initial mode list to obtain a residual value of each mode;

[0019] a rate-distortion optimization circuit configured to calculate a rate-distortion cost of each prediction mode in the initial mode list according to the residual value of each mode, and select a prediction mode corresponding to a minimum rate-distortion cost as an optimal intra prediction mode.

[0020] Optionally, for the non-4x4 block types included in the plurality of target block types, the initial prediction modes of the non-4x4 blocks are adaptively screened by using image region consistency, including:

[0021] collecting the initial prediction modes of the 4x4 blocks of the V 4x4 block types in the same region; V is related to the size of N in the 4xN; when N is 8, V is 2;

[0022] determining whether the initial prediction modes of the 4x4 blocks of the V 4x4 block types are consistent;

[0023] if the initial prediction modes of the 4x4 blocks of the V 4x4 block types are consistent, inheriting the initial prediction modes of the V 4x4 blocks as the initial prediction modes of the non-4x4 blocks in the same region;

[0024] if the initial prediction modes of the 4x4 blocks of the V 4x4 block types are inconsistent, merging the initial prediction modes of the 4x4 blocks of the V 4x4 block types, eliminating the repeated prediction modes after the merging, calculating the Hadamard transform values of each prediction mode after the elimination of the repeated prediction modes, sorting the calculated Hadamard transform values and selecting the prediction modes corresponding to the preset number of minimum values as the initial prediction modes of the non-4x4 blocks in the same region.

[0025] Optionally, according to the coarse selection result, an initial mode list of a mode decision stage is constructed, including:

[0026] traversing the preset intra prediction mode set supported by the AVS3 standard to obtain the remaining 33 intra prediction results of modes 33 to 65;

[0027] constructing the initial mode list of the mode decision stage according to the coarse selection result and in combination with the modes 33 to 65;

[0028] determining whether the initial prediction modes of the 4x4 blocks and the initial prediction modes of the non-4x4 blocks contain a preset plurality of angle modes, the mode indexes of the preset plurality of angle modes being K and L;

[0029] If all are included, calculate the adjacent mode index corresponding to the preset plurality of angle modes, and the calculation method of the adjacent mode index is: (32+K-1) and (32+L-2); based on the adjacent mode index, obtain the adjacent mode corresponding to the preset plurality of angle modes from the remaining 33 intra prediction results; introduce the adjacent mode corresponding to the preset plurality of angle modes into the mode decision to obtain the initial mode list of the mode decision stage.

[0030] If one is included, no new mode is introduced into the mode decision.

[0031] Optionally, the seamless switching of the intra prediction mode is controlled, including:

[0032] When the switching occurs between the prediction modes with the same number of pipeline stages, the next period opened in the current prediction mode is directly switched; or,

[0033] When switching from a prediction mode with a smaller number of pipeline stages to a prediction mode with a larger number of pipeline stages, the new prediction mode is opened (m-n-1) periods in advance, the number of pipeline stages of the smaller number of pipeline stages is n, and the number of pipeline stages of the larger number of pipeline stages is m, m>n; or,

[0034] When switching from a prediction mode with a larger number of pipeline stages to a prediction mode with a smaller number of pipeline stages, the new prediction mode is opened (m-n+1) periods in advance.

[0035] Optionally, the pipeline control module is further configured to configure the prediction mode 0, the prediction mode 1 and the prediction mode 2 in the preset intra prediction mode set supported by the AVS3 standard as 3-stage pipeline, the prediction mode 12 and the prediction mode 24 as 2-stage pipeline, and other prediction modes as 4-stage pipeline.

[0036] Optionally, the internal circuit of the mode coarse selection module and the mode decision module adopts a pipeline architecture design; and the plurality of target block types include block types of 4x 4, 4x 8, 8x 4, 8x 8, 4x 16, 16x 4, 8x 16, 16x 8, 16x 16, 4x 32, 32x 4, 8x 32, 32x 8, 16x 32, 32x 16, 32x 32.

[0037] In a second aspect, an intra prediction mode decision method for AVS3 includes:

[0038] The block partition information of the maximum coding unit is acquired, and block types supported by the AVS3 standard are filtered from the block partition information to eliminate the 64xN and Nx64 block types, to obtain a plurality of target block types; for the 4x4 block type included in the plurality of target block types, the initial prediction mode of the 4x4 block is filtered; for the non-4x4 block type included in the plurality of target block types, the initial prediction mode of the non-4x4 block is adaptively filtered by using image region consistency; and the initial prediction mode of the 4x4 block and the initial prediction mode of the non-4x4 block are taken as the rough selection result.

[0039] According to the rough selection result, an initial mode list of the mode decision stage is constructed; rate-distortion costs of each mode in the initial mode list are calculated; and a prediction mode corresponding to the minimum rate-distortion cost is taken as the optimal intra prediction mode.

[0040] The intra prediction mode is controlled to be seamlessly switched to ensure that there is no error in the calculation result and one calculation result is generated in each clock cycle.

[0041] Optionally, for the 4x4 block type included in the plurality of target block types, the initial prediction mode of the 4x4 block is filtered, including:

[0042] For the 4x4 block type included in the plurality of target block types, prediction modes 0 to 32 in a preset intra prediction mode set supported by the AVS3 standard are traversed to obtain 33 intra prediction results.

[0043] The 33 intra prediction results are used to calculate Hadamard transform values;

[0044] All the calculated Hadamard transform values are sorted;

[0045] The initial prediction mode of the 4x4 block corresponding to the sorted N minimum values is taken.

[0046] In the embodiments of the present application, on the one hand, in order to adapt the coding algorithm to the hardware platform, the block types supported by the AVS3 standard are filtered to eliminate the 64xN and Nx64 block types, the number of original block types is reduced from 23 to 16, the complexity of the algorithm is greatly reduced, the coding efficiency is effectively improved, and the area and power consumption of FPGA or ASIC can be effectively reduced in hardware implementation. On the other hand, the present application adaptively uses the correlation between modes of different coding blocks in the same region by using image region consistency to adaptively filter the modes, which effectively reduces the number of prediction modes, which also greatly reduces the complexity of the algorithm. On the other hand, by controlling the seamless switching of the prediction mode, the prediction results are free of bubbles, one calculation result is generated in each cycle, and the calculation efficiency of the algorithm is effectively improved.

[0047] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application, as claimed. BRIEF DESCRIPTION OF DRAWINGS

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and serve to explain the principles of the application, in which, like reference numerals designate corresponding parts throughout the several views.

[0049] Figure 1 is a device structure schematic diagram of an intra prediction mode decision device for AVS3 provided by an embodiment of the application;

[0050] Figure 2 is a schematic diagram of different block types in the same region provided by an embodiment of the application;

[0051] Figure 3 is an intra prediction mode schematic diagram provided by an embodiment of the application;

[0052] Figure 4A is an intra prediction flow control schematic diagram corresponding to a 4x4 coding block provided by an embodiment of the application;

[0053] Figure 4B is another intra prediction flow control schematic diagram corresponding to a 4x4 coding block provided by an embodiment of the application;

[0054] Figure 5 is a flow schematic diagram of an intra prediction mode decision method for AVS3 provided by an embodiment of the application;

[0055] Figure 6 is a structure schematic diagram of an electronic device provided by an embodiment of the application. DETAILED DESCRIPTION

[0056] The following description and drawings are illustrative of the specific embodiments of the present application and are not intended to limit the generality of the application as set forth in the appended claims.

[0057] It should be noted that the described embodiments are merely a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0058] The following description refers to the accompanying drawings. Unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements. The following description, given together with the accompanying drawings, is a description of the embodiments of the application, and is not representative of all the embodiments consistent with the application. Rather, they are merely examples of devices and methods consistent with some aspects of the application, as detailed in the appended claims.

[0059] In the description of the present application, it is understood that the terms "first", "second" and the like are used only for descriptive purposes, and cannot be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances. In addition, in the description of the present application, "a plurality of" means two or more, unless otherwise specified. The association relationship of the associated objects is described, and it means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. The character " / " generally represents that the associated objects before and after are a kind of "or" relationship.

[0060] In order to solve the problems existing in the prior art, the present application provides an intra prediction mode decision device and method for AVS3. In the embodiments of the present application, on the one hand, in order to adapt the coding algorithm to the hardware platform, the block types supported in the AVS3 standard are screened, and the block types of 64x N and N x 64 are deleted, reducing the number of original block types from 23 to 16, greatly reducing the complexity of the algorithm, effectively improving the coding efficiency, and effectively reducing the area and power consumption of FPGA or ASIC in hardware implementation. On the other hand, the present application utilizes the consistency of image regions, adaptively utilizes the correlation between different coding blocks in the same region, and adaptively screens the modes, effectively reducing the number of prediction modes, which also greatly reduces the complexity of the algorithm. On the other hand, by controlling the seamless switching of the prediction modes, the prediction results are free of bubbles, one cycle one calculation result, which effectively improves the calculation efficiency of the algorithm. The following will be described in detail by exemplary embodiments.

[0061] Please refer to Figure 1 , Figure 1An apparatus structure diagram of an intra prediction mode decision device for AVS3 is provided in the embodiment of the present application. The device comprises a mode coarse selection module, a mode decision module, and a pipeline control module. The mode coarse selection module, the mode decision module, and the pipeline control module are electrically connected. The mode coarse selection module is configured to obtain block division information of a maximum coding unit, and filter block types supported by the AVS3 standard from the block division information to eliminate 64xN and Nx64 block types, to obtain a plurality of target block types. For a 4x4 block type included in the plurality of target block types, an initial prediction mode of the 4x4 block is filtered. For a non-4x4 block type included in the plurality of target block types, an initial prediction mode of the non-4x4 block is adaptively filtered by using image region consistency. The initial prediction mode of the 4x4 block and the initial prediction mode of the non-4x4 block are taken as a coarse selection result. The mode decision module is configured to construct an initial mode list of a mode decision stage according to the coarse selection result, calculate rate-distortion costs of each prediction mode in the initial mode list, and take a prediction mode corresponding to a minimum rate-distortion cost as an optimal intra prediction mode. The pipeline control module is configured to control seamless switching of the intra prediction mode, to ensure that there is no error in a calculation result and one calculation result is generated in each clock cycle.

[0062] The AVS3 standard is a high-definition video coding technology standard, and is used for compression of video data. A maximum coding unit (LCU) is a largest processing unit in video coding, and an image is divided into a plurality of LCUs for coding. Block division information is information describing that an LCU is further divided into smaller blocks. The 64xN and Nx64 block types are specific coding block sizes, and these block types are eliminated in the present application to simplify the coding process. The initial prediction mode of the 4x4 block is a prediction mode determined for the 4x4 block type. The initial prediction mode of the non-4x4 block is a prediction mode adaptively filtered based on image region consistency for the non-4x4 block type. The rate-distortion cost (RDC) is an index for evaluating the advantages and disadvantages of a prediction mode, and considers the number of coding bits and the quality of a reconstructed image.

[0063] For example, an AVS3 video encoder is implemented.

[0064] Experiments show that the block type of 64x N, N x 64 has little effect on the performance of the encoder, and the removal of the above-mentioned block type increases the code rate of the encoder by less than 1%, and the number of block types of the encoder is reduced from 23 to 16, greatly reducing the computational complexity of the encoder, therefore the block type of 64x N, N x 64 is removed, and the block types supported by the application are 16: 4x 4, 4x8, 8x 4, 8x 8, 4x 16, 16x 4, 8x 16, 16x 8, 16x 16, 4x 32, 32x 4, 8x 32, 32x8, 16x 32, 32x 16, 32x 32. The encoding speed of the mode coarsely selected module is not only affected by the block division type, but also affected by the number of prediction modes, therefore how to adaptively select the prediction mode is also a problem that needs to be studied in the hardware implementation of video encoding. The application fully considers the consistency of the image region, such as Figure 2 The correlation between the prediction modes of different prediction blocks of the same region is used to adaptively select the number of modes that need to be traversed by the mode coarsely selected module.

[0065] Therefore, the processing process of the mode coarsely selected module is: receiving the block division information of the LCU, determining which block types the LCU is divided into. According to the AVS3 standard, the block types of 64xN and Nx64 are removed, and 16 target block types are reserved. For the 4x4 block type, the initial prediction mode of the 4x4 block is selected, such as the plane mode, the vertical mode and the like. For the non-4x4 block type, the initial prediction mode of the non-4x4 block is adaptively selected by using the consistency of the image region; the initial prediction mode of the 4x4 block and the initial prediction mode of the non-4x4 block are taken as the coarsely selected result. The initial prediction mode of the 4x4 block and the initial prediction mode of the non-4x4 block are taken as the coarsely selected result.

[0066] The processing process of the mode decision module is: receiving the coarsely selected result of the mode coarsely selected module. Building an initial mode list for the mode decision stage. Calculating the rate-distortion cost of each prediction mode in the initial mode list. Selecting the prediction mode corresponding to the minimum rate-distortion cost as the optimal intra prediction mode.

[0067] In order to meet the needs of real-time application scenarios, the application adopts a pipeline mode for calculation. Since there are 5 categories of intra prediction modes, a total of 65, and the calculation methods of each category of prediction mode are different, the same number of pipeline stages cannot be adopted, which requires the design of a pipeline control module to realize seamless switching of intra prediction modes and ensure that there is no bubble in the calculation result, and one calculation result is output per clock cycle.

[0068] The hardware implementation architecture based on the pipeline design is to improve the coding efficiency. Unlike the software platform implementation, the hardware platform needs to balance power consumption, coding efficiency and coding performance under limited resources. Abandoning the software programming idea, the parallel characteristics of the hardware platform are utilized, and the pipeline architecture is reasonably designed, so that the algorithm can be implemented on the hardware platform with limited resources. Therefore, the processing process of the pipeline control module is to control the seamless switching of the intra prediction mode, ensure the smooth transition of the prediction mode of different block types, ensure that there is no error in the calculation result, and generate one calculation result per clock cycle to improve the coding efficiency.

[0069] In the embodiments of the present application, on the one hand, in order to adapt the coding algorithm to the hardware platform, the block types supported in the AVS3 standard are screened, and the block types of 64x N and N x 64 are deleted, reducing the number of original block types from 23 to 16, greatly reducing the complexity of the algorithm, effectively improving the coding efficiency, and also effectively reducing the area and power consumption of FPGA or ASIC in hardware implementation. On the other hand, the present application utilizes the consistency of image regions, adaptively utilizes the correlation between different coding modes of the same region, and adaptively selects modes to effectively reduce the number of prediction modes, which also greatly reduces the complexity of the algorithm. On the other hand, by controlling the seamless switching of the prediction mode, the prediction results are free of bubbles, one calculation result per cycle, and the calculation efficiency of the algorithm is effectively improved.

[0070] In some embodiments of the present application, the mode coarse selection module includes a first intra prediction circuit, a Hadamard transform cost value calculation circuit and a cost value sorting circuit; the first intra prediction circuit, the Hadamard transform cost value calculation circuit and the cost value sorting circuit are electrically connected; the first intra prediction circuit adopts 64-pixel parallel calculation, and the different prediction modes are calculated in a pipeline manner; the first intra prediction circuit is used for, for the 4x4 block type included in the plurality of target block types, traversing the prediction mode 0 to the prediction mode 32 in the preset intra prediction mode set supported by the AVS3 standard to obtain 33 intra prediction results; the Hadamard transform cost value calculation circuit is used for calculating the Hadamard transform cost value by using the 33 intra prediction results; and the cost value sorting circuit is used for sorting all the calculated Hadamard transform cost values, and taking the initial prediction mode of the 4x4 block corresponding to the N smallest values in the sorting as the initial prediction mode.

[0071] The first intra prediction circuit is part of the mode coarse selection module and is specially used for intra prediction of image blocks. It processes image data to generate prediction blocks based on different prediction modes. The Hadamard transform cost value calculation circuit is used to calculate the Hadamard transform cost value of the prediction blocks obtained using different prediction modes, which is a way to measure prediction error. The cost value sorting circuit is responsible for sorting the Hadamard transform cost values of all prediction modes in order to quickly identify the mode with the smallest cost.

[0072] It should be noted that in the software platform, the intra prediction is calculated in a serial manner, i.e., mode n is calculated, and then mode n+1 is calculated.

[0073] For example, for a 4x4 coding block, the prediction modes in Table 3 are traversed, the Hadamard transform cost values are calculated, the cost values are compared, and the initial prediction mode of the 4x4 block is selected from the N smallest prediction modes.

[0074] In the embodiments of the present application, the first intra prediction circuit calculates in parallel with 64 pixels, greatly improving the prediction rate, so that the encoder can process a large amount of data in a short time. The Hadamard transform cost value calculation circuit analyzes the error of each prediction block, ensuring that the selected prediction mode not only considers the accuracy of the prediction, but also considers the efficiency of the encoding. The cost value sorting circuit ensures that the optimal N modes can be quickly selected from a large number of modes, and the prediction error of these modes is the smallest, thereby providing a solid foundation for the subsequent encoding decision.

[0075] In some embodiments of the present application, for non-4x4 block types included in a plurality of target block types, the specific processing process of adaptively screening the initial prediction mode of the non-4x4 block type includes: collecting the initial prediction modes of the 4x4 blocks of the V 4x4 block types in the same region; V is related to the size of N in 4xN; when N is 8, V is 2; determining whether the initial prediction modes of the 4x4 blocks of the V 4x4 block types are consistent; if consistent, inheriting the initial prediction modes of the V 4x4 blocks as the initial prediction mode of the non-4x4 block in the same region; if not consistent, merging the initial prediction modes of the 4x4 blocks of the V 4x4 block types, eliminating the repeated prediction modes after merging, calculating the Hadamard transform cost value of each prediction mode after eliminating the repeated prediction modes, and sorting the calculated Hadamard transform cost values and selecting the prediction mode corresponding to the preset number of minimum values as the initial prediction mode of the non-4x4 block in the same region.

[0076] For example, for other coding blocks except 4x4 block, using image region consistency, adaptive screening prediction mode, the specific method is as follows, for example, the initial prediction mode list of 4x8 coding block is generated by the mode screening results of two 4x4 coding blocks in the same region, when the mode screening results of two 4x4 coding blocks are consistent, the 8x8 coding block directly inherits the mode screening results of 4x4 coding block, without the need of Hadamard transform value calculation, and without the need of obtaining the mode screening results by comparing the value, the mode screening results of the 4x8 block are consistent with the mode screening results of the 4x4 block in the same region, the Hadamard transform value is the sum of the Hadamard transform values of the two 4x4 blocks in the same region; when the mode screening results of two 4x4 coding blocks are inconsistent, the mode screening results of two 4x4 blocks are combined to form the initial prediction mode list of 4x8 mode screening module, the repeated mode is removed, the Hadamard transform value is calculated, and the mode screening results are obtained by comparing the value; the mode screening results of other non-4x4 coding blocks are obtained in the same way as the 4x8 coding block. The code rate of this mode adaptive screening method increases by less than 1%, but can greatly reduce the number of prediction modes. This method can greatly reduce the calculation complexity of the hardware encoder by sacrificing a little performance of the encoder, which is a good way to balance the performance, power consumption and area of the hardware encoder.

[0077] In some embodiments of the present application, the mode decision module includes a mode initial list construction circuit, an intra prediction circuit and a rate-distortion optimization circuit; the mode initial list construction circuit, the second intra prediction circuit and the rate-distortion optimization circuit are electrically connected; the mode initial list construction circuit is configured to construct an initial mode list of the mode decision stage according to the screening results; the second intra prediction circuit is configured to perform intra prediction on each mode in the initial mode list to obtain a residual value of each mode; the rate-distortion optimization circuit is configured to calculate a rate-distortion cost of each prediction mode in the initial mode list according to the residual value of each mode; and the prediction mode corresponding to the minimum rate-distortion cost is taken as the optimal intra prediction mode. The rate-distortion optimization circuit mainly realizes transformation, quantization, inverse quantization, inverse transformation and reconstruction of the residual, and decides the optimal intra prediction mode of the current coding block by comparing the rate-distortion cost.

[0078] The mode initial list construction circuit is a component in the mode decision module that constructs a more detailed mode list based on the coarse selection results provided by the mode coarse selection module for the next decision process. The second intra prediction circuit is used to perform intra prediction on each mode in the initial mode list to generate residual values for the largest coding unit (LCU). The residual value refers to the difference between the original image block and the predicted image block. The rate-distortion optimization circuit is responsible for calculating the rate-distortion cost (RDC) of each prediction mode, which is a trade-off between the number of encoding bits (rate) and the quality of the reconstructed image (distortion).

[0079] In the embodiments of the present application, the mode initial list construction circuit first generates a comprehensive mode list based on the coarse selection results to provide a basis for fine selection. Then, the second intra prediction circuit performs detailed intra prediction on these modes to calculate the corresponding image residuals for each mode. Then, the rate-distortion optimization circuit analyzes these residuals to calculate the rate-distortion cost of each mode, ensuring that the selected mode is not only accurate in prediction but also efficient in encoding. Finally, the mode with the smallest rate-distortion cost is selected as the optimal prediction mode, which not only ensures the encoding quality but also optimizes the bit rate of the encoding, achieving efficient and high-quality video encoding. This fine decision-making process enables the encoder to maintain high efficiency and excellent encoding performance when processing complex or varied video content.

[0080] In some embodiments of the present application, the specific process of constructing the initial mode list in the mode decision stage based on the coarse selection results includes: traversing the preset intra prediction mode set supported by the AVS3 standard to obtain 33 remaining intra prediction results; constructing the initial mode list in the mode decision stage based on the coarse selection results and in combination with modes 33 to 65; determining whether the initial prediction mode of the 4x4 block and the initial prediction mode of the non-4x4 block contain a preset plurality of angle modes, the mode indices of the preset plurality of angle modes being K and L; if all contain, calculating the adjacent mode indices corresponding to the preset plurality of angle modes, the calculation method of the adjacent mode indices being (32+K-1) and (32+L-2); based on the adjacent mode indices, obtaining the adjacent modes corresponding to the preset plurality of angle modes from the remaining 33 intra prediction results; introducing the adjacent modes corresponding to the preset plurality of angle modes into the mode decision to obtain the initial mode list in the mode decision stage; if one contains, no new mode is introduced into the mode decision.

[0081] For example, in the mode coarse selection stage, only 33 modes (modes 0 to 32) are used, and the remaining 33 modes (modes 33 to 65) are not used, such as Figure 3The shown intra prediction mode, therefore, in the mode decision stage, will adaptively use the remaining 33 modes. The remaining 33 modes are angular prediction modes, and the mode of the remaining 33 modes is adaptively selected from the angular prediction modes in the mode coarsely selected result and enters the mode decision module, that is, the angular prediction mode adjacent to the angular prediction mode in the mode coarsely selected result is introduced into the mode decision module, and the specific operation is as follows: the mode coarsely selected result contains mode K and mode L, then mode (32+K-1) and mode (32+L-2) are introduced into the mode decision, and if there is only one angular mode in the mode coarsely selected result, no new mode needs to be introduced into the mode decision, for example: the mode coarsely selected result contains mode 4 and mode 5, and therefore mode 35 adjacent to mode 4 and mode 5 is introduced into the mode decision; the mode coarsely selected result contains mode 4 and mode 6, and therefore mode 35 adjacent to mode 4 and mode 36 adjacent to mode 6 are introduced into the mode decision. The modes in the coarsely selected mode list in the mode decision are traversed, rate distortion cost (RDC) calculation is performed, the rate distortion cost values are compared, and the intra prediction mode of the coding block is determined.

[0082] In some embodiments of the present application, the pipeline control module is further configured to support, for the preset intra prediction mode set supported by the AVS3 standard, prediction mode 0, prediction mode 1 and prediction mode 2 being configured as 3-stage pipelining, prediction mode 12 and prediction mode 24 being configured as 2-stage pipelining, and other prediction modes being configured as 4-stage pipelining.

[0083] In some embodiments of the present application, the specific process of controlling the seamless switching of the intra prediction mode includes: when the switching occurs between the prediction modes with the same number of pipelining stages, controlling the direct switching in the next period when the current prediction mode is started; or when switching from a prediction mode with a smaller number of pipelining stages to a prediction mode with a larger number of pipelining stages, controlling the starting of the new prediction mode (m-n-1) periods in advance, the number of pipelining stages of the smaller one being n, and the number of pipelining stages of the larger one being m, m>n; or when switching from a prediction mode with a larger number of pipelining stages to a prediction mode with a smaller number of pipelining stages, controlling the starting of the new prediction mode (m-n+1) periods in delay.

[0084] For example, modes 0, 1, and 2 use a 3-stage pipeline, modes 12 and 24 use a 2-stage pipeline, and other modes, which are angle prediction modes, use a 4-stage pipeline. For mode switching with the same number of pipeline stages, the switch can occur in the next cycle after the current mode is started. For mode switching with different numbers of pipeline stages, the mode with fewer pipeline stages (n stages) switches to the mode with more pipeline stages (m stages), with the mode with more pipeline stages starting (mn-1) cycles earlier. The mode with more pipeline stages (m stages) switches to the mode with fewer pipeline stages (n stages), with the mode with more pipeline stages starting (m-n+1) cycles later.

[0085] like Figure 4A and 4B The 4x4 coding block shown uses a pipelined mode switching control method in the mode coarse selection stage. Modes 0, 1, and 2 have the same number of pipeline stages, so switching between the three modes requires a one-clock-cycle delay. Modes 2 and 3 have different numbers of pipeline stages, so they start calculations simultaneously. Since mode 2 has three pipeline stages and mode 3 has four, the calculation result of mode 2 is one clock cycle earlier than that of mode 3. Mode 11 has four pipeline stages and mode 12 has two pipeline stages. When switching from mode 11 to mode 12, the time interval between their start-up is three cycles. Mode 12 has two pipeline stages and mode 13 has four pipeline stages. Mode 13 starts two cycles earlier than mode 12.

[0086] In this application's embodiments, on the one hand, to adapt the encoding algorithm to the hardware platform, the block types supported in the AVS3 standard are filtered, reducing the number of 64xN and Nx64 block types from 23 to 16, significantly reducing the algorithm's complexity and effectively improving encoding efficiency. In hardware implementation, this also effectively reduces the area and power consumption of the FPGA or ASIC. On the other hand, this application utilizes image region consistency, adaptively leveraging the correlation between modes of different encoding blocks within the same region, and adaptively filtering modes to effectively reduce the number of prediction modes, further reducing algorithm complexity. Furthermore, by controlling seamless switching of prediction modes, prediction results are bubble-free, with one calculation result per cycle, effectively improving the algorithm's computational efficiency.

[0087] Please see Figure 5 This document provides a flowchart illustrating an intra-frame prediction mode decision-making method for AVS3, applicable to the server side, as an embodiment of this application. Figure 5 As shown, the detection method in this application embodiment may include the following steps:

[0088] S101, obtain the block partition information of the maximum coding unit, and filter the block types supported by the AVS3 standard from the block partition information to eliminate the 64xN and Nx64 block types, to obtain a plurality of target block types; for the 4x4 block type included in the plurality of target block types, filter the initial prediction mode of the 4x4 block; for the non-4x4 block type included in the plurality of target block types, adaptively filter the initial prediction mode of the non-4x4 block by using the image region consistency; and take the initial prediction mode of the 4x4 block and the initial prediction mode of the non-4x4 block as the rough selection result.

[0089] In some embodiments of the present application, the specific process of filtering the initial prediction mode of the 4x4 block for the 4x4 block type included in the plurality of target block types includes: for the 4x4 block type included in the plurality of target block types, traversing the preset intra prediction mode set supported by the AVS3 standard from prediction mode 0 to prediction mode 32 to obtain 33 intra prediction results; calculating the Hadamard transform cost value by using the 33 intra prediction results; sorting all the calculated Hadamard transform cost values; and taking the prediction mode of the 4x4 block corresponding to the N smallest values in the sorting as the initial prediction mode of the 4x4 block. The prediction mode corresponding to the N smallest values can be 1, 2, 3, 4 or 5, which can be set according to actual needs.

[0090] S102, constructing an initial mode list of the mode decision stage according to the rough selection result; calculating the rate-distortion cost of each mode in the initial mode list; and taking the prediction mode corresponding to the minimum rate-distortion cost as the optimal intra prediction mode.

[0091] S103, controlling the seamless switching of the intra prediction mode to ensure that the calculation result is correct and that one calculation result is generated in each clock cycle.

[0092] Assuming N = 3 in the method, the size of the maximum coding block (LCU) is 64x64, the number of cycles required for a maximum coding block (LCU) to complete the intra prediction mode decision in the pipeline architecture of the method can be calculated, and the method supports a total of 16 block types: 4x4, 4x8, 8x4, 8x8, 4x16, 16x4, 8x16, 16x8, 16x16, 4x32, 32x4, 8x32, 32x8, 16x32, 32x16, 32x32. For a 4x4 coding block, the maximum coding block contains 256 4x4 coding blocks, and a total of 256x16 / 64x33 = 2112 cycles are required in the prediction mode coarse selection stage. Other types of coding blocks do not need to traverse 33 prediction modes, and the maximum number of prediction modes for other coding blocks is 2N = 6. In the prediction mode coarse selection stage, a total of 15x256x16 / 64x6 = 5760 cycles are required for the remaining 15 types of blocks other than the 4x4 coding block. Therefore, the maximum coding block (LCU) requires 2112 + 5760 = 7872 cycles to complete the intra prediction mode coarse selection. In the mode decision stage, the number of modes in the initial mode list corresponding to each coding block is less than or equal to 2N = 6. According to the maximum 2N = 6, a total of 16x256x16 / 64x6 = 6144 cycles are required. Since the prediction mode preliminary selection module and the prediction mode decision module are designed using a pipeline architecture, the maximum coding block (LCU) requires 7872 cycles to complete the intra prediction mode decision, which meets the 4k@30fps throughput requirement under a 500MHz clock pulse.

[0093] In the embodiments of the present application, on the one hand, in order to adapt the coding algorithm to the hardware platform, the block types supported in the AVS3 standard are screened, and the block types of 64xN and Nx64 are deleted, reducing the number of original block types from 23 to 16, greatly reducing the complexity of the algorithm, and effectively improving the coding efficiency. In hardware implementation, the area and power consumption of FPGA or ASIC can also be effectively reduced. On the other hand, the present application utilizes the image region consistency, adaptively utilizes the correlation between the modes of different coding blocks in the same region, adaptively screens the modes, and effectively reduces the number of prediction modes, which also greatly reduces the complexity of the algorithm. On the other hand, by controlling the seamless switching of the prediction modes, the prediction results are free of bubbles, one cycle one calculation result, and the calculation efficiency of the algorithm is effectively improved.

[0094] The present application also provides a computer readable medium having program instructions stored thereon, which, when executed by a processor, implement the AVS3-oriented intra prediction mode decision method provided by each of the above method embodiments.

[0095] The application further provides a computer program product comprising instructions which, when executed on a computer, cause the computer to perform the AVS3-oriented intra prediction mode decision method of each method embodiment described above.

[0096] Please refer to Figure 6 A structural schematic diagram of an electronic device is provided for the embodiments of the application. As shown in the figure, the electronic device 1000 can include at least one processor 1001, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002. Figure 6

[0097] The communication bus 1002 is configured to realize the connection and communication between the components.

[0098] The user interface 1003 can include a display screen (Display) and a camera (Camera). Optionally, the user interface 1003 can further include a standard wired interface and a wireless interface.

[0099] The network interface 1004 can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0100] The processor 1001 can include one or more processing cores. The processor 1001 connects various parts in the entire electronic device 1000 through various interfaces and lines, executes various functions of the electronic device 1000 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 1005, and calling data stored in the memory 1005. Optionally, the processor 1001 can be realized in at least one of the following hardware forms: Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 1001 can be integrated with a combination of one or more of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU is mainly used to process operating systems, user interfaces, and application programs. The GPU is responsible for rendering and drawing the content to be displayed on the display screen. The modem is used to process wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 1001, but can be realized by a separate chip.

[0101] ​The memory 1005 can include a random access memory (RAM) and can also include a read-only memory (ROM). Optionally, the memory 1005 includes a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, codes, code sets, or instruction sets. The memory 1005 can include a program storage area and a data storage area, where the program storage area can store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area can store data involved in the above-mentioned various method embodiments, etc. The memory 1005 can also be at least one storage system located away from the aforementioned processor 1001. As shown in Figure 6 The memory 1005 as a computer storage medium can include an operating system, a network communication module, a user interface module, and an AVS3-oriented intra prediction mode decision application program.

[0102] In the electronic device 1000 shown in Figure 6 In the electronic device 1000 shown in

[0103] Obtain the block partition information of the largest coding unit, and from the block partition information, filter the block types supported by the AVS3 standard to eliminate the 64xN and Nx64 block types, to obtain a plurality of target block types; for the 4x4 block type included in the plurality of target block types, filter the initial prediction mode of the 4x4 block; for the non-4x4 block type included in the plurality of target block types, use image region consistency to adaptively filter the initial prediction mode of the non-4x4 block; and take the initial prediction mode of the 4x4 block and the initial prediction mode of the non-4x4 block as the rough selection result;

[0104] According to the rough selection result, construct an initial mode list of the mode decision stage; calculate the rate-distortion cost of each mode in the initial mode list; and take the prediction mode corresponding to the minimum rate-distortion cost as the optimal intra prediction mode;

[0105] Control the seamless switching of the intra prediction mode to ensure that there is no error in the calculation result and that one calculation result is generated per clock cycle.

[0106] In an embodiment, the processor 1001, when performing the initial prediction mode screening of the 4x4 block type included for a plurality of target block types, specifically performs the following operations:

[0107] For the 4x4 block type included for a plurality of target block types, the prediction modes 0 to 32 in the preset intra prediction mode set supported by the AVS3 standard are traversed to obtain 33 intra prediction results;

[0108] The 33 intra prediction results are used to calculate Hadamard transform value;

[0109] All the calculated Hadamard transform values are sorted;

[0110] The prediction mode of the 4x4 block corresponding to the N smallest values in the sorting is the initial prediction mode.

[0111] In the embodiments of the present application, on the one hand, in order to adapt the coding algorithm to the hardware platform, the block types supported in the AVS3 standard are screened, and the 64x N and N x 64 block types are deleted, so that the number of original block types is reduced from 23 to 16, which greatly reduces the complexity of the algorithm and effectively improves the coding efficiency, and in the hardware implementation, the area and power consumption of FPGA or ASIC can also be effectively reduced. On the other hand, the present application utilizes the image region consistency, and adaptively utilizes the correlation between different coding block modes in the same region, and adaptively screens the modes, which effectively reduces the number of prediction modes, which also greatly reduces the complexity of the algorithm. On the other hand, by controlling the seamless switching of the prediction mode, the prediction results are free of bubbles, one cycle one calculation result, which effectively improves the calculation efficiency of the algorithm.

[0112] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware, and the program for the AVS3-oriented intra prediction mode decision can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments. The storage medium of the program for the AVS3-oriented intra prediction mode decision can be a disk, an optical disc, a read-only memory or a random access memory, etc.

[0113] The above only describes the preferred embodiments of the present application, and of course cannot limit the scope of the rights of the present application, so the equivalent changes made according to the claims of the present application still fall within the scope of the present application.

[0114] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware, and the program for the AVS3-oriented intra prediction mode decision can be stored in a computer readable storage medium. When the program is executed, the program can include the processes of the above-mentioned embodiment methods. The storage medium for the AVS3-oriented intra prediction mode decision can be a disk, an optical disk, a read-only memory or a random access memory, etc.

[0115] The above disclosure is only the preferred embodiments of the present application, and of course cannot limit the scope of the rights of the present application, so the equivalent changes made according to the claims of the present application still fall within the scope of the present application.

Claims

1. A frame prediction mode decision-making device for AVS3, characterized in that, The device includes: The module comprises a pattern coarse selection module, a pattern decision module, and a pipeline control module; among which, The pattern coarse selection module, pattern decision module, and pipeline control module are electrically connected; wherein... The coarse selection module is used to obtain the block partitioning information of the largest coding unit, and filter the block types supported by the AVS3 standard from the block partitioning information to eliminate 64xN and Nx64 block types, thereby obtaining multiple target block types; for the 4x4 block types included in the multiple target block types, the initial prediction mode of the 4x4 block is selected; for the non-4x4 block types included in the multiple target block types, the initial prediction mode of the non-4x4 block is adaptively selected using image region consistency; the initial prediction modes of the 4x4 block and the non-4x4 block are used as the coarse selection results. The mode decision module is used to construct an initial mode list for the mode decision stage based on the coarse selection results; calculate the rate-distortion cost of each prediction mode in the initial mode list; and select the prediction mode corresponding to the minimum rate-distortion cost as the optimal intra-frame prediction mode. The step of constructing an initial pattern list for the pattern decision-making stage based on the coarse selection results includes: Traverse the preset intra-prediction modes 33 to 65 supported by the AVS3 standard to obtain the remaining 33 intra-prediction results. Based on the preliminary selection results and in combination with modes 33 to 65, an initial mode list for the mode decision-making stage is constructed. Determine whether the initial prediction mode of the 4x4 block and the initial prediction mode of the non-4x4 block contain multiple preset angle modes, wherein the mode indices of the multiple preset angle modes are K and L; If all are included, calculate the adjacent mode index corresponding to the preset multiple angle modes. The calculation method of the adjacent mode index is (32+K-1) and (32+L-2). Based on the adjacent mode index, obtain the adjacent modes corresponding to the preset multiple angle modes from the remaining 33 intra-frame prediction results. Introduce the adjacent modes corresponding to the preset multiple angle modes into the mode decision to obtain the initial mode list of the mode decision stage. If one is included, no new pattern is introduced into the pattern decision; The pipeline control module is used to control the seamless switching of intra-frame prediction modes to ensure that the calculation results are error-free and that one calculation result is generated per clock cycle. The seamless switching of the control frame prediction mode includes: When a switch occurs between forecast modes with the same number of pipeline stages, control switches directly in the next cycle after the current forecast mode starts; or, When switching from a prediction mode with fewer flow stages to a prediction mode with more flow stages, the control starts the new prediction mode (mn-1) cycles in advance, with n flow stages for fewer flow stages and m flow stages for more flow stages, where m>n; or, When switching from a prediction mode with more flow stages to a prediction mode with fewer flow stages, the control delays the start of the new prediction mode by (m-n+1) cycles. The pipeline control module is used to configure prediction mode 0, prediction mode 1 and prediction mode 2 as 3-level pipelines, prediction mode 12 and prediction mode 24 as 2-level pipelines, and other prediction modes as 4-level pipelines, in the preset intra-frame prediction mode set supported by the AVS3 standard.

2. The apparatus according to claim 1, characterized in that, The mode coarse selection module includes a first intra-frame prediction circuit, a Hadma transform cost calculation circuit, and a cost ranking circuit; the first intra-frame prediction circuit, the Hadma transform cost calculation circuit, and the cost ranking circuit are electrically connected; the first intra-frame prediction circuit uses 64-pixel parallel calculation, and different prediction modes are calculated in a pipeline manner. The first intra-frame prediction circuit is used to traverse the preset intra-frame prediction mode set supported by the AVS3 standard from prediction mode 0 to prediction mode 32 for the 4x4 block types including the multiple target block types, and obtain 33 intra-frame prediction results. The Hadma transform cost calculation circuit is used to calculate the Hadma transform cost using 33 intra-frame prediction results. The value sorting circuit is used to sort all the calculated Hadema transform values ​​and use the prediction patterns corresponding to the N minimum values ​​as the initial prediction patterns for the 4x4 block.

3. The apparatus according to claim 1, characterized in that, The mode decision module includes a mode initialization list construction circuit, an intra-frame prediction circuit, and a rate-distortion optimization circuit; wherein, the mode initialization list construction circuit, the second intra-frame prediction circuit, and the rate-distortion optimization circuit are electrically connected; wherein... The initial pattern list construction circuit is used to construct an initial pattern list for the pattern decision-making stage based on the coarse selection results. The second intra-frame prediction circuit is used to perform intra-frame prediction on each mode in the initial mode list to obtain the residual value of each mode. The rate-distortion optimization circuit is used to calculate the rate-distortion cost of each prediction mode in the initial mode list based on the residual value of each mode; and to select the prediction mode corresponding to the minimum rate-distortion cost as the optimal intra-frame prediction mode.

4. The apparatus according to claim 1, characterized in that, The method for adaptively filtering the initial prediction pattern for non-4x4 blocks, including the various target block types, using image region consistency, includes: Collect the initial prediction patterns of 4x4 blocks of each of the V 4x4 block types within the same region; V is related to the size of N in 4xN; when N is 8, V is 2; Determine whether the initial prediction modes of the 4x4 blocks of each of the V 4x4 block types are consistent; If they are consistent, the initial prediction patterns of V 4x4 blocks are inherited as the initial prediction patterns of non-4x4 blocks in the same region. If there is a discrepancy, the initial prediction patterns of the 4x4 blocks of each of the V 4x4 block types are merged, and duplicate prediction patterns after merging are removed. The Hadma transform value of each prediction pattern after removing duplicate prediction patterns is calculated. The calculated Hadma transform values ​​are sorted, and the prediction pattern corresponding to the minimum value of a preset number is selected as the initial prediction pattern of the non-4x4 blocks in the same region.

5. The apparatus according to claim 1, characterized in that, The internal circuits of the pattern coarse selection module and the pattern decision module adopt a pipelined architecture design; multiple target block types include 4x 4, 4x 8, 8x 4, 8x 8, 4x 16, 16x 4, 8x 16, 16x 8, 16x 16, 4x 32, 32x 4, 8x 32, 32x 8, 16x 32, 32x 16, and 32x 32 block types.

Images