Intra-frame mode determination method and device for video coding and decoding, equipment and storage medium

By selecting the reference image unit as the intra mode according to the priority information in video encoding and decoding, the problem of unreasonable dependence of the inter prediction image unit in intra mode selection is solved, and the parallelism and encoding and decoding efficiency are improved.

CN119011824APending Publication Date: 2024-11-22TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202310581494.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-22
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

During the video encoding and decoding process, when the image units using inter prediction cannot effectively select the intra prediction mode, the prior art needs to introduce unreasonable dependencies, resulting in a low degree of parallelism.

Method used

By determining the reference image unit of the image unit that adopts the inter prediction mode, and selecting a reference image unit as the intra mode based on the prioritized information (such as whether to use the intra prediction mode, quantization parameters, frame level and interframe distance) is the intra-frame mode. , remove dependencies on other image units.

Benefits of technology

The parallelism of intra mode derivation is improved, the intra modes of multiple image units can be deduced simultaneously, and the efficiency and compression rate of the encoding and decoding process are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119011824A_ABST
    Figure CN119011824A_ABST
Patent Text Reader

Abstract

The invention discloses an intra-frame mode determination method and device for video coding and decoding, equipment and a storage medium. The method comprises the following steps: for a first image unit adopting an inter-frame prediction mode, determining a reference image unit of the first image unit; under the condition that the number of the reference image units is greater than or equal to 2, selecting one reference image unit from the reference image units according to the first information to serve as a first reference image unit; and determining the intra-frame mode of the first reference image unit as the intra-frame mode of the first image unit. In the intra-frame mode derivation process, the dependency relationship on the image units except the first image unit and the reference image unit is removed, the intra-frame modes of multiple image units can be derived at the same time, and the degree of parallelism is high.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to the field of video coding and decoding technology, and in particular, to a method, device, equipment and storage medium for determining an intra-frame mode of video coding and decoding. Background Art

[0002] In the video encoding and decoding process, intra-frame prediction can effectively remove the spatial redundancy of the video frame. For an image unit in a video frame, its corresponding intra-frame prediction mode can be selected by referring to other adjacent image units in the video frame. However, there may be image units using inter-frame prediction in the video frame, and these image units cannot provide reference for the selection of the above intra-frame prediction mode.

[0003] In the related art, for a certain image unit, no matter whether the image unit uses intra-frame prediction or inter-frame prediction, an intra-frame mode can be derived. For an image unit using inter-frame prediction, this intra-frame mode is obtained based on the intra-frame mode of the reference image unit of the image unit, which can provide a reference for the selection of intra-frame prediction modes of other image units.

[0004] When there are multiple reference image units for a certain image unit, it is necessary to determine the priorities of these reference image units. In the related art, when making priority decisions, it is necessary to use information from other image units except the reference image unit to assist in the decision, which introduces unreasonable dependencies. When performing intra-frame mode derivation, the degree of parallelism is low. Summary of the invention

[0005] The embodiment of the present application provides a method, apparatus, device and storage medium for determining an intra-frame mode of video coding and decoding. The technical solution provided by the embodiment of the present application is as follows:

[0006] According to one aspect of an embodiment of the present application, a method for determining an intra-frame mode of video coding and decoding is provided, the method comprising:

[0007] For a first image unit using an inter-frame prediction mode, determining a reference image unit of the first image unit;

[0008] In a case where the number of the reference image units is greater than or equal to 2, a reference image unit is selected from each of the reference image units as a first reference image unit according to first information; wherein the first information includes at least one of the following information arranged in priority: whether the reference image unit adopts an intra-frame prediction mode, a quantization parameter (Quantizer Parameter, referred to as QP) of the reference image unit, a level of a frame to which the reference image unit belongs, a distance between a frame to which the reference image unit belongs and a frame to which the first image unit belongs, and a quantization parameter of a frame to which the reference image unit belongs;

[0009] The intra-frame mode of the first reference image unit is determined as the intra-frame mode of the first image unit.

[0010] According to one aspect of an embodiment of the present application, a device for determining an intra-frame mode of video coding and decoding is provided, the device comprising:

[0011] A first determining module, configured to determine, for a first image unit adopting an inter-frame prediction mode, a reference image unit of the first image unit;

[0012] A selection module, configured to select, when the number of the reference image units is greater than or equal to 2, a reference image unit from each of the reference image units according to the first information as a first reference image unit;

[0013] The second determining module is used to determine the intra-frame mode of the first reference image unit as the intra-frame mode of the first image unit.

[0014] According to one aspect of an embodiment of the present application, a computer device is provided, comprising a processor and a memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the intra-frame mode determination method of the above-mentioned video encoding and decoding.

[0015] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is loaded and executed by a processor to implement the above-mentioned intra-frame mode determination method of video encoding and decoding.

[0016] According to one aspect of an embodiment of the present application, a computer program product is provided, which includes a computer program, the computer program is stored in a computer-readable storage medium, and a processor reads and executes the computer program from the computer-readable storage medium to implement the above-mentioned intra-frame mode determination method of video encoding and decoding.

[0017] The technical solution provided by the embodiments of the present application includes at least the following beneficial effects:

[0018] According to at least one of the following information arranged in priority: whether the reference image unit adopts the intra-frame prediction mode, the quantization parameter of the reference image unit, the level of the frame to which the reference image unit belongs, the distance between the frame to which the reference image unit belongs and the frame to which the first image unit belongs, and the quantization parameter of the frame to which the reference image unit belongs, a first reference image unit is selected from each reference image unit of the first image unit, and the intra-frame mode of the first reference image unit is determined as the intra-frame mode of the first image unit. In the intra-frame mode derivation process, the dependency on image units other than the first image unit and its reference image unit is removed, and the intra-frame modes of more than one image unit can be derived at the same time, with high parallelism. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a simplified block diagram of a communication system provided by one embodiment of the present application;

[0020] Figure 2 It is a schematic diagram of a placement method of a video encoder and a video decoder in a streaming transmission environment provided by an embodiment of the present application;

[0021] Figure 3 is a block diagram of intra-frame mode storage provided by one embodiment of the present application;

[0022] Figure 4 is a flowchart of a method for determining an intra-frame mode provided by an embodiment of the present application;

[0023] Figure 5 is a flowchart of a method for determining an intra-frame mode provided by another embodiment of the present application;

[0024] Figure 6 is a schematic diagram of the hierarchy of frames provided by an embodiment of the present application;

[0025] Figure 7 It is a block diagram of an overall solution of a method for determining an intra-frame mode provided by an embodiment of the present application;

[0026] Figure 8 is a block diagram of a priority comparison method of an image reference unit provided by an embodiment of the present application;

[0027] Fig. 9 is a block diagram of an intra-frame mode determination device provided by an embodiment of the present application;

[0028] Fig.10 It is a structural block diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0029] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0030] Please refer to Figure 1 , which shows a simplified block diagram of a communication system provided by an embodiment of the present application. The communication system 100 includes a plurality of devices, which can communicate with each other through, for example, a network 150. For example, the communication system 100 includes a first device 110 and a second device 120 interconnected through the network 150. Figure 1 In an embodiment, the first device 110 and the second device 120 perform unidirectional data transmission. For example, the first device 110 may encode video data, such as a video picture stream captured by the first device 110, for transmission to the second device 120 via the network 150. The encoded video data is transmitted in the form of one or more encoded video code streams. The second device 120 may receive the encoded video data from the network 150, decode the encoded video data to recover the video data, and display the video picture based on the recovered video data. Unidirectional data transmission is more common in applications such as media services.

[0031] In another embodiment, the communication system 100 includes a third device 130 and a fourth device 140 that perform bidirectional transmission of encoded video data, which may occur, for example, during a video conference. For bidirectional data transmission, each of the third device 130 and the fourth device 140 may encode video data (e.g., a video picture stream collected by the device) for transmission to the other of the third device 130 and the fourth device 140 via the network 150. Each of the third device 130 and the fourth device 140 may also receive the encoded video data transmitted by the other of the third device 130 and the fourth device 140, and may decode the encoded video data to recover the video data, and may display the video picture on an accessible display device based on the recovered video data.

[0032] exist Figure 1 In the embodiment of the present invention, the first device 110, the second device 120, the third device 130 and the fourth device 140 may be computer devices such as servers, personal computers and smart phones, but the principles disclosed in the present application may not be limited thereto. The present application embodiment is applicable to PC (Personal Computer), mobile phones, tablet computers, media players and / or dedicated video conferencing equipment. The network 150 represents any number of networks that transmit encoded video data between the first device 110, the second device 120, the third device 130 and the fourth device 140, including, for example, wired and / or wireless communication networks. The communication network 150 can exchange data in circuit switching and / or packet switching channels. The network may include a telecommunications network, a local area network, a wide area network and / or the Internet. For the purpose of the present application, unless explained below, the architecture and topology of the network 150 may be irrelevant to the operation disclosed in the present application.

[0033] As an example, Figure 2 The video encoder and the video decoder are shown in a streaming environment. The subject matter disclosed in the present application is equally applicable to other video-supported applications, including, for example, video conferencing, digital TV, storing compressed video on digital media including CD (Compact Disc), DVD (Digital Versatile Disc), memory stick, etc.

[0034] The streaming system may include an acquisition subsystem 213, which may include a video source 201 such as a digital camera, which creates an uncompressed video picture stream 202. In an embodiment, the video picture stream 202 includes samples taken by a digital camera. The video picture stream 202 is depicted as a thick line to emphasize the high data volume of the video picture stream compared to the encoded video data 204 (or the encoded video code stream), and the video picture stream 202 can be processed by an electronic device 220, which includes a video encoder 203 coupled to the video source 201. The video encoder 203 may include hardware, software, or a combination of hardware and software to implement or implement various aspects of the disclosed subject matter as described in more detail below. Compared to the video picture stream 202, the encoded video data 204 (or the encoded video code stream 204) is depicted as a thin line to emphasize the lower data volume of the encoded video data 204 (or the encoded video code stream 204), which can be stored on the streaming server 205 for future use. One or more streaming client subsystems, such as Figure 2 , can access the streaming server 205 to retrieve the copies 207 and 209 of the encoded video data 204. The client subsystem 206 may include, for example, a video decoder 210 in an electronic device 230. The video decoder 210 decodes the incoming copy 207 of the encoded video data and produces an output video picture stream 211 that can be presented on a display 212 (e.g., a display screen) or another presentation device (not depicted). In some streaming systems, the encoded video data 204, video data 207, and video data 209 (e.g., video bitstreams) may be encoded according to certain video encoding / compression standards.

[0035] It should be noted that the electronic device 220 and the electronic device 230 may include other components (not shown). For example, the electronic device 220 may include a video decoder (not shown), and the electronic device 230 may further include a video encoder (not shown). The video decoder is used to decode the received encoded video data; the video encoder is used to encode the video data.

[0036] It should be noted that the technical solution provided in the embodiments of the present application can be applied to H.266 / VVC (Versatile Video Coding), H.265 / HEVC (High Efficient Video Coding) standards, AVS (Audio Video coding Standard) (such as AVS3) or the next generation of video coding standards, and the embodiments of the present application are not limited to this.

[0037] Another point that needs to be explained is that the execution subject of each step of the method for determining the intra-frame mode of video encoding and decoding provided in the embodiment of the present application can be a decoding end device or an encoding end device. Both the decoding end device and the encoding end device can be computer devices, which refer to electronic devices with data calculation, processing and storage capabilities, such as PCs, mobile phones, tablet computers, media players, dedicated video conferencing equipment, servers, etc.

[0038] In addition, the method provided in the present application can be used alone or in combination with other methods in any order. The encoder and decoder based on the method provided in the present application can be implemented by one or more processors or one or more integrated circuits.

[0039] Before introducing the technical solution of this application, some related technical knowledge involved in this application is first introduced and explained. The following related technologies can be arbitrarily combined with the technical solution of the embodiment of this application as optional solutions, and they all belong to the protection scope of the embodiment of this application. The embodiment of this application includes at least part of the following contents.

[0040] During the video encoding and decoding process, the video frame image will be divided into different image units for processing. When encoding an image unit, the prediction mode used will be selected first, and then the optimal mode information will be stored and written into the bitstream. When encoding or decoding subsequent image units, a list of candidate modes with high probability of being selected can be derived based on the above mode information. The corresponding prediction mode can be selected from this list, which can reduce the bitrate overhead of the video encoding and decoding process.

[0041] Mode information includes both frame-level information and image unit-level information. Frame-level information is the same for each image unit in the same frame, and usually includes frame-level quantization parameters, temporal levels, etc. Image unit-level information may include motion information, such as motion vectors, reference frames, etc.

[0042] In the related art, the mode information may also include an intra-frame mode, please refer to Figure 3After obtaining the mode information of an image unit, the intra-frame mode of the image unit can be derived, and the derived intra-frame mode can be stored as the mode information of the image unit. The intra-frame mode of the image unit can provide a reference for selecting the intra-frame prediction mode of other adjacent image units, and can also be used to derive the intra-frame mode of subsequent image units.

[0043] Next, the intra-frame mode derivation method provided by the related art is introduced. In this method, it is first determined whether the current image unit is an intra-frame prediction unit. If it is, the intra-frame prediction mode of the current image unit is directly determined as the intra-frame mode of the current image unit. If not, the intra-frame mode of the reference image unit of the current image unit is determined as the intra-frame mode of the current image unit. If there are multiple reference image units in the current image unit, the following information arranged in priority is obtained in sequence: whether the reference image unit is an intra-frame prediction unit, the distance between the frame to which the reference image unit belongs and the frame to which the current image unit belongs, the quantization parameter of the last image unit of the frame to which the reference image unit belongs, and the sequence number of the reference image unit. And according to the above information, an optimal reference image unit is compared and selected from multiple reference image units, and finally the intra-frame mode of the optimal reference image unit is determined as the intra-frame mode of the current image unit. Among them, the intra-frame prediction unit refers to an image unit that adopts the intra-frame prediction mode.

[0044] Those skilled in the art should understand that since the video frame sequence of video encoding and decoding must contain a key frame in which all internal image units adopt the intra-frame prediction mode, and this key frame will be used as a reference frame for the subsequent encoding and decoding process, the above method can determine an intra-frame mode for all image units in the video frame sequence.

[0045] However, when determining the optimal reference image unit, the above method does not fully utilize the pattern information of the reference image unit, but uses the quantization parameter of the last image unit of the frame to which the reference image unit belongs, thereby introducing unreasonable dependencies and reducing parallelism.

[0046] The present application optimizes the method for selecting the above-mentioned optimal reference image unit, which will be described in detail in the following embodiments.

[0047] Please refer to Figure 4 , which shows a flow chart of a method for determining an intra-frame mode of video encoding and decoding provided by an embodiment of the present application. The method can be executed by an encoding end device or a decoding end device. The method may include at least one of the following steps 410 to 430:

[0048] Step 410: Determine a reference image unit of the first image unit.

[0049] The first image unit may be any image unit. The image unit may be an image block obtained by partitioning an image according to a certain method, may be a CU (Coding Unit) obtained by recursively dividing a CTU (Coding Tree Unit) in a quadtree manner, or may be a pixel string composed of a series of pixel points. The present application does not limit the specific form of the image unit.

[0050] Any image unit whose corresponding intra-frame prediction mode cannot be obtained can be used as the first image unit. For example, in some embodiments, the first image unit adopts the inter-frame prediction mode. In other embodiments, the first image unit does not adopt the intra-frame prediction mode.

[0051] The reference image unit refers to an image unit that is referenced by the first image unit during predictive coding.

[0052] In some embodiments, the reference image unit of the first image unit can be determined based on the reference frame and motion vector (MV) of the first image unit. Exemplarily, the coordinates of the upper left corner pixel of each image unit can be used as the coordinates of the image unit. If the coordinates of an image unit are (16, 16) and the motion vector is (-3, -1), the coordinates of its reference image unit on the reference frame are (13, 15). If the first image unit has multiple motion vectors and multiple reference frames, then the first image unit should also have multiple reference image units.

[0053] Step 420: When the number of reference image units is greater than or equal to 2, select a reference image unit from each reference image unit as a first reference image unit according to the first information, wherein the first information includes at least one of the following information arranged in priority: whether the reference image unit adopts an intra-frame prediction mode, a quantization parameter of the reference image unit, a level of a frame to which the reference image unit belongs, a distance between a frame to which the reference image unit belongs and a frame to which the first image unit belongs, and a quantization parameter of a frame to which the reference image unit belongs.

[0054] The first information is used to select an optimal reference image unit from various reference image units, and the optimal reference image unit is the first reference image unit.

[0055] The priority of the information included in the first information refers to the order of utilizing the aforementioned information in the process of selecting the first reference image unit. That is, if the first reference image unit can be determined based on the information with a higher priority, there is no need to consider the information with a lower priority.

[0056] In some embodiments, the first information may be mode information of each reference image unit, and may include first sub-information and second sub-information, wherein the first sub-information is information at the image unit level, and the second sub-information is information at the frame level.

[0057] In some embodiments, the first sub-information includes at least one of the following information: whether the reference image unit adopts an intra-frame prediction mode, and a quantization parameter of the reference image unit.

[0058] In some embodiments, the second sub-information includes at least one of the following information: a level of the frame to which the reference image unit belongs, a distance between the frame to which the reference image unit belongs and the frame to which the first image unit belongs, and a quantization parameter of the frame to which the reference image unit belongs.

[0059] It is worth noting that, in some embodiments, the expression of the information included in the first information may be modified, and the amount of information included in the first information may also be modified.

[0060] In some embodiments, the first information includes at least one of the following information: whether the reference image unit is an intra-frame prediction unit, the quantization parameter of the reference image unit, the level of the frame to which the reference image unit belongs, the distance between the frame to which the reference image unit belongs and the frame to which the first image unit belongs, and the quantization parameter of the frame to which the reference image unit belongs.

[0061] In some embodiments, the first information includes at least one of the following information: a quantization parameter of the reference image unit, a level of a frame to which the reference image unit belongs, and a quantization parameter of the frame to which the reference image unit belongs.

[0062] Please refer to Figure 5 In some embodiments, step 420 may include at least one of the following sub-steps 421 to 427.

[0063] Step 421, if there are at least two reference image units that adopt the intra-frame prediction mode, then select a reference image unit whose adopted intra-frame mode belongs to the first set from the at least two reference image units that adopt the intra-frame prediction mode as the first reference image unit; wherein the first set includes at least one intra-frame mode.

[0064] Intra-frame prediction refers to predicting pixels to be reconstructed using reconstructed pixels in the same frame image. For an image unit, whether to adopt the intra-frame prediction mode determines whether intra-frame prediction is used in the encoding and decoding process.

[0065] In an embodiment of the present application, the first set is a predefined set, and at least one intra-frame mode included in the first set may be an intra-frame prediction mode in the related technology. For example, in the H.265 / HEVC standard, the first set may include all intra-frame prediction modes in the standard except the DC / Planar mode.

[0066] Exemplarily, if there are at least two reference image units that use intra-frame prediction modes, and only one of them uses an intra-frame mode that belongs to the first set, the reference image unit whose intra-frame mode belongs to the first set is used as the first reference image unit. For example, if there are three reference image units A, B, and C, all of which use intra-frame prediction modes, and the intra-frame modes of B and C do not belong to the first set, and the intra-frame mode of A belongs to the first set, then A can be used as the first reference image unit.

[0067] In some embodiments, a reference image unit whose adopted intra-frame mode belongs to the first set and has the smallest sequence number can be selected from at least two reference image units adopting the intra-frame prediction mode as the first reference image unit, and the sequence number of the reference image unit is related to the encoding and decoding order of the reference image unit.

[0068] The setting of the reference image unit serial number is described below. Taking two reference image units as an example, in some embodiments, the reference image unit that is first coded and decoded may be assigned serial number 0, and the reference image unit that is later coded and decoded may be assigned serial number 1. In other embodiments, the reference image unit that is later coded and decoded may be assigned serial number 0, and the reference image unit that is first coded and decoded may be assigned serial number 1. In the present application, the serial number of the reference image unit is set by a technician during the coding and decoding process, and the present application does not limit how to set it.

[0069] Exemplarily, if among at least two reference image units that adopt the intra-frame prediction mode, there are two reference image units whose intra-frame modes belong to the first set, the reference image unit with the smaller sequence number among the two reference image units is used as the first reference image unit. For example, if there are three reference image units A, B, and C, all of which adopt the intra-frame prediction mode, the sequence number of A is 0, the sequence number of B is 1, and the sequence number of C is 2, and the intra-frame modes of A and B belong to the first set, and the intra-frame mode of C does not belong to the first set, then A can be used as the first reference image unit.

[0070] The above method introduces a predefined set to assist in selecting the first reference image unit, which can select the first reference image unit more quickly and accurately, thereby improving the efficiency of intra-frame mode determination and obtaining a better intra-frame mode.

[0071] Step 422: If there is only one reference image unit that adopts the intra-frame prediction mode, the reference image unit that adopts the intra-frame prediction mode is determined as the first reference image unit.

[0072] Exemplarily, if there are two or more reference image units, and only one of them adopts the intra-frame prediction mode, the reference image unit adopting the intra-frame prediction mode is determined as the first reference image unit. For example, if there are three reference image units A, B, and C, and only A adopts the intra-frame prediction mode, A can be used as the first reference image unit.

[0073] The above method excludes image units that do not adopt intra-frame prediction mode in the reference image units, so that an intra-frame mode that is more consistent with the actual probability can be derived in the end.

[0074] Step 423: If no reference image unit adopts the intra prediction mode, and among the reference image units, the number of reference image units with the smallest quantization parameter is a and a is equal to 1, the reference image unit with the smallest quantization parameter is determined as the first reference image unit.

[0075] The quantization parameter of the reference image unit can reflect the compression of the spatial details of the reference image unit. The lower the value is, the higher the compression rate of the reference image unit is.

[0076] Exemplarily, if there are two reference image units and none of them adopts the intra-frame prediction mode, the reference image unit with the smallest quantization parameter is used as the first reference image unit. For example, if there are three reference image units A, B, and C, none of which adopts the intra-frame prediction mode, and the quantization parameter of A is the smallest among the three, A can be determined as the first reference image unit. For another example, if there are two reference image units A and B, neither of which adopts the intra-frame prediction mode, and the quantization parameter of A is small, A can be determined as the first reference image unit.

[0077] The above method determines the first reference image unit by the quantization parameter of the reference image unit, and excludes the reference image unit with a higher quantization parameter, so that the intra-frame mode closer to the actual optimal mode can be finally derived, thereby improving the compression rate of the encoding and decoding process.

[0078] Step 424, if the number of reference image units with the smallest quantization parameter is a and a is greater than 1, and among the a reference image units with the smallest quantization parameter, the number of reference image units with the smallest level of the frame to which it belongs is b and b is equal to 1, then the reference image unit with the smallest level of the frame to which it belongs is determined as the first reference image unit.

[0079] The level of a frame refers to the temporal level of a frame. In some embodiments, the encoding may adopt a hierarchical frame structure. The temporal level of a frame is divided as follows: Figure 6As shown, based on the image frames of the first level, the image frames of the second level can be obtained, and based on the image frames of the first and second levels, the image frames of the third level can be obtained, wherein the image frames of the first level have the smallest level.

[0080] Exemplarily, if there are three reference image units A, B, and C with the smallest quantization parameters, and the level of the frame to which A belongs is the smallest, then A can be determined as the first reference image unit.

[0081] Exemplarily, if there are two reference image units A and B with the smallest quantization parameters, and the level of the frame to which A belongs is relatively small, then A may be determined as the first reference image unit.

[0082] The above method determines the first reference image unit by the temporal level of the frame to which the reference image unit belongs, and excludes reference image units with higher temporal levels of the frame to which they belong, so that a better intra-frame mode can be derived in the end.

[0083] Step 425, if the number of reference image units with the smallest level of the belonging frame is b and b is greater than 1, and among the b reference image units with the smallest level of the belonging frame, the number of reference image units with the smallest distance between the frame to which the reference image unit belongs and the frame to which the first image unit belongs is c and c is equal to 1, then the reference image unit with the smallest distance is determined as the first reference image unit.

[0084] In some embodiments, the distance between the frame to which the reference image unit belongs and the frame to which the first image unit belongs can be the time interval between the two. In other embodiments, the distance between the frame to which the reference image unit belongs and the frame to which the first image unit belongs can also be measured by the number of frames between the two.

[0085] Exemplarily, if there are three reference image units A, B, and C belonging to frames with the smallest hierarchy, and the distance between the frame belonging to A and the frame belonging to the first image unit is the smallest, then A can be determined as the first reference image unit.

[0086] Exemplarily, if there are two reference image units A and B belonging to frames with the smallest hierarchy, and the distance between the frame belonging to A and the frame belonging to the first image unit is small, then A can be determined as the first reference image unit.

[0087] The above method determines the first reference image unit by the distance between the frame to which the reference image unit belongs and the frame to which the first image unit belongs, and excludes the reference image unit whose distance between the frame to which the first image unit belongs is larger, so that a better intra-frame mode can be finally derived.

[0088] Step 426, if the number of the reference image units with the smallest distance is c and c is greater than 1, and among the c reference image units with the smallest distance, the number of the reference image units with the smallest quantization parameter of the frame to which they belong is d and d is equal to 1, then the reference image unit with the smallest quantization parameter of the frame to which they belong is determined as the first reference image unit.

[0089] The quantization parameter of the frame to which the reference image unit belongs can reflect the compression of spatial details of the frame to which the reference image unit belongs. The lower the value is, the higher the compression rate of the frame to which the reference image unit belongs is.

[0090] Exemplarily, if there are three reference image units A, B, and C whose distances from the frame to which the first image unit belongs are the smallest, and the quantization parameter of the frame to which A belongs is the smallest, then A can be determined as the first reference image unit.

[0091] Exemplarily, if there are reference image units whose distances between the frames A and B and the frame to which the first image unit belongs are the smallest, and the quantization parameter of the frame to which A belongs is small, then A may be determined as the first reference image unit.

[0092] The above method determines the first reference image unit by using the quantization parameter of the frame to which the reference image unit belongs, and excludes the reference image unit with a larger quantization parameter of the frame to which it belongs, so that a better intra-frame mode can be derived in the end.

[0093] Step 427, if the number of reference image units with the smallest quantization parameters of the frame is d and d is greater than 1, then select a reference image unit with the smallest serial number from the d reference image units with the smallest quantization parameters of the frame as the first reference image unit, and the serial number of the reference image unit is related to the encoding and decoding order of the reference image unit.

[0094] Exemplarily, if there are three reference image units A, B, and C, whose quantization parameters belong to the smallest frames, and A has the smallest sequence number, then A may be determined as the first reference image unit.

[0095] Exemplarily, if there are two reference image units A and B, whose frames have the smallest quantization parameters, and A has a smaller sequence number, then A may be determined as the first reference image unit.

[0096] The above method determines the first reference image unit by the sequence number of the reference image unit, so that no matter what the situation of the reference image unit is, the intra-frame mode of the first image unit can be finally derived.

[0097] Step 430: determine the intra-mode of the first reference image unit as the intra-mode of the first image unit.

[0098] In some embodiments, there is only one reference unit for the first image unit, and the reference unit can be used as the first reference unit.

[0099] In some embodiments, the first reference image unit may be an image unit using an inter-frame prediction mode, and its corresponding intra-frame mode may also be obtained according to the intra-frame mode determination method provided in an embodiment of the present application.

[0100] The technical solution provided by the embodiment of the present application selects a first reference image unit from each reference image unit of the first image unit, and determines the intra-frame mode of the first reference unit as the intra-frame mode of the first image unit, according to at least one of the following information arranged in priority: whether the reference image unit adopts the intra-frame prediction mode, the quantization parameter of the reference image unit, the level of the frame to which the reference image unit belongs, the distance between the frame to which the reference image unit belongs and the frame to which the first image unit belongs, and the quantization parameter of the frame to which the reference image unit belongs. In the process of deriving the intra-frame mode, the dependency on image units other than the first image unit and the reference image unit is removed, and the intra-frame modes of more than one image unit can be derived at the same time, with high parallelism.

[0101] In addition, it should be added that, at both the encoding end and the decoding end, the intra-frame mode determination method provided by the embodiment of the present application can be used to derive the intra-frame mode of the first image unit, and this intra-frame mode can participate in constructing the intra-frame mode candidate list of the second image unit adjacent to the first image unit. During the encoding process, the second image unit selects the optimal intra-frame prediction mode from the above intra-frame mode candidate list, and writes the index of the above optimal mode into the bitstream. During the decoding process, the second image unit can construct the intra-frame mode candidate list in the same way, and directly obtain the index of the optimal mode from the bitstream. Compared with the related art, since the present application introduces information such as the quantization parameter of the reference image unit, the level of the frame to which the reference image unit belongs, and the quantization parameter of the frame to which the reference image unit belongs for the derivation of the intra-frame mode of the first image unit, the division of the priority of the first information is also more reasonable, so a better intra-frame mode candidate list can be obtained for the second image unit, thereby improving the compression rate and speed of the video encoding and decoding process.

[0102] Please refer to Figure 7 , which shows a block diagram of the overall solution of the intra-frame mode determination method provided by an embodiment of the present application.

[0103] In some embodiments, the first image unit may be a first image block, and it is first determined whether the first image block is an intra-frame prediction block using an intra-frame prediction mode. If so, the intra-frame prediction mode of the first image block is directly used as its intra-frame mode. If not, the block-level mode information and frame-level mode information of the reference block of the first image block are obtained according to priority, and based on the above mode information, the optimal reference block is selected, and finally the intra-frame mode of the optimal reference block is used as the intra-frame mode of the first image block. In this scheme, the specific details of selecting the optimal reference block can be found in the above embodiment, and will not be repeated here.

[0104] Please refer to Figure 8 , which shows a block diagram of a priority comparison method of reference image units provided by an embodiment of the present application.

[0105] In some embodiments, the first image unit may have only two reference image units, namely reference image unit A and reference image unit B, where A has a smaller sequence number. When performing priority comparison, it can be first determined whether A and B are both intra-frame prediction units. If not, the intra-frame prediction unit in A and B is selected as the first reference unit. If so, it is further examined whether the intra-frame modes of A and B belong to a predefined set. If neither A nor B is an intra-frame prediction unit, the quantization parameters of A and B, the levels of the frames to which A and B belong, the distance between the frames to which A and B belong and the frame to which the first image unit belongs, and the quantization parameters of the frames to which A and B belong are compared in turn, and finally the first reference unit is selected.

[0106] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0107] Please refer to Fig. 9 , which shows a block diagram of an intra-frame mode determination device for video encoding and decoding provided by an embodiment of the present application. The device has the function of implementing the above-mentioned intra-frame mode determination method, and the function can be implemented by hardware, or by hardware executing corresponding software. The device can be a computer device, or can be set in a computer device. The device 900 may include: a first determination module 910, a selection module 920 and a second determination module 930.

[0108] The first determination module 910 is configured to determine, for a first image unit adopting an inter-frame prediction mode, a reference image unit of the first image unit.

[0109] The selection module 920 is configured to select, when the number of the reference image units is greater than or equal to 2, one reference image unit from the reference image units according to the first information as a first reference image unit.

[0110] The second determination module 930 is configured to determine the intra-frame mode of the first reference image unit as the intra-frame mode of the first image unit.

[0111] In some embodiments, the selection module 920 is used to:

[0112] If there are at least two reference image units that adopt the intra-frame prediction mode, then, from the at least two reference image units that adopt the intra-frame prediction mode, select a reference image unit whose adopted intra-frame mode belongs to a first set as the first reference image unit; wherein the first set includes at least one intra-frame mode;

[0113] If there is only one reference image unit that adopts the intra-frame prediction mode, the reference image unit that adopts the intra-frame prediction mode is determined as the first reference image unit.

[0114] In some embodiments, the selection module 920 is also used to determine the reference image unit with the smallest quantization parameter as the first reference image unit if there is no reference image unit adopting the intra-frame prediction mode, and the number of reference image units with the smallest quantization parameter among the reference image units is a and a is equal to 1.

[0115] In some embodiments, the selection module 920 is also used to determine the reference image unit with the smallest level of the frame as the first reference image unit if the number of reference image units with the smallest quantization parameter is a and a is greater than 1, and among the a reference image units with the smallest quantization parameter, the number of reference image units with the smallest level of the frame is b and b is equal to 1.

[0116] In some embodiments, the selection module 920 is also used to determine the reference image unit with the smallest distance as the first reference image unit if the number of reference image units with the smallest level of the frame to which it belongs is b and b is greater than 1, and among the b reference image units with the smallest level of the frame to which the reference image unit belongs, the number of reference image units with the smallest distance between the frame to which the reference image unit belongs and the frame to which the first image unit belongs is c and c is equal to 1.

[0117] In some embodiments, the selection module 920 is also used to determine the reference image unit with the smallest quantization parameter of the frame as the first reference image unit if the number of the reference image units with the smallest distance is c and c is greater than 1, and among the c reference image units with the smallest distance, the number of the reference image units with the smallest quantization parameter of the frame is d and d is equal to 1.

[0118] In some embodiments, the selection module 920 is also used to select a reference image unit with the smallest serial number from the d reference image units with the smallest quantization parameters of the belonging frame as the first reference image unit if the number of reference image units with the smallest quantization parameters of the belonging frame is d and d is greater than 1, and the serial number of the reference image unit is related to the encoding and decoding order of the reference image unit.

[0119] In some embodiments, the selection module 920 is also used to select, from the at least two reference image units that adopt the intra-frame prediction mode, a reference image unit whose adopted intra-frame mode belongs to the first set and has the smallest serial number, as the first reference image unit, and the serial number of the reference image unit is related to the encoding and decoding order of the reference image unit.

[0120] It should be noted that the device provided in the above embodiment, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0121] Please refer to Fig.10 , which exemplarily shows a structural block diagram of a computer device provided by an embodiment of the present application.

[0122] Typically, the computer device 1000 includes a processor 1001 and a memory 1002 .

[0123] The processor 1001 may include one or more processing cores, such as a 4-core processor, a 14-core processor, etc. The processor 1001 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1001 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 1001 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1001 may also include an AI processor, which is used to process computing operations related to machine learning.

[0124] The memory 1002 may include one or more computer-readable storage media, which may be tangible and non-transitory. The memory 1002 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1002 stores a computer program, which is loaded and executed by the processor 1001 to implement the above-mentioned intra-frame mode determination method.

[0125] Those skilled in the art will understand that Fig.10 The structure shown in the figure does not constitute a limitation on the computer device 1000, and the computer device 1000 may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0126] In some embodiments, a computer-readable storage medium is further provided, wherein a computer program is stored in the storage medium, and the computer program is loaded and executed by a processor to implement the above intra-frame mode determination method.

[0127] Optionally, the computer readable storage medium may include: ROM (Read-Only Memory), RAM (Random-Access Memory), SSD (Solid State Drives) or optical disks, etc. Among them, the random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0128] In some embodiments, a computer program product is also provided, which includes a computer program stored in a computer-readable storage medium, and a processor reads and executes the computer program from the computer-readable storage medium to implement the above-mentioned intra-frame mode determination method.

[0129] It should be understood that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. In addition, the step numbers described in this article only illustrate a possible execution sequence between the steps. In some other embodiments, the above steps may not be executed in the order of the numbers, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in the opposite order to the diagram. The embodiments of the present application are not limited to this.

[0130] The above are merely exemplary embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for determining intra-frame mode of video encoding and decoding, It is characterized in that The method comprises: For a first image unit using an inter-frame prediction mode, determining a reference image unit of the first image unit; In a case where the number of the reference image units is greater than or equal to 2, selecting a reference image unit from each of the reference image units as a first reference image unit according to first information; wherein the first information includes at least one of the following information arranged in priority: whether the reference image unit adopts an intra-frame prediction mode, a quantization parameter of the reference image unit, a level of a frame to which the reference image unit belongs, a distance between a frame to which the reference image unit belongs and a frame to which the first image unit belongs, and a quantization parameter of a frame to which the reference image unit belongs; The intra-frame mode of the first reference image unit is determined as the intra-frame mode of the first image unit.

2. The method according to claim 1, It is characterized in that The step of selecting a reference image unit from each of the reference image units as a first reference image unit according to the first information includes: If there are at least two reference image units that adopt the intra-frame prediction mode, then, from the at least two reference image units that adopt the intra-frame prediction mode, select a reference image unit whose adopted intra-frame mode belongs to a first set as the first reference image unit; wherein the first set includes at least one intra-frame mode; If there is only one reference image unit that adopts the intra-frame prediction mode, the reference image unit that adopts the intra-frame prediction mode is determined as the first reference image unit.

3. The method according to claim 2, It is characterized in that The method further comprises: If no reference image unit adopts the intra-frame prediction mode, and among the reference image units, the number of reference image units with the smallest quantization parameter is a and a is equal to 1, then the reference image unit with the smallest quantization parameter is determined as the first reference image unit.

4. The method according to claim 3, It is characterized in that The method further comprises: If the number of reference image units with the smallest quantization parameter is a and a is greater than 1, and among the a reference image units with the smallest quantization parameter, the number of reference image units with the smallest level of the frame to which they belong is b and b is equal to 1, then the reference image unit with the smallest level of the frame to which they belong is determined as the first reference image unit.

5. The method according to claim 4, It is characterized in that The method further comprises: If the number of reference image units with the smallest level of the belonging frame is b and b is greater than 1, and among the b reference image units with the smallest level of the belonging frame, the number of reference image units with the smallest distance between the frame to which the reference image unit belongs and the frame to which the first image unit belongs is c and c is equal to 1, then the reference image unit with the smallest distance is determined as the first reference image unit.

6. The method according to claim 5, It is characterized in that The method further comprises: If the number of reference image units with the smallest distance is c and c is greater than 1, and among the c reference image units with the smallest distance, the number of reference image units with the smallest quantization parameter of the frame to which they belong is d and d is equal to 1, then the reference image unit with the smallest quantization parameter of the frame to which they belong is determined as the first reference image unit.

7. The method according to claim 6, It is characterized in that The method further comprises: If the number of reference image units with the smallest quantization parameter of the belonging frame is d and d is greater than 1, then a reference image unit with the smallest serial number is selected from the d reference image units with the smallest quantization parameter of the belonging frame as the first reference image unit, and the serial number of the reference image unit is related to the encoding and decoding order of the reference image unit.

8. The method according to claim 2, It is characterized in that The method further comprises: selecting, from the at least two reference image units adopting the intra prediction mode, a reference image unit whose adopted intra prediction mode belongs to a first set as the first reference image unit, comprising: From the at least two reference image units adopting the intra-frame prediction mode, select a reference image unit whose adopted intra-frame mode belongs to the first set and has the smallest sequence number as the first reference image unit, and the sequence number of the reference image unit is related to the encoding and decoding order of the reference image unit.

9. A device for determining intra-frame mode of video encoding and decoding, It is characterized in that The device comprises: A first determining module, configured to determine, for a first image unit adopting an inter-frame prediction mode, a reference image unit of the first image unit; A selection module, configured to select, when the number of the reference image units is greater than or equal to 2, a reference image unit from each of the reference image units according to the first information as a first reference image unit; The second determining module is used to determine the intra-frame mode of the first reference image unit as the intra-frame mode of the first image unit.

10. A computer device, It is characterized in that The computer device comprises a processor and a memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement the method according to any one of claims 1 to 8.

12. A computer program product, It is characterized in that The computer program product comprises a computer program, wherein the computer program is stored in a computer-readable storage medium, and a processor reads and executes the computer program from the computer-readable storage medium to implement the method according to any one of claims 1 to 8.

Citation Information

Patent Citations

  • Reference block selection method and device of coding unit, electronic equipment and storage medium

    CN109688407A

  • Video data processing method and device, computer equipment and storage medium

    CN112235582A

  • Method and device for bidirectional inter frame prediction

    CN112567753A

  • Device for encoding moving image, method of controlling the same, and program

    JP2011041014A

  • Image prediction method, apparatus, and codec

    WO2019128716A1