Hash table construction method, video coding method, coder, electronic device and medium
By introducing motion vector information into video encoding, selectively constructing hash tables and regional-level table building for large motion areas, the problem of high complexity in the construction of hash tables in the existing technology is solved, and a more efficient coding process is achieved .
Patent Information
- Application Number
- PCT/CN2024/087328
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-15
- Filing Date
- 2024-04-11
- Publication Date
- 2024-11-21
AI Technical Summary
The existing hash table construction method has high complexity in the video encoding process, resulting in low encoding efficiency. Especially when facing SSC video, the granularity construction method of the whole frame by pixel increases computing power consumption and time complexity.
By introducing motion vector information, a hash table is selectively constructed based on preset motion vector thresholds, and a region-level table is constructed for large motion state areas. Combined with the frame-level table construction method of the entire frame-by-pixel frame, the hash table construction time and Coding complexity.
It shortens the hash table construction time, reduces the coding complexity, and maintains the improvement of compression efficiency, which is suitable for the encoding requirements of SSC videos.
Smart Images

Figure CN2024087328_21112024_PF_FP_ABST
Abstract
Description
Hash table construction and video encoding method, encoder, electronic device and medium
[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on May 15, 2023, with application number 202310556020.6 and invention name “Hash table construction and video encoding method, encoder, electronic device and medium”, the entire contents of which are incorporated by reference into this application. Technical Field
[0002] The present application relates to the field of video processing technology, and in particular to a hash table construction method, a video encoding method, a corresponding encoder, a corresponding electronic device, and a corresponding computer-readable storage medium. Background Art
[0003] With the rise of video services such as short videos, video conferencing, live streaming, and online video on demand, video has gradually replaced some traditional text and image formats, becoming a primary means of entertainment, information acquisition, work, and study. Compared to ordinary natural videos, SSC (Screen Content Coding) videos have the characteristics of less noise, weaker spatial texture continuity, and a wider range of temporal motion.
[0004] HashMe is one of the efficient coding tools for SCC videos. During the encoding process, the encoder calculates the features of the current coding block when performing motion search. It usually relies on the index of its reference frame hash table to find the coding block with the same feature value as the reference block of the current coding block. However, the method of constructing a hash table based on extracting the features of the coded blocks during the encoding process easily increases the coding complexity of HashMe.
[0005] Summary of the Invention
[0006] In view of the above problems, embodiments of the present application are proposed to provide a hash table construction method, a video encoding method, a corresponding encoder, a corresponding electronic device, and a corresponding computer-readable storage medium that overcome the above problems or at least partially solve the above problems.
[0007] The present invention discloses a method for constructing a hash table, which includes:
[0008] Acquire a target video frame, and acquire motion vector information contained in a preset selected area of the target video frame;
[0009] In response to the motion vector information being greater than a preset motion vector threshold, acquiring pixel block information of a target pixel block within a preset selected area of the target video frame;
[0010] A hash table of a preset selected area of the target video frame is constructed according to the pixel block information.
[0011] Optionally, in response to the motion vector information being greater than a preset motion vector threshold, acquiring pixel block information of a target pixel block within a preset selected area of the target video frame includes:
[0012] Obtaining a pixel window; the pixel window is determined based on a preset pixel block size;
[0013] In response to the motion vector information being greater than a preset motion vector threshold, the pixel window is used to collect pixel block information of a target pixel block within a preset selected area of the target video frame.
[0014] Optionally, the target video frame includes a plurality of pieces of motion vector information within a preset selected area; and in response to the motion vector information being greater than a preset motion vector threshold, collecting pixel block information of a target pixel block within the preset selected area of the target video frame using the pixel window includes:
[0015] In response to at least one piece of motion vector information being greater than a preset motion vector threshold, pixel block information of a target pixel block within a preset selected area of the target video frame is collected using the pixel window.
[0016] Optionally, the motion vector information includes first-dimensional motion vector information and second-dimensional motion vector information, and the preset motion vector threshold includes a preset first-dimensional motion vector threshold and a preset second-dimensional motion vector threshold;
[0017] In response to the existence of at least one motion vector information greater than a preset motion vector threshold, using the pixel window to collect pixel block information of a target pixel block within a preset selected area of the target video frame includes:
[0018] In response to the existence of at least one motion vector information having a first dimension motion vector information greater than a preset first dimension motion vector threshold, or the existence of at least one motion vector information having a second dimension motion vector information greater than a preset second dimension motion vector threshold, the pixel window is used to collect pixel block information of the target pixel block within a preset selected area of the target video frame.
[0019] Optionally, the using the pixel window to collect pixel block information of a target pixel block within a preset selected area of the target video frame includes:
[0020] Determining a pixel block within a pixel window of the target video frame as a target pixel block, and collecting pixel block information of the target pixel block;
[0021] Within the preset selected area of the target video frame, the pixel window is continuously slid according to a preset horizontal step size and a preset vertical step size, and pixel blocks within the pixel window of the target video frame are continuously determined as target pixel blocks, and pixel block information of the target pixel blocks is collected until pixel block information of each target pixel block contained in the preset selected area of the target video frame is collected.
[0022] Optionally, also include:
[0023] Based on the size of the preset selected area, the next preset selected area is continuously determined in the target video frame, and motion vector information contained in the target video frame within the range of the next preset selected area is obtained. In response to the newly obtained motion vector information being greater than a preset motion vector threshold, pixel block information of the target pixel block in the next preset selected area is obtained, and a hash table of the next preset selected area is constructed based on the newly obtained pixel block information, until the determined preset selected area covers the target video frame.
[0024] Optionally, the target pixel block includes at least one target pixel block, and the pixel block information includes a coding feature value and pixel block coordinate information; and constructing a hash table of a preset selected area of the target video frame according to the pixel block information includes:
[0025] Extracting a coding feature value of at least one target pixel block, and obtaining pixel block coordinate information of at least one target pixel block;
[0026] A hash table of a preset selected area of the target video frame is constructed using the encoded feature value of each target pixel block as an index and the pixel block coordinate information of each target pixel block as content; wherein the hash table of the preset selected area of at least one target video frame constitutes the hash table of the target video frame.
[0027] Optionally, also include:
[0028] In response to the target video frame being a scene switch frame reference frame, a pixel window is obtained, and pixel block information of pixel blocks included in the scene switch frame reference frame is collected using the pixel window, and a hash table of the scene switch frame reference frame is constructed based on the pixel block information of the pixel blocks included in the scene switch frame reference frame.
[0029] Optionally, also include:
[0030] Obtaining a video frame to be encoded, and encoding the video frame to be encoded using a hash table of a preset selected area of the target video frame; the video frame to be encoded is a video frame that refers to the target video frame;
[0031] And / or, the video frame to be encoded is encoded using the hash table of the preset selected area of the target video frame and the hash table of the scene switch frame reference frame.
[0032] The present application also discloses a video encoding method, which includes:
[0033] In response to an input instruction for a video file, obtaining a hash table of a to-be-encoded video frame and a target video frame of the video file, wherein the to-be-encoded video frame is a video frame that references the target video frame; wherein the hash table of the target video frame includes a hash table of a preset selected area of the target video frame, the hash table of the preset selected area being constructed based on pixel block information of a target pixel block within the preset selected area of the target video frame when the motion vector information of the target video frame is greater than a preset motion vector threshold;
[0034] The to-be-encoded video frame is encoded using the hash table of the target video frame.
[0035] Optionally, the hash table of the target video frame further includes a hash table of a scene switching frame reference frame, and the hash table of the scene switching frame reference frame is constructed based on pixel block information of pixel blocks included in the scene switching frame reference frame.
[0036] The present application also discloses a hash table construction device, which includes:
[0037] A motion vector information acquisition module, configured to acquire a target video frame and acquire motion vector information contained in a preset selected area of the target video frame;
[0038] a pixel block information acquisition module, configured to acquire pixel block information of a target pixel block in a preset selected area of the target video frame in response to the motion vector information being greater than a preset motion vector threshold;
[0039] The first hash table construction module constructs a hash table of a preset selected area of the target video frame according to the pixel block information.
[0040] The present application also discloses a video encoding device, comprising:
[0041] a hash table acquisition module, in response to an input instruction of a video file, acquiring hash tables of a to-be-encoded video frame and a target video frame of the video file, wherein the to-be-encoded video frame is a video frame that references the target video frame; the hash table of the target video frame includes a hash table of a preset selected area of the target video frame, the hash table of the preset selected area being constructed based on pixel block information of a target pixel block within the preset selected area of the target video frame when the motion vector information of the target video frame is greater than a preset motion vector threshold;
[0042] The video encoding module encodes the to-be-encoded video frame using the hash table of the target video frame.
[0043] An embodiment of the present application further discloses an encoder, which includes a computer program executed in the encoder, and when the computer program is executed by the encoder, implements any one of the hash table construction methods or any one of the video encoding methods.
[0044] An embodiment of the present application also discloses an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, any one of the hash table construction methods or any one of the video encoding methods is implemented.
[0045] An embodiment of the present application further discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, any one of the hash table construction methods or any one of the video encoding methods is implemented.
[0046] The embodiments of the present application include the following advantages:
[0047] In an embodiment of the present application, motion vector information contained in a preset selected area of a target video frame is obtained. Then, when the obtained motion vector information is greater than a preset motion vector threshold, pixel block information of a target pixel block within the preset selected area is obtained, and a hash table for the preset selected area is constructed. Motion vector information is introduced into the hash table construction process, and a hash table is constructed for the preset selected area of the target video frame based on motion vector information greater than the preset motion vector threshold. This achieves selective partitioning of the hash table, shortens the hash table construction time, and thereby reduces time complexity and encoding complexity. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] FIG1 is a flowchart of a method for constructing a hash table according to an embodiment of the present invention;
[0049] FIG2 is a flowchart of another embodiment of a hash table construction method of the present application;
[0050] FIG3 is a flowchart of a video encoding method according to an embodiment of the present application;
[0051] FIG4 is a schematic diagram of a framework of a video encoder provided in an embodiment of the present application;
[0052] FIG5 is a schematic diagram of an application scenario of hash table construction and video encoding provided by an embodiment of the present application;
[0053] FIG6 is a block diagram of a hash table construction device according to an embodiment of the present invention;
[0054] FIG7 is a structural block diagram of an embodiment of a video encoding device of the present application. DETAILED DESCRIPTION
[0055] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.
[0056] To facilitate those skilled in the art to understand the present application, the following explains the terms or nouns involved in the following embodiments of the present application:
[0057] SCC: Screen Content Coding, screen content coding, refers to the compression coding and algorithm redesign specifically for screen content based on its inherent characteristics and the differences in characteristics from natural scene content.
[0058] H.265 / HEVC: Play High Efficiency Video Coding, high-efficiency video coding, is an international video coding standard.
[0059] HashMe: A coding technology of HEVC, it is one of the efficient coding tools for SCC videos.
[0060] IBC (Intra Block Copy) is a HEVC coding technology. A traditional video coding technology that is very similar to IBC is Motion Compensation (MC) during inter-frame coding. Both technologies are based on block matching to obtain predicted pixel values.
[0061] Palette: A HEVC encoding technology, mainly used for screen content compression encoding.
[0062] Transform Skip: Transform skip mode is a technology introduced in HEVC, which has a significant effect on video sequences with slow scene changes.
[0063] RDCost: Rate Distortion cost refers to the data rate distortion cost, which mainly comprehensively considers the impact of encoding tools on bit rate and quality.
[0064] CU: Coding Unit, which is the coding unit.
[0065] QP: Quantization Parameter, which indicates the quantization step size that can be derived.
[0066] Motion search: It is mainly the process of finding a matching block on the reference frame that has the smallest difference with the current block.
[0067] Motion estimation: refers to the technology of extracting motion information from video sequences.
[0068] MV: Motion Vector, which refers to the vector connecting the current block and the matching block, is used to indicate the direction of movement of the object in the video.
[0069] BDrate: The Bitrate, or Bientgaard-Delta rate, is used to evaluate the rate-distortion performance of different video encoders. In this embodiment, it is a measure of the effectiveness of video coding algorithms, indicating changes in bitrate while maintaining the same reconstruction quality. Negative numbers generally indicate bitrate savings and improved coding efficiency.
[0070] Skip: Skip mode is a special merge mode that saves bit rate.
[0071] Merge: Merge mode, which mainly uses the vector information of adjacent blocks in the time domain and spatial domain to predict the vector information of the current block.
[0072] AMVP: Advanced motion vector prediction, an advanced motion vector prediction technology, mainly uses the correlation of motion vectors in space and time to establish spatial candidate lists and temporal candidate lists respectively, and then selects MVP from the candidate lists.
[0073] Intra: Internal mode, which is mainly based on the technology of cutting macroblocks into sub-blocks of corresponding sizes to the mode, and using the sub-blocks to predict the pixels of the currently encoded adjacent blocks.
[0074] With the rise of video services such as short videos, video conferencing, live broadcasts, and online on-demand, videos have gradually replaced some traditional text and image methods, becoming one of the main ways for people to entertain, obtain information, work, and study in their lives. In the future, video services will continue to develop rapidly in terms of scale and video format.
[0075] Compared to traditional graphics and text, videos offer more intuitive and engaging content presentations. With the rapid development of video services, the amount of data generated by video transmission and storage is significant, placing greater demands on transmission bandwidth and storage space. Video encoding, a key component of video compression, ensures minimal degradation of video file size while maintaining visual quality. This reduces enterprise bandwidth costs and provides users with a clearer, smoother visual experience. Video encoding is performed by video encoders.
[0076] In video services, from the perspective of video content, they can be divided into SSC videos and ordinary natural videos. Compared with ordinary natural videos, SSC videos have less noise, weaker spatial texture continuity, and a larger temporal motion range. As a result, the encoder strategies for encoding SCC and natural videos are very different. Some encoding technologies are mainly aimed at SCC videos, such as IBC, Palette, Transform Skip, HashMe, etc. The rise of video conferencing and video teaching has led to a gradual increase in the demand for SCC videos, and naturally, the demand for improved SCC video compression efficiency is also increasing.
[0077] Among the coding technologies for SCC video, HashMe is an example of an efficient coding tool for SCC video. Its main feature is that the encoder extracts the features of the coded block during the encoding process and then uses these features as the subscript index of a hash table. The coordinates of the coded block are used as the content pointed to by the index, allowing one index to point to multiple contents, thus constructing a hash table. During motion search, the encoder calculates the features of the current coded block by indexing its reference frame hash table to find a coded block with the same feature value, and uses this coded block as the reference block for the current coded block. In other words, HashMe's encoding method for SCC video relies on the construction of hash tables.
[0078] However, among the technologies related to hash table construction, HashMe in the HM reference software uses a method of constructing a hash table based on the granularity of each pixel of the entire frame. Under this hash table construction method, the hash table is constructed in the dimension of each frame. The construction of the hash table consumes a lot of computing power. Although HashMe can improve compression efficiency, it still leads to high time complexity and encoding complexity, making it unsuitable for practical applications.
[0079] In an embodiment of the present application, a hash table is selectively constructed based on an estimation of video motion. Specifically, based on the introduction of motion vector information, the state of a preset selected area in a target video frame can be determined based on a preset motion vector threshold. A hash table is selectively constructed for a preset selected area in a large motion state, and the frame-level table building method based on the entire frame pixel-by-pixel of HashMe is replaced with a region-level table building method for areas with large motion states. This reduces the complexity of HashMe while not affecting the compression efficiency gain brought by HashMe. In addition, when the target video frame hits a scene switch, that is, the target video frame belongs to a scene switch frame, a frame-level table is supplemented for it. By combining the frame-level table building method based on the entire frame pixel-by-pixel and the region-level table building method for areas with large motion states, the utilization rate of HashMe is improved as much as possible, and the complexity of HashMe is reduced as much as possible while ensuring its performance gain.
[0080] 1 , a flowchart of an embodiment of a hash table construction method of the present application is shown, which may specifically include the following steps:
[0081] Step 101: obtaining a target video frame and obtaining motion vector information contained in a preset selected area of the target video frame;
[0082] The target video frame refers to an encoded video frame, which may include several encoded pixel blocks (referred to as encoded blocks). The features of the encoded blocks may be used as reference blocks for the blocks to be encoded contained in the frame to be encoded, so as to perform motion estimation during the encoding process of the blocks to be encoded based on the reference blocks, thereby facilitating the prediction and encoding of the blocks to be encoded.
[0083] For SCC videos, areas experiencing high motion contain textures where objects are moving rapidly. Traditional coding techniques, such as traditional Skip, Merge, Amvp, and Intra modes, make it difficult to match ideal reference blocks. In this case, a hash table can be used to index reference blocks to find reference blocks. This means that the determination of reference blocks relies on a hash table constructed for the encoded blocks. However, most areas of a video frame typically do not experience high motion continuously. In particular, for SCC sequences, high motion often occurs in localized areas and lasts for very short periods of time. To reduce HashMe complexity while maintaining the compression efficiency gains achieved by HashMe, a region-by-region hash table can be constructed for the target video frame, taking into account the characteristics of localized areas experiencing high motion in the SCC sequence.
[0084] The region-based hash table construction method can be manifested as selective hash table construction. Specifically, the hash table can be selectively constructed based on an estimate of video motion. Specifically, this can be manifested by introducing motion vector information, determining the state of a preset selected region in the target video frame based on a preset motion vector threshold, and selectively constructing a hash table for the preset selected region experiencing significant motion.
[0085] In one embodiment of the present application, a hash table is constructed for the target video frame by region. The analysis and construction of the hash table can be started by selecting a region. The selected region can be a preset selected region. The range size or size of the preset selected region is generally smaller than the threshold of the set coding unit size, that is, it can be determined based on the coding unit size adopted by the encoder when encoding. For example, in an encoder adopting the H.265 / HEVC coding standard, the range size or size of the preset selected region is generally smaller than 64px×64px.
[0086] Regarding the introduction of motion vector information, in practical applications, the motion vector information contained in a preset selected area of the target video frame can be obtained so as to determine the motion state of the current local area based on the motion vector information of the preset selected area. The preset selected area may include multiple pixel blocks, and the number of motion vector information contained in the preset selected area may be multiple, that is, a single preset selected area may include multiple motion vector information corresponding to multiple pixel blocks, wherein the motion vector of the preset selected area may refer to a vector connecting the current block and the matching block, the current block may refer to the current pixel block in the current preset selected area, and the matching block is the pixel block used for reference found by motion search on the reference frame during encoding of the current pixel block, and its motion vector information can be mainly used to measure the motion direction and motion distance of the object content related to the current pixel block, that is, the motion vector information may include the motion distance of the object content related to the current pixel block in a certain direction.
[0087] It should be noted that the range size or dimension of a single preset selected area is difficult to cover the entire target video frame. In the embodiment of the present application, the position of the next preset selected area in the target video frame can be determined according to the range size of the preset selected area, or the current preset selected area can be moved on the target video frame, and the motion state of the local area corresponding to the preset selected area can be continued to be determined until the motion state of the area included in the entire target video frame is determined.
[0088] Step 102 , in response to the motion vector information being greater than a preset motion vector threshold, obtaining pixel block information of a target pixel block within a preset selected area of a target video frame;
[0089] After determining the motion state of the current local area based on the motion vector information of the preset selected area, in order to facilitate the construction of a hash table for areas in a large motion state, pixel block information of the target pixel block in the preset selected area of the target video frame can be obtained.
[0090] The motion state of the current local area can be determined mainly based on the motion vector information and the preset motion vector threshold. Specifically, when the motion vector information is greater than the preset motion vector threshold, it can be determined that the motion state of the current local area (i.e., the current preset selected area) is in a large motion state, so as to selectively construct a hash table for the preset selected area in a large motion state based on the acquisition of pixel block information in this case.
[0091] It should be noted that the preset threshold can be a threshold pre-set according to an experimental value or an empirical value, or it can be a threshold adaptively derived from historical coding information. The derived threshold can improve the robustness of the algorithm. The embodiment of the present application does not limit the method of determining the preset threshold; and the motion vector information can be generated in the original process when the target video frame is video encoded. At this time, based on the position and range size of the preset selected area in the target video frame, the motion vector information contained in the preset selected area can be obtained. The embodiment of the present application does not limit the original process of video encoding the target video frame.
[0092] Among them, the large motion state can refer to the state of rapid motion of an object, which can be specifically determined based on motion vector information. The motion vector information can be used to represent the current motion vector direction and motion distance. Usually, when the motion distance in the current motion vector direction is greater than the motion distance threshold set in the current motion vector direction, it is considered that the situation meets the conditions of being in a large motion state.
[0093] In practical applications, pixel block information of the target pixel block can be acquired based on a pixel window. After determining that the motion state of the current local area is in a large motion state, the aforementioned set pixel window can be used to acquire corresponding pixel block information of the target pixel block contained in the current local area.
[0094] Specifically, the pixel window can generally be determined based on a preset pixel block size. The preset pixel block size can refer to the size of a single pixel block of the target video frame, or can also refer to the size of multiple pixel blocks. This is not limited in the embodiments of the present application. For example, the size of the pixel block can be expressed as w×h. In this case, the size of the pixel window can be expressed as w×h, where w represents the number of pixels corresponding to the width of the pixel block, and h represents the number of pixels corresponding to the height of the pixel block.
[0095] It should be noted that, for the collection of pixel block information of a target pixel block in a preset selected area, the target pixel block of a preset selected area refers to the various pixel blocks contained in the preset selected area. At this time, the target pixel blocks contained in a single preset selected area may include multiple pixel blocks, and the size of the pixel collection window is determined based on the size of a single pixel block. The range size or size of the preset selected area is determined based on the coding unit size adopted by the encoder when encoding. The range size or size of the preset selected area is larger than the size of the pixel collection window. In order to ensure the complete collection of pixel block information of each pixel block contained in the preset selected area, the pixel window can be moved within the preset selected area, and the pixel block information of the target pixel block located in the pixel window is continuously collected based on the moved pixel window until the pixel window is completely moved within the preset selected area, and the collection of pixel block information of each target pixel block contained in the preset selected area is completed.
[0096] Step 103: construct a hash table of a preset selected area of the target video frame according to the pixel block information.
[0097] In one embodiment of the present application, after obtaining pixel block information of a preset selected area in a large motion state, a hash table of the preset selected area of the target video frame can be constructed based on the pixel block information, so as to provide reference block index assistance for subsequent frames to be encoded based on the constructed hash table, find an ideal reference block to perform motion estimation in the encoding process of the block to be encoded based on the reference block, and complete the prediction and encoding of the block to be encoded.
[0098] The preset selected areas in a large motion state usually have the characteristics of local areas in the SCC video. In the process of constructing the hash table, the table is mainly constructed selectively according to the preset selected areas. That is, the hash table of the target video frame can be constructed based on the hash tables of the preset selected areas of multiple target video frames, and the hash table used to construct the target video frame includes the hash table of the preset selected areas in a large motion state. Since the preset selected areas not in a large motion state can be well predicted and encoded through traditional modes such as Skip, Merge, Amvp, and Intra, the hash table used to construct the target video frame does not include the hash table of the preset selected areas not in a large motion state.
[0099] In practical applications, the process of constructing a hash table can be selectively constructed according to a preset selected area. After the hash table is constructed for the target pixel block contained in a preset selected area in a large motion state, the range size or size of a single preset selected area is difficult to cover the entire target video frame. In order to completely traverse and analyze each local area contained in the target video frame, the next local area of the target video frame can be analyzed at this time.
[0100] That is, the next preset selected area can be continuously determined in the target video frame according to the preset area range of the preset selected area, and the motion vector information contained in the target video frame within the next preset area range can be obtained. Based on the motion vector information and the preset motion vector threshold, the pixel block information of the target pixel block within the preset selected area of the target video frame can be obtained, and a hash table of the preset selected area of the target video frame can be constructed based on the pixel block information until the determined preset selected area completely covers the target video frame. In this case, there is no pixel overlap between the areas, that is, there are no identical pixel blocks between the areas, thereby avoiding analysis of the same pixel blocks in the target video frame during the analysis process.
[0101] The pixel block information is the pixel block information for the target pixel block. The target pixel block belongs to the target video frame. The target video frame refers to the encoded video frame. The target pixel block is the encoded block contained in the target video frame. When constructing a hash table for the encoded block, the encoding feature value of the target pixel block at each position can be used as an index, and the pixel block coordinate information of the target pixel block at each position can be used as the content pointed to by the corresponding index, so that one index can point to multiple contents to construct a hash table. The content specifically refers to the position of the texture block in the image. The texture content at different positions may be consistent, so as to provide index assistance for the reference block when encoding the block to be encoded later.
[0102] It should be noted that the hash table construction method proposed in the embodiment of the present application can be applied to an encoder for SCC video. The encoder can be an encoder developed based on the H.265 / HEVC coding standard, which can use the constructed hash table to encode the SCC video.
[0103] In an embodiment of the present application, motion vector information contained in a preset selected area of a target video frame is obtained. Then, when the obtained motion vector information is greater than a preset motion vector threshold, pixel block information of a target pixel block within the preset selected area is obtained, and a hash table for the preset selected area is constructed. Motion vector information is introduced into the hash table construction process, and a hash table is constructed for the preset selected area of the target video frame based on motion vector information greater than the preset motion vector threshold. This achieves selective partitioning of the hash table, shortens the hash table construction time, and thereby reduces time complexity and encoding complexity.
[0104] 2 , there is shown a flowchart of another embodiment of a hash table construction method of the present application, which may specifically include the following steps:
[0105] Step 201: In response to at least one motion vector information being greater than a preset motion vector threshold, pixel block information of a target pixel block within a preset selected area of a target video frame is collected using a pixel window;
[0106] In an embodiment of the present application, a hash table can be selectively constructed based on an estimation of video motion. Specifically, based on the introduction of motion vector information, the state of a preset selected area in a target video frame can be determined based on a preset motion vector threshold. A hash table can be selectively constructed for the preset selected area in a large motion state, and the frame-level table construction method of HashMe based on pixel-by-pixel of the entire frame can be replaced with a region-level table construction method for areas in a large motion state. This reduces the complexity of HashMe without affecting the compression efficiency gain brought by HashMe.
[0107] Specifically, large motions often occur in local areas and last for a short time. Therefore, a hash table can be constructed for the target video frame by region.
[0108] The region-based hash table construction method can be manifested as selective hash table construction. Specifically, the hash table can be selectively constructed based on an estimate of video motion. Specifically, this can be manifested by introducing motion vector information, determining the state of a preset selected region in the target video frame based on a preset motion vector threshold, and selectively constructing a hash table for the preset selected region experiencing significant motion.
[0109] The large motion state may refer to the state of rapid motion of an object, which may be specifically determined based on motion vector information. The motion vector information may be used to indicate the current motion vector direction and motion distance. Usually, when the motion distance in the current motion vector direction is greater than the motion distance threshold set in the current motion vector direction, it may be considered that the situation meets the conditions for being in a large motion state.
[0110] In practical applications, the motion vector information contained in the preset selected area of the target video frame may generally include multiple pieces of motion vector information.
[0111] In one embodiment of the present application, the number of motion vector information contained in a single preset selected area can be multiple. When there is at least one motion vector information greater than a preset motion vector threshold, a pixel window can be used to collect pixel block information of the target pixel block within the preset selected area of the target video frame.
[0112] Specifically, the motion vector information may include first-dimensional motion vector information and second-dimensional motion vector information, and the preset motion vector threshold may include a preset first-dimensional motion vector threshold and a preset second-dimensional motion vector threshold. At this time, in response to the existence of at least one motion vector information having first-dimensional motion vector information greater than the preset first-dimensional motion vector threshold, or the existence of at least one motion vector information having second-dimensional motion vector information greater than the preset second-dimensional motion vector threshold, it can be determined that there is large motion in the video frame within the range of the preset selected area, that is, the local area corresponding to the preset selected area is in a large motion state. At this time, a pixel window can be used to collect pixel block information of the target pixel block within the preset selected area of the target video frame.
[0113] Exemplarily, motion vector information may refer to MV information, which is a two-dimensional vector that can be written in the mathematical form of (x, y), where x is the horizontal coordinate and y is the vertical coordinate. In the MV information, (x, y) can be used to express the direction and movement distance of the current MV. The threshold for measuring the magnitude of MV movement, i.e., the preset motion vector threshold, can also be a two-dimensional vector and can be expressed using (x, y). That is, the first-dimensional motion vector information contained in the MV information can be represented by mv.x, which represents the current movement distance in the x direction. The second-dimensional motion vector information contained in the MV information can be represented by mv.y, which represents the current movement distance in the y direction. The preset first-dimensional motion vector threshold can be represented by x_thr, which represents the threshold for the movement distance in the x direction. The preset second-dimensional motion vector threshold can be represented by y_thr, which represents the threshold for the movement distance in the y direction.
[0114] When analyzing the MV information contained in a preset selected area, assuming there are multiple MVs, if one of the multiple MVs satisfies mv.x > x_thr, or if one of the multiple MVs satisfies mv.y > y_thr, this indicates that the current MV has moved a significant distance in at least one of the x and y directions, known as large motion. Therefore, the current preset selected area can be determined to be in a large motion state and belongs to a large motion region. Large motion regions are characterized by rapid object movement in the video, and contain texture regions with rapidly moving objects. For texture regions with rapidly moving objects, traditional inter-frame prediction techniques are limited by complexity and find it difficult to match an ideal reference block. In this case, using a hash table to index the reference block makes it easier to find the appropriate reference block, thereby improving compression efficiency during video encoding.
[0115] In one embodiment of the present application, after determining that the current preset selected area belongs to a large motion area, a pixel window may be used to collect pixel block information of a target pixel block within the preset selected area of the target video frame to construct a hash table.
[0116] The pixel block information of the target pixel block can be acquired based on the pixel window. The pixel window can generally be determined based on a preset pixel block size. The preset pixel block size can refer to the size of a single pixel block of the target video frame, or the size of multiple pixel blocks. The embodiment of the present application is not limited to this. For example, the size of the pixel block can be expressed as w×h. In this case, the size of the pixel window can be expressed as w×h, where w is used to represent the number of pixels corresponding to the width of the pixel block, and h is used to represent the number of pixels corresponding to the height of the pixel block.
[0117] For the collection of pixel block information of a target pixel block in a preset selected area, the target pixel block of a preset selected area refers to the various pixel blocks contained in the preset selected area. At this time, the target pixel blocks contained in a single preset selected area may include multiple pixel blocks, and the size of the pixel collection window is determined based on the size of the single pixel block. The range size or size of the preset selected area is determined based on the coding unit size adopted by the encoder when encoding. The range size or size of the preset selected area is larger than the size of the pixel collection window. In order to ensure the complete collection of pixel block information of each pixel block contained in the preset selected area, the pixel window can be moved within the preset selected area, and the pixel block information of the target pixel block located in the pixel window is continuously collected based on the moved pixel window until the pixel window is completely moved within the preset selected area, and the collection of pixel block information of each target pixel block contained in the preset selected area in a large motion state is completed.
[0118] Specifically, a pixel block within a pixel window of a target video frame can be determined as a target pixel block, and pixel block information of the target pixel block can be collected. Then, the pixel window can be continuously slid within a preset area of the target video frame according to a preset horizontal step size and a preset vertical step size, and pixel blocks within the pixel window of the target video frame can be continuously determined as target pixel blocks, and pixel block information of the target pixel blocks can be collected, until pixel block information of each target pixel block contained in a preset selected area of the target video frame is collected. In actual applications, the movement of the pixel window within the preset selected area can be specifically manifested as traversing the pixels contained in the preset selected area based on the upper left corner pixel point of the pixel window.
[0119] For example, for a preset selected area, a pixel window of w×h can be used to slide on the current preset selected area according to the preset horizontal step size sh and the preset vertical step size sv. The pixel block information of the pixel blocks at each position in the current preset selected area is collected and gradually added to the hash table of the target video frame until the current preset selected area is traversed.
[0120] In some embodiments of the present application, when analyzing the MV information contained in the preset selected area, there are usually multiple MV information, and none of the multiple MV information satisfies mv.x>x_thr or mv.y>y_thr, that is, the MV information contained in the current preset selected area does not satisfy the large motion state. At this time, the object in the video is in a nearly static state, such as a static background area captured by a static camera. Traditional inter-frame prediction technology can predict and encode this situation, that is, the preset selected area that is not in a large motion state can be well predicted and encoded through traditional Skip, Merge, Amvp, Intra and other modes. There is no need to match the reference block through the hash method, and the hash table construction operation can be omitted for the pixel blocks of the current preset selected area. The hash table construction process can be skipped to maximize the utilization rate of HashMe.
[0121] Step 202: construct a hash table of at least one preset selected area of the target video frame according to the pixel block information, and obtain a hash table of the target video frame according to the hash table of the at least one preset selected area;
[0122] In one embodiment of the present application, after obtaining pixel block information of a preset selected area in a large motion state, a hash table of the preset selected area of the target video frame can be constructed based on the pixel block information, so as to provide reference block index assistance for subsequent frames to be encoded based on the constructed hash table, find an ideal reference block to perform motion estimation in the encoding process of the block to be encoded based on the reference block, and complete the prediction and encoding of the block to be encoded.
[0123] The preset selected areas in a large motion state usually have the characteristics of local areas in the SCC video. In the process of constructing the hash table, the table is mainly constructed selectively according to the preset selected areas. That is, the hash table of the target video frame can be constructed based on the hash tables of the preset selected areas of multiple target video frames, and the hash table used to construct the target video frame includes the hash table of the preset selected areas in a large motion state. Since the preset selected areas not in a large motion state can be well predicted and encoded through traditional modes such as Skip, Merge, Amvp, and Intra, the hash table used to construct the target video frame does not include the hash table of the preset selected areas not in a large motion state.
[0124] In practical applications, the process of constructing a hash table can be selectively constructed according to a preset selected area. After the hash table is constructed for the target pixel block contained in a preset selected area in a large motion state, the range size or size of a single preset selected area is difficult to cover the entire target video frame. In order to completely traverse and analyze each local area contained in the target video frame, the next local area of the target video frame can be analyzed at this time.
[0125] That is, the next preset selected area can be continuously determined in the target video frame according to the preset area range of the preset selected area, and the motion vector information contained in the target video frame within the next preset area range can be obtained. Based on the motion vector information and the preset motion vector threshold, the pixel block information of the target pixel block within the preset selected area of the target video frame can be obtained, and a hash table of the preset selected area of the target video frame can be constructed based on the pixel block information until the determined preset selected area completely covers the target video frame. In this case, there is no pixel overlap between the areas, that is, there are no identical pixel blocks between the areas, thereby avoiding analysis of the same pixel blocks in the target video frame during the analysis process.
[0126] The pixel block information is pixel block information for a target pixel block. The target pixel block belongs to a target video frame. The target video frame refers to an encoded video frame. The target pixel block is an encoded block contained in the target video frame.
[0127] The target pixel block may include at least one target pixel block at a different position, and the pixel block information may include a coding feature value and pixel block coordinate information. When constructing a hash table for the encoded block, the coding feature value of at least one target pixel block may be extracted, and the pixel block coordinate information of at least one target pixel block may be obtained. The coding feature value of the target pixel block at each position is used as an index, and the pixel block coordinate information of the target pixel block at each position is used as the content pointed to by the corresponding index. The pixel block coordinate information of each target pixel block may refer to the coordinates (x, y) of the upper left corner pixel point of the pixel block. The content specifically refers to the position of the texture block in the image. The texture content at different positions may be consistent, so that one index can point to multiple contents. A hash table of a preset selected area of the target video frame is constructed to provide index assistance for the reference block when encoding the block to be encoded subsequently.
[0128] In a specific implementation, the encoded eigenvalue refers to a numerical value that can represent a texture block with the highest possible specificity. As an example, the specific way of generating eigenvalues may include, for example, generating eigenvalues based on the pixels and / or gradients of each pixel block, but the eigenvalues generated in this way have low specificity, that is, the eigenvalues generated by the contents of multiple different textures will be the same, resulting in the inability to efficiently judge the similarity of the two pixel blocks by comparing the eigenvalues of the two pixel blocks; as another example, it may include a eigenvalue generation method such as a CRC cyclic redundancy check code, which has high specificity, but has the disadvantage of matching two identical pixel blocks, but cannot match two pixel blocks that are not exactly the same but highly similar. It should be noted that the embodiment of the present application does not limit the specific way of generating eigenvalues.
[0129] Step 203: In response to the target video frame being a scene switch frame reference frame, pixel block information of pixel blocks included in the scene switch frame reference frame is collected using a pixel window, and a hash table of the scene switch frame reference frame is constructed based on the pixel block information of the pixel blocks included in the scene switch frame reference frame.
[0130] In some embodiments of the present application, when the target video frame is a scene switching frame reference frame, it manifests as a phenomenon similar to turning pages of a PPT or quickly moving up a word document. At this time, its reference blocks will be very different. In this case, traditional inter-frame prediction technology is difficult to match ideal reference blocks within complexity limits for most coding blocks. It can be supplemented by frame-level table building. By combining a frame-level table building method based on pixel-by-pixel for the entire frame and a region-level table building method for areas with large motion states, the utilization rate of HashMe can be improved as much as possible, and the complexity of HashMe can be reduced as much as possible while ensuring its performance gain.
[0131] It should be noted that the frame-level table building of the supplementary reference frame can help most coding blocks. Since there are usually not many scene switches in a video, its frame-level table building method based on pixel-by-pixel of the entire frame will not add too much complexity.
[0132] In practical applications, it is possible to determine whether the current target video frame hits the scene switch. In response to the target video frame being a scene switch frame reference frame, a pixel window can be used to collect pixel block information of the pixel blocks contained in the scene switch frame reference frame, and then a hash table of the scene switch frame can be constructed based on the pixel block information of the pixel blocks contained in the scene switch frame reference frame.
[0133] The method for determining whether a target video frame hits a scene cut can be performed based on the output information of the video encoding pre-processing analysis module. The video encoding pre-analysis module and the video core encoding module can be modules of the encoder. The process implemented by the video encoding pre-analysis module can be located before the implementation process of the video core encoding module. Its function can be to output information such as the frame-level quantization parameter and frame type of the current frame through analysis of the video frame, and provide it to the core encoding module.
[0134] In a specific implementation, the video encoding pre-analysis module can generate rough MV, intra-frame / inter-frame prediction cost and other information during the analysis process. This information can be used to determine whether the current target video frame is a scene switching frame reference frame. For example, when the inter-frame prediction cost is close to or even higher than the intra-frame prediction cost, it can be said that the current target video frame is likely to be a scene switching frame reference frame.
[0135] In the process of using a pixel window to capture pixel block information of a pixel block included in a scene switching frame reference frame, the pixel window can generally be determined based on a preset pixel block size. The preset pixel block size can refer to the size of a single pixel block of the target video frame, or can also refer to the size of multiple pixel blocks. This embodiment of the present application is not limited to this. For example, the size of the pixel block can be represented by w×h. In this case, the size of the pixel window can be represented as w×h, where w is used to represent the number of pixels corresponding to the width of the pixel block, and h is used to represent the number of pixels corresponding to the height of the pixel block.
[0136] The pixel block information is the pixel block information for the target pixel block, and the target pixel block belongs to the target video frame. The target video frame refers to the encoded video frame, and the target pixel block is the encoded block contained in the target video frame. When constructing a hash table for the encoded block, the encoding feature value of the target pixel block at each position can be used as an index, and the pixel block coordinate information of the target pixel block at each position can be used as the content pointed to by the corresponding index, so that one index can point to multiple contents to construct a hash table. The content specifically refers to the position of the texture block in the image. The texture content at different positions may be consistent, so as to provide index assistance for the reference block when subsequently encoding the block to be encoded that refers to the reference frame of the scene switching frame.
[0137] Specifically, assuming that the pixel window is determined based on the size of a single pixel block of the target video frame, in the process of collecting pixel block information of the entire target video frame based on the pixel window, it is difficult for a single pixel window to cover the entire target video frame. In order to completely collect the pixel block information of the target video frame contained in the target video frame, the pixel window can be moved on the entire target video frame, and the pixel block information of the target pixel block located in the pixel window is continuously collected based on the moved pixel window until the collection of pixel block information of each target pixel block contained in the entire target video frame is completed.
[0138] The frame-level table building method is that the table building is performed on the dimension of each frame. For example, there will be a w×h pixel window for the hash table construction of the w×h block. At this time, it can be slid on the target video frame according to the preset horizontal step size sh and the preset vertical step size sv. The pixel block information of the pixel blocks at various positions in the target video frame is collected and gradually added to the hash table of the target video frame, that is, the hash table of the scene switching frame reference frame, until the scene switching frame reference frame is traversed. It should be noted that the smaller the values of the preset horizontal step size sh and the preset vertical step size sv, the more candidate blocks there are for table building, and the greater the performance help to HashMe. Usually, sh=1 and sv=1 can be set, and this application does not impose any restrictions on this.
[0139] In one embodiment of the present application, when constructing a hash table for an encoded block, the encoding feature value of at least one target pixel block can be extracted, and the pixel block coordinate information of at least one target pixel block can be obtained. The encoding feature value of the target pixel block at each position is used as an index, and the pixel block coordinate information of the target pixel block at each position is used as the content pointed to by the corresponding index. The pixel block coordinate information of each target pixel block may refer to the coordinates (x, y) of the upper left corner pixel point of the pixel block. The content specifically refers to the position of the texture block in the image. The texture content at different positions may be consistent, so that one index can point to multiple contents, and a hash table of the scene switching frame reference frame is constructed to provide index assistance for the reference block when subsequently encoding the block to be encoded that refers to the scene switching frame reference frame.
[0140] In the embodiment of the present application, the utilization rate of HashMe can be improved as much as possible by combining the frame-level table building method based on the pixel-by-pixel of the entire frame and the region-level table building method for the large motion state area, while ensuring its performance gain and reducing the complexity of HashMe as much as possible.
[0141] It should be noted that the combination of the frame-level table building method based on pixel-by-pixel construction of the entire frame and the region-level table building method for large motion state areas can be specifically reflected in the construction of a hash table for the target video frame according to the region-level table building method, and then when it is determined that the current target video frame is a scene switching frame reference frame, the whole frame table building of its reference frame is supplemented, that is, the reference frame for which the whole frame table building is to be supplemented has undergone the aforementioned region-level table building.
[0142] Step 204: Encode the video frame to be encoded using the constructed hash table.
[0143] It should be noted that the target video frame can construct a hash table while being encoded, and the hash table used by the target video frame during the encoding process is the hash table of its reference frame, and the hash table of the target video frame constructed is mainly used for subsequent frames that reference the target video frame. The frame that references the target video frame can be the video frame to be encoded. When the hash table constructed for the target video frame is used, the video frame to be encoded can be obtained.
[0144] The constructed hash table can be a hash table constructed solely based on a frame-level table building method based on a pixel-by-pixel basis for the entire frame, or it can be a hash table constructed by combining a frame-level table building method based on a pixel-by-pixel basis for the entire frame and a region-level table building method for areas with large motion states. In this case, the constructed hash table can be used to encode the video frame to be encoded of the reference target video frame.
[0145] The video frame to be encoded may include blocks to be encoded. During the encoding process of the video frame to be encoded, motion estimation can be performed on the blocks to be encoded based on reference blocks. In one case, a hash table of a preset selected area of a target video frame can be used to encode the video frame to be encoded; in another case, a hash table of a preset selected area of a target video frame and a hash table of a scene switching frame reference frame can be used to encode the video frame to be encoded.
[0146] Specifically, in the process of encoding using a hash table, the same-position block of the next frame can be predicted and encoded based on the constructed hash table. That is, for the current block to be encoded, its reference block can be determined based on the index of the hash table, so that motion estimation can be performed on the block to be encoded based on the determined reference block, thereby achieving prediction and encoding of the current block to be encoded. Motion estimation is specifically manifested in estimating the pixel difference between the encoding block and its reference block. For example, the content indexed by the feature value is a candidate for a reference block, and then a reference block with a small pixel difference with the block to be encoded is determined as an ideal reference block among multiple candidate reference blocks.
[0147] In an embodiment of the present application, motion vector information contained in a preset selected area of a target video frame is obtained. Then, when the obtained motion vector information is greater than a preset motion vector threshold, pixel block information of a target pixel block within the preset selected area is obtained, and a hash table for the preset selected area is constructed. Motion vector information is introduced into the hash table construction process, and a hash table is constructed for the preset selected area of the target video frame when the motion vector information is greater than the preset motion vector threshold. This achieves selective partitioning of the hash table, shortens the hash table construction time, and thereby reduces time complexity and encoding complexity.
[0148] 3 , there is shown a flowchart of another video encoding method embodiment of the present application, which may specifically include the following steps:
[0149] Step 301, in response to an input instruction of a video file, obtaining a hash table of a to-be-encoded video frame and a target video frame of the video file;
[0150] The hash table construction method proposed in the embodiment of the present application can be applied to an encoder for SCC video. The encoder can be an encoder developed based on the H.265 / HEVC coding standard, which can use the constructed hash table to encode the SCC video.
[0151] It should be noted that the target video frame can construct a hash table while being encoded, and the hash table used by the target video frame during the encoding process is the hash table of its reference frame, and the hash table of the target video frame constructed is mainly used for subsequent reference target video frames. The frame of the reference target video frame can be the video frame to be encoded.
[0152] In practical applications, the obtained video file may be a SCC video to be encoded. Specifically, an encoder according to the H.265 / HEVC encoding standard may be used to respond to input instructions of the video file and perform video encoding on the SCC video. Specifically, a video frame to be encoded of the SCC video may be obtained, the video frame to be encoded including a current block to be encoded. At this time, a reference block for performing motion estimation on the current block to be encoded may be obtained. Specifically, a hash table of the target video frame may be obtained, so that the reference block can be determined based on the hash table of the target video frame.
[0153] In the embodiment of the present application, the target video frame refers to an encoded video frame that can be used for motion estimation.
[0154] The obtained hash table of the target video frame may include a hash table of a preset selected area of the target video frame. The hash table of the preset selected area may be constructed based on pixel block information of the target pixel block within the preset selected area of the target video frame according to motion vector information and a preset motion vector threshold.
[0155] The acquired hash table of the target video frame may further include a hash table of a scene switch frame reference frame. The hash table of the scene switch frame reference frame may be constructed based on pixel block information of pixel blocks included in the scene switch frame reference frame.
[0156] For the specific construction process of the hash table of the target video frame, reference may be made to the contents of the hash table construction method described in the aforementioned embodiment of the present application, which will not be elaborated here.
[0157] Step 302: Encode the video frame to be encoded using the hash table of the target video frame.
[0158] In the process of encoding using a hash table, the co-located block of the next frame can be predicted and encoded based on the constructed hash table. That is, for the block to be encoded, its reference block can be determined based on the index of the hash table to achieve prediction and encoding of the current block to be encoded.
[0159] Motion estimation is specifically manifested as estimating the pixel gap between the coding block and its reference block. For example, the content indexed by the feature value is a candidate for a reference block, and then the reference block with the smallest pixel gap with the block to be coded is determined as the ideal reference block among multiple candidate reference blocks.
[0160] It should be noted that, since for the preset selected area that is not in a state of large motion in the target video frame, the ordinary video encoding mode can already well predict and encode the same-position block of the next frame, there is no need to hash the pixel blocks in this preset selected area. Therefore, for the pixel blocks that have not been hashed, when the pixel blocks are used to achieve subsequent prediction and encoding of the same-position block in the next frame, it can be achieved through traditional video encoding technology. That is, there can also be a coding method that combines the use of traditional video coding technology and the use of hash tables constructed based on frame-level table building methods and region-level table building methods to achieve video coding. The specific combined use of video coding is not limited in the embodiments of this application.
[0161] In some embodiments of the present application, an encoder may be provided, which includes a computer program executed in the encoder, and when the computer program is executed by the encoder, it implements any hash table construction method or any video encoding method in the above embodiments.
[0162] Specifically, referring to FIG4 , a schematic diagram of the framework of a video encoder provided in an embodiment of the present application is shown. The video encoder 410 may involve a video encoding pre-analysis module 41 and a video core encoding module 42. The video encoder 410 is an encoder that adopts the H.265 / HEVC encoding standard, which can implement encoding compression operations for SCC videos. The process implemented by the video encoding pre-analysis module 41 may be located before the implementation process of the video core encoding module 42. The function of the video encoding pre-analysis module 41 may be to output the frame-level quantization parameter, frame type, and other information of the current frame through analysis of the video frame, and provide it to the video core encoding module 42. The video core encoding module 42 may be used to construct a hash table and implement video encoding, that is, to use the constructed hash table.
[0163] In an embodiment of the present application, a hash table can be selectively constructed based on the estimation of video motion. Specifically, based on the introduction of motion vector information and the state of a preset selected area in a target video frame based on a preset motion vector threshold, a hash table can be selectively constructed for the preset selected area in a large motion state, and the frame-level table building method based on the whole frame pixel-by-pixel of HashMe is replaced by a region-level table building method for large motion state areas, so that while reducing the complexity of HashMe, the compression efficiency gain brought by HashMe is not affected; in addition, when the target video frame hits the scene switch, that is, the target video frame belongs to the scene switch frame reference frame, a frame-level table is supplemented for it. By combining the frame-level table building method based on the whole frame pixel-by-pixel and the region-level table building method for large motion state areas, the utilization rate of HashMe is improved as much as possible, and the complexity of HashMe is reduced as much as possible while ensuring its performance gain.
[0164] 5 , a schematic diagram of an application scenario of hash table construction and video coding provided in an embodiment of the present application is shown. In the video encoder shown in FIG4 , HashMe is used as an example in related coding technologies for SCC videos. HashMe is one of the efficient coding tools for SCC videos. It mainly involves extracting features of coded blocks during the encoding process, using these features as subscript indexes in a hash table, and using the coordinates of the coded block as the content pointed to by the index, so that one index can point to multiple contents, thereby constructing a hash table. During motion search, the encoder calculates the features of the current coded block, primarily by finding a coded block with the same feature value through an index into its reference frame hash table, and using this coded block as a reference block for the current coded block.
[0165] Specifically, in actual encoding applications of the video encoder shown in FIG4 , there are steps involved in constructing a hash table and using the hash table.
[0166] During hash table construction, hash tables can be selectively constructed for pre-selected regions experiencing significant motion. Furthermore, when a target video frame encounters a scene change, i.e., the target video frame belongs to a scene change frame reference frame, and frame-level table construction can be performed to supplement it. Frame-level table construction is performed on a frame-by-frame basis, while region-level table construction has a smaller dimension than frame-level table construction.
[0167] In practical applications, when a hash table is selectively constructed for a preset selected area in a large motion state, as shown in (A) of FIG5 , the motion vector information contained in the preset selected area of the target video frame can be obtained. The range size or size of the preset selected area is generally smaller than the threshold of the set coding unit size, assuming it is 32×32. At this time, the motion vector information contained in the preset selected area with a range size of 32×32 at a certain position in the target video frame can be obtained. The obtained motion vector information MV information may include mv.x representing the current motion distance in the x direction, and mv.x representing the current motion distance in the y direction. Assume that the threshold value x_thr of the motion distance in the x direction is 100, and the threshold value y_thr of the motion distance in the y direction is 100. If there is one MV information satisfying mv.x>x_thr among multiple MV information, or there is one MV information satisfying mv.y>y_thr among multiple MV information, it can be determined that the preset selected area with a range of 32×32 at the current position of the target video frame is in a large motion state and belongs to a large motion area. At this time, the pixel block information of the target pixel block in the preset selected area of 32×32 can be collected using a pixel window.
[0168] Specifically, the size of the pixel block can be expressed as w×h, so the size of the pixel window can be expressed as w×h, where w is used to represent the number of pixels corresponding to the width of the pixel block, and h is used to represent the number of pixels corresponding to the height of the pixel block. Assuming that w is 8 and h is 8, an 8×8 pixel window can be used to collect pixel block information of the target pixel block contained in the current 8×8 pixel window in a preset selected area of 32×32.
[0169] Since the range size of the 32×32 preset selected area is larger than the size of the 8×8 pixel window, in order to ensure the complete collection of pixel block information of each pixel block included in the preset selected area, the 8×8 pixel window can be slid in the 32×32 preset selected area according to the preset horizontal step size sh=1 and the preset vertical step size sv=1 until all target pixel blocks at various positions included in the 32×32 preset selected area are slid, and the pixel block information collection of the target pixel blocks at various positions is completed.
[0170] In some embodiments of the present application, when the target video frame is a scene switching frame reference frame, it is manifested as a phenomenon similar to turning pages of a PPT or quickly moving up a word document. At this time, its reference blocks will be very different. In this case, it is difficult for traditional inter-frame prediction technology to match the ideal reference blocks within the complexity limit for most coding blocks, and its reference blocks can be supplemented by frame-level table building. The process of frame-level table building, as shown in (B) of Figure 5, can be manifested as using an 8×8 pixel window to collect pixel block information of the target pixel block contained in the current 8×8 pixel window in the current target video frame, and the 8×8 pixel window can be slid in the target video frame according to a preset horizontal step size sh=1 and a preset vertical step size sv=1, until the target pixel blocks at various positions contained in the target video frame are slid, and the pixel block information of the target pixel blocks at various positions is collected.
[0171] The method for collecting pixel block information can be expressed as a cyclic redundancy check code generation method, which is not limited in the embodiments of the present application.
[0172] Pixel block information can include encoded feature values and pixel block coordinate information. At this time, each target pixel block can be traversed and its feature value calculated. Then, the memory space pointed to by the corresponding address is found, and the pixel block coordinate information of the target pixel block is stored in the memory space. After gradually completing the storage of each target pixel block, the feature value of each target pixel block and the memory space it points to can be called a hash table, completing the construction of the hash table. Among them, the construction of the hash table can be implemented through an array. The feature value can be regarded as an address, which points to the starting position of a continuous memory space. This continuous memory space stores the pixel block coordinates (x, y) whose feature value is the feature value corresponding to the address, so that an index can point to multiple contents, which is used to provide index assistance for the reference block when encoding the block to be encoded later.
[0173] During the use of the hash table, when searching for an ideal reference block for a block to be coded, the characteristic value of the current block to be coded can be calculated, and then the memory space corresponding to the characteristic value can be found through the hash table. If there are pixel block coordinates, each pixel block can be used as a reference block for motion estimation. Specifically, the reference block can be determined based on the index of the hash table, so that motion estimation can be performed on the block to be coded based on the determined reference block, thereby realizing prediction and coding of the block to be coded. Motion estimation is specifically manifested in estimating the pixel gap between the coding block and its reference block. For example, the content indexed by the characteristic value is a candidate for a reference block, and then a reference block with a small pixel gap with the block to be coded is determined as the ideal reference block among multiple candidate reference blocks.
[0174] It should be noted that the motion vector information can be generated in the original process when the target video frame is video encoded. At this time, based on the position and range size of the preset selected area in the target video frame, the motion vector information contained in the preset selected area can be obtained. The original process of video encoding the target video frame is not limited in the embodiments of the present application.
[0175] In a preferred embodiment, after the lookahead window is set, the motion vector information actually encoded can also be obtained by the motion vector information searched by the encoder's pre-processing analysis module instead, wherein the lookahead window can be set by means of encoding parameter configuration, and its size will affect the video delay. Usually, different video scenes such as on-demand, live broadcast, and RTC have different setting ranges. For example, if it is set to 5, the pre-processing analysis module will pre-analyze the current frame and the following 4 frames, and start encoding the first frame after the analysis is completed; after the lookahead window is set, the motion of the following frames can be known in advance based on the set lookahead window. These motion conditions can replace the original motion vector information in the regional table building method to continue to select the table building area. In this regard, the embodiments of the present application are not limited.
[0176] In an embodiment of the present application, a hash table is selectively constructed based on the estimation of video motion. Specifically, based on the introduction of motion vector information, the state of a preset selected area in a target video frame can be determined based on a preset motion vector threshold. A hash table is selectively constructed for a preset selected area in a large motion state, and the frame-level table building method based on the entire frame pixel-by-pixel of HashMe is replaced with a region-level table building method for large motion state areas. This reduces the complexity of HashMe while not affecting the compression efficiency gain brought by HashMe. In addition, when the target video frame hits a scene switch, that is, the target video frame belongs to a scene switch frame reference frame, a frame-level table is supplemented for it. By combining the frame-level table building method based on the entire frame pixel-by-pixel and the region-level table building method for large motion state areas, the utilization rate of HashMe is improved as much as possible, and the complexity of HashMe is reduced as much as possible while ensuring its performance gain.
[0177] It should be noted that for the method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present application are not limited by the order of the actions described, because according to the embodiments of the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present application.
[0178] 6 , a block diagram of an embodiment of a hash table construction device of the present application is shown, which may specifically include the following modules:
[0179] The motion vector information acquisition module 601 is used to acquire a target video frame and acquire motion vector information contained in a preset selected area of the target video frame;
[0180] The pixel block information acquisition module 602 is configured to acquire pixel block information of a target pixel block in a preset selected area of the target video frame in response to the motion vector information being greater than a preset motion vector threshold;
[0181] The first hash table construction module 603 constructs a hash table of a preset selected area of the target video frame according to the pixel block information.
[0182] In one embodiment of the present application, the pixel block information acquisition module 602 may include the following submodules:
[0183] A pixel window acquisition submodule, configured to acquire a pixel window; the pixel window is determined based on a preset pixel block size;
[0184] The pixel block information acquisition submodule is configured to acquire pixel block information of a target pixel block within a preset selected area of the target video frame using the pixel window in response to the motion vector information being greater than a preset motion vector threshold.
[0185] In one embodiment of the present application, the target video frame includes a plurality of pieces of motion vector information within a preset area; and the pixel block information acquisition submodule may include the following units:
[0186] The pixel block information acquisition unit is configured to acquire pixel block information of a target pixel block within a preset selected area of the target video frame using the pixel window in response to at least one motion vector information being greater than a preset motion vector threshold.
[0187] In one embodiment of the present application, the motion vector information includes first-dimensional motion vector information and second-dimensional motion vector information, and the preset motion vector threshold includes a preset first-dimensional motion vector threshold and a preset second-dimensional motion vector threshold;
[0188] The pixel block information acquisition unit may include the following subunits:
[0189] A pixel block information acquisition subunit is configured to respond to the existence of first-dimensional motion vector information of at least one motion vector information being greater than a preset first-dimensional motion vector threshold, or the existence of second-dimensional motion vector information of at least one motion vector information being greater than a preset second-dimensional motion vector threshold, by using the pixel window to acquire pixel block information of a target pixel block within a preset selected area of the target video frame.
[0190] Among them, the using the pixel window to collect pixel block information of the target pixel block in the preset selected area of the target video frame includes determining the pixel block in the pixel window of the target video frame as the target pixel block, and collecting pixel block information of the target pixel block; within the range of the preset selected area of the target video frame, the pixel window is continuously sliding according to a preset horizontal step size and a preset vertical step size, and the pixel block in the pixel window of the target video frame is continuously determined as the target pixel block, and the pixel block information of the target pixel block is collected until the pixel block information of each target pixel block contained in the preset selected area of the target video frame is collected.
[0191] In one embodiment of the present application, the first hash table construction module 603 can also be used to continuously determine the next preset selected area in the target video frame based on the range size of the preset selected area, and obtain the motion vector information contained in the target video frame within the range of the next preset selected area, obtain the pixel block information of the target pixel block within the next preset selected area based on the newly obtained motion vector information and the preset motion vector threshold, and construct a hash table of the next selected area of the target video frame based on the newly obtained pixel block information, until the determined preset selected area completes the target video frame.
[0192] In one embodiment of the present application, the target pixel block includes at least one target pixel block, and the pixel block information includes a coding feature value and pixel block coordinate information; the first hash table construction module 603 may include the following submodules:
[0193] A coding feature value extraction submodule, configured to extract a coding feature value of at least one target pixel block and obtain pixel block coordinate information of the at least one target pixel block;
[0194] A hash table construction submodule is used to construct a hash table of a preset selected area of the target video frame using the encoded feature value of each target pixel block as an index and the pixel block coordinate information of each target pixel block as content; wherein the hash table of the preset selected area of at least one target video frame constitutes the hash table of the target video frame.
[0195] In one embodiment of the present application, the hash table construction device provided in the embodiment of the present application may further include the following modules:
[0196] The second hash table construction module is configured to obtain a pixel window in response to the target video frame being a scene switching frame reference frame, and use the pixel window to collect pixel block information of the pixel blocks included in the scene switching frame reference frame, and construct a hash table of the scene switching frame reference frame based on the pixel block information of the pixel blocks included in the scene switching frame reference frame.
[0197] In one embodiment of the present application, the hash table construction device provided in the embodiment of the present application may further include the following modules:
[0198] A video encoding module is configured to obtain a video frame to be encoded and encode the video frame to be encoded using a hash table of a preset selected area of the target video frame; and / or to encode the video frame to be encoded using a hash table of a preset selected area of the target video frame and a hash table of the scene switching frame; the video frame to be encoded is a video frame that references the target video frame.
[0199] In an embodiment of the present application, a hash table construction device provided in an embodiment of the present application obtains motion vector information contained in a preset selected area of a target video frame, and then obtains pixel block information of a target pixel block within the preset selected area based on the obtained motion vector information being greater than a preset motion vector threshold, and constructs a hash table for the preset selected area. Motion vector information is introduced into the hash table construction process, and a hash table is constructed for the preset selected area of the target video frame based on the motion vector information being greater than the preset motion vector threshold, thereby achieving selective partitioning and constructing the hash table, shortening the hash table construction time, and thereby reducing time complexity and coding complexity.
[0200] 7 , a structural block diagram of an embodiment of a video encoding device of the present application is shown, which may specifically include the following modules:
[0201] A hash table acquisition module 701, in response to an input instruction of a video file, acquires hash tables of a to-be-encoded video frame and a target video frame of the video file, wherein the to-be-encoded video frame is a video frame that references the target video frame; the hash table of the target video frame includes a hash table of a preset selected area of the target video frame, the hash table of the preset selected area being constructed based on pixel block information of a target pixel block within the preset selected area of the target video frame when the motion vector information of the target video frame is greater than a preset motion vector threshold;
[0202] The video encoding module 702 encodes the to-be-encoded video frame using the hash table of the target video frame.
[0203] In one embodiment of the present application, the hash table of the target video frame further includes a hash table of a scene switching frame reference frame, and the hash table of the scene switching frame reference frame is constructed based on pixel block information of pixel blocks included in the scene switching frame reference frame.
[0204] In an embodiment of the present application, the video encoding device provided by the embodiment of the present application introduces motion vector information in the process of constructing a hash table, and constructs a hash table for a preset selected area in the target video frame based on when the motion vector information is greater than a preset motion vector threshold, thereby realizing selective partitioning to construct the hash table, shortening the construction time of the hash table, and thereby reducing the time complexity and encoding complexity in the encoding process.
[0205] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0206] An embodiment of the present application further provides an electronic device, including:
[0207] The present invention includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, the various processes of the above-mentioned hash table construction method or video encoding method embodiment are implemented, and the same technical effects can be achieved. To avoid repetition, they are not described here.
[0208] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned hash table construction method or video encoding method embodiment are implemented, and the same technical effects can be achieved. To avoid repetition, they are not described here.
[0209] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referenced to each other.
[0210] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0211] The present application embodiment is described with reference to the flow chart and / or block diagram of the method, terminal device (system), and computer program product according to the embodiment of the present application. It should be understood that each process and / or box in the flow chart and / or block diagram and the combination of the process and / or box in the flow chart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing terminal device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device produce a device for realizing the function specified in one process or multiple processes and / or one box or multiple boxes of the flow chart.
[0212] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0213] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce computer-implemented processing, so that the instructions executed on the computer or other programmable terminal device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0214] Although preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they become aware of the underlying inventive concepts. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as changes and modifications that fall within the scope of the embodiments of the present invention.
[0215] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0216] It should also be noted that, in this document, relational terms such as first and second, etc., are used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or terminal device that includes the element.
[0217] The above is a detailed introduction to a hash table construction method, a video encoding method, a corresponding encoder, a corresponding electronic device, and a corresponding computer-readable storage medium provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the ideas of the present application, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present application.
Claims
1. A hash table construction method, characterized in that: The method comprises: Acquire a target video frame, and acquire motion vector information contained in a preset selected area of the target video frame; In response to the motion vector information being greater than a preset motion vector threshold, acquiring pixel block information of a target pixel block in a preset selected area of the target video frame; A hash table of a preset selected area of the target video frame is constructed according to the pixel block information.
2. The method according to claim 1, characterized in that In response to the motion vector information being greater than a preset motion vector threshold, acquiring pixel block information of a target pixel block in a preset selected area of the target video frame includes: Acquire a pixel window; the pixel window is determined based on a preset pixel block size; In response to the motion vector information being greater than a preset motion vector threshold, the pixel window is used to collect pixel block information of a target pixel block within a preset selected area of the target video frame.
3. The method according to claim 2, characterized in that The target video frame contains a plurality of pieces of motion vector information in a preset area; in response to the motion vector information being greater than a preset motion vector threshold, using the pixel window to collect pixel block information of a target pixel block in a preset selected area of the target video frame, comprises: In response to at least one motion vector information being greater than a preset motion vector threshold, the pixel window is used to collect pixel block information of a target pixel block within a preset selected area of the target video frame.
4. The method according to claim 3, characterized in that The motion vector information includes first-dimensional motion vector information and second-dimensional motion vector information, and the preset motion vector threshold includes a preset first-dimensional motion vector threshold and a preset second-dimensional motion vector threshold; In response to the existence of at least one motion vector information greater than a preset motion vector threshold, using the pixel window to collect pixel block information of a target pixel block in a preset selected area of the target video frame includes: In response to the existence of at least one motion vector information of a first dimension being greater than a preset first dimension motion vector threshold, or the existence of at least one motion vector information of a second dimension being greater than a preset second dimension motion vector threshold, the pixel window is used to collect pixel block information of a target pixel block within a preset selected area of the target video frame.
5. The method according to claim 3 or 4, characterized in that: The step of using the pixel window to collect pixel block information of a target pixel block in a preset selected area of the target video frame includes: Determine a pixel block within a pixel window of the target video frame as a target pixel block, and collect pixel block information of the target pixel block; Within the preset selected area of the target video frame, the pixel window is continuously slid according to a preset horizontal step size and a preset vertical step size, and pixel blocks within the pixel window of the target video frame are continuously determined as target pixel blocks, and pixel block information of the target pixel blocks is collected until pixel block information of each target pixel block contained in the preset selected area of the target video frame is collected.
6. The method according to claim 1, characterized in that Also includes: According to the range size of the preset selected area, the next preset selected area is continuously determined in the target video frame, and the motion vector information contained in the target video frame within the range of the next preset selected area is obtained, and in response to the newly obtained motion vector information being greater than the preset motion vector threshold, the pixel block information of the target pixel block in the next preset selected area is obtained, and a hash table of the next preset selected area is constructed according to the newly obtained pixel block information, until the determined preset selected area covers the target video frame.
7. The method according to claim 1 or 6, characterized in that: The target pixel block includes at least one target pixel block, and the pixel block information includes a coding feature value and pixel block coordinate information; and constructing a hash table of a preset selected area of the target video frame according to the pixel block information includes: Extracting a coding feature value of at least one target pixel block, and obtaining pixel block coordinate information of at least one target pixel block; A hash table of a preset selected area of the target video frame is constructed using the encoded feature value of each target pixel block as an index and the pixel block coordinate information of each target pixel block as content; wherein the hash table of a preset selected area of at least one target video frame constitutes the hash table of the target video frame.
8. The method according to claim 1 or 6, characterized in that: Also includes: In response to the target video frame being a scene switching frame reference frame, a pixel window is obtained, and the pixel window is used to collect pixel block information of pixel blocks included in the scene switching frame reference frame, and a hash table of the scene switching frame reference frame is constructed based on the pixel block information of the pixel blocks included in the scene switching frame reference frame.
9. The method according to claim 8, characterized in that Also includes: Obtain a video frame to be encoded, and use a hash table of a preset selected area of the target video frame to encode a reference frame of the video frame to be encoded; The to-be-encoded video frame is a video frame that refers to the target video frame; And / or, the to-be-encoded video frame is encoded using a hash table of a preset selected area of the target video frame and a hash table of a scene switching frame reference frame.
10. A video encoding method, characterized in that: The method comprises: In response to an input instruction of a video file, a hash table of a to-be-encoded video frame and a target video frame of the video file is obtained, wherein the to-be-encoded video frame is a video frame that refers to the target video frame; wherein the hash table of the target video frame includes a hash table of a preset selected area of the target video frame, and the hash table of the preset selected area is constructed based on pixel block information of a target pixel block in a preset selected area of the target video frame when the motion vector information is greater than a preset motion vector threshold; The to-be-encoded video frame is encoded using the hash table of the target video frame.
11. The method according to claim 10, characterized in that The hash table of the target video frame also includes a hash table of a scene switching frame reference frame, and the hash table of the scene switching frame reference frame is constructed based on pixel block information of pixel blocks included in the scene switching frame reference frame.
12. An encoder, characterized in that: The encoder comprises a computer program executed in the encoder, and when the computer program is executed by the encoder, the hash table construction method according to any one of claims 1 to 9 or the video encoding method according to any one of claims 10 to 11 is implemented.
13. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the hash table construction method as described in any one of claims 1 to 9 or the video encoding method as described in any one of claims 10 to 11 is implemented.
14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the hash table construction method according to any one of claims 1 to 9 or the video encoding method according to any one of claims 10 to 11 is implemented.
Citation Information
Patent Citations
Screen content coding method and device, equipment and medium
CN111669595A
Screen content processing method, device and equipment
CN112804528A
Screen video coding method and device, computer equipment and storage medium
CN115811615A
Method for encoding image about intra prediction based on hash information
KR1020180107659A
Cited By
Operation and maintenance audit video storage and playback method, medium, equipment and product
CN121542460A
