Reference block searching method and circuit, chip, and device

Abstract

The present application belongs to the field of video coding. Disclosed are a reference block searching method and circuit, a chip, and a device. The method comprises: acquiring a first image region, the first image region covering part or all of a reference region of a current block to be coded (1020); performing reciprocal scanning within the first image region by means of a search window at a first step length, the first step length being the side length of the minimum coding block, the size of the search window being the size of the maximum coding block, the minimum coding block being the coding block with the minimum size, and the maximum coding block being the coding block with the maximum size (1040); calculating respective reference indicator values of a plurality of coding blocks generated every time the search window slides during reciprocal scanning (1060); and, on the basis of the reference indicator values of the coding blocks generated during reciprocal scanning, determining a reference block of the current block to be coded (1080). The present application provides a novel reference block searching method.

Description

Reference block search method, circuit, chip and device

[0001] This application claims priority to Chinese patent application No. 202410104689.6 filed on January 24, 2024, entitled “Reference block search method, circuit, device, processor and chip,” the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present application relates to the field of video coding, and in particular to a reference block search method, circuit, chip and device. Background Art

[0003] Video data is large in raw data size and requires compression before storage or transmission. Compression, also known as encoding, is a crucial step in the encoding process. The IBC (Intra Block Copy) prediction mode is used to search for reference blocks within the current frame using block matching from reconstructed, already-encoded areas.

[0004] In the VVC (Versatile Video Coding) standard algorithm model (a new generation video coding standard), the IBC prediction mode supports a hash search method, which can obtain a high degree of matching of reference blocks but requires a large amount of computing and storage resources. Summary of the Invention

[0005] The present application provides a reference block search method, circuit, chip and device, which provides a new reference block search method. The technical solution includes the following contents.

[0006] According to one aspect of the present application, a method for searching a reference block is provided, and the method includes the following steps.

[0007] Acquire a first image area, where the first image area covers a portion or the entirety of a reference area of ​​a current block to be encoded;

[0008] Performing a reciprocating scan in the first image region with a first step length through a search window, where the first step length is the side length of the minimum coding block, the size of the search window is the size of the maximum coding block, the minimum coding block is the coding block with the smallest size, and the maximum coding block is the coding block with the largest size;

[0009] Calculate the reference index values ​​of the multiple coding blocks generated by each sliding of the search window during the reciprocating scanning process;

[0010] According to the reference index values ​​of the coding blocks generated during the reciprocating scanning process, a reference block of the current block to be coded is determined.

[0011] According to another aspect of the present application, a reference block search circuit is provided, the circuit comprising a first circuit unit, a second circuit unit, a third circuit unit, and a fourth circuit unit; the four circuit units are connected in series via conductive lines.

[0012] A first circuit unit is configured to obtain a first image area, where the first image area covers a portion or the entirety of a reference area of ​​a current block to be encoded;

[0013] a second circuit unit, configured to perform a reciprocating scan in the first image region using a search window with a first step length, where the first step length is a side length of a minimum coding block, a size of the search window is a size of a maximum coding block, the minimum coding block is a coding block with a minimum size, and the maximum coding block is a coding block with a maximum size;

[0014] The third circuit unit is used to calculate the reference index values ​​of the plurality of coding blocks generated by each sliding of the search window during the reciprocating scanning process;

[0015] The fourth circuit unit is configured to determine a reference block of a current block to be encoded according to a reference indicator value of the encoding block generated during the reciprocating scanning process.

[0016] According to another aspect of the present application, a device for searching a reference block is provided, and the device includes the following modules.

[0017] An acquisition module, configured to acquire a first image area, where the first image area covers a portion or the entirety of a reference area of ​​a current block to be encoded;

[0018] a scanning module, configured to perform a reciprocating scan in a first image region through a search window with a first step length, where the first step length is a side length of a minimum coding block, a size of the search window is a size of a maximum coding block, where the minimum coding block is a coding block with a minimum size, and the maximum coding block is a coding block with a maximum size;

[0019] A calculation module, configured to calculate reference index values ​​of respective multiple coding blocks generated by each sliding of the search window during the reciprocating scanning process;

[0020] The determination module is used to determine a reference block of a current block to be encoded according to a reference index value of the encoding block generated during the reciprocating scanning process.

[0021] According to one aspect of the present application, a processor is provided, comprising the above reference block search circuit.

[0022] According to one aspect of the present application, a chip is provided, the chip including a processor, the processor including the search circuit of the reference block as described above.

[0023] According to one aspect of the present application, a computer device is provided. The computer device includes a processor. The processor includes the above reference block search circuit.

[0024] According to another aspect of the present application, a video encoding method is provided, the method comprising: encoding a current encoding block according to a reference block determined by the above-mentioned reference block determination method to obtain a video code stream.

[0025] According to another aspect of the present application, a video decoding method is provided, the method comprising: decoding a video code stream, where the video code stream is obtained by encoding a current coding block using a reference block determined according to the reference block determination method.

[0026] According to another aspect of the present application, a video encoding device is provided, which includes an encoding module, and the encoding module is used to encode a current encoding block according to a reference block determined by the above-mentioned reference block determination method to obtain a video code stream.

[0027] According to another aspect of the present application, a video decoding device is provided, which includes a decoding module for decoding a video code stream, which is obtained by encoding a current coding block according to a reference block determined by the above-mentioned reference block determination method.

[0028] According to another aspect of the present application, a method for storing or sending a video code stream is provided, wherein the video code stream is generated according to the above-mentioned video encoding method, or the video code stream can be decoded based on the above-mentioned video decoding method.

[0029] According to another aspect of the present application, a computer storage medium is provided, storing instructions, wherein the instructions can be executed by at least one processor to implement the above-mentioned video encoding method, generate a video code stream, and store it.

[0030] According to another aspect of the present application, a computer device is provided, comprising a processor and a memory, wherein the memory stores instructions, and the processor loads and executes the instructions to implement the above-mentioned video encoding method or the above-mentioned video decoding method.

[0031] In an embodiment of the present application, a reciprocating scan is performed in the first image area through a search window, and then, based on the reference index value, a reference block of the current block to be coded is determined from the coded blocks generated by the reciprocating scan. That is, the present application provides a new reference block search method based on reciprocating scanning. In addition, by setting the side length of the minimum coding block as the step length of the reciprocating scan and the size of the maximum coding block as the size of the search window of the reciprocating scan, when calculating the reference index value of the coding block generated after sliding, the reference index value of the coding block obtained before sliding can be used, thereby achieving data reuse. For example, if the size of the minimum coding block is a*a and the size of the maximum coding block is 4a*4a, the embodiment of the present application can achieve 75% data reuse, thereby saving the computing resources and storage resources required for calculating the reference index value. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] FIG1 is a basic flow chart of a video encoding process exemplarily shown in the present application;

[0033] FIG2 is a schematic diagram of an inter-frame prediction mode provided by an embodiment of the present application;

[0034] FIG3 is a schematic diagram of candidate motion vectors provided by one embodiment of the present application;

[0035] FIG4 is a schematic diagram of an intra-block copy mode provided by an embodiment of the present application;

[0036] FIG5 is a schematic diagram of an intra-frame string copy mode provided by an embodiment of the present application;

[0037] FIG6 is a simplified block diagram of a communication system provided by one embodiment of the present application;

[0038] FIG7 is a schematic diagram of a placement of a video encoder and a video decoder in a streaming environment provided by one embodiment of the present application;

[0039] FIG8 is a schematic diagram showing the principle of a reference block search method provided by one embodiment of the present application;

[0040] FIG9 is a schematic diagram of a reference area in the IBC prediction mode specified by the VVC protocol;

[0041] FIG10 is a flowchart of a reference block search method provided by one embodiment of the present application;

[0042] FIG11 is a schematic diagram of a search window before and after a slide according to an embodiment of the present application;

[0043] FIG12 is a flowchart of a method for calculating reference index values ​​of multiple coding blocks generated by each sliding of the search window during a reciprocating scan according to an embodiment of the present application;

[0044] FIG13 is a schematic diagram of a method for multiplexing reference index values ​​of coding blocks in overlapping search areas provided by one embodiment of the present application;

[0045] FIG14 is a schematic diagram of a method for multiplexing reference index values ​​of coding blocks in overlapping search areas provided by one embodiment of the present application;

[0046] FIG15 is a schematic diagram of a hardware computing architecture provided by one embodiment of the present application;

[0047] FIG16 is a flowchart of a reference block search method provided by another embodiment of the present application;

[0048] FIG17 is a flowchart of a reference block search method provided in yet another embodiment of the present application;

[0049] FIG18 is a structural block diagram of a reference block search circuit provided by one embodiment of the present application;

[0050] FIG19 is a structural block diagram of a reference block search apparatus provided by one embodiment of the present application;

[0051] FIG20 is a structural block diagram of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0052] First, a brief introduction is given to the terms involved in the embodiments of this application.

[0053] Before introducing the embodiments of the present application, a brief introduction to video encoding technology is first given in conjunction with Figure 1. Figure 1 exemplarily shows a basic flow chart of a video encoding process.

[0054] A video signal is a sequence of images consisting of multiple frames. A frame represents the spatial information of a video signal. Taking the YUV format as an example, a frame consists of a matrix of luminance samples (Y) and two matrices of chrominance samples (Cb and Cr). Video signals can be acquired using two methods: those captured by a camera and those generated by a computer. Due to differences in statistical properties, the corresponding compression encoding methods may also differ.

[0055] Some mainstream video coding technologies, such as H.265 / HEVC (High Efficient Video Coding), H.266 / VVC (Versatile Video Coding), and AVS (Audio Video Coding Standard) (such as AVS3), use a hybrid coding framework to perform the following operations and processing on the input raw video signal:

[0056] 1. Block partition structure: The input image is divided into several non-overlapping processing units, each of which performs similar compression operations. This processing unit is called a CTU (Coding Tree Unit) or LCU. Below the CTU, further subdivision can be performed to obtain one or more basic coding units, called CUs. Each CU is the most basic element in the encoding process. The following describes the various possible encoding methods for each CU.

[0057] 2. Predictive Coding: This includes methods such as intra-frame prediction and inter-frame prediction. The original video signal is predicted by a selected reconstructed video signal to produce a residual video signal. The encoder needs to select the most suitable predictive coding mode for the current CU to be encoded from among many possible modes and inform the decoder. Intra-frame prediction means that the predicted signal comes from an already encoded and reconstructed area within the same image. Inter-frame prediction means that the predicted signal comes from an already encoded image that is different from the current image (called a reference image).

[0058] 3. Transform Coding and Quantization: The residual video signal undergoes transform operations such as the Discrete Fourier Transform (DFT) and Discrete Cosine Transform (DCT) to convert the signal into a transform domain, known as transform coefficients. The signal in the transform domain is further subjected to a lossy quantization operation, which loses some information, making the quantized signal more suitable for compression. Some video coding standards may offer more than one transform method. Therefore, the encoder must select one for the current CU to be encoded and inform the decoder. The level of quantization is typically determined by the quantization parameter (QP). A larger QP value means that coefficients with a larger value range will be quantized to the same output, which generally results in greater distortion and a lower bitrate. Conversely, a smaller QP value means that coefficients with a smaller value range will be quantized to the same output, which generally results in less distortion and a higher bitrate.

[0059] 4. Entropy Coding or Statistical Coding: The quantized transform domain signal will be statistically compressed and encoded according to the frequency of occurrence of each value, and finally a binary (0 or 1) compressed code stream will be output. At the same time, the encoding generates other information, such as the selected mode, motion vector, etc., which also need to be entropy coded to reduce the bit rate. Statistical coding is a lossless coding method that can effectively reduce the bit rate required to express the same signal. Common statistical coding methods include variable length coding (VLC) or context-based binary arithmetic coding (CABAC).

[0060] 5. Loop Filtering: The encoded image undergoes inverse quantization, inverse transformation, and prediction compensation (the reverse of 2 to 4 above) to obtain a reconstructed decoded image. Compared with the original image, the reconstructed image has some information that is different from the original image due to the influence of quantization, resulting in distortion. Filtering the reconstructed image, such as deblocking filtering, SAO (Sample Adaptive Offset), or ALF (Adaptive Lattice Filter), can effectively reduce the distortion caused by quantization. Since these filtered reconstructed images will serve as a reference for subsequent encoded images and be used to predict future signals, the above filtering operation is also called loop filtering, that is, filtering operations within the encoding loop.

[0061] As can be seen from the encoding process described above, at the decoding end, after receiving the compressed bitstream for each CU, the decoder first performs entropy decoding to obtain various mode information and quantized transform coefficients. Each coefficient undergoes inverse quantization and inverse transformation to produce a residual signal. Furthermore, based on the known coding mode information, the prediction signal corresponding to the current CU to be encoded can be obtained. Adding these two together yields the reconstructed signal. Finally, the reconstructed value of the decoded image undergoes loop filtering to produce the final output signal.

[0062] Some mainstream video coding standards, such as HEVC, VVC, AVS3 and other standards, all adopt a block-based hybrid coding framework. They divide the original video data into a series of coding blocks, and combine video coding methods such as prediction, transform and entropy coding to achieve video data compression. Among them, motion compensation is a commonly used prediction method for video coding. Motion compensation is based on the redundant characteristics of video content in the time domain or spatial domain, and derives the prediction value of the current coding block from the encoded area. This type of prediction method includes: inter-frame prediction, intra-frame block copy prediction, intra-frame string copy prediction, etc. In specific coding implementations, these prediction methods may be used alone or in combination. For coding blocks using these prediction methods, it is usually necessary to explicitly or implicitly encode one or more two-dimensional displacement vectors in the code stream to indicate the displacement of the current block (or the same-position block of the current block) relative to one or more reference blocks.

[0063] It's important to note that displacement vectors may have different names in different prediction modes and implementations. This article uniformly describes them as follows: 1) The displacement vector in inter-frame prediction mode is called a motion vector (MV); 2) The displacement vector in IBC (Intra Block Copy) prediction mode is called a block vector (BV); 3) The displacement vector in ISC (Intra String Copy) prediction mode is called a string vector (SV). Intra string copy is also known as "string prediction" or "string matching."

[0064] MV refers to the displacement vector used in the inter-frame prediction mode, which points from the current image to the reference image, and its value is the coordinate offset between the current block and the reference block, where the current block and the reference block are in two different images. In the inter-frame prediction mode, motion vector prediction can be introduced. By predicting the motion vector of the current block, the predicted motion vector corresponding to the current block is obtained, and the difference between the predicted motion vector corresponding to the current block and the actual motion vector is encoded and transmitted. Compared with directly encoding and transmitting the actual motion vector corresponding to the current block, it is beneficial to save bit overhead. In the embodiment of the present application, the predicted motion vector refers to the predicted value of the motion vector of the current block obtained by the motion vector prediction technology.

[0065] BV refers to the displacement vector used in the IBC prediction mode, and its value is the coordinate offset between the current block and the reference block, where both the current block and the reference block are in the current image. In the IBC prediction mode, block vector prediction can be introduced. By predicting the block vector of the current block, a predicted block vector corresponding to the current block is obtained. The difference between the predicted block vector corresponding to the current block and the actual block vector is encoded and transmitted. Compared with directly encoding and transmitting the actual block vector corresponding to the current block, this helps save bit overhead. In the embodiment of the present application, the predicted block vector refers to the predicted value of the block vector of the current block obtained by block vector prediction technology.

[0066] SV refers to the displacement vector used in the ISC prediction mode, and its value is the coordinate offset between the current string and the reference string, where both the current string and the reference string are in the current image. In the ISC mode, string vector prediction can be introduced. By predicting the string vector of the current string, a predicted string vector corresponding to the current string is obtained. The difference between the predicted string vector and the actual string vector corresponding to the current string is encoded and transmitted. Compared with directly encoding and transmitting the actual string vector corresponding to the current string, this helps save bit overhead. In the embodiment of the present application, the predicted string vector refers to the predicted value of the string vector of the current string obtained through string vector prediction technology.

[0067] The following are some different prediction modes:

[0068] 1. Inter-frame prediction mode

[0069] As shown in Figure 2, inter-frame prediction uses the correlation in the video time domain to predict the pixels of the current image using the pixels of the adjacent encoded images to effectively remove the temporal redundancy of the video and effectively save bits of the encoded residual data. Here, P is the current frame, Pr is the reference frame, B is the current block to be encoded, and Br is the reference block of B. B' and B have the same coordinate position in the image, with Br's coordinates being (xr, yr) and B''s coordinates being (x, y). The displacement between the current block to be encoded and its reference block is called a motion vector (MV), i.e., MV = (xr-x, yr-y).

[0070] Considering the strong correlation between adjacent blocks in the temporal or spatial domains, MV prediction technology can be used to further reduce the bits required to encode MVs. In H.265 / HEVC, inter-frame prediction includes two MV prediction technologies: Merge and AMVP (Advanced Motion Vector Prediction).

[0071] Merge mode will establish an MV candidate list for the current PU (Prediction Unit), which contains 5 candidate MVs (and their corresponding reference images). It traverses these 5 candidate MVs and selects the one with the lowest rate-distortion cost as the optimal MV. If the codec establishes the candidate list in the same way, the encoder only needs to transmit the index of the optimal MV in the candidate list. It should be noted that HEVC's MV prediction technology also has a skip mode, which is a special case of Merge mode. After finding the optimal MV in Merge mode, if the current block and the reference block are basically the same, then there is no need to transmit residual data, only the index of the MV and a skip flag.

[0072] The MV candidate list established by the Merge mode includes two situations: spatial domain and temporal domain. For B Slice (B frame image), it also includes a combined list method. Among them, the spatial domain provides up to 4 candidate MVs, and its establishment is shown in part (a) of Figure 3. The spatial domain list is established in the order of A1→B1→B0→A0→B2, where B2 is a substitute, that is, when one or more of A1, B1, B0, and A0 do not exist, the motion information of B2 needs to be used; the temporal domain provides up to 1 candidate MV, and its establishment is shown in part (b) of Figure 3, which is obtained by scaling the MV of the same-position PU according to the following formula: curMV=td*colMV / tb;

[0073] Among them, curMV represents the MV of the current PU, colMV represents the MV of the co-located PU, td represents the distance between the current image and the reference image, and tb represents the distance between the co-located image and the reference image. If the PU at position D0 on the co-located block is not available, it is replaced by the co-located PU at position D1. For the PU in B Slice, since there are two MVs, its MV candidate list also needs to provide two MVPs (Motion Vector Predictor, predicted motion vector). HEVC generates a combination list for B Slice by combining the first 4 candidate MVs in the MV candidate list in pairs.

[0074] Similarly, the AMVP mode uses the MV correlation of adjacent blocks in the spatial and temporal domains to establish a MV candidate list for the current PU. Unlike the Merge mode, the AMVP mode selects the optimal predicted MV from the MV candidate list, and performs differential encoding with the optimal MV obtained through motion search of the current block to be encoded, that is, encoding MVD = MV-MVP, where MVD is the motion vector residual (Motion Vector Difference); by establishing the same list, the decoding end only needs the sequence number of MVD and MVP in the list to calculate the MV of the current decoding block. The MV candidate list of the AMVP mode also includes both spatial and temporal domain situations. The difference is that the length of the MV candidate list of the AMVP mode is only 2.

[0075] As mentioned above, in HEVC's AMVP mode, MVD (Motion Vector Difference) must be encoded. In HEVC, the MVD resolution is controlled by the use_integer_mv_flag in the slice_header. When the flag is 0, the MVD is encoded at 1 / 4 (luminance) pixel resolution; when the flag is 1, the MVD is encoded at integer (luminance) pixel resolution. VVC uses an adaptive motion vector resolution (AMVR) method. This method allows each CU to adaptively select the resolution of the MV to be encoded. In normal AMVP mode, the optional resolutions include 1 / 4, 1 / 2, 1, and 4 pixel resolutions. For a CU with at least one non-zero MVD component, a flag is first encoded to indicate whether quarter-luminance sample MVD precision is used for the CU. If the flag is 0, the MVD of the current CU is encoded at 1 / 4 pixel resolution. Otherwise, a second flag is encoded to indicate whether the CU uses 1 / 2 pixel resolution or another MVD resolution. Otherwise, a third flag is encoded to indicate whether 1-pixel resolution or 4-pixel resolution is used for the CU.

[0076] 2. IBC Prediction Model

[0077] IBC is an intra-frame coding tool adopted in the HEVC Screen Content Coding (SCC) extension, which significantly improves the coding efficiency of screen content. In AVS3 and VVC, IBC technology is also adopted to improve the performance of screen content coding. IBC uses the spatial correlation of screen content video and uses the coded image pixels on the current image to predict the pixels of the current block to be coded, which can effectively save the bits required to encode the pixels. As shown in Figure 4, the displacement between the current block and its reference block in IBC is called BV (block vector). H.266 / VVC uses a BV prediction technology similar to inter-frame prediction to further save the bits required to encode BV, and allows the use of 1 or 4 pixel resolution to encode BVD (Block Vector Difference).

[0078] 3. ISC Forecasting Model

[0079] ISC technology divides a coding block into a series of pixel strings or unmatched pixels according to a specific scanning sequence (such as raster scan, round-trip scan, and zig-zag scan). Similar to IBC, each string searches for a reference string of the same shape within the coded area of ​​the current image to derive a predicted value for the current string. By encoding the residual between the current string pixel value and the predicted value instead of directly encoding the pixel value, it can effectively save bits. Figure 5 shows a schematic diagram of intra-frame string copying. The dark gray area represents the coded area, the 28 white pixels represent string 1, the 35 light gray pixels represent string 2, and the single black pixel represents an unmatched pixel. The displacement between string 1 and its reference string is string vector 1 in Figure 4; the displacement between string 2 and its reference string is string vector 2 in Figure 4.

[0080] Intra-frame string replication technology requires encoding the SV corresponding to each string in the current coding block, the string length, and a flag indicating whether there is a matching string. SV represents the displacement of the string to be encoded from its reference string. The string length represents the number of pixels contained in the string. In different implementations, there are multiple ways to encode the string length. The following are several examples (some examples may be used in combination): 1) Encode the string length directly in the bitstream; 2) Encode the number of pixels to be processed after the string is processed in the bitstream. The decoder calculates the length of the current string, L = N-N1-N2, based on the current block size N, the number of processed pixels N1, and the decoded number of pixels to be processed N2; 3) Encode a flag in the bitstream to indicate whether the string is the last string. If it is the last string, the length of the current string, L = N-N1, is calculated based on the current block size N and the number of processed pixels N1. If a pixel does not find a corresponding reference in the reference area, the pixel value of the unmatched pixel will be directly encoded.

[0081] As shown in Figure 6, it shows a simplified block diagram of a communication system provided by an embodiment of the present application. The communication system 600 includes multiple devices, which can communicate with each other via, for example, a network 650. For example, the communication system 600 includes a first device 610 and a second device 620 interconnected via the network 650. In the embodiment of Figure 6, the first device 610 and the second device 620 perform unidirectional data transmission. For example, the first device 610 can encode video data, such as a video picture stream collected by the first device 610, for transmission to the second device 620 via the network 650. The encoded video data is transmitted in the form of one or more encoded video code streams. The second device 620 can receive the encoded video data from the network 650, decode the encoded video data to recover the video data, and display the video picture based on the recovered video data. Unidirectional data transmission is more common in applications such as media services.

[0082] In another embodiment, the communication system 600 includes a third device 630 and a fourth device 640 that perform bidirectional transmission of encoded video data, which can occur, for example, during a video conference. For bidirectional data transmission, each of the third device 630 and the fourth device 640 can encode video data (e.g., a video picture stream captured by the device) for transmission to the other of the third device 630 and the fourth device 640 via the network 650. Each of the third device 630 and the fourth device 640 can also receive the encoded video data transmitted by the other of the third device 630 and the fourth device 640, decode the encoded video data to recover the video data, and display the video pictures on an accessible display device based on the recovered video data.

[0083] In the embodiment of Figure 6, the first device 610, the second device 620, the third device 630 and the fourth device 640 can be computer devices such as servers, personal computers and smart phones, but the principles disclosed in this application are not limited to this. The embodiments of the present application are applicable to PCs (Personal Computers), mobile phones, tablet computers, media players and / or dedicated video conferencing equipment. Network 650 represents any number of networks that transmit encoded video data between the first device 610, the second device 620, the third device 630 and the fourth device 640, including, for example, wired and / or wireless communication networks. The communication network 650 can exchange data in circuit switching and / or packet switching channels. The network may include a telecommunications network, a local area network, a wide area network and / or the Internet. For the purposes of this application, unless explained below, the architecture and topology of the network 650 may be irrelevant to the operations disclosed in this application.

[0084] As an example, FIG7 illustrates the placement of a video encoder and a video decoder in a streaming environment. The subject matter disclosed herein is equally applicable to other video-enabled applications, including, for example, video conferencing, digital TV, and storing compressed video on digital media such as CDs (Compact Discs), DVDs (Digital Versatile Discs), memory sticks, and the like.

[0085] Streaming system 700 includes an acquisition subsystem 713, which may include a video source 701, such as a digital camera. Video source 701 creates an uncompressed video picture stream 702. In one embodiment, video picture stream 702 includes samples captured by a digital camera. Video picture stream 702 is depicted as a thicker line to emphasize the higher data volume of the video picture stream compared to encoded video data 704 (or encoded video stream). Video picture stream 702 may be processed by an electronic device 720, which includes a video encoder 703 coupled to video source 701. Video encoder 703 may include hardware, software, or a combination of hardware and software to implement or embody various aspects of the disclosed subject matter, as described in greater detail below. Encoded video data 704 (or encoded video stream 704) is depicted as a thinner line to emphasize the lower data volume of encoded video data 704 (or encoded video stream 704), which may be stored on a streaming server 705 for future use. One or more streaming client subsystems, such as client subsystem 706 and client subsystem 708 in FIG7 , can access streaming server 705 to retrieve copies 707 and 709 of encoded video data 704. Client subsystem 706 can include, for example, a video decoder 710 in electronic device 730. Video decoder 710 decodes the incoming copy 707 of the encoded video data and produces an output video picture stream 711 that can be presented on a display 712 (e.g., a screen) or another presentation device (not depicted). In some streaming systems, the encoded video data 704, video data 707, and video data 709 (e.g., video code streams) can be encoded according to certain video encoding / compression standards.

[0086] It should be noted that the electronic device 720 and the electronic device 730 may include other components (not shown). For example, the electronic device 720 may include a video decoder (not shown), and the electronic device 730 may also include a video encoder (not shown). The video decoder is used to decode the received encoded video data; the video encoder is used to encode the video data. The encoded video data is transmitted between the video encoder and the video decoder in the form of a video code stream. It should be noted that the technical solution provided in the embodiment of the present application can be applied to the H.266 / VVC standard, the H.265 / HEVC standard, AVS (such as AVS3) or the next generation video coding and decoding standard, and the embodiment of the present application is not limited to this.

[0087] It should also be noted that the execution entity of each step of the video encoding method provided in the embodiment of the present application can be an encoding end device. During the video encoding process in the IBC prediction mode, the encoding scheme provided in the embodiment of the present application can be used to encode the block vector of the current block. Both the decoding end device and the encoding end device can be computer devices, which refer to electronic devices with data calculation, processing, and storage capabilities, such as PCs, mobile phones, tablet computers, media players, dedicated video conferencing equipment, servers, etc.

[0088] In addition, the method provided in this application can be used alone or in combination with other methods in any order. The encoder based on the method provided in this application can be implemented by one or more processors or one or more integrated circuits. The following describes the technical solution of this application through several embodiments.

[0089] In the related art, in the VVC standard algorithm model, the IBC prediction mode supports two search methods: Pattern search and Hash search. The calculation speed of Pattern search is faster, but there is a possibility of local optimality; the reference block obtained by hash search has a high matching degree, but it requires a large amount of computing resources and storage resources. Based on this, the present application provides a method for searching the reference block of the current block to be encoded in the IBC prediction mode. The IBC prediction mode needs to search for the reference block of the current block to be encoded from the area that has been encoded and reconstructed in the current encoding frame based on block matching.

[0090] Figure 8 is a schematic diagram showing the principle of a reference block search method provided by an exemplary embodiment of the present application. The reference block search method shown in Figure 8 is executed by an encoding device, and optionally, the search method shown in Figure 8 is applied to an IBC prediction mode.

[0091] In the search process shown in FIG8 , the encoding device executes step 801: obtaining a first image area. The first image area covers part or all of the reference area of ​​the current block to be encoded. The reference area is an area used to search for a reference block for the current block to be encoded, and the reference area includes the reference block of the current block to be encoded. In an embodiment of the present application, the reference area of ​​the current block to be encoded and the current block to be encoded are located in the same frame image. Optionally, the first image area is part or all of the reference area of ​​the current block to be encoded, that is, the first image area is located in the reference area of ​​the current block to be encoded. In one embodiment, the search method of the present application is applied to the IBC prediction mode of the VVC protocol. In the IBC prediction mode specified by the VVC protocol, the reference area of ​​the current block to be encoded is shown in FIG9 . The reference area of ​​the current block to be encoded includes the current CTU and the left CTU. The current CTU refers to the CTU in which the current block to be encoded is located. A CTU is 128*128 in size and contains four CUs of 64*64 in size, all in pixels. The reference area of ​​the current block to be coded includes three complete 64*64 CUs and a coded area in a 64*64 CU. Optionally, the first image area is any one of the four 64*64 CUs where the reference area is located, so the size of the first image area is 64*64. Optionally, the search process shown in Figure 8 is performed once in each of the four 64*64 CUs for the current block to be coded.

[0092] In the search process shown in FIG8 , the encoding device executes step 802: performing a reciprocating scan with a first step length in the first image area through the search window. Optionally, multiple rounds of reciprocating scans are performed in the first image area through the search window, and the search window will slide multiple times in each round of reciprocating scans, and each slide will generate multiple coding blocks. During a round of reciprocating scans, the search window slides from the left side of the first image area to the right, each time by the first step length. After reaching the right critical position, it slides downward by the first step length. The search window then slides from the right to the left, each time by the first step length. After reaching the left critical position, it slides downward by the first step length. The above sliding is repeated until the search window reaches the bottom and can no longer slide downward.

[0093] In the embodiment of the present application, the first step length is the side length of the minimum coding block, the size of the search window is the size of the maximum coding block, and the sizes of the minimum coding block and the maximum coding block are pre-set. Taking the size of the minimum coding block as 4*4 and the size of the maximum coding block as 16*16 as an example, the bold solid line in Figure 8 shows the search window before sliding, and the bold dotted line shows the search window after sliding. Schematically, before the first slide begins, the pixel coordinates of the upper left corner of the search window before sliding are (0, 0), and after one slide, the pixel coordinates of the upper left corner of the search window after sliding are (4, 0).

[0094] In one embodiment, multiple rounds of reciprocating scanning are performed, and the starting points of the search windows of the multiple rounds of reciprocating scanning constitute a pixel matrix, the length and width of the pixel matrix are both the first step length, and the size of the pixel matrix is ​​the size of the minimum coding block. For example, as shown in Figure 8, in the first round of reciprocating scanning (ROUND 0), the starting point of the search window is (0, 0). In the second round of reciprocating scanning (ROUND 1), the starting point of the search window is (1, 0); in the third round of reciprocating scanning (ROUND 2), the starting point of the search window is (2, 0); in the fourth round of reciprocating scanning (ROUND 3), the starting point of the search window is (3, 0); in the fifth round of reciprocating scanning (ROUND 4), the starting point of the search window is (0, 1); and so on, until the search window performs the sixteenth round of reciprocating scanning, and in the sixteenth round of reciprocating scanning (ROUND 15), the starting point of the search window is (3, 3).

[0095] In the search process shown in FIG8 , the encoding device executes step 803: calculating the reference index values ​​of the multiple coding blocks generated by each sliding of the search window during the reciprocating scanning process. During the reciprocating scanning process, after each sliding of the search window, the search window after sliding includes an updated search area and an overlapping search area. As shown in FIG8 , the updated search area is an area in the search window after sliding that does not overlap with the search window before sliding, and the overlapping search area is an area in the search window after sliding that overlaps with the search window before sliding. After each sliding, the updated search area obtained includes four new minimum coding blocks. The four new minimum coding blocks are coding blocks included in the search window after sliding but not in the search window before sliding. The size of the minimum coding block is 4*4. Optionally, four hardware computing units (optionally, SAD (Sum of Absolute Difference, absolute error sum) computing unit, SATD (Sum of Absolute Transformed Difference, transformed absolute error sum) computing unit, SSE (Sum of Squared Error, square difference sum) computing unit, etc.) are used to calculate the reference index values ​​of the four minimum coding blocks (optionally, SAD, SATD, SSE, etc.). One hardware computing unit calculates the reference index value of a minimum coding block. Based on the reference index values ​​of the four minimum coding blocks, combined with the reference index values ​​of the coding blocks in the overlapping search area, the reference index values ​​of the coding blocks of each size are obtained.

[0096] Schematically, the size of the four new minimum coding blocks is 4*4. Combined with the reference index values ​​of the coding blocks in the overlapping search area, reference index values ​​of coding blocks represented as 4*8, 4*16, 8*16, 16*16, 8*4, 16*4, 16*8, and 8*8 can be obtained. It can be understood that the reference index values ​​of the coding blocks in the overlapping search area are reusable data. When the next slide is performed, the reference index values ​​of the coding blocks in the overlapping search area can be used to calculate the reference index values ​​of the coding blocks generated by the next slide. Under the data relationship shown in Figure 8, when calculating the reference index values ​​of coding blocks of various sizes generated after each slide, 75% data reuse can be achieved.

[0097] In the search process shown in FIG8 , the encoding device executes step 804 : determining a reference block of the current block to be encoded according to the reference index value of the encoding block generated during the reciprocating scanning process.

[0098] In the first image area, the search window performs multiple rounds of reciprocating scanning. During each round of reciprocating scanning, the search window slides multiple times, and each slide generates multiple coding blocks. The coding device determines, from all coding blocks generated by the multiple rounds of reciprocating scanning, a coding block whose reference index value (optionally, SAD) meets the selection condition as a reference block for the current block to be encoded. Exemplarily, from all coding blocks generated by the multiple rounds of reciprocating scanning, the coding block with the smallest reference index value is determined as the reference block for the current block to be encoded.

[0099] To summarize, the embodiment of the present application performs a reciprocating scan with a first step length in the first image area through a search window, where the first step length is the side length of the minimum coding block, and the size of the search window is the size of the maximum coding block. This enables the reference index values ​​of the coding blocks in the overlapping search area to be used when calculating the reference index values ​​of multiple coding blocks generated by one slide. Moreover, if the size of the minimum coding block is a*a and the size of the maximum coding block is 4a*4a, the above scheme can achieve 75% data reuse, save computing resources and storage resources, and improve computing efficiency.

[0100] FIG10 shows a flowchart of a reference block search method provided by an exemplary embodiment of the present application. The method is illustrated by an example in which the method is executed by an encoding device. The method includes:

[0101] Step 1020: Acquire a first image region, where the first image region covers a portion or the entire reference region of the current block to be encoded.

[0102] The reference area is used to search for a reference block for the current block to be coded. The reference area includes the reference block of the current block to be coded. The current block to be coded refers to the image block currently being pre-coded. The reference area and the current block to be coded are located in the same frame image to implement intra-frame prediction. Optionally, the first image area includes part, all, or all of the reference area, i.e., the first image area is located in the reference area of ​​the current block to be coded.

[0103] In one embodiment, the search method provided in this application is applied to the IBC prediction mode in the VVC protocol. In this IBC prediction mode, a reference block for the current coding block is searched from the already coded and reconstructed area in the current coded frame based on block matching. Optionally, the search method provided in this application can be applied to a graphics processing unit (GPU), a video processing unit (VPU), etc., and implemented by an integrated circuit in the processor. The reference area specified in the VVC protocol is shown in Figure 9. The reference area of ​​the current block to be coded is located in the current CTU and the left CTU. A CTU has a size of 128*128 and contains four CUs of a size of 64*64, all in pixels. The reference area of ​​the current block to be coded includes three complete 64*64 CUs and an already coded area in a 64*64 CU. Optionally, the first image area is any one of the four 64*64 CUs where the reference area is located, that is, the size of the first image area is 64*64. Optionally, the current block to be coded performs the reference block search process in each of the four 64*64 CUs.

[0104] Figure 9 shows the reference areas in four cases. ① Referring to part (A) of Figure 9, if the current block to be coded is located in the 64*64 CU in the upper left corner of the current CTU, the reference area of ​​the current block to be coded includes the 64*64 CUs in the upper right, lower left, and lower right corners of the left CTU, and the coded area in the 64*64 CU in the upper left corner of the current CTU. ② Referring to part (B) of Figure 9, if the current block to be coded is located in the 64*64 CU in the upper right corner of the current CTU, the reference area of ​​the current block to be coded includes the 64*64 CUs in the lower left and lower right corners of the left CTU, the 64*64 CU in the upper left corner of the current CTU, and the coded area in the 64*64 CU in the upper right corner of the current CTU. ③ Referring to part (C) of Figure 9, if the current block to be coded is located in the 64*64 CU at the lower left corner of the current CTU, the reference area of ​​the current block to be coded includes the 64*64 CU at the lower right corner of the left CTU, the 64*64 CUs at the upper left and upper right corners of the current CTU, and the coded area in the 64*64 CU at the lower left corner of the current CTU. ④ Referring to part (D) of Figure 9, if the current block to be coded is located in the 64*64 CU at the lower right corner of the current CTU, the reference area of ​​the current block to be coded includes the 64*64 CUs at the upper left, upper right, and lower left corners of the current CTU, and the coded area in the 64*64 CU at the lower right corner of the current CTU.

[0105] In another embodiment, the search method provided in the present application can be applied to the IBC prediction mode in the HEVC compression standard protocol. The reference area specified in the HEVC compression standard protocol is the entire frame where the current coding block is located, and the entire frame is used as the first image area to execute the search method provided in the present application.

[0106] Step 1040: Perform a reciprocating scan in the first image region using a search window with a first step length, where the first step length is the side length of the minimum coding block, and the size of the search window is the size of the maximum coding block, where the minimum coding block is the coding block with the smallest size, and the maximum coding block is the coding block with the largest size.

[0107] Reciprocating scanning refers to the process of scanning the search window back and forth in opposite directions. In one embodiment, reciprocating scanning involves scanning in a first direction followed by scanning in a second direction, where the first and second directions are opposite directions. For example, the search box may be scanned from left to right, then from right to left, or vice versa. Another example is the search box may be scanned from top to bottom, then from bottom to top, or vice versa. The reciprocating scanning process is described in detail in step 802 above.

[0108] In an embodiment of the present application, the size range of the coding block is the size range supported by the coding device in the IBC prediction mode. Optionally, the size range of the coding block is a pre-defined size range. Optionally, the size range of the coding block is 4*4 to 16*16, that is, the size of the minimum coding block is 4*4, and the size of the maximum coding block is 16*16. Optionally, the size range of the coding block can also be 8*8 to 32*32, that is, the size of the minimum coding block is 8*8, and the size of the maximum coding block is 32*32, and the units are all pixels. In this application, the size range of the coding block will be mainly 4*4 to 16*16 for example, and the size range of 8*8 to 32*32 can also be implemented based on a similar search process.

[0109] Step 1060 , calculating reference index values ​​of the plurality of coding blocks generated by each sliding of the search window during the reciprocating scanning process;

[0110] During the reciprocating scanning process, the search window will slide multiple times, and each slide will generate new 4*4, 4*8, 4*16, 8*16, 16*16, 8*4, 16*4, 16*8, and 8*8 coding blocks. Figure 11 shows the search window before and after one slide. The bold solid line in Figure 11 is the search window before sliding, and the bold dotted line is the search window after sliding.

[0111] In conjunction with reference Figure 11, the search window after sliding generates the following new coding blocks, which refer to the coding blocks that are not included in the search window before sliding but are included in the search window after sliding. Coding blocks with a size of 4*4: a coding block composed of pixel matrix 17, a coding block composed of pixel matrix 18, a coding block composed of pixel matrix 19, and a coding block composed of pixel matrix 20. Coding blocks with a size of 4*8: a coding block composed of pixel matrices 17 and 18, and a coding block composed of pixel matrices 19 and 20. Coding blocks with a size of 4*16: a coding block composed of pixel matrices 17, 18, 19, and 20. Coding blocks with a size of 8*4: a coding block composed of pixel matrices 13 and 17, a coding block composed of pixel matrices 14 and 18, a coding block composed of pixel matrices 15 and 19, and a coding block composed of pixel matrices 16 and 20. Coding blocks of size 16*4: coding blocks consisting of pixel matrices 5, 9, 13, and 17; coding blocks consisting of pixel matrices 6, 10, 14, and 18; coding blocks consisting of pixel matrices 7, 11, 15, and 19; and coding blocks consisting of pixel matrices 8, 12, 16, and 20. Coding blocks of size 8*8: coding blocks consisting of pixel matrices 13, 17, 14, and 18; and coding blocks consisting of pixel matrices 15, 19, 16, and 20. Coding blocks of size 16*16: coding blocks consisting of pixel matrices 5 to 20. After one slide, multiple new coding blocks will be generated based on the updated pixel matrices 17, 18, 19, and 20, and reference index values ​​for the generated multiple new coding blocks will be calculated. Optionally, the reference index value is any one of SAD, SATD, and SSE.

[0112] Step 1080 , determining a reference block for the current block to be coded based on the reference index values ​​of the coding blocks generated during the reciprocating scanning process;

[0113] During a round of reciprocating scanning, multiple slides are performed, each of which generates multiple coding blocks. Based on the reference index values ​​of the coding blocks, a reference block for the current block to be coded is determined from all the coding blocks generated by the multiple slides. In one embodiment, the reference index value is any one of SAD, SATD, and SSE. From all the coding blocks generated by the multiple slides, the coding block with the smallest reference index value is determined as the reference block for the current block to be coded.

[0114] In summary, this application will perform a reciprocating scan in the first image region using a search window with a first step length, where the first step length is the side length of the smallest coding block, and the size of the search window is the size of the largest coding block. This application will also calculate the reference index values ​​of the multiple coding blocks generated by each sliding of the search window during the reciprocating scan, and determine the reference block of the current block to be coded based on the reference index values. In other words, this application provides a new reference block search method based on reciprocating scanning.

[0115] Based on the optional embodiment shown in FIG10 , FIG12 shows a flow chart of a method for calculating reference index values ​​of respective multiple coding blocks generated by each sliding of the search window during a reciprocating scan process. The method includes the following steps:

[0116] Step 1061: During the reciprocating scanning process, after each sliding of the search window, calculate reference index values ​​for each of the plurality of minimum coding blocks in an updated search area, where the updated search area is an area of ​​the search window after the sliding that does not overlap with the search window before the sliding.

[0117] During the reciprocating scanning process, the search window will slide multiple times, and each slide will generate multiple new coding blocks. For each slide, the search window after the slide will include an updated search area and an overlapping search area. With reference to Figure 8, the updated search area is the area of ​​the search window after the slide that does not overlap with the search window before the slide, and the overlapping search area is the area of ​​the search window after the slide that overlaps with the search window before the slide. The updated search area includes multiple minimum coding blocks. In one embodiment, a hardware computing unit is used to calculate the reference index value of each of the a minimum coding blocks in the updated search area. The a hardware computing unit corresponds one-to-one to the a minimum coding blocks, and a is a preset positive integer.

[0118] Figure 8 takes the minimum coding block size of 4*4 and the maximum coding block size of 16*16 as an example. The updated search area includes 4 minimum coding blocks of size 4*4. The reference index values ​​of the 4 minimum coding blocks are calculated by 4 hardware computing units. The 4 hardware computing units correspond one-to-one to the 4 minimum coding blocks. Optionally, the reference index value is any one of SAD, SATD, and SSE. The hardware computing units can be correspondingly called SAD computing units, SATD computing units, and SSE computing units.

[0119] Step 1062, based on the reference index values ​​of each of the multiple minimum coding blocks in the updated search area and combined with the reference index values ​​of the coding blocks in the overlapping search area, obtain the reference index values ​​of the coding blocks of various sizes generated after the search window slides. The overlapping search area is the area in the search window after sliding that overlaps with the search window before sliding.

[0120] After each slide of the search window, the reference index values ​​of the multiple minimum coding blocks in the updated search area will be calculated. In the previous slide, the reference index values ​​of the coding blocks in the overlapping search area have been calculated. Therefore, the two are combined to obtain the reference index values ​​of the coding blocks of various sizes generated after the search window slides.

[0121] In one embodiment, multiple adders are used to update the reference index values ​​of the multiple minimum coding blocks in the search area, and along the row and column directions of the first image area, combined with the reference index values ​​of the coding blocks in the overlapping search area, to obtain reference index values ​​of coding blocks of various sizes generated after the search window slides. The adder is used to add the reference index values ​​of two coding blocks of the same size to obtain the reference index value of the coding block of the new size. For example, a 4*8 adder is used to add the reference index values ​​of two coding blocks of size 4*4 to obtain the reference index value of a coding block of size 4*8. The 4*8 adder is used to output the reference index value of a coding block with a row dimension of 4 and a column dimension of 8. An 8*4 adder is used to add the reference index values ​​of two coding blocks of size 4*4 to obtain the reference index value of a coding block with a row dimension of 8 and a column dimension of 4.

[0122] With reference to FIG13 , the update search area includes four minimum coding blocks: a coding block consisting of pixel matrix 17 , a coding block consisting of pixel matrix 18 , a coding block consisting of pixel matrix 19 , and a coding block consisting of pixel matrix 20 .

[0123] Based on the reference index values ​​of the four smallest coding blocks, along the row and column directions of the image, the reference index values ​​of the overlapping search area (the shaded area in Figure 13) are two-dimensionally combined to calculate the reference index values ​​of the newly generated 4*4, 4*8, 4*16, 8*16, 16*16, 8*4, 16*4, 16*8, and 8*8 coding blocks after the search window slides. For the newly generated coding blocks of various sizes, refer to the detailed description of step 1060 above. A more detailed data reuse method for the overlapping search area will be described below.

[0124] In summary, the above embodiment uses a search window to perform a reciprocating scan in the first image region with a first step length, where the first step length is the side length of the minimum coding block, and the search window size is the size of the maximum coding block. This allows the use of data in the overlapping search area when calculating the reference index values ​​of multiple coding blocks generated by a single slide. The overlapping search area is the area in the search window after sliding that overlaps with the search window before sliding, and the data in the overlapping search area is reusable data. Given the data relationship shown in Figure 4 (the minimum coding block size is a*a, and the maximum coding block size is 4a*4a), the above scheme can achieve 75% data reuse, saving computing and storage resources and improving computing efficiency.

[0125] Furthermore, the above embodiment provides a method for reusing data in overlapping search areas, so that when calculating reference index values ​​of multiple coding blocks generated by one slide, the above method can be used to reuse data in the overlapping search areas, saving computing resources and storage resources and improving computing efficiency.

[0126] Data reuse methods for overlapping search areas include the following:

[0127] ① Through the first adder, the reference index values ​​of the two minimum coding blocks in the update search area are added together to obtain a reference index value of the first coding block, the size of the minimum coding block is a*a, and the size of the first coding block is a*2a; through the second adder, the reference index values ​​of the other two minimum coding blocks in the update search area are added together to obtain another reference index value of the first coding block, and the update search area includes four minimum coding blocks.

[0128] In conjunction with sub-figure (1) of reference Figure 14, the size of the minimum coding block is 4*4, and the updated search area includes four minimum coding blocks: a coding block composed of pixel matrix 17, a coding block composed of pixel matrix 18, a coding block composed of pixel matrix 19, and a coding block composed of pixel matrix 20.

[0129] With reference to sub-graph (2) of FIG14 , a 4*8 adder is used to add the reference index value of the coding block composed of pixel matrix 17 to the reference index value of the coding block composed of pixel matrix 18, thereby obtaining the reference index value of the coding block composed of pixel matrices 17 and 18. Another 4*8 adder is used to add the reference index value of the coding block composed of pixel matrix 19 to the reference index value of the coding block composed of pixel matrix 20, thereby obtaining the reference index value of the coding block composed of pixel matrices 19 and 20.

[0130] Refer to Figure 15, which shows the hardware computing architecture. Four 4*4 hardware computing units can be used to calculate the reference index values ​​for four 4*4 minimum coding blocks. One reference index value for one 4*4 minimum coding block corresponds to one 4*4 hardware computing unit. Two 4*8 adders are used, each 4*8 adder is used to add the reference index values ​​of two 4*4 coding blocks, resulting in a total of two reference index values ​​for 4*8 coding blocks.

[0131] ② The reference index values ​​of the two first coding blocks are added together by a third adder to obtain the reference index value of the second coding block. The size of the second coding block is a*4a.

[0132] In combination with sub-graph (3) of reference Figure 14, the reference index value of the coding block composed of pixel matrices 17 and 18 is added to the reference index value of the coding block composed of pixel matrices 19 and 20 through a 4*16 adder to obtain the reference index value of the coding block composed of pixel matrices 17-20.

[0133] With reference to FIG15 , the reference index values ​​of two 4*8 coding blocks are added together through a 4*16 adder to obtain a reference index value of a 4*16 coding block.

[0134] ③ Through the fourth adder, the reference index value of the second coding block is added to the reference index value of a coding block of size a*4a adjacent to the second coding block in the overlapping search area to obtain the reference index value of the third coding block, and the size of the third coding block is 2a*4a.

[0135] In conjunction with sub-graph (4) of reference Figure 14, an 8*16 adder is used to add the reference index value of the coding block composed of pixel lattices 17-20 to the reference index value of the coding block composed of pixel lattices 13-16 to obtain the reference index value of the coding block composed of pixel lattices 13-20.

[0136] With reference to FIG15 , the reference index values ​​of two 4*16 coding blocks are added together through an 8*16 adder to obtain a reference index value of an 8*16 coding block.

[0137] ④ Through the fifth adder, the reference index value of the third coding block is added to the reference index value of a coding block with a size of 2a*4a adjacent to the third coding block in the overlapping search area to obtain the reference index value of the largest coding block. The size of the largest coding block is 4a*4a.

[0138] In combination with sub-graph (5) of reference Figure 14, through a 16*16 adder, the reference index value of the coding block composed of pixel lattices 13-20 is added to the reference index value of the coding block composed of pixel lattices 5-12 to obtain the reference index value of the coding block composed of pixel lattices 5-20.

[0139] 15 , the reference index values ​​of two 8*16 coding blocks are added together through a 16*16 adder to obtain a reference index value of a 16*16 coding block.

[0140] ⑤ Through the sixth adder, the reference index value of the first minimum coding block is added to the reference index value of a coding block with a size of a*a adjacent to the first minimum coding block in the overlapping search area to obtain the reference index value of the first fourth coding block, where the size of the fourth coding block is 2a*a; through the seventh adder, the reference index value of the second minimum coding block is added to the reference index value of a coding block with a size of a*a adjacent to the second minimum coding block in the overlapping search area to obtain the reference index value of the second fourth coding block; through the eighth adder, the reference index value of the third minimum coding block is added to the reference index value of a coding block with a size of a*a adjacent to the third minimum coding block in the overlapping search area to obtain the reference index value of the third fourth coding block; through the ninth adder, the reference index value of the fourth minimum coding block is added to the reference index value of a coding block with a size of a*a adjacent to the fourth minimum coding block in the overlapping search area to obtain the reference index value of the fourth fourth coding block.

[0141] In combination with sub-graph (6) of reference Figure 14, through an 8*4 adder, the reference index value of the coding block composed of pixel matrix 17 is added to the reference index value of the coding block composed of pixel matrix 13 to obtain the reference index value of the coding block composed of pixel matrices 13 and 17.

[0142] Through another 8*4 adder, the reference index value of the coding block composed of pixel matrix 18 is added to the reference index value of the coding block composed of pixel matrix 14 to obtain the reference index value of the coding block composed of pixel matrices 14 and 18.

[0143] Through another 8*4 adder, the reference index value of the coding block composed of pixel matrix 19 is added to the reference index value of the coding block composed of pixel matrix 15 to obtain the reference index value of the coding block composed of pixel matrices 15 and 19.

[0144] Through another 8*4 adder, the reference index value of the coding block composed of pixel matrix 20 is added to the reference index value of the coding block composed of pixel matrix 16 to obtain the reference index value of the coding block composed of pixel matrices 16 and 20.

[0145] 15 , four 8*4 adders are used, and each 8*4 adder performs addition calculation on the reference index values ​​of two 4*4 coding blocks, thereby obtaining a total of four reference index values ​​of 8*4 coding blocks.

[0146] ⑥ Through the tenth adder, the reference index value of the first fourth coding block is added to the coding block with a size of 2a*a adjacent to the first fourth coding block in the overlapping search area to obtain the reference index value of the first fifth coding block, and the size of the fifth coding block is 4a*a; through the eleventh adder, the reference index value of the second fourth coding block is added to the coding block with a size of 2a*a adjacent to the second fourth coding block in the overlapping search area to obtain the reference index value of the second fifth coding block; through the twelfth adder, the reference index value of the third fourth coding block is added to the coding block with a size of 2a*a adjacent to the third fourth coding block in the overlapping search area to obtain the reference index value of the third fifth coding block; through the thirteenth adder, the reference index value of the fourth fourth coding block is added to the coding block with a size of 2a*a adjacent to the fourth fourth coding block in the overlapping search area to obtain the reference index value of the fourth fifth coding block.

[0147] In combination with sub-graph (7) of reference Figure 14, through a 16*4 adder, the reference index value of the coding block composed of pixel matrices 5 and 9 is added to the reference index value of the coding block composed of pixel matrices 13 and 17 to obtain the reference index value of the coding block composed of pixel matrices 5, 9, 13, and 17.

[0148] Through another 16*4 adder, the reference index value of the coding block composed of pixel matrices 6 and 10 is added to the reference index value of the coding block composed of pixel matrices 14 and 18 to obtain the reference index value of the coding block composed of pixel matrices 6, 10, 14, and 18.

[0149] Through another 16*4 adder, the reference index value of the coding block composed of pixel matrices 7 and 11 is added to the reference index value of the coding block composed of pixel matrices 15 and 19 to obtain the reference index value of the coding block composed of pixel matrices 7, 11, 15, and 19.

[0150] Through another 16*4 adder, the reference index value of the coding block composed of pixel matrices 8 and 12 is added to the reference index value of the coding block composed of pixel matrices 16 and 20 to obtain the reference index value of the coding block composed of pixel matrices 8, 12, 16, and 20.

[0151] 15 , four 16*4 adders are used, and each 16*4 adder performs addition calculation on the reference index values ​​of two 8*4 coding blocks, thereby obtaining a total of four reference index values ​​of 16*4 coding blocks.

[0152] ⑦ Through the fourteenth adder, the reference index values ​​of the two fifth coding blocks are added together to obtain the reference index value of a sixth coding block, and the size of the sixth coding block is 4a*2a; through the fifteenth adder, the reference index values ​​of the other two fifth coding blocks are added together to obtain the reference index value of another sixth coding block.

[0153] In conjunction with sub-graph (8) of reference Figure 14, through a 16*8 adder, the reference index value of the coding block composed of pixel lattices 5, 9, 13, and 17 is added to the reference index value of the coding block composed of pixel lattices 6, 10, 14, and 18 to obtain the reference index value of the coding block composed of pixel lattices 5, 9, 13, 17, 6, 10, 14, and 18.

[0154] Through another 16*8 adder, the reference index value of the coding block composed of pixel matrices 7, 11, 15, and 19 is added to the reference index value of the coding block composed of pixel matrices 8, 12, 16, and 20 to obtain the reference index value of the coding block composed of pixel matrices 7, 11, 15, 19, 8, 12, 16, and 20.

[0155] 15 , two 16*8 adders are used, and each 16*8 adder performs addition calculation on the reference index values ​​of two 16*4 coding blocks, thereby obtaining a total of two reference index values ​​of 16*8 coding blocks.

[0156] ⑧ Through the sixteenth adder, the reference index value of a first coding block is added to the reference index value of a coding block with a size of a*2a adjacent to the first coding block in the overlapping search area to obtain a reference index value of a seventh coding block, and the size of the seventh coding block is 2a*2a; through the seventeenth adder, the reference index value of another first coding block is added to the reference index value of a coding block with a size of a*2a adjacent to the other first coding block in the overlapping search area to obtain another reference index value of the seventh coding block.

[0157] In combination with sub-graph (9) of reference Figure 14, the reference index value of the coding block composed of pixel matrices 17 and 18 is added to the reference index value of the coding block composed of pixel matrices 13 and 14 through an 8*8 adder to obtain the reference index value of the coding block composed of pixel matrices 13, 14, 17, and 18.

[0158] Through another 8*8 adder, the reference index value of the coding block composed of pixel matrices 19 and 20 is added to the reference index value of the coding block composed of pixel matrices 15 and 16 to obtain the reference index value of the coding block composed of pixel matrices 15, 16, 19, and 20.

[0159] 15 , two 8*8 adders are used, and each 8*8 adder performs addition calculation on the reference index values ​​of two 4*4 coding blocks, thereby obtaining the reference index values ​​of two 8*8 coding blocks in total.

[0160] In summary, when the minimum coding block size is a*a and the maximum coding block size is 4a*4a, the above data multiplexing method only requires 4 hardware computing units and 17 adders, saving a lot of computing resources and computing time.

[0161] FIG16 shows a flowchart of a reference block search method provided by an exemplary embodiment of the present application. The method is illustrated by an example in which the method is executed by an encoding device. The method includes:

[0162] Step 1620: for each of the N square areas, for a corresponding to each square area 2 For each search window, a round of reciprocating scanning is performed on each search window in the first image region with a first step length;

[0163] The N square areas cover the complete reference area of ​​the current block to be coded as specified by the Versatile Video Coding Protocol (VVC Protocol). Referring to Figure 9 , the VVC Protocol specifies that the reference area of ​​the current block to be coded includes three complete 64*64 CUs and a coded area in a 64*64 CU. That is, the reference area of ​​the current block to be coded is covered by four square areas, and the first image area is any one of the four square areas.

[0164] In each square area, a 2 Rounds of reciprocating scanning, each round of reciprocating scanning is performed based on a search window, then a 2 The reciprocating scanning is based on a 2 A search window is executed, 2 Reciprocating scanning and a 2 There is a one-to-one correspondence between the search windows. 2 The starting points of the reciprocating scans corresponding to the search windows will form a pixel matrix, and the length and width of the pixel matrix are both the first step length a. That is, the embodiment of the present application provides a full search solution, through a 2 A search window 2 In a round-trip scanning, each pixel in the first image area will participate in the search as a vertex of the search window (the search window includes four vertices), which ensures the comprehensiveness of the search and the reference block finally determined has a high matching degree and better quality.

[0165] For example, if a is 4, there are 16 search windows. The starting points of the reciprocating scan corresponding to the 16 search windows are (0, 0), (1, 0), (2, 0), (3, 0), (0, 1), (1, 1), (2, 1), (3, 1), (0, 2), (1, 2), (2, 2), (3, 2), (0, 3), (1, 3), (2, 3), (3, 3). That is, a 2 The starting points of the reciprocating scans corresponding to the respective search windows constitute a minimum coding block.

[0166] Step 1640: For N square areas corresponding to N*a 2 A reciprocating scan is performed, and reference index values ​​of multiple coding blocks generated by each sliding of the search window during each reciprocating scan are calculated;

[0167] N square areas correspond to N*a executions 2 During a round of reciprocating scanning, the search window will slide multiple times. Assume that there are p slides. Each slide will generate multiple coding blocks. Assume that each slide generates q coding blocks, then a total of N*a 2 *p*q coding blocks. The calculation method of the reference index value of each coding block has been introduced in detail above and will not be repeated here.

[0168] Step 1660, from N*a 2 In the coding block cluster obtained by the reciprocating scanning, a reference block of the current block to be coded is determined based on the reference indicator value.

[0169] N*a 2 The coding blocks obtained by the reciprocating scanning constitute a coding block cluster. Based on the reference index value, a reference block of the current coding block can be selected from the coding block cluster. Optionally, the reference index value is SAD, and the coding block with the smallest SAD is selected from the coding block cluster as the reference block.

[0170] In summary, the embodiment of the present application provides a full search solution, through a 2 A search window 2 In a round-trip scanning, each pixel in the first image area will participate in the search as a vertex of the search window (the search window includes four vertices), which ensures the comprehensiveness of the search and the reference block finally determined has a high matching degree and better quality.

[0171] FIG17 shows a flowchart of a reference block search method provided by an exemplary embodiment of the present application. The method is illustrated by an example in which the method is executed by an encoding device. The method includes:

[0172] Step 1701, frame start;

[0173] Start searching for the reference block of the current frame to be encoded.

[0174] Step 1702, CTU starts;

[0175] The current frame to be encoded includes multiple CTUs, and a reference block search is started in the current CTU.

[0176] Step 1703, loading original data and reference data;

[0177] In one embodiment, a CTU is 128*128 in size and contains 1024 4*4 CUs. The reference block search method for a single 4*4 CU is described above, i.e., a single-path reference block search process. Alternatively, a 256-path parallel reference block search process can be performed for 256 4*4 CUs.

[0178] In a single-pass process, original data refers to pixel data of a current block to be encoded, and reference data refers to pixel data of a reference area of ​​the current block to be encoded.

[0179] Step 1704, reciprocating scanning starts;

[0180] In the embodiment of the present application, a total of 16 rounds of reciprocating scanning are performed in a single-pass process.

[0181] The first round of reciprocating scanning is recorded as ROUND 0. The second round of reciprocating scanning is recorded as ROUND 1, and so on, and the 16th round of reciprocating scanning is recorded as ROUND 15.

[0182] Step 1705, selecting a starting point according to reciprocating scanning;

[0183] The starting points of the search windows corresponding to the 16 rounds of reciprocating scanning are different, and the starting point of the search window of the current round of reciprocating scanning is selected.

[0184] Step 1706: Start reciprocating motion from the starting point in the current area and perform sliding window calculation;

[0185] The search window starts to move back and forth from the starting point in the current area and performs sliding window calculation to obtain reference index values ​​of multiple coding blocks generated by each sliding of the search window in the current round of reciprocating scanning.

[0186] Step 1707: After the sliding window calculation is completed, the number of rounds of reciprocating scanning is increased by 1;

[0187] After the sliding window calculation is completed, the number of rounds of reciprocating scanning is increased by 1;

[0188] Step 1708, determining whether the number of rounds of reciprocating scanning is greater than 15;

[0189] Determine whether the round of reciprocating scanning is greater than 15. If so, execute step 1709; if not, execute step 1704.

[0190] Step 1709, determine whether encoding has reached the last CTU;

[0191] Determine whether the current encoding has reached the last CTU. If so, execute step 1710; if not, execute step 1702.

[0192] Step 1710, the frame ends.

[0193] Reference block search has been performed on all CUs to be encoded in the current frame.

[0194] In this application, the compression rate performance test of the above scheme was carried out. The test results show that although the size of the supported coding blocks is limited, the full search method is superior to the original hash and pattern search methods. Therefore, the PSNR (Peak Signal to Noise Ratio), SSIM (Structural Similarity Index), and VMAF (Video Muiti Method Assessment Fusion) all have significant performance improvements (negative values ​​represent positive benefits).

[0195] In terms of hardware architecture, this application uses 256-way parallel computing, and each way performs search calculations in the above manner, which can achieve a processing performance of 1080p@144fps at 1GHz.

[0196] FIG18 shows a reference block search circuit 1800 provided by an exemplary embodiment of the present application. The circuit includes a first circuit unit 1801 , a second circuit unit 1802 , a third circuit unit 1803 and a fourth circuit unit 1804 ; the four circuit units are connected in series via wires.

[0197] The first circuit unit 1801 is configured to obtain a first image area, where the first image area covers a portion or the entire reference area of ​​the current block to be encoded;

[0198] A second circuit unit 1802 is configured to perform a reciprocating scan in the first image region using a search window with a first step length, where the first step length is the side length of the minimum coding block, the size of the search window is the size of the maximum coding block, the minimum coding block is the coding block with the smallest size, and the maximum coding block is the coding block with the largest size;

[0199] The third circuit unit 1803 is configured to calculate reference index values ​​of the plurality of coding blocks generated by each sliding of the search window during the reciprocating scanning process;

[0200] The fourth circuit unit 1804 is configured to determine a reference block for the current block to be coded according to the reference index value of the coding block generated during the reciprocating scanning process.

[0201] In an optional embodiment, the third circuit unit 1803 is further configured to calculate, during the reciprocating scanning process, after each sliding of the search window, reference index values ​​of the plurality of minimum coding blocks in an updated search area, where the updated search area is an area of ​​the search window after sliding that does not overlap with the search window before sliding.

[0202] The third circuit unit 1803 is further used to obtain reference index values ​​of coding blocks of various sizes generated after the search window slides based on the reference index values ​​of each of the multiple minimum coding blocks in the updated search area and in combination with the reference index values ​​of the coding blocks in the overlapping search area. The overlapping search area is the area in the search window after sliding that overlaps with the search window before sliding.

[0203] In an optional embodiment, the third circuit unit 1803 includes a hardware computing units 1805. The a hardware computing units 1805 are configured to calculate reference index values ​​for each of the a minimum coding blocks in the updated search area in the sliding search window. The a hardware computing units correspond one-to-one to the a minimum coding blocks, where a is a positive integer.

[0204] In an optional embodiment, the third circuit unit includes a plurality of adders 1806. The plurality of adders 1806 is configured to obtain, based on the reference index values ​​of the plurality of minimum coding blocks in the updated search area, reference index values ​​of coding blocks of various sizes generated after the search window slides along the row and column directions of the first image area in combination with the reference index values ​​of coding blocks in the overlapping search area.

[0205] In an optional embodiment, the plurality of adders 1806 includes first to seventeenth adders.

[0206] a first adder, configured to add the reference index values ​​of the two smallest coding blocks in the update search area to obtain a reference index value of a first coding block, where the size of the smallest coding block is a*a and the size of the first coding block is a*2a; and a second adder, configured to add the reference index values ​​of the other two smallest coding blocks in the update search area to obtain a reference index value of another first coding block, where the update search area includes four smallest coding blocks;

[0207] A third adder is used to add the reference index values ​​of the two first coding blocks to obtain the reference index value of the second coding block, where the size of the second coding block is a*4a;

[0208] a fourth adder, configured to add the reference index value of the second coding block to the reference index value of a coding block of a size of a*4a adjacent to the second coding block in the overlapping search area to obtain a reference index value of a third coding block, where the size of the third coding block is 2a*4a;

[0209] a fifth adder, configured to add the reference index value of the third coding block to the reference index value of a coding block of size 2a*4a adjacent to the third coding block in the overlapping search area to obtain a reference index value of a largest coding block, where the size of the largest coding block is 4a*4a;

[0210] a sixth adder, configured to add the reference index value of the first minimum coding block to the reference index value of a coding block of size a*a adjacent to the first minimum coding block in the overlapping search area, to obtain the reference index value of the first fourth coding block, where the size of the fourth coding block is 2a*a; a seventh adder, configured to add the reference index value of the second minimum coding block to the reference index value of a coding block of size a*a adjacent to the second minimum coding block in the overlapping search area, to obtain the reference index value of the second fourth coding block; an eighth adder, configured to add the reference index value of the third minimum coding block to the reference index value of a coding block of size a*a adjacent to the third minimum coding block in the overlapping search area, to obtain the reference index value of the third fourth coding block; a ninth adder, configured to add the reference index value of the fourth minimum coding block to the reference index value of a coding block of size a*a adjacent to the fourth minimum coding block in the overlapping search area, to obtain the reference index value of the fourth fourth coding block;

[0211] a tenth adder, configured to add the reference index value of the first fourth coding block to a coding block of size 2a*a adjacent to the first fourth coding block in the overlapping search area, to obtain the reference index value of the first fifth coding block, where the size of the fifth coding block is 4a*a; an eleventh adder, configured to add the reference index value of the second fourth coding block to a coding block of size 2a*a adjacent to the second fourth coding block in the overlapping search area, to obtain the reference index value of the second fifth coding block; a twelfth adder, configured to add the reference index value of the third fourth coding block to a coding block of size 2a*a adjacent to the third fourth coding block in the overlapping search area, to obtain the reference index value of the third fifth coding block; a thirteenth adder, configured to add the reference index value of the fourth fourth coding block to a coding block of size 2a*a adjacent to the fourth fourth coding block in the overlapping search area, to obtain the reference index value of the fourth fifth coding block;

[0212] a fourteenth adder, configured to add the reference index values ​​of the two fifth coding blocks to obtain a reference index value of a sixth coding block, where the size of the sixth coding block is 4a*2a; a fifteenth adder, configured to add the reference index values ​​of the other two fifth coding blocks to obtain a reference index value of another sixth coding block;

[0213] The sixteenth adder is used to add the reference index value of a first coding block and the reference index value of a coding block with a size of a*2a adjacent to the first coding block in the overlapping search area to obtain a reference index value of a seventh coding block, where the size of the seventh coding block is 2a*2a; the seventeenth adder is used to add the reference index value of another first coding block and the reference index value of a coding block with a size of a*2a adjacent to the other first coding block in the overlapping search area to obtain a reference index value of another seventh coding block.

[0214] In an optional embodiment, the size of the first image area is 64*64, the size of the minimum coding block is 4*4, and the size of the maximum coding block is 16*16; or, the size of the first image area is 64*64, the size of the minimum coding block is 8*8, and the size of the maximum coding block is 32*32, all in pixels.

[0215] In an optional embodiment, the search window is a 2 Any search window in the search windows, a 2 The starting points of the reciprocating scans corresponding to each search window form a pixel matrix, and the length and width of the pixel matrix are both the first step length a;

[0216] The second circuit unit 1802 is also used for a 2For each search window, a round of reciprocating scanning is performed on each search window in the first image region with a first step length;

[0217] The third circuit unit 1803 is also used for a 2 a corresponding to the search window 2 A reciprocating scan is performed, and reference index values ​​of multiple coding blocks generated by each sliding of the search window during each reciprocating scan are calculated.

[0218] In an optional embodiment, the first image area is any one of N square areas, and the N square areas cover a complete reference area of ​​the current block to be encoded as specified by the multifunctional video coding protocol;

[0219] The second circuit unit 1802 is further configured to: for each square area in the N square areas, for a corresponding to each square area 2 For each search window, a round of reciprocating scanning is performed on each search window in the first image region with a first step length;

[0220] The third circuit unit 1803 is also used for N*a corresponding to the N square areas. 2 A reciprocating scan is performed, and reference index values ​​of multiple coding blocks generated by each sliding of the search window during each reciprocating scan are calculated;

[0221] The fourth circuit unit 1804 is further configured to generate a 2 In the coding block cluster obtained by the reciprocating scanning, a reference block of the current block to be coded is determined based on the reference indicator value.

[0222] It should be noted that for a more detailed description of the search circuit of the reference block, please refer to the above method embodiment. The relevant content introduced in the above method embodiment can be transformed into execution by a corresponding integrated circuit, which can be added to the embodiment of the search circuit of the reference block.

[0223] FIG19 shows a block diagram of a reference block search apparatus provided by an exemplary embodiment of the present application, the apparatus comprising:

[0224] An acquisition module 1901 is configured to acquire a first image region, where the first image region covers a portion or the entirety of a reference region of a current block to be encoded;

[0225] A scanning module 1902 is configured to perform a reciprocating scan in a first image region using a search window with a first step length, where the first step length is the side length of a minimum coding block, and the size of the search window is the size of a maximum coding block, where the minimum coding block is the coding block with the smallest size, and the maximum coding block is the coding block with the largest size;

[0226] A calculation module 1903 is configured to calculate reference index values ​​of respective coding blocks generated by each sliding of the search window during the reciprocating scanning process;

[0227] The determination module 1904 is configured to determine a reference block of the current block to be encoded according to the reference index value of the encoding block generated during the reciprocating scanning process.

[0228] In an optional embodiment, the calculation module 1903 is further used to calculate the reference index values ​​of multiple minimum coding blocks in the updated search area after each sliding of the search window during the reciprocating scanning process, where the updated search area is the area in the search window after sliding that does not overlap with the search window before sliding; based on the reference index values ​​of the multiple minimum coding blocks in the updated search area, combined with the reference index values ​​of the coding blocks in the overlapping search area, the reference index values ​​of the coding blocks of various sizes generated after the search window slides are obtained, where the overlapping search area is the area in the search window after sliding that overlaps with the search window before sliding.

[0229] In an optional embodiment, the calculation module 1903 is further used to calculate the reference index values ​​of the a minimum coding blocks in the updated search area in the sliding search window through a hardware computing units, and the a hardware computing units correspond one-to-one to the a minimum coding blocks.

[0230] In an optional embodiment, the calculation module 1903 is also used to obtain, through multiple adders, reference index values ​​of coding blocks of various sizes generated after the search window slides, based on the reference index values ​​of each of the multiple minimum coding blocks in the updated search area, along the row and column directions of the first image area, combined with the reference index values ​​of the coding blocks in the overlapping search area.

[0231] In an optional embodiment, the calculation module 1903 is further configured to add, by a first adder, the respective reference index values ​​of two minimum coding blocks in the updated search area to obtain a reference index value of a first coding block, where the size of the minimum coding block is a*a, and the size of the first coding block is a*2a; and add, by a second adder, the respective reference index values ​​of another two minimum coding blocks in the updated search area to obtain a reference index value of another first coding block, where the updated search area includes four minimum coding blocks.

[0232] Adding the reference index values ​​of the two first coding blocks together through a third adder to obtain a reference index value of a second coding block, where the size of the second coding block is a*4a;

[0233] Adding, by a fourth adder, the reference index value of the second coding block and the reference index value of a coding block of size a*4a adjacent to the second coding block in the overlapping search area to obtain a reference index value of a third coding block, where the size of the third coding block is 2a*4a;

[0234] Adding, by a fifth adder, the reference index value of the third coding block and the reference index value of a coding block of size 2a*4a adjacent to the third coding block in the overlapping search area to obtain a reference index value of a largest coding block, where the size of the largest coding block is 4a*4a;

[0235] The sixth adder adds the reference index value of the first minimum coding block and the reference index value of a coding block of size a*a adjacent to the first minimum coding block in the overlapping search area to obtain the reference index value of the first fourth coding block, where the size of the fourth coding block is 2a*a; the seventh adder adds the reference index value of the second minimum coding block and the reference index value of a coding block of size a*a adjacent to the second minimum coding block in the overlapping search area to obtain the reference index value of the second fourth coding block; the eighth adder adds the reference index value of the third minimum coding block and the reference index value of a coding block of size a*a adjacent to the third minimum coding block in the overlapping search area to obtain the reference index value of the third fourth coding block; the ninth adder adds the reference index value of the fourth minimum coding block and the reference index value of a coding block of size a*a adjacent to the fourth minimum coding block in the overlapping search area to obtain the reference index value of the fourth fourth coding block;

[0236] By means of a tenth adder, the reference index value of the first fourth coding block is added to a coding block of size 2a*a adjacent to the first fourth coding block in the overlapping search area to obtain a reference index value of the first fifth coding block, where the size of the fifth coding block is 4a*a; by means of an eleventh adder, the reference index value of the second fourth coding block is added to a coding block of size 2a*a adjacent to the second fourth coding block in the overlapping search area to obtain a reference index value of the second fifth coding block; by means of a twelfth adder, the reference index value of the third fourth coding block is added to a coding block of size 2a*a adjacent to the third fourth coding block in the overlapping search area to obtain a reference index value of the third fifth coding block; by means of a thirteenth adder, the reference index value of the fourth fourth coding block is added to a coding block of size 2a*a adjacent to the fourth fourth coding block in the overlapping search area to obtain a reference index value of the fourth fifth coding block;

[0237] The reference index values ​​of the two fifth coding blocks are added together by a fourteenth adder to obtain a reference index value of a sixth coding block, where the size of the sixth coding block is 4a*2a. The reference index values ​​of the other two fifth coding blocks are added together by a fifteenth adder to obtain another reference index value of the sixth coding block.

[0238] The reference index value of a first coding block is added to the reference index value of a coding block with a size of a*2a adjacent to the first coding block in the overlapping search area through the sixteenth adder to obtain a reference index value of a seventh coding block, where the size of the seventh coding block is 2a*2a; the reference index value of another first coding block is added to the reference index value of a coding block with a size of a*2a adjacent to the other first coding block in the overlapping search area through the seventeenth adder to obtain another reference index value of the seventh coding block.

[0239] In an optional embodiment, the size of the first image area is 64*64, the size of the minimum coding block is 4*4, and the size of the maximum coding block is 16*16; or, the size of the first image area is 64*64, the size of the minimum coding block is 8*8, and the size of the maximum coding block is 32*32, all in pixels.

[0240] In an optional embodiment, the search window is a 2 Any search window in the search windows, a 2 The starting points of the reciprocating scans corresponding to the respective search windows form a pixel matrix, and the length and width of the pixel matrix are both the first step length a. The scanning module 1902 is also used for a 2 Each search window in the search windows is scanned back and forth in the first image region with the first step length. The calculation module 1903 is also used for a 2 a corresponding to the search window 2 A reciprocating scan is performed, and reference index values ​​of multiple coding blocks generated by each sliding of the search window during each reciprocating scan are calculated.

[0241] In an optional embodiment, the first image area is any one of the N square areas, and the N square areas cover the complete reference area of ​​the current block to be coded as specified by the multifunctional video coding protocol. The scanning module 1902 is further configured to scan each square area in the N square areas for a corresponding 2 Each search window in the N search windows is scanned back and forth in the first image region with the first step length. The calculation module 1903 is also used to calculate N*a corresponding to the N square regions. 2A reciprocating scan is performed, and reference index values ​​of multiple coding blocks generated by each sliding of the search window during each reciprocating scan are calculated.

[0242] The determination module 1904 is further configured to determine the 2 In the coding block cluster obtained by the reciprocating scanning, a reference block of the current block to be coded is determined based on the reference indicator value.

[0243] In summary, a search window is used to perform a reciprocating scan in the first image region with a first step length, where the first step length is the side length of the smallest coding block and the size of the search window is the size of the largest coding block. Reference index values ​​are calculated for each of the multiple coding blocks generated by each sliding movement of the search window during the reciprocating scan. These reference index values ​​are used to determine a reference block for the current block to be coded. In other words, this application provides a new reference block search method based on reciprocating scanning.

[0244] FIG20 shows a block diagram of a computer device 2000 provided in accordance with an exemplary embodiment of the present application. The computer device 2000 may be a portable mobile terminal, such as a smartphone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer, or a desktop computer. The computer device 2000 may also be referred to as a user device, a portable terminal, a laptop terminal, a desktop terminal, or other similar names. Typically, the computer device 2000 includes a processor 2001 and a memory 2002.

[0245] Processor 2001 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 2001 may be implemented in at least one of the following hardware forms: a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), or a PLA (Programmable Logic Array). Processor 2001 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, processor 2001 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the display screen. In some embodiments, processor 2001 may also include an AI (Artificial Intelligence) processor, which is responsible for processing computing operations related to machine learning. In some embodiments, processor 2001 includes the reference block search circuit described in the above embodiments.

[0246] The memory 2002 may include one or more computer-readable storage media, which may be non-transitory, and may also include high-speed random access memory and non-volatile memory, such as one or more magnetic disk storage devices and flash memory storage devices.

[0247] In some embodiments, computer device 2000 may optionally include a peripheral device interface 2003 and at least one peripheral device. Processor 2001, memory 2002, and peripheral device interface 2003 may be connected via a bus or signal lines. Each peripheral device may be connected to peripheral device interface 2003 via a bus, signal lines, or a circuit board. Those skilled in the art will appreciate that the structure shown in FIG. 20 does not limit computer device 2000 and may include more or fewer components than shown, combine certain components, or employ a different component arrangement.

[0248] According to one aspect of the present application, a processor is provided, comprising the reference block search circuit provided by the above embodiment.

[0249] According to one aspect of the present application, a chip is provided, the chip including a processor, the processor including the reference block search circuit provided by the above embodiment.

[0250] According to one aspect of the present application, a computer device is provided. The computer device includes a processor. The processor includes the reference block search circuit provided by the above embodiment.

[0251] According to one aspect of the present application, a video encoding method is provided, the method comprising: encoding a current encoding block according to a reference block determined by the above-mentioned reference block determination method to obtain a video code stream.

[0252] According to one aspect of the present application, a video decoding method is provided, the method comprising: decoding a video code stream, where the video code stream is obtained by encoding a current coding block using a reference block determined according to the reference block determination method.

[0253] According to one aspect of the present application, a video encoding device is provided, which includes an encoding module, which is used to encode a current encoding block according to a reference block determined by the above-mentioned reference block determination method to obtain a video code stream.

[0254] According to one aspect of the present application, a video decoding device is provided, which includes a decoding module for decoding a video code stream, which is obtained by encoding a current coding block based on a reference block determined according to the above-mentioned reference block determination method.

[0255] According to one aspect of the present application, a method for storing or sending a video code stream is provided, wherein the video code stream is generated according to the above-mentioned video encoding method, or the video code stream can be decoded based on the above-mentioned video decoding method.

[0256] According to one aspect of the present application, a computer storage medium is provided, storing instructions, wherein the instructions can be executed by at least one processor to implement the above-mentioned video encoding method, generate a video code stream, and store it.

[0257] According to one aspect of the present application, a computer device is provided, comprising a processor and a memory, wherein the memory stores instructions, and the processor loads and executes the instructions to implement the above-mentioned video encoding method or the above-mentioned video decoding method.

Claims

1. A search method for reference blocks, applied to a processor, the method comprising: Obtaining a first image region, where the first image region covers a partial region or all regions of the referenceable region of a current block to be encoded; Performing a reciprocating scan in the first image region with a first step size through a search window, where the first step size is the side length of the smallest coding block, the size of the search window is the size of the largest coding block, the smallest coding block is the coding block with the smallest size, and the largest coding block is the coding block with the largest size; Calculating respective reference index values of multiple coding blocks generated each time the search window slides during the reciprocating scan; Determining a reference block of the current block to be encoded according to the reference index values of the coding blocks generated during the reciprocating scan.

2. The method according to claim 1, wherein The calculating respective reference index values of multiple coding blocks generated each time the search window slides during the reciprocating scan includes: During the reciprocating scan, after each slide of the search window, calculating respective reference index values of a plurality of the smallest coding blocks in an updated search region, where the updated search region is a region in the slid search window that does not overlap with the search window before sliding; Based on the respective reference index values of a plurality of the smallest coding blocks in the updated search region, combining the reference index values of the coding blocks in an overlapping search region to obtain respective reference index values of coding blocks of various sizes generated after the search window slides, where the overlapping search region is a region in the slid search window that overlaps with the search window before sliding.

3. The method according to claim 2, wherein The calculating respective reference index values of a plurality of the smallest coding blocks in the updated search region includes: Calculating respective reference index values of a smallest coding blocks in the updated search region through a hardware computing units, where the a hardware computing units correspond to the a smallest coding blocks one by one, and a is a positive integer.

4. The method according to claim 2 or 3, wherein, The based on the respective reference index values of a plurality of the smallest coding blocks in the updated search region, combining the reference index values of the coding blocks in the overlapping search region to obtain respective reference index values of coding blocks of various sizes generated after the search window slides includes: Through a plurality of adders, based on the respective reference index values of a plurality of the smallest coding blocks in the updated search region, along the row direction and column direction of the first image region, combining the reference index values of the coding blocks in the overlapping search region to obtain respective reference index values of coding blocks of various sizes generated after the search window slides.

5. The method according to claim 4, wherein, The through a plurality of adders, based on the respective reference index values of a plurality of the smallest coding blocks in the updated search region, along the row direction and column direction of the first image region, combining the reference index values of the coding blocks in the overlapping search region to obtain respective reference index values of coding blocks of various sizes generated after the search window slides includes: Through a first adder, the reference index values of two smallest coding blocks in the updated search area are added respectively to obtain the reference index value of a first coding block, the size of the smallest coding block being a*a, and the size of the first coding block being a*2a; through a second adder, the reference index values of another two smallest coding blocks in the updated search area are added respectively to obtain the reference index value of another first coding block, the updated search area including four smallest coding blocks; Through a third adder, the reference index values of two first coding blocks are added respectively to obtain the reference index value of a second coding block, the size of the second coding block being a*4a; Through a fourth adder, the reference index value of the second coding block is added to the reference index value of a coding block with a size of a*4a adjacent to the second coding block in the overlapping search area to obtain the reference index value of a third coding block, the size of the third coding block being 2a*4a; Through a fifth adder, the reference index value of the third coding block is added to the reference index value of a coding block with a size of 2a*4a adjacent to the third coding block in the overlapping search area to obtain the reference index value of the largest coding block, the size of the largest coding block being 4a*4a; Through a sixth adder, the reference index value of the first smallest coding block is added to the reference index value of a coding block with a size of a*a adjacent to the first smallest coding block in the overlapping search area to obtain the reference index value of a first fourth coding block, the size of the fourth coding block being 2a*a; through a seventh adder, the reference index value of the second smallest coding block is added to the reference index value of a coding block with a size of a*a adjacent to the second smallest coding block in the overlapping search area to obtain the reference index value of a second fourth coding block; through an eighth adder, the reference index value of the third smallest coding block is added to the reference index value of a coding block with a size of a*a adjacent to the third smallest coding block in the overlapping search area to obtain the reference index value of a third fourth coding block; through a ninth adder, the reference index value of the fourth smallest coding block is added to the reference index value of a coding block with a size of a*a adjacent to the fourth smallest coding block in the overlapping search area to obtain the reference index value of a fourth fourth coding block; Through the tenth adder, add the reference index value of the first fourth coding block to a coding block with a size of 2a*a adjacent to the first fourth coding block in the overlapping search area to obtain the reference index value of the first fifth coding block, where the size of the fifth coding block is 4a*a; through the eleventh adder, add the reference index value of the second fourth coding block to a coding block with a size of 2a*a adjacent to the second fourth coding block in the overlapping search area to obtain the reference index value of the second fifth coding block; through the twelfth adder, add the reference index value of the third fourth coding block to a coding block with a size of 2a*a adjacent to the third fourth coding block in the overlapping search area to obtain the reference index value of the third fifth coding block; through the thirteenth adder, add the reference index value of the fourth fourth coding block to a coding block with a size of 2a*a adjacent to the fourth fourth coding block in the overlapping search area to obtain the reference index value of the fourth fifth coding block; Through the fourteenth adder, add the reference index values of the two fifth coding blocks respectively to obtain the reference index value of a sixth coding block, where the size of the sixth coding block is 4a*2a; through the fifteenth adder, add the reference index values of the other two fifth coding blocks respectively to obtain the reference index value of another sixth coding block; Through the sixteenth adder, add the reference index value of a first coding block to the reference index value of a coding block with a size of a*2a adjacent to the first coding block in the overlapping search area to obtain the reference index value of a seventh coding block, where the size of the seventh coding block is 2a*2a; through the seventeenth adder, add the reference index value of another first coding block to the reference index value of a coding block with a size of a*2a adjacent to the other first coding block in the overlapping search area to obtain the reference index value of another seventh coding block.

6. According to the method of any one of claims 1 to 5, wherein The size of the first image area is 64*64, the size of the smallest coding block is 4*4, and the size of the largest coding block is 16*16; Alternatively, the size of the first image area is 64*64, the size of the smallest coding block is 8*8, and the size of the largest coding block is 32*32, with the unit being pixels.

7. According to the method of any one of claims 1 to 6, wherein The search window is any one of a 2 search windows, and the a 2 search windows correspond one-to-one with a 2 round of reciprocating scanning, and the starting points of the reciprocating scanning corresponding to each of the a 2 search windows form a pixel lattice, and the length and width of the pixel lattice are both the first step length a; The reciprocating scan of the search window in the first image area with a first step size includes: For the a 2 For each of the search windows, perform one round of the reciprocating scan on the first image area with the first step length for each of the search windows; Calculating the reference index values of the multiple coding blocks generated each time the search window slides during the reciprocating scan includes: For the said a 2 a corresponding to the search window 2 Round-trip scanning is performed, and the reference index values of multiple coded blocks generated each time the search window slides during each round-trip scanning are calculated.

8. The method according to claim 7, wherein The first image area is any one of the N square areas, and the N square areas cover the complete reference area of the current block to be coded specified by the multi-functional video coding protocol; For the said a 2 For each of the 2 search windows, perform one round of the reciprocating scan on the first image area with the first step length for each of the search windows, including: For each of the N square regions, for each of the a 2 search windows corresponding to each of the square regions, perform one round of the reciprocating scan on the first image region with the first step size for each of the search windows; For the said a 2 a corresponding to the search window 2 Round-trip scanning, calculating the respective reference index values of multiple coding blocks generated each time the search window slides during each round-trip scanning, including: For the N*a corresponding to the N square regions 2 For the round-trip scanning, calculate the respective reference index values of the multiple coded blocks generated each time the search window slides during each round of round-trip scanning; Determining the reference block of the current block to be coded according to the reference index values of the coding blocks generated during the reciprocating scan includes: From the N*a 2 Among the coded block clusters obtained by the wheel reciprocating scan, based on the reference index value, determine the reference block of the current block to be coded.

9. A search circuit for reference blocks, the circuit comprising a first circuit unit, a second circuit unit, a third circuit unit, and a fourth circuit unit; the four circuit units are connected in series through wires; The first circuit unit is configured to obtain a first image region, where the first image region covers a partial region or all regions of the referenceable region of the current block to be encoded; The second circuit unit is configured to perform a reciprocating scan in the first image region with a first step length through a search window, where the first step length is the side length of the smallest coding block, the size of the search window is the size of the largest coding block, the smallest coding block is the coding block with the smallest size, and the largest coding block is the coding block with the largest size; The third circuit unit is configured to calculate the respective reference index values of a plurality of coding blocks generated each time the search window slides during the reciprocating scan; The fourth circuit unit is configured to determine the reference block of the current block to be encoded according to the reference index values of the coding blocks generated during the reciprocating scan.

10. The circuit according to claim 9, wherein, The third circuit unit is further configured to, during the reciprocating scan, after each slide of the search window, calculate the respective reference index values of a plurality of the smallest coding blocks in the updated search region, where the updated search region is the region in the slid search window that does not overlap with the search window before sliding; The third circuit unit is further configured to, based on the respective reference index values of a plurality of the smallest coding blocks in the updated search region, and in combination with the reference index values of the coding blocks in the overlapping search region, obtain the reference index values of the coding blocks of various sizes generated after the search window slides, where the overlapping search region is the region in the slid search window that overlaps with the search window before sliding.

11. The circuit according to claim 10, wherein, The third circuit unit includes a hardware computing units; The a hardware computing units are configured to calculate the respective reference index values of a smallest coding blocks in the updated search region, and the a hardware computing units correspond to the a smallest coding blocks one by one, and a is a positive integer.

12. The circuit according to claim 10 or 11, wherein, The third circuit unit includes a plurality of adders; The plurality of adders are configured to, based on the respective reference index values of a plurality of the smallest coding blocks in the updated search region, along the row direction and column direction of the first image region, and in combination with the reference index values of the coding blocks in the overlapping search region, obtain the reference index values of the coding blocks of various sizes generated after the search window slides.

13. The circuit according to claim 12, wherein, The plurality of adders include a first to seventeenth adder; The first adder is configured to add the respective reference index values of two smallest coding blocks in the updated search region to obtain the reference index value of a first coding block, where the size of the smallest coding block is a*a, and the size of the first coding block is a*2a; the second adder is configured to add the respective reference index values of another two smallest coding blocks in the updated search region to obtain the reference index value of another first coding block, and the updated search region includes four smallest coding blocks; A third adder, configured to add the respective reference index values of two first coding blocks to obtain the reference index value of a second coding block, where the size of the second coding block is a * 4a; A fourth adder, configured to add the reference index value of the second coding block and the reference index value of a coding block with a size of a * 4a adjacent to the second coding block in the overlapping search area to obtain the reference index value of a third coding block, where the size of the third coding block is 2a * 4a; A fifth adder, configured to add the reference index value of the third coding block and the reference index value of a coding block with a size of 2a * 4a adjacent to the third coding block in the overlapping search area to obtain the reference index value of the largest coding block, where the size of the largest coding block is 4a * 4a; A sixth adder, configured to add the reference index value of the first smallest coding block and the reference index value of a coding block with a size of a * a adjacent to the first smallest coding block in the overlapping search area to obtain the reference index value of a first fourth coding block, where the size of the fourth coding block is 2a * a; A seventh adder, configured to add the reference index value of the second smallest coding block and the reference index value of a coding block with a size of a * a adjacent to the second smallest coding block in the overlapping search area to obtain the reference index value of a second fourth coding block; an eighth adder, configured to add the reference index value of the third smallest coding block and the reference index value of a coding block with a size of a * a adjacent to the third smallest coding block in the overlapping search area to obtain the reference index value of a third fourth coding block; a ninth adder, configured to add the reference index value of the fourth smallest coding block and the reference index value of a coding block with a size of a * a adjacent to the fourth smallest coding block in the overlapping search area to obtain the reference index value of a fourth fourth coding block; A tenth adder, configured to add the reference index value of the first fourth coding block and a coding block with a size of 2a * a adjacent to the first fourth coding block in the overlapping search area to obtain the reference index value of a first fifth coding block, where the size of the fifth coding block is 4a * a; an eleventh adder, configured to add the reference index value of the second fourth coding block and a coding block with a size of 2a * a adjacent to the second fourth coding block in the overlapping search area to obtain the reference index value of a second fifth coding block; a twelfth adder, configured to add the reference index value of the third fourth coding block and a coding block with a size of 2a * a adjacent to the third fourth coding block in the overlapping search area to obtain the reference index value of a third fifth coding block; a thirteenth adder, configured to add the reference index value of the fourth fourth coding block and a coding block with a size of 2a * a adjacent to the fourth fourth coding block in the overlapping search area to obtain the reference index value of a fourth fifth coding block; The fourteenth adder is used to add the respective reference index values of two fifth coding blocks to obtain the reference index value of a sixth coding block, and the size of the sixth coding block is 4a*2a; the fifteenth adder is used to add the respective reference index values of another two fifth coding blocks to obtain the reference index value of another sixth coding block. The sixteenth adder is used to add the reference index value of a first coding block and the reference index value of a coding block with a size of a*2a adjacent to the first coding block in the overlapping search area to obtain the reference index value of a seventh coding block, and the size of the seventh coding block is 2a*2a; the seventeenth adder is used to add the reference index value of another first coding block and the reference index value of a coding block with a size of a*2a adjacent to the other first coding block in the overlapping search area to obtain the reference index value of another seventh coding block.

14. The circuit according to any one of claims 9 to 13, wherein, The size of the first image area is 64*64, the size of the smallest coding block is 4*4, and the size of the largest coding block is 16*16. Alternatively, the size of the first image area is 64*64, the size of the smallest coding block is 8*8, and the size of the largest coding block is 32*32, with the unit being pixels.

15. The circuit according to any one of claims 9 to 13, wherein, The search window is any one of a 2 search windows, and the a 2 search windows correspond one-to-one with a 2 round of reciprocating scanning, and the starting points of the reciprocating scanning corresponding to each of the a 2 search windows form a pixel lattice, and the length and width of the pixel lattice are both the first step length a; The second circuit unit is further configured to, for each of the 2 search windows, perform one round of the reciprocating scan on each of the search windows in the first image area with the first step size; The third circuit unit is further configured to, for 2 a number of search windows corresponding to a 2 rounds of reciprocating scans, calculate the respective reference index values of multiple coding blocks generated each time the search window slides during each round of reciprocating scans.

16. The circuit according to claim 15, wherein, The first image area is any one of N square areas, and the N square areas cover the complete reference area of the current coding block to be coded as specified by the multi-functional video coding protocol. The second circuit unit is further configured to, for each of the N square regions, for each of the a 2 search windows corresponding to each square region, perform one round of the reciprocating scan on the first image region with the first step size for each of the search windows; The third circuit unit is further configured to, for the N*a 2 rounds of reciprocating scanning, calculate the respective reference index values of the multiple coding blocks generated each time the search window slides during each round of reciprocating scanning; The fourth circuit unit is further configured to obtain from the N*a 2 cluster of coded blocks obtained by the round-trip scan, and determine a reference block of the current block to be coded based on the reference metric value.

17. A chip, the chip includes a processor, and the processor includes a search circuit for a reference block as described in any one of claims 9 to 16.

18. A computer device, the computer device includes a processor, and the processor includes a search circuit for a reference block as described in any one of claims 9 to 16.

19. A video encoding method, the method comprising: Encoding the current coding block to obtain a video bitstream using the reference block determined by the method according to any one of claims 1 to 8.

20. A video decoding method, the method comprising: Decoding a video bitstream, where the video bitstream is obtained by encoding the current coding block using the reference block determined by the method according to any one of claims 1 to 8.

21. A video coding device, the device includes an encoding module, and the encoding module is used to encode the current coding block to obtain a video bitstream using the reference block determined by the method according to any one of claims 1 to 8.

22. A video decoding device, the device includes a decoding module, and the decoding module is used to decode a video bitstream, where the video bitstream is obtained by encoding the current coding block using the reference block determined by the method according to any one of claims 1 to 8.

23. A method for storing or transmitting a video bitstream, where the video bitstream is generated by the video coding method according to claim 19, or the video bitstream can be decoded based on the video decoding method according to claim 20.

24. A computer storage medium storing instructions that can be executed by at least one processor to implement the video coding method according to claim 19, generate a video bitstream and store it.

25. A computer device, comprising a processor and a memory, wherein instructions are stored in the memory, and the processor loads and executes the instructions to implement at least one of the video encoding method according to claim 19 and the video decoding method according to claim 20.

Images