Disparity vector calculation method and device, parallax vector prediction method and device, equipment and storage medium

Abstract

The invention discloses a parallax vector calculation method, a parallax vector prediction method, a parallax vector prediction device, parallax vector prediction equipment and a storage medium, which can improve the derivation process of a parallax vector and improve the coding efficiency. The method comprises the following steps: determining a parallax range through a depth value of a current coding block, determining a candidate block number and a candidate block set of a reference frame corresponding to a parallax domain through the parallax range, calculating a first parallax value of each candidate block corresponding to the current coding block, and finding a candidate block with the minimum parallax as a target candidate block, when the depth of the first parallax domain candidate block is correspondingly converted into the difference of the second parallax value, the second parallax value is used as the parallax vector, so that limitation of the same depth value is avoided, and a more accurate parallax vector can be obtained; according to the embodiment of the invention, the application effect of enhancing the motion compensation block and the inter-view corresponding block is achieved; the calculation complexity and memory occupation are reduced, the adaptability in a complex scene is improved, and the coding performance is remarkably improved.

Description

Technical Field

[0001] This application belongs to the field of video processing technology, and in particular relates to a disparity vector calculation method, prediction method, device, equipment and storage medium. Background Technology

[0002] With the continuous advancement of display technology, the refresh rate of 3D video systems has been continuously improved. However, factors affecting the playback effect of 3D video include residual prediction. Existing residual prediction methods include basic residual prediction, inter-view residual prediction (IVRP), and advanced residual prediction (ARP). Basic residual prediction generates residuals by calculating the difference between the current block and the reference block. It has the advantages of simple implementation and low computational complexity, and is suitable for scenarios with limited computing resources and low compression efficiency requirements. IVRP utilizes the decoded residual information of adjacent views to reduce data redundancy and significantly improves compression efficiency, making it suitable for applications with strong inter-view correlation. ARP, based on IVRP, further optimizes residual prediction by introducing complex models such as affine motion compensation, and is suitable for high-precision applications that need to handle complex motion and deformation scenes.

[0003] However, current residual prediction techniques suffer from high computational complexity, requiring substantial computing resources and resulting in poor performance in real-time applications or on resource-constrained devices. Secondly, their large memory footprint impacts overall system efficiency. Furthermore, existing technologies have limited adaptability, particularly when handling fast-moving or complex textured scenes, where prediction results are unsatisfactory and coding efficiency is reduced. Summary of the Invention

[0004] This application provides a disparity vector calculation method, prediction method, apparatus, device, and storage medium, which can improve the calculation and derivation process of disparity vectors. By adding more conditional factors, it reduces residual energy and improves the coding efficiency of inter-view residual prediction algorithms. It is no longer affected by the time domain or disparity domain and can perform residual prediction on videos in both the disparity domain and the time domain.

[0005] In a first aspect, embodiments of this application provide a disparity vector calculation method, comprising: determining the disparity range of a current coded block in a reference view, the disparity range being determined based on the maximum depth value and the minimum depth value of the reference view; determining a candidate block set for the current coded block based on the width of the current coded block and the disparity range; calculating a first disparity value between each candidate block in the candidate block set and the current coded block; selecting the candidate block with the smallest disparity difference from the candidate block set as a target candidate block, the disparity difference being the difference between the first disparity value between the current coded block and the corresponding candidate block and a second disparity value converted from the depth of the corresponding candidate block; and using the second disparity value corresponding to the target candidate block as the disparity vector of the current coded block relative to the reference view.

[0006] In one possible example scenario, the second disparity value is calculated using the following formula: Where d represents the second disparity value, Z represents the depth value of the corresponding candidate block, f represents the camera focal length, and Baseline represents the baseline distance.

[0007] In one possible example scenario, the first disparity value is calculated using the following formula:

[0008] Where, d n The disparity between the nth candidate block and the current coded block is represented by d, where width is the width of the current coded block. max The maximum disparity value converted to correspond to the maximum depth of the current view.

[0009] In a second aspect, embodiments of this application provide a video residual prediction method, comprising: obtaining a reference frame corresponding to the current frame in which the current coding block is located, the reference frame including a temporal reference frame under the current view and a disparity domain reference frame under the reference view; determining the motion vector and disparity vector of the current coding block, the disparity vector being obtained by the disparity vector calculation method as described in any one of the first aspects; determining a motion compensation block and an interview correspondence block corresponding to the current coding block, and a temporal motion compensation block corresponding to the interview correspondence block, based on the motion compensation block and the interview correspondence block corresponding to the current coding block, and a temporal motion compensation block corresponding to the interview correspondence block; and determining the target residual of the current coding block based on the motion compensation block and the interview correspondence block corresponding to the current coding block, and the temporal motion compensation block corresponding to the interview correspondence block.

[0010] In one possible example scenario, when the target residual to be determined is a temporal residual, determining the motion compensation block and the interview correspondence block corresponding to the current coding block, and the temporal motion compensation block corresponding to the interview correspondence block, based on the motion vector, the disparity vector, and the reference frame, includes: determining the motion compensation block corresponding to the current coding block in the temporal reference frame based on the current coding block and the motion vector; determining the interview correspondence block corresponding to the current coding block in the disparity domain reference frame based on the current coding block and the disparity vector; and determining the temporal motion compensation block corresponding to the interview correspondence block in the reference view based on the interview correspondence block and the motion vector.

[0011] In one possible example scenario, determining the target residual of the current coding block based on the motion compensation block and the inter-view correspondence block corresponding to the current coding block, and the temporal motion compensation block corresponding to the inter-view correspondence block, includes: obtaining a first residual of the current coding block based on the difference between the current coding block and the motion compensation block corresponding to the current coding block; determining a second residual of the current coding block based on the difference between the inter-view correspondence block and the temporal motion compensation block corresponding to the inter-view correspondence block; and calculating the target residual based on the first residual and the second residual.

[0012] In one possible example scenario, calculating the target residual based on the first residual and the second residual includes: determining a first product value of the second residual and a set first weight; and obtaining the target residual based on the difference between the first residual and the first product value.

[0013] In one possible example scenario, when the target residual to be determined is a disparity domain residual, determining the motion compensation block and the interview correspondence block corresponding to the current coding block, and the temporal motion compensation block corresponding to the interview correspondence block, based on the motion vector, the disparity vector, and the reference frame, includes: determining the interview correspondence block corresponding to the current coding block in the disparity domain reference frame based on the current coding block and the disparity vector; determining the temporal motion compensation block corresponding to the interview correspondence block in the reference view based on the interview correspondence block and the motion vector; and determining the motion compensation block corresponding to the current coding block in the temporal reference frame based on the temporal motion compensation block and the disparity vector.

[0014] In one possible example scenario, determining the target residual of the current coding block based on the motion compensation block and the inter-view correspondence block corresponding to the current coding block, and the temporal motion compensation block corresponding to the inter-view correspondence block, includes: obtaining a third residual of the current coding block based on the difference between the current coding block and the inter-view correspondence block; determining a fourth residual of the current coding block based on the difference between the temporal motion compensation block corresponding to the inter-view correspondence block and the motion compensation block corresponding to the current coding block; and calculating the target residual based on the third residual and the fourth residual. In another possible example scenario, calculating the target residual based on the third residual and the fourth residual includes: determining a second product value of the fourth residual and a set second weight; and obtaining the target residual based on the difference between the third residual and the second product value.

[0015] Secondly, embodiments of this application provide a video residual prediction apparatus, the apparatus comprising: an acquisition module, configured to acquire a reference frame corresponding to the current frame in which the current coding block is located, the reference frame including a temporal reference frame under the current view and a disparity domain reference frame under the reference view; a vector determination module, configured to determine the motion vector and disparity vector of the current coding block, the disparity vector being obtained by the disparity vector calculation method described in any one of the first aspects; a compensation module, configured to determine a motion compensation block and an inter-view correspondence block corresponding to the current coding block, and a temporal motion compensation block corresponding to the inter-view correspondence block, based on the motion vector, the disparity vector, and the reference frame; and a residual prediction module, configured to determine the target residual of the current coding block based on the motion compensation block and the inter-view correspondence block corresponding to the current coding block, and the temporal motion compensation block corresponding to the inter-view correspondence block.

[0016] Thirdly, embodiments of this application provide a video residual prediction device, the device comprising: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the video residual prediction method as described in any one of the second aspects.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the video residual prediction method described in any one of the second aspects.

[0018] The disparity vector calculation method, prediction method, apparatus, device, and storage medium of this application can improve the derivation process of disparity vectors and increase coding efficiency. By determining the disparity range through the depth value of the current coding block, and then determining the number of candidate blocks and the candidate block set of the reference frame corresponding to the disparity domain through the disparity range, the method calculates the first disparity value corresponding to each candidate block in the current coding block, finds the candidate block with the smallest disparity as the target candidate block, and uses the second disparity value as the disparity vector when the difference between the depth correspondence of the candidate block with the first disparity value and the second disparity value is minimized. This eliminates the limitation of the same depth value, resulting in a more accurate disparity vector. This makes the motion compensation blocks and inter-view correspondence blocks obtained using motion vectors and disparity vectors more accurate. It helps reduce computational complexity and memory usage, improves adaptability in complex scenes, and significantly enhances coding performance. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating a disparity vector calculation method provided in an embodiment of this application; Figure 2 This is a block diagram illustrating how to determine the interview corresponding block of the current coded block, as provided in an embodiment of this application. Figure 3 This is a flowchart illustrating a video residual prediction method provided in an embodiment of this application; Figure 4 This is a flowchart illustrating another video residual prediction method provided in an embodiment of this application; Figure 5 This is an example diagram illustrating the determination of motion compensation blocks and inter-view corresponding blocks provided in an embodiment of this application. Figure 6 This is a schematic diagram of the structure of a time-domain residual prediction method provided in an embodiment of this application; Figure 7 This is a flowchart illustrating another video residual prediction method provided in the embodiments of this application; Figure 8 This is a schematic diagram of the structure of a disparity domain residual prediction method provided in an embodiment of this application; Figure 9 This is a block diagram of another video residual prediction method provided in the embodiments of this application; Figure 10 This is a schematic diagram of the structure of a video residual prediction device provided in an embodiment of this application; Figure 11This is a schematic diagram of the structure of a video residual prediction device provided in an embodiment of this application. Detailed Implementation

[0021] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0022] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0023] Residual prediction coding is a data compression technique based on prediction error, widely used in image / video coding. Its core idea is to predict the value of the current pixel / block and encode only the prediction residual (the difference between the actual and predicted values), thereby reducing the amount of data that needs to be transmitted. It uses information from neighboring pixels or already encoded blocks to predict the spatial prediction (e.g., intra-frame prediction) or temporal prediction (e.g., inter-frame prediction) corresponding to the current block, and then obtains the prediction residual based on the difference between the actual and predicted values.

[0024] The third-generation audio and video coding standard (AVS3) is a Chinese-developed standard for audio and video coding, and the world's first audio and video source coding standard designed for 8K ultra-high-definition and 5G industrial applications. AVS3 saves approximately 30% bit rate compared to AVS2 for 4K ultra-high-definition video, and significantly improves coding efficiency for 8K video. It creatively employs key technologies such as more flexible coding tree block partitioning, internal prediction filters, and final motion vector expressions.

[0025] Existing Advanced Residual Prediction (ARP) techniques have high computational complexity, requiring significant computing resources and resulting in poor performance in real-time applications or on resource-constrained devices. Secondly, their large memory footprint impacts overall system efficiency. Furthermore, existing techniques have limited adaptability, particularly when handling fast-moving or complex texture scenes, where prediction results are unsatisfactory and coding efficiency is reduced. While inter-view prediction techniques are already applied in existing HTM reference software, their potential for performance improvement is limited by stringent conditions. For example, current coding blocks can only apply this technique when referencing a temporal reference frame, and the vector information at the decoding end can only be selected from either the motion vector (MV) or the disparity vector (DV). This restricts the acquisition and application of Motion Compensation Partitions (MCPs) and Disparity Compensation Partitions (DCPs).

[0026] To address the aforementioned technical problems, embodiments of this application provide a disparity vector calculation method, prediction method, apparatus, device, and storage medium. By improving the derivation process of the disparity vector, coding efficiency is enhanced. The disparity range is determined by the depth value of the current coding block, and then the number of candidate blocks and the candidate block set corresponding to the reference frame in the disparity domain are determined based on the disparity range. By calculating the first disparity value corresponding to each candidate block in the current coding block, the candidate block with the smallest disparity is found as the target candidate block. When the difference between the depth correspondence of the candidate blocks in the first disparity domain and the second disparity value is minimized, the second disparity value is used as the disparity vector. This eliminates the limitation of the same depth value, resulting in a more accurate disparity vector and enhancing the application effect of motion compensation blocks and inter-view correspondence blocks. It also helps reduce computational complexity and memory usage, improves adaptability in complex scenes, and significantly improves coding performance. The disparity vector calculation method provided in this application embodiment is described below.

[0027] Figure 1 This is a flowchart illustrating a disparity vector calculation method provided in an embodiment of this application. Figure 1 As shown, the steps of the disparity vector calculation method specifically include: S101~S105.

[0028] S101. Determine the disparity range of the current coded block in the reference view. The disparity range is determined based on the maximum depth value and minimum depth value of the reference view.

[0029] The parallax range mentioned here can be understood as the maximum depth value Z of the reference view. max Determine the maximum disparity value d of the current coded block. max Minimum depth value Z minThe minimum disparity value d of the current coded block is calculated. min The set consisting of the minimum and maximum disparity values ​​is used as the disparity range of the current coded block [d]. min ,d max ].

[0030] Furthermore, given the current coding block, a reference view for the current coding block is determined, thereby obtaining a disparity domain reference frame. The disparity value is calculated based on the depth value of the current reference view. Then, the maximum and minimum disparity values ​​of the disparity domain are determined based on the maximum and minimum depth values ​​of the current coding block within the disparity domain, thereby determining the disparity range of the current coding block and providing reference data for the next step of calculating candidate blocks in the disparity domain.

[0031] S102. Based on the width and disparity range of the current coding block, determine the candidate block set of the current coding block.

[0032] The width of the current coding block mentioned here can be understood as the average width.

[0033] Furthermore, the region of candidate blocks in the disparity domain reference frame corresponding to the current coding block is determined as a coding block strip region composed of disparity ranges. The candidate block region within the disparity range is then segmented according to the width of the current coding block, resulting in the number of candidate blocks and the candidate block set. However, each candidate block has a different disparity vector from the current coding block, and the disparity differences between the candidate blocks and the current coding block vary. Therefore, it is necessary to find candidate blocks with small disparity differences from the current coding block as target candidate blocks, i.e., as disparity compensation blocks for the current coding block in the disparity domain reference frame.

[0034] In one possible example scenario, Figure 2 This is a block diagram illustrating how to determine the inter-view corresponding block of the current coded block, as provided in an embodiment of this application. According to... Figure 2 As shown in the provided diagram, the dashed square area represents the current coded block area. The selected disparity range is uniformly divided from the maximum disparity value to the minimum disparity value, and divided according to the width of the current coded block to obtain a candidate block set containing N candidate blocks.

[0035] The number of candidate blocks is obtained using Formula 1: Formula 1 in, The width of the current coded block is given. The number of candidate blocks N is obtained by averaging the difference between the maximum and minimum disparity values ​​using the width of the current coded block.

[0036] S103. Calculate the first disparity value between each candidate block in the candidate block set and the current coded block.

[0037] The first disparity value mentioned here can be understood as the disparity between each candidate block and the current coded block.

[0038] Furthermore, N candidate blocks are calculated using the disparity range and the width of the current coded block. The process of calculating the disparity between any candidate block and the current coded block requires using Formula 2, which calculates the first disparity: Formula 2 Where, d n d represents the disparity between the nth candidate block and the current coded block, where width is the width of the current coded block. max The maximum disparity value is the maximum depth corresponding to the current view. According to Formula 1, the disparity value between the nth candidate block and the current coded block is determined in several cases. When the maximum disparity value is greater than zero, (n-1)*width represents the n-1 disparity ranges in the candidate block set. Figure 2 The provided diagram shows that the disparity of the first candidate block is d. max To calculate the disparity value of the nth candidate block, based on a value greater than zero, we remove n-1 candidate blocks from the positions following the first candidate block; the maximum disparity d... max If the value is less than zero, use the disparity d of the first candidate block. max The disparity value is obtained by moving n-1 blocks to the left. The disparity of any candidate block can then be calculated.

[0039] S104. Select the candidate block with the smallest disparity difference from the candidate block set as the target candidate block. The disparity difference is the difference between the first disparity value between the current coded block and the corresponding candidate block and the second disparity value converted from the depth of the corresponding candidate block.

[0040] To find the candidate block with the smallest disparity to the current coded block, Equation 3 is used: Formula 3 Where Disparity represents the disparity value, and Conv(B) represents the transformation of block-coded block B from depth to disparity vector. This disparity value is derived from the maximum depth value of the four corners and the center point of the target block in the reference view. Let B(x+d) represent the center point of the current coded block, and let B(x+d) represent the candidate block with disparity d in the current view. Therefore, if the target candidate block satisfies Formula 3, the obtained disparity and the transformed disparity and depth values ​​can be used as the disparity vector for encoding the current coded block.

[0041] The second disparity value is calculated using formula 4: Formula 4 Where d represents the second disparity value, Z represents the depth value of the corresponding candidate block, f represents the camera focal length, and Baseline represents the baseline distance. Equation 4 is used to convert the depth value to the disparity value.

[0042] The target candidate block is selected as the inter-view correspondence block of the current coding block in the disparity domain by analyzing the difference between the first disparity value between each candidate block and the current coding block and the candidate block during the process of transforming the depth into disparity value.

[0043] S105. Use the second disparity value corresponding to the target candidate block as the disparity vector of the current coded block relative to the reference view.

[0044] According to Formulas 3 and 4, the second disparity value corresponding to the smallest difference between the first disparity value and the second disparity value during the disparity value transformation process from depth value to disparity value between the selected target candidate block and the current coding block is used as the disparity vector of the current coding block on the reference map. Unlike the commonly used method for calculating the disparity vector of adjacent blocks, this method is not constrained by the requirement that pixels in the current coding block and pixels in adjacent candidate blocks must have the same depth value. It transforms different disparity values ​​based on different depth values, and selects the disparity vector with the smallest difference, thus improving coding efficiency.

[0045] This application provides a disparity vector calculation method that improves coding efficiency by refining the derivation process of the disparity vector. The method determines the disparity range using the depth value of the current coding block, and then determines the number of candidate blocks and the candidate block set corresponding to the reference frame in the disparity domain. By calculating the first disparity value corresponding to each candidate block in the current coding block, the candidate block with the smallest disparity is found as the target candidate block. When the difference between the depth correspondence of the candidate blocks in the first disparity domain and the second disparity value is minimized, the second disparity value is used as the disparity vector. This eliminates the limitation of the same depth value, resulting in a more accurate disparity vector and enhancing the application effect of motion compensation blocks and inter-view correspondence blocks. It also helps reduce computational complexity and memory usage, improves adaptability in complex scenes, and significantly improves coding performance.

[0046] Figure 3 This is a flowchart illustrating a video residual prediction method provided in an embodiment of this application. According to... Figure 3 The provided diagram shows that the steps of the video residual prediction method specifically include: S301~S304.

[0047] S301. Obtain the reference frame corresponding to the current frame where the current coding block is located. The reference frame includes the temporal reference frame under the current view and the disparity domain reference frame under the reference view.

[0048] The reference frame corresponding to the current frame is found based on the position of the current coding block in the view. The reference frame is then used to determine whether the current coding block's reference frame is in the temporal or disparity domain. In the temporal domain, reference frames corresponding to different times in the current view are used as temporal reference frames. In the reference view environment corresponding to the current coding block, a disparity domain reference frame is obtained. This facilitates the next step of calculating the temporal residual under the temporal reference frame corresponding to the current coding block, or calculating the disparity domain residual under the disparity domain reference frame environment. For example, the temporal and disparity domain reference frames are placed in a reference frame list. All reference frames in this list can be used for coding prediction of the current frame. Each frame can be compensated based on different reference frames to reduce redundant data.

[0049] S302. Determine the motion vector and disparity vector of the current coding block. The disparity vector is obtained through the disparity vector calculation method.

[0050] The disparity vector mentioned here is obtained through... Figures 1-2 The steps for calculating the disparity vector are provided. They will not be repeated here.

[0051] The calculation of motion vectors begins in the initial stage of video coding by dividing the current input frame into multiple 16x16 pixel macroblocks. This block division makes each macroblock the basic unit of the coding process, providing a more refined analytical foundation for subsequent processing. The segmented macroblocks are then used for motion estimation and block matching to more efficiently utilize spatial and temporal information and reduce redundancy. Motion estimation is performed after obtaining the macroblocks. Motion vectors describe the motion relationship between each macroblock and its corresponding region in the reference frame. The motion vectors indicate how the current coded block compensates for a region in the reference view, thus laying the foundation for subsequent block matching.

[0052] Based on the motion estimation results, block matching is performed on each macroblock, i.e., finding the temporal reference block most similar to the current coded block in the reference view. Two methods can be used: forward prediction (P-frame) and bidirectional prediction (B-frame). Frames preceding the P-frame serve as references, while the B-frame utilizes information from both the preceding and following frames for more accurate prediction. This matching not only helps improve coding performance but also effectively enhances image quality. After obtaining the match, the encoder performs differential coding on the motion vector of each block. By performing differential calculations with the motion vectors of surrounding macroblocks, the amount of data that needs to be transmitted is reduced, thus obtaining the motion vector. Based on the obtained motion vector and disparity vector, a reference is provided for the next step of determining the temporal compensation block and the corresponding block in the disparity domain. The motion vector is used in the temporal reference frame to describe how the image block moves along the time axis. During decoding, the encoder finds the corresponding motion compensation block in the reference frame based on the motion vector. The disparity vector is used in the inter-view reference frame to describe the difference between image blocks at different viewpoints. During decoding, the encoder finds the corresponding disparity compensation block in the reference frame based on the disparity vector. These two compensation methods predict the data of the current frame based on different temporal or spatial differences. The current coded block selects a block in the temporal reference frame or an inter-view reference frame based on the motion vector or disparity vector, and performs content prediction through motion compensation or disparity compensation.

[0053] S303. Based on the motion vector, disparity vector, and reference frame, determine the motion compensation block and interview corresponding block corresponding to the current coding block, as well as the temporal motion compensation block corresponding to the interview corresponding block.

[0054] The motion compensation block mentioned here can be understood as a compensation block on the current view determined based on temporal motion vectors, based on the current coded block. The inter-view correspondence block mentioned here can be understood as a compensation block in the disparity domain reference frame of the reference view obtained using disparity vectors, based on the current coded block. The temporal motion compensation block mentioned here can be understood as the predicted temporal motion compensation block calculated using the inter-view correspondence block and motion vectors, while assuming a temporal reference frame exists for the inter-view correspondence block.

[0055] Furthermore, based on the obtained motion vectors, a motion compensation block is calculated for the current coded block in the temporal reference frame of the current view. Simultaneously, using the obtained disparity vectors, the inter-view correspondence block of the current coded block in the disparity domain reference frame is calculated, which is also a disparity compensation block. Since the inter-view correspondence block exists in a temporal reference frame, the predicted temporal motion compensation block is calculated using the motion vectors and the temporal correspondence block. Then, compensation blocks for the temporal and disparity domain reference frames can be calculated using the current coded block, motion vectors, and disparity vectors, providing reference data for the next step of calculating the temporal and disparity domain residuals.

[0056] S304. Based on the motion compensation block and the inter-view corresponding block corresponding to the current coding block, and the temporal motion compensation block corresponding to the inter-view corresponding block, determine the target residual of the current coding block.

[0057] The target residual mentioned here can be understood as the result of predicting the residual between the current coding block and the prediction block in the temporal or disparity domain, under the conditions of the current frame where the current coding block is located, as well as the temporal reference frame and the disparity domain reference frame.

[0058] Furthermore, based on the calculated motion compensation blocks, the temporal residual of the current coding block can be calculated; based on the inter-view correspondence blocks, the disparity domain residual of the current coding block can be calculated; based on the motion compensation blocks and temporal motion compensation blocks, the temporal prediction residual can be predicted; based on the inter-view correspondence blocks and temporal motion compensation blocks, the disparity domain prediction residual can be predicted; based on the temporal residual and the prediction residual, the temporal target residual is obtained; based on the disparity domain residual and the disparity domain prediction residual, the disparity domain target residual is obtained. This allows for the processing of residuals in either the temporal or disparity domains, no longer limited by a single reference frame, improving coding efficiency, overcoming the limitations of existing technologies, providing a more efficient and flexible residual prediction solution, further improving the performance of 3D video coding, and making it suitable for various application scenarios in 3D video systems.

[0059] This application provides a video residual prediction method. By calculating the inter-view correspondence block corresponding to the current coding block based on disparity vectors, and the motion compensation block of the current coding block based on motion vectors, and by calculating the corresponding temporal motion compensation block based on the obtained inter-view correspondence blocks and motion vectors, the residuals in the temporal domain and the disparity domain can be inferred. Different target residuals are obtained depending on the conditions, no longer limited to the case where the current coding block references a temporal reference frame. Simultaneously, by improving the disparity vector acquisition method, the application effect of the motion compensation block and the inter-view correspondence block is enhanced. This helps reduce computational complexity and memory consumption, improves adaptability in complex scenes, and significantly improves coding performance. The residual prediction method of this application overcomes the limitations of existing technologies, providing a more efficient and flexible residual prediction solution, further improving the performance of 3D video coding, and is applicable to various application scenarios of 3D video systems.

[0060] Figure 4 This is a flowchart illustrating another video residual prediction method provided in an embodiment of this application. Figure 4 This description is based on the above embodiments. Figure 4 The provided diagram shows that the steps of the video residual prediction method specifically include: S401~S408.

[0061] S401. Based on the current coding block and motion vector, determine the motion compensation block corresponding to the current coding block in the temporal reference frame.

[0062] S402. Based on the current coding block and the disparity vector, determine the inter-view corresponding block of the current coding block in the disparity domain reference frame.

[0063] Based on the position of the current coded block, and assuming the motion vector is known, the corresponding compensation block is found according to the motion vector based on the current coded block, and this corresponding compensation block is used as the motion compensation block. If the disparity vector is known, the corresponding compensation block is found according to the direction of the disparity vector and the corresponding reference view based on the current coded block, and this is used as the inter-view corresponding block, also called the disparity compensation block.

[0064] S403. Based on the inter-view correspondence block and motion vector, determine the temporal motion compensation block corresponding to the inter-view correspondence block in the reference view.

[0065] Starting with calculating the position of the corresponding block between views in the temporal domain based on the motion vector, and according to the direction of the motion vector and the corresponding reference view, the prediction compensation block in the temporal domain corresponding to the reference view is found, and the prediction compensation block is used as the temporal motion compensation block. Then, the compensation block corresponding to the current view and the reference view is calculated based on the current coded block.

[0066] In one possible example scenario, Figure 5 This is an illustration of the effect of determining motion compensation blocks and inter-view correspondence blocks according to an embodiment of this application. Figure 5 The provided illustration shows that motion compensation predicts the current frame based on a time series. Using motion vectors, the decoder can then determine the frame from a reference frame (e.g., ...). Figure 5 In frames where R=2, a compensation block corresponding to the current coded block is found, thereby reducing the difference between the current frame and the reference frame. Figure 5As can be seen, the motion compensation process refers to a block in the encoded reconstructed frame, which is predicted using motion compensation. The interview correspondence block (i.e., the disparity compensation block) is predicted based on different viewpoints. Using disparity vectors, the decoder finds the corresponding block in the disparity domain reference frame and performs compensation prediction based on the disparity difference. During encoding and decoding, the prediction method for the current block (whether to use motion compensation or disparity compensation) depends on the settings during encoding. Depending on the application scenario, the encoder can choose to use motion compensation (MCP) or disparity compensation (DCP), or a combination of both for optimal prediction. Generally, for scenes with strong temporal continuity (such as static backgrounds or uniformly moving objects), similar blocks can be found in the preceding and following frames from the same viewpoint, so MCP is chosen. For scenes with high interview similarity (such as static or slow-motion scenes shot by multiple cameras), matching blocks can be found in the same frame at adjacent viewpoints, so DCP is chosen. When MCP is selected, the encoder uses motion vectors from the temporal reference frame for compensation. When DCP is selected, the encoder uses disparity vectors from the interview reference frame for compensation. During decoding, the decoder can reconstruct the content of the current coded block by selecting the corresponding motion compensation block (MCP) or disparity compensation block (DCP) from the reference frame based on the motion vector (MV) or disparity vector (DV). This selection process significantly improves coding efficiency and reduces redundant data.

[0067] S404. Based on the difference between the current coding block and the motion compensation block corresponding to the current coding block, obtain the first residual of the current coding block.

[0068] S405. Based on the difference between the inter-view correspondence block and the temporal motion compensation block corresponding to the inter-view correspondence block, determine the second residual of the current coding block.

[0069] The residual is calculated based on the pixel differences between the current coded block and the corresponding temporal reference block in the reference view. The residual is calculated by subtracting the pixel values ​​of the reference block; the resulting residual block represents the difference between the current coded block and the reference block. The residual values ​​represent the details of the image, and this information needs to be effectively transmitted during encoding to reconstruct the original image.

[0070] Based on the motion compensation block corresponding to the current coding block, the difference between the current coding block and the motion compensation block can be calculated, and this difference is used as the first residual of the current coding block in the temporal domain. According to the disparity vectors of the reference view and the current view, the temporal residual in the temporal reference frame is calculated, i.e., the difference between the inter-view corresponding block and the temporal motion compensation block is calculated, and this difference is used as the second residual of the prediction of the current coding block in the disparity reference frame.

[0071] S406. Based on the first residual and the second residual, the target residual is calculated.

[0072] S407. Determine the first product value of the second residual and the set first weight.

[0073] S408. Based on the difference between the first residual and the first product value, the target residual is obtained.

[0074] Based on the first and second residuals of the current coding block in the time domain, the constraint weight value of the second residual is selected as the first weight to constrain the influence ratio of the second residual on the target residual. Based on the first product value obtained by multiplying the second residual by the first weight, the difference between the first residual and the first product value is taken as the target residual of the current coding block in the time domain. Then, the difference of the current view in the time domain is obtained through the target residual.

[0075] To further illustrate the calculation process of the target residual in the time domain, let's consider a possible example scenario. Figure 6 This is a schematic diagram of the structure of a time-domain residual prediction method provided in an embodiment of this application. According to... Figure 6 The provided diagram shows that, based on the motion vector and disparity vector, the residual can be corrected through motion compensation. When the current coding block uses a temporal motion compensation block (MCP), Curr represents the current coding block, CurrRef represents the temporal motion compensation block, mvLX represents the motion vector, Base represents the interview corresponding block, BaseRef represents the temporal motion compensation block found by Base using the motion vector mvLX in the reference view, POC represents the image sequence number, and DV represents the disparity vector obtained using the improved disparity vector calculation method proposed in this proposal.

[0076] according to Figure 6 The provided diagram first calculates the residual of the current coding block in the current frame using the difference between the current coding block and the motion compensation block, and then obtains the first residual Rc using Formula 5: Formula 5 Based on the first residual Rc, the residual of the inter-view corresponding block is calculated using the difference between the inter-view corresponding block and the temporal motion compensation block, and the second residual Rb is obtained through formula 6: Formula 6 Based on the first and second residuals, and with the first weight α, the target residual in the time domain corresponding to the current coding block can be obtained. The target residual Rt is then obtained using Equation 7: Formula 7 The target residual is obtained by calculating the residuals of different reference frames twice.

[0077] Through the embodiments provided in this application, the first residual of the current coding block in the time domain is calculated from the perspective of the time domain, and the second residual of the corresponding inter-view block of the current coding block is calculated. The target residual in the time domain is obtained based on the first residual and the second residual, thereby realizing the residual prediction of the current coding block in the time domain.

[0078] Figure 7 This is a flowchart illustrating another video residual prediction method provided in the embodiments of this application. Figure 7 Is Figures 1-3 This explanation is based on [the above]. Figure 7 The provided diagram shows that the steps of the video residual prediction method specifically include: S701~S708.

[0079] S701. Based on the current coding block and the disparity vector, determine the inter-view corresponding block of the current coding block in the disparity domain reference frame.

[0080] S702. Based on the inter-view correspondence block and motion vector, determine the temporal motion compensation block corresponding to the inter-view correspondence block in the reference view.

[0081] Based on the position of the current coded block and assuming the disparity vector is known, the corresponding compensation block is found according to the disparity vector, and this compensation block is used as the inter-view corresponding block. If the motion vector is known, based on the inter-view corresponding block of the current coded block, the corresponding compensation block is found according to the direction of the motion vector and the corresponding reference view, and this is used as the temporal motion compensation block.

[0082] S703. Based on the temporal motion compensation block and disparity vector, determine the motion compensation block corresponding to the current coding block in the temporal reference frame.

[0083] Starting from the position of the temporal motion compensation block based on the disparity vector in the disparity domain, and following the direction of the disparity vector and the corresponding reference view, the compensation block in the disparity domain corresponding to the reference view is found. This compensation block is then used as the motion compensation block of the temporal motion compensation block relative to the disparity domain. Furthermore, the motion compensation block corresponding to the current view is calculated based on the current coded block. This provides a reference compensation block for the next step of calculating the residual in the disparity domain.

[0084] S704. Based on the difference between the current coding block and the corresponding inter-view block, obtain the third residual of the current coding block.

[0085] S705. Based on the difference between the temporal motion compensation block corresponding to the inter-view corresponding block and the motion compensation block corresponding to the current coding block, determine the fourth residual of the current coding block.

[0086] Based on the inter-view corresponding block of the current coding block, the difference between the current coding block and the inter-view corresponding block can be calculated, and the obtained difference is used as the third residual of the current coding block in the disparity domain. According to the temporal motion compensation block in the reference view and the motion compensation block in the current view, the fourth residual in the disparity domain of the temporal reference frame is calculated, that is, the difference between the temporal motion compensation block and the motion compensation block is calculated, and the difference is used as the fourth residual of the prediction.

[0087] S706. Based on the third and fourth residuals, the target residual is calculated.

[0088] S707. Determine the second product value of the fourth residual and the set second weight.

[0089] S708. Based on the difference between the third residual and the second product value, the target residual is obtained.

[0090] Based on the third and fourth residuals of the current coding block in the disparity domain, the constraint weight value of the fourth residual is selected as the second weight to constrain the influence ratio of the fourth residual on the target residual. Based on the product of the fourth residual and the second weight to obtain the second product value, the difference between the third residual and the second product value is taken as the target residual of the current coding block in the disparity domain. Then, the difference of the current view in the disparity domain is obtained through the target residual.

[0091] To further illustrate the calculation process of the target residual in the disparity domain, in a possible example scenario, Figure 8 This is a schematic diagram of the structure of a disparity domain residual prediction method provided in an embodiment of this application. According to... Figure 8 As shown in the provided diagram, when the current coding block uses the interview corresponding block (DCP) in the disparity domain, Curr represents the current coding block, DMV represents the disparity vector of the interview prediction, and Base represents the interview corresponding block. If Base contains a temporal prediction block, the temporal motion compensation block BaseRef on the reference view is found through the motion vector mvLX, and then the motion compensation block Curref of the temporal motion compensation block BaseRef under the reference view is found through the disparity vector DMV of the interview prediction.

[0092] according to Figure 8 The provided diagram first calculates the residual of the current coding block in the reference frame using the difference between the current coding block Curr and the corresponding interview block Base, and then obtains the third residual Rs using Equation 8: Formula 8 Based on the obtained third residual Rs, the residual of the motion compensation block is calculated using the difference between the motion compensation block and the time-domain motion compensation block, and the fourth residual Ry is obtained through formula 9: Formula 9 Based on the third and fourth residuals, and with the second weight β, the target residual in the disparity domain corresponding to the current coding block can be obtained. The target residual Ru is then obtained using Equation 10: Formula 10 The target residual is obtained by calculating the residuals of different reference frames twice.

[0093] The embodiments provided in this application calculate the third residual of the current coding block in the disparity domain and the fourth residual of the motion compensation block in the disparity domain. Based on the third and fourth residuals, the target residual in the disparity domain is obtained, thereby achieving residual prediction of the current coding block in the disparity domain. By calculating the compensation block using disparity vectors, the obtained residual prediction results are more accurate, improving coding efficiency.

[0094] In one possible example scenario, Figure 9 This is a block diagram of another video residual prediction method provided in the embodiments of this application. According to... Figure 9 The provided diagram illustrates the process of residual prediction for the current coding block. First, motion vector calculation 2 and disparity vector calculation 3 are performed on the current frame 1 containing the current coding block to obtain the corresponding motion vector and disparity vector. Then, the motion compensation block and inter-view correspondence block corresponding to the current coding block are calculated using the motion vector and disparity vector, as well as the temporal motion compensation block in the temporal domain where the inter-view correspondence block is located. The first disparity is calculated using the current coding block and the motion compensation block, and the second residual is obtained using the inter-view correspondence block and the temporal motion compensation block. The difference between the first and second residuals is used as the temporal residual prediction result 4. The third residual is calculated using the current coding block and the inter-view correspondence block, and the fourth residual is calculated using the motion compensation block and the temporal motion compensation block. The difference between the third and fourth residuals is used as the disparity domain residual prediction result 5. Based on the disparity domain residual prediction and the temporal residual prediction, the residual prediction result 6 of the current coding block is obtained, thus providing residual prediction methods under different constraints. Based on the obtained residual prediction results, the encoder performs quantization processing on the residuals. The quantization process maps continuous residual values ​​to a discrete numerical range, effectively reducing the amount of data. The quantized residual data is then entropy encoded to further compress the data, resulting in residual code 7. Decoding is then performed using the inverse processing of the encoding process 8, enabling more efficient subsequent transmission or storage.

[0095] The encoded residual data, along with the reference frame and motion / disparity vectors, constitutes the video stream and is transmitted to the decoder. During this process, not the complete image data is transmitted, but rather compressed information based on inter-frame prediction, including the reference frame, motion / disparity vectors, and compressed residual data. At the decoder, the decoder dequantizes the received quantized residual to recover values ​​as close as possible to the original residual. Then, the decoder extracts the corresponding prediction blocks from the reference frame based on the motion or disparity vectors. Finally, the dequantized residual is superimposed onto the prediction blocks to reconstruct image content close to the original. The decoder repeats this process, decoding block by block, until the complete video frame is finally recovered. This completes the encoding and decoding process of 3D video.

[0096] Furthermore, based on the video residual prediction method provided in this application, residual prediction results for 3D videos are obtained. To further verify the effectiveness of the residual prediction method of this application, residual prediction is performed on different data, yielding different prediction results.

[0097] First, four sets of test sequence data were obtained by capturing spatial scenes from different angles at the same moment through multi-view video, as shown in Table 1.

[0098] Table 1

[0099] Based on the test sequence in Table 1, an environment architecture for 3D-HTM was built using reference software, and residual prediction was performed using the video residual prediction method provided in this application. The prediction results are shown in Table 2, where the selected benchmark test data are test data provided by the standards organization.

[0100] Table 2

[0101] As can be seen from the residual prediction results provided in Table 2, the performance comparison is based on the test data of the different groups listed in the first column. (For video1...) Figure 1 ) and video2 (view) Figure 2 The difference in BD rate between the texture images is shown, with a negative BD rate indicating that the proposed method has better performance. It can be seen that this method achieves performance improvements of 1.7% and 1.6% in video1 and video2, respectively. In the decoded view, there are performance improvements of 0.6% and 0.6%, respectively. In the synthesized view, there is a performance improvement of 0.5%. Encoding time, decoding time, and synthesis time remain almost unchanged.

[0102] The video residual prediction method provided in this application calculates the inter-view correspondence block corresponding to the current coding block based on the disparity vector, the motion compensation block of the current coding block based on the motion vector, and the corresponding temporal motion compensation block based on the obtained inter-view correspondence block and motion vector. This allows for the prediction of the temporal residual and the disparity domain residual, resulting in different target residuals depending on the conditions; it is no longer limited to the case where the current coding block references a temporal reference frame. Furthermore, by improving the disparity vector acquisition method, the application effect of the motion compensation block and the inter-view correspondence block is enhanced. This helps reduce computational complexity and memory usage, improves adaptability in complex scenes, and significantly improves coding performance. The residual prediction method of this application overcomes the limitations of existing technologies, providing a more efficient and flexible residual prediction solution, further improving the performance of 3D video coding, and is applicable to various application scenarios of 3D video systems.

[0103] Figure 10 This is a schematic diagram of the structure of a video residual prediction device provided in an embodiment of this application. According to... Figure 10 The provided diagram shows that the structure of the video residual prediction device 100 specifically includes: The acquisition module 1001 is used to acquire the reference frame corresponding to the current frame where the current coding block is located. The reference frame includes the temporal reference frame under the current view and the disparity domain reference frame under the reference view. The vector determination module 1002 is used to determine the motion vector and disparity vector of the current coding block. The disparity vector is determined by, for example,... Figures 1-2 The disparity vector is obtained by calculating any one of the following methods; The compensation module 1003 is used to determine the motion compensation block and the inter-view corresponding block corresponding to the current coding block, as well as the temporal motion compensation block corresponding to the inter-view corresponding block, based on the motion vector, disparity vector and reference frame. The residual prediction module 1004 is used to determine the target residual of the current coding block based on the motion compensation block and the inter-view corresponding block corresponding to the current coding block, and the temporal motion compensation block corresponding to the inter-view corresponding block.

[0104] The video residual prediction device provided in this embodiment can be as follows: Figure 10 The residual prediction device 100 for the video shown can achieve Figures 1-9 For a detailed description of the technical effectiveness of the residual prediction method in the video shown, please refer to [link / reference]. Figures 1-9 The corresponding explanation is concise and will not be elaborated upon here.

[0105] Figure 11 This is a schematic diagram of the structure of a video residual prediction device provided in an embodiment of this application. The video residual prediction device may include a processor 1101 and a memory 1102 storing computer program instructions.

[0106] Specifically, the processor 1101 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0107] Memory 1102 may include mass storage for data or instructions. For example, and not limitingly, memory 1102 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 1102 may include removable or non-removable (or fixed) media. Where appropriate, memory 1102 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 1102 is non-volatile solid-state memory.

[0108] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the methods according to one aspect of this disclosure.

[0109] The processor 1101 reads and executes computer program instructions stored in the memory 1102 to implement any of the video residual prediction methods in the above embodiments.

[0110] In one example, the video residual prediction device may further include a communication interface 1103 and a bus 1110. Wherein, as Figure 11 As shown, the processor 1101, memory 1102, and communication interface 1103 are connected through bus 1110 and complete communication with each other.

[0111] The communication interface 1103 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0112] Bus 1110 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 1110 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0113] Furthermore, in conjunction with the video residual prediction methods described in the above embodiments, this application embodiment can provide a computer storage medium for implementation. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the video residual prediction methods described in the above embodiments.

[0114] This application also provides a computer program product, including a computer program that, when executed, implements any of the video residual prediction methods described in the above embodiments.

[0115] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0116] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0117] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0118] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0119] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for calculating disparity vectors, characterized in that, include: Determine the disparity range of the current coded block in the reference view, the disparity range being determined based on the maximum and minimum depth values ​​of the reference view; Based on the width of the current coded block and the disparity range, a candidate block set for the current coded block is determined; Calculate the first disparity value between each candidate block in the candidate block set and the current coded block; The candidate block with the smallest disparity difference is selected from the candidate block set as the target candidate block. The disparity difference is the difference between the first disparity value between the current coded block and the corresponding candidate block and the second disparity value converted from the depth of the corresponding candidate block. The second disparity value corresponding to the target candidate block is used as the disparity vector of the current coded block relative to the reference view.

2. The method according to claim 1, characterized in that, The second disparity value is calculated using the following formula: Where d represents the second disparity value, Z represents the depth value of the corresponding candidate block, f represents the camera focal length, and Baseline represents the baseline distance.

3. The method according to claim 1, characterized in that, The first disparity value is calculated using the following formula: Where, d n The disparity between the nth candidate block and the current coded block is represented by d, where width is the width of the current coded block. max The maximum disparity value converted to correspond to the maximum depth of the current view.

4. A method for predicting residuals in video, characterized in that, include: Obtain the reference frame corresponding to the current frame where the current coding block is located. The reference frame includes the temporal reference frame under the current view and the disparity domain reference frame under the reference view. Determine the motion vector and disparity vector of the current coding block, wherein the disparity vector is obtained by the disparity vector calculation method as described in any one of claims 1 to 3; Based on the motion vector, the disparity vector, and the reference frame, determine the motion compensation block and the inter-view correspondence block corresponding to the current coding block, as well as the temporal motion compensation block corresponding to the inter-view correspondence block; Based on the motion compensation block and the inter-view correspondence block corresponding to the current coding block, and the temporal motion compensation block corresponding to the inter-view correspondence block, the target residual of the current coding block is determined.

5. The method according to claim 4, characterized in that, When the target residual to be determined is a temporal residual, the step of determining the motion compensation block and inter-view correspondence block corresponding to the current coding block, and the temporal motion compensation block corresponding to the inter-view correspondence block, based on the motion vector, the disparity vector, and the reference frame, includes: Based on the current coding block and the motion vector, determine the motion compensation block corresponding to the current coding block in the time-domain reference frame; Based on the current coding block and the disparity vector, determine the inter-view corresponding block of the current coding block in the disparity domain reference frame; Based on the inter-view correspondence block and the motion vector, the temporal motion compensation block corresponding to the inter-view correspondence block is determined in the reference view.

6. The method according to claim 5, characterized in that, The step of determining the target residual of the current coding block based on the motion compensation block and the inter-view correspondence block corresponding to the current coding block, and the temporal motion compensation block corresponding to the inter-view correspondence block, includes: Based on the difference between the current coding block and the motion compensation block corresponding to the current coding block, the first residual of the current coding block is obtained; Based on the difference between the inter-view correspondence block and the temporal motion compensation block corresponding to the inter-view correspondence block, the second residual of the current coding block is determined; The target residual is calculated based on the first residual and the second residual.

7. The method according to claim 6, characterized in that, The step of calculating the target residual based on the first residual and the second residual includes: Determine the first product value of the second residual and the set first weight; The target residual is obtained based on the difference between the first residual and the first product value.

8. The method according to claim 4, characterized in that, When the target residual to be determined is a disparity domain residual, the step of determining the motion compensation block and inter-view correspondence block corresponding to the current coding block, and the temporal motion compensation block corresponding to the inter-view correspondence block, based on the motion vector, the disparity vector, and the reference frame, includes: Based on the current coding block and the disparity vector, determine the inter-view corresponding block of the current coding block in the disparity domain reference frame; Based on the inter-view correspondence block and the motion vector, determine the temporal motion compensation block corresponding to the inter-view correspondence block in the reference view; Based on the temporal motion compensation block and the disparity vector, the motion compensation block corresponding to the current coding block is determined in the temporal reference frame.

9. The method according to claim 8, characterized in that, The step of determining the target residual of the current coding block based on the motion compensation block and the inter-view correspondence block corresponding to the current coding block, and the temporal motion compensation block corresponding to the inter-view correspondence block, includes: Based on the difference between the current coded block and the corresponding inter-view block, the third residual of the current coded block is obtained; Based on the difference between the temporal motion compensation block corresponding to the inter-view corresponding block and the motion compensation block corresponding to the current coding block, the fourth residual of the current coding block is determined; The target residual is calculated based on the third residual and the fourth residual.

10. The method according to claim 9, characterized in that, The calculation of the target residual based on the third residual and the fourth residual includes: Determine the second product value of the fourth residual and the set second weight; The target residual is obtained based on the difference between the third residual and the second product value.

11. A residual prediction device for video, characterized in that, The device includes: The acquisition module is used to acquire the reference frame corresponding to the current frame where the current coding block is located. The reference frame includes the temporal reference frame under the current view and the disparity domain reference frame under the reference view. The vector determination module is used to determine the motion vector and disparity vector of the current coding block, wherein the disparity vector is obtained by the disparity vector calculation method as described in any one of claims 1 to 3; The compensation module is used to determine, based on the motion vector, the disparity vector, and the reference frame, the motion compensation block and the interview correspondence block corresponding to the current coding block, as well as the temporal motion compensation block corresponding to the interview correspondence block; The residual prediction module is used to determine the target residual of the current coding block based on the motion compensation block and the inter-view correspondence block corresponding to the current coding block, and the temporal motion compensation block corresponding to the inter-view correspondence block.

12. A video residual prediction device, characterized in that, The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the video residual prediction method as described in any one of claims 4 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the video residual prediction method as described in any one of claims 4 to 10.

Images