Coding and decoding method and device, equipment and storage medium

Abstract

The invention provides a coding and decoding method and device, equipment and a storage medium, which can perform predictive coding on at least one viewpoint of a first level in a frame coding process according to a hierarchical prediction structure of a multi-viewpoint array to obtain a reference frame, and then perform predictive coding on a viewpoint of a second level according to the reference frame to obtain a reconstructed image. Therefore, a reference relationship can be established between different position viewpoints (such as different rows or different columns) of the second hierarchy and the first hierarchy, the spatial position correlation between the viewpoints in two-dimensional distribution can be better utilized, and the video compression efficiency of a multi-viewpoint array can be improved.

Description

Coding and decoding method, device, equipment, and storage medium Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a coding and decoding method, apparatus, device, and storage medium. Background Art

[0002] In 3D applications such as virtual reality (VR), augmented reality (AR), and mixed reality (MR), multi-view video can be used as a visual media object. Multi-view video is captured by multiple cameras, contains different perspectives, and supports user interaction in immersive media.

[0003] A multi-view video encoding and decoding technology uses a 2-D hierarchical coding structure to combine multi-view images into a video sequence. This video sequence is then compressed using existing video compression tools to produce a compressed video stream. However, this method has low video compression efficiency.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a coding and decoding method, apparatus, device, and storage medium, which can better utilize the spatial position correlation between two-dimensionally distributed viewpoints, thereby helping to improve the video compression efficiency of a multi-viewpoint array.

[0006] In a first aspect, the present application provides an encoding method, comprising:

[0007] Determining a hierarchical prediction structure for a multi-view array, wherein the hierarchical prediction structure comprises a first level and a second level, the first level comprising at least one view of the multi-view array, the second level comprising views other than the first level;

[0008] performing predictive encoding on at least one viewpoint of the first level to obtain a reference frame;

[0009] According to the reference frame, predictive encoding is performed on the viewpoint of the second level to obtain a reconstructed image.

[0010] In a second aspect, an embodiment of the present application provides a decoding method, including:

[0011] Determining a hierarchical prediction structure for a multi-view array, wherein the hierarchical prediction structure comprises a first level and a second level, the first level comprising at least one view of the multi-view array, the second level comprising views other than the first level;

[0012] Performing predictive decoding on at least one viewpoint of the first layer to obtain a reference frame;

[0013] According to the reference frame, the viewpoint of the second level is predictively decoded to obtain a reconstructed image.

[0014] In a third aspect, the present application provides an encoding device for executing the method of the first aspect or its respective implementations. Specifically, the prediction device includes a functional unit for executing the method of the first aspect or its respective implementations.

[0015] In a fourth aspect, the present application provides a decoding device for executing the method in the second aspect or its respective implementations. Specifically, the prediction device includes a functional unit for executing the method in the second aspect or its respective implementations.

[0016] In a fifth aspect, an encoder is provided, comprising a processor and a memory, wherein the memory is configured to store a computer program, and the processor is configured to call and execute the computer program stored in the memory to execute the method of the first aspect or its respective implementations.

[0017] In a sixth aspect, a decoder is provided, comprising a processor and a memory, wherein the memory is configured to store a computer program, and the processor is configured to call and execute the computer program stored in the memory to execute the method of the second aspect or its respective implementations.

[0018] In a seventh aspect, a coding and decoding system is provided, comprising an encoder and a decoder. The encoder is configured to execute the method of the first aspect or its respective implementations, and the decoder is configured to execute the method of the second aspect or its respective implementations.

[0019] In an eighth aspect, a chip is provided for implementing the method described in any one of the first and second aspects above, or their respective implementations. Specifically, the chip includes a processor configured to load and execute a computer program from a memory, causing a device equipped with the chip to perform the method described in any one of the first and second aspects above, or their respective implementations.

[0020] In a ninth aspect, a computer-readable storage medium is provided for storing a computer program, which enables a computer to execute the method of any one of the first to second aspects or their respective implementations.

[0021] In a tenth aspect, a computer program product is provided, comprising computer program instructions, which enable a computer to execute the method of any one of the first to second aspects or their respective implementations.

[0022] In an eleventh aspect, a computer program is provided, which, when executed on a computer, enables the computer to execute the method in any one of the first to second aspects or their respective implementations.

[0023] In a twelfth aspect, a code stream is provided, which is generated based on the method of the first aspect.

[0024] Based on the above technical solution, the embodiment of the present application can first predictively encode at least one viewpoint of the first level to obtain a reference frame according to the hierarchical prediction structure of the multi-viewpoint array during the frame encoding process, and then predictively encode the viewpoint of the second level based on the reference frame to obtain a reconstructed image, so that a reference relationship can be established between viewpoints at different positions (such as different rows or different columns) of the second level and the first level, and the spatial position correlation between viewpoints distributed in two dimensions can be better utilized, which helps to improve the video compression efficiency of the multi-viewpoint array.

[0025] In addition, since the hierarchical prediction structure can include spatial position information between multiple views, the hierarchical prediction structure of the embodiment of the present application can include the reference position of the viewpoint in the multi-view array without the need for additional means, such as establishing a position lookup table to supplement the reference position of each viewpoint, which helps to further improve the compression efficiency of the multi-view array. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] FIG1 is a schematic block diagram of a video encoding and decoding system according to an embodiment of the present application;

[0027] FIG2A is a schematic block diagram of a video encoder according to an embodiment of the present application;

[0028] FIG2B is a schematic block diagram of a video decoder according to an embodiment of the present application;

[0029] FIG3A is a schematic diagram of a multi-view array;

[0030] FIG3B is a schematic diagram of the encoding order of a multi-view array;

[0031] FIG4 is a schematic diagram of a coding method flow chart provided in an embodiment of the present application;

[0032] FIG5A is a schematic diagram of DO of a multi-view array provided in one embodiment of the present application;

[0033] FIG5B is a schematic diagram of a hierarchical prediction structure of a multi-view array provided by an embodiment of the present application;

[0034] FIG6 is a schematic diagram of an encoding process provided by another embodiment of the present application;

[0035] FIG7 is a schematic diagram of an EO of a multi-view array provided in one embodiment of the present application;

[0036] FIG8 is a schematic diagram of an EO of a multi-view array provided by another embodiment of the present application;

[0037] FIG9 is a flowchart of a decoding method provided in an embodiment of the present application;

[0038] FIG10 is a schematic block diagram of an encoding device provided in an embodiment of the present application;

[0039] FIG11 is a schematic block diagram of a decoding device provided in an embodiment of the present application;

[0040] FIG12 is a schematic block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The present application can be applied to the field of image coding and decoding, the field of video coding and decoding, the field of hardware video coding and decoding, the field of dedicated circuit video coding and decoding, the field of real-time video coding and decoding, etc. For example, the solution of the present application can be combined with an audio and video coding standard (AVS), such as the H.264 / Advanced Video Coding (AVC) standard, the H.265 / High Efficiency Video Coding (HEVC) standard, and the H.266 / Versatile Video Coding (VVC) standard. Alternatively, the solution of the present application can be combined with other proprietary or industry standards and operated, the standards including ITU-TH.261, ISO / IEC MPEG-1 Visual, ITU-TH.262 or ISO / IEC MPEG-2 Visual, ITU-TH.263, ISO / IEC MPEG-4 Visual, ITU-TH.264 (also known as ISO / IEC MPEG-4 AVC), including scalable video coding (SVC) and multi-view video coding (MVC) extensions. It should be understood that the technology of this application is not limited to any specific coding standard or technology.

[0042] The encoding involved in the embodiment of the present application is mainly video encoding and decoding. For ease of understanding, the video encoding and decoding system involved in the embodiment of the present application is first introduced in conjunction with Figure 1.

[0043] FIG1 is a schematic block diagram of a video encoding and decoding system involved in an embodiment of the present application. It should be noted that FIG1 is only an example, and the video encoding and decoding system of the embodiment of the present application includes but is not limited to that shown in FIG1. ​​As shown in FIG1, the video encoding and decoding system 100 includes an encoding device 110 and a decoding device 120. The encoding device is used to encode (which can be understood as compressing) the video data to generate a code stream, and transmit the code stream to the decoding device. The decoding device decodes the code stream generated by the encoding device to obtain decoded video data.

[0044] The encoding device 110 of the embodiment of the present application can be understood as a device with a video encoding function, and the decoding device 120 can be understood as a device with a video decoding function, that is, the embodiment of the present application includes a wider range of devices for the encoding device 110 and the decoding device 120, such as smartphones, desktop computers, mobile computing devices, notebook (e.g., laptop) computers, tablet computers, set-top boxes, televisions, cameras, display devices, digital media players, video game consoles, car computers, etc.

[0045] In some embodiments, the encoding device 110 may transmit the encoded video data (eg, a code stream) to the decoding device 120 via a channel 130. The channel 130 may include one or more media and / or devices capable of transmitting the encoded video data from the encoding device 110 to the decoding device 120.

[0046] In one example, the channel 130 includes one or more communication media that enable the encoding device 110 to transmit the encoded video data directly to the decoding device 120 in real time. In this example, the encoding device 110 can modulate the encoded video data according to a communication standard and transmit the modulated video data to the decoding device 120. The communication media includes wireless communication media, such as radio frequency spectrum. Optionally, the communication media may also include wired communication media, such as one or more physical transmission lines.

[0047] In another example, channel 130 includes a storage medium that can store the video data encoded by encoding device 110. The storage medium includes various locally accessible data storage media, such as optical disks, DVDs, and flash memories. In this example, decoding device 120 can retrieve the encoded video data from the storage medium.

[0048] In another example, the channel 130 may include a storage server that can store the video data encoded by the encoding device 110. In this example, the decoding device 120 can download the stored encoded video data from the storage server. Alternatively, the storage server can store the encoded video data and transmit the encoded video data to the decoding device 120, such as a web server (e.g., for a website), a file transfer protocol (FTP) server, etc.

[0049] In some embodiments, the encoding device 110 includes a video encoder 112 and an output interface 113. The output interface 113 may include a modulator / demodulator (modem) and / or a transmitter.

[0050] In some embodiments, the encoding device 110 may further include a video source 111 in addition to the video encoder 112 and the input interface 113 .

[0051] The video source 111 may include at least one of a video acquisition device (eg, a camera), a video archive, a video input interface, and a computer graphics system, wherein the video input interface is used to receive video data from a video content provider, and the computer graphics system is used to generate video data.

[0052] The video encoder 112 encodes the video data from the video source 111 to generate a bitstream. The video data may include one or more pictures or a sequence of pictures. The bitstream contains the encoding information of the pictures or picture sequences in the form of a bitstream.

[0053] The video encoder 112 transmits the encoded video data directly to the decoding device 120 via the output interface 113. The encoded video data may also be stored in a storage medium or a storage server for subsequent reading by the decoding device 120.

[0054] In some embodiments, the decoding device 120 includes an input interface 121 and a video decoder 122 .

[0055] In some embodiments, the decoding device 120 may further include a display device 123 in addition to the input interface 121 and the video decoder 122 .

[0056] The input interface 121 includes a receiver and / or a modem and can receive the encoded video data via the channel 130 .

[0057] The video decoder 122 is configured to decode the encoded video data to obtain decoded video data, and transmit the decoded video data to the display device 123 .

[0058] The decoded video data is displayed on the display device 123. The display device 123 may be integrated with the decoding device 120 or external to the decoding device 120. The display device 123 may include various display devices, such as a liquid crystal display (LCD), a plasma display, an organic light emitting diode (OLED) display, or other types of display devices.

[0059] In addition, Figure 1 is only an example, and the technical solution of the embodiment of the present application is not limited to Figure 1. For example, the technology of the present application can also be applied to unilateral video encoding or unilateral video decoding.

[0060] The following is an introduction to the video encoding framework involved in the embodiments of the present application.

[0061] FIG2A is a schematic block diagram of a video encoder according to an embodiment of the present application.

[0062] As shown in FIG2A , the video encoder 200 may include a prediction unit 210, a residual unit 220, a transform / quantization unit 230, an inverse transform / quantization unit 240, a reconstruction unit 250, a loop filter unit 260, a decoded image buffer 270, and an entropy coding unit 280. It should be noted that the video encoder 200 may include more, fewer, or different functional components.

[0063] In some embodiments, the prediction unit 210 includes an inter-frame prediction unit 211 and an intra-frame estimation unit 212. Because there is a strong correlation between adjacent pixels in a video frame, intra-frame prediction is used in video coding and decoding technologies to eliminate spatial redundancy between adjacent pixels. Because there is a strong similarity between adjacent frames in a video, inter-frame prediction is used in video coding and decoding technologies to eliminate temporal redundancy between adjacent frames, thereby improving coding efficiency.

[0064] The inter-frame prediction unit 211 can be used for inter-frame prediction, which can include motion estimation and motion compensation. It can refer to image information of different frames. Inter-frame prediction uses motion information to find a reference block from a reference frame and generates a prediction block based on the reference block to eliminate temporal redundancy. The frames used for inter-frame prediction can be P frames and / or B frames, where P frames refer to forward predicted frames and B frames refer to bidirectionally predicted frames.

[0065] The intra-frame estimation unit 212 only refers to the information of the same frame image to predict the pixel information in the current image block to eliminate spatial redundancy. The frame used for intra-frame prediction can be an I frame.

[0066] FIG2B is a schematic block diagram of a video decoder according to an embodiment of the present application.

[0067] As shown in FIG2B , the video decoder 300 includes an entropy decoding unit 310, a prediction unit 320, an inverse quantization / transformation unit 330, a reconstruction unit 340, a loop filter unit 350, and a decoded picture buffer 360. The prediction unit 320 includes an inter-frame prediction unit 321 and an intra-frame estimation unit 322. It should be noted that the video decoder 300 may include more, fewer, or different functional components.

[0068] The video decoder 300 may receive a bitstream. The entropy decoding unit 310 may parse the bitstream to extract syntax elements from the bitstream. As part of parsing the bitstream, the entropy decoding unit 310 may parse the entropy-encoded syntax elements in the bitstream. The prediction unit 320, the inverse quantization / transform unit 330, the reconstruction unit 340, and the loop filter unit 350 may decode the video data based on the syntax elements extracted from the bitstream, thereby generating decoded video data.

[0069] The basic process of video encoding and decoding is as follows: At the encoder end, a frame of image is divided into blocks. For the current block, the prediction unit 210 uses intra-frame prediction or inter-frame prediction to generate a prediction block for the current block. The residual unit 220 calculates a residual block based on the predicted block and the original block of the current block. This residual block is the difference between the predicted block and the original block of the current block. This residual block is also called residual information. This residual block undergoes transformation and quantization by the transform / quantization unit 230, removing information that is insensitive to the human eye and eliminating visual redundancy. Optionally, the residual block before transformation and quantization by the transform / quantization unit 230 is called a time-domain residual block, and the time-domain residual block after transformation and quantization by the transform / quantization unit 230 is called a frequency residual block or a frequency-domain residual block. The entropy coding unit 280 receives the quantized change coefficients output by the change quantization unit 230 and performs entropy coding on these quantized change coefficients to output a bitstream. For example, the entropy coding unit 280 can eliminate character redundancy based on the target context model and probability information of the binary bitstream.

[0070] At the decoding end, the entropy decoding unit 310 can parse the code stream to obtain the prediction information, quantization coefficient matrix, etc. of the current block. The prediction unit 320 uses intra-frame prediction or inter-frame prediction for the current block based on the prediction information to generate a prediction block for the current block. The inverse quantization / transformation unit 330 uses the quantization coefficient matrix obtained from the code stream to inverse quantize and inverse transform the quantization coefficient matrix to obtain a residual block. The reconstruction unit 340 adds the prediction block and the residual block to obtain a reconstructed block. The reconstructed blocks constitute a reconstructed image, and the loop filtering unit 350 performs loop filtering on the reconstructed image based on the image or block to obtain a decoded image. The encoding end also requires similar operations as the decoding end to obtain a decoded image. The decoded image can also be called a reconstructed image, and the reconstructed image can be used as a reference frame for inter-frame prediction for subsequent frames.

[0071] It should be noted that the block division information determined by the encoder, as well as mode information or parameter information such as prediction, transform, quantization, entropy coding, and loop filtering, etc., are carried in the bitstream when necessary. The decoder parses the bitstream and analyzes the existing information to determine the same block division information, prediction, transform, quantization, entropy coding, loop filtering, etc. mode information or parameter information as the encoder, thereby ensuring that the decoded image obtained by the encoder and the decoder are identical.

[0072] The above is the basic process of the video codec under the block-based hybrid coding framework. With the development of technology, some modules or steps of the framework or process may be optimized. This application is applicable to the basic process of the video codec under the block-based hybrid coding framework, but is not limited to the framework and process.

[0073] In some application scenarios, multi-viewpoint video can appear in a 3D scene. Multi-viewpoint video is captured by multiple cameras, contains different perspectives, and supports user interaction. It is also called multi-view video or free-viewpoint video.

[0074] Multi-view video is typically produced by capturing the same 3D scene from multiple angles using a camera array. The cameras in the array are strategically positioned during the capture process so that each camera captures the scene from a unique viewpoint. The images captured by these cameras are called multi-view images. These multi-view images can be arranged into a multi-view image array based on their spatial relationships, also known as a multi-view array.

[0075] Accordingly, multiple cameras will capture multiple video sequences corresponding to multiple viewpoints. To provide more viewpoints, more cameras are used to generate multi-view videos with a large number of viewpoint-related video sequences. After the multi-view video is captured, it needs to be compressed and encoded. For example, in existing multi-view video technologies, video compression algorithms can be implemented using AVS3 encoding technology, HEVC encoding technology, etc.

[0076] A multi-view video encoding and decoding technology uses a 2-D hierarchical coding structure to combine a multi-view array into a video sequence (or image sequence). This video sequence is then compressed using existing video compression tools to produce a compressed video bitstream. For example, in the encoding device 110 of Figure 1 , the video source 111 can assemble a video sequence from a multi-view image array acquired by a video capture device. This video sequence is then input as video data into the video encoder 112, which encodes the video sequence to produce a bitstream.

[0077] The key to this approach is determining the frame coding order for each multi-view image in the multi-view array, which corresponds to the order of the video sequence. Specifically, during the frame coding process, this approach extends the 1-D hierarchical coding structure used in conventional video coding to a two-dimensional case.

[0078] For example, for standard video encoding, taking a 16-frame sequence as an example, a typical coding order is 0, 16, 8, 4, 2, 1, 3, 6, 5, 7, 12, 10, 9, 11, 14, 13, 15. Specifically, frame 0 can be an I-frame, where each coding unit can only use information from that frame for prediction; frame 16 can be a P-frame, which can use forward inter-frame information for prediction; and frames 1 through 15 can support bidirectional prediction. This coding order reduces the memory usage of the reference frame buffer.

[0079] FIG3A shows a specific example of a multi-view array. As shown in FIG3A , the multi-view array includes 165 viewpoints, and each viewpoint is numbered as shown in FIG1 , where the central viewpoint is numbered 0 and the remaining viewpoints are numbered row by row from 1 to 164. These numbers are called picture order counts (POCs). Taking the multi-view array as an example, the existing two-dimensional hierarchical coding structure scheme divides all viewpoints (i.e., the multi-view array) into four parts for encoding, as shown by the dotted lines in FIG1 . As shown in FIG3B , taking the upper left part as an example, the aforementioned one-dimensional hierarchical coding structure is used for encoding in the horizontal and vertical directions respectively. Specifically, the 0th row is encoded first, followed by the 6th row, then the 3rd row, and so on. For the internal encoding of each row, the 0th column is encoded first, then the 6th column, then the 3rd column, and so on.

[0080] The above scheme extends the one-dimensional layered coding structure used in conventional video coding to two dimensions during the frame encoding process. This is a simple imitation and extension of the one-dimensional layered coding structure, ignoring the spatial correlation of each viewpoint in the multi-view array on the two-dimensional plane. This reduces the video compression efficiency of the multi-view array. Furthermore, this scheme requires additional means, such as establishing a position lookup table, to supplement the reference position of each viewpoint.

[0081] In order to solve the above technical problems, an embodiment of the present application determines a hierarchical prediction structure of a multi-view array, and first predictively encodes at least one viewpoint of the first level to obtain a reference frame during the frame encoding process, and then predictively encodes the viewpoint of the second level based on the reference frame to obtain a reconstructed image, so that a reference relationship can be established between viewpoints at different positions (such as different rows or columns) of the second level and the first level, and the spatial position correlation between viewpoints distributed in two dimensions can be better utilized, which helps to improve the video compression efficiency of the multi-view array.

[0082] 4 , the video encoding method provided in the embodiment of the present application is introduced by taking the encoding end as an example.

[0083] FIG4 is a flow chart of an encoding method 400 provided in an embodiment of the present application. As shown in FIG4 , the method 400 in the embodiment of the present application includes:

[0084] S401 , determining a hierarchical prediction structure of a multi-view array, the hierarchical prediction structure comprising a first level and a second level, the first level comprising at least one view of the multi-view array, the second level comprising other views except the first level.

[0085] In three-dimensional application scenarios, such as virtual reality (VR), augmented reality (AR), and mixed reality (MR), multi-viewpoint videos can be used as visual media objects.

[0086] Multi-view video is produced by capturing the same 3D scene from multiple angles using multiple cameras (e.g., a camera array). The images captured by these cameras are called multi-view images. These multi-view images can be arranged into a multi-view array based on their spatial relationships. Each viewpoint in the multi-view array has horizontal and vertical parallax.

[0087] As an example, the multi-view array may be obtained by arranging dense multi-view images according to a spatial position relationship, wherein each multi-view image is arranged at a high density.

[0088] As an example, the multi-view array may include a centrosymmetric multi-view array, such as a square multi-view array.

[0089] As an example, the multi-view array may include multi-view images acquired by multiple cameras at the same time or at different times, which is not limited in this application.

[0090] In embodiments of the present application, a hierarchical prediction structure (HPS) of a multi-view array can be determined. Because the multi-view array is formed based on the spatial positional relationships of the multi-view images, the spatial positional relationships of each level in the hierarchical prediction structure differ. For example, the spatial positions of the viewpoints in the first level and the viewpoints in the second level described above differ. In other words, the hierarchical prediction structure can include spatial position information between multiple views.

[0091] The first-level viewpoints may be basic viewpoints in a multi-viewpoint array, such as relatively important viewpoints, and may provide reference information for the second-level viewpoints, serving as a reference for the second-level viewpoints.

[0092] Furthermore, each image in the multi-view array corresponds to a different camera shooting angle, and each viewpoint position is related to the corresponding camera shooting angle. Therefore, the hierarchical prediction structure obtained for the multi-view array is related to the camera shooting angle. In some embodiments, this hierarchical prediction structure may also be referred to as an angular hierarchy prediction structure (AHPS), which is not limited in this application.

[0093] In some embodiments, the multi-view array may also be referred to as a multi-view image array, a multi-view array, etc., which is not limited in this application.

[0094] In the embodiment of the present application, the multi-viewpoint images in the multi-viewpoint array can be rearranged into a video sequence (or image sequence). Display order (DO), also known as viewpoint order count (POC), refers to the sequential index of each viewpoint image in the video sequence.

[0095] As an example, the central viewpoint image (i.e., the central viewpoint image) is designated as the first frame in the video sequence and numbered 0 (i.e., DO#0). The remaining viewpoints (i.e., viewpoint images) are assigned DOs sequentially from left to right and from top to bottom. Figure 5A shows a specific example of DOs for a 9×9 multi-view array. Each square in Figure 5A represents a viewpoint, and the number inside the square is the DO of the corresponding viewpoint.

[0096] In some embodiments, the first level includes a first sub-level. The first sub-level includes a central viewpoint. Because the central viewpoint is located at the center of the multi-view array, it has the smallest average disparity with all viewpoints in the multi-view array and can provide more accurate reference information, serving as a reference for subsequent frames.

[0097] In some embodiments, the first level may further include a second sub-level. The second sub-level includes at least two viewpoints evenly distributed in the multi-view array. Exemplarily, the second sub-level may include at least two viewpoints sparsely distributed in the multi-view array.

[0098] In an embodiment of the present application, the first level, such as the first sub-level and the second sub-level, is the most basic constituent element of the hierarchical prediction structure, and can be the basic viewpoint in the multi-viewpoint array, serving as a reference for subsequent frames (such as frames corresponding to the second-level viewpoint), thereby being able to utilize the spatial position correlation between two-dimensionally distributed viewpoints during the frame encoding process.

[0099] As a possible implementation, the second sub-level may include at least one of a plurality of viewpoints evenly distributed along the edge lines (i.e., edge positions) of the multi-view array, and viewpoints located between the center viewpoint and the vertex viewpoints on the diagonal lines (i.e., diagonal positions) of the multi-view array. It should be noted that the second sub-level does not include the center viewpoint.

[0100] Exemplarily, the edge lines of the multi-view array may include at least one of the leftmost column (e.g., the first column), the rightmost column (the last column), the topmost row (the first row), and the bottommost row (the last row). The diagonal lines of the multi-view array may include at least one of a first line connecting the top left vertex viewpoint and the bottom right vertex viewpoint of the multi-view array, and a second line connecting the top right vertex viewpoint and the bottom left vertex viewpoint. The center viewpoint is located at the intersection of the first and second lines.

[0101] Figure 5B shows a specific example of a hierarchical prediction structure for a 9×9 multi-view array. As shown, the first level includes a first sub-level H0 and a second sub-level H1. The first sub-level H0 includes the central viewpoint DO#0 located at the center of the multi-view array. The second sub-level H1 includes eight viewpoints evenly distributed along the edge lines of the multi-view array, namely DO#1, DO#5, DO#9, DO#37, DO#44, DO#72, DO#76, and DO#80, as well as four viewpoints evenly distributed between the center viewpoint and the four vertex viewpoints on two diagonals of the multi-view array, namely viewpoint DO#21 located halfway between center viewpoint DO#0 and vertex viewpoint DO#1, viewpoint DO#25 located halfway between center viewpoint DO#0 and vertex viewpoint DO#9, viewpoint DO#56 located halfway between center viewpoint DO#0 and vertex viewpoint DO#72, and viewpoint DO#60 located halfway between center viewpoint DO#0 and vertex viewpoint DO#80, for a total of 12 viewpoints. These 12 viewpoints in the second sub-level H1 are sparsely and evenly distributed throughout the view.

[0102] In some embodiments, the second level includes a third sub-level, where the third sub-level includes at least two viewpoints located between the first sub-level and the second sub-level on the horizontal and vertical axes of the multi-view array. The viewpoints of the third sub-level serve to connect the first sub-level and the second sub-level.

[0103] Continuing to refer to Figure 5B, the second level includes the third sub-level H2, which includes viewpoint DO#39 between viewpoint DO#0 of the first sub-level H0 and viewpoint DO#37 of the second sub-level H1 on the horizontal center axis of the multi-viewpoint array, viewpoint DO#42 between viewpoint DO#0 of the first sub-level H0 and viewpoint DO#44 of the second sub-level H1, viewpoint DO#23 between viewpoint DO#0 of the first sub-level H0 and viewpoint DO#5 of the second sub-level H1 on the vertical center axis, and viewpoint DO#58 between viewpoint DO#0 of the first sub-level H0 and viewpoint DO#76 of the second sub-level H1, for a total of four viewpoints.

[0104] In some embodiments, the second level further includes a fourth sub-level. The fourth sub-level includes viewpoints located between the viewpoints of the first sub-level on an edge line of the multi-view array, and at least two viewpoints located between the third sub-level and the first sub-level, and between the third sub-level and the second sub-level, on the horizontal and vertical central axes of the multi-view array. The viewpoints of the fourth sub-level fill the gap between the third sub-level and the second sub-level.

[0105] Continuing to refer to FIG5B , the second level further includes a fourth sub-level H3 , which includes 16 viewpoints, namely:

[0106] viewpoint DO#3 between viewpoints DO#1 and DO#5, viewpoint DO#7 between viewpoints DO#5 and DO#9, viewpoint DO#27 between viewpoints DO#9 and DO#44, viewpoint DO#62 between viewpoints DO#44 and DO#80, viewpoint DO#78 between viewpoints DO#80 and DO#76, viewpoint DO#74 between viewpoints DO#76 and DO#72, viewpoint DO#54 between viewpoints DO#72 and DO#37, and viewpoint DO#19 between viewpoints DO#37 and DO#1, for a total of 8 viewpoints; and

[0107] On the horizontal center axis, there are four viewpoints: viewpoint DO#40 between viewpoint DO#39 of the third sub-level H2 and viewpoint DO#0 of the first sub-level H1; viewpoint DO#41 between viewpoint DO#42 of the third sub-level H2 and viewpoint DO#0 of the first sub-level H1; viewpoint DO#38 between viewpoint DO#39 of the third sub-level H2 and viewpoint DO#37 of the second sub-level H1; and viewpoint DO#43 between viewpoint DO#42 of the third sub-level H2 and viewpoint DO#44 of the second sub-level H1; and

[0108] On the vertical center axis, there are viewpoint DO#32 between viewpoint DO#23 of the third sub-level H2 and viewpoint DO#0 of the first sub-level H1, viewpoint DO#49 between viewpoint DO#58 of the third sub-level H2 and viewpoint DO#0 of the first sub-level H1, viewpoint DO#14 between viewpoint DO#23 of the third sub-level H2 and viewpoint DO#5 of the second sub-level H1, and viewpoint DO#67 between viewpoint DO#58 of the third sub-level H2 and viewpoint DO#76 of the second sub-level H1, for a total of 4 viewpoints.

[0109] In some embodiments, the second level also includes a fifth sub-level, the fifth sub-level including at least two viewpoints between the second sub-level and the fourth sub-level located on the edge line of the multi-view array, at least two viewpoints between the third sub-level and the second sub-level on a row of the multi-view array other than the edge line and the horizontal central axis, and between the second sub-level and the fourth sub-level.

[0110] Continuing to refer to FIG5B , the second level further includes a fifth sub-level H4 , which includes 24 viewpoints, namely:

[0111] Viewpoints between the viewpoints of the second sub-level H1 and the viewpoints of the fourth sub-level H3 on the edge line of the multi-view array, such as DO#2, DO#4, DO#6, DO#8, DO#18, DO#36, DO#53, DO#71, DO#79, DO#77, DO#75, DO#73, DO#63, DO#45, DO#28, DO#10, etc., for a total of 16 viewpoints; and

[0112] The viewpoints between the viewpoints of the third sub-level H2 and the viewpoints of the second sub-level H1 on the third row, such as DO#22 and DO#24, and the viewpoints between the viewpoints of the second sub-level H1 and the viewpoints of the fourth sub-level H3, such as DO#20 and DO#26, for a total of 4 viewpoints; and

[0113] There are 4 viewpoints in total, including the viewpoints between the viewpoints of the third sub-level H2 and the viewpoints of the second sub-level H1 on the seventh row, such as DO#57 and DO#59, and the viewpoints between the viewpoints of the second sub-level H1 and the viewpoints of the fourth sub-level H3, such as DO#55 and DO#61.

[0114] As shown in Figure 5B , the viewpoints of the fourth and fifth sub-levels H3 and H4 cover nearly half of the dense multi-view array. Therefore, when the viewpoints of the fourth and fifth sub-levels H3 and H4 are inter-predicted using the viewpoints of the first or second sub-level as a reference, bitrate savings can be achieved. In other words, the fourth and fifth sub-levels H3 and H4 are the primary source of bitrate savings.

[0115] In some embodiments, the second level also includes a sixth sub-level, which includes at least two viewpoints between the third sub-level and the second sub-level, and between the second sub-level and the fourth sub-level, located on columns of the multi-viewpoint array other than edge lines and the vertical central axis.

[0116] Continuing to refer to FIG5B , the second level further includes a sixth sub-level H5 , which includes 8 viewpoints, namely:

[0117] The viewpoints between the viewpoints of the third sub-level H2 and the viewpoints of the second sub-level H1 on the third column, such as DO#30 and DO#47, and at least two viewpoints between the viewpoints of the second sub-level H1 and the viewpoints of the fourth sub-level H3, such as DO#12 and DO#65, for a total of 4 viewpoints;

[0118] The viewpoints between the viewpoints of the third sub-level H2 and the viewpoints of the second sub-level H1 on the seventh column, such as DO#34 and DO#51, and at least two viewpoints between the viewpoints of the second sub-level H1 and the viewpoints of the fourth sub-level H3, such as DO#16 and DO#69, totaling 4 viewpoints.

[0119] In some embodiments, the second level further includes a seventh sub-level including at least two views located in the same row as the sixth sub-level in rows excluding edge lines of the multi-view array.

[0120] Continuing to refer to Figure 5B, the second level also includes a seventh sub-level H6, which includes 16 viewpoints, namely: viewpoints DO#11, DO#13, DO#15, and DO#17 in the second row, viewpoints DO#29, DO#31, DO#33, and DO#35 in the fourth row, viewpoints DO#46, DO#48, DO#50, and DO#52 in the sixth row, and viewpoints DO#64, DO#66, DO#68, and DO#70 in the eighth row.

[0121] In some embodiments, the multi-view images in the multi-view array can be rearranged into a video sequence based on a hierarchical prediction structure, where the images are ordered from low to high levels (sub-levels). In this case, the video sequence is arranged in encoder order (EO), which refers to the actual order in which the view images are encoded. Therefore, the multi-view images in the multi-view array can be encoded from low to high levels (sub-levels). Optionally, a precise encoding order can be used within a specific level.

[0122] The following steps S402 and S403 describe the encoding sequence process according to the hierarchical prediction structure.

[0123] S402 : Perform predictive coding on at least one viewpoint of the first layer to obtain a reference frame.

[0124] Specifically, at least one viewpoint of the first level of the hierarchical prediction structure is first compression-encoded.

[0125] In some embodiments, as shown in FIG6 , when the first level includes a first sub-level, step S402 may specifically include S4021:

[0126] S4021 , performing intra-frame prediction coding on the central viewpoint of the first sub-level to obtain a first reference frame.

[0127] Specifically, the central viewpoint of the first sub-level is designated as the first frame of the coding order EO, ​​numbered 0 (EO#0). As shown in Figure 7, the coding order of the central viewpoint (DO#0) of the first sub-level H0 is EO#0. Since there are no coded viewpoint images to serve as reference, the central viewpoint is encoded using intra-frame prediction mode, resulting in the first reference frame. This first reference frame can be used as a reference frame for any viewpoint in subsequent video sequences.

[0128] In some embodiments, referring to FIG6 , when the first level further includes a second sub-level, step S402 may further include S4022:

[0129] S4022: Perform predictive coding on at least two viewpoints of the second sub-level to obtain a second reference frame.

[0130] The above-mentioned reference frame includes a first reference frame and a second reference frame.

[0131] Specifically, after the central viewpoint of the first sub-level is encoded, at least two viewpoints of the second sub-level are respectively predictively encoded to obtain at least two second reference frames.

[0132] In some embodiments, intra-frame prediction encoding or inter-frame prediction encoding can be performed on at least two viewpoints of the second sub-level to obtain a second reference frame. When inter-frame prediction encoding is performed on the viewpoints of the second sub-level, a reference frame can be adaptively selected from adjacent (including the most adjacent or next adjacent) viewpoints. For example, the first reference frame of the encoded central viewpoint of the first sub-level can be used as the reference frame, or the second reference frame of the already encoded viewpoint in the second sub-level can be used as the reference frame. This application is not limited to this.

[0133] Optionally, the viewpoints of the first sub-level can be divided into an upper half and a lower half, and compression encoding can be performed on each half independently. By performing compression encoding on the upper half and the lower half independently, it is possible to delete some of the encoded images of the upper half (e.g., the encoded images corresponding to viewpoints other than the viewpoint at the junction of the upper half and the lower half) after the encoding of the upper half is completed, thereby saving encoding cache.

[0134] For example, referring to Figure 7 , after encoding the central viewpoint (DO#0) of the first sub-level, the viewpoints of the second sub-level H1 can be divided into an upper half and a lower half. First, the upper half of the second sub-level H1 can be encoded and compressed in the coding order from EO#1 to EO#7. As a specific example, the viewpoints DO#44, DO#9, DO#25, DO#5, DO#21, DO#1, and DO#37 of the upper half of the second sub-level H1 can be encoded and compressed in the coding order from EO#1 to EO#7 from right to left.

[0135] Optionally, after all viewpoints in the upper half of the multi-view array are coded, the lower half of the second sub-level can be compressed and coded. The specific compression order is similar to that of the upper half, and the description of the upper half can be referred to and will not be repeated here.

[0136] S403 , performing predictive coding on the viewpoints of the second level according to the reference frame to obtain a reconstructed image.

[0137] After the first-level viewpoints are coded, they can provide reference information for the second-level viewpoints, serving as a reference for the second-level viewpoints. Specifically, inter-frame prediction coding can be performed on the second-level viewpoints based on the reference frames obtained by coding the first-level viewpoints to obtain a reconstructed image.

[0138] Therefore, the embodiment of the present application determines the hierarchical prediction structure of the multi-view array, and first predictively encodes at least one viewpoint of the first level to obtain a reference frame during the frame encoding process, and then predictively encodes the viewpoint of the second level based on the reference frame to obtain a reconstructed image, so that a reference relationship can be established between viewpoints at different positions (such as different rows or columns) of the second level and the first level, and the spatial position correlation between viewpoints distributed in two dimensions can be better utilized, which helps to improve the video compression efficiency of the multi-view array.

[0139] In addition, since the hierarchical prediction structure can include spatial position information between multiple views, the hierarchical prediction structure of the embodiment of the present application can include the reference position of the viewpoint in the multi-view array without the need for additional means, such as establishing a position lookup table to supplement the reference position of each viewpoint, which helps to further improve the compression efficiency of the multi-view array.

[0140] In some embodiments, when the second level further includes multiple sub-levels, step 403 may be specifically implemented as follows:

[0141] According to the reference frame corresponding to the first level, the viewpoints in each sub-level in the second level are sequentially encoded in order from the lower sub-level to the higher sub-level to obtain a reconstructed image, wherein the viewpoint of each sub-level in each sub-level in the second level is predictively encoded with reference to the viewpoint of the same or lower sub-level as each sub-level.

[0142] In some embodiments, referring to FIG6 , when the first level further includes a first sub-level and a second sub-level, step 403 may specifically be the following steps S4031:

[0143] S4031, based on the first reference frame and the second reference frame, the viewpoints in each sub-level in the second level are sequentially encoded in order from the lower sub-level to the higher sub-level, wherein the viewpoint of each sub-level in each sub-level is predictively encoded with reference to the viewpoint of the same sub-level or lower sub-level of each sub-level.

[0144] For example, the viewpoint of the third sub-level in the second level can use the encoded viewpoint in at least one of the first sub-level, the second sub-level, and the third sub-level as a reference frame for inter-frame prediction encoding; the viewpoint of the fourth sub-level can use the encoded viewpoint in at least one of the first sub-level to the fourth sub-level as a reference frame for inter-frame prediction encoding, and so on.

[0145] In some embodiments, each sub-level may adaptively select a reference frame from adjacent (including the most adjacent or next adjacent) viewpoints during encoding, which is not limited in this application.

[0146] Therefore, in the embodiment of the present application, during the frame encoding process, first predictively encodes the viewpoints of the lower sub-level to obtain a reference frame, and then predictively encodes the viewpoints of the higher sub-level based on the reference frame to obtain a reconstructed image. That is, the viewpoints in a certain sub-level can only be encoded with reference to the viewpoints in the same sub-level or lower sub-level, so that a reference relationship can be established between viewpoints at different positions in different sub-levels (such as different rows or different columns), and the spatial position correlation between viewpoints distributed in two dimensions can be utilized to a greater extent, which helps to improve the video compression efficiency of the multi-viewpoint array.

[0147] In some embodiments, in the above-mentioned step 4031, at least two rows of the multi-view array can be encoded row by row according to the encoding order of the one-dimensional hierarchical coding structure, wherein the views other than the first sub-level and the second sub-level in each row of the multi-view array are encoded sequentially according to the order from the lower sub-level to the higher sub-level.

[0148] For example, referring to FIG7 , when the multi-view array includes 9 rows of views, encoding can be performed row by row according to the classic encoding order 1, 9, 5, 3, 2, 4, 7, 6, 8. For each view in each row, except for the views of the first and second sub-levels that have already been encoded, the views are encoded sequentially from the lower sub-level to the higher sub-level.

[0149] In some embodiments, when a row includes at least two viewpoints of the same sub-level, the at least two viewpoints may be encoded one by one according to the coding order of the one-dimensional hierarchical coding structure.

[0150] For example, referring to FIG7 , when encoding multiple viewpoints in the first row, the viewpoints DO#1, DO#5, and DO#9 of the second sub-level H1 have already been encoded and the corresponding second reference frames have been obtained. At this time, the viewpoints DO#3 and DO#7 of the fourth sub-level H3 can first be inter-frame predicted and encoded, and then the viewpoints DO#2, DO#4, DO#6, and DO#8 of the fifth sub-level H4 can be inter-frame predicted and encoded. For example, DO#3 and DO#7 can be encoded in sequence according to the encoding order of the one-dimensional layered coding structure, and DO#2, DO#4, DO#6, and DO#8 can be encoded in sequence according to the encoding order of the one-dimensional layered coding structure.

[0151] In some embodiments, a first partial multi-view array may be determined in the multi-view array, wherein the first partial multi-view array includes the center viewpoint. Then, at least two rows of the first partial multi-view array may be encoded row by row according to a coding order of the one-dimensional hierarchical coding structure.

[0152] In an embodiment of the present application, the first part of the multi-view array can be independently encoded, so that after the encoding of the first part of the multi-view array is completed, part of the encoded images of the first part of the multi-view array (such as the encoded images corresponding to the viewpoints other than the viewpoints at the intersection of the first part of the multi-view array and the rest of the multi-view array) can be deleted, thereby saving the encoding cache.

[0153] Exemplarily, the first part of the multi-view array may include the upper half of the multi-view array, the lower half of the multi-view array, the upper right part of the multi-view array, the upper left part of the multi-view array, the lower left part of the multi-view array, or the lower right part of the multi-view array, which is not limited in this application.

[0154] As shown in Figure 8, taking the first part of the multi-viewpoint array as the upper right part of the multi-viewpoint array as an example, after completing the predictive encoding of the first sub-level H0 and the upper half of the viewpoints of the second sub-level H1, the viewpoints of the remaining third sub-level H2 to the seventh sub-level H6 in the upper half are further divided into two parts, which are left and right. These two parts are also compressed independently to save the encoding cache.

[0155] Continuing with Figure 8 , taking the upper right portion of the multi-view array as an example, at least two rows of this portion of the multi-view array can be encoded row by row according to the coding order of the one-dimensional hierarchical coding structure, for example, row by row in the order of 5, 1, 3, 4, 2. Within each row of this portion of the multi-view array, encoding is performed in hierarchical order, with lower sub-level views being prioritized.

[0156] For example, the viewpoints in row 5 are encoded first. Specifically, the viewpoint DO#42 of the third sub-level H2 may be encoded first, with the encoding order being EO#8. Then, the viewpoints DO#43 and DO#41 of the fourth sub-level H3 may be encoded, with the encoding orders being EO#9 and EO#10 respectively.

[0157] Then, the viewpoints in the first row are encoded. Specifically, the viewpoint DO#7 of the fourth sub-level H3 is encoded first, and its encoding order is EO#11. Then, the viewpoints DO#8 and DO#6 of the fifth sub-level H4 are encoded, and their encoding orders are EO#12 and EO#13 respectively.

[0158] Then, the viewpoints in the third row are encoded. Specifically, the viewpoint DO#23 of the third sub-level H2 is encoded first, with the encoding order EO#14. Then, the viewpoint DO#27 of the fourth sub-level H3 is encoded with the encoding order EO#15. Then, the viewpoints DO#26 and DO#24 of the fifth sub-level H4 are encoded with the encoding orders EO#16 and EO#17 respectively.

[0159] Then, the viewpoints in the fourth row are encoded. Specifically, the viewpoint DO#32 of the fourth sub-level H3 is encoded first, and its encoding order is EO#18. Then, the viewpoint DO#36 of the fifth sub-level H4 is encoded, and its encoding order is EO#19. Then, the viewpoint DO#34 of the sixth sub-level H5 is encoded, and its encoding order is EO#20. Then, the viewpoints DO#35 and DO#33 of the seventh sub-level H6 are encoded, and their encoding orders are EO#21 and EO#22 respectively.

[0160] Then the viewpoints of the second row are encoded. Specifically, the viewpoint DO#14 of the fourth sub-level H3 is encoded first, and its encoding order is EO#23. Then the viewpoint DO#18 of the fifth sub-level H4 is encoded, and its encoding order is EO#24. Then the viewpoint DO#16 of the sixth sub-level H5 is encoded, and its encoding order is EO#25. Then the viewpoints DO#17 and DO#15 of the seventh sub-level H6 are encoded, and their encoding orders are EO#26 and EO#27 respectively.

[0161] Optionally, when encoding a viewpoint within a row, a reference frame can be adaptively selected from adjacent (including the most adjacent or second-adjacent) viewpoints. For example, an encoded viewpoint can be adaptively selected as a reference frame in the same row, the same column, different rows, or different columns. This application does not limit this.

[0162] In some embodiments, referring to FIG8 , after the encoding of the upper right multi-view array is completed, the encoding of the upper left multi-view array can proceed. The encoding sequence of the upper left multi-view array is similar to that of the upper right multi-view array, and reference can be made to the above description. After the encoding of the upper right multi-view array is completed, the encoding of the upper multi-view array is complete.

[0163] Furthermore, after the encoding of the upper multi-view array is complete, the encoding of the lower multi-view array can continue. The encoding order for the lower multi-view array is similar to that for the upper multi-view array, and reference can be made to the description above. Once the encoding of the lower multi-view array is complete, the encoding of the entire multi-view array is complete. Figure 8 also shows the encoding order for all views in the multi-view array.

[0164] In some embodiments, different partial multi-view arrays share at least two viewpoints at their intersections. For example, assuming the first partial multi-view array is the upper right partial multi-view array and the second partial multi-view array is the upper left partial multi-view array, the intersections of the first and second partial multi-view arrays share viewpoints DO#5, DO#14, DO#23, DO#32, and DO#0.

[0165] Optionally, when all viewpoints in the first part of the multi-viewpoint array have been encoded and the viewpoints in the second part of the multi-viewpoint array have not yet been encoded, the reconstructed images corresponding to the viewpoints in the first part of the multi-viewpoint array except for at least two viewpoints shared at the intersection can be deleted to save the encoding cache and maintain a lightweight encoding cache area.

[0166] For example, referring to Figure 8 , when the encoding of all viewpoints in the upper right multi-view array is complete, the reconstructed images of all viewpoints except for the shared viewpoints DO#5, DO#14, DO#23, DO#32, and DO#0 at the junction with the upper left multi-view array, and the shared viewpoints DO#41, DO#42, DO#43, and DO#44 at the junction with the lower right multi-view array can be deleted. Because each portion of the multi-view array is encoded independently, deleting the reconstructed images corresponding to viewpoints other than the at least two shared viewpoints at the junction in the first multi-view array does not affect the encoding of other portions of the multi-view array, thereby saving viewpoint encoding cache and maintaining a lightweight encoding cache.

[0167] In addition, in the embodiment of the present application, in each independently coded region, the coded image of the last coded viewpoint will not be used as a reference frame for other viewpoints. Such viewpoints can be referred to as non-reference viewpoints, such as the viewpoints of the seventh sub-level H6 in the embodiment of the present application, and the second, fifth, seventh, and tenth columns of the multi-viewpoint array in Figure 3A. Compared to the prior art where an entire column of viewpoints is non-reference viewpoints, the embodiment of the present application can significantly reduce the number of non-reference viewpoints, thereby further improving video compression efficiency.

[0168] The above describes the encoding method of the present application using the encoding end as an example. The following describes the video decoding method provided in the embodiment of the present application using the decoding end as an example.

[0169] FIG9 is a schematic flow chart of a decoding method 500 provided in an embodiment of the present application. As shown in FIG9 , the decoding method in the embodiment of the present application includes:

[0170] S501 : Determine a hierarchical prediction structure of a multi-view array, where the hierarchical prediction structure includes a first level and a second level, wherein the first level includes at least one view of the multi-view array, and the second level includes other views except the first level.

[0171] S502 : Perform predictive decoding on at least one viewpoint of the first layer to obtain a reference frame.

[0172] S503 : Perform predictive decoding on the viewpoint of the second level according to the reference frame to obtain a reconstructed image.

[0173] Specifically, after the decoder obtains the bitstream, it can determine the hierarchical prediction structure of the multi-view array based on the bitstream. Then, after the decoder obtains the video frames by reverse decoding according to the coding order of the viewpoint images in the first and second layers of the multi-view array, it can obtain the multi-view array based on the hierarchical prediction structure of the multi-view array.

[0174] In some embodiments, the multi-view array is obtained by shooting the same three-dimensional scene from multiple angles using multiple cameras.

[0175] In some embodiments, it is characterized in that the multi-view array includes a centrally symmetric multi-view array.

[0176] In some embodiments, the first level includes a first sub-level including a central viewpoint.

[0177] In some embodiments, performing predictive decoding on at least one viewpoint of the first layer to obtain a reference frame includes:

[0178] Intra-frame prediction decoding is performed on the central viewpoint of the first sub-level to obtain a first reference frame.

[0179] In some embodiments, the first level further includes a second sub-level including at least two views evenly distributed in the multi-view array.

[0180] In some embodiments, the second sub-level includes at least one of a plurality of viewpoints evenly distributed on edge lines of the multi-view array and viewpoints located between a center viewpoint and a vertex viewpoint on a diagonal line of the multi-view array.

[0181] In some embodiments, performing predictive decoding on at least one viewpoint of the first layer to obtain a reference frame includes:

[0182] Intra-frame prediction decoding or inter-frame prediction decoding is performed on at least two viewpoints of the second sub-level to obtain a second reference frame.

[0183] In some embodiments, the second level includes a third sub-level including at least two viewpoints located between the first sub-level and the second sub-level on a horizontal center axis and a vertical center axis of the multi-view array.

[0184] In some embodiments, the second level further includes a fourth sub-level, the fourth sub-level including viewpoints between viewpoints of the first sub-level located on edge lines of the multi-view array, and at least two viewpoints between the third sub-level and the first sub-level, and between the third sub-level and the second sub-level on the horizontal center axis and the vertical center axis of the multi-view array.

[0185] In some embodiments, the second level further includes a fifth sub-level, the fifth sub-level including at least two viewpoints between the second sub-level and the fourth sub-level on an edge line of the multi-view array, at least two viewpoints between the third sub-level and the second sub-level on a row of the multi-view array other than the edge line and the horizontal central axis, and between the second sub-level and the fourth sub-level.

[0186] In some embodiments, the second level further includes a sixth sub-level, and the sixth sub-level includes at least two viewpoints between the third sub-level and the second sub-level, and between the second sub-level and the fourth sub-level, located on columns of the multi-viewpoint array excluding edge lines and vertical central axes.

[0187] In some embodiments, the second level further includes a seventh sub-level including at least two views located in the same row as the sixth sub-level in rows excluding edge lines of the multi-view array.

[0188] In some embodiments, performing predictive decoding on the viewpoint of the second level according to the reference frame to obtain a reconstructed image includes:

[0189] According to the reference frame, the viewpoints in each sub-level in the second level are decoded in sequence from a lower sub-level to a higher sub-level, wherein the viewpoint of each sub-level in the sub-levels is predictively decoded with reference to the viewpoint of the same sub-level or a lower sub-level as the sub-level.

[0190] In some embodiments, decoding the views in each sub-level of the second level in sequence from a lower sub-level to a higher sub-level according to the reference frame comprises:

[0191] At least two rows of the multi-view array are decoded row by row according to a decoding order of a one-dimensional hierarchical decoding structure, wherein, within each row of the multi-view array, views other than the first sub-level and the second sub-level are decoded sequentially from a lower sub-level to a higher sub-level.

[0192] In some embodiments, it further includes:

[0193] determining a first partial multi-view array in the multi-view array, wherein the first partial multi-view array includes the central view;

[0194] The step of decoding at least two rows of the multi-view array row by row according to the decoding order of the one-dimensional layered decoding structure includes:

[0195] At least two rows of the first part of the multi-view array are decoded row by row according to a decoding order of a one-dimensional hierarchical decoding structure.

[0196] In some embodiments, the first portion of the multi-view array includes an upper half of the multi-view array, a lower half of the multi-view array, an upper right portion of the multi-view array, an upper left portion of the multi-view array, a lower left portion of the multi-view array, or a lower right portion of the multi-view array.

[0197] In some embodiments, the multi-view array further includes a second partial multi-view array, the second partial multi-view array includes the central viewpoint, and the first partial multi-view array and the second partial multi-view array share at least two viewpoints at a junction;

[0198] The method further comprises:

[0199] Deleting reconstructed images corresponding to viewpoints other than the at least two viewpoints shared by the boundary in the first portion of the multi-viewpoint array.

[0200] It should be noted that, in the embodiments of the present application, the specific process of the decoding method can refer to the process of the encoding method, which will not be repeated here. The encoding method provided by the embodiments of the present application can achieve a better encoding effect at the encoding end, improve the encoding compression efficiency, and correspondingly, can also improve the decoding performance at the decoder.

[0201] The preferred embodiments of the present application are described in detail above in conjunction with the accompanying drawings. However, the present application is not limited to the specific details in the above embodiments. Within the technical concept of the present application, a variety of simple modifications can be made to the technical solution of the present application, and these simple modifications all fall within the scope of protection of the present application. For example, the various specific technical features described in the above specific embodiments can be combined in any suitable manner unless there is any contradiction. In order to avoid unnecessary repetition, the present application will not further explain various possible combinations. For another example, the various different embodiments of the present application can also be arbitrarily combined, and as long as they do not violate the ideas of the present application, they should also be regarded as the contents disclosed in the present application.

[0202] It should also be understood that in the various method embodiments of the present application, the size of the sequence numbers of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. In addition, in the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships can exist. Specifically, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this application generally indicates that the related objects before and after are in an "or" relationship.

[0203] The method embodiment of the present application is described in detail above in conjunction with Figures 4 to 9 , and the device embodiment of the present application is described in detail below in conjunction with Figures 10 to 12 .

[0204] FIG10 is a schematic block diagram of an encoding device 10 provided in an embodiment of the present application, where the encoding device 10 is applied to the above-mentioned video decoding end.

[0205] As shown in FIG10 , the encoding device 10 includes:

[0206] a determining unit 11, configured to determine a hierarchical prediction structure of a multi-view array, wherein the hierarchical prediction structure comprises a first level and a second level, wherein the first level comprises at least one view of the multi-view array, and the second level comprises views other than the first level;

[0207] an encoding unit 12, configured to perform predictive encoding on at least one viewpoint of the first layer to obtain a reference frame;

[0208] The encoding unit 12 is further configured to perform predictive encoding on the viewpoints of the second level according to the reference frame to obtain a reconstructed image.

[0209] In some embodiments, the first level includes a first sub-level including a central viewpoint.

[0210] In some embodiments, the encoding unit 12 is specifically configured to:

[0211] Intra-frame prediction encoding is performed on the central viewpoint of the first sub-level to obtain a first reference frame.

[0212] In some embodiments, the first level further includes a second sub-level including at least two views evenly distributed in the multi-view array.

[0213] In some embodiments, the second sub-level includes at least one of a plurality of viewpoints evenly distributed on edge lines of the multi-view array and viewpoints located between a center viewpoint and a vertex viewpoint on a diagonal line of the multi-view array.

[0214] In some embodiments, the encoding unit 12 is specifically configured to:

[0215] Intra-frame prediction coding or inter-frame prediction coding is performed on at least two viewpoints of the second sub-level to obtain a second reference frame.

[0216] In some embodiments, the second level includes a third sub-level including at least two viewpoints located between the first sub-level and the second sub-level on a horizontal center axis and a vertical center axis of the multi-view array.

[0217] In some embodiments, the second level further includes a fourth sub-level, the fourth sub-level including viewpoints between viewpoints of the first sub-level located on edge lines of the multi-view array, and at least two viewpoints between the third sub-level and the first sub-level, and between the third sub-level and the second sub-level on the horizontal center axis and the vertical center axis of the multi-view array.

[0218] In some embodiments, the second level further includes a fifth sub-level, the fifth sub-level including at least two viewpoints between the second sub-level and the fourth sub-level on an edge line of the multi-view array, at least two viewpoints between the third sub-level and the second sub-level on a row of the multi-view array other than the edge line and the horizontal central axis, and between the second sub-level and the fourth sub-level.

[0219] In some embodiments, the second level further includes a sixth sub-level, and the sixth sub-level includes at least two viewpoints between the third sub-level and the second sub-level, and between the second sub-level and the fourth sub-level, located on columns of the multi-viewpoint array excluding edge lines and vertical central axes.

[0220] In some embodiments, the second level further includes a seventh sub-level including at least two views located in the same row as the sixth sub-level in rows excluding edge lines of the multi-view array.

[0221] In some embodiments, the encoding unit 12 is specifically configured to:

[0222] According to the reference frame, the viewpoints in each sub-level in the second level are sequentially encoded in an order from a lower sub-level to a higher sub-level, wherein the viewpoint of each sub-level in the sub-levels is predictively encoded with reference to the viewpoint of the same sub-level or a lower sub-level as each sub-level.

[0223] In some embodiments, the encoding unit 12 is specifically configured to:

[0224] At least two rows of the multi-view array are encoded row by row according to a coding order of a one-dimensional hierarchical coding structure, wherein the viewpoints other than the first sub-level and the second sub-level in each row of the multi-view array are encoded sequentially from a lower sub-level to a higher sub-level.

[0225] In some embodiments, the encoding unit 12 is specifically configured to:

[0226] determining a first partial multi-view array in the multi-view array, wherein the first partial multi-view array includes the central view;

[0227] At least two rows of the first part of the multi-view array are encoded row by row according to a coding order of a one-dimensional hierarchical coding structure.

[0228] In some embodiments, the first portion of the multi-view array includes an upper half of the multi-view array, a lower half of the multi-view array, an upper right portion of the multi-view array, an upper left portion of the multi-view array, a lower left portion of the multi-view array, or a lower right portion of the multi-view array.

[0229] In some embodiments, the multi-view array further includes a second partial multi-view array, the second partial multi-view array includes the central viewpoint, and the first partial multi-view array and the second partial multi-view array share at least two viewpoints at a junction;

[0230] The encoding unit 12 is further configured to delete reconstructed images corresponding to viewpoints other than the at least two viewpoints shared by the boundary in the first portion of the multi-viewpoint array.

[0231] In some embodiments, the multi-view array is obtained by shooting the same three-dimensional scene from multiple angles using multiple cameras.

[0232] It should be understood that the device embodiments and the method embodiments may correspond to each other, and similar descriptions may refer to the method embodiments. To avoid repetition, they will not be described here. Specifically, the device 10 shown in Figure 10 can execute the encoding method of the encoding end of the embodiment of the present application, and the aforementioned and other operations and / or functions of the various units in the device 10 are respectively for implementing the corresponding processes in each method such as the encoding method of the encoding end, etc., for the sake of brevity, they will not be described here.

[0233] FIG11 is a schematic block diagram of a decoding device provided in an embodiment of the present application, which is applied to the above-mentioned decoding end.

[0234] As shown in FIG11 , the decoding device 20 may include:

[0235] a determining unit 21 configured to determine a hierarchical prediction structure of a multi-view array, wherein the hierarchical prediction structure comprises a first level and a second level, wherein the first level comprises at least one view of the multi-view array, and the second level comprises views other than the first level;

[0236] a decoding unit 22, configured to perform predictive decoding on at least one viewpoint of the first layer to obtain a reference frame;

[0237] The decoding unit 22 is further configured to perform predictive decoding on the viewpoint of the second level according to the reference frame to obtain a reconstructed image.

[0238] In some embodiments, the first level includes a first sub-level including a central viewpoint.

[0239] In some embodiments, the decoding unit 22 is specifically configured to:

[0240] Intra-frame prediction decoding is performed on the central viewpoint of the first sub-level to obtain a first reference frame.

[0241] In some embodiments, the first level further includes a second sub-level including at least two views evenly distributed in the multi-view array.

[0242] In some embodiments, the second sub-level includes at least one of a plurality of viewpoints evenly distributed on edge lines of the multi-view array and viewpoints located between a center viewpoint and a vertex viewpoint on a diagonal line of the multi-view array.

[0243] In some embodiments, the decoding unit 22 is specifically configured to:

[0244] Intra-frame prediction decoding or inter-frame prediction decoding is performed on at least two viewpoints of the second sub-level to obtain a second reference frame.

[0245] In some embodiments, the second level includes a third sub-level including at least two viewpoints located between the first sub-level and the second sub-level on a horizontal center axis and a vertical center axis of the multi-view array.

[0246] In some embodiments, the second level further includes a fourth sub-level, the fourth sub-level including viewpoints between viewpoints of the first sub-level located on edge lines of the multi-view array, and at least two viewpoints between the third sub-level and the first sub-level, and between the third sub-level and the second sub-level on the horizontal center axis and the vertical center axis of the multi-view array.

[0247] In some embodiments, the second level further includes a fifth sub-level, the fifth sub-level including at least two viewpoints between the second sub-level and the fourth sub-level on an edge line of the multi-view array, at least two viewpoints between the third sub-level and the second sub-level on a row of the multi-view array other than the edge line and the horizontal central axis, and between the second sub-level and the fourth sub-level.

[0248] In some embodiments, the second level further includes a sixth sub-level, and the sixth sub-level includes at least two viewpoints between the third sub-level and the second sub-level, and between the second sub-level and the fourth sub-level, located on columns of the multi-viewpoint array excluding edge lines and vertical central axes.

[0249] In some embodiments, the second level further includes a seventh sub-level including at least two views located in the same row as the sixth sub-level in rows excluding edge lines of the multi-view array.

[0250] In some embodiments, performing inter-frame prediction decoding on the viewpoint of the second level according to the reference frame to obtain a reconstructed image includes:

[0251] According to the reference frame, the viewpoints in each sub-level in the second level are decoded in sequence from a lower sub-level to a higher sub-level, wherein the viewpoint of each sub-level in the sub-levels is predictively decoded with reference to the viewpoint of the same sub-level or a lower sub-level as the sub-level.

[0252] In some embodiments, the decoding unit 22 is specifically configured to:

[0253] At least two rows of the multi-view array are decoded row by row according to a decoding order of a one-dimensional hierarchical decoding structure, wherein, within each row of the multi-view array, views other than the first sub-level and the second sub-level are decoded sequentially from a lower sub-level to a higher sub-level.

[0254] In some embodiments, the decoding unit 22 is specifically configured to:

[0255] determining a first partial multi-view array in the multi-view array, wherein the first partial multi-view array includes the central view;

[0256] At least two rows of the first part of the multi-view array are decoded row by row according to a decoding order of a one-dimensional hierarchical decoding structure.

[0257] In some embodiments, the first portion of the multi-view array includes an upper half of the multi-view array, a lower half of the multi-view array, an upper right portion of the multi-view array, an upper left portion of the multi-view array, a lower left portion of the multi-view array, or a lower right portion of the multi-view array.

[0258] In some embodiments, the multi-view array further includes a second partial multi-view array, the second partial multi-view array includes the central viewpoint, and the first partial multi-view array and the second partial multi-view array share at least two viewpoints at a junction;

[0259] The decoding unit 22 is further configured to delete reconstructed images corresponding to viewpoints other than the at least two viewpoints shared by the boundary in the first portion of the multi-viewpoint array.

[0260] In some embodiments, the multi-view array is obtained by shooting the same three-dimensional scene from multiple angles using multiple cameras.

[0261] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions can refer to the method embodiment. To avoid repetition, no further description is given here. Specifically, the device 20 shown in Figure 11 can correspond to the corresponding subject in the prediction method of the decoding end of the embodiment of the present application, and the aforementioned and other operations and / or functions of each unit in the device 20 are respectively for implementing the corresponding processes in each method such as the decoding method of the decoding end. For the sake of brevity, no further description is given here.

[0262] The above describes the apparatus and system of the embodiment of the present application from the perspective of functional units in conjunction with the accompanying drawings. It should be understood that the functional unit can be implemented in the form of hardware, can be implemented by instructions in the form of software, or can be implemented by a combination of hardware and software units. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software units in the decoding processor. Optionally, the software unit can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps in the above method embodiment in conjunction with its hardware.

[0263] FIG12 is a schematic block diagram of an electronic device provided in an embodiment of the present application.

[0264] As shown in FIG12 , the electronic device 30 may be a video encoder or a video decoder as described in an embodiment of the present application. The electronic device 30 may include:

[0265] The memory 33 and the processor 32 are configured to store a computer program 34 and transmit the program code 34 to the processor 32. In other words, the processor 32 can call and run the computer program 34 from the memory 33 to implement the method in the embodiment of the present application.

[0266] For example, the processor 32 may be configured to execute the steps of the above-described method 400 or 500 according to the instructions in the computer program 34 .

[0267] In some embodiments of the present application, the processor 32 may include but is not limited to:

[0268] General-purpose processor, Digital Signal Processor (DSP), Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0269] In some embodiments of the present application, the memory 33 includes but is not limited to:

[0270] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0271] In some embodiments of the present application, the computer program 34 may be divided into one or more units, which are stored in the memory 33 and executed by the processor 32 to implement the method provided by the present application. The one or more units may be a series of computer program instruction segments capable of implementing specific functions, and the instruction segments are used to describe the execution process of the computer program 34 in the electronic device 30.

[0272] As shown in FIG12 , the electronic device 30 may further include:

[0273] The transceiver 33 may be connected to the processor 32 or the memory 33 .

[0274] The processor 32 can control the transceiver 33 to communicate with other devices. Specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 33 may include a transmitter and a receiver. The transceiver 33 may further include an antenna, and the number of antennas may be one or more.

[0275] It should be understood that the various components in the electronic device 30 are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0276] The present application also provides a computer storage medium having a computer program stored thereon, which, when executed by a computer, enables the computer to perform the method of the above-mentioned method embodiment. In other words, the present application also provides a computer program product containing instructions, which, when executed by a computer, enables the computer to perform the method of the above-mentioned method embodiment.

[0277] The present application also provides a code stream, which is generated according to the above encoding method. Optionally, the code stream includes the above first flag, or includes the first flag and the second flag.

[0278] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (DVD)), or a semiconductor medium (e.g., a solid state drive (SSD)).

[0279] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0280] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the unit is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0281] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment. For example, the functional units in the various embodiments of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0282] The above content is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A coding method, characterized in that: include: Determining a hierarchical prediction structure for a multi-view array, wherein the hierarchical prediction structure comprises a first level and a second level, the first level comprising at least one view of the multi-view array, the second level comprising views other than the first level; performing predictive encoding on at least one viewpoint of the first level to obtain a reference frame; According to the reference frame, predictive encoding is performed on the viewpoint of the second level to obtain a reconstructed image.

2. The method according to claim 1, characterized in that The first hierarchy includes a first sub-hierarchy including a central viewpoint.

3. The method according to claim 2, characterized in that The predictive encoding of at least one viewpoint of the first level to obtain a reference frame includes: Intra-frame prediction encoding is performed on the central viewpoint of the first sub-level to obtain a first reference frame.

4. The method according to claim 2, characterized in that The first level further includes a second sub-level including at least two views evenly distributed in the multi-view array.

5. The method according to claim 4, characterized in that The second sub-level includes at least one of a plurality of viewpoints evenly distributed on edge lines of the multi-viewpoint array and viewpoints located between a center viewpoint and a vertex viewpoint on a diagonal line of the multi-viewpoint array.

6. The method according to claim 4, characterized in that The predictive encoding of at least one viewpoint of the first level to obtain a reference frame includes: Intra-frame prediction coding or inter-frame prediction coding is performed on at least two viewpoints of the second sub-level to obtain a second reference frame.

7. The method according to claim 4, characterized in that The second level includes a third sub-level including at least two viewpoints located between the first sub-level and the second sub-level on a horizontal central axis and a vertical central axis of the multi-viewpoint array.

8. The method according to claim 7, characterized in that The second level also includes a fourth sub-level, which includes viewpoints between the viewpoints of the first sub-level located on the edge line of the multi-view array, and at least two viewpoints between the third sub-level and the first sub-level, and between the third sub-level and the second sub-level on the horizontal central axis and the vertical central axis of the multi-view array.

9. The method according to claim 8, characterized in that The second level also includes a fifth sub-level, which includes at least two viewpoints between the second sub-level and the fourth sub-level on the edge line of the multi-view array, between the third sub-level and the second sub-level on the rows of the multi-view array other than the edge line and the horizontal central axis, and between the second sub-level and the fourth sub-level.

10. The method according to claim 9, characterized in that The second level also includes a sixth sub-level, which includes at least two viewpoints located between the third sub-level and the second sub-level, and between the second sub-level and the fourth sub-level on columns of the multi-viewpoint array excluding edge lines and a vertical central axis.

11. The method according to claim 10, characterized in that The second hierarchy further includes a seventh sub-level including at least two views located in the same row as the sixth sub-level in rows excluding edge lines of the multi-view array.

12. The method according to any one of claims 7 to 11, characterized in that: The step of performing predictive coding on the viewpoint of the second level according to the reference frame to obtain a reconstructed image includes: According to the reference frame, the viewpoints in each sub-level in the second level are sequentially encoded in an order from a lower sub-level to a higher sub-level, wherein the viewpoint of each sub-level in the sub-levels is predictively encoded with reference to the viewpoint of the same sub-level or a lower sub-level as each sub-level.

13. The method according to claim 12, characterized in that The encoding of the viewpoints in each sub-level of the second level in sequence from a lower sub-level to a higher sub-level according to the reference frame includes: At least two rows of the multi-view array are encoded row by row according to a coding order of a one-dimensional hierarchical coding structure, wherein the viewpoints other than the first sub-level and the second sub-level in each row of the multi-view array are encoded sequentially from a lower sub-level to a higher sub-level.

14. The method according to claim 13, characterized in that Also includes: determining a first partial multi-view array in the multi-view array, wherein the first partial multi-view array includes the central view; The step of encoding at least two rows of the multi-view array row by row according to the coding order of the one-dimensional layered coding structure comprises: At least two rows of the first part of the multi-view array are encoded row by row according to a coding order of a one-dimensional hierarchical coding structure.

15. The method according to claim 14, characterized in that The first portion of the multi-view array includes an upper half of the multi-view array, a lower half of the multi-view array, an upper right portion of the multi-view array, an upper left portion of the multi-view array, a lower left portion of the multi-view array, or a lower right portion of the multi-view array.

16. The method according to claim 14 or 15, characterized in that The multi-view array further includes a second partial multi-view array, the second partial multi-view array includes the central viewpoint, and the first partial multi-view array and the second partial multi-view array share at least two viewpoints at a junction; The method further comprises: Deleting reconstructed images corresponding to viewpoints other than the at least two viewpoints shared by the boundary in the first portion of the multi-viewpoint array.

17. The method according to any one of claims 1 to 16, characterized in that The multi-viewpoint array is obtained by shooting the same three-dimensional scene from multiple angles using multiple cameras.

18. A decoding method, characterized in that: include: Determining a hierarchical prediction structure for a multi-view array, wherein the hierarchical prediction structure comprises a first level and a second level, the first level comprising at least one view of the multi-view array, the second level comprising views other than the first level; Performing predictive decoding on at least one viewpoint of the first layer to obtain a reference frame; According to the reference frame, the viewpoint of the second level is predictively decoded to obtain a reconstructed image.

19. The method according to claim 18, characterized in that The first hierarchy includes a first sub-hierarchy including a central viewpoint.

20. The method according to claim 19, characterized in that The predictive decoding of at least one viewpoint of the first layer to obtain a reference frame includes: Intra-frame prediction decoding is performed on the central viewpoint of the first sub-level to obtain a first reference frame.

21. The method according to claim 19, wherein The first level further includes a second sub-level including at least two views evenly distributed in the multi-view array.

22. The method according to claim 21, characterized in that The second sub-level includes at least one of a plurality of viewpoints evenly distributed on edge lines of the multi-viewpoint array and viewpoints located between a center viewpoint and a vertex viewpoint on a diagonal line of the multi-viewpoint array.

23. The method according to claim 21, characterized in that The predictive decoding of at least one viewpoint of the first layer to obtain a reference frame includes: Intra-frame prediction decoding or inter-frame prediction decoding is performed on at least two viewpoints of the second sub-level to obtain a second reference frame.

24. The method according to claim 21, characterized in that The second level includes a third sub-level including at least two viewpoints located between the first sub-level and the second sub-level on a horizontal central axis and a vertical central axis of the multi-viewpoint array.

25. The method according to claim 24, characterized in that The second level also includes a fourth sub-level, which includes viewpoints between the viewpoints of the first sub-level located on the edge line of the multi-view array, and at least two viewpoints between the third sub-level and the first sub-level, and between the third sub-level and the second sub-level on the horizontal central axis and the vertical central axis of the multi-view array.

26. The method according to claim 25, characterized in that The second level also includes a fifth sub-level, which includes at least two viewpoints between the second sub-level and the fourth sub-level on the edge line of the multi-view array, between the third sub-level and the second sub-level on the rows of the multi-view array other than the edge line and the horizontal central axis, and between the second sub-level and the fourth sub-level.

27. The method according to claim 26, characterized in that The second level also includes a sixth sub-level, which includes at least two viewpoints located between the third sub-level and the second sub-level, and between the second sub-level and the fourth sub-level on columns of the multi-viewpoint array excluding edge lines and a vertical central axis.

28. The method according to claim 27, characterized in that The second hierarchy further includes a seventh sub-level including at least two views located in the same row as the sixth sub-level in rows excluding edge lines of the multi-view array.

29. The method according to any one of claims 24 to 28, characterized in that The predictive decoding of the second-level viewpoint according to the reference frame to obtain a reconstructed image includes: According to the reference frame, the viewpoints in each sub-level in the second level are decoded in sequence from a lower sub-level to a higher sub-level, wherein the viewpoint of each sub-level in the sub-levels is predictively decoded with reference to the viewpoint of the same sub-level or a lower sub-level as the sub-level.

30. The method according to claim 29, wherein The decoding of the viewpoints in each sub-level in the second level in sequence from a lower sub-level to a higher sub-level according to the reference frame includes: At least two rows of the multi-view array are decoded row by row according to a decoding order of a one-dimensional hierarchical decoding structure, wherein, within each row of the multi-view array, views other than the first sub-level and the second sub-level are decoded sequentially from a lower sub-level to a higher sub-level.

31. The method according to claim 30, characterized in that Also includes: determining a first partial multi-view array in the multi-view array, wherein the first partial multi-view array includes the central view; The step of decoding at least two rows of the multi-view array row by row according to the decoding order of the one-dimensional layered decoding structure includes: At least two rows of the first part of the multi-view array are decoded row by row according to a decoding order of a one-dimensional hierarchical decoding structure.

32. The method according to claim 31, characterized in that The first portion of the multi-view array includes an upper half of the multi-view array, a lower half of the multi-view array, an upper right portion of the multi-view array, an upper left portion of the multi-view array, a lower left portion of the multi-view array, or a lower right portion of the multi-view array.

33. The method according to claim 31 or 32, characterized in that The multi-view array further includes a second partial multi-view array, the second partial multi-view array includes the central viewpoint, and the first partial multi-view array and the second partial multi-view array share at least two viewpoints at a junction; The method further comprises: Deleting reconstructed images corresponding to viewpoints other than the at least two viewpoints shared by the boundary in the first portion of the multi-viewpoint array.

34. The method according to any one of claims 18 to 33, wherein: The multi-viewpoint array is obtained by shooting the same three-dimensional scene from multiple angles using multiple cameras.

35. An encoding device, characterized in that include: a determining unit, configured to determine a hierarchical prediction structure of a multi-view array, wherein the hierarchical prediction structure comprises a first level and a second level, the first level comprising at least one view of the multi-view array, and the second level comprising views other than the first level; a coding unit, configured to perform predictive coding on at least one viewpoint of the first layer to obtain a reference frame; The encoding unit is further configured to perform inter-frame prediction encoding on the viewpoint of the second level according to the reference frame to obtain a reconstructed image.

36. A decoding device, characterized in that: include: a determining unit, configured to determine a hierarchical prediction structure of a multi-view array, wherein the hierarchical prediction structure comprises a first level and a second level, the first level comprising at least one view of the multi-view array, and the second level comprising views other than the first level; a decoding unit, performing predictive decoding on at least one viewpoint of the first layer to obtain a reference frame; The decoding unit is further configured to perform inter-frame prediction decoding on the viewpoint of the second level according to the reference frame to obtain a reconstructed image.

37. An electronic device, characterized in that: including processor and memory; The memory shown is used to store computer programs; The processor is configured to call and execute the computer program stored in the memory to implement the method according to any one of claims 1 to 34.

38. A computer-readable storage medium, characterized in that For storing computer programs; The computer program enables a computer to execute the method according to any one of claims 1 to 34.

39. A code stream, characterized in that The code stream is generated based on the method according to any one of claims 1 to 16.

Images