Encoding and decoding method, code stream, encoder, decoder and storage medium

Abstract

The embodiment of the invention discloses an encoding and decoding method, a code stream, an encoder, a decoder and a storage medium, the method comprising: decoding the code stream, and determining filtering identification information of a first video image (S401); if the filtering identification information indicates that the first video image uses the first filtering processing, decoding the code stream, and determining a filtering coefficient of the first video image (S402); and when at least one pixel point in the first video image satisfies a preset neighborhood condition, performing first filtering processing on the at least one pixel point according to the filtering coefficient, and determining a filtered image corresponding to the first video image (S403). Therefore, the time complexity can be reduced, the geometric reconstruction quality of the point cloud can be improved, and the coding and decoding performance of the point cloud is further improved.

Description

Coding and decoding method, code stream, encoder, decoder and storage medium Technical Field

[0001] The embodiments of the present application relate to the field of point cloud encoding and decoding technology, and in particular to an encoding and decoding method, a bit stream, an encoder, a decoder, and a storage medium. Background Art

[0002] Currently, the Video-based Point Cloud Compression (V-PCC) codec framework includes both smoothing operations on the decoded video frames after lossy compression and post-processing algorithms for the reconstructed point cloud. After basic reconstruction of the decoded video frames, only image smoothing is performed, leaving significant room for improvement in the quality of the reconstructed point cloud.

[0003] To improve the quality of reconstructed point clouds, filtering can be performed on geometric video frames in the V-PCC codec framework. However, existing filtering solutions still have some problems.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide a coding and decoding method, code stream, encoder, decoder and storage medium, which can not only reduce the time complexity, but also improve the geometric reconstruction quality of the point cloud, thereby improving the coding and decoding performance of the point cloud.

[0006] The technical solution of the embodiment of the present application can be implemented as follows:

[0007] In a first aspect, an embodiment of the present application provides a decoding method, applied to a decoder, the method comprising:

[0008] Decoding the code stream to determine filter identification information of the first video image;

[0009] If the filter identification information indicates that the first video image is to be processed using the first filter, decoding the bitstream to determine a filter coefficient for the first video image;

[0010] When at least one pixel point in the first video image meets a preset neighborhood condition, a first filtering process is performed on the at least one pixel point according to a filtering coefficient to determine a filtered image corresponding to the first video image.

[0011] In a second aspect, an embodiment of the present application provides an encoding method, applied to an encoder, the method comprising:

[0012] Determining at least one pixel point in the first video image that meets a preset neighborhood condition;

[0013] Determining a filter coefficient of the first video image according to at least one pixel point;

[0014] Performing a first filtering process on at least one pixel point according to a filtering coefficient to determine a filtered image of the first video image;

[0015] Determining filtering identification information of the first video image according to the first video image and the filtered image;

[0016] If the filter identification information indicates that the first video image uses the first filter processing, the filter identification information and the corresponding filter coefficients of the first video image are encoded, and the obtained encoding bits are written into the bitstream.

[0017] In a third aspect, an embodiment of the present application provides a code stream, which is generated by bit encoding based on information to be encoded; wherein the information to be encoded includes at least one of the following:

[0018] Filter activation identification information, total number of frames, category information, filter identification information of at least one video image, and corresponding filter coefficients;

[0019] Among them, the filter enable identification information is used to indicate whether the video sequence uses the first filtering process, the filter identification information is used to indicate whether the first video image uses the first filtering process, the video sequence includes at least one video image, and the first video image is any one of the at least one video image.

[0020] In a fourth aspect, an embodiment of the present application provides an encoder, comprising a first determining unit, a first filtering unit, and an encoding unit; wherein,

[0021] A first determining unit is configured to determine at least one pixel point in the first video image that meets a preset neighborhood condition; and determine a filter coefficient of the first video image based on the at least one pixel point;

[0022] a first filtering unit configured to perform a first filtering process on at least one pixel point according to a filtering coefficient to determine a filtered image of the first video image;

[0023] The first determining unit is further configured to determine filtering identification information of the first video image based on the first video image and the filtered image;

[0024] The encoding unit is configured to encode the filter identification information and the corresponding filter coefficients of the first video image if the filter identification information indicates that the first video image uses the first filter processing, and write the obtained encoding bits into the bit stream.

[0025] In a fifth aspect, an embodiment of the present application provides an encoder, comprising a first memory and a first processor; wherein,

[0026] a first memory for storing a computer program capable of running on the first processor;

[0027] The first processor is configured to execute the method according to the second aspect when running a computer program.

[0028] In a sixth aspect, an embodiment of the present application provides a decoder, the decoder comprising a decoding unit and a second filtering unit; wherein,

[0029] a decoding unit configured to decode the code stream and determine filter identification information of the first video image;

[0030] The decoding unit is further configured to decode the code stream and determine the filter coefficient of the first video image if the filter identification information indicates that the first video image uses the first filter processing;

[0031] The second filtering unit is configured to perform a first filtering process on the at least one pixel point in the first video image according to the filtering coefficient when the at least one pixel point in the first video image meets the preset neighborhood condition, and determine a filtered image corresponding to the first video image.

[0032] In a seventh aspect, an embodiment of the present application provides a decoder, the decoder comprising a second memory and a second processor; wherein,

[0033] a second memory for storing a computer program capable of running on the second processor;

[0034] The second processor is configured to execute the method according to the first aspect when running a computer program.

[0035] In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which, when executed, implements the method described in the first aspect or the method described in the second aspect.

[0036] The embodiments of the present application provide a coding and decoding method, a bitstream, an encoder, a decoder, and a storage medium. At the encoding end, after determining at least one pixel point that meets a preset neighborhood condition based on a first video image, a filter coefficient of the first video image is determined based on the at least one pixel point; then, a first filtering process is performed on the at least one pixel point based on the filter coefficient to determine a filtered image of the first video image; and, based on the first video image and the filtered image, filter identification information of the first video image is determined; if the filter identification information indicates that the first video image uses the first filtering process, the filter identification information of the first video image and the corresponding filter coefficient are encoded, and the obtained coded bits are written into the bitstream. At the decoding end, the bitstream is decoded to determine the filter identification information of the first video image; if the filter identification information indicates that the first video image uses the first filtering process, the bitstream is decoded to determine the filter coefficient of the first video image; when at least one pixel point in the first video image meets the preset neighborhood condition, a first filtering process is performed on the at least one pixel point based on the filter coefficient to determine the filtered image corresponding to the first video image. In this way, since the preset neighborhood condition at the encoding end is pre-set based on the bit rate of the first video image, there is no need to perform multiple coefficient calculations to determine the optimal preset neighborhood condition, thereby reducing time complexity. In addition, after the encoding end determines at least one pixel point that meets the preset neighborhood condition from the first video image, it can also determine the corresponding filter coefficient based on the at least one pixel point. And only when the first video image needs to be processed by the first filtering, the corresponding filter coefficient and the category information indicating the preset neighborhood condition are transmitted to the decoding end. Accordingly, the decoding end can directly decode to obtain the filter coefficient and determine the corresponding preset neighborhood condition, and then use the filter coefficient to perform the first filtering process on the at least one pixel point that meets the preset neighborhood condition. This not only reduces time complexity, but also optimizes the reconstructed video image and improves the reconstructed geometric quality of the point cloud. At the same time, the present application also standardizes the design of the writing / reading method of the code stream to ensure the stability of the codec system operation and the rationality of the code stream structure. In this way, not only is the quality of the reconstructed point cloud enhanced, but the quality of the video frame can also be improved, further improving compression efficiency and improving codec performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a schematic diagram of a point cloud encoding and decoding network architecture;

[0038] FIG2 is a schematic diagram of a V-PCC encoder framework;

[0039] FIG3 is a schematic diagram of a V-PCC decoder component framework;

[0040] FIG4 is a schematic diagram of a flow chart of a decoding method provided in an embodiment of the present application;

[0041] FIG5A is a first schematic diagram of a pixel neighborhood provided in an embodiment of the present application;

[0042] FIG5B is a second schematic diagram of a pixel neighborhood provided in an embodiment of the present application;

[0043] FIG5C is a third schematic diagram of a pixel neighborhood provided in an embodiment of the present application;

[0044] FIG6A is a first schematic diagram of pixel values ​​of a pixel point and its neighboring pixel points provided in an embodiment of the present application;

[0045] FIG6B is a second schematic diagram of pixel values ​​of a pixel point and its neighboring pixel points provided in an embodiment of the present application;

[0046] FIG6C is a third schematic diagram of pixel values ​​of a pixel point and its neighboring pixel points provided in an embodiment of the present application;

[0047] FIG7 is a schematic diagram of a diamond-shaped filter window provided in an embodiment of the present application;

[0048] FIG8 is a flowchart diagram 1 of an encoding method provided in an embodiment of the present application;

[0049] FIG9 is a second flow chart of an encoding method provided in an embodiment of the present application;

[0050] FIG10 is a third flow chart of an encoding method provided in an embodiment of the present application;

[0051] FIG11 is a schematic diagram of a code stream structure provided in an embodiment of the present application;

[0052] FIG12 is a schematic diagram of a partial architecture of a V-PCC encoder with Wiener filtering added according to an embodiment of the present application;

[0053] FIG13 is a second schematic diagram of a partial architecture of a V-PCC encoder with Wiener filtering added, provided by an embodiment of the present application;

[0054] FIG14 is a schematic diagram of an exemplary video geometric frame image provided in an embodiment of the present application;

[0055] FIG15 is a schematic diagram of the structure of an encoder provided in an embodiment of the present application;

[0056] FIG16 is a schematic diagram of a specific hardware structure of an encoder provided in an embodiment of the present application;

[0057] FIG17 is a schematic diagram of the structure of a decoder provided in an embodiment of the present application;

[0058] FIG18 is a schematic diagram of a specific hardware structure of a decoder provided in an embodiment of the present application;

[0059] FIG19 is a schematic diagram of the composition structure of a coding and decoding system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0060] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below with reference to the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0061] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0062] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0063] It should also be pointed out that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0064] Before further explaining the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained first. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations:

[0065] Geometry-based Point Cloud Compression (G-PCC or GPCC);

[0066] Video-based Point Cloud Compression (V-PCC or VPCC);

[0067] Wiener Filter;

[0068] Occupancy map;

[0069] Geometry video / frame;

[0070] Texture video / frame;

[0071] K Nearest Neighbor (KNN);

[0072] Three Dimension (3D);

[0073] Sum Squared Error (SSE);

[0074] Minimum Mean Squared Error (MMSE);

[0075] Rate Distortion Optimization (RDO);

[0076] Peak Signal to Noise Ratio (PSNR);

[0077] Performance indicators (Bjφntegaard Delta Rate, BD-Rate);

[0078] Rate Distortion Cost (RDCost);

[0079] Luminance component (L or Y);

[0080] Chroma blue (Cb);

[0081] Red chroma component (Chroma red, Cr);

[0082] Bitstream;

[0083] V-PCC code stream unit (Sample Stream V3C Unit, SSVU).

[0084] Point cloud is a three-dimensional representation of the surface of an object. Point cloud (data) of the surface of an object can be collected through acquisition equipment such as photoelectric radar, lidar, laser scanner, and multi-view camera.

[0085] In a two-dimensional image, each pixel contains information and is distributed regularly, so there's no need to record its location. However, the distribution of points in a point cloud in three-dimensional space is random and irregular, so recording the location of each point in space is necessary to fully represent the point cloud. Similar to a two-dimensional image, each location in the acquisition process has corresponding attribute information, typically an RGB color value, which reflects the object's color. For a point cloud, in addition to color information, each point's attribute information often includes a reflectance value, which reflects the surface texture of the object. Therefore, a point in a point cloud can include both location information and attribute information. For example, the location information of a point can be its three-dimensional coordinates (x, y, z). The location information of a point can also be referred to as its geometric information. For example, the attribute information of a point can include color information (three-dimensional color information) and / or reflectance (one-dimensional reflectance information r). For example, the color information can be information in any color space. For example, the color information can be RGB information, where R represents red (R), G represents green (G), and B represents blue (B). For another example, the color information may be luminance and chrominance (YCbCr, YUV) information, where Y represents brightness (Luma), Cb (U) represents blue color difference, and Cr (V) represents red color difference.

[0086] For example, a point cloud generated using laser measurement principles can include both its 3D coordinate information and its reflectivity. For another example, a point cloud generated using photogrammetry principles can include both its 3D coordinate information and its 3D color information. For another example, a point cloud generated using a combination of laser measurement and photogrammetry principles can include both its 3D coordinate information, its reflectivity value, and its 3D color information.

[0087] Point clouds can be divided into the following categories according to the acquisition method:

[0088] The first type of static point cloud: the object is stationary and the device used to obtain the point cloud is also stationary;

[0089] The second type of dynamic point cloud: the object is moving, but the device that obtains the point cloud is stationary;

[0090] The third type of dynamic point cloud acquisition: the device that acquires the point cloud is moving.

[0091] For example, point clouds can be divided into two categories according to their usage:

[0092] Category 1: Machine perception point cloud, which can be used in scenarios such as autonomous navigation systems, real-time inspection systems, geographic information systems, visual sorting robots, and disaster relief robots;

[0093] Category 2: Human eye perception point cloud, which can be used in point cloud application scenarios such as digital cultural heritage, free viewpoint broadcasting, 3D immersive communication, and 3D immersive interaction.

[0094] Since point clouds are a collection of massive points, storing point clouds not only consumes a lot of memory, but is also not conducive to transmission. There is also not enough bandwidth to support direct transmission of point clouds at the network layer without compression. Therefore, point clouds need to be compressed.

[0095] Currently, point cloud coding frameworks that can compress point clouds include the G-PCC or V-PCC codecs provided by the Moving Picture Experts Group (MPEG), or the AVS-PCC codec provided by the Audio Video Standard (AVS). The G-PCC codec can be used to compress the first type of static point clouds and the third type of dynamically acquired point clouds, while the V-PCC codec can be used to compress the second type of dynamic point clouds.

[0096] In an embodiment of the present application, a network architecture of a point cloud encoding and decoding system including a decoding method and an encoding method is provided. FIG1 is a schematic diagram of a network architecture of a point cloud encoding and decoding system. As shown in FIG1 , the network architecture includes one or more electronic devices 13 to 1N and a communication network 01, wherein the electronic devices 13 to 1N can perform video interaction through the communication network 01. During the implementation process, the electronic device can be various types of devices with point cloud encoding and decoding functions. For example, the electronic device can include a mobile phone, a tablet computer, a personal computer, a personal digital assistant, a navigator, a digital phone, a video phone, a television, a sensor device, a server, etc., which is not limited by the embodiment of the present application. Among them, the decoder or encoder in the embodiment of the present application can be the above-mentioned electronic device.

[0097] Among them, the electronic device in the embodiment of the present application has a point cloud encoding and decoding function, generally including a point cloud encoder (ie, encoder) and a point cloud decoder (ie, decoder).

[0098] The following describes the relevant technologies using the V-PCC codec framework as an example.

[0099] Figure 2 is a schematic diagram of the composition framework of a V-PCC encoder. As shown in Figure 2, the V-PCC encoder is applied to a point cloud encoder. In the V-PCC encoding framework, for the point cloud frame to be encoded, the 3D patch generation module (3D patch generation) 11 generates a 3D patch based on the input point cloud frame, and then outputs it to the patch packing module (patch packing) 13, the geometry image generation module (Geometry image generation) 19, the texture frame generation module (Attribute image generation) 15, the patch sequence compression module (Patch sequence compression) 27 and the smoothing module (Smoothing) 17 respectively. The patch encapsulation module 13 encapsulates the patch, generates an occupancy map and outputs it to the geometry frame generation module 19 and the first video compression module 21. The first video compression module 21 compresses the occupancy map, outputs the compressed occupancy substream to the multiplexer 33, and outputs the reconstructed occupancy map to the geometry frame padding module 23, the texture frame padding module 25, the texture frame generation module 15 and the smoothing module 17. The geometry frame generation module 19 generates a geometry frame based on the input occupancy map, 3D patch, and point cloud frame, and outputs it to the geometry frame filling module 23. The geometry frame filling module 23 outputs a padded geometry frame (padded geometry) based on the input geometry frame and the reconstructed occupancy map to the second video compression module 31. The second video compression module 31 outputs a compressed geometry substream to the multiplexer 33 and a reconstructed geometry frame to the smoothing module 17. The smoothing module 17 smoothes the reconstructed geometry frame based on the reconstructed occupancy map and 3D patch, and outputs the smoothed reconstructed geometry frame to the texture frame generation module 15. The texture frame generation module 15 generates a texture frame based on the input smoothed reconstructed geometry frame, 3D patch, reconstructed occupancy map, and point cloud frame, and outputs it to the texture frame filling module 25. The texture frame filling module 25 outputs the padded texture frame to the third video compression module 29. The third video compression module 29 outputs the compressed texture substream to the multiplexer 33. The Patch sequence compression module 27 outputs the compressed Patch sub-stream to the multiplexer 33 .The multiplexer 33 multiplexes the input patch sub-stream, texture sub-stream, geometry sub-stream and occupancy sub-stream and outputs a compressed bitstream.

[0100] Figure 3 is a schematic diagram of the composition framework of a V-PCC decoder. As shown in Figure 3, the V-PCC decoder is applied to the point cloud decoder. In the V-PCC decoding framework, the compressed code stream is demultiplexed by the demultiplexer 41, and the sequence parameter set (SPS), patch substream, texture substream, geometry substream and occupancy substream are output. The SPS parsing module 43 performs syntax analysis on the SPS and outputs syntax elements to the patch sequence decompression module 45, the first video decompression module 47, the second video decompression module 49, the third video decompression module 51, the geometry and texture reconstruction module 53, the geometry post-processing module 55 and the texture conversion and smoothing module 57. The patch sequence decompression module 45 decompresses the input patch substream according to the syntax elements and outputs patch information (patch information). The first video decompression module 47 decompresses the input occupancy substream according to the syntax elements and outputs an occupancy map to the geometry and texture reconstruction module 53. The second video decompression module 49 decompresses the input geometry substream according to the syntax elements and outputs a geometry frame to the geometry and texture reconstruction module 53. The third video decompression module 51 decompresses the input texture substream according to the syntax elements and outputs a texture frame to the geometry and texture reconstruction module 53. The geometry and texture reconstruction module 53 obtains a reconstructed geometry frame and a reconstructed texture frame based on the input syntax elements, patch information, occupancy map, geometry frame, and texture frame. The geometry and texture reconstruction module 53 outputs the reconstructed geometry frame to the geometry post-processing module 55, which then outputs the reconstructed texture frame to the texture conversion and smoothing module 57. The geometry post-processing module 55 smoothes the reconstructed geometry frame according to the syntax elements and outputs it to the texture conversion and smoothing module 57. The texture conversion and smoothing module 57 performs texture conversion and smoothing based on the input syntax elements, texture frame, and the smoothed reconstructed geometry frame, and outputs a smoothed reconstructed point cloud frame.

[0101] V-PCC point cloud smoothing post-processing includes both smoothing the video obtained by decoding after lossy compression and post-processing algorithms for the reconstructed point cloud. Figures 2 and 3 illustrate the V-PCC encoding and decoding framework. Specifically, V-PCC performs point cloud smoothing twice: the first smoothing operation is performed after the reconstructed frame is generated; the second smoothing operation is performed after the reconstructed point cloud is generated.

[0102] (1) For texture video frames, V-PCC addresses the problem that non-adjacent patches in three-dimensional space may be very close to each other within the image, and patch-based encoders may confuse adjacent pixels, resulting in reconstruction artifacts. A color smoothing algorithm is proposed. In the reconstructed frame, the occupancy map is used to find the pixels corresponding to the patch boundary points, and then a median filter is used for smoothing. For geometric video frames, V-PCC addresses the problem that patches may be discontinuous in the reconstructed point cloud. A boundary filtering algorithm is proposed to find the pixels corresponding to the patch boundary points and locally change the depth value of the patch edge.

[0103] (2) For the reconstructed point cloud, to address the problem of patch discontinuity, 3D points are added to make the boundary continuous. To address the problem of outliers and noise caused by distortion, the reconstructed point cloud is gridded and trilinear filtering is performed on the patch edge points using the center of the grid to improve the visual effect of the point cloud.

[0104] However, after basic reconstruction of decoded video frames, the V-PCC codec framework only smooths the image or point cloud, leaving significant room for improvement in the quality of the reconstructed point cloud. To further improve compression performance, it is necessary to achieve enhanced point cloud quality while maintaining minimal changes in the bitrate.

[0105] Meanwhile, Wiener filtering for video frames in the V-PCC codec framework can significantly improve the geometric quality of reconstructed point clouds. However, Wiener filtering for images has some drawbacks. For example, when the filter order K is constant, using the same set of coefficients to filter large-scale images or images with significant local variations does not yield good image quality enhancement. Increasing K alone only slightly improves filtering effectiveness, while also increasing the bitrate size and time complexity, potentially worsening overall performance. The more commonly used adaptive loop filter, also based on Wiener filtering, classifies filters into 25 categories. The gradient direction and degree of variation for each 4×4 patch in the image are calculated, and the filter category is determined based on this information. The coefficients are then calculated and filtered. However, this method is difficult to apply to codec quality enhancement because it does not specifically address the densely packed patches and large neighborhood variations in V-PCC image frames. Furthermore, this method requires a large amount of data to transmit, resulting in high bitrate overhead.

[0106] Existing Wiener filtering quality enhancement techniques in related technologies still face significant challenges, primarily high time complexity and irregularities in writing filtering information into the bitstream. First, searching for multiple filtering combinations is unnecessary. Experiments have shown that, in the vast majority of cases, only the first filtered combination achieves positive gain. Furthermore, removing these groups from the overall search and searching for the optimal combination again is complex and ineffective. Second, this technique searches for the optimal combination through a double loop: continuously adding new groups to the current combination, recalculating the optimal coefficients, filtering the pixel values, and comparing the PSNR improvement for the entire video frame after filtering. This operation has extremely high time complexity. Experiments have also revealed that the optimal combination can be determined with a regular pattern and is also related to the bitrate. Therefore, if the optimal combination can be directly determined, the entire operation can be completed by searching for pixels within the neighborhood difference range and performing coefficient calculation, filtering, and rate-distortion analysis in a single pass. Furthermore, this technique does not specify a detailed bitstream writing method. Because V-PCC has a strict syntax and bitstream structure, writing information such as coefficients into the correct location in the bitstream while ensuring that it does not affect the encoding and decoding of other modules is a crucial task. Related technologies write the geometry bitstream to a temporary file, add flags, group information, filter coefficients, and other information after the first few characters of the temporary file, and then read the temporary file from V-PCC to complete the bitstream writing. While this method can ensure that the written information is read, it is not rigorous and cannot guarantee that the coding information of the geometric video will not be affected.

[0107] In response to the above technical problems, the embodiment of the present application proposes an optimized V-PCC point cloud geometric frame Wiener filtering quality enhancement technology. At the encoding end, the preset neighborhood condition can be pre-set based on the bit rate of the first video image, eliminating the need for multiple coefficient calculations to determine the optimal preset neighborhood condition, thereby reducing time complexity. In addition, after the encoding end determines at least one pixel point that meets the preset neighborhood condition from the first video image, it can also determine a corresponding filter coefficient based on the at least one pixel point. Only when the first video image requires the first filtering process will the corresponding filter coefficient and category information indicating the preset neighborhood condition be transmitted to the decoding end. Accordingly, the decoding end can directly decode to obtain the filter coefficient and determine the corresponding preset neighborhood condition, and then use the filter coefficient to perform the first filtering process on the at least one pixel point that meets the preset neighborhood condition. This not only reduces time complexity, but also optimizes the reconstructed video image and improves the reconstructed geometric quality of the point cloud. At the same time, the present application also standardizes the design of the code stream writing / reading method to ensure the stability of the code stream operation and the rationality of the code stream structure. In this way, not only the quality of the reconstructed point cloud is enhanced, but also the quality of the video frame can be improved, further improving the compression efficiency and improving the codec performance.

[0108] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0109] In one embodiment of the present application, referring to FIG4 , a schematic flow chart of a decoding method provided by an embodiment of the present application is shown. As shown in FIG4 , the method may include:

[0110] S401: Decode a code stream and determine filter identification information of a first video image.

[0111] It should be noted that the decoding method in the embodiment of the present application specifically refers to a point cloud decoding method, which can be applied to the point cloud decoder in V-PCC (in the embodiment of the present application, it can be referred to as "decoder" for short). However, it should be noted that the decoding method described in the embodiment of the present application is not limited to the field of V-PCC, but can also be applied to other fields of quality enhancement of lossy point clouds / video images. V-PCC is taken as an example here because executing it on V-PCC can maximize its advantages, and it can process both geometric images and texture images. The overall quality of the point cloud is improved, with better results.

[0112] It should also be noted that in the embodiment of the present application, the decoding method can also be called a quality enhancement method. More specifically, it can be an optimized V-PCC point cloud geometric image Wiener filtering quality enhancement method, so as to achieve the purpose of geometric denoising and enhancing point cloud quality.

[0113] It should also be noted that, in the embodiment of the present application, the decoding end can determine the filter identification information of the first video image by decoding the bitstream, wherein the filter identification information can be used to indicate whether the first video image uses the first filter processing. Here, the decoding end can specifically determine the value of the filter identification information of the first video image by decoding the bitstream. In some embodiments, the method may further include:

[0114] If the value of the filter identification information is equal to the first value, determining that the filter identification information indicates that the first video image is processed using the first filter;

[0115] If the value of the filter identification information is equal to the second value, it is determined that the filter identification information indicates that the first video image does not use the first filtering process.

[0116] In the embodiment of the present application, the first value and the second value are different, and the first value and the second value can be in parameter form or in digital form. Specifically, the filter identification information here can be a parameter written in the profile or a flag value, which is not specifically limited here.

[0117] In an embodiment of the present application, the first value can be set to 1 and the second value can be set to 0; or, the first value can be set to 0 and the second value can be set to 1; or, the first value can also be set to true and the second value can also be set to false; or, the first value can be set to false and the second value can be set to true; but no specific limitation is made here.

[0118] For example, taking the filter identification information as a flag as an example, assuming that the first value is set to 1 and the second value is set to 0, if the decoding determines that the value of the filter identification information is 1, then it can be determined that the filter identification information indicates that the first video image uses the first filtering processing; conversely, if the decoding determines that the value of the filter identification information is 0, then it can be determined that the filter identification information indicates that the first video image does not use the first filtering processing.

[0119] It should also be noted that, in the embodiment of the present application, the first video image includes but is not limited to video frames, such as immersive video (Immersive Video), image video, etc. The video image here may have geometric information and texture information, and the geometric information refers to the position information of the pixels in the image, such as the three-dimensional coordinate information (x, y, z) of the point; the texture information may be the attribute information of the pixels in the image, such as the color information and reflectivity of the point. In addition, the first video image may also be a smaller video unit such as a slice or a slice segment in a video frame, or a larger video unit, such as multiple video frames. Here, the first video image may be a reconstructed video image, such as a lossy reconstructed geometric video image or a lossy reconstructed texture video image, and this is not specifically limited here.

[0120] S402: If the filter identification information indicates that the first video image is to be processed using the first filter, the code stream is decoded to determine the filter coefficients of the first video image.

[0121] It should be noted that in the embodiment of the present application, to improve coding efficiency, the filter coefficients for the first video image are written into the bitstream only when the first video image uses the first filtering process. In other words, after decoding the filter identification information of the first video image, the decoding end will only decode and determine the filter coefficients for the first video image if the filtering identification information indicates that the first video image uses the first filtering process.

[0122] It should also be noted that, in the embodiments of the present application, the first filtering process may include a Wiener filtering process. That is, the filter used for the first filtering process may be an adaptive filter, such as a neural network-based filter, a Wiener filter, etc., but this is not specifically limited to this. Taking the Wiener filter as an example, the filter coefficients described in the embodiments of the present application may be used for Wiener filtering, i.e., the filter coefficients are the coefficients of the Wiener filtering process.

[0123] Here, Wiener filtering is a quality enhancement algorithm, which is a linear optimal filtering based on the minimum mean square error criterion. Wiener filters are widely used in signal and image processing and other fields. Taking Wiener filtering for images as an example, usually for lossy or distorted images, Wiener filtering can obtain a restored image that is close to the original image in terms of mean square error, and it will also have a better subjective effect. For a K-order Wiener filter, the algorithm will obtain K coefficients based on the correspondence between the lossy image and the attribute value of each pixel in the original image, as the optimal coefficients of Wiener filtering. The coefficients can be used to perform Wiener filtering on the image to obtain a quality-enhanced image. Among them, the filter coefficients described in the embodiment of the present application specifically refer to the optimal coefficients of Wiener filtering, and K is an integer greater than 0.

[0124] S403: When at least one pixel point in the first video image meets a preset neighborhood condition, perform a first filtering process on the at least one pixel point according to a filtering coefficient to determine a filtered image corresponding to the first video image.

[0125] It should be noted that, in the embodiments of the present application, the preset neighborhood condition may refer to the optimal neighborhood difference combination (or simply referred to as "optimal combination" or "best combination") under the bit rate corresponding to the first video image. In other words, the preset neighborhood condition is associated with the bit rate of the first video image. For example, if the bit rate of the first video image is a first bit rate, the preset neighborhood condition at this time may be the optimal combination corresponding to the first bit rate; if the bit rate of the first video image is a second bit rate, the preset neighborhood condition at this time may be the optimal combination corresponding to the second bit rate.

[0126] In some embodiments, at least one pixel point in the first video image satisfies a preset neighborhood condition, which may include: a neighborhood difference value of a first pixel point in the first video image satisfies a preset neighborhood condition; wherein the first pixel point is any one of the at least one pixel point.

[0127] It should be noted that, in an embodiment of the present application, for the first video image, the neighborhood difference value of the first pixel point can be obtained by performing a neighborhood difference calculation based on the pixel point and each pixel point in the adjacent area corresponding to the pixel point. The adjacent area corresponding to the pixel point may include at least one neighboring pixel point. That is, in some embodiments, the method may further include: performing a neighborhood difference calculation based on the first pixel point in the first video image and at least one neighboring pixel point of the first pixel point to determine at least one absolute value of the difference; performing statistical processing based on the at least one absolute value of the difference to determine the neighborhood difference value of the first pixel point in the first video image.

[0128] It should also be noted that, in the embodiment of the present application, the statistical processing may include at least one of the following: averaging processing, summing processing, and maximum value processing.

[0129] It should also be noted that, in an embodiment of the present application, at least one neighborhood pixel point is a pixel point in the adjacent area corresponding to the first pixel point. Among them, the adjacent area corresponding to the first pixel point may refer to the eight neighborhoods, four neighborhoods, or diagonal neighborhoods of the pixel point. For example, Figure 5A shows the eight neighborhoods of pixel point A, i.e., the area with hatching, and the eight neighborhoods include 8 pixels in a circle around pixel point A. Figure 5B shows the four neighborhoods of pixel point A, i.e., the area with hatching, and the four neighborhoods include 4 pixels located at the top, bottom, left, and right around pixel A. Figure 5C shows the diagonal neighborhoods of pixel point A, i.e., the area with hatching, and the diagonal neighborhoods include pixels located at the four corners around pixel A. However, this is not specifically limited.

[0130] Further, in some embodiments, performing neighborhood difference calculation based on the first pixel point in the first video image and at least one neighboring pixel point of the first pixel point to determine at least one absolute value of the difference may include: performing absolute value calculation of the difference based on the pixel value of the first pixel point and the pixel value of at least one neighboring pixel point to determine at least one absolute value of the difference.

[0131] It should be noted that, in the embodiment of the present application, the neighborhood difference value of the first pixel point in the first video image can be obtained by calculating the absolute value of the difference between the pixel value of the first pixel point and the pixel value of at least one neighboring pixel point. For example, taking the sum as an example, assuming that a represents the pixel value of the first pixel point, a i Represents the i-th neighboring pixel value of at least one neighboring pixel point, then the neighborhood difference value of the first pixel point is represented by diff, as follows:

[0132] Wherein, m represents the number of at least one neighboring pixel point, and m is an integer greater than 0.

[0133] In one possible implementation, taking the eight-neighborhood region of pixel A as an example, the neighborhood difference value of pixel A in the first video image can be obtained by summing the absolute values ​​of the differences between the pixel value of pixel A and the pixel value of each pixel in its eight-neighborhood region. As shown in Figure 6A , the pixel value of pixel A is 2, and the pixel values ​​of each pixel in its eight-neighborhood region are 3, 1, 3, 2, 4, 2, 2, and 4, respectively. The resulting neighborhood difference value, represented by diff, is as follows: diff = |2-3| + |2-1| + |2-3| + |2-2| + |2-4| + |2-2| + |2-2| + |2-4| = 7.

[0134] As shown in the example of Figure 6B, the pixel value of pixel A is 1, and the pixel values ​​of each pixel in the eight neighborhoods are 1, 1, 1, 2, 1, 2, 2, 1 respectively. The obtained neighborhood difference value is represented by diff, as follows: diff = |1-1|+|1-1|+|1-1|+|1-2|+|1-1|+|1-2|+|1-2|+|1-1|=3.

[0135] As shown in the example of Figure 6C, the pixel value of pixel A is 5, and the pixel values ​​of each pixel in the eight neighborhoods are 4, 7, 8, 8, 6, 5, 2, and 3 respectively. The obtained neighborhood difference value is represented by diff, as follows: diff = |5-4|+|5-7|+|5-8|+|5-8|+|5-6|+|5-5|+|5-2|+|5-3|=15.

[0136] It should also be noted that in the embodiment of the present application, for the calculation of neighborhood difference values, in addition to summing, the statistical processing here can also be to find the average, that is, to divide the sum by the number of pixels in the neighborhood; or the statistical processing here can also be to find the maximum value, that is, to select the maximum value from at least one absolute difference value obtained. In addition, after obtaining the absolute difference value diff, diff can be used to directly represent the neighborhood difference value of pixel point A. However, the neighborhood of pixel point A in the first video image can also be normalized, such as mapping diff to a specific interval, and using the normalized value to represent the neighborhood difference value of pixel point A. There is no specific limitation on this.

[0137] In another possible implementation, the neighborhood difference value of pixel A in the first video image can also be determined based on the difference between pixel A and the pixels in its neighborhood. Taking the data in Figure 3A as an example, the neighborhood difference value of pixel A is obtained by the difference in pixel values ​​across the entire 3×3 region. For example, considering the differences of 2, 3, 1, 3, 2, 4, 2, 2, 4, this difference can be expressed using, for example, range, mean square error, or the like.

[0138] In this way, since the neighborhood difference value of the former implementation method is obtained based on the absolute value of the difference between the pixel value of pixel point A and the pixel value of each pixel in the neighborhood of the pixel point, it is more sensitive to pixel-by-pixel changes, can capture the drastic changes in grayscale values ​​at the edge of the object, and more accurately classify the pixels at the edge into the same category, generate appropriate filtering coefficients, and thus improve the filtering effect.

[0139] It is understood that in the embodiments of the present application, the decoding end can determine the preset neighborhood condition by decoding the corresponding category information. Therefore, in some embodiments, the method may further include: decoding the bitstream to determine the category information of the first video image; wherein the category information is used to indicate the preset neighborhood condition.

[0140] In a specific embodiment, when the preset neighborhood condition represents a preset range, decoding the bitstream and determining the category information of the first video image may include: decoding the bitstream and determining start information and end information corresponding to the preset range.

[0141] It should be noted that in this embodiment of the present application, the encoder may write category information indicating a preset neighborhood condition into the bitstream, so that the decoder can determine the preset neighborhood condition by decoding the bitstream. Here, the preset neighborhood condition can be a preset range, or also called a preset interval, such as [4,10], [4,8], etc.

[0142] It should also be noted that in this embodiment of the present application, the category information is used to record the start information (i.e., the starting group number) and end information (i.e., the ending group number) of a preset range. The start information and end information each occupy 4 bits; therefore, the category information occupies 8 bits in the code stream.

[0143] For example, for the preset neighborhood condition, if the preset range is [4, 10], the category information written in the code stream is 4 and 10, 4 represents the start information, and 10 represents the end information; if the preset range is [4, 8], the category information written in the code stream is 4 and 8, 4 represents the start information, and 8 represents the end information.

[0144] It can also be understood that in an embodiment of the present application, the method may further include: if the pixel value of the first pixel point in the first video image is equal to 0, determining that the first pixel point in the first video image does not meet the preset neighborhood condition.

[0145] It should be noted that in the embodiment of the present application, if the pixel value of the first pixel is 0, it can be said that the pixel is most likely a background pixel; since filtering the background pixels is completely meaningless, it is possible to choose not to calculate the neighborhood difference value of the pixel, that is, it is possible to directly determine that the pixel does not meet the preset neighborhood conditions, thereby reducing time complexity.

[0146] Furthermore, with respect to the filter identification information of the first video image, in some embodiments, the method may further include: if the filter identification information indicates that the first video image does not use the first filter processing, then the decoding code stream is not continued, and the step of determining the filter coefficient of the first video image is performed.

[0147] That is, if the value of the decoded filter identification information is a first value, the filter identification information indicates that the first video image uses the first filter processing, then the code stream can be decoded to determine the filter coefficients of the first video image; then, the first filter processing is performed on at least one pixel in the first video image based on the filter coefficients to determine the filtered image corresponding to the first video image. Conversely, if the value of the decoded filter identification information is a second value, the filter identification information indicates that the first video image does not use the first filter processing, then the steps of decoding the code stream and determining the filter coefficients of the first video image are not performed, and the first video image can be directly determined as the filtered image.

[0148] It can also be understood that in the embodiment of the present application, the first video image is a video image in a video sequence. In some embodiments, the method may further include: decoding the bit stream and determining filtering parameters of the video sequence.

[0149] Among them, the filtering parameters may include at least one of the following: filtering activation identification information, total number of frames, category information, filtering identification information of at least one video image in the video sequence and corresponding filtering coefficients, and the first video image is any one of the at least one video image.

[0150] It should be noted that in the embodiment of the present application, the filter activation identification information (i.e., the filter activation flag) can be used to indicate whether the video sequence uses the first filtering process, represented here by Wiener_filtering_flag, which occupies 1 bit in the bitstream; the total number of frames can be used to indicate the total number of frames in the video sequence, represented here by frame_num, which occupies 32 bits in the bitstream; the category information can be used to indicate the preset neighborhood conditions of each video image in the video sequence, which can also be called combination information, so it is represented here by groups, which occupies 8 bits in the bitstream; the filter identification information (i.e., the filter flag) of each video image can be used to indicate whether the video image uses the first filtering process, represented here by filtering_flag_of_each_frame, which occupies 1 bit in the bitstream; the filter coefficient of each video image can be represented by WFcoef, which occupies a maximum of 32×k×frame_num bits. Here, only when a video image uses the first filtering process, the corresponding filter coefficient will be written here; otherwise, if a video image does not use the first filtering process, the filter coefficient will not be written.

[0151] In addition, it should be noted that in the embodiment of the present application, each frame of video image can generate two geometric video images, specifically a far frame video image and a near frame video image; then, for each of these two frames of video image, whether to use the first filtering process and the corresponding filter coefficients are determined. Therefore, if the video sequence includes n frames of video image, the total number of frames here is 2n, where n is an integer greater than 0.

[0152] Furthermore, in some embodiments, decoding the bitstream and determining the filtering parameters of the video sequence may include: decoding the bitstream and determining the syntax object information in the first bitstream unit; reading the filtering enablement identification information according to the syntax object information, and if the filtering enablement identification information indicates that the video sequence uses the first filtering processing, then continuing to read at least one of the total number of frames, category information, the filtering identification information of at least one video image, and the corresponding filtering coefficient according to the syntax object information.

[0153] It should also be noted that in this embodiment of the present application, the first code stream unit can be represented by V3C_WF. The syntax object information here may include a first syntax object, a second syntax object, a third syntax object, a fourth syntax object, and a fifth syntax object. Among them, the first syntax object is represented by syntax.Wiener_filtering_flag, the second syntax object is represented by syntax.frame_num, the third syntax object is represented by syntax.groups, the fourth syntax object is represented by syntax.filtering_flag_of_each_frame, and the fifth syntax object is represented by syntax.WFcoef.

[0154] It should also be noted that, in the embodiment of the present application, the syntax object information in the first code stream unit may be decoded using an entropy decoding method or a binarization method, etc., and then decoded to determine the value corresponding to at least one syntax object.

[0155] In addition, in the embodiments of the present application, for these grammatical objects, the value range of each field has certain restrictions (i.e., conformance). In one possible implementation, the value range can be as follows:

[0156] Filtering enable flag information Wiener_filtering_flag: true or false;

[0157] Total frame number frame_num: an integer greater than 1.

[0158] Category information groups: The first four digits represent the starting group number (4), and the last four digits represent the ending group number (8 or 10). In the experiment, the value is 72 or 74 (if other combinations are selected, the value will be different, ranging from 0 to 255).

[0159] Each frame filtering flag information filtering_flag_of_each_frame: true or false;

[0160] The filter coefficient of each frame Wfcoef is from -231 to 231-1, i.e., from -2147483648 to 2147483647.

[0161] In this way, during the decoding process, the filtering enable identification information can first be read from syntax.Wiener_filtering_flag; if the filtering enable identification information indicates that the video sequence uses the first filtering processing, then the total number of frames can be read from syntax.frame_num, the category information can be read from syntax.groups, the filtering identification information of at least one video image can be read from syntax.filtering_flag_of_each_frame, and the corresponding filtering coefficient can be read from syntax.WFcoef; among them, if the filtering identification information of a video image indicates that the video image does not use the first filtering processing, then the filtering coefficient of the video image will not be read.

[0162] Furthermore, in some embodiments, the method may also include: if the filter enable identification information indicates that the video sequence does not use the first filtering process, then the step of reading at least one of the total number of frames, category information, filter identification information of at least one video image, and corresponding filter coefficients according to the grammatical object information is not continued.

[0163] In some embodiments, for filtering activation identification information, the method may further include:

[0164] If the value of the filter activation identification information is equal to the first value, determining that the filter activation identification information indicates that the video sequence uses the first filtering process;

[0165] If the value of the filtering enabling flag information is equal to the second value, it is determined that the filtering enabling flag information indicates that the video sequence does not use the first filtering process.

[0166] In the embodiment of the present application, the first value and the second value are different, and the first value and the second value can be in parameter form or in digital form. Specifically, the filtering activation identification information here can be a parameter written in the profile or a flag value, which is not specifically limited here.

[0167] In an embodiment of the present application, the first value can be set to 1 and the second value can be set to 0; or, the first value can be set to 0 and the second value can be set to 1; or, the first value can also be set to true and the second value can also be set to false; or, the first value can be set to false and the second value can be set to true; but no specific limitation is made here.

[0168] For example, assuming that the first value is set to 1 and the second value is set to 0, if the value of Wiener_filtering_flag is equal to 1, then it can be determined that the filtering enable identification information indicates that the video sequence uses the first filtering processing, and it is necessary to continue reading other syntax objects; conversely, if the value of Wiener_filtering_flag is equal to 0, then it can be determined that the filtering enable identification information indicates that the video sequence does not use the first filtering processing, and there is no need to read other syntax objects.

[0169] That is to say, in an embodiment of the present application, the first code stream unit can be represented by V3C_WF. During the decoding process, it is first necessary to read the valid information in V3C_WF into the variables of the syntax object. Subsequently, the variables of the corresponding syntax object can be read in the first filtering link to obtain the category information, the filter flag of each video image, and the corresponding filter coefficients. It should be noted that if the first filtering process is not turned on in the video sequence, that is, Wiener_filtering_flag == 0, then other variables will no longer be read. Similarly, if the filter flag of a video image is 0, the filter coefficient of the video image will not be read.

[0170] Furthermore, in an embodiment of the present application, the first filtering process may be a Wiener filtering process. After the filtering coefficients are obtained through decoding, they may be used to determine a filtered image corresponding to the first video image. Specifically, in some embodiments, performing the first filtering process on at least one pixel point according to the filtering coefficients to determine the filtered image corresponding to the first video image may include: performing a Wiener filtering process on the pixel value of at least one pixel point according to the filtering coefficients to determine the filtered pixel value of the at least one pixel point; and determining the filtered image based on the filtered pixel value of the at least one pixel point.

[0171] It should be noted that, in the embodiment of the present application, the first video image may be a reconstructed video image. If the filter identification information indicates that the reconstructed video image uses the first filtering process, and at least one pixel point that satisfies the preset neighborhood condition is found in the reconstructed video image, then Wiener filtering can be performed on the at least one pixel point in the reconstructed video image according to the filter coefficient to determine the filtered pixel value of the at least one pixel point, while other pixels in the reconstructed video image are not processed (retaining the reconstructed pixel value), thereby determining a filtered image corresponding to the reconstructed video image.

[0172] It should also be noted that, in an embodiment of the present application, Wiener filtering is performed on the pixel value of at least one pixel point according to the filter coefficient to determine the filtered pixel value of the at least one pixel point. Taking the first pixel point of the at least one pixel point as an example, in some embodiments, determining the filtered pixel value of the first pixel point may include: determining K reference pixel points corresponding to the first pixel point in the reconstructed video image; performing Wiener filtering on the K reference pixel points corresponding to the first pixel point according to the filter coefficient to determine the filtered pixel value of the first pixel point; wherein K is an integer greater than 0, and the K reference pixel points include the first pixel point and (K-1) neighboring pixel points adjacent to the first pixel point.

[0173] In a specific embodiment, taking the first pixel point as an example, the method may further include: based on the first pixel point in the reconstructed video image, searching for a preset number of candidate pixel points in the reconstructed video image using a K-nearest neighbor search method; respectively calculating the distance values ​​between the first pixel point and the preset number of candidate points, and selecting (K-1) distance values ​​from the obtained preset number of distance values, and the (K-1) distance values ​​are all less than or equal to the remaining distance values ​​in the preset number of distance values; determining (K-1) neighbor pixel points based on the candidate pixel points corresponding to the (K-1) distance values, and determining the first pixel point and the (K-1) neighbor pixel points as the K reference pixel points corresponding to the first pixel point.

[0174] It should be noted that in the embodiment of the present application, the first pixel point can be any one of the at least one pixel points in the reconstructed video image. Taking the first pixel point as an example, a K-nearest neighbor search method can be used to search for a preset number of candidate points in the reconstructed video image, calculate the distance values ​​between the first pixel point and these candidate points, and then select the (K-1) neighboring pixel points closest to the first pixel point from these candidate points; the first pixel point itself and the (K-1) neighboring pixel points are determined as the final K reference pixel points.

[0175] It should also be noted that, in an embodiment of the present application, when performing Wiener filtering on at least one pixel point using a filter coefficient, the filter correlation matrix (expressed as P(n,k)) can be determined first based on the order of the Wiener filter and the reconstructed pixel values ​​of the K reference pixels corresponding to the at least one pixel point; then the filtered pixel value of the at least one pixel point can be determined based on the product of the filter correlation matrix and the filter coefficient, thereby determining the filtered image.

[0176] It should also be noted that, in an embodiment of the present application, a preset window can also be used to scan each pixel of the at least one pixel point, and the pixel values ​​of all the pixel points in the preset window are weighted averaged, and the pixel value of the pixel point located at the center of the preset window is updated to the result of the weighted average to obtain the filtered pixel value of each pixel point; wherein, the set of weighting coefficients used for the weighted average is the filtering coefficient obtained by decoding, and the shape of the preset window can be a rectangle or a diamond.

[0177] In the embodiments of the present application, the "preset window" here refers to the window used for neighborhood filtering scanning, and may also be referred to as a template, template window, convolution kernel, box, etc. The scanning operation may also be referred to as convolution. However, an example of an embodiment of the present application uses a diamond-shaped filter window, as shown in FIG7 . Compared to the original rectangular filter, this example changes the shape of the window, which can better adapt to boundary changes and more effectively extract neighborhood information. The window in FIG7 includes 25 pixels, corresponding to a filter of order k=25. The embodiments of the present application may also use filters of other orders such as 9th order, 16th order, 36th order, etc. for neighborhood filtering.

[0178] Furthermore, in the embodiments of the present application, for a filter, the filter type can be used to indicate the filter order, and / or the filter shape, and / or the filter dimension. The filter shape includes diamond, rectangle, etc., and the filter dimension includes one-dimensional, two-dimensional, or even more dimensions.

[0179] Furthermore, in an embodiment of the present application, the filter may be a Wiener filter, an adaptive filter, or the like. In addition, the filter type may correspond to filters of different orders, for example, filters of orders 12, 16, 20, 32, or 128; different types may also correspond to filters of different dimensions, for example, one-dimensional filters, two-dimensional filters, etc., which are not specifically limited here. In other words, if a 16-order filter needs to be determined, 16 points may be used to determine a 16-order asymmetric filter, or an 8-order one-dimensional symmetric filter, or other numbers of filters (for example, more special two-dimensional or three-dimensional filters, etc.), etc., and the filter is not specifically limited here.

[0180] Furthermore, in the embodiment of the present application, for the order of the filter, the value of the order can be 16, 20, 32, 64, etc. Exemplarily, the value of the order is 20, but this is not specifically limited here.

[0181] Furthermore, in an embodiment of the present application, the original video image can be directly obtained through the point cloud reading function of the codec program, and the reconstructed video image is obtained after all encoding operations are completed. In addition, the reconstructed video image in the embodiment of the present application can be a reconstructed image output after decoding, or it can be used as a reference for decoding subsequent image frames; in addition, the decoding method here can be applied not only within the prediction loop, that is, used as an in-loop filter, and can be used as a reference for decoding subsequent image frames; it can also be applied outside the prediction loop, that is, used as a post filter, and not used as a reference for decoding subsequent image frames; the embodiment of the present application does not specifically limit this.

[0182] In this way, taking the Wiener filter as an example, when using the Wiener filter to perform the first filtering processing on the reconstructed video image, the reconstructed video image and the filter coefficient can be input into the Wiener filter at the same time, that is, the input of the Wiener filter is the filter coefficient and the reconstructed video image, and finally the filtering processing of the reconstructed video image can be completed based on the filter coefficient to obtain the corresponding filtered image.

[0183] That is, in the embodiment of the present application, the filter coefficients are obtained based on the original video image and the reconstructed video image. Therefore, applying the filter coefficients to the reconstructed video image can restore the original video image to the greatest extent possible. For example, in the embodiment of the present application, taking a lossy reconstructed video image in a point cloud sequence as an example, assuming that the order of the filter is k and the total number of pixels in the reconstructed video image is n, the filter correlation matrix P(n,k) is used to represent the reconstructed pixel values ​​of k reference pixels corresponding to all pixels in the reconstructed video image under a certain color component (such as Y component / U component / V component).

[0184] In this way, based on the formula R(n)=P(n,k)×H(k), the filter coefficient corresponding to a certain color component (represented by H(k)) is applied to the reconstructed video image, that is, the filter correlation matrix P(n,k), so that the filtered pixel value R(n) under the color component can be obtained, and then the filtered image can be determined.

[0185] Furthermore, in the embodiments of the present application, during the filtering process, some peripheral features of the image, such as image boundaries and patch boundaries, are considered. For image boundaries, in order to apply the Wiener filter to the outermost pixels of the image, the image exterior can be filled (for example, with a fill value of 0), or these outer pixels can be left unprocessed. For patch boundaries, no special processing is required and calculation can be performed according to the methods of the embodiments of the present application.

[0186] It should also be noted that, in the embodiment of the present application, the pixel value of a pixel point can be the value of any component of the three color components (also referred to as three color channels). For example, in an image in YUV format, "Y" represents the brightness component, and the value of the brightness component is usually called the grayscale value; "U" and "V" represent the chrominance component, and the chrominance component can be stored as Cb and Cr, where Cb is the blue chrominance component and Cr is the red chrominance component. The pixel value of a pixel point can be the value of the brightness component, the blue chrominance component, or the red chrominance component; in an image in RGB format, the pixel value of a pixel point can be the value of red, green, or blue, and so on. In some cases, when the first video image has only some components, for example, in V-PCC, when filtering the reconstructed geometric video image, the pixel value of the pixel point refers to the grayscale value. When filtering the reconstructed texture video image, the two chrominance components can be filtered separately. When filtering the blue chrominance component, the pixel value of the above-mentioned pixel point refers to the value of the blue chrominance component; when filtering the red chrominance component, the pixel value of the above-mentioned pixel point refers to the value of the red chrominance component; this is not specifically limited here.

[0187] It should also be noted that, in the embodiment of the present application, the first video image may include a reconstructed video image, wherein the reconstructed video image may include a lossy reconstructed geometric video image and / or a lossy reconstructed texture video image.

[0188] In some embodiments, when the first video image includes a lossy reconstructed geometric video image, the method may further include: after performing a first filtering process on the reconstructed geometric video image, performing quality enhancement processing on the corresponding texture video image based on the filtered reconstructed geometric video image.

[0189] That is to say, in the embodiment of the present application, the method can not only be applied to lossy geometrically reconstructed video images to implement the neighborhood difference-based Wiener filtering post-processing technology to achieve the purpose of geometric denoising and enhancing point cloud quality; it can also be applied to filtering lossy texture-reconstructed video images to achieve the purpose of enhancing attribute quality.

[0190] It should also be noted that, in an embodiment of the present application, a new first code stream unit: V3C_WF can be created in the V-PCC code stream to write the filtering parameters of the video sequence; alternatively, these filtering parameters can also be written into the existing code stream unit: V3C_GVD, which is regarded as part of the geometric codec code stream.

[0191] It should also be noted that in the embodiment of the present application, the category information (i.e., the optimal combination) is associated with the bit rate of the first video image. Here, the same optimal combination can be set for the same video sequence, or different video sequences can be set with different optimal combinations at different bit rates. Alternatively, two optimal combinations can be specified for each bit rate, and different optimal combinations can be used for filtering and comparing performance, and then the combination with better performance can be selected as the final optimal combination; this is not specifically limited here.

[0192] This embodiment provides a decoding method, which decodes a bitstream and determines filter identification information of a first video image; if the filter identification information indicates that the first video image uses a first filter process, the bitstream is decoded to determine the filter coefficient of the first video image; when at least one pixel in the first video image meets a preset neighborhood condition, the first filter process is performed on the at least one pixel according to the filter coefficient to determine a filtered image corresponding to the first video image. In this way, the decoding end can directly decode to obtain the filter coefficient and determine the corresponding preset neighborhood condition, and then use the filter coefficient to perform the first filter process on the at least one pixel that meets the preset neighborhood condition; thereby not only reducing the time complexity, but also optimizing the reconstructed video image and improving the reconstructed geometric quality; at the same time, the present application also standardizes the design of the bitstream writing / reading method to ensure the stability of the codec system operation and the rationality of the bitstream structure; thus, for the geometric lossy and attribute lossy methods in V-PCC, not only the quality of the reconstructed point cloud is enhanced, but also the quality of the video frame can be improved, thereby improving the compression efficiency and improving the codec performance.

[0193] In another embodiment of the present application, referring to FIG8 , a schematic diagram of an encoding method provided by an embodiment of the present application is shown. As shown in FIG8 , the method includes:

[0194] S801: Determine at least one pixel point in a first video image that meets a preset neighborhood condition.

[0195] It should be noted that the encoding method in the embodiment of the present application specifically refers to a point cloud encoding method, which can be applied to the point cloud encoder in V-PCC (in the embodiment of the present application, it can be simply referred to as "encoder"). However, it should be noted that the encoding method described in the embodiment of the present application is not limited to the V-PCC field, and can also be applied to other lossy point cloud / video image quality enhancement fields. V-PCC is taken as an example here because executing it on V-PCC can maximize its advantages, and can process both geometric images and texture images. The overall quality of the point cloud is improved, with better results.

[0196] It should also be noted that in the embodiments of the present application, the encoding method can also be called a quality enhancement method. More specifically, it can be an optimized V-PCC point cloud geometric image Wiener filtering quality enhancement method, thereby achieving the purpose of geometric denoising and enhancing point cloud quality.

[0197] It should also be noted that, in the embodiment of the present application, the first video image includes but is not limited to video frames, such as immersive video (Immersive Video), image video, etc. The video image here may have geometric information and texture information, and the geometric information refers to the position information of the pixels in the image, such as the three-dimensional coordinate information (x, y, z) of the point; the texture information may be the attribute information of the pixels in the image, such as the color information and reflectivity of the point. In addition, the first video image may also be a smaller video unit such as a slice or a slice segment in a video frame, or a larger video unit, such as multiple video frames. Here, the first video image may be a reconstructed video image, such as a lossy reconstructed geometric video image or a lossy reconstructed texture video image, and this is not specifically limited here.

[0198] It is understandable that in the embodiments of the present application, the preset neighborhood condition may refer to the optimal neighborhood difference combination (or simply referred to as "optimal combination" or "best combination") under the bit rate corresponding to the first video image. In other words, the preset neighborhood condition is associated with the bit rate of the first video image. For example, if the bit rate of the first video image is a first bit rate, the preset neighborhood condition at this time may be the optimal combination corresponding to the first bit rate; if the bit rate of the first video image is a second bit rate, the preset neighborhood condition at this time may be the optimal combination corresponding to the second bit rate.

[0199] For example, all point clouds in the V-PCC standard test sequence were compressed into 32 frames under intra-frame coding conditions, and the best combinations at various bit rates were calculated. Here are the top two most selected best combinations and their occurrence times at each bit rate:

[0200] R01: 4–10 were selected 82 times; 1–11 were selected 58 times.

[0201] R02: 1–10 was selected 64 times; 4–10 was selected 53 times.

[0202] R03: 4–8 was selected 82 times; 1–8 was selected 69 times.

[0203] R04: 4–8 was selected 124 times; 1–8 was selected 36 times.

[0204] R05: 4–8 was selected 153 times; 1–8 was selected 59 times.

[0205] As can be seen, at low bitrates (R01 and R02), the 4–10 combination (i.e., combinations of pixels with neighborhood differences falling within the range {4, 5, 6, 7, 8, 9, 10}) appears most frequently, while at high bitrates (R03 and above), the 4–8 combination is most frequently selected. Based on these experimental results, the optimal combination can be pre-set: at R01 and R02 bitrates, the optimal combination can be [4, 10]; at R03, R04, and R05 bitrates, the optimal combination can be [4, 8]. In other words, at R01 and R02 bitrates, the preset neighborhood condition can be [4, 10]; at R03, R04, and R05 bitrates, the preset neighborhood condition can be [4, 8]. This significantly reduces time loss while barely compromising filtering performance.

[0206] Further, in some embodiments, at least one pixel point in the first video image satisfies a preset neighborhood condition, which may include: a neighborhood difference value of a first pixel point in the first video image satisfies a preset neighborhood condition; wherein the first pixel point is any one of the at least one pixel point.

[0207] In an embodiment of the present application, for the first video image, the neighborhood difference value of the first pixel point can be obtained by performing a neighborhood difference calculation based on the pixel point and each pixel point in the adjacent area corresponding to the pixel point. The adjacent area corresponding to the pixel point may include at least one neighboring pixel point. That is, in some embodiments, the method may further include: performing a neighborhood difference calculation based on the first pixel point in the first video image and at least one neighboring pixel point of the first pixel point to determine at least one absolute difference value; performing statistical processing based on the at least one absolute difference value to determine the neighborhood difference value of the first pixel point in the first video image.

[0208] In an embodiment of the present application, at least one neighborhood pixel point is a pixel point in the adjacent area corresponding to the first pixel point. The adjacent area corresponding to the first pixel point may refer to the eight neighborhoods, four neighborhoods, or diagonal neighborhoods of the pixel point. For example, FIG5A shows the eight neighborhoods of pixel point A, i.e., the area with hatching, and the eight neighborhoods include 8 pixels in a circle around pixel point A. FIG5B shows the four neighborhoods of pixel point A, i.e., the area with hatching, and the four neighborhoods include 4 pixels located at the top, bottom, left, and right around pixel A. FIG5C shows the diagonal neighborhoods of pixel point A, i.e., the area with hatching, and the diagonal neighborhoods include the pixels located at the four corners around pixel A. However, this is not specifically limited.

[0209] In a specific embodiment, taking the first pixel in the first video image as an example, for calculating the neighborhood difference of the first pixel, referring to FIG9 , the method may include:

[0210] S901: Calculate the absolute value of the difference between a pixel value of a first pixel and a pixel value of at least one neighboring pixel to determine at least one absolute value of the difference.

[0211] S902: Perform statistical processing on at least one absolute difference value to determine a neighborhood difference value of the first pixel.

[0212] It should be noted that, in the embodiment of the present application, the statistical processing may include at least one of the following: averaging processing, summing processing and maximum value processing.

[0213] It should also be noted that, in the embodiment of the present application, the neighborhood difference value of the first pixel point in the first video image can be obtained by statistically calculating the absolute value of the difference between the pixel value of the first pixel point and the pixel value of at least one neighboring pixel point. For example, taking the sum as an example, assuming that a represents the pixel value of the first pixel point, a i Represents the i-th neighboring pixel value of at least one neighboring pixel point, then the neighborhood difference value of the first pixel point is represented by diff, as follows:

[0214] Wherein, m represents the number of at least one neighboring pixel point, and m is an integer greater than 0.

[0215] In one possible implementation, taking the eight-neighborhood region of pixel A as an example, the neighborhood difference value of pixel A in the first video image can be obtained by summing the absolute values ​​of the differences between the pixel value of pixel A and the pixel value of each pixel in its eight-neighborhood region. As shown in Figure 6A , the pixel value of pixel A is 2, and the pixel values ​​of each pixel in its eight-neighborhood region are 3, 1, 3, 2, 4, 2, 2, and 4, respectively. The resulting neighborhood difference value, represented by diff, is as follows: diff = |2-3| + |2-1| + |2-3| + |2-2| + |2-4| + |2-2| + |2-2| + |2-4| = 7.

[0216] As shown in the example of Figure 6B, the pixel value of pixel A is 1, and the pixel values ​​of each pixel in the eight neighborhoods are 1, 1, 1, 2, 1, 2, 2, 1 respectively. The obtained neighborhood difference value is represented by diff, as follows: diff = |1-1|+|1-1|+|1-1|+|1-2|+|1-1|+|1-2|+|1-2|+|1-1|=3.

[0217] As shown in the example of Figure 6C, the pixel value of pixel A is 5, and the pixel values ​​of each pixel in the eight neighborhoods are 4, 7, 8, 8, 6, 5, 2, and 3 respectively. The obtained neighborhood difference value is represented by diff, as follows: diff = |5-4|+|5-7|+|5-8|+|5-8|+|5-6|+|5-5|+|5-2|+|5-3|=15.

[0218] It should also be noted that in the embodiment of the present application, for the calculation of neighborhood difference values, in addition to summing, the statistical processing here can also be to find the average, that is, to divide the sum by the number of pixels in the neighborhood; or the statistical processing here can also be to find the maximum value, that is, to select the maximum value from at least one absolute difference value obtained. In addition, after obtaining the absolute difference value diff, diff can be used to directly represent the neighborhood difference value of pixel point A. However, the neighborhood of pixel point A in the first video image can also be normalized, such as mapping diff to a specific interval, and using the normalized value to represent the neighborhood difference value of pixel point A. There is no specific limitation on this.

[0219] In another possible implementation, the neighborhood difference value of pixel A in the first video image can also be determined based on the difference between pixel A and the pixels in its neighborhood. Taking the data in Figure 3A as an example, the neighborhood difference value of pixel A is obtained by the difference in pixel values ​​across the entire 3×3 region. For example, considering the differences of 2, 3, 1, 3, 2, 4, 2, 2, 4, this difference can be expressed using, for example, range, mean square error, or the like.

[0220] In this way, since the neighborhood difference value of the former implementation method is obtained based on the absolute value of the difference between the pixel value of pixel point A and the pixel value of each pixel in the neighborhood of the pixel point, it is more sensitive to pixel-by-pixel changes, can capture the drastic changes in grayscale values ​​at the edge of the object, and more accurately classify the pixels at the edge into the same category, generate appropriate filtering coefficients, and thus improve the filtering effect.

[0221] It is also understood that in embodiments of the present application, to reduce time complexity, the encoder may also determine corresponding category information for a preset neighborhood condition and write it into the bitstream, so that the decoder can quickly determine the preset neighborhood condition through decoding. In some embodiments, the method may further include: determining category information for the first video image based on the preset neighborhood condition; encoding the category information for the first video image, and writing the resulting coded bits into the bitstream.

[0222] In a specific embodiment, when the preset neighborhood condition represents a preset range, determining the category information of the first video image according to the preset neighborhood condition may include: determining start information and end information corresponding to the preset range according to the preset neighborhood condition.

[0223] It should be noted that in this embodiment of the present application, the encoder may write category information indicating a preset neighborhood condition into the bitstream, so that the decoder can determine the preset neighborhood condition by decoding the bitstream. Here, the preset neighborhood condition can be a preset range, or also called a preset interval, i.e., the aforementioned optimal neighborhood difference combination, or "best combination"), such as [4,10], [4,8], and so on.

[0224] It should also be noted that in this embodiment of the present application, the category information is used to record the start information (i.e., the starting group number) and end information (i.e., the ending group number) of a preset range. The start information and end information each occupy 4 bits; therefore, the category information occupies 8 bits in the code stream.

[0225] For example, for the preset neighborhood condition, if the preset range is [4, 10], the category information written in the code stream is 4 and 10, 4 represents the start information, and 10 represents the end information; if the preset range is [4, 8], the category information written in the code stream is 4 and 8, 4 represents the start information, and 8 represents the end information.

[0226] It can also be understood that in an embodiment of the present application, the method may further include: if the pixel value of the first pixel point in the first video image is equal to 0, determining that the first pixel point in the first video image does not meet the preset neighborhood condition.

[0227] It should be noted that in the embodiment of the present application, if the pixel value of the first pixel is 0, it can be said that the pixel is most likely a background pixel; since filtering the background pixels is completely meaningless, it is possible to choose not to calculate the neighborhood difference value of the pixel, that is, it is possible to directly determine that the pixel does not meet the preset neighborhood conditions, thereby reducing time complexity.

[0228] S802: Determine a filter coefficient of a first video image according to at least one pixel point.

[0229] It should be noted that in the embodiments of the present application, the filter coefficients are related to the first video image and the corresponding original video image. In order to find the correspondence between each pixel in the first video image and the pixel in the original video image in the geometric lossy and attribute lossy encoding method, a KNN search method is used to traverse each pixel in the first video image. The search range is the entire video image. The purpose is to find the nearest neighbors of each pixel and use them as the K reference pixels corresponding to the pixel for calculating the filter coefficients.

[0230] In some embodiments, determining the filter coefficient of the first video image based on at least one pixel point may include: determining the filter coefficient based on K reference pixels corresponding to the pixel point in the original video image and the pixel point in the first video image.

[0231] In an embodiment of the present application, the first video image is a lossy reconstructed video image. Before the input filter determines the filter coefficient, it is first necessary to determine the K reference pixels corresponding to the pixel in the reconstructed video image. Here, taking the first pixel as an example, the K reference pixels corresponding to the first pixel can include the first pixel and (K-1) neighboring pixels adjacent to the first pixel, where K is an integer greater than 0.

[0232] In a specific embodiment, determining K reference pixels corresponding to a first pixel in the reconstructed video image may include:

[0233] Based on the first pixel point in the reconstructed video image, searching for a preset number of candidate pixel points in the reconstructed video image using a K-nearest neighbor search method;

[0234] Calculate the distance values ​​between the first pixel and a preset number of candidate points respectively, and select (K-1) distance values ​​from the obtained preset number of distance values, and the (K-1) distance values ​​are all less than or equal to the remaining distance values ​​in the preset number of distance values;

[0235] Determine (K-1) neighboring pixels according to the candidate pixels corresponding to the (K-1) distance values, and determine the first pixel and the (K-1) neighboring pixels as K reference pixels corresponding to the first pixel.

[0236] It should be noted that in the embodiment of the present application, the first pixel is any one of at least one pixel in the reconstructed video image. Here, taking the first pixel as an example, a K-nearest neighbor search method can be used to search for a preset number of candidate pixels in the reconstructed video image, calculate the distance between the first pixel and these candidate pixels, and then select the (K-1) neighboring pixels closest to the first pixel from these candidate pixels; the first pixel itself and the (K-1) neighboring pixels are determined as the final K reference pixels.

[0237] It should also be noted that in the embodiment of the present application, the filter can be an adaptive filter, for example, a filter based on a neural network, a Wiener filter, etc., which is not specifically limited here.

[0238] For example, taking the Wiener filter as an example, the main function of the Wiener filter is to calculate the filter coefficient and determine whether the point cloud quality after the Wiener filter is improved. In other words, the filter coefficients described in the embodiments of the present application can be the coefficients of the Wiener filter processing, that is, the filter coefficients are the coefficients output by the Wiener filter.

[0239] Here, the Wiener filter is a linear filter whose optimization criterion is to minimize the mean square error. Under certain constraints, the square of the difference between its output and a given function (often called the desired output) is minimized. Through mathematical calculations, this problem can be transformed into a solution to the Toblitz equation. The Wiener filter is also known as the least squares filter or the least squares filter.

[0240] Among them, Wiener filtering is a method that uses the correlation characteristics and spectral characteristics of a stationary random process to filter a signal mixed with noise. It is currently one of the basic filtering methods. The specific algorithm of Wiener filtering is as follows:

[0241] For a series of (noisy) input signals, the output when the filter length or order is M is as follows,

[0242] Where M is the length or order of the filter, y(n) is the output signal, and x(n) is a column of input signals (mixed with noise).

[0243] Expressed in matrix form: y(n)=H(m)×X(n).

[0244] If the expected signal is known to be d(n), the error between the known signal and the expected signal can be calculated and expressed as e(n), specifically: e(n) = d(n) - y(n) = d(n) - H(m) × X(n), m = 0, 1, ...M.

[0245] The Wiener filter takes the minimum mean square error as the objective function, so let the objective function be: Min E(e(n) 2 )=E[(d(n)-H(m)×X(n)) 2 ].

[0246] When the filter coefficient is optimal, the inverse of the objective function to the coefficient should be 0, that is: 2E[(d(n)-H(m)×X(n))]×X(n)=0; E[d(n)X(n)]-H(m)E[X(n)X(n)]=0.

[0247] It can be further expressed as: Rxd-H×Rxx=0.

[0248] Among them, Rxd and Rxx are the correlation matrix of the input signal and the expected signal and the autocorrelation matrix of the input signal respectively. According to the optimal solution calculation of the Wiener-Hoff equation, the filter coefficient H can be obtained: H=Rxx -1 ×Rxd.

[0249] Furthermore, when performing Wiener filtering, a noisy signal and a desired signal are required; and in the embodiment of the present application, for the field of image processing, the two correspond to a lossy image (or called a "distorted image") and an original image (or called a "true image"), respectively, so that the input of the Wiener filter can be determined. For a k-order Wiener filter, a set of filter coefficients can be calculated based on the pixel values ​​of each pixel in the lossy image and the original image. As the optimal coefficients for Wiener filtering, this set of filter coefficients can be used to perform Wiener filtering on the lossy image, and a restored image that is close to the original image in terms of mean square error can be obtained, and it will also have a better subjective effect.

[0250] In a specific embodiment, taking the first video image as a lossy reconstructed video image as an example, with reference to FIG10 , the method for determining the filter coefficient of the first video image may include:

[0251] S1001: Determine a filtering correlation matrix according to reconstructed pixel values ​​of K reference pixels corresponding to at least one pixel in a reconstructed video image.

[0252] S1002: Determine a filtering correlation vector according to an original pixel value of at least one pixel point in an original video image corresponding to the reconstructed video image.

[0253] S1003: Determine a mutual correlation matrix according to the filter correlation matrix and the filter correlation vector, and determine an autocorrelation matrix according to the filter correlation matrix.

[0254] S1004: Calculate coefficients based on the mutual correlation matrix and the autocorrelation matrix to determine filter coefficients.

[0255] It should be noted that, in an embodiment of the present application, the filter coefficient of the first video image is determined based on at least one pixel point in the reconstructed video image. Specifically, it can be: determining the filter correlation matrix based on the reconstructed pixel values ​​of K reference pixel points corresponding to each of the at least one pixel point in the reconstructed video image; and determining the filter correlation vector based on the original pixel value of at least one pixel point in the original video image corresponding to the reconstructed video image; and then determining the filter coefficient based on the filter correlation matrix and the filter correlation vector.

[0256] It should also be noted that in the embodiment of the present application, the filter coefficients are determined based on the filter correlation matrix and the filter correlation vector. Specifically, the mutual correlation matrix is ​​determined based on the filter correlation matrix and the filter correlation vector; and the autocorrelation matrix is ​​determined based on the filter correlation matrix; and the coefficients are calculated based on the mutual correlation matrix and the autocorrelation matrix to determine the filter coefficients.

[0257] For example, taking the filtering of a video image as an example, assuming that the filter order is k and the total number of pixels in the video image is n, the filtering process when the sliding window is a square is: if the side length of the sliding window is l, then the filter order K=l 2 , that is, use the K reference pixels of the current pixel (including the current pixel itself) to calculate the optimal coefficient.

[0258] Among them, the total number of pixels in the video image is n = h × w; the filtering correlation matrix P (n, k) represents the reconstructed pixel values ​​of the k reference pixels of all pixels in the lossy image under this component; the filtering correlation vector S (n) represents the original pixel values ​​of all pixels in the original image under this component.

[0259] From the above algorithm we can get:

[0260] Cross-correlation matrix B(k): B(k) = P(n, k) T ×S(n);

[0261] Autocorrelation matrix A(k,k): A(k,k) = P(n,k) T ×P(n,k);

[0262] Then the optimal coefficient (vector) H(k): H(k)=A(k,k) -1 ×B(k).

[0263] Wherein, H(k) is a set of optimal coefficients of the k-th order filter under this component, that is, the filter coefficients described in the embodiment of the present application, which can also be called "Wiener coefficients" here, including a total of k filter coefficient values.

[0264] S803: Perform a first filtering process on at least one pixel point according to the filtering coefficient to determine a filtered image of the first video image.

[0265] It should be noted that in an embodiment of the present application, after the filter coefficient is calculated, the first video image can be subjected to a first filtering process according to the filter coefficient, so that the original image can be restored to the maximum extent according to the filtered image, thereby achieving the purpose of geometric denoising and enhancing the point cloud reconstruction quality.

[0266] It should also be noted that, in the embodiment of the present application, the first filtering process may include Wiener filtering. That is, the filter used for the first filtering process here may be an adaptive filter, such as a filter based on a neural network, a Wiener filter, etc., but this is not specifically limited. Taking the Wiener filter as an example, the filter coefficients described in the embodiment of the present application can be used for Wiener filtering, that is, the filter coefficients are the coefficients of the Wiener filtering process.

[0267] In a specific embodiment, performing a first filtering process on at least one pixel point according to a filtering coefficient to determine a filtered image of the first video image may include: performing Wiener filtering process on the pixel value of at least one pixel point according to the filtering coefficient to determine the filtered pixel value of at least one pixel point; and determining the filtered image according to the filtered pixel value of at least one pixel point.

[0268] In an embodiment of the present application, the first video image may be a reconstructed video image. After finding at least one pixel point that satisfies a preset neighborhood condition in the reconstructed video image, Wiener filtering may be performed on the at least one pixel point in the reconstructed video image according to a filter coefficient to determine a filtered pixel value of the at least one pixel point. Meanwhile, other pixels in the reconstructed video image are not processed (the reconstructed pixel values ​​are retained), thereby determining a filtered image corresponding to the reconstructed video image.

[0269] In an embodiment of the present application, Wiener filtering is performed on the pixel value of at least one pixel point according to the filter coefficient to determine the filtered pixel value of the at least one pixel point. Taking the first pixel point of the at least one pixel point as an example, in some embodiments, determining the filtered pixel value of the first pixel point may include: determining K reference pixels corresponding to the first pixel point in the reconstructed video image; performing Wiener filtering on the K reference pixels corresponding to the first pixel point according to the filter coefficient to determine the filtered pixel value of the first pixel point; wherein K is an integer greater than 0, and the K reference pixels include the first pixel point and (K-1) neighboring pixels adjacent to the first pixel point.

[0270] It should be noted that in the embodiment of the present application, the first pixel point can be any one of the at least one pixel points in the reconstructed video image. Taking the first pixel point as an example, a K-nearest neighbor search method can be used to search for a preset number of candidate pixel points in the reconstructed video image, calculate the distance value between the first pixel point and these candidate pixel points, and then select the (K-1) neighboring pixel points closest to the first pixel point from these candidate pixel points; the first pixel point itself and the (K-1) neighboring pixel points are determined as the final K reference pixel points.

[0271] It should also be noted that, in an embodiment of the present application, when performing Wiener filtering on at least one pixel point using a filter coefficient, the filter correlation matrix (expressed as P(n,k)) can be determined first based on the order of the Wiener filter and the reconstructed pixel values ​​of the K reference pixels corresponding to the at least one pixel point; then the filtered pixel value of the at least one pixel point can be determined based on the product of the filter correlation matrix and the filter coefficient, thereby determining the filtered image.

[0272] That is to say, after obtaining the filter coefficients, the set of filter coefficients H(k) is applied to the filter correlation matrix P(n,k), which can restore the original video image to the maximum extent, that is: R(n) = P(n,k) × H(k), where R(n) represents the filtered pixel values ​​corresponding to the n pixels in the filtered image under this component.

[0273] It should also be noted that, in the embodiments of the present application, a preset window method may also be used to determine the filtered pixel value of the at least one pixel. In some embodiments, performing Wiener filtering on the pixel value of the at least one pixel according to the filter coefficient to determine the filtered pixel value of the at least one pixel may include:

[0274] Use a preset window to scan each pixel in at least one pixel point, perform weighted averaging on the pixel values ​​of all the pixel points in the preset window, and update the pixel value of the pixel point located at the center of the preset window to the result of the weighted averaging to obtain the filtered pixel value of each pixel point; wherein a set of weighting coefficients used in the weighted averaging is the filtering coefficient, and the shape of the preset window is a rectangle or a diamond.

[0275] That is to say, here the shape of the preset window is first determined, and then the preset window is used to scan each pixel of the at least one pixel, the pixel values ​​of all the pixels in the preset window are weighted averaged, and the pixel value of the pixel located at the center of the preset window is updated to the result of the weighted average to obtain the filtered pixel value of each pixel, and then the filtered image is determined; wherein, the set of weighting coefficients used for weighted averaging is the filter coefficient.

[0276] In the embodiments of the present application, the "preset window" here refers to the window used for neighborhood filtering scanning, and may also be referred to as a template, template window, convolution kernel, box, etc. The scanning operation may also be referred to as convolution. However, an example of an embodiment of the present application uses a diamond-shaped filter window, as shown in FIG7 . Compared to the original rectangular filter, this example changes the shape of the window, which can better adapt to boundary changes and more effectively extract neighborhood information. The window in FIG7 includes 25 pixels, corresponding to a filter of order k=25. The embodiments of the present application may also use filters of other orders such as 9th order, 16th order, 36th order, etc. for neighborhood filtering.

[0277] S804: Determine filtering identification information of the first video image according to the first video image and the filtered image.

[0278] It should be noted that in the embodiment of the present application, after determining the filtered image, the encoding end may further determine filter identification information based on the first video image and the filtered image, wherein the filter identification information is used to indicate whether the first video image is processed using the first filter.

[0279] It should also be noted that in the embodiment of the present application, the first video image is a lossy reconstructed video image. In one possible implementation, determining the filter identification information of the first video image based on the first video image and the filtered image may include: determining a first-generation value of the reconstructed video image relative to the original video image, and determining a second-generation value of the filtered image relative to the original video image; and determining the filter identification information based on the first-generation value and the second-generation value.

[0280] In an embodiment of the present application, determining the first generation value of the reconstructed video image relative to the original video image may include: calculating the cost of the reconstructed video image and the original video image using a rate-distortion cost method, and using the obtained first rate-distortion value as the first generation value.

[0281] In an embodiment of the present application, determining the second-generation value of the filtered image relative to the original video image may include: using a rate-distortion cost method to calculate the cost of the filtered image and the original video image, and using the obtained second rate-distortion value as the second-generation value.

[0282] In this implementation, the filter identification information can be determined using a rate-distortion cost approach. First, the first-generation value of the reconstructed video image relative to the original video image and the second-generation value of the filtered image relative to the original video image are determined using the rate-distortion cost approach. Then, based on the comparison between the first-generation value and the second cost value, the filter identification information for the first video image is determined. The cost value here can be a distortion value used for distortion measurement, a rate-distortion cost result, or the like, without specific limitation.

[0283] It should be noted that in order to more accurately measure the performance improvement before and after filtering, the embodiment of the present application simultaneously performs a rate-distortion trade-off on the filtered image and the reconstructed video image. Here, the rate-distortion cost method can be used to calculate the rate-distortion value after the comprehensive quality improvement and bitrate increase. Among them, the first rate-distortion value and the second rate-distortion value can respectively represent the rate-distortion cost results of the original video image relative to the reconstructed video image or the filtered image, and are used to represent the compression efficiency of the point cloud before and after filtering. For the calculation of the first rate-distortion value and the second rate-distortion value, the specific calculation formula is as follows: J = D + λ × R.

[0284] Here, J is the rate-distortion value, D is the SSE of the original video image and the reconstructed video image or the filtered image, that is, the sum of squares of the errors of corresponding points; λ is a quantity related to the quantization parameter QP. R is the total geometric bitstream size divided by the number of frames.

[0285] In a specific embodiment, determining the filter identification information based on the first generation value and the second generation value may include: if the second generation value is less than the first generation value, determining the value of the filter identification information to be the first value; if the second generation value is greater than the first generation value, determining the value of the filter identification information to be the second value.

[0286] It should be noted that in the embodiment of the present application, the first value and the second value are different, and the first value and the second value can be in parameter form or in digital form. Specifically, the filter identification information here can be a parameter written in the profile or a flag value, which is not specifically limited here.

[0287] It should also be noted that in the embodiment of the present application, the first value can be set to 1 and the second value can be set to 0; or, the first value can be set to 0 and the second value can be set to 1; or, the first value can also be set to true and the second value can also be set to false; or, the first value can be set to false and the second value can be set to true; but this is not specifically limited here.

[0288] For example, assuming that the first value is set to 1 and the second value is set to 0, if the cost J of the filtered image f Less than the cost of reconstructing the video image J r , the filter identification information of the video image is set to 1, and the corresponding filter coefficients and optimal combination and other filtering information are recorded; otherwise, the filter identification information of the video image is set to 0, and the corresponding filter coefficients and optimal combination and other filtering information are no longer recorded.

[0289] In another possible implementation, determining the filtering identification information of the first video image based on the first video image and the filtered image may include: determining a first performance gain value of the reconstructed video image relative to the original video image, and determining a second performance gain value of the filtered image relative to the original video image; and determining the filtering identification information based on the first performance gain value and the second performance gain value.

[0290] In an embodiment of the present application, determining a first performance gain value of the reconstructed video image relative to the original video image may include: using a preset performance measurement indicator to calculate the performance gain of the reconstructed video image and the original video image to obtain the first performance gain value.

[0291] In an embodiment of the present application, determining a second performance gain value of the filtered image relative to the original video image may include: using a preset performance measurement indicator to calculate the performance gain of the filtered image and the original video image to obtain the second performance gain value.

[0292] It should be noted that the preset performance measurement indicator here can be PSNR or other performance indicators, which are not specifically limited here. For example, the first performance gain value can be the PSNR of the reconstructed video image relative to the original video image, and the second performance gain value can be the PSNR of the filtered image relative to the original video image.

[0293] In a specific embodiment, determining the filtering identification information based on the first performance gain value and the second performance gain value may include: if the second performance gain value is greater than the first performance gain value, determining the value of the filtering identification information to be the first value; if the second performance gain value is less than the first performance gain value, determining the value of the filtering identification information to be the second value.

[0294] Exemplarily, assuming that the first value is set to 1 and the second value is set to 0, if the performance gain PSNR of the filtered image is greater than the performance gain PSNR of the reconstructed video image, the filter identification information of the video image is set to 1, and the corresponding filter coefficients, optimal combination and other filtering information are recorded; otherwise, the filter identification information of the video image is set to 0, and the corresponding filter coefficients, optimal combination and other filtering information are no longer recorded.

[0295] In another possible implementation, the rate-distortion cost and performance gain can be comprehensively considered. Specifically, in some embodiments, determining the filter identification information based on the first performance gain value and the second performance gain value may include: if the second performance gain value is greater than the first performance gain value and the second generation value is less than the first generation value, determining the value of the filter identification information to be the first value; if the second performance gain value is less than the first performance gain value or the second generation value is greater than the first generation value, determining the value of the filter identification information to be the second value.

[0296] In the embodiments of the present application, not only the rate-distortion value determined by the rate-distortion cost method can be considered, but also the performance value determined by a preset performance metric (e.g., PSNR value). In other words, not only the improvement in the reconstructed geometric quality is considered, but also the cost required to write filtering information such as filter coefficients and optimal combinations into the bitstream is calculated. The performance of the two is comprehensively considered to determine whether the compression performance has been improved after filtering, thereby determining whether the encoder should transmit filtering information such as filter coefficients and optimal combinations.

[0297] In addition, it should be noted that when the first generation value is equal to the second generation value, or the first performance gain value is equal to the second performance gain value, the value of the filter identification information can be determined to be the first value; or, the value of the filter identification information can be determined to be the second value; there is no specific limitation on this.

[0298] Furthermore, in some embodiments, the method may further include:

[0299] If the value of the filter identification information is the first value, determining that the filter identification information indicates that the first video image is processed using the first filter;

[0300] If the value of the filter identification information is the second value, it is determined that the filter identification information indicates that the first video image does not use the first filtering process.

[0301] It should be noted that in the embodiment of the present application, the first value and the second value are different, and the first value and the second value can be in parameter form or in digital form. Specifically, the filter identification information here can be a parameter written in the profile or a flag value, which is not specifically limited here.

[0302] It should also be noted that in the embodiment of the present application, the first value can be set to 1 and the second value can be set to 0; or, the first value can be set to 0 and the second value can be set to 1; or, the first value can also be set to true and the second value can also be set to false; or, the first value can be set to false and the second value can be set to true; but this is not specifically limited here.

[0303] Exemplarily, assuming that the first value is set to 1 and the second value is set to 0, taking the rate-distortion cost and performance gain into consideration as an example, if the PSNR1 corresponding to the reconstructed video image is greater than the PSNR2 corresponding to the filtered image, or the Cost1 corresponding to the reconstructed video image is less than the Cost2 corresponding to the filtered image, that is, the PSNR value of the filtered image decreases, or the rate-distortion cost of the filtered image increases, then it can be considered that the filtering effect is poor. At this time, the value of the filtering identification information is determined to be 0, which is used to indicate that the first video image does not undergo the first filtering process; correspondingly, if the PSNR1 corresponding to the reconstructed video image is less than the PSNR2 corresponding to the filtered image and the Cost1 corresponding to the reconstructed video image is greater than the Cost2 corresponding to the filtered image, that is, the PSNR of the filtered image is improved and the rate-distortion cost is reduced, then it can be considered that the filtering effect is good. At this time, the value of the filtering identification information is determined to be 1, which is used to indicate that the first video image undergoes the first filtering process.

[0304] S805: If the filter identification information indicates that the first video image uses the first filter processing, the filter identification information and the corresponding filter coefficients of the first video image are encoded, and the obtained encoding bits are written into the bitstream.

[0305] It should be noted that in an embodiment of the present application, after the encoder determines the filter identification information based on the reconstructed video image and the filtered image, if the filter identification information indicates that the reconstructed video image is to be subjected to a first filtering process, then the filter identification information can be written into the bitstream, and the filter coefficient can also be selectively written into the bitstream.

[0306] It should also be noted that, in an embodiment of the present application, the method may further include: if the filter identification information indicates that the first video image does not use the first filter processing, encoding the filter identification information of the first video image and writing the obtained encoded bits into the bitstream.

[0307] In an embodiment of the present application, if the filter identification information indicates that the first video image uses the first filtering processing, then the filter identification information of the first video image and the corresponding filter coefficient can be written into the bitstream; if the filter identification information indicates that the first video image does not use the first filtering processing, then at this time there is no need to write the filter coefficient into the bitstream, only the filter identification information needs to be written into the bitstream so that it can be subsequently transmitted to the decoding end through the bitstream.

[0308] It is understood that in the embodiment of the present application, the first video image is a video image in a video sequence. In some embodiments, the method further includes: determining filtering parameters of the video sequence; encoding the filtering parameters of the video sequence, and writing the obtained encoded bits into the bitstream.

[0309] Among them, the filtering parameters may include at least one of the following: filtering activation identification information, total number of frames, category information, filtering identification information of at least one video image in the video sequence and corresponding filtering coefficients, and the first video image is any one of the at least one video image.

[0310] It should be noted that in the embodiment of the present application, the filter activation identification information (i.e., the filter activation flag) can be used to indicate whether the video sequence uses the first filtering process, represented here by Wiener_filtering_flag, which occupies 1 bit in the bitstream; the total number of frames can be used to indicate the total number of frames in the video sequence, represented here by frame_num, which occupies 32 bits in the bitstream; the category information can be used to indicate the preset neighborhood conditions of each video image in the video sequence, which can also be called combination information, so it is represented here by groups, which occupies 8 bits in the bitstream; the filter identification information (i.e., the filter flag) of each video image can be used to indicate whether the video image uses the first filtering process, represented here by filtering_flag_of_each_frame, which occupies 1 bit in the bitstream; the filter coefficient of each video image can be represented by WFcoef, which occupies a maximum of 32×k×frame_num bits. Here, only when a video image uses the first filtering process, the corresponding filter coefficient will be written here; otherwise, if a video image does not use the first filtering process, the filter coefficient will not be written.

[0311] In addition, it should be noted that in the embodiment of the present application, each frame of video image can generate two geometric video images, specifically a far frame video image and a near frame video image; then, for each of these two frames of video image, whether to use the first filtering process and the corresponding filter coefficients are determined. Therefore, if the video sequence includes n frames of video image, the total number of frames here is 2n, where n is an integer greater than 0.

[0312] Furthermore, in some embodiments, the code stream may include a first code stream unit. Accordingly, encoding the filtering parameters of the video sequence and writing the obtained coded bits into the code stream may include:

[0313] According to the filtering parameters of the video sequence, filter activation identification information is stored in a syntax object in the first bitstream unit; if the filter activation identification information indicates that the video sequence uses the first filtering process, at least one of the total number of frames, category information, filter identification information of at least one video image, and corresponding filter coefficients is further stored in the syntax object of the first bitstream unit to obtain syntax object information of the first bitstream unit; the syntax object information in the first bitstream unit is encoded, and the obtained encoded bits are written into the bitstream.

[0314] It should be noted that in this embodiment of the present application, the first bitstream unit can be represented by V3C_WF. The syntax object information here may include a first syntax object, a second syntax object, a third syntax object, a fourth syntax object, and a fifth syntax object. The first syntax object is represented by syntax.Wiener_filtering_flag, the second syntax object is represented by syntax.frame_num, the third syntax object is represented by syntax.groups, the fourth syntax object is represented by syntax.filtering_flag_of_each_frame, and the fifth syntax object is represented by syntax.WFcoef. Here, the filtering parameters are written to the syntax objects of the first bitstream unit according to actual needs.

[0315] In this way, during the encoding process, the filter activation identification information is first determined and can be written into syntax.Wiener_filtering_flag; if the filter activation identification information indicates that the video sequence uses the first filtering process, then the total number of frames, category information, the filter identification information of at least one video image in the video sequence, and at least one of the corresponding filter coefficients can be determined; specifically, it can be: the total number of frames is written into syntax.frame_num, the category information is written into syntax.groups, the filter identification information of at least one video image is written into syntax.filtering_flag_of_each_frame, and the corresponding filter coefficient is written into syntax.WFcoef; wherein, if the filter identification information of a video image indicates that the video image does not use the first filtering process, then the filter coefficient corresponding to the video image will not be written into syntax.WFcoef.

[0316] Furthermore, in some embodiments, the method may also include: if the filter enable identification information indicates that the video sequence does not use the first filtering process, then the step of storing at least one of the total number of frames, category information, filter identification information of at least one video image, and corresponding filter coefficients in the syntax object of the first code stream unit is not continued.

[0317] In some embodiments, for filtering activation identification information, the method may further include:

[0318] If the video sequence is processed using the first filter, determining that the value of the filter activation identification information is equal to the first value;

[0319] If the video sequence does not use the first filtering process, it is determined that the value of the filtering enabling identification information is equal to the second value.

[0320] In the embodiment of the present application, the first value and the second value are different, and the first value and the second value can be in parameter form or in digital form. Specifically, the filtering activation identification information here can be a parameter written in the profile or a flag value, which is not specifically limited here.

[0321] In an embodiment of the present application, the first value can be set to 1 and the second value can be set to 0; or, the first value can be set to 0 and the second value can be set to 1; or, the first value can also be set to true and the second value can also be set to false; or, the first value can be set to false and the second value can be set to true; but no specific limitation is made here.

[0322] Exemplarily, the filtering parameters of the video sequence are encoded, and the code is expressed as:

[0323] Among them, syntax is a syntax object of the PCCHighLevelSyntax class. After filtering, various information will be stored in the variables of the object members so as to be written into the code stream.

[0324] It should also be noted that, in the embodiment of the present application, the first video image may include a reconstructed video image, wherein the reconstructed video image may include a lossy reconstructed geometric video image and / or a lossy reconstructed texture video image.

[0325] In some embodiments, when the first video image includes a lossy reconstructed geometric video image, the method may further include: after performing a first filtering process on the reconstructed geometric video image, performing quality enhancement processing on the corresponding texture video image based on the filtered reconstructed geometric video image.

[0326] That is to say, in the embodiment of the present application, the method can not only be applied to lossy geometrically reconstructed video images to implement the neighborhood difference-based Wiener filtering post-processing technology to achieve the purpose of geometric denoising and enhancing point cloud quality; it can also be applied to filtering lossy texture-reconstructed video images to achieve the purpose of enhancing attribute quality.

[0327] Furthermore, in the embodiments of the present application, for a filter, the filter type can be used to indicate the filter order, and / or the filter shape, and / or the filter dimension. The filter shape includes diamond, rectangle, etc., and the filter dimension includes one-dimensional, two-dimensional, or even more dimensions.

[0328] Furthermore, in an embodiment of the present application, the filter may be a Wiener filter, an adaptive filter, or the like. In addition, the filter type may correspond to filters of different orders, for example, filters of orders 12, 16, 20, 32, or 128; different types may also correspond to filters of different dimensions, for example, one-dimensional filters, two-dimensional filters, etc., which are not specifically limited here. In other words, if a 16-order filter needs to be determined, 16 points may be used to determine a 16-order asymmetric filter, or an 8-order one-dimensional symmetric filter, or other numbers of filters (for example, more special two-dimensional or three-dimensional filters, etc.), etc., and the filter is not specifically limited here.

[0329] Furthermore, in the embodiment of the present application, for the order of the filter, the value of the order can be 16, 20, 32, 64, etc. Exemplarily, the value of the order is 20, but this is not specifically limited here.

[0330] Furthermore, in the embodiment of the present application, the original video image can be directly obtained through the point cloud reading function of the codec program, and the reconstructed video image is obtained after all encoding operations are completed. In addition, the reconstructed video image in the embodiment of the present application can be a reconstructed image output after decoding, or it can be used as a reference for decoding subsequent image frames; in addition, the encoding method here can be applied not only within the prediction loop, that is, used as an in-loop filter, which can be used as a reference for decoding subsequent image frames; it can also be applied outside the prediction loop, that is, used as a post filter, and not used as a reference for decoding subsequent image frames; the embodiment of the present application does not specifically limit this.

[0331] In this way, taking the Wiener filter as an example, when using the Wiener filter to perform the first filtering processing on the reconstructed video image, the reconstructed video image and the filter coefficient can be input into the Wiener filter at the same time, that is, the input of the Wiener filter is the filter coefficient and the reconstructed video image, and finally the filtering processing of the reconstructed video image can be completed based on the filter coefficient to obtain the corresponding filtered image.

[0332] That is, in the embodiment of the present application, the filter coefficients are obtained based on the original video image and the reconstructed video image. Therefore, applying the filter coefficients to the reconstructed video image can restore the original video image to the greatest extent possible. For example, in the embodiment of the present application, taking a lossy reconstructed video image in a point cloud sequence as an example, assuming that the order of the filter is k and the total number of pixels in the reconstructed video image is n, the filter correlation matrix P(n,k) is used to represent the reconstructed pixel values ​​of k reference pixels corresponding to all pixels in the reconstructed video image under a certain color component (such as Y component / U component / V component).

[0333] In this way, based on the formula R(n)=P(n,k)×H(k), the filter coefficient corresponding to a certain color component (represented by H(k)) is applied to the reconstructed video image, that is, the filter correlation matrix P(n,k), so that the filtered pixel value R(n) under the color component can be obtained, and then the filtered image can be determined.

[0334] Furthermore, in the embodiments of the present application, during the filtering process, some peripheral features of the image, such as image boundaries and patch boundaries, are considered. For image boundaries, in order to apply the Wiener filter to the outermost pixels of the image, the image exterior can be filled (for example, with a fill value of 0), or these outer pixels can be left unprocessed. For patch boundaries, no special processing is required and calculation can be performed according to the methods of the embodiments of the present application.

[0335] It should also be noted that, in the embodiment of the present application, the pixel value of a pixel point can be the value of any component of the three color components (also referred to as three color channels). For example, in an image in YUV format, "Y" represents the brightness component, and the value of the brightness component is usually called the grayscale value; "U" and "V" represent the chrominance component, and the chrominance component can be stored as Cb and Cr, where Cb is the blue chrominance component and Cr is the red chrominance component. The pixel value of a pixel point can be the value of the brightness component, the blue chrominance component, or the red chrominance component; in an image in RGB format, the pixel value of a pixel point can be the value of red, green, or blue, and so on. In some cases, when the first video image has only some components, for example, in V-PCC, when filtering the reconstructed geometric video image, the pixel value of the pixel point refers to the grayscale value. When filtering the reconstructed texture video image, the two chrominance components can be filtered separately. When filtering the blue chrominance component, the pixel value of the above-mentioned pixel point refers to the value of the blue chrominance component; when filtering the red chrominance component, the pixel value of the above-mentioned pixel point refers to the value of the red chrominance component; this is not specifically limited here.

[0336] It should also be noted that in the embodiment of the present application, the category information (i.e., the optimal combination) is associated with the bit rate of the first video image. Here, the same optimal combination can be set for the same video sequence, or different video sequences can be set with different optimal combinations at different bit rates. Alternatively, two optimal combinations can be specified for each bit rate, and different optimal combinations can be used for filtering and comparing performance, and then the combination with better performance can be selected as the final optimal combination; this is not specifically limited here.

[0337] Furthermore, in the embodiment of the present application, if different code rates correspond to at least two neighborhood conditions, then for determining the optimal preset neighborhood condition, in some embodiments, the method may further include:

[0338] Determine at least two neighborhood conditions corresponding to the first video image; wherein the at least two neighborhood conditions include a first neighborhood condition and a second neighborhood condition;

[0339] Determining at least one pixel point in the first video image that meets a first neighborhood condition and a corresponding first filter coefficient, and performing a first filtering process based on the first filter coefficient and the at least one pixel point that meets the first neighborhood condition to determine a first filtered image;

[0340] Determining at least one pixel point in the first video image that meets a second neighborhood condition and a corresponding second filter coefficient, and performing a second filtering process based on the second filter coefficient and the at least one pixel point that meets the second neighborhood condition to determine a second filtered image;

[0341] Determining a preset neighborhood condition from the first neighborhood condition and the second neighborhood condition according to the first filtered image and the second filtered image includes at least one of the following methods:

[0342] Determining, according to the performance gain values ​​of the first filtered image and the second filtered image relative to the first video image, a neighborhood condition corresponding to a maximum performance gain value as a preset neighborhood condition;

[0343] According to the cost values ​​of the first filtered image and the second filtered image relative to the first video image, a neighborhood condition corresponding to a minimum cost value is determined as a preset neighborhood condition.

[0344] It should be noted that in the embodiment of the present application, at least two neighborhood conditions can be set for different bit rates, such as the first neighborhood condition and the second neighborhood condition; then the first filtered image can be obtained using the method of the embodiment of the present application according to the first neighborhood condition, or the second filtered image can be obtained using the method of the embodiment of the present application according to the second neighborhood condition; then, the performance gain or cost value is compared based on the first filtered image and the second filtered image, and the best combination is determined, that is, the preset neighborhood condition. It should be noted that, regarding the comparison of performance gain or cost value, the specific process can be referred to the above content and will not be described in detail here. That is to say, two optimal combinations can be specified for each bit rate, and different optimal combinations are filtered separately, and the performance or cost value is compared, and the combination with better performance is selected as the final preset neighborhood condition.

[0345] In another embodiment of the present application, a code stream is provided, which can be generated by bit encoding based on the information to be encoded; wherein the information to be encoded includes at least one of the following: filter activation identification information, the total number of frames, category information, filter identification information of at least one video image, and the corresponding filter coefficient.

[0346] In an embodiment of the present application, the filter enable identification information is used to indicate whether the video sequence uses the first filter processing, and the filter identification information is used to indicate whether the first video image uses the first filter processing. The video sequence includes at least one video image, and the first video image is any one of the at least one video image.

[0347] In this embodiment of the present application, the codestream can be composed of five sub-codestreams, each of which records corresponding V3C_Unit information. V-PCC already has five codestream units: V3C_VPS, V3C_AD, V3C_OVD, V3C_GVD, and V3C_AVD. These represent point cloud parameter sets, patch data groups, occupancy map video data, geometry video data, and attribute video data. V-PCC also reserves multiple unnamed codestream units for future use.

[0348] In some embodiments, in addition to the five code stream units mentioned above, the code stream may also include:

[0349] A first bitstream unit is configured to store filtering parameters for a video sequence. The filtering parameters include at least one of the following: filter activation information, a total number of frames, category information, and filter identification information and corresponding filter coefficients for at least one video image. The first bitstream unit can be represented by V3C_WF.

[0350] Furthermore, in some embodiments, the first code stream unit may be composed of header information and payload. The payload may include:

[0351] A first syntax object is configured to store filtering enabling identification information;

[0352] A second syntax object is configured to store a total number of frames when the filter enabling flag information indicates that the video sequence is to be processed using the first filter;

[0353] A third syntax object is configured to store category information when the filter enabling flag information indicates that the video sequence uses the first filter processing;

[0354] A fourth syntax object is configured to store filter identification information of each of at least one video image when the filter enabling identification information indicates that the video sequence uses the first filter processing;

[0355] The fifth syntax object is configured to store a filter coefficient corresponding to the at least one video image when the filter identification information indicates that the at least one video image uses the first filter processing.

[0356] It should also be noted that the first code stream unit V3C_WF is used to store Wiener filter related information. The header information of V3C_WF has been configured by V-PCC and does not need to be processed. In the payload (actual data), the following information can be adaptively written here:

[0357] (1) Filtering activation identification information Wiener_filtering_flag, occupies 1 bit; if V-PCC does not use Wiener filtering, the following information does not need to be written.

[0358] (2) Total frame number frame_num, occupies 32 bits;

[0359] (3) Category information groups, which takes up 8 bits (records the beginning and end group numbers, each taking up 4 bits);

[0360] (4) Each frame filtering flag information filtering_flag_of_each_frame, occupies 1×frame_num bits;

[0361] (5) Each frame filter coefficient WFcoef (not written if a frame does not need filtering); occupies a maximum of 32×k×frame_num bits.

[0362] It should also be noted that, in the embodiment of the present application, the syntax object information in the first code stream unit may be encoded using an entropy coding method or a binarization method, etc., and then the value corresponding to each syntax object is written into the code stream.

[0363] In addition, in the embodiments of the present application, for these grammatical objects, the value range of each field has certain restrictions (i.e., conformance). In one possible implementation, the value range can be as follows:

[0364] Filtering enable flag information Wiener_filtering_flag: true or false;

[0365] Total frame number frame_num: an integer greater than 1.

[0366] Category information groups: The first four digits represent the starting group number (4), and the last four digits represent the ending group number (8 or 10). In the experiment, the value is 72 or 74 (if other combinations are selected, the value will be different, ranging from 0 to 255).

[0367] Each frame filtering flag information filtering_flag_of_each_frame: true or false;

[0368] Each frame filter coefficient Wfcoef: -2 31 to 231 -1, that is, -2147483648 to 2147483647.

[0369] For example, Figure 11 shows a schematic diagram of a bitstream structure provided by an embodiment of the present application. As shown in Figure 11, a bitstream unit (V3C_Unit) may include: a V3C_VPS unit, a V3C_AD unit, a V3C_OVD unit, a V3C_WF unit, a V3C_GVD unit, and a V3C_AVD unit; wherein the V3C_WF unit may be composed of header information (V3C_Unit_Header) and payload (Wiener_filtering_flag, frame_num, groups_information, filtering_flag_of_each_frame, Wiener_Filtering_codfficients). Here, V3C_Unit_Header occupies 27 bits, represented by u(27); Wiener_filtering_flag occupies 1 bit, represented by u(1); frame_num occupies 32 bits, represented by u(32); groups_information occupies 8 bits, represented by u(8); filtering_flag_of_each_frame occupies 1×frame_num bits, represented by u(1)×frame_num; and Wiener_Filtering_codfficients occupies 32×k×frame_num bits, represented by u(32)×k×frame_num. These code stream units are used as the pixel-level stream (SampleStream V3CUnit) of V3C and are written into the code stream (Bitstream). In addition, although the V3C_WF unit is located between the V3C_OVD unit and the V3C_GVD unit in Figure 11, the V3C_WF unit can also be located in other positions and is not limited to this.

[0370] It should also be noted that, in an embodiment of the present application, a new first code stream unit: V3C_WF can be created in the V-PCC code stream to write the filtering parameters of the video sequence; alternatively, these filtering parameters can also be written into the existing code stream unit: V3C_GVD, which is regarded as part of the geometric codec code stream.

[0371] This embodiment provides an encoding method, which determines, based on a first video image, at least one pixel point that satisfies a preset neighborhood condition, and then determines a filter coefficient for the first video image based on the at least one pixel point; then performs a first filtering process on the at least one pixel point based on the filter coefficient to determine a filtered image of the first video image; and determines filter identification information for the first video image based on the first video image and the filtered image; if the filter identification information indicates that the first video image uses the first filtering process, the filter identification information and the corresponding filter coefficient of the first video image are encoded, and the resulting encoded bits are written into a bitstream. In this way, since the preset neighborhood condition of the encoding end is pre-set according to the bit rate of the first video image, there is no need to perform multiple coefficient calculations to determine the optimal preset neighborhood condition, thereby reducing the time complexity; in addition, after the encoding end determines at least one pixel point that meets the preset neighborhood condition from the first video image, it can also determine the corresponding filter coefficient based on the at least one pixel point; and only when the first video image needs to use the first filtering process, the corresponding filter coefficient and the category information used to indicate the preset neighborhood condition will be passed to the decoding end; thereby further reducing the time complexity, and optimizing the reconstructed video image, which can improve the reconstructed geometric quality of the point cloud; at the same time, the present application also standardizes the design of the code stream writing method to ensure the stability of the codec system operation and the rationality of the code stream structure; in this way, for the geometric lossy and attribute lossy methods in V-PCC, not only the quality of the reconstructed point cloud is enhanced, but also the quality of the video frame can be improved, further improving the compression efficiency and improving the codec performance.

[0372] In another embodiment of the present application, based on the encoding and decoding method described in the aforementioned embodiment, a V-PCC point cloud frame adaptive Wiener filtering quality enhancement technology based on neighborhood differences has been proposed. In this technology, a technology for performing adaptive Wiener filtering post-processing based on neighborhood differences on V-PCC geometric reconstructed images is proposed to achieve the purpose of point cloud geometry denoising and improving subjective effects. For the geometric video image generated by V-PCC, 8 neighborhood differences are calculated for each pixel, and pixels are grouped according to the differences. Pixels with the same total difference are considered to have similar neighborhood differences and are divided into the same group (a maximum difference threshold is set, and if the threshold is exceeded, they are all divided into the last group). This technology compares the filtering effects of different group combinations, selects the best filtering effects, and selects them as the final filtering combination. In particular, for different group combinations, the principle of adjacent combinations must be followed. For example, groups with neighborhood differences of {1,2} can be combined, while groups {1,3} or {1,2,4} will not be combined because the neighborhood differences are not adjacent. In addition, a single group can also be regarded as a combination. Then, all cases (from {0} to {0, 1, 2…, d}) are traversed, and the filter coefficients for each combination are calculated and compared. The filtering effects (such as the PSNR improvement) are then compared. The optimal coefficients for the Wiener filter are calculated using the pixels in this combination, and the pixels in this combination are filtered (the remaining pixel values ​​remain unchanged). The rate-distortion cost (RDCost) is also calculated. If the cost increases after filtering, no filtering is performed at the decoder, and the filter flag is set to zero. After filtering this combination, the filtered pixels are removed, and the iterative search for the optimal combination continues for the remaining groups until the set filtering threshold is reached. Finally, the flag, combination information (if the flag is set), and filter coefficients (if the flag is set) are written into the bitstream. Similarly, at the decoder, pixel combination is performed using the same method, the filter information is decoded, and filtering is performed.

[0373] The main problems with this technology are high time complexity and irregular writing of filtering information into the bitstream. First, finding three combinations for filtering is unnecessary. Experiments have shown that, in the vast majority of cases, only the first filtering combination achieves positive gain. Furthermore, removing these groups from the overall search and re-finding the optimal combination is highly complex and ineffective. Second, this technology searches for the optimal combination through a double loop: continuously adding new groups to the current combination, recalculating the optimal coefficients, filtering the pixel values, and comparing the improvement in PSNR for the entire video after filtering. This operation has extremely high time complexity. Experiments have also revealed that the optimal combination follows a pattern. The optimal combination is also related to the bitrate. Therefore, if the optimal combination can be directly determined, the pixels within the neighborhood difference range can be found directly, and the coefficient calculation, filtering, and rate-distortion analysis can be performed only once, the entire operation can be completed. Furthermore, this technology does not provide detailed information on how to write the bitstream. Given the strict syntax and bitstream structure of V-PCC, writing information such as coefficients into the correct location in the bitstream while ensuring that it does not affect the encoding and decoding of other modules is a crucial task. This technique writes the geometry bitstream to a temporary file, adds flags, group information, filter coefficients, and other information after the first few characters of the temporary file, and then reads the temporary file from the V-PCC to complete the bitstream writing. While this method ensures that the written information is read, it is not rigorous and cannot guarantee that the geometry video encoding information will not be affected.

[0374] Based on this, in an embodiment of the present application, an optimized technique for performing neighborhood-difference-based Wiener filtering post-processing on V-PCC geometrically reconstructed images is proposed to achieve geometric denoising and enhance point cloud quality. This technique continues to utilize a pixel grouping method for geometric images based on neighborhood differences, employing only one Wiener filter at the codec end while maintaining the optimal combination method for different bitrates. At the encoder end, the pixels in this combination are used to calculate the corresponding optimal coefficients, which are then filtered and the rate-distortion analysis is performed. If the rate-distortion cost is lower than that of the reconstructed geometric frame, the following information is transmitted to the decoder: whether V-PCC Wiener filtering is enabled, the total number of frames, the filter flag for each frame, the combination information for each frame, and the optimal coefficient for each frame. At the decoder end, after decoding this information, pixel grouping, combination, filtering, and other post-processing are performed on the point cloud geometrically reconstructed image. It should be noted that this technique targets V-PCC geometrically lossy and attribute-lossy coding methods, which can include inter-frame and intra-frame coding. Extensive experiments have been conducted to demonstrate the effectiveness of the proposed technique.

[0375] In order to execute the encoding method of the embodiment of the present application, a Wiener filtering module can be added to the V-PCC encoding end. Figure 12 shows a related local architecture schematic diagram provided by the embodiment of the present application. As shown in Figure 12, a Wiener filtering module 35 is added here, which can also be called a Wiener filter. The Wiener filtering module 35 receives the smoothed reconstructed geometric video frame (referred to as the reconstructed geometric frame for short) output from the smoothing module 17, and the original geometric video frame (referred to as the original geometric frame for short) output from the geometric frame filling module 23 (or the geometric frame generation module 19). The reconstructed geometric video frame is Wiener filtered, and the filtered reconstructed geometric video frame is output to the texture frame generation module 15 for the generation of the corresponding texture video frame. After Wiener filtering, the quality of the reconstructed geometric video frame is enhanced, which can also improve the quality of the generated texture video frame. In the embodiment of the present application, the Wiener filtering module 35 can also record the generated filter coefficients, which are sent together with the geometric video frame data after encoding, or sent as syntax elements.

[0376] FIG13 illustrates another exemplary architecture that can implement the encoding method of an embodiment of the present application. As shown in FIG13 , the added Wiener filtering module 35 receives the reconstructed geometric video frame output from the second video compression module 31 and the original geometric video frame output from the geometric frame filling module 23 (or geometric frame generation module 19). The filtered reconstructed geometric video frame is output to the smoothing module 17. This architecture can also achieve similar effects to the architecture of FIG12 .

[0377] It should also be noted that in this embodiment of the present application, the smoothing module 17 can also be eliminated. The Wiener filtering module 35 receives the reconstructed geometric video frame output from the second video compression module 31 and the original geometric video frame output from the geometric frame filling module 23. The filtered reconstructed geometric video frame is output to the texture frame generation module 15.

[0378] It should also be noted that, in the embodiment of the present application, the Wiener filtering method of this embodiment can also be used to filter the reconstructed texture video frame, generate corresponding filter coefficients, and encode and transmit them.

[0379] In a specific embodiment, the encoding operation is as follows:

[0380] (1) According to the V-PCC encoding process, each frame of input point cloud is divided into patches before encoding. The patches are closely arranged and mapped to two single-channel images of the far layer and the near layer, thus generating a geometry video. The texture information of each patch is mapped to two three-channel images, thus generating an attribute video. The difference between the two geometry / texture images in each frame is very small. The occupancy map is used to indicate the occupancy of useful pixels in the video. In the V-PCC geometry encoding part, the input of the Wiener filter can be obtained here: the reconstructed (lossy) geometry video frame and the original (lossless) geometry video frame.

[0381] (2) In this technology, considering that the background of the geometric video frame is black, and the grayscale distribution of the foreground patches is uneven, and the brightness changes dramatically at the junction with the background, as shown in Figure 14, a pixel grouping method based on neighborhood differences is used. If the traditional Wiener filtering method is used, a set of optimal coefficients is calculated for the entire image and filtered, the effect is difficult to achieve as expected. In fact, after testing, if the Wiener filtering is directly performed with a filter of k=25, the image PSNR will even drop by about 0.05dB. This is because the large neighborhood differences between each pixel in the geometric video frame are not fully considered.

[0382] Based on this, the technique calculates the sum of the absolute differences between each pixel and each of its eight neighboring pixels, denoted as diff, and uses this value for classification (as shown in Figure 5A). Furthermore, if the current pixel value is 0, it indicates that the pixel is most likely a background point. Since filtering background points is meaningless, the technique chooses not to calculate the neighborhood differences for that pixel to reduce time complexity.

[0383] Here, the neighborhood difference diff calculation method is, Among them, a is the current pixel value, a i is the pixel value of the ith neighborhood. In Figure 5A, the absolute value of the difference between the center point and the eight neighborhoods is calculated and summed.

[0384] Furthermore, all point clouds in the V-PCC standard test sequence were compressed into 32 frames under intra-frame coding conditions, and the best combinations at various bit rates were calculated. Here are the top two most selected best combinations and their occurrence times at each bit rate:

[0385] R01: 4–10 was selected 82 times. 1–11 was selected 58 times.

[0386] R02: 1–10 was selected 64 times. 4–10 was selected 53 times.

[0387] R03: 4–8 was selected 82 times. 1–8 was selected 69 times.

[0388] R04: 4–8 was selected 124 times. 1–8 was selected 36 times.

[0389] R05: 4–8 was selected 153 times. 1–8 was selected 59 times.

[0390] As can be seen, at low bitrates (R01 and R02), the 4–10 combination (i.e., combinations of pixels with neighborhood differences falling within the range {4, 5, 6, 7, 8, 9, 10}) appears most frequently, while at bitrates R03 and above, the 4–8 combination is most frequently chosen. Based on these experimental results, this technique artificially sets the optimal combination to [4, 10] at R01 and R02 bitrates and [4, 8] at R03, R04, and R05 bitrates. This significantly reduces time loss while barely compromising filtering performance.

[0391] After obtaining pixels whose neighborhood differences fall within the optimal combination range, this technique uses the Wiener filter algorithm to filter these pixels (the remaining pixels are not processed, and their reconstructed pixel values ​​are retained). Specifically, the Wiener filter uses a diamond shape (as shown in Figure 7). In this technique, a filter with an order of k = 25 is used. This process yields the optimal coefficients for the filter and the resulting geometric video frame. A cost function is then used to balance the rate and distortion, i.e., J = D + λ × R.

[0392] Where D is the SSE of the original video frame and the reconstructed or filtered video frame, that is, the sum of squares of the corresponding point errors; λ is a quantity related to the quantization parameter QP. R is the total geometric bitstream size divided by the number of frames. If the filtered cost J f Less than the cost J of reconstructing the point set r , the filter flag of the frame is set to 1, and the group of coefficients and the best combination information are recorded; otherwise, the flag is set to 0.

[0393] Considering that each frame of point cloud will generate two frames of geometric video frames (far and near frames), this technology will filter these two frames separately, calculate the rate-distortion cost, and record information such as coefficients.

[0394] (3) Finally, the information that needs to be passed to the decoding end is written into the bitstream. This technology specifies the bitstream writing method (as shown in Figure 11). Specifically, since the total bitstream is composed of 5 sub-bitstreams, each sub-bitstream records the corresponding V3C_Unit information. There are already 5 bitstream units in V-PCC, namely: V3C_VPS, V3C_AD, V3C_OVD, V3C_GVD, V3C_AVD. That is, point cloud parameter set, Patch data group, occupancy map video data, geometric video data, attribute video data. At the same time, V-PCC reserves multiple unnamed bitstream units for subsequent supplementary use. This technology creates a new bitstream unit, namely V3C_WF, to store Wiener filter related information. Each unit consists of header information and payload. The header information of the self-created unit has been configured by V-PCC and does not need to be processed. In the payload (actual data), the following information is written here in sequence:

[0395] (a) Wiener filtering enable flag Wiener_filtering_flag. This flag occupies one bit and is used to indicate whether the V-PCC uses Wiener filtering; if so, it is set. The following situations are all written when Wiener_filtering_flag is set.

[0396] (b) Total frame number frame_num, occupies 32 bits.

[0397] (c) Group information: occupies 8 bits (records the beginning and ending group numbers of the group, each occupies 4 bits).

[0398] (d) Each frame filtering flag, filtering_flag_of_each_frame, occupies 1×frame_num bits.

[0399] (e) Each frame filter coefficient WFcoef (not written if a frame does not need filtering). It occupies a maximum of 32×k×frame_num bits.

[0400] For example, it is expressed in code as:

[0401] Among them, syntax is an object of the PCCHighLevelSyntax class. This technology will store various information in the object member variables after filtering so that it can be written into the code stream.

[0402] For example, in one implementation, the syntax structure of the encoding end is as shown in Table 1.

[0403] Table 1

[0404] In another specific embodiment, the decoding operation is as follows:

[0405] (1) During the decoding process, the valid information in V3C_WF is read into the variables of the syntax object. The corresponding variables can then be read in the Wiener filtering phase to obtain group information, per-frame flags, and filter coefficients. Note that if Wiener filtering is not enabled (Wiener_filtering_flag == 0), no further reading is performed. Similarly, if the flag for a frame is 0, the filter coefficients for that frame are not read.

[0406] (2) After obtaining the reconstructed geometric frame, each pixel is classified according to the neighborhood according to the method adopted by the encoding end. The pixels belonging to the best combination are selected and Wiener filtering is performed using the read coefficients to obtain the filtered video frame. Since the geometric reconstruction of the point cloud depends on the quality of the geometric frame, this post-processing can improve the geometric quality of the point cloud and achieve a denoising effect. At the same time, due to the reduction of the rate-distortion cost, this method will also improve the point cloud compression performance.

[0407] For example, in one implementation, the syntax structure of the encoding end is as shown in Table 2.

[0408] Table 2

[0409] For example, all standard test sequences were comprehensively tested on the latest version of V-PCC, TMC2v21.0, under conditions of loss of geometry and attributes, including 32-frame intra, 32-frame inter, and 300-frame intra. Among them, Table 3 shows the test results of 32-frame intra-frame_sequence under the case of geometric lossy and attribute lossy (C2_ai: lossy geonetry, lossy attributes), Table 4 shows the test results of 32-frame intra-frame_average under the case of geometric lossy and attribute lossy (C2_ai: lossy geonetry, lossy attributes), Table 5 shows the test results of 32-frame inter-frame_sequence under the case of geometric lossy and attribute lossy (C2_ai: lossy geonetry, lossy attributes), Table 6 shows the test results of 32-frame inter-frame_average under the case of geometric lossy and attribute lossy (C2_ai: lossy geonetry, lossy attributes), Table 7 shows the test results of 300-frame intra-frame_average under the case of geometric lossy and attribute lossy (C2_ai: lossy geonetry, lossy attributes), and Table 8 shows the test results of 300-frame intra-frame_sequence under the case of geometric lossy and attribute lossy (C2_ai: lossy geonetry, lossy attributes).

[0410] Table 3

[0411] Table 4

[0412] Table 5

[0413] Table 6

[0414] Table 7

[0415] Table 8

[0416] As can be seen from Tables 3 to 8, compared to the original program, the point cloud's PSNR is significantly improved after Wiener filtering with geometric video frame optimization, while compression efficiency is further increased and the geometric BD-Rate is significantly reduced. At the same time, the quality and BD-Rate of color attributes also show some improvement. This is achieved by better geometric guidance attribute reconstruction. Furthermore, in terms of time complexity, the time consumed by Wiener filtering is almost negligible (approximately 1% time increase) on both the encoding and decoding ends, which is a significant improvement compared to previous technologies. The performance gain remains almost unchanged, or even slightly improved, thereby improving encoding and decoding efficiency.

[0417] As can be seen from the table, compared to the original program, after the Wiener filter with geometric video frame optimization, the PSNR of the point cloud has been greatly improved, while the compression efficiency has been further increased, and the geometric BD-Rate has been significantly reduced. At the same time, the quality of the color attributes and the BD-Rate have also increased to a certain extent. This is achieved by better geometric guidance attribute reconstruction. At the same time, in terms of time complexity, the time consumed by Wiener filtering is almost negligible (approximately 1% time increase) on both the encoding and decoding ends. This is a significant improvement compared to previous technologies. The performance gain is almost unchanged, or even slightly improved, which reflects the effectiveness of this technology.

[0418] Furthermore, in the embodiment of the present application, the same method is adopted for filtering texture frames to enhance the attribute quality. In addition, the filtering information can also be written into the V3C_GVD unit and regarded as part of the geometry codec stream.

[0419] Furthermore, in the embodiment of the present application, the optimal combination can be determined for each sequence at each bit rate. Alternatively, two optimal combinations can be specified at each bit rate, and different optimal combinations are filtered separately, and the performance is compared, and the combination with the better performance is selected as the final combination.

[0420] This embodiment provides a V-PCC point cloud post-processing geometric quality enhancement technology that uses an optimized Wiener filtering algorithm. On the one hand, based on the bit rate, the optimal combination method for groups obtained based on neighborhood classification is determined in advance. The combination of [4, 10] is used at low bit rates, and the combination of [4, 8] is used at high bit rates. On the other hand, for pixels with a pixel value of 0, there is no need to calculate neighborhood differences. On the other hand, a new sub-stream unit, V3C_WF, is created in the V-PCC bitstream. Five types of information are written into this unit: the Wiener filter enable flag, the number of frames, the optimal combination information, the per-frame filter flag, and the per-frame optimal coefficient. On the other hand, compared to related technologies, this technology significantly reduces time complexity while ensuring unaffected performance. The bitstream writing / reading method is standardized, ensuring the stability of V-PCC operation and the rationality of its structure. This can improve the geometric reconstruction quality of the point cloud, thereby improving the encoding and decoding performance of the point cloud.

[0421] In another embodiment of the present application, based on the same inventive concept as the above embodiment, see Figure 15, which shows a schematic diagram of the composition structure of an encoder provided by an embodiment of the present application. As shown in Figure 15, the encoder 150 may include: a first determining unit 1501, a first filtering unit 1502 and an encoding unit 1503; wherein,

[0422] The first determining unit 1501 is configured to determine at least one pixel point in the first video image that meets a preset neighborhood condition; and determine a filter coefficient of the first video image based on the at least one pixel point;

[0423] A first filtering unit 1502 is configured to perform a first filtering process on at least one pixel point according to a filtering coefficient to determine a filtered image of the first video image;

[0424] The first determining unit 1501 is further configured to determine filtering identification information of the first video image based on the first video image and the filtered image;

[0425] The encoding unit 1503 is configured to encode the filter identification information and the corresponding filter coefficients of the first video image if the filter identification information indicates that the first video image uses the first filter processing, and write the obtained encoding bits into the bitstream.

[0426] In some embodiments, the first determining unit 1501 is further configured to determine whether a neighborhood difference value of a first pixel point in the first video image satisfies a preset neighborhood condition; wherein the first pixel point is any one of the at least one pixel point.

[0427] In some embodiments, the preset neighborhood condition is associated with the bit rate of the first video image.

[0428] In some embodiments, the first determination unit 1501 is further configured to perform neighborhood difference calculation based on the first pixel point in the first video image and at least one neighboring pixel point of the first pixel point to determine at least one absolute value of the difference; and perform statistical processing based on the at least one absolute value of the difference to determine the neighborhood difference value of the first pixel point in the first video image.

[0429] In some embodiments, the statistical processing includes at least one of the following: averaging processing, summing processing, and maximizing processing.

[0430] In some embodiments, the first determining unit 1501 is further configured to calculate an absolute value of a difference between a pixel value of the first pixel and a pixel value of at least one neighboring pixel to determine at least one absolute value of the difference.

[0431] In some embodiments, the first determining unit 1501 is further configured to determine category information of the first video image according to a preset neighborhood condition;

[0432] The encoding unit 1503 is further configured to encode the category information of the first video image and write the obtained encoding bits into the bitstream.

[0433] In some embodiments, when the preset neighborhood condition represents a preset range, the first determining unit 1501 is further configured to determine the start information and end information corresponding to the preset range according to the preset neighborhood condition.

[0434] In some embodiments, the first determining unit 1501 is further configured to determine that the first pixel in the first video image does not meet a preset neighborhood condition if the pixel value of the first pixel in the first video image is equal to 0.

[0435] In some embodiments, the first video image is a lossy reconstructed video image; the first determination unit 1501 is further configured to determine a filter correlation matrix based on the reconstructed pixel values ​​of K reference pixels corresponding to each of at least one pixel in the reconstructed video image; wherein K is an integer greater than 0; determine a filter correlation vector based on the original pixel value of at least one pixel in the original video image corresponding to the reconstructed video image; and determine a filter coefficient based on the filter correlation matrix and the filter correlation vector.

[0436] In some embodiments, the first determination unit 1501 is further configured to determine the mutual correlation matrix based on the filter correlation matrix and the filter correlation vector; determine the autocorrelation matrix based on the filter correlation matrix; and perform coefficient calculation based on the mutual correlation matrix and the autocorrelation matrix to determine the filter coefficients.

[0437] In some embodiments, the first determination unit 1501 is further configured to search for a preset number of candidate pixel points in the reconstructed video image based on the first pixel point in the reconstructed video image using a K-nearest neighbor search method; calculate the distance values ​​between the first pixel point and the preset number of candidate pixel points respectively, and select (K-1) distance values ​​from the obtained preset number of distance values, and the (K-1) distance values ​​are all less than or equal to the remaining distance values ​​in the preset number of distance values; determine (K-1) neighbor pixel points based on the candidate pixel points corresponding to the (K-1) distance values, and determine the first pixel point and the (K-1) neighbor pixel points as K reference pixel points corresponding to the first pixel point; wherein the first pixel point is any one of at least one pixel point in the reconstructed video image.

[0438] In some embodiments, the first filtering unit 1502 is further configured to perform Wiener filtering on the pixel value of at least one pixel point according to the filtering coefficient to determine the filtered pixel value of at least one pixel point; and determine the filtered image based on the filtered pixel value of at least one pixel point.

[0439] In some embodiments, the first filtering unit 1502 is further configured to use a preset window to scan each pixel point in at least one pixel point, perform weighted averaging on the pixel values ​​of all pixel points in the preset window, and update the pixel value of the pixel point located at the center of the preset window to the result of the weighted averaging to obtain the filtered pixel value of each pixel point; wherein a set of weighting coefficients used for the weighted averaging is a filtering coefficient, and the shape of the preset window is a rectangle or a diamond.

[0440] In some embodiments, the first video image is a lossy reconstructed video image; the first determination unit 1501 is further configured to determine a first generation value of the reconstructed video image relative to the original video image, and to determine a second generation value of the filtered image relative to the original video image; and to determine the filter identification information based on the first generation value and the second generation value.

[0441] In some embodiments, the first determination unit 1501 is further configured to determine that the value of the filter identification information is the first value if the second generation value is less than the first generation value; and to determine that the value of the filter identification information is the second value if the second generation value is greater than the first generation value.

[0442] In some embodiments, the first determination unit 1501 is further configured to determine a first performance gain value of the reconstructed video image relative to the original video image, and to determine a second performance gain value of the filtered image relative to the original video image; and to determine the filtering identification information based on the first performance gain value and the second performance gain value.

[0443] In some embodiments, the first determination unit 1501 is further configured to determine that the value of the filter identification information is the first value if the second performance gain value is greater than the first performance gain value; if the second performance gain value is less than the first performance gain value, determine that the value of the filter identification information is the second value.

[0444] In some embodiments, the first determination unit 1501 is further configured to determine that the value of the filter identification information is the first value if the second performance gain value is greater than the first performance gain value and the second generation value is less than the first generation value; if the second performance gain value is less than the first performance gain value, or the second generation value is greater than the first generation value, determine that the value of the filter identification information is the second value.

[0445] In some embodiments, the first determination unit 1501 is further configured to, if the value of the filter identification information is a first value, determine that the filter identification information indicates that the first video image uses the first filtering processing; if the value of the filter identification information is a second value, determine that the filter identification information indicates that the first video image does not use the first filtering processing.

[0446] In some embodiments, the first determination unit 1501 is further configured to determine at least two neighborhood conditions corresponding to the first video image; wherein the at least two neighborhood conditions include a first neighborhood condition and a second neighborhood condition; determine at least one pixel point in the first video image that meets the first neighborhood condition and a corresponding first filter coefficient, and perform a first filtering process based on the first filter coefficient and the at least one pixel point that meets the first neighborhood condition to determine a first filtered image; determine at least one pixel point in the first video image that meets the second neighborhood condition and a corresponding second filter coefficient, and perform a second filtering process based on the second filter coefficient and the at least one pixel point that meets the second neighborhood condition to determine a second filtered image; and determine a preset neighborhood condition from the first neighborhood condition and the second neighborhood condition based on the first filtered image and the second filtered image, including at least one of the following methods:

[0447] Determining, according to the performance gain values ​​of the first filtered image and the second filtered image relative to the first video image, a neighborhood condition corresponding to a maximum performance gain value as a preset neighborhood condition;

[0448] According to the cost values ​​of the first filtered image and the second filtered image relative to the first video image, a neighborhood condition corresponding to a minimum cost value is determined as a preset neighborhood condition.

[0449] In some embodiments, the encoding unit 1503 is further configured to encode the filtering identification information of the first video image and write the obtained encoding bits into the bitstream if the filtering identification information indicates that the first video image does not use the first filtering process.

[0450] In some embodiments, the first filtering process comprises a Wiener filtering process.

[0451] In some embodiments, the first determining unit 1501 is further configured to determine a filtering parameter of the video sequence;

[0452] The encoding unit 1503 is further configured to encode the filtering parameters of the video sequence and write the obtained encoding bits into the bitstream; wherein the filtering parameters include at least one of the following: filtering activation identification information, the total number of frames, category information, filtering identification information of each of at least one video image in the video sequence and the corresponding filtering coefficient, and the first video image is any one of the at least one video image.

[0453] In some embodiments, the encoding unit 1503 is further configured to store, based on the filtering parameters of the video sequence, filtering enablement identification information in a syntax object in the first bitstream unit; if the filtering enablement identification information indicates that the video sequence uses the first filtering process, then continue to store at least one of the total number of frames, category information, filtering identification information of at least one video image, and corresponding filtering coefficients in the syntax object of the first bitstream unit to obtain syntax object information of the first bitstream unit; and is further configured to encode the syntax object information in the first bitstream unit and write the obtained coded bits into the bitstream.

[0454] In some embodiments, the encoding unit 1503 is further configured to not continue the step of storing at least one of the total number of frames, category information, filter identification information of at least one video image, and corresponding filter coefficients in the syntax object of the first code stream unit if the filter enable identification information indicates that the video sequence does not use the first filtering process.

[0455] In some embodiments, the first video image comprises a reconstructed video image.

[0456] In some embodiments, the first video image includes a lossy reconstructed geometric video image, and the first filtering unit 1502 is further configured to perform quality enhancement processing on the corresponding texture video image based on the filtered reconstructed geometric video image after performing the first filtering processing on the reconstructed geometric video image.

[0457] In some embodiments, the first video image comprises a lossy reconstructed texture video image.

[0458] It is understandable that in the embodiments of the present application, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and of course it can also be a module, or it can be non-modular. Moreover, the various components in this embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional modules.

[0459] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, or the portion that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in this embodiment. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0460] Therefore, an embodiment of the present application provides a computer-readable storage medium, which is applied to the encoder 150. The computer-readable storage medium stores a computer program, and when the computer program is executed by the first processor, it implements the method described in any one of the aforementioned embodiments.

[0461] Based on the composition of the above-mentioned encoder 150 and the computer-readable storage medium, refer to Figure 16, which shows a specific hardware structure diagram of the encoder 150 provided in an embodiment of the present application. As shown in Figure 16, the encoder 150 may include: a first communication interface 1601, a first memory 1602 and a first processor 1603; each component is coupled together through a first bus system 1604. It can be understood that the first bus system 1604 is used to realize the connection and communication between these components. In addition to the data bus, the first bus system 1604 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as the first bus system 1604 in Figure 16. Among them,

[0462] The first communication interface 1601 is used to receive and send signals when sending and receiving information with other external network elements;

[0463] A first memory 1602 is used to store computer programs that can be run on the first processor 1603;

[0464] The first processor 1603 is configured to, when running the computer program, execute:

[0465] Determining at least one pixel point in the first video image that meets a preset neighborhood condition;

[0466] Determining a filter coefficient of the first video image according to at least one pixel point;

[0467] Performing a first filtering process on at least one pixel point according to a filtering coefficient to determine a filtered image of the first video image;

[0468] Determining filtering identification information of the first video image according to the first video image and the filtered image;

[0469] If the filter identification information indicates that the first video image uses the first filter processing, the filter identification information and the corresponding filter coefficients of the first video image are encoded, and the obtained encoding bits are written into the bitstream.

[0470] It is understood that the first memory 1602 in the embodiment of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The first memory 1602 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0471] The first processor 1603 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits or software instructions in the first processor 1603. The above-mentioned first processor 1603 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the first memory 1602 , and the first processor 1603 reads the information in the first memory 1602 and completes the steps of the above method in combination with its hardware.

[0472] It is to be understood that these embodiments described in the present application can be implemented with hardware, software, firmware, middleware, microcode or its combination.For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (Application Specific Integrated Circuits, ASIC), digital signal processor (Digital Signal Processing, DSP), digital signal processing equipment (DSP Device, DSPD), programmable logic device (Programmable Logic Device, PLD), field programmable gate array (Field-Programmable Gate Array, FPGA), general-purpose processor, controller, microcontroller, microprocessor, other electronic units for performing functions described in the present application or its combination.For software implementation, the technology described in the present application can be realized by the module (such as process, function etc.) that performs functions described in the present application. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0473] Optionally, as another embodiment, the first processor 1603 is further configured to execute the method described in any one of the aforementioned embodiments when running the computer program.

[0474] This embodiment provides an encoder, in which, since the preset neighborhood condition of the encoding end is pre-set according to the bit rate of the first video image, there is no need to perform multiple coefficient calculations to determine the optimal preset neighborhood condition, thereby reducing time complexity; in addition, after the encoding end determines at least one pixel point that meets the preset neighborhood condition from the first video image, it can also determine the corresponding filter coefficient based on the at least one pixel point; and only when the first video image needs to use the first filtering processing, the corresponding filter coefficient and the category information used to indicate the preset neighborhood condition are transmitted to the decoding end; thereby further reducing time complexity, and optimizing the reconstructed video image, which can improve the reconstructed geometric quality of the point cloud; at the same time, a standardized design is performed for the writing / reading method of the code stream, ensuring the stability of the operation of the encoding and decoding system and the rationality of the code stream structure; in this way, not only the quality of the reconstructed point cloud is enhanced, but also the quality of the video frame can be improved, further improving the compression efficiency and improving the encoding and decoding performance.

[0475] In another embodiment of the present application, based on the same inventive concept as the above embodiment, see FIG17 , which shows a schematic diagram of the structure of a decoder 170 provided in an embodiment of the present application. As shown in FIG17 , the decoder 170 may include: a decoding unit 1701 and a second filtering unit 1702; wherein,

[0476] The decoding unit 1701 is configured to decode the code stream and determine the filter identification information of the first video image;

[0477] The decoding unit 1701 is further configured to decode the code stream and determine the filter coefficient of the first video image if the filter identification information indicates that the first video image uses the first filter processing;

[0478] The second filtering unit 1702 is configured to perform a first filtering process on the at least one pixel point in the first video image according to the filtering coefficient when the at least one pixel point in the first video image meets the preset neighborhood condition, and determine a filtered image corresponding to the first video image.

[0479] In some embodiments, referring to FIG. 17 , the decoder 170 may further include a second determining unit 1703 configured to ensure that a neighborhood difference value of a first pixel point in the first video image satisfies a preset neighborhood condition; wherein the first pixel point is any one of at least one pixel point.

[0480] In some embodiments, the preset neighborhood condition is associated with the bit rate of the first video image.

[0481] In some embodiments, the second determination unit 1703 is further configured to perform neighborhood difference calculation based on the first pixel point in the first video image and at least one neighboring pixel point of the first pixel point to determine at least one absolute value of the difference; and perform statistical processing based on the at least one absolute value of the difference to determine the neighborhood difference value of the first pixel point in the first video image.

[0482] In some embodiments, the statistical processing includes at least one of the following: averaging processing, summing processing, and maximizing processing.

[0483] In some embodiments, the second determining unit 1703 is further configured to calculate an absolute value of a difference between a pixel value of the first pixel point and a pixel value of at least one neighboring pixel point to determine at least one absolute value of the difference.

[0484] In some embodiments, the decoding unit 1701 is further configured to decode the code stream to determine category information of the first video image; wherein the category information is used to indicate a preset neighborhood condition.

[0485] In some embodiments, when the preset neighborhood condition represents a preset range, the decoding unit 1701 is further configured to decode the code stream to determine the start information and end information corresponding to the preset range.

[0486] In some embodiments, the second determining unit 1703 is further configured to determine that the first pixel in the first video image does not meet a preset neighborhood condition if the pixel value of the first pixel in the first video image is equal to 0.

[0487] In some embodiments, the second filtering unit 1702 is further configured to not continue the steps of decoding the code stream and determining the filtering coefficients of the first video image if the filtering identification information indicates that the first video image does not use the first filtering process.

[0488] In some embodiments, the decoding unit 1701 is further configured to decode the code stream and determine the filtering parameters of the video sequence; wherein the filtering parameters include at least one of the following: filtering activation identification information, the total number of frames, category information, filtering identification information of at least one video image in the video sequence and the corresponding filtering coefficient, and the first video image is any one of the at least one video image.

[0489] In some embodiments, the decoding unit 1701 is further configured to decode the code stream and determine the syntax object information in the first code stream unit; read the filter activation identification information according to the syntax object information; if the filter activation identification information indicates that the video sequence uses the first filtering processing, then continue to read at least one of the total number of frames, category information, the filter identification information of at least one video image, and the corresponding filter coefficient according to the syntax object information.

[0490] In some embodiments, the decoding unit 1701 is further configured to not continue to execute the step of reading at least one of the total number of frames, category information, filter identification information of at least one video image, and corresponding filter coefficients according to the grammatical object information if the filter enable identification information indicates that the video sequence does not use the first filtering process.

[0491] In some embodiments, the second filtering unit 1702 is further configured to perform Wiener filtering on the pixel value of at least one pixel point according to the filtering coefficient to determine the filtered pixel value of at least one pixel point; and determine the filtered image based on the filtered pixel value of at least one pixel point.

[0492] In some embodiments, the second determination unit 1703 is further configured to determine K reference pixel points corresponding to a first pixel point among at least one pixel point; and perform Wiener filtering on the K reference pixel points corresponding to the first pixel point according to the filtering coefficient to determine the filtered pixel value of the first pixel point; wherein the first pixel point is any one of the at least one pixel point; K is an integer greater than 0, and the K reference pixel points include the first pixel point and (K-1) neighboring pixel points adjacent to the first pixel point.

[0493] In some embodiments, the first filtering process comprises a Wiener filtering process.

[0494] In some embodiments, the first video image comprises a reconstructed video image.

[0495] In some embodiments, the first video image includes a lossy reconstructed geometric video image, and the second filtering unit 1702 is further configured to perform quality enhancement processing on the corresponding texture video image based on the filtered reconstructed geometric video image after performing the first filtering processing on the reconstructed geometric video image.

[0496] In some embodiments, the first video image comprises a lossy reconstructed texture video image.

[0497] It is understood that in this embodiment, a "unit" can be a portion of a circuit, a portion of a processor, a portion of a program or software, etc., and can also be a module or a non-modular system. Furthermore, the various components in this embodiment can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit. The aforementioned integrated units can be implemented in the form of hardware or software functional modules.

[0498] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, this embodiment provides a computer-readable storage medium, which is applied to the decoder 170 and stores a computer program. When the computer program is executed by the second processor, it implements any of the methods in the aforementioned embodiments.

[0499] Based on the composition of the above-mentioned decoder 170 and the computer-readable storage medium, refer to Figure 18, which shows a specific hardware structure diagram of the decoder 170 provided in an embodiment of the present application. As shown in Figure 18, the decoder 170 may include: a second communication interface 1801, a second memory 1802 and a second processor 1803; each component is coupled together through a second bus system 1804. It can be understood that the second bus system 1804 is used to achieve connection and communication between these components. In addition to the data bus, the second bus system 1804 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, various buses are labeled as the second bus system 1804 in Figure 18. Among them,

[0500] The second communication interface 1801 is used to receive and send signals when sending and receiving information with other external network elements;

[0501] The second memory 1802 is used to store computer programs that can be run on the second processor 1803;

[0502] The second processor 1803 is configured to, when running the computer program, execute:

[0503] Decoding the code stream to determine filter identification information of the first video image;

[0504] If the filtering identification information indicates that the first video image is to be processed using a first filtering method, decoding the bitstream to determine a filtering coefficient for the first video image;

[0505] When at least one pixel point in the first video image meets a preset neighborhood condition, a first filtering process is performed on the at least one pixel point according to the filtering coefficient to determine a filtered image corresponding to the first video image.

[0506] Optionally, as another embodiment, the second processor 1803 is further configured to execute any one of the methods described in the foregoing embodiments when running the computer program.

[0507] It can be understood that the hardware functions of the second memory 1802 are similar to those of the first memory 1602, and the hardware functions of the second processor 1803 are similar to those of the first processor 1603; they will not be described in detail here.

[0508] This embodiment provides a decoder, in which the filter coefficients can be directly decoded to obtain the corresponding preset neighborhood conditions and then the filter coefficients can be used to perform a first filtering process on at least one pixel point that meets the preset neighborhood conditions; thereby not only reducing the time complexity, but also optimizing the reconstructed video image, and improving the reconstructed geometric quality of the point cloud; at the same time, the present application also standardizes the design of the writing / reading method of the code stream, ensuring the stability of the operation of the encoding and decoding system and the rationality of the code stream structure; in this way, not only the quality of the reconstructed point cloud is enhanced, but also the quality of the video frame can be improved, further improving the compression efficiency and improving the encoding and decoding performance.

[0509] In yet another embodiment of the present application, referring to FIG19 , a schematic diagram of the structure of a coding and decoding system provided by an embodiment of the present application is shown. As shown in FIG19 , the coding and decoding system 190 may include an encoder 1901 and a decoder 1902 .

[0510] In the embodiment of the present application, the encoder 1901 may be the encoder described in any one of the aforementioned embodiments, and the decoder 1902 may be the decoder described in any one of the aforementioned embodiments.

[0511] It should be noted that, in this application, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.

[0512] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0513] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0514] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0515] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0516] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims. Industrial Applicability

[0517] In an embodiment of the present application, at the encoding end, after determining at least one pixel point that meets a preset neighborhood condition based on the first video image, a filter coefficient of the first video image is determined based on the at least one pixel point; then, a first filtering process is performed on the at least one pixel point based on the filtering coefficient to determine a filtered image of the first video image; and based on the first video image and the filtered image, the filter identification information of the first video image is determined; if the filtering identification information indicates that the first video image uses the first filtering process, the filtering identification information of the first video image and the corresponding filtering coefficient are encoded, and the obtained encoding bits are written into the bitstream. At the decoding end, the bitstream is decoded to determine the filtering identification information of the first video image; if the filtering identification information indicates that the first video image uses the first filtering process, the bitstream is decoded to determine the filtering coefficient of the first video image; when at least one pixel point in the first video image meets the preset neighborhood condition, a first filtering process is performed on the at least one pixel point based on the filtering coefficient to determine the filtered image corresponding to the first video image. In this way, since the preset neighborhood condition at the encoding end is pre-set based on the bit rate of the first video image, there is no need to perform multiple coefficient calculations to determine the optimal preset neighborhood condition, thereby reducing time complexity. In addition, after the encoding end determines at least one pixel point that meets the preset neighborhood condition from the first video image, it can also determine the corresponding filter coefficient based on the at least one pixel point. And only when the first video image needs to be processed by the first filtering, the corresponding filter coefficient and the category information indicating the preset neighborhood condition are transmitted to the decoding end. Accordingly, the decoding end can directly decode to obtain the filter coefficient and determine the corresponding preset neighborhood condition, and then use the filter coefficient to perform the first filtering process on the at least one pixel point that meets the preset neighborhood condition. This not only reduces time complexity, but also optimizes the reconstructed video image and improves the reconstructed geometric quality of the point cloud. At the same time, the present application also standardizes the design of the writing / reading method of the code stream to ensure the stability of the codec system operation and the rationality of the code stream structure. In this way, not only is the quality of the reconstructed point cloud enhanced, but the quality of the video frame can also be improved, further improving compression efficiency and improving codec performance.

Claims

1. A decoding method, applied to a decoder, the method include: Decoding the bitstream to determine filter identification information of the first video image; If the filter identification information indicates that the first video image is processed using a first filter, decoding the bitstream to determine a filter coefficient of the first video image; When at least one pixel point in the first video image meets a preset neighborhood condition, a first filtering process is performed on the at least one pixel point according to the filtering coefficient to determine a filtered image corresponding to the first video image.

2. The method according to claim 1, in, At least one pixel point in the first video image satisfies a preset neighborhood condition, including: A neighborhood difference value of a first pixel point in the first video image satisfies the preset neighborhood condition; wherein the first pixel point is any one of the at least one pixel point.

3. The method according to claim 1, in, The preset neighborhood condition is associated with a bit rate of the first video image.

4. The method according to claim 2, in, The method further comprises: Performing neighborhood difference calculation based on a first pixel point in the first video image and at least one neighboring pixel point of the first pixel point to determine at least one absolute value of the difference; Statistical processing is performed according to the at least one absolute difference value to determine a neighborhood difference value of a first pixel in the first video image.

5. The method according to claim 4, in, The statistical processing includes at least one of the following: Mean processing, sum processing and maximum processing.

6. The method according to claim 4, in, The performing neighborhood difference calculation based on a first pixel point in the first video image and at least one neighboring pixel point of the first pixel point to determine at least one absolute value of the difference includes: The at least one difference absolute value is determined by calculating the absolute value of the difference between the pixel value of the first pixel point and the pixel value of the at least one neighborhood pixel point.

7. The method according to claim 2, in, The method further comprises: Decode the code stream to determine category information of the first video image; wherein the category information is used to indicate the preset neighborhood condition.

8. The method according to claim 7, in, When the preset neighborhood condition represents a preset range, the decoding bitstream determines the category information of the first video image, including: The code stream is decoded to determine the start information and the end information corresponding to the preset range.

9. The method according to claim 1, in, The method further comprises: If the pixel value of the first pixel in the first video image is equal to 0, it is determined that the first pixel in the first video image does not meet the preset neighborhood condition.

10. The method according to claim 1, in, The method further comprises: If the filter identification information indicates that the first video image does not use the first filter processing, the step of decoding the code stream and determining the filter coefficient of the first video image is not continued.

11. The method according to any one of claims 1 to 10, in, The method further comprises: Decoding the bitstream and determining filtering parameters of the video sequence; wherein the filtering parameters include at least one of the following: Filter activation identification information, total number of frames, category information, filter identification information of at least one video image in the video sequence and corresponding filter coefficients, and the first video image is any one of the at least one video image.

12. The method according to claim 11, in, The decoding bit stream determines the filtering parameters of the video sequence, including: Decoding the code stream to determine the syntax object information in the first code stream unit; The filter activation identification information is read according to the grammatical object information. If the filter activation identification information indicates that the video sequence uses the first filtering process, the total number of frames, the category information, the filter identification information of each of the at least one video image, and at least one of the corresponding filter coefficients are continued to be read according to the grammatical object information.

13. The method according to claim 12, in, The method further comprises: If the filter activation identification information indicates that the video sequence does not use the first filtering process, the step of reading at least one of the total number of frames, the category information, the filter identification information of each of the at least one video image, and the corresponding filter coefficient according to the grammatical object information will not be continued.

14. The method according to claim 1, in, The performing a first filtering process on the at least one pixel point according to the filtering coefficient to determine a filtered image corresponding to the first video image includes: The pixel value of the at least one pixel point is subjected to Wiener filtering according to the filtering coefficient to determine the pixel value of the at least one pixel point. Filter pixel values; The filtered image is determined according to the filtered pixel value of the at least one pixel point.

15. The method according to claim 14, in, The performing Wiener filtering on the pixel value of the at least one pixel point according to the filtering coefficient to determine the filtered pixel value of the at least one pixel point includes: Determine K reference pixel points corresponding to a first pixel point among the at least one pixel point; Performing Wiener filtering on K reference pixels corresponding to the first pixel according to the filtering coefficient to determine a filtered pixel value of the first pixel; The first pixel point is any one of the at least one pixel point; K is an integer greater than 0, and the K reference pixel points include the first pixel point and (K-1) neighboring pixel points adjacent to the first pixel point.

16. The method according to any one of claims 1 to 15, in, The first filtering process includes a Wiener filtering process.

17. The method according to any one of claims 1 to 15, in, The first video image includes a reconstructed video image.

18. The method according to claim 17, in, The first video image comprises a lossy reconstructed geometric video image, and the method further comprises: After performing a first filtering process on the reconstructed geometric video image, performing a quality enhancement process on the corresponding texture video image according to the filtered reconstructed geometric video image; or, The first video image comprises a lossy reconstructed texture video image.

19. A coding method, applied to an encoder, the method include: Determine at least one pixel point in the first video image that meets a preset neighborhood condition; Determining a filter coefficient of the first video image according to the at least one pixel point; Performing a first filtering process on the at least one pixel point according to the filtering coefficient to determine a filtered image of the first video image; Determining filtering identification information of the first video image according to the first video image and the filtered image; If the filter identification information indicates that the first video image uses the first filter processing, the filter identification information and the corresponding filter coefficients of the first video image are encoded, and the obtained encoding bits are written into the bitstream.

20. The method according to claim 19, in, At least one pixel point in the first video image satisfies a preset neighborhood condition, including: A neighborhood difference value of a first pixel point in the first video image satisfies the preset neighborhood condition; wherein the first pixel point is any one of the at least one pixel point.

21. The method according to claim 19, in, The preset neighborhood condition is associated with a bit rate of the first video image.

22. The method according to claim 20, in, The method further comprises: Performing neighborhood difference calculation based on a first pixel point in the first video image and at least one neighboring pixel point of the first pixel point to determine at least one absolute value of the difference; Statistical processing is performed according to the at least one absolute difference value to determine a neighborhood difference value of a first pixel in the first video image.

23. The method according to claim 22, in, The statistical processing includes at least one of the following: Mean processing, sum processing and maximum processing.

24. The method according to claim 22, in, The performing neighborhood difference calculation based on a first pixel point in the first video image and at least one neighboring pixel point of the first pixel point to determine at least one absolute value of the difference includes: The at least one difference absolute value is determined by calculating the absolute value of the difference between the pixel value of the first pixel point and the pixel value of the at least one neighborhood pixel point.

25. The method according to claim 19, in, The method further comprises: Determining category information of the first video image according to the preset neighborhood condition; The category information of the first video image is encoded, and the obtained encoding bits are written into a bit stream.

26. The method according to claim 25, in, When the preset neighborhood condition represents a preset range, determining the category information of the first video image according to the preset neighborhood condition includes: According to the preset neighborhood condition, the start information and the end information corresponding to the preset range are determined.

27. The method according to claim 19, in, The method further comprises: If the pixel value of the first pixel in the first video image is equal to 0, it is determined that the first pixel in the first video image does not meet the preset neighborhood condition.

28. The method according to claim 19, in, The first video image is a lossy reconstructed video image; and determining a filter coefficient of the first video image according to the at least one pixel point includes: Determine a filtering correlation matrix according to the reconstructed pixel values ​​of K reference pixels corresponding to each of the at least one pixel in the reconstructed video image; wherein K is an integer greater than 0; Determining a filtering correlation vector according to an original pixel value of at least one pixel point in an original video image corresponding to the reconstructed video image; The filter coefficients are determined according to the filter correlation matrix and the filter correlation vector.

29. The method according to claim 28, in, The determining the filter coefficient according to the filter correlation matrix and the filter correlation vector comprises: Determining a cross-correlation matrix according to the filter correlation matrix and the filter correlation vector; determining an autocorrelation matrix based on the filtered correlation matrix; Coefficient calculation is performed according to the mutual correlation matrix and the autocorrelation matrix to determine the filter coefficient.

30. The method according to claim 28, in, The method further comprises: Based on the first pixel point in the reconstructed video image, searching for a preset number of candidate pixel points in the reconstructed video image using a K nearest neighbor search method; Calculating the distance values ​​between the first pixel and the preset number of candidate pixel points respectively, selecting (K-1) distance values ​​from the obtained preset number of distance values, and the (K-1) distance values ​​are all less than or equal to the remaining distance values ​​in the preset number of distance values; Determine (K-1) neighboring pixel points according to the candidate pixel points corresponding to the (K-1) distance values, and determine the first pixel point and the (K-1) neighboring pixel points as K reference pixel points corresponding to the first pixel point; The first pixel point is any one of the at least one pixel point in the reconstructed video image.

31. The method according to claim 19, in, The performing a first filtering process on the at least one pixel point according to the filtering coefficient to determine a filtered image of the first video image includes: Performing Wiener filtering on the pixel value of the at least one pixel point according to the filtering coefficient to determine a filtered pixel value of the at least one pixel point; The filtered image is determined according to the filtered pixel value of the at least one pixel point.

32. The method according to claim 31, in, The performing Wiener filtering on the pixel value of the at least one pixel point according to the filtering coefficient to determine the filtered pixel value of the at least one pixel point includes: Use a preset window to scan each pixel point of the at least one pixel point, perform weighted averaging on the pixel values ​​of all the pixel points in the preset window, and update the pixel value of the pixel point located at the center of the preset window to the result of the weighted averaging to obtain the filtered pixel value of each pixel point; wherein a set of weighting coefficients used in the weighted averaging is the filtering coefficient, and the shape of the preset window is a rectangle or a diamond.

33. The method according to claim 19, in, The first video image is a lossy reconstructed video image; and determining the filtering identification information of the first video image according to the first video image and the filtered image includes: Determining a first generation value of the reconstructed video image relative to the original video image, and determining a second generation value of the filtered image relative to the original video image; The filtering identification information is determined according to the first generation value and the second generation value.

34. The method according to claim 33, in, The determining the filtering identification information according to the first generation value and the second generation value includes: If the second generation value is less than the first generation value, determining the value of the filtering identification information to be the first value; If the second generation value is greater than the first generation value, the value of the filtering identification information is determined to be the second value.

35. The method according to claim 33, in, The determining, according to the first video image and the filtered image, filtering identification information of the first video image includes: Determining a first performance gain value of the reconstructed video image relative to the original video image, and determining a second performance gain value of the filtered image relative to the original video image; The filter identification information is determined according to the first performance gain value and the second performance gain value.

36. The method according to claim 35, in, The determining the filtering identification information according to the first performance gain value and the second performance gain value includes: If the second performance gain value is greater than the first performance gain value, determining that the value of the filtering identification information is the first value; If the second performance gain value is less than the first performance gain value, the value of the filtering identification information is determined to be a second value.

37. The method according to claim 35, in, The determining the filtering identification information according to the first performance gain value and the second performance gain value includes: If the second performance gain value is greater than the first performance gain value and the second generation value is less than the first generation value, determining that the value of the filtering identification information is the first value; If the second performance gain value is smaller than the first performance gain value, or the second generation value is larger than the first generation value, then the value of the filtering identification information is determined to be a second value.

38. The method according to claim 34, 36 or 37, in, The method further comprises: If the value of the filter identification information is a first value, determining that the filter identification information indicates that the first video image uses a first filter process; If the value of the filter identification information is the second value, it is determined that the filter identification information indicates that the first video image does not use the first filter processing.

39. The method according to claim 19, in, The method further comprises: Determine at least two neighborhood conditions corresponding to the first video image; wherein the at least two neighborhood conditions include a first neighborhood condition and a second neighborhood condition; Determine at least one pixel point satisfying a first neighborhood condition and a corresponding first filter coefficient in the first video image, and perform a first filtering process according to the first filter coefficient and the at least one pixel point satisfying the first neighborhood condition to determine a first filtered image; Determine at least one pixel point in the first video image that meets a second neighborhood condition and a corresponding second filter coefficient, and perform a second filtering process according to the second filter coefficient and the at least one pixel point that meets the second neighborhood condition to determine a second filtered image; Determining the preset neighborhood condition from the first neighborhood condition and the second neighborhood condition according to the first filtered image and the second filtered image includes at least one of the following methods: According to the performance gain values ​​of the first filtered image and the second filtered image relative to the first video image, the neighborhood condition corresponding to the maximum performance gain value is determined as the preset neighborhood condition; According to the cost values ​​of the first filtered image and the second filtered image relative to the first video image, a neighborhood condition corresponding to a minimum cost value is determined as the preset neighborhood condition.

40. The method according to claim 19, in, The method further comprises: If the filter identification information indicates that the first video image does not use the first filter processing, the filter identification information of the first video image is encoded, and the obtained encoding bits are written into the bitstream.

41. The method according to any one of claims 19 to 40, in, The first filtering process includes a Wiener filtering process.

42. The method according to any one of claims 19 to 40, in, The method further comprises: determining filtering parameters of a video sequence; Encoding the filtering parameters of the video sequence and writing the obtained coded bits into a bitstream; The filtering parameters include at least one of the following: Filter activation identification information, total number of frames, category information, filter identification information of at least one video image in the video sequence and corresponding filter coefficients, and the first video image is any one of the at least one video image.

43. The method according to claim 42, in, The bitstream includes a first bitstream unit; the step of encoding the filtering parameters of the video sequence and writing the obtained coded bits into the bitstream includes: According to the filtering parameters of the video sequence, the filtering enabling identification information is stored in a syntax object in the first code stream unit; if the filtering enabling identification information indicates that the video sequence uses the first filtering process, at least one of the total number of frames, the category information, the filtering identification information of each of the at least one video image, and the corresponding filtering coefficient is continuously stored in the syntax object of the first code stream unit to obtain the syntax object information of the first code stream unit; The syntax object information in the first code stream unit is encoded, and the obtained encoded bits are written into the code stream.

44. The method according to claim 43, in, The method further comprises: If the filter activation identification information indicates that the video sequence does not use the first filtering process, the step of storing at least one of the total number of frames, the category information, the filter identification information of each of the at least one video image, and the corresponding filter coefficient in the syntax object of the first code stream unit is not continued.

45. The method according to any one of claims 19 to 44, in, The first video image includes a reconstructed video image.

46. ​​The method according to claim 45, in, The first video image comprises a lossy reconstructed geometric video image, and the method further comprises: After performing a first filtering process on the reconstructed geometric video image, performing a quality enhancement process on the corresponding texture video image according to the filtered reconstructed geometric video image; or, The first video image comprises a lossy reconstructed texture video image.

47. A code stream, the code stream is generated by bit encoding according to information to be encoded; in, The information to be encoded includes at least one of the following: Filter activation identification information, total number of frames, category information, filter identification information of at least one video image and corresponding filter coefficients; The filter enable identification information is used to indicate whether the video sequence uses the first filter processing, the filter identification information is used to indicate whether the first video image uses the first filter processing, the video sequence includes at least one video image, and the first video image is any one of the at least one video image.

48. The code stream according to claim 47, in, The code stream includes: The first code stream unit is configured to store filtering parameters of the video sequence; wherein the filtering parameters include at least one of the following: the filtering activation identification information, the total number of frames, the category information, the filtering identification information of each of the at least one video image, and the corresponding filtering coefficient.

49. The code stream according to claim 48, in, The first code stream unit is composed of header information and payload, and the payload includes: A first syntax object, configured to store the filtering enabling identification information; A second syntax object is configured to store the total number of frames when the filter enabling identification information indicates that the video sequence uses a first filter process; A third syntax object is configured to store the category information when the filter enabling identification information indicates that the video sequence uses a first filter process; A fourth syntax object is configured to store filter identification information of each of the at least one video image when the filter enabling identification information indicates that the video sequence uses a first filter process; The fifth syntax object is configured to store the filter coefficient corresponding to the at least one video image when the filter identification information indicates that the at least one video image uses the first filter processing.

50. An encoder, comprising a first determining unit, a first filtering unit and an encoding unit; in, The first determining unit is configured to determine at least one pixel point in the first video image that meets a preset neighborhood condition; and determine a filter coefficient of the first video image according to the at least one pixel point; The first filtering unit is configured to perform a first filtering process on the at least one pixel point according to the filtering coefficient to determine a filtered image of the first video image; The first determining unit is further configured to determine filtering identification information of the first video image according to the first video image and the filtered image; The encoding unit is configured to encode the filter identification information and the corresponding filter coefficient of the first video image if the filter identification information indicates that the first video image uses the first filter processing, and write the obtained encoding bits into the bit stream.

51. An encoder comprising a first memory and a first processor; in, The first memory is used to store a computer program that can be run on the first processor; The first processor is configured to execute the method according to any one of claims 19 to 46 when running the computer program.

52. A decoder, the decoder comprising a decoding unit and a second filtering unit; in, The decoding unit is configured to decode the code stream and determine the filter identification information of the first video image; The decoding unit is further configured to decode the bitstream and determine the filter coefficient of the first video image if the filter identification information indicates that the first video image uses the first filter processing; The second filtering unit is configured to perform a first filtering process on the at least one pixel point in the first video image according to the filtering coefficient when the at least one pixel point in the first video image meets a preset neighborhood condition, so as to determine a filtered image corresponding to the first video image.

53. A decoder comprising a second memory and a second processor; in, The second memory is used to store a computer program that can be run on the second processor; The second processor is configured to execute the method according to any one of claims 1 to 18 when running the computer program.

54. A computer readable storage medium, in, The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 18 is implemented, or the method according to any one of claims 19 to 46 is implemented.

Images