Information processing method and apparatus, device, and storage medium

Abstract

Embodiments of the present application disclose information processing methods and apparatuses, devices and storage media, wherein the method comprises: determining an original pixel set of a to-be-encoded block in a current video frame; determining attribute information of each original pixel in the original pixel set; selecting original pixels whose attribute information satisfies a screening condition to obtain a target pixel set; and performing encoding processing on the to-be-encoded block based on the attribute information of each original pixel in the target pixel set.

Description

[0001] Case Analysis

[0002] This application is a divisional application of Chinese patent application No. 201980057448.8, entitled "Information Processing Method, Apparatus, Device, Storage Medium", which entered the Chinese national phase of PCT international patent application PCT / CN2019 / 087300, filed on May 16, 2019. Technical Field

[0003] This application relates to video encoding and decoding technology, and to, but is not limited to, information processing methods, apparatus, devices, and storage media. Background Technology

[0004] In recent years, video services have developed rapidly in the fields of the Internet and mobile communications. Video services require encoding the source video data first, and then transmitting the encoded video data to the user terminal through the channels of the Internet or mobile communication network.

[0005] For users, video smoothness directly impacts their viewing experience. And the computational complexity of predictive coding in video encoding directly affects video smoothness. Summary of the Invention

[0006] In view of this, embodiments of this application provide information processing methods, apparatus, devices, and storage media to solve at least one problem existing in the related art.

[0007] The technical solution of this application embodiment is implemented as follows:

[0008] In a first aspect, embodiments of this application provide an information processing method, the method comprising: determining the original pixel set of a block to be encoded in a current video frame; and determining the attribute information of each original pixel in the original pixel set;

[0009] Select original pixels whose attribute information meets the filtering conditions to obtain a target pixel set; based on the attribute information of each original pixel in the target pixel set, encode the block to be encoded.

[0010] Secondly, embodiments of this application provide an information processing apparatus, comprising: an original pixel set determination module configured to determine the original pixel set of a block to be encoded in a current video frame; an attribute information determination module configured to determine the attribute information of each original pixel in the original pixel set; a target pixel selection module configured to select original pixels whose attribute information satisfies the filtering conditions to obtain a target pixel set; and an encoding processing module configured to perform encoding processing on the block to be encoded based on the attribute information of each original pixel in the target pixel set.

[0011] Thirdly, embodiments of this application provide an electronic device, including a memory and a processor. The memory stores a computer program that can run on the processor, and the processor executes the program to implement the steps in the above-mentioned information processing.

[0012] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described information processing method.

[0013] In this embodiment, original pixels whose attribute information meets the filtering conditions are selected from the original pixel set of the block to be encoded to obtain a target pixel set; the block to be encoded is encoded based on the attribute information of each original pixel in the target pixel set; in this way, the block to be encoded is encoded based on the attribute information of a selected portion of the original pixels (that is, the original pixels in the target pixel set), rather than based on the attribute information of all the original pixels in the original pixel set. This reduces the computational complexity of the encoding process, thereby improving video encoding efficiency and thus improving the smoothness of the video for the user. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the network architecture for video encoding and decoding in an embodiment of this application;

[0015] Figure 2A This is a schematic diagram of the composition structure of the video encoder in an embodiment of this application;

[0016] Figure 2B This is a schematic diagram of the composition structure of the video decoder according to an embodiment of this application;

[0017] Figure 3A This is a schematic diagram illustrating the implementation flow of the information processing method in an embodiment of this application;

[0018] Figure 3B This is a schematic diagram illustrating the implementation flow of another information processing method according to an embodiment of this application;

[0019] Figure 3C This is a schematic diagram illustrating the relationship between the block to be encoded and the reference pixel in an embodiment of this application;

[0020] Figure 3D This is a schematic diagram illustrating the implementation flow of another information processing method according to an embodiment of this application;

[0021] Figure 4 This is a schematic diagram illustrating the implementation flow of another information processing method according to an embodiment of this application;

[0022] Figure 5A This is a schematic diagram illustrating the implementation flow of another information processing method according to an embodiment of this application;

[0023] Figure 5B A schematic diagram illustrating the implementation flow of the method for constructing a prediction model in the embodiments of this application;

[0024] Figure 6A This is a schematic diagram of the composition structure of the information processing device according to an embodiment of this application;

[0025] Figure 6B This is a schematic diagram of the composition of another information processing device according to an embodiment of this application;

[0026] Figure 6C This is a schematic diagram of the composition structure of another information processing device according to an embodiment of this application;

[0027] Figure 6D This is a schematic diagram of the composition structure of another information processing device according to an embodiment of this application;

[0028] Figure 7 This is a schematic diagram of a hardware entity of an electronic device according to an embodiment of this application. Detailed Implementation

[0029] This embodiment first provides a network architecture for video encoding and decoding. Figure 1 This is a schematic diagram of the network architecture for video encoding and decoding in an embodiment of this application, as shown below. Figure 1 As shown, the network architecture includes one or more electronic devices 11 to 1N and a communication network 01, wherein the electronic devices 11 to 1N can perform video interaction through the communication network 01. The electronic devices can be various types of devices with video encoding and decoding capabilities, such as mobile phones, tablets, personal computers, personal digital assistants, navigators, digital phones, video phones, televisions, sensing devices, servers, etc.

[0030] The electronic device has video encoding and decoding capabilities, and generally includes a video encoder and a video decoder, for example, see [link to relevant documentation]. Figure 2AAs shown, the video encoder 21 comprises the following structures: a transform and quantization unit 211, an intra-frame estimation unit 212, an intra-frame prediction unit 213, a motion compensation unit 214, a motion estimation unit 215, an inverse transform and inverse quantization unit 216, a filter control and analysis unit 217, a filtering unit 218, an encoding unit 219, and a decoded image buffer unit 210. The filtering unit 218 can implement deblocking filtering and Sample Adaptive Offset (SAO) filtering, while the encoding unit 219 can implement header information encoding and Context-based Adaptive Binary Arithmetic Coding (CABAC). For the input source video data, the encoder uses coding tree blocks (Coding Tree Blocks) to perform the encoding of the input source video data. The partitioning of a TreeUnit (CTU) yields a block to be encoded in the current video frame. After performing intra-frame prediction or inter-frame prediction on this block, the resulting residual information is transformed by the transform and quantization unit 211. This transformation includes converting the residual information from the pixel domain to the transform domain and quantizing the resulting transform coefficients to further reduce the bit rate. Intra-frame estimation unit 212 and intra-frame prediction unit 213 perform intra-frame prediction on the block to be encoded, for example, determining the intra-frame prediction mode for encoding the block. Motion compensation unit 214 and motion estimation unit 215 perform inter-frame prediction coding of the block to be encoded relative to one or more blocks in one or more reference frames to provide temporal prediction information. The motion estimation unit 215 estimates motion vectors, which can estimate the motion of the block to be encoded, and then the motion compensation unit 214 performs motion compensation based on these motion vectors. After determining the intra-frame prediction mode, the intra-frame prediction... The prediction unit 213 is also used to provide the selected intra-frame prediction data to the coding unit 219, and the motion estimation unit 215 also sends the calculated motion vector data to the coding unit 219. Furthermore, the inverse transform and inverse quantization unit 216 is used to reconstruct the block to be encoded, reconstructing a residual block in the pixel domain. This reconstructed residual block is then processed by the filter control analysis unit 217 and the filtering unit 218 to remove block artifacts. The reconstructed residual block is then added to a predictive block in the frame of the decoding image buffer unit 210 to generate a reconstructed video coding block. The coding unit 219 is used to encode various coding parameters and quantized transform coefficients. In the CABAC-based coding algorithm, the context content can be based on adjacent coding blocks and can be used to encode information indicating the determined intra-frame prediction mode, outputting the bitstream of the video data. The decoding image buffer unit 210 is used to store the reconstructed video coding blocks for prediction reference. As video coding progresses, new reconstructed video coding blocks are continuously generated, and these reconstructed video coding blocks are stored in the decoding image buffer unit 210.

[0031] The video decoder 22 corresponding to the video encoder 21 has the following structure: Figure 2B As shown, it includes: a decoding unit 221, an inverse transform and inverse quantization unit 222, an intra-frame prediction unit 223, a motion compensation unit 224, a filtering unit 225, and a decoded image buffer unit 226, etc. The decoding unit 221 can perform header information decoding and CABAC decoding, and the filtering unit 225 can perform deblocking filtering and SAO filtering. The input video signal is processed... Figure 2A After encoding, the video signal bitstream is output; this bitstream is input to the video decoder 22, first passing through the decoding unit 221 to obtain the decoded transform coefficients; these transform coefficients are then processed by the inverse transform and inverse quantization unit 222 to generate residual blocks in the pixel domain; the intra-frame prediction unit 223 can be used to generate prediction data for the current decoded block based on the determined intra-frame prediction mode and data from previously decoded blocks in the current frame or image; the motion compensation unit 224 determines the prediction information for the current decoded block by analyzing motion vectors and other associated syntax elements, and uses this information... Predictive information is used to generate a predictive block for the current decoded block being decoded; a decoded video block is formed by summing the residual block from the inverse transform and inverse quantization unit 222 with the corresponding predictive block generated by the intra-prediction unit 223 or the motion compensation unit 224; the decoded video block is passed through the filtering unit 225 to remove block artifacts, thereby improving video quality; then the decoded video block is stored in the decoded image buffer unit 226, which stores reference images for subsequent intra-prediction or motion compensation, and is also used for the output display of the video signal.

[0032] Based on this, the technical solution of this application will be further described in detail below with reference to the accompanying drawings and embodiments. The information processing method provided in the embodiments of this application can be applied to both the video encoder 21 and the video decoder 22, and the embodiments of this application do not specifically limit it in this regard.

[0033] This application provides an information processing method applied to an electronic device. The function implemented by this method can be achieved by the processor in the electronic device calling program code. Of course, the program code can be stored in a computer storage medium. It can be seen that the electronic device includes at least a processor and a storage medium.

[0034] Figure 3A This is a schematic diagram illustrating the implementation flow of the information processing method in an embodiment of this application, as shown below. Figure 3A As shown, the method includes steps S301 to S304:

[0035] S301. Determine the original pixel set of the block to be encoded in the current video frame;

[0036] It should be noted that the block to be encoded refers to the image region in the current video frame that needs to be encoded. Different encoding processes result in different types of blocks to be encoded, and the contents of the original pixel set also differ. For example, when the encoding process is predictive coding, the block to be encoded is an image block including luma and chroma components. The reconstructed pixels outside the block to be encoded can be determined as the original pixel set, that is, the reference pixels used when performing predictive coding on the block to be encoded. However, the contents of the original pixel set are not limited; this is just an example. When the encoding process is transform coding, the block to be encoded is an image block including residual values, that is, the residual block output after predictive coding. The pixels in the residual block can be determined as the original pixel set, but the contents of the original pixel set are not limited; this is just an example. When the encoding process is entropy coding, the block to be encoded is an image block including transform coefficients, that is, the coefficient block output after transform coding. The pixels in the coefficient block can be determined as the original pixel set, but the contents of the original pixel set are not limited; this is just an example.

[0037] S302. Determine the attribute information of each original pixel in the original pixel set;

[0038] Understandably, when the original pixel set is the reconstructed pixels outside the block to be encoded, the attribute information of each original pixel in the original pixel set is the position information, luminance component, chrominance component, etc. of the reconstructed pixel; when the original pixel set is the pixels in the residual block, the attribute information of each original pixel in the original pixel set is the position information, residual value, etc. of the pixels in the residual block; when the original pixel set is the pixels in the coefficient block, the attribute information of each original pixel in the original pixel set is the position information, transform coefficient, etc. of some pixels in the coefficient block.

[0039] S303. Select the original pixels whose attribute information meets the filtering conditions to obtain the target pixel set;

[0040] S304. Based on the attribute information of each original pixel in the target pixel set, the block to be encoded is encoded.

[0041] It should be noted that the type of encoding process is not limited here. For example, the encoding process can be one of predictive coding, transform coding, and entropy coding.

[0042] In this embodiment of the application, before encoding the block to be encoded, a portion of the original pixels (i.e. the target pixel set) are selected from the original pixel set of the block to be encoded. Then, based on the attribute information of the portion of the original pixels (rather than all the original pixels in the original pixel set), the block to be encoded is encoded, thereby reducing the complexity of the encoding process, reducing the time spent on the encoding process, and thus improving the smoothness of the user watching the video.

[0043] It should be noted that the information processing method provided in this application is actually an improvement on traditional video coding techniques (such as video coding standards H.264, H.265, etc.). Specifically, it adds the concept of a subset to the traditional video coding technique. For example, before predictive coding of the block to be coded, a subset of original pixels is selected from the original pixel set (including all reconstructed pixels adjacent to the block to be coded). Then, downsampling or filtering is performed on this subset of original pixels, instead of downsampling or filtering the entire original pixel set. This reduces the complexity of predictive coding and improves coding efficiency.

[0044] This application provides another information processing method. Figure 3B This is a schematic diagram illustrating the implementation flow of another information processing method according to an embodiment of this application, such as... Figure 3B As shown, the method includes steps S311 to S314:

[0045] S311. Determine the original pixel set of the block to be encoded in the current video frame, wherein each original pixel in the original pixel set is a reconstructed pixel outside the block to be encoded;

[0046] Understandably, at least one reconstructed pixel outside the block to be encoded is defined here as an original pixel in the original pixel set. Generally, the reconstructed pixels outside the block to be encoded are called reference pixels because when predicting the block to be encoded, the attribute information (e.g., luma components, chroma components, etc.) of these reconstructed pixels needs to be referenced to predict the pixels to be predicted in the block. Therefore, a reconstructed pixel is a pixel that has already been predicted. For example, Figure 3C The original pixel set may include all reconstructed pixels in the upper row region 31 of the block to be encoded 30, or it may include all reconstructed pixels in the left column region 32 of the block to be encoded 30. Generally, the original pixel set includes pixel identifiers of the reconstructed pixels, such as pixel numbers.

[0047] S312. Determine the attribute information of each reconstructed pixel in the original pixel set;

[0048] For example, based on the video data of the current video frame, the position information, luminance component, chrominance component, and other attribute information of each reconstructed pixel in the original pixel set can be obtained.

[0049] S313. Select the reconstructed pixels whose attribute information meets the filtering conditions to obtain the target pixel set;

[0050] Generally, the filtering conditions are used to select reconstructed pixels that are strongly correlated with the block to be encoded, thereby obtaining a target pixel set. It should be noted that the target pixel set includes attribute information of the reconstructed pixels.

[0051] S314. Based on the attribute information of each reconstructed pixel in the target pixel set, predictive coding is performed on the block to be encoded.

[0052] Understandably, predictive coding is a processing step within the coding process. It primarily utilizes the spatial or temporal correlation of adjacent pixels to predict the currently encoded pixel using already transmitted pixels, and then encodes and transmits the difference between the predicted value and the true value (i.e., the prediction error). For example, in predictive coding, the attribute information of each reconstructed pixel in the target pixel set is used to perform downsampling or filtering processing on the target pixel set; or, for instance, the attribute information of each reconstructed pixel in the target pixel set is used to determine the prediction model of the block to be encoded.

[0053] In other embodiments, the block to be encoded can also be transformed or entropy encoded based on the attribute information of each reconstructed pixel in the target pixel set.

[0054] Understandably, the transform coding is another processing step in the coding process. It mainly involves transforming the image described in the spatial domain (e.g., discrete cosine transform, discrete sine transform, Hadamard transform, etc.) to form data (coefficients) in the transform domain, thereby changing the data distribution and reducing the amount of effective data. For example, in transform coding, the attribute information of each reconstructed pixel in the target pixel set is used to derive the transform coding mode information.

[0055] Predictive coding and transform coding are two different compression coding methods. Combining these two methods constitutes hybrid coding. In the hybrid coding framework, entropy coding is a processing step after predictive coding and transform coding. It is a variable-length coding (VLC) method that can further improve the compression efficiency of hybrid coding. In entropy coding, the attribute information of each reconstructed pixel in the target pixel set can be used to deduce the information of the entropy coding context model.

[0056] In this embodiment, reconstructed pixels whose attribute information meets the filtering conditions are selected from the original pixel set of the block to be encoded to obtain a target pixel set; based on the attribute information of each reconstructed pixel in the target pixel set, predictive coding, transform coding, or entropy coding are performed on the block to be encoded; thus, by performing any of the above coding processes on the block to be encoded based on the attribute information of a selected portion of the original pixels (rather than all the original pixels in the original pixel set), the computational complexity of the coding process can be reduced, thereby improving video coding efficiency and thus improving the smoothness of the video for the user.

[0057] This application provides yet another information processing method. Figure 3D This is a schematic diagram illustrating the implementation flow of another information processing method according to an embodiment of this application, as shown below. Figure 3D As shown, the method includes steps S321 to S324:

[0058] S321. Determine at least one pixel to be encoded in the block to be encoded in the current video frame as the original pixel, and obtain the original pixel set;

[0059] It should be noted that the attribute information of the pixels to be encoded in the block to be encoded is different in different stages of the encoding process. For example, if steps S321 to S324 perform predictive coding, then the attribute information of the pixels to be encoded includes luminance components, chrominance components, and position information. If steps S321 to S324 perform transform coding, then the attribute information of the pixels to be encoded includes the residual value output after predictive coding and position information; that is, the block to be encoded is a residual block. Furthermore, if steps S321 to S324 perform entropy coding, then the attribute information of the pixels to be encoded includes the transform coefficients output after transform coding and position information; that is, the block to be encoded is a coefficient block.

[0060] S322. Determine the attribute information of each original pixel in the original pixel set;

[0061] S323. Select the original pixels whose attribute information meets the filtering conditions to obtain the target pixel set;

[0062] For example, in transform coding, based on the position information of each pixel to be encoded in the residual block, a portion of the pixels to be encoded can be sampled from the residual block, and the sampled portion of the pixels to be encoded can be determined as the target pixel set; as another example, in transform coding, pixels to be encoded with residual values ​​greater than a preset threshold can be selected from the residual block to obtain the target pixel set; as yet another example, in entropy coding, based on the position information of each pixel to be encoded in the coefficient block, a portion of the pixels to be encoded can be sampled from the coefficient block, and the sampled portion of the pixels to be encoded can be determined as the target pixel set.

[0063] S324. Based on the attribute information of each original pixel in the target pixel set, perform predictive coding, transform coding, or entropy coding on the block to be encoded.

[0064] In this embodiment of the application, only a portion of the pixels to be encoded in the block to be encoded are encoded, such as predictive coding, transform coding, or entropy coding, instead of encoding all the pixels in the block to be encoded. This can effectively reduce the complexity of the encoding process, improve the data processing speed, thereby improving the encoding efficiency and thus improving the smoothness of the user's video viewing.

[0065] This application provides another information processing method. Figure 4 This is a schematic diagram illustrating the implementation flow of another information processing method according to an embodiment of this application, such as... Figure 4 As shown, the method includes steps S401 to S406:

[0066] S401. Determine the N reconstructed pixels within a preset range of the block to be encoded in the current video frame as the original pixel set; where N is a preset integer greater than or equal to 1;

[0067] For example, the reconstructed pixels in the reference rows or reference columns adjacent to the block to be encoded are determined as the original pixel set. The number of reference rows and reference columns is not limited here; it can be one or more reference rows, or one or more reference columns. Figure 3C As shown, the reconstructed pixels in the upper row region 31 of the block to be encoded 30 can be determined as the original pixel set; or, the reconstructed pixels in the left column region 32 of the block to be encoded 30 can also be determined as the original pixel set. It should be noted that the value of N can be preset, for example, N = the side length of the block to be encoded + n, where n is an integer greater than or equal to 0, and the side length of the block to be encoded can be represented by the number of pixels.

[0068] It should also be noted that step S401 is actually an implementation example of step S301 in the above embodiments.

[0069] S402. Determine the attribute information of each reconstructed pixel in the original pixel set;

[0070] S403. Based on the attribute information of the reconstructed pixels in the original pixel set, determine the correlation between each reconstructed pixel and the block to be encoded;

[0071] Understandably, the correlation is used to characterize the degree of association between the reconstructed pixel and the block to be encoded (or the pixel to be predicted in the block to be encoded). For example, the positional relationship between the reconstructed pixel in the original pixel set and the block to be encoded can be determined, and this positional relationship can be defined as the correlation between the reconstructed pixel and the block to be encoded. In other embodiments, reconstructed pixels whose image components are within a preset range can be selected from the original pixel set as target pixels to obtain the target pixel set; wherein, the image components are luminance components or chrominance components. For example, the preset range is (x... min ,x max ), where x min and x max These refer to the maximum and minimum values ​​of the image components in the original pixel set, respectively. The stronger the correlation between the reconstructed pixel and the block to be encoded, the more similar the attribute information of the reconstructed pixel is to the attribute information of the predicted pixels in the block to be encoded. Therefore, based on the correlation between the reconstructed pixel and the block to be encoded, reliable reconstructed pixels can be quickly and effectively selected as the reconstructed pixels in the target pixel set. Furthermore, when selecting reconstructed pixels that meet the screening criteria based on correlation, some reconstructed pixels with low correlation are filtered out. Therefore, the reconstructed pixels in the resulting target pixel set are all pixels strongly correlated with the block to be encoded. Thus, predictive encoding of the block to be encoded is performed based on the attribute information of each reconstructed pixel in the target pixel set, improving the robustness of the algorithm.

[0072] S404. Select the reconstructed pixels whose correlation meets the filtering conditions to obtain the target pixel set;

[0073] For example, assuming the distance between the reconstructed pixel and the block to be encoded is used to characterize the correlation between them, then reconstructed pixels with a distance greater than or equal to a preset distance threshold can be selected as reconstructed pixels in the target pixel set. It should be noted that steps S403 and S404 are actually an implementation example of step S303 in the above embodiments.

[0074] S405. Preprocess the attribute information of each reconstructed pixel in the target pixel set to obtain the preprocessed target pixel set.

[0075] Understandably, if only some reference pixels (i.e., reconstructed pixels in the target pixel set) are preprocessed before predictive coding of the block to be coded, and predictive coding of the block to be coded is performed based on the preprocessed results, the complexity of preprocessing can be reduced, thereby reducing the computational complexity of predictive coding and improving video smoothness.

[0076] For example, the preprocessing could be downsampling, where only the attribute information of each reconstructed pixel in the target pixel set is downsampled, rather than the original pixel set. Understandably, the purpose of downsampling is to unify the resolution among the multiple image components of the reconstructed pixel, i.e., to make the multiple image components of the reconstructed pixel have the same size in the spatial domain. Therefore, downsampling only a portion of the reconstructed pixels reduces the number of downsampling operations, thereby improving the coding efficiency of predictive coding.

[0077] Alternatively, the preprocessing can also be filtering (e.g., interpolation filtering), that is, filtering the attribute information of each reconstructed pixel in the target pixel set to obtain a filtered target pixel set. Similarly, compared to filtering the original pixel set, this method only filters the attribute information of each reconstructed pixel in the target pixel set, instead of filtering all reconstructed pixels in the original pixel set. This reduces the number of filtering iterations and increases the filtering speed.

[0078] S406. Based on the attribute information of each reconstructed pixel in the preprocessed target pixel set, predictive coding is performed on the block to be encoded.

[0079] It should be noted that steps S405 and S406 are actually an implementation example of step S304 in the above embodiments.

[0080] This application provides yet another information processing method. Figure 5A This is a schematic diagram illustrating the implementation flow of another information processing method according to an embodiment of this application, as shown below. Figure 5A As shown, the method includes steps S501 to S506:

[0081] S501. Determine the original pixel set of the block to be encoded in the current video frame, wherein each reference pixel in the original pixel set is a reconstructed pixel outside the block to be encoded;

[0082] S502. Determine the attribute information of each reconstructed pixel in the original pixel set;

[0083] S503. Select the reconstructed pixels whose attribute information meets the filtering conditions to obtain the target pixel set;

[0084] S504. The attribute information of each reconstructed pixel in the target pixel set is downsampled to obtain the target pixel set after downsampling.

[0085] S505. Based on the attribute information of each reconstructed pixel in the target pixel set after downsampling, a prediction model is constructed. The prediction model is used to characterize the prediction relationship between multiple image components of the pixel to be encoded in the block to be encoded.

[0086] It should be noted that the prediction model can be a linear model or a nonlinear model. In this embodiment, the structure of the prediction model is not specifically limited. Based on the attribute information of each reconstructed pixel in the target pixel set after downsampling, a prediction model can be fitted so that multiple image components of the pixels to be encoded in the block conform to the prediction relationship represented in the prediction model.

[0087] For example, suppose the prediction model is as shown in formula (1):

[0088] C′=α*Y+β (1);

[0089] In the formula, α and β are the model parameters of the prediction model, Y is the luminance component, and C' is the predicted chrominance component. Based on the luminance and chrominance components of multiple reconstructed pixels in the downsampled target pixel set, the values ​​of model parameters α and β can be quickly derived, thereby obtaining the prediction model. The chrominance component of the pixel to be predicted in the block to be encoded can then be predicted using the obtained prediction model. For example, the maximum and minimum values ​​of the luminance component can be found in the downsampled target pixel set. Then, based on the maximum and minimum values ​​of the luminance component, the values ​​of model parameters α and β can be determined. This method, compared to determining the values ​​of model parameters α and β based on the original pixel set, can obtain the model parameter values ​​of the prediction model more efficiently, thereby improving data processing speed. Furthermore, the information processing method provided in this application embodiment is beneficial for improving coding efficiency, especially when the screening conditions are reasonable, the improvement in coding efficiency is more significant.

[0090] In other embodiments, steps S504 and S505 can be replaced by: filtering the attribute information of each reconstructed pixel in the target pixel set to obtain a filtered target pixel set; and constructing a prediction model based on the filtered target pixel set, wherein the prediction model is used to characterize the prediction relationship between multiple image components of the pixels to be encoded in the block to be encoded. Steps S505 and S506 are actually an implementation example of step S406 in the above embodiments.

[0091] S506. Based on the prediction model, predictive coding is performed on the block to be encoded.

[0092] In other embodiments, for step S505, a prediction model is constructed based on the downsampled target pixel set. This prediction model characterizes the prediction relationships between multiple image components of the pixels to be encoded in the block to be encoded, such as... Figure 5B As shown, this can be achieved through the following steps S5051 and S5052:

[0093] S5051. Based on the chromaticity component and luminance component of each reconstructed pixel in the target pixel set after downsampling, determine the parameter values ​​of the first parameter and the second parameter of the prediction model.

[0094] Of course, in other embodiments, the parameter values ​​of the first parameter and the second parameter of the prediction model can also be determined based on the chromaticity component and the luminance component of each reconstructed pixel in the filtered target pixel set. Compared with determining the parameter values ​​of the first parameter and the second parameter based on the filtered original pixel set, the former has lower computational complexity.

[0095] S5052. Construct the prediction model based on the parameter values ​​of the first parameter and the second parameter.

[0096] In this application embodiment, a concept of a reference pixel subset (i.e., the target pixel set described in the above embodiments) is proposed. In predictive coding, reference pixels are often used to predict the current coding block (i.e., the block to be encoded as described in step S311 above). Assuming that reference points in certain regions of the reference pixels are more effective for predicting the current coding block, a reference pixel subset consisting of these points is used to predict the current coding block. This reduces complexity without sacrificing or even improving coding performance. Therefore, in this application embodiment, by selecting appropriate reference pixel samples before any processing step (e.g., downsampling and filtering) before predicting the current coding block, a reference pixel subset is obtained. Predictive coding is then performed based on this subset, which improves the coding efficiency of predictive coding.

[0097] In predictive coding, reference pixels are frequently used to predict the current coding block. For example, available reference pixels in the upper and left regions of the current coding block are used to predict it. These reference pixels typically consist of one or more columns to the left of the current coding block and one or more rows to the top of it. These pixels are generally reconstructed pixels, i.e., pixels that have already undergone predictive coding. Sometimes, it is also necessary to downsample these pixels to form reference pixels.

[0098] For example, see Figure 3C As shown, the current block to be encoded is a 2N*2N encoding block. The original pixel set of this encoding block includes the 2N reconstructed pixels adjacent to its upper side and the 2N reconstructed pixels adjacent to its left side.

[0099] For a given coding block, the importance and relevance of each reference pixel are different. Furthermore, reference pixels in close proximity to the current coding block may have a similar impact on the prediction of the current coding block. In other words, some pixels in the vicinity of the current coding block may be very effective in prediction, while others are detrimental. By using only selected, relatively important reference pixels as a subset of reference pixels (i.e., the target pixel set) and using this subset to predict the current coding block, good prediction results with lower computational complexity can be obtained. In this embodiment, when selecting reference pixels, a subset of reference pixels is constructed based on factors such as the importance and relevance of each reference pixel. Using this subset to predict the current coding block reduces computational complexity while improving coding performance.

[0100] For example, original neighboring reference pixels (i.e., reference pixels in the original pixel set) can be obtained from the left or top region of the current coding block; then, pixels that meet the conditions can be selected from multiple original neighboring reference pixels according to the position and / or pixel features (such as intensity), to obtain a subset of reference pixels; after obtaining the subset of reference pixels, the subset can be used in any process before predicting the current coding block, such as downsampling or filtering the subset. Another example is the use of the subset in the Cross-Component Linear Model (CCLM) prediction method. Understandably, the Cross-Component Linear Model (CCLM) is a coding tool in H.266 / VVC. This model uses the reconstructed luminance (Y) to predict the corresponding chrominance (C). For example, the chrominance value C' of the current coding block can be derived using the following linear model formula (2).

[0101] C′=α*Y+β (2);

[0102] Here, the values ​​of parameters α and β can be derived from the brightness and chromaticity of adjacent reference pixels.

[0103] In CCLM, certain points in the original reference region can be selected. Then, a subset of reference pixels formed by these points is further processed. For example, downsampling is performed using the reference pixel subset, and the maximum and minimum values ​​are found from the downsampled subset. Based on these maximum and minimum values, the values ​​of parameters α and β are determined. This downsampling of the reference pixel subset reduces the number of downsampling operations compared to downsampling the original pixel set. Simultaneously, the maximum and minimum values ​​can be found more quickly from the downsampled subset, thereby rapidly determining the prediction model and improving data processing speed. Of course, if the selection criteria are appropriate, the coding efficiency of predictive coding will be improved. In other embodiments, using a subset of reference pixels allows for faster derivation of the values ​​of parameters α and β compared to the original pixel set. The resulting prediction model is then used to predict the current coding block, further improving data processing speed.

[0104] In the embodiments of this application, video coding is performed by using a subset of reference pixels in any processing prior to predicting the current coding block, which reduces the complexity of video coding and improves the robustness of the algorithm.

[0105] Based on the foregoing embodiments, this application provides an information processing device, which includes the included units and the modules included in each unit, which can be implemented by a processor in an electronic device; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP) or field programmable gate array (FPGA), etc.

[0106] Figure 6A This is a schematic diagram of the composition structure of the information processing device according to an embodiment of this application, as shown below. Figure 6A As shown, the device 6 includes an original pixel set determination module 61, an attribute information determination module 62, a target pixel selection module 63, and an encoding processing module 64, wherein: the original pixel set determination module 61 is configured to determine the original pixel set of the block to be encoded in the current video frame; the attribute information determination module 62 is configured to determine the attribute information of each original pixel in the original pixel set; the target pixel selection module 63 is configured to select original pixels whose attribute information meets the filtering conditions to obtain a target pixel set; and the encoding processing module 64 is configured to perform encoding processing on the block to be encoded based on the attribute information of each original pixel in the target pixel set.

[0107] In other embodiments, the original pixel set determination module 61 is configured to determine at least one reconstructed pixel outside the block to be encoded as an original pixel in the original pixel set; the encoding processing module 64 is configured to perform predictive encoding, transform encoding or entropy encoding on the block to be encoded based on the attribute information of each reconstructed pixel in the target pixel set.

[0108] In other embodiments, the original pixel set determination module 61 is configured to determine at least one pixel to be encoded within the block to be encoded as an original pixel in the original pixel set; the encoding processing module 64 is configured to perform predictive encoding, transform encoding, or entropy encoding on the target pixel set based on the attribute information of each pixel to be encoded in the target pixel set.

[0109] In other embodiments, such as Figure 6B As shown, the target pixel selection module 63 includes: a correlation determination unit 631, configured to determine the correlation between each reconstructed pixel and the block to be encoded based on the attribute information of the reconstructed pixels in the original pixel set; and a selection unit 632, configured to select reconstructed pixels whose correlation satisfies the filtering conditions to obtain a target pixel set.

[0110] In other embodiments, the correlation determination unit 631 is configured to determine the positional relationship between the reconstructed pixels in the original pixel set and the block to be encoded; and to determine the positional relationship as the correlation between the reconstructed pixels and the block to be encoded.

[0111] In other embodiments, the target pixel selection module 63 is configured to select reconstructed pixels whose image components are within a preset range from the original pixel set as target pixels, thereby obtaining the target pixel set.

[0112] In other embodiments, the image component is a luminance component or a chrominance component.

[0113] In other embodiments, such as Figure 6C As shown, the encoding processing module 64 includes: a preprocessing unit 641 configured to preprocess the attribute information of each reconstructed pixel in the target pixel set to obtain a preprocessed target pixel set; and a prediction encoding unit 642 configured to perform prediction encoding on the block to be encoded based on the attribute information of each reconstructed pixel in the preprocessed target pixel set.

[0114] In other embodiments, the preprocessing unit 641 is configured to perform downsampling processing on the attribute information of each reconstructed pixel in the target pixel set to obtain a downsampled target pixel set; or, it is configured to perform filtering processing on the attribute information of each reconstructed pixel in the target pixel set to obtain a filtered target pixel set.

[0115] In other embodiments, such as Figure 6D As shown, the predictive coding unit 641 includes: a model building subunit 6411, configured to build a predictive model based on the attribute information of each reconstructed pixel in the target pixel set after the downsampling process or the filtering process, wherein the predictive model is used to characterize the predictive relationship between multiple image components of the pixel to be encoded in the block to be encoded; and a predictive coding subunit 6412 configured to perform predictive coding on the block to be encoded based on the predictive model.

[0116] In other embodiments, the model construction subunit 6411 is configured to determine the parameter values ​​of the first parameter and the second parameter of the prediction model based on the chromaticity component and luminance component of each reconstructed pixel in the target pixel set after the downsampling process or the filtering process; and to construct the prediction model based on the parameter values ​​of the first parameter and the second parameter.

[0117] In other embodiments, the original pixel set determination module 61 is configured to determine N reconstructed pixels within a preset range where the block to be encoded is located as the original pixel set; wherein N is a preset integer greater than or equal to 1.

[0118] In other embodiments, the original pixel set determination module 61 is configured to determine the reconstructed pixels in the reference row or reference column adjacent to the block to be encoded as the original pixel set.

[0119] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0120] It should be noted that, in the embodiments of this application, if the above-described information processing method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a mobile phone, tablet computer, personal computer, personal digital assistant, navigator, digital phone, video phone, television, sensor device, server, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0121] This application provides an electronic device, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements the steps in the information processing method provided in the above embodiments.

[0122] This application provides a computer-readable storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the steps in the information processing method provided in the above embodiments.

[0123] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0124] It should be noted that, Figure 7 This is a schematic diagram of the hardware entity of the electronic device in the embodiment of this application, such as... Figure 7 As shown, the electronic device 700 includes a memory 701 and a processor 702. The memory 701 stores a computer program that can run on the processor 702. When the processor 702 executes the program, it implements the steps in the information processing method provided in the above embodiments.

[0125] It should be noted that the memory 701 is configured to store instructions and applications executable by the processor 702, and can also cache data to be processed or already processed by the processor 702 and various modules in the electronic device 700 (e.g., image data, audio data, voice communication data and video communication data), which can be implemented by flash memory or random access memory (RAM).

[0126] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0127] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0128] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0129] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0130] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0131] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0132] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a mobile phone, tablet computer, personal computer, personal digital assistant, navigator, digital phone, video phone, television, sensor device, server, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, magnetic disks, or optical disks.

[0133] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0134] Industrial applicability

[0135] In this embodiment, original pixels whose attribute information meets the filtering conditions are selected from the original pixel set of the block to be encoded to obtain a target pixel set; the block to be encoded is encoded based on the attribute information of each original pixel in the target pixel set; thus, encoding the block to be encoded based on the attribute information of a selected portion of the original pixels can reduce the computational complexity of the encoding process, thereby improving video encoding efficiency and improving the smoothness of video viewing for users.

Claims

1. An information processing method applied to an encoder, the method comprising: Determine the sample set of the block to be encoded in the current video frame; wherein the sample set includes at least one reconstructed sample outside the block to be encoded; Based on the attribute information of the samples in the sample set, at least one sample is selected from the sample set to obtain the target sample set; wherein, the attribute information includes positional relationship, luminance component and chrominance component; Based on the attribute information of the reconstructed samples in the target sample set, the reconstructed samples in the target sample set are filtered to obtain the filtered target sample set. Based on the attribute information of the reconstructed samples in the target sample set after filtering, the parameter values ​​of the prediction model are calculated. The prediction model is used to characterize the prediction relationship between two or more image components in the block to be encoded. Based on the prediction model, predictive coding is performed on the block to be encoded.

2. The method according to claim 1, characterized in that, The sample set includes: at least one row of reconstructed samples adjacent to the upper side of the block to be encoded, and / or at least one column of reconstructed samples adjacent to the left side of the block to be encoded.

3. The method according to claim 2, characterized in that, The step of selecting at least one sample from the sample set based on the attribute information of the samples in the sample set to obtain the target sample set includes: Based on the attribute information of the reconstructed samples in the sample set, the positional relationship between the reconstructed samples in the sample set and the block to be encoded is determined; Based on the described positional relationships, the target sample set is obtained.

4. The method according to claim 2, characterized in that, The step of selecting at least one sample from the sample set based on the attribute information of the samples in the sample set to obtain the target sample set includes: Based on the attribute information, select a portion of the reconstructed samples from at least one row of reconstructed samples adjacent to the upper side of the block to be encoded, and / or select a portion of the reconstructed samples from at least one column of reconstructed samples adjacent to the left side of the block to be encoded. The target sample set contains the selected reconstructed samples.

5. The method according to claim 4, characterized in that, The step of selecting a portion of reconstructed samples from at least one row of reconstructed samples adjacent to the upper side of the block to be encoded, based on the attribute information, includes: Based on the attribute information, at least one row of reconstructed samples adjacent to the upper side of the block to be encoded is downsampled to obtain partial reconstructed samples; The step of selecting a portion of reconstructed samples from at least one column of reconstructed samples adjacent to the left side of the block to be encoded, based on the attribute information, includes: Based on the attribute information, at least one column of reconstructed samples adjacent to the left side of the block to be encoded is downsampled to obtain partial reconstructed samples.

6. The method according to claim 1, characterized in that, The input to the filtering process includes the reconstructed sample and one or more adjacent reconstructed samples.

7. The method according to claim 1, characterized in that, The step of reconstructing the attribute information of samples from the filtered target sample set and constructing a prediction model includes: Based on the chromaticity and luminance components of the reconstructed samples in the target sample set after filtering, the parameter values ​​of the first parameter and the second parameter of the prediction model are determined, wherein the prediction model is a linear model represented by the first parameter and the second parameter.

8. An information processing method applied to a decoder, the method comprising: Determine the sample set of the processing block to be decoded; wherein the sample set includes at least one reconstructed sample outside the processing block; Based on the attribute information of the samples in the sample set, at least one sample is selected from the sample set to obtain the target sample set; wherein, the attribute information includes positional relationship, luminance component and chrominance component; Based on the attribute information of the reconstructed samples in the target sample set, the reconstructed samples in the target sample set are filtered to obtain the filtered target sample set. Based on the attribute information of the reconstructed samples in the target sample set after filtering, the parameter values ​​of the prediction model are calculated. The prediction model is used to characterize the prediction relationship between two or more image components in the processing block. Based on the prediction model, the predicted value of the processing block is determined; The processing block is decoded based on the predicted value of the processing block.

9. The method according to claim 8, characterized in that, The sample set includes: at least one row of reconstructed samples adjacent to the upper side of the processing block, and / or at least one column of reconstructed samples adjacent to the left side of the processing block.

10. The method according to claim 9, characterized in that, The step of selecting at least one sample from the sample set based on the attribute information of the samples in the sample set to obtain the target sample set includes: Based on the attribute information of the reconstructed samples in the sample set, the positional relationship between the reconstructed samples in the sample set and the processing block is determined; Based on the described positional relationships, the target sample set is obtained.

11. The method according to claim 9, characterized in that, The step of selecting at least one sample from the sample set based on the attribute information of the samples in the sample set to obtain the target sample set includes: Based on the attribute information, select a portion of the reconstructed samples from at least one row of reconstructed samples adjacent to the top side of the processing block, and / or select a portion of the reconstructed samples from at least one column of reconstructed samples adjacent to the left side of the processing block. The target sample set contains the selected reconstructed samples.

12. The method according to claim 11, characterized in that, The step of selecting a portion of the reconstructed samples from at least one row of reconstructed samples adjacent to the upper side of the processing block based on the attribute information includes: Based on the attribute information, at least one row of reconstructed samples adjacent to the upper side of the processing block is downsampled to obtain partial reconstructed samples; The step of selecting a portion of reconstructed samples from at least one column of reconstructed samples adjacent to the left side of the processing block based on the attribute information includes: Based on the attribute information, at least one column of reconstructed samples adjacent to the left side of the processing block is downsampled to obtain partial reconstructed samples.

13. The method according to claim 8, characterized in that, The input to the filtering process includes the reconstructed sample and one or more adjacent reconstructed samples.

14. The method according to claim 8, characterized in that, The step of reconstructing the attribute information of samples from the filtered target sample set and constructing a prediction model includes: Based on the chromaticity and luminance components of the reconstructed samples in the target sample set after filtering, the parameter values ​​of the first parameter and the second parameter of the prediction model are determined, wherein the prediction model is a linear model represented by the first parameter and the second parameter.

15. An information processing apparatus applied to an encoder, the apparatus comprising: The sample set determination module is configured to determine the sample set of the block to be encoded in the current video frame; wherein, the sample set includes at least one reconstructed sample outside the block to be encoded; The target sample selection module is configured to select at least one sample from the sample set based on the attribute information of the samples in the sample set, thereby obtaining a target sample set; wherein the attribute information includes positional relationship, luminance component and chrominance component; The encoding processing module is configured to perform filtering processing on the reconstructed samples in the target sample set based on the attribute information of the reconstructed samples in the target sample set to obtain a filtered target sample set; calculate the parameter values ​​of the prediction model based on the attribute information of the reconstructed samples in the filtered target sample set, wherein the prediction model is used to characterize the prediction relationship between two or more image components in the block to be encoded; and perform predictive encoding on the block to be encoded based on the prediction model.

16. An information processing apparatus applied to a decoder, the apparatus comprising: The sample set determination module is configured to determine the sample set of the processing block to be decoded; wherein the sample set includes at least one reconstructed sample outside the processing block; The target sample selection module is configured to select at least one sample from the sample set based on the attribute information of the samples in the sample set, thereby obtaining a target sample set; wherein the attribute information includes positional relationship, luminance component and chrominance component; The decoding processing module is configured to: filter the reconstructed samples in the target sample set based on the attribute information of the reconstructed samples in the target sample set to obtain a filtered target sample set; calculate the parameter values ​​of a prediction model based on the attribute information of the reconstructed samples in the filtered target sample set, wherein the prediction model is used to characterize the prediction relationship between two or more image components in the processing block; determine the prediction value of the processing block based on the prediction model; and decode the processing block according to the prediction value of the processing block.

17. An electronic device comprising a memory and a processor, the memory storing a computer program executable on the processor, the processor executing the program to implement the steps of the information processing method of any one of claims 1 to 7 or the steps of the information processing method of any one of claims 8 to 14.

18. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the information processing method according to any one of claims 1 to 7 or the steps of the information processing method according to any one of claims 8 to 14.

19. A method for transmitting a code stream, characterized in that, Generate a bitstream by performing the method according to any one of claims 1 to 7; and transmit the bitstream.

Images