Information processing method and apparatus, and device and storage medium

Abstract

To provide an information processing method and apparatus, and a device and a storage medium that improve video encoding efficiency by reducing the computational complexity of an encoding process and also improve the smoothness of video playback as viewed by users.SOLUTION: The method includes: determining an original pixel set of a block to be encoded in a current video frame; determining attribute information of each original pixel in the original pixel set; selecting original pixels of which attribute information satisfies a screening condition and acquiring a target pixel set; and executing encoding processing on the block to be encoded, on the basis of the attribute information of each original pixel of the target pixel set.SELECTED DRAWING: Figure 3A

Description

[Technical field]

[0001] The present embodiment relates to the technical field of video encoding and decoding, and relates to, but is not limited to, an information processing method, an apparatus, a device, and a storage medium. [Background technology]

[0002] In recent years, video services have been rapidly developed in the fields of the Internet and mobile communications. Video services require first encoding source video data, and then transmitting the encoded video data to user terminals through channels of the Internet or mobile communications networks.

[0003] For users, video fluency directly affects their video viewing experience. Moreover, the computational complexity of predictive coding in video coding directly affects video fluency. Summary of the Invention [Problem to be solved by the invention]

[0004] In view of the above, the present embodiments provide an information processing method, an apparatus, a device, and a storage medium for solving at least one problem in the related art. [Means for solving the problem]

[0005] The technical solution according to the embodiment of the present application can be realized as follows.

[0006] According to a first aspect, an embodiment of the present application provides an information processing method, the method including: determining an original pixel set of a block to be coded in a current video frame; determining attribute information of each original pixel of the original pixel set; The method includes selecting original pixels whose attribute information satisfies a screening condition to obtain a target pixel set, and performing an encoding process on the block to be encoded based on the attribute information of each original pixel of the target pixel set.

[0007] According to a second aspect, an embodiment of the present application provides an information processing device, the device comprising: an original pixel set determination module configured to determine an original pixel set of a block to be encoded in a current video frame; an attribute information determination module configured to determine attribute information of each original pixel of the original pixel set; a target pixel selection module configured to select original pixels whose attribute information satisfies a screening condition to obtain a target pixel set; and an encoding processing module configured to perform an encoding process on the block to be encoded based on the attribute information of each original pixel of the target pixel set.

[0008] According to a third aspect, an embodiment of the present application provides an electronic device comprising a memory and a processor, the memory configured to store a computer program executable by the processor, and the processor configured to realize steps in the above information processing method when executing the program.

[0009] According to a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, the computer program implementing the steps of the above information processing method when executed by a processor. Effect of the Invention

[0010] In the embodiment of the present application, an original pixel whose attribute information satisfies a screening condition is selected from the original pixel set of the block to be coded to obtain a target pixel set, and an encoding process is performed on the block to be coded based on the attribute information of each original pixel of the target pixel set. In this way, instead of performing an encoding process on the block to be coded based on the attribute information of all original images in the original pixel set, the encoding process is performed on the block to be coded based on the attribute information of a part of the selected original pixels (i.e., the original pixels in the target pixel set), thereby reducing the computational complexity of the encoding process, thereby improving the efficiency of video coding and improving the fluency of the video played by the user. [Brief description of the drawings]

[0011] [Figure 1] FIG. 2 is an exemplary structural diagram of a network architecture configuration for video encoding and decoding according to an embodiment of the present application. [Figure 2A] FIG. 2 is an exemplary structural diagram of a video encoder according to an embodiment of the present invention; [Figure 2B] FIG. 2 is an exemplary structural diagram of a video decoder according to an embodiment of the present invention; [Figure 3A] 1 is an exemplary implementation flowchart of an information processing method according to an embodiment of the present application; [Figure 3B] 1 is an exemplary implementation flowchart of another information processing method according to an embodiment of the present application; [Figure 3C] FIG. 2 is a schematic diagram of the relationship between a block to be coded and reference pixels according to an embodiment of the present application; [Figure 3D] 10 is an exemplary implementation flowchart of yet another information processing method according to an embodiment of the present application; [Figure 4] 1 is an exemplary implementation flowchart of another information processing method according to an embodiment of the present application; [Figure 5A] 10 is an exemplary implementation flowchart of yet another information processing method according to an embodiment of the present application; [Figure 5B]1 is an exemplary implementation flowchart of a method for building a predictive model according to an embodiment of the present disclosure. [Figure 6A] 1 is an exemplary structural diagram of an information processing device according to an embodiment of the present invention; [Figure 6B] FIG. 13 is an exemplary structural diagram of another information processing device according to an embodiment of the present application. [Figure 6C] FIG. 13 is an exemplary structural diagram of a configuration of yet another information processing device according to an embodiment of the present application. [Figure 6D] FIG. 13 is an exemplary structural diagram of a configuration of yet another information processing device according to an embodiment of the present application. [Figure 7] FIG. 2 is a schematic diagram of hardware entities of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] This embodiment first provides a network architecture for video encoding and decoding. FIG. 1 is an exemplary structural diagram of a configuration of a network architecture for video encoding and decoding according to an embodiment of the present application. As shown in FIG. 1, the network architecture includes one or more electronic devices 11-1N and a communication network 01, where the electronic devices 11-1N may perform video interaction through the communication network 01. In the implementation process, the electronic devices may be various types of devices with video encoding and decoding functions. For example, the electronic devices may include mobile phones, tablet computers, personal computers, personal digital assistants, navigators, digital phones, video phones, televisions, sensing devices, servers, etc.

[0013] The electronic device has a video encoding and decoding function, and generally includes a video encoder and a video decoder. For example, referring to FIG. 2A, the configuration structure of the video encoder 21 includes a transform and quantization unit 211, an intraframe estimation unit 212, an intraframe prediction unit 213, a motion compensation unit 214, a motion estimation unit 215, an inverse transform and inverse quantization unit 216, a filter control analysis unit 217, a filtering unit 218, an encoding unit 219, and a decoded image cache unit 210, etc. Here, the filtering unit 218 can realize deblocking filtering and Sample Adaptive Offset (SAO) filtering, and the encoding unit 219 can realize header information coding and Context-based Adaptive Binary Arithmetic Coding (CABAC). For the input source video data, a to-be-coded block of one current video frame can be obtained by dividing a coding tree unit (CTU), and then, after performing intraframe prediction or interframe prediction on the to-be-coded block, the obtained residual information is transformed by the transform and quantization unit 211 (including transforming the residual information from pixel domain to transform domain and quantizing the obtained transform coefficients to further reduce the bit rate). The intraframe estimation unit 212 and the intraframe prediction unit 213 are configured to perform intraframe prediction on the to-be-coded block, such as determining an intraframe prediction mode used to code the to-be-coded block. The motion compensation unit 214 and the motion estimation unit 215 are configured to perform interframe prediction coding on the to-be-coded block in relation to one or more blocks in one or more reference frames to provide temporal prediction information. Here, the motion estimation unit 215 is configured to estimate a motion vector, and the motion of the to-be-coded block can be estimated according to the motion vector, and then the motion compensation unit 214 performs motion compensation based on the motion vector.After performing the intra prediction mode, the intra prediction unit 213 is further configured to provide the selected intra prediction data to the coding unit 219, and the motion estimation unit 215 sends the motion vector data determined by the calculation to the coding unit 219. Furthermore, the inverse transform and inverse quantization unit 216 is configured to reconstruct the block to be coded, i.e., reconstruct a residual block in the pixel domain, and block artifacts of the reconstructed residual block are removed by the filter control analysis unit 217 and the filtering unit 218, and then add the reconstructed residual block to one prediction block in the frame of the decoded image cache unit 210 to generate a reconstructed video coding block. The coding unit 219 is configured to code various coding parameters and quantized transform coefficients, and in a coding algorithm based on CABAC, the context content can be based on neighboring coding blocks, and can be used to code information indicating the determined intra prediction mode to output a bitstream of the video data. The decoded image cache unit 210 is configured to store the reconstructed video coding block used for prediction reference. As the video is encoded, new reconstructed video coding blocks are continually generated and these reconstructed video coding blocks are stored in a decoded picture cache unit 210 .

[0014] As shown in FIG. 2B, the configuration structure of the video decoder 22 corresponding to the video encoder 21 includes a decoding unit 221, an inverse transform and inverse quantization unit 222, an intraframe prediction unit 223, a motion compensation unit 224, a filtering unit 225, and a decoded image cache unit 226, etc. Here, the decoding unit 221 can realize header information decoding and CABAC decoding, and the filtering unit 225 can realize deblocking filtering and SAO filtering. After performing the encoding process shown in FIG. 2A on the input video signal, output a bitstream of the video signal. The bitstream is input to the video decoder 22, and the bitstream is first processed by the decoding unit 221 to obtain decoded transform coefficients. The transform coefficients are processed by the inverse transform and inverse quantization unit 222 to generate a residual block in the pixel domain. The intraframe prediction unit 223 can be configured to generate prediction data of a current decoded block according to the determined intraframe prediction mode and data of a previous decoded block from a current frame or image. The motion compensation unit 224 analyzes the motion vectors and other relevant syntax elements to determine prediction information for the currently decoded block, and uses the prediction information to generate a prediction block for the currently decoded block being decoded. The residual block from the inverse transform and inverse quantization unit 222 is summed with the corresponding prediction block generated by the intraframe prediction unit 223 or the motion compensation unit 224 to generate a decoded video block. Block artifacts in the decoded video block are removed by the filtering unit 225, thereby improving quality. The decoded video block is then stored in the decoded image cache unit 226, which stores reference images used for subsequent intraframe prediction or motion compensation, and is also used for output display of the video signal.

[0015] Based on this, the technical solution of the present application will be described in more detail below with reference to the accompanying drawings and embodiments. The information processing method according to the embodiment of the present application may be applied to the video encoder 21 or the video decoder 22, but the embodiment of the present application is not limited thereto.

[0016] The present embodiment provides an information processing method, the method is provided in an electronic device, and the functions realized by the method can be realized by calling a program by a processor in the electronic device. Of course, the program code can be stored in a computer storage medium, and obviously, the electronic device includes at least a processor and a storage medium.

[0017] FIG. 3A is an exemplary implementation flowchart of an information processing method according to an embodiment of the present application. As shown in FIG. 3A, the method includes steps S301 to S304.

[0018] In step S301, the original pixel set of the block to be coded in the current video frame is determined.

[0019] It should be explained that the block to be coded refers to an image area in a current video frame that needs to be coded, and the types of the block to be coded corresponding to different coding processes are different, and the contents included in the original pixel set are also different. For example, when the coding process is predictive coding, the block to be coded is an image block including luma and chroma components, and the reconstructed pixels outside the block to be coded can be determined as the original pixel set, i.e., the reference pixels used when performing predictive coding on the block to be coded, but the contents of the original pixel set are not limited in this specification and are merely examples here. When the coding process is transform coding, the block to be coded is an image block including residual values, i.e., the residual block output after predictive coding, and the pixels in the residual block can be determined as the original pixel set, but the contents of the original pixel set are not limited in this specification and are merely examples here. When the encoding process is entropy encoding, the block to be encoded is an image block containing transform coefficients, i.e., a coefficient block output after transform encoding, and the pixels in the coefficient block can be determined as the original pixel set, but the content of the original pixel set is not limited in this specification and is merely an example here.

[0020] In step S302, attribute information of each original pixel in the original pixel set is determined.

[0021] It can be understood that when the original pixel set is a reconstructed pixel outside the block to be encoded, the attribute information of each original pixel of the original pixel set is position information of the reconstructed pixel, a luma component, a chroma component, etc.; when the original pixel set is a pixel in a residual block, the attribute information of each original pixel of the original pixel set is position information of a pixel in the residual block, a residual value, etc.; when the original pixel set is a pixel in a coefficient block, the attribute information of each original pixel of the original pixel set is position information of a part of a pixel in the coefficient block, a transform coefficient, etc.

[0022] In step S303, the source pixels whose attribute information satisfies the screening condition are selected to obtain the target pixel set.

[0023] In step S304, an encoding process is performed on the block to be encoded based on the attribute information of each original pixel of the target pixel set.

[0024] It should be noted that this specification does not limit the type of the encoding process, for example, the encoding process may be one of predictive coding, transform coding, and entropy coding.

[0025] In the embodiment of the present application, before performing the encoding process on the block to be encoded, first, a portion of original pixels (i.e., the target pixel set) is selected from the original pixel set of the block to be encoded, and then the encoding process is performed on the block to be encoded based on attribute information of the portion of the original pixels (not all of the original pixels in the original pixel set), thereby reducing the complexity of the encoding process and the time required for the encoding process, thereby improving the fluency of the playback of the video viewed by the user.

[0026] It should be noted that the information processing method according to the embodiment of the present application is actually an improvement of conventional video coding techniques (such as video coding standards H.264, H.265, etc.), that is, the concept of subsets is added to conventional video coding techniques. For example, before performing predictive coding on a block to be coded, first select a part of original pixels (i.e., obtain a subset) from the original pixel set (including all reconstructed pixels adjacent to the block to be coded), and then perform a downsampling process or a filtering process, etc. on the original pixel set, instead of performing a downsampling process or a filtering process, etc. on the original pixel set, thereby reducing the complexity of predictive coding and improving coding efficiency.

[0027] The present embodiment provides another information processing method, and FIG. 3B is an exemplary implementation flowchart of the another information processing method according to the present embodiment. As shown in FIG. 3B, the method includes steps S311 to S314.

[0028] In step S311, an original pixel set of a block to be coded in a current video frame is determined, and each original pixel of the original pixel set is a reconstructed pixel outside the block to be coded.

[0029] It can be understood that at least one reconstructed pixel outside the block to be coded is determined as an original pixel in the original pixel set. Generally, the reconstructed pixels outside the block to be coded are called reference pixels, because when performing prediction on the block to be coded, it is necessary to refer to attribute information (e.g., luminance component, chrominance component, etc.) of these reconstructed pixels to predict a predicted pixel in the block to be coded. Therefore, the reconstructed pixel is a pixel point where prediction is completed. For example, as shown in FIG. 3C, the original pixel set may include all reconstructed pixels in the top row region 31 of the block to be coded 30, or the original pixel set may further include all reconstructed pixels in the region 32 of a left column of the block to be coded 30. Generally, the original pixel set includes pixel identifiers (e.g., pixel numbers) of reconstructed pixels.

[0030] In step S312, attribute information of each reconstructed pixel of the original pixel set is determined.

[0031] For example, attribute information such as position information, luminance component, chrominance component, etc. of each reconstructed pixel in the original pixel set can be obtained based on the video data of the current video frame.

[0032] In step S313, reconstruction pixels whose attribute information satisfies the screening condition are selected to obtain a target pixel set.

[0033] In general, the screening condition is used to filter reconstructed pixels that are strongly related to the block to be coded to obtain a target pixel set. It should be noted that the target pixel set includes attribute information of the reconstructed pixels.

[0034] In step S314, predictive coding is performed on the block to be coded based on the attribute information of each reconstructed pixel of the target pixel set.

[0035] It can be understood that the predictive coding is a processing step in the coding process, which mainly uses the spatial or temporal correlation of neighboring pixels to predict the pixel to be coded using transmitted pixels, and then codes and transmits the difference between the predicted value and the true value (i.e., the prediction error). For example, predictive coding uses attribute information of each reconstructed pixel in the target pixel set to perform a downsampling process or a filtering process, etc., on the target pixel set. In another example, the attribute information of each reconstructed pixel in the target pixel set is used to determine a prediction model of the block to be coded.

[0036] In another embodiment, transform coding or entropy coding, etc. may be performed on the block to be coded based on attribute information of each reconstructed pixel in the target pixel set.

[0037] It can be understood that the transform coding is another processing step in the coding process, which mainly involves performing a certain transformation (such as discrete cosine transform, discrete sine transform, Hadamard transform, etc.) on an image described in a spatial domain to generate data (coefficients) in a transform domain, thereby achieving the purpose of changing data distribution and reducing effective data amount. For example, transform coding utilizes attribute information of each reconstructed pixel in the target pixel set to derive information of a transform coding mode.

[0038] Predictive coding and transform coding are two different types of compression coding methods, and the combination of the two methods forms hybrid coding, and in the hybrid coding framework, entropy coding is a processing step after predictive coding and transform coding, which is variable length coding (VLC), which can further improve the compression efficiency of hybrid coding. In entropy coding, attribute information of each reconstructed pixel in the target pixel set can be utilized to derive information of an entropy coding context model.

[0039] In the embodiment of the present application, reconstructed pixels whose attribute information satisfies a screening condition are selected from the original pixel set of the block to be coded to obtain a target pixel set, and predictive coding, transform coding, or entropy coding is performed on the block to be coded based on the attribute information of each reconstructed pixel of the target pixel set. In this way, by performing any of the above coding processes on the block to be coded based on the attribute information of a part of the selected original pixels (not all of the original pixels in the original pixel set), the computational complexity of the coding process can be reduced, thereby improving the efficiency of video coding and further improving the fluency of the video playback viewed by the user.

[0040] The present embodiment provides yet another information processing method, and FIG. 3D is an exemplary implementation flowchart of the yet another information processing method according to the present embodiment. As shown in FIG. 3D, the method includes steps S321 to S324.

[0041] In step S321, at least one coded pixel in a coded block of a current video frame is determined as an original pixel to obtain an original pixel set.

[0042] It should be noted that at different stages of the encoding process, the attribute information of the pixel to be encoded in the block to be encoded is different. For example, what is performed in steps S321 to S324 is predictive encoding, in which case the attribute information of the pixel to be encoded includes luminance components, chrominance components, position information, etc. As another example, what is performed in steps S321 to S324 is transform encoding, in which case the attribute information of the pixel to be encoded is residual values ​​output after predictive encoding, position information, etc., that is, the block to be encoded is a residual block; as another example, what is performed in steps S321 to S324 is entropy encoding, in which case the attribute information of the pixel to be encoded is transform coefficients output after transform encoding, position information, etc., that is, the block to be encoded is a coefficient block.

[0043] In step S322, attribute information of each original pixel in the original pixel set is determined.

[0044] In step S323, the source pixels whose attribute information satisfies the screening condition are selected to obtain the target pixel set.

[0045] For example, in transform coding, a portion of pixels to be coded can be sampled from the residual block based on position information of each pixel to be coded in the residual block, and the sampled pixels to be coded are determined as the target pixel set. In another example, in transform coding, pixels to be coded whose residual value exceeds a preset threshold are selected from the residual block to obtain a target pixel set. In yet another example, in entropy coding, a portion of pixels to be coded are sampled from the coefficient block based on position information of each pixel to be coded in the coefficient block, and the sampled pixels to be coded are determined as the target pixel set.

[0046] In step S324, predictive coding, transform coding, or entropy coding is performed on the block to be coded according to each original pixel attribute information of the target pixel set.

[0047] In the present embodiment, instead of performing the encoding process on all the encoded pixels of a block to be encoded, the encoding process (e.g., predictive encoding, transform encoding, entropy encoding, etc.) is performed on only a portion of the encoded pixels of the block to be encoded, thereby effectively reducing the complexity of the encoding process and improving the data processing speed, thereby improving the encoding efficiency and further improving the fluency of the playback of the video watched by the user.

[0048] An embodiment of the present application provides another information processing method, and FIG. 4 is an exemplary implementation flowchart of another information processing method according to an embodiment of the present application. As shown in FIG. 4, the method includes steps S401 to S406.

[0049] In step S401, N reconstructed pixels (N is a preset integer greater than or equal to 1) within a preset range in which a block to be coded of a current video frame is located are determined as the original pixel set.

[0050] For example, the reconstructed pixels in a reference row or reference example adjacent to the block to be coded are determined as the original pixel set. The number of the reference row and the reference example is not limited here, and may be one or more reference rows, and may be one or more reference examples. As shown in Fig. 3C, the reconstructed pixels in the top row area 31 of the block to be coded 30 can be determined as the original pixel set, or the reconstructed pixels in the left column area 32 of the block to be coded 30 can be determined as the original pixel set. It should be noted that the value of N can be set in advance, for example, N=side length of the block to be coded+n (n is an integer greater than or equal to 0), and the side length of the block to be coded can be expressed by the number of pixel points.

[0051] It should be noted that step S401 is actually an embodiment of step S301 in the above embodiment.

[0052] In step S402, attribute information of each reconstructed pixel of the original pixel set is determined.

[0053] In step S403, a correlation degree between each reconstructed pixel and the block to be coded is determined based on attribute information of the reconstructed pixels in the original pixel set.

[0054] It can be understood that the correlation degree is used to represent the degree of association between the reconstructed pixel and the block to be coded (or a predicted pixel in the block to be coded). For example, a positional relationship between a reconstructed pixel in the original pixel set and the block to be coded is determined, and the positional relationship is determined as a correlation degree between the reconstructed pixel and the block to be coded. In another embodiment, the target pixel set can also be obtained by selecting a reconstructed pixel having an image component within a preset range from the original pixel set as a target pixel, where the image component is a luma component or a chroma component. For example, when the preset range is (X min ,X max ), where X min and X maxrespectively refer to the maximum and minimum values ​​of the image components of the original pixel set. The stronger the correlation between the reconstructed pixel and the block to be coded, the more similar the attribute information of the reconstructed pixel is to the attribute information of the predicted pixel in the block to be coded. Therefore, a reliable reconstructed pixel can be quickly and effectively selected as a reconstructed pixel in the target pixel set based on the correlation between the reconstructed pixel and the block to be coded. Furthermore, when selecting a reconstructed pixel that satisfies a screening condition based on the correlation, reconstructed pixels with a relatively low correlation are excluded, so that all reconstructed pixels in the obtained target pixel set are pixels that are strongly correlated with the block to be coded. In this way, the robustness of the algorithm can be improved by performing predictive coding on the block to be coded based on the attribute information of each reconstructed pixel of the target pixel set.

[0055] In step S404, the reconstruction pixels whose correlation degree satisfies the screening condition are selected to obtain the target pixel set.

[0056] For example, if the distance between the reconstructed pixel and the block to be coded represents the correlation degree between the reconstructed pixel and the block to be coded, the reconstructed pixel whose distance is greater than or equal to a preset distance threshold can be selected as the reconstructed pixel in the target pixel set. It should be noted that step S403 and step S404 are actually embodiments of step S303 in the above embodiment.

[0057] In step S405, pre-processing is performed on the attribute information of each reconstruction pixel of the target pixel set to obtain a pre-processed target pixel set.

[0058] It can be understood that before performing predictive coding on the block to be coded, only a portion of reference pixels (i.e., reconstructed pixels in the target pixel set) can be pre-processed, and predictive coding can be performed on the block to be coded based on the pre-processed result, thereby reducing the complexity of pre-processing, and further reducing the computational complexity of predictive coding, and improving the playback fluency of the video watched by the user.

[0059] For example, the pre-processing may be a down-sampling process, i.e., the down-sampling process is not performed on the original pixel set, but only on the attribute information of each reconstructed pixel in the target pixel set. It can be understood that the purpose of the down-sampling process is to unify the resolution between the multiple image components of the reconstructed pixels, i.e., to make the multiple image components of the reconstructed pixels have the same size in the spatial domain. Therefore, by performing the down-sampling process only on a part of the reconstructed pixels, the number of down-samplings can be reduced, and thus the coding efficiency of the predictive coding can be improved.

[0060] Alternatively, the pre-processing may be a filtering process (e.g., interpolation filtering, etc.), that is, the pre-processing is to perform a filtering process on attribute information of each reconstructed pixel of the target pixel set to obtain a filtered target pixel set. Similarly, compared with the filtering process on the original pixel set, the filtering process here is not performed on all reconstructed pixels in the original pixel set, but only on attribute information of each reconstructed pixel in the target pixel set, so that the number of filtering processes can be reduced and the speed of the filtering process can be improved.

[0061] In step S406, predictive coding is performed on the block to be coded based on the attribute information of each reconstructed pixel of the pre-processed target pixel set.

[0062] It should be noted that step S405 and step S406 are actually embodiments of step S304 in the above embodiment.

[0063] The present embodiment provides yet another information processing method, and FIG. 5A is an exemplary implementation flowchart of the yet another information processing method according to the present embodiment. As shown in FIG. 5A, the method includes steps S501 to S506.

[0064] In step S501, an original pixel set of a block to be coded in a current video frame is determined, and each reference pixel in the original pixel set is a reconstructed pixel outside the block to be coded.

[0065] In step S502, attribute information of each reconstructed pixel of the original pixel set is determined.

[0066] In step S503, reconstruction pixels whose attribute information satisfies the screening condition are selected to obtain a target pixel set.

[0067] In step S504, a down-sampling process is performed on the attribute information of each reconstruction pixel of the target pixel set to obtain a down-sampled target pixel set.

[0068] In step S505, a prediction model is constructed based on attribute information of each reconstructed pixel of the downsampled target pixel set, and the prediction model is used to represent a predictive relationship between multiple image components of the pixels to be coded in the block to be coded.

[0069] It should be noted that the prediction model may be a linear model or a nonlinear model, and the present embodiment does not particularly limit the structure of the prediction model. A prediction model can be constructed based on attribute information of each reconstructed pixel in the downsampled target pixel set, so that multiple image components of the coded pixels in the coded block can be matched to a prediction relationship represented by the prediction model.

[0070] For example, assume that the prediction model is as shown in equation (1).

number

[0071] In another embodiment, step S504 and step S505 can be replaced by performing a filtering process on attribute information of each reconstructed pixel of the target pixel set to obtain a filtered target pixel set, and constructing a prediction model based on the filtered target pixel set, where the prediction model is used to represent a prediction relationship between multiple image components of pixels to be coded in the block to be coded. Step S505 and step S506 are actually embodiments of step S406 in the above embodiment.

[0072] In step S506, predictive coding is performed on the block to be coded based on the prediction model.

[0073] In another embodiment, in step S505, a prediction model is constructed based on the downsampled target pixel set, and the prediction model is used to represent a prediction relationship between multiple image components of the pixels to be coded in the block to be coded. As shown in Figure 5B, the step S505 can be realized through steps S5051 and S5052.

[0074] In step S5051, a parameter value of a first parameter and a parameter value of a second parameter of a prediction model are determined based on chrominance and luma components of each reconstructed pixel in the downsampled target pixel set.

[0075] Of course, in other embodiments, the parameter values ​​of the first parameter and the second parameter of the prediction model can also be determined based on the chrominance and luminance components of each reconstructed pixel in the filtered target pixel set, and such a method has lower computational complexity than the method of determining the parameter values ​​of the first parameter and the second parameter based on the filtered original pixel set.

[0076] In step S5052, the prediction model is constructed based on the parameter value of the first parameter and the parameter value of the second parameter.

[0077] In the present embodiment, a concept of a reference pixel subset (i.e., a target pixel set described in the above embodiment) is proposed. In predictive coding, reference pixels are usually used to predict a current coding block (i.e., a block to be coded in the step S311). Assuming that reference points in a certain area of ​​reference pixels are more effective for predicting a current coding block, a reference pixel subset constituted by these points can be used to predict a current coding block, thus preventing coding performance loss, and even improving coding performance and reducing complexity at the same time. Therefore, in the present embodiment, before any step (e.g., downsampling and filtering, etc.) before performing prediction on a current coding block, a reference pixel sample is appropriately selected to obtain a reference pixel subset, and predictive coding is performed based on the subset, thereby improving coding efficiency of predictive coding.

[0078] In predictive coding, reference pixels are usually used to predict a current coding block. For example, available reference pixels in the upper and left regions of the current coding block are used to predict the current coding block, and these reference pixels are usually composed of pixels in one or more columns to the left of the current coding block and one or more rows to the upper side of the current coding block, and these pixels are generally reconstructed pixels, i.e., pixels that have completed predictive coding. In some cases, downsampling needs to be performed on these pixels to construct the reference pixels.

[0079] For example, as shown in FIG. 3C, the currently coded block is a 2N*2N coding block, whose original pixel set includes 2N adjacent reconstructed pixels to its upper side and 2N adjacent reconstructed pixels to its left side.

[0080] For a coding block, the importance and correlation of each reference pixel is different. At the same time, reference pixels located near the current coding block may have similar effects on the prediction of the current coding block. That is, a certain pixel point in an area near the current coding block may be very effective in prediction, while other pixel points may not be beneficial for the prediction of the current coding block. By using only the selected relatively important reference pixel points as a reference pixel subset (i.e., the target pixel set), and using the reference pixel subset to perform prediction of the current coding block, a good prediction result with lower calculation complexity can be obtained. In the embodiment of the present application, when selecting reference pixels, a reference pixel subset is constructed according to factors such as the importance and correlation of each reference pixel point, and the reference pixel subset is used to perform prediction of the current coding block, thereby reducing the calculation complexity and improving the coding performance.

[0081] For example, obtain original neighboring reference pixels (i.e., reference pixels in the original pixel set) from the left or top region of the current coding block, and then select pixels that satisfy a condition from the multiple original neighboring reference pixels according to the positions and / or pixel features (such as intensity) of the original neighboring reference pixels to obtain a reference pixel subset. After obtaining the reference pixel subset, the subset can be used in any process before performing prediction on the current coding block, for example, performing downsampling or filtering on the subset. In another example, the subset is used in a cross-component linear model (CCLM) prediction method. It can be understood that the cross-component linear model (CCLM) is an encoding tool in H.266 / VVC. The model uses the reconstructed luminance (Y) to predict the corresponding chrominance (C). For example, the chrominance value C′ of the current coding block can be derived through the following linear model equation (2):

number

[0082] In the CCLM, a specific point in the original reference region can be selected. Then, the reference pixel subset composed of these points is used to perform further processing. For example, downsampling is performed using the reference pixel subset, and the maximum and minimum values ​​are found from the downsampled reference pixel subset, and the values ​​of the parameters α and β are determined based on the maximum and minimum values. In this way, the method of performing downsampling on the reference pixel subset can reduce the number of downsamplings compared to the method of performing downsampling on the original pixel set, and can also find the maximum and minimum values ​​from the downsampled reference pixel subset more quickly, which can quickly determine the prediction model, and can further improve the speed of data processing. Of course, when the screening conditions are reasonable, the coding efficiency of predictive coding is improved, and in other embodiments, compared with the method of using the original pixel set, the method of using the reference pixel subset here can more quickly derive the values ​​of the parameters α and β, and then predict the current coding block through the prediction model obtained by processing, thereby improving the speed of data processing.

[0083] In the present embodiment, the video encoding is performed using a subset of reference pixels in any process before predicting the current encoding block, thereby reducing the complexity of the video encoding and improving the robustness of the algorithm.

[0084] Based on the above-mentioned embodiment, the embodiment of the present application provides an information processing device, and each unit included in the device and each module included in each unit can be realized by a processor in an electronic device, and of course, can be realized by a specific logic circuit. In the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0085] 6A is an exemplary structural diagram of the configuration of an information processing device according to an embodiment of the present application. As shown in FIG. 6A, the device 60 includes an original pixel set determining module 61, an attribute information determining module 62, a target pixel selecting module 63, and an encoding processing module 64. Here, the original pixel set determining module 61 is configured to determine an original pixel set of a block to be encoded in a current video frame. The attribute information determining module 62 is configured to determine attribute information of each original pixel of the original pixel set. The target pixel selecting module 63 is configured to select an original pixel whose attribute information satisfies a screening condition to obtain a target pixel set. The encoding processing module 64 is configured to perform an encoding process on the block to be encoded according to the attribute information of each original pixel of the target pixel set.

[0086] In another embodiment, the original pixel set determination module 61 is configured to determine at least one reconstructed pixel outside the block to be encoded as an original pixel within the original pixel set, and the encoding processing module 64 is configured to perform predictive encoding, transform encoding, or entropy encoding on the block to be encoded based on attribute information of each reconstructed pixel of the target pixel set.

[0087] In another embodiment, the original pixel set determination module 61 is configured to determine at least one encoded pixel in the encoded block as an original pixel in the original pixel set, and the encoding processing module 64 is configured to perform predictive encoding, transform encoding, or entropy encoding on the target pixel set based on attribute information of each encoded pixel of the target pixel set.

[0088] In another embodiment, as shown in FIG. 6B , the target pixel selection module 63 includes a correlation degree determination unit 631 configured to determine a correlation degree between each reconstructed pixel and the block to be encoded based on attribute information of the reconstructed pixels in the original pixel set, and a selection unit 632 configured to select reconstructed pixels whose correlation degrees satisfy a screening condition to obtain a target pixel set.

[0089] In another embodiment, the correlation determination unit 631 is configured to determine a positional relationship between a reconstructed pixel in the original pixel set and the block to be coded, and to determine the positional relationship as a correlation between the reconstructed pixel and the block to be coded.

[0090] In another embodiment, the target pixel selection module 63 is configured to select, from the original pixel set, reconstructed pixels having image content within a preset range as target pixels to obtain the target pixel set.

[0091] In another embodiment, the image component is a luma component or a chroma component.

[0092] In another embodiment, as shown in FIG. 6C , the encoding processing module 64 includes a pre-processing unit 641 configured to perform pre-processing on attribute information of each reconstructed pixel of the target pixel set to obtain a pre-processed target pixel set, and a predictive encoding unit 642 configured to perform predictive encoding on the block to be encoded based on the attribute information of each reconstructed pixel of the pre-processed target pixel set.

[0093] In another embodiment, the pre-processing unit 641 is configured to perform a downsampling operation on the attribute information of each reconstructed pixel of the target pixel set to obtain a downsampled target pixel set, or to perform a filtering operation on the attribute information of each reconstructed pixel of the target pixel set to obtain a filtered target pixel set.

[0094] In another embodiment, as shown in FIG. 6D , the predictive encoding unit 641 includes a model construction subunit 6411 configured to construct a predictive model based on attribute information of each reconstructed pixel in the downsampled target pixel set or the filtered target pixel set, the predictive model being used to represent a predictive relationship between multiple image components of pixels to be encoded in the block to be encoded, and a predictive encoding subunit 6412 configured to perform predictive encoding on the block to be encoded based on the predictive model.

[0095] In another embodiment, the model construction subunit 6411 is configured to determine a parameter value of a first parameter and a parameter value of a second parameter of a prediction model based on chrominance and luma components of each reconstructed pixel of the downsampled target pixel set or the filtered target pixel set, and construct the prediction model based on the parameter value of the first parameter and the parameter value of the second parameter.

[0096] In another embodiment, the original pixel set determination module 61 is configured to determine N reconstructed pixels (N is a preset integer greater than or equal to 1) within a preset range in which the block to be encoded is located as the original pixel set.

[0097] In another embodiment, the original pixel set determination module 61 is configured to determine reconstructed pixels in a reference row or reference example adjacent to the block to be coded as the original pixel set.

[0098] The above description of the apparatus embodiment is similar to the above description of the method embodiment, and has the same beneficial effects as the method embodiment. The technical details not disclosed in the apparatus embodiment of the present application can be understood by referring to the description of the method embodiment of the present application.

[0099] In addition, in the embodiments of the present application, when the above information processing method is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the essential part of the technical solution of the embodiments of the present application, i.e., the part that contributes to the related art, can be embodied in the form of a software product, and the computer software product is stored in one storage medium and includes several instructions for causing an electronic device (which may be a mobile phone, a tablet computer, a personal computer, a personal digital assistant, a navigator, a digital telephone, a video telephone, a television, a sensing device, a server, etc.) to execute all or part of the method described in each embodiment of the present application. The above storage medium includes various media that can store program code, such as a U disk, a mobile hard disk, a read only memory (ROM), a magnetic disk, or an optical disk. Thus, the embodiments of the present application are not limited to a specific combination of hardware and software.

[0100] An embodiment of the present application provides an electronic device comprising a memory and a processor, the memory being configured to store a computer program executable by the processor, and the processor, when executing the program, realizing steps in the information processing method according to the embodiment described above.

[0101] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the information processing method according to the above embodiment.

[0102] It should be noted here that the above description of the storage medium and device embodiments is similar to the description of the method embodiments above, and has the same beneficial effects as the method embodiments. The technical details not disclosed in the storage medium and device embodiments of the present application can be understood by referring to the description of the method embodiments of the present application.

[0103] It should be noted that FIG. 7 is a schematic diagram of hardware entities of an electronic device according to an embodiment of the present application, and as shown in FIG. 7, the electronic device 700 includes a memory 701 and a processor 702, the memory 701 is configured to store a computer program executable by the processor 702, and the processor 702 is configured to realize steps in the information processing method according to the above embodiment when executing the program.

[0104] It should be noted that the memory 701 is configured to store instructions and applications executable by the processor 702, and can be realized by a flash memory (FLASH) or a random access memory (RAM), which can also cache data being processed or processed by the processor 702 and each module of the electronic device 700 (e.g., image data, audio data, voice communication data, video communication data, etc.).

[0105] It should be understood that "one embodiment" or "an embodiment" referred to throughout this specification means that a particular feature, structure, or characteristic associated with the embodiment is included in at least one embodiment of the present application. Thus, "in one embodiment" or "in one embodiment" in this specification do not necessarily refer to the same embodiment. Moreover, these particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that in various embodiments of the present application, the magnitude of the sequence numbers of each process above does not mean the execution sequence, and the execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The numbers of the embodiments of the present application described above do not represent the superiority or inferiority of the embodiments, but are for convenience of explanation.

[0106] It should be noted that, as used herein, the terms "comprising," "including," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that includes a set of elements includes not only those elements, but also other elements not expressly listed or inherent elements of such process, method, article, or apparatus. Unless otherwise limited, an element defined with the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0107] In some embodiments provided in the present application, it should be understood that the disclosed devices and methods may be realized in other ways. The above-described device embodiments are merely exemplary, and the division of the units is merely a logical division of functions. In actual implementation, other division methods may exist, for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not implemented. Furthermore, the mutual coupling or direct coupling or communication connection between each component shown or described may be an indirect coupling or communication connection via some interfaces, devices or units, and may be in an electrical, mechanical or other form.

[0108] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, the units may be located in one place or distributed among multiple network units, and some or all of the units may be selected according to actual needs to achieve the objective of the technical solution of the embodiment.

[0109] In addition, the functional units in each embodiment of the present invention may all be integrated into one second processing unit, each unit may be used individually as one unit, or two or more units may be integrated into one unit. The integrated unit may be embodied in the form of hardware or a combination of hardware and software functional units.

[0110] It will be obvious to those skilled in the art that all or some of the steps of implementing the above method embodiments can be completed by hardware associated with program instructions, and the program can be stored in a computer-readable storage medium, and when the program is executed, the steps of the above method embodiments are performed, and the storage medium includes various media capable of storing program code, such as removable storage, read-only memory (ROM), magnetic memory, or optical disk.

[0111] Alternatively, the above-mentioned integrated units of the present application may be realized in the form of a software functional module and stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the essential part of the technical solution of the embodiments of the present application, i.e., the part that contributes to the related art, may be embodied in the form of a software product, and the computer software product is stored in a storage medium and includes several instructions for causing an electronic device (which may be a mobile phone, a tablet computer, a personal computer, a personal digital assistant, a navigator, a digital telephone, a video telephone, a television, a sensing device, a server, etc.) to execute all or part of the method described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program code, such as removable storage, ROM, RAM, magnetic memory, or optical disk.

[0112] The above content is merely a specific embodiment of the present application, and the protection scope of the present application is not limited thereto. Any modifications or replacements that a person skilled in the art can easily think of within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims. [Industrial Applicability]

[0113] In the embodiment of the present application, an original pixel whose attribute information satisfies a screening condition is selected from an original pixel set of a block to be coded to obtain a target pixel set, and an encoding process is performed on the block to be coded according to the attribute information of each original pixel of the target pixel set. In this way, the encoding process is performed on the block to be coded according to the attribute information of a part of the selected original pixels, thereby reducing the computational complexity of the encoding process, thereby improving the efficiency of video coding, and further improving the fluency of the video played by the user.

Claims

1. 1. An information processing method performed by a decoder, comprising: determining a sample set for a currently decoded processing block; Selecting at least one sample from the sample set based on attribute information of samples in the sample set to obtain a target sample set, the attribute information including position information, a luminance component, and a chrominance component; performing a filtering process on the target sample set based on the attribute information to obtain a filtered target sample set; constructing a predictive model based on the filtered target sample set, the predictive model being used to represent a predictive relationship between two or more image components within the processing block; and determining a predicted value for the processing block based on the prediction model; and decoding the processing block based on a prediction value of the processing block; Determining a sample set of the currently decoded processing block comprises: determining at least one reconstructed sample outside the processing block as a sample within the sample set; Constructing a predictive model based on the filtered target sample set includes: determining a parameter value for a first parameter and a parameter value for a second parameter of a predictive model based on chrominance and luminance components of samples in the filtered target sample set, the predictive model being a linear model represented by the first parameter and the second parameter; Determining parameter values ​​for a first parameter and a second parameter of a predictive model based on chrominance and luminance components of samples in the filtered target sample set includes: obtaining maximum and minimum values ​​of luminance components of samples in the filtered target sample set; determining a parameter value for a first parameter and a parameter value for a second parameter of a predictive model based on maximum and minimum values ​​of the chrominance and luminance components of samples in the filtered target sample set.

2. the set of samples includes at least one row of reconstructed samples adjacent to an upper side of the processing block and / or at least one column of reconstructed samples adjacent to a left side of the processing block. The information processing method according to claim 1 .

3. the attribute information includes at least a positional relationship between the reconstructed sample in the sample set and the processing block; The information processing method according to claim 2 .

4. Selecting at least one sample from the sample set based on attribute information of samples in the sample set to obtain a target sample set, determining a positional relationship between the reconstructed samples in the sample set and the processing block based on attribute information of the reconstructed samples in the sample set; and acquiring a target sample set based on the positional relationship. The information processing method according to claim 3.

5. Selecting at least one sample from the sample set based on attribute information of samples in the sample set to obtain a target sample set, selecting one or more reconstructed samples from at least one row of reconstructed samples adjacent to an upper side of the processing block based on the attribute information, and / or selecting one or more reconstructed samples from at least one column of reconstructed samples adjacent to a left side of the processing block based on the attribute information; the target sample set includes selected reconstruction samples; The information processing method according to claim 2 .

6. Selecting one or more reconstructed samples from at least one row of reconstructed samples adjacent an upper side of the processing block based on the attribute information includes: performing a downsampling operation on at least one row of reconstructed samples adjacent to an upper side of the processing block based on the attribute information to obtain one or more reconstructed samples; Selecting one or more reconstructed samples from at least one column of reconstructed samples adjacent to a left side of the processing block based on the attribute information includes: performing a downsampling operation on at least one column of reconstructed samples adjacent to a left side of the processing block based on the attribute information to obtain one or more reconstructed samples. The information processing method according to claim 5.

7. The filtering process is a downsampling process. The information processing method according to claim 1 .

8. an input of the filtering process includes the reconstructed sample and one or more reconstructed samples adjacent to the reconstructed sample; The information processing method according to claim 1 .

9. 1. An information processing method performed by an encoder, comprising: determining a sample set of a block to be coded within a current video frame; Selecting at least one sample from the sample set based on attribute information of samples in the sample set to obtain a target sample set, the attribute information including position information, a luminance component, and a chrominance component; performing a filtering process on the target sample set based on the attribute information to obtain a filtered target sample set; building a prediction model based on the filtered target sample set, the prediction model being used to represent a predictive relationship between two or more image components within the block to be coded; determining a prediction value for the block to be coded based on the prediction model; performing an encoding process on the block to be encoded based on a predicted value of the block to be encoded; Determining a sample set of a block to be coded within the current video frame includes: determining at least one reconstructed sample outside the block to be coded as a sample in the sample set; Constructing a predictive model based on the filtered target sample set includes: determining a parameter value for a first parameter and a parameter value for a second parameter of a predictive model based on chrominance and luminance components of samples in the filtered target sample set, the predictive model being a linear model represented by the first parameter and the second parameter; Determining parameter values ​​for a first parameter and a second parameter of a predictive model based on chrominance and luminance components of samples in the filtered target sample set includes: obtaining maximum and minimum values ​​of luminance components of samples in the filtered target sample set; determining a parameter value for a first parameter and a parameter value for a second parameter of a predictive model based on the maximum and minimum values ​​of the chrominance components and the luminance components of samples in the filtered target sample set.

10. A decoder configured to carry out a method for processing information according to any one of claims 1 to 8.

11. An encoder configured to perform a method for processing information according to claim 9.

Images