Information processing method and device, equipment and storage medium

Abstract

The embodiment of the invention discloses an information processing method and device, equipment and a storage medium, and the method comprises the steps: determining an original pixel set of a to-be-coded block in a current video frame; determining attribute information of each original pixel in the original pixel set; selecting the original pixels of which the attribute information meets the screening condition to obtain a target pixel set; and coding the block to be coded based on the attribute information of each original pixel in the target pixel set.

Description

[0001] Divisional Statement This application is a divisional application of PCT / CN2019 / 087300, filed on May 16, 2019, entitled "Information Processing Method and Device, Equipment, and Storage Medium", which entered the Chinese national phase as Chinese Patent Application No. 201980057448.8. TECHNICAL FIELD

[0002] Embodiments of the present application relate to video coding technology, and relate to but are not limited to an information processing method and device, equipment, and storage medium. BACKGROUND

[0003] In recent years, video services have rapidly developed in the Internet and mobile communication fields. Video services need to encode source video data first, and then transmit the encoded video data to user terminals through channels of the Internet or mobile communication networks.

[0004] For users, the smoothness of a video directly affects the user's video viewing experience. The computational complexity of predictive coding in video encoding directly affects the smoothness of the video. SUMMARY

[0005] Embodiments of the present application provide an information processing method and device, equipment, and storage medium to solve at least one problem in the related art.

[0006] The technical solutions of embodiments of the present application are implemented as follows: In a first aspect, an embodiment of the present application provides an information processing method, which includes: determining an original pixel set of a to-be-encoded block in a current video frame; determining attribute information of each original pixel in the original pixel set; selecting original pixels whose attribute information meets a screening condition to obtain a target pixel set; and performing encoding processing on the to-be-encoded block based on the attribute information of each original pixel in the target pixel set.

[0007] In a second aspect, an embodiment of the present application provides an information processing device, which includes: an original pixel set determination module configured to determine an original pixel set of a to-be-encoded block in a current video frame; an attribute information determination module configured to determine attribute information of each original pixel in the original pixel set; a target pixel selection module configured to select original pixels whose attribute information meets a screening condition to obtain a target pixel set; and an encoding processing module configured to perform encoding processing on the to-be-encoded block based on the attribute information of each original pixel in the target pixel set.

[0008] In a third aspect, an electronic device is provided, which includes a memory and a processor. The memory stores a computer program capable of running on the processor. The processor implements the steps of the above information processing when executing the program.

[0009] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the steps of the above information processing method.

[0010] In the embodiments of the present application, the original pixels whose attribute information meets the screening condition are selected from the original pixel set of the to-be-encoded block to obtain a target pixel set. The to-be-encoded block is encoded based on the attribute information of each original pixel in the target pixel set. In this way, the to-be-encoded block is encoded based on the attribute information of the selected part of the original pixels (that is, the original pixels in the target pixel set), instead of the attribute information of all the original pixels in the original pixel set. Thus, the computational complexity of the encoding process can be reduced, the video encoding efficiency is improved, and the user's video watching fluency is improved. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 A schematic diagram of the component structure of the network architecture for the video encoding and decoding of the embodiments of the present application; Figure 2A A schematic diagram of the component structure of the video encoder of the embodiments of the present application; Figure 2B A schematic diagram of the component structure of the video decoder of the embodiments of the present application; Figure 3A A schematic diagram of the implementation flow of the information processing method of the embodiments of the present application; Figure 3B A schematic diagram of the implementation flow of another information processing method of the embodiments of the present application; Figure 3C A schematic diagram of the relationship between the to-be-encoded block and the reference pixel of the embodiments of the present application; Figure 3D A schematic diagram of the implementation flow of another information processing method of the embodiments of the present application; Figure 4 A schematic diagram of the implementation flow of another information processing method of the embodiments of the present application; Figure 5A A schematic diagram of the implementation flow of another information processing method of the embodiments of the present application; Figure 5B A schematic diagram of the implementation flow of the method for constructing a prediction model of the embodiments of the present application; Figure 6A A schematic diagram of the component structure of the information processing device of the embodiments of the present application; Figure 6BThis is a schematic diagram of the composition of another information processing device according to an embodiment of this application; Figure 6C This is a schematic diagram of the composition structure of another information processing device according to an embodiment of this application; Figure 6D This is a schematic diagram of the composition structure of another information processing device according to an embodiment of this application; Figure 7 This is a schematic diagram of a hardware entity of an electronic device according to an embodiment of this application. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0013] This embodiment first provides a network architecture for video encoding and decoding. Figure 1 This is a schematic diagram of the network architecture for video encoding and decoding in an embodiment of this application, as shown below. Figure 1 As shown, the network architecture includes one or more electronic devices 11 to 1N and a communication network 01, wherein the electronic devices 11 to 1N can perform video interaction through the communication network 01. The electronic devices can be various types of devices with video encoding and decoding capabilities, such as mobile phones, tablets, personal computers, personal digital assistants, navigators, digital phones, video phones, televisions, sensing devices, servers, etc.

[0014] The electronic device has video encoding and decoding capabilities, and generally includes a video encoder and a video decoder, for example, see [link to relevant documentation]. Figure 2AAs shown, the constituent structure of the video encoder 21 includes a transform and quantization unit 211, an intra estimation unit 212, an intra prediction unit 213, a motion compensation unit 214, a motion estimation unit 215, an inverse transform and inverse quantization unit 216, a filter control analysis unit 217, a filter unit 218, an encoding unit 219, a decoded picture buffer 210, and the like. The filter unit 218 can implement deblocking filtering and sample adaptive offset (SAO) filtering, and the encoding unit 219 can implement header information encoding and context-based adaptive binary arithmatic coding (CABAC). For input source video data, a current video frame to be encoded can be obtained by dividing a coding tree unit (CTU), and then the intra or inter prediction is performed on the to-be-encoded block to obtain residual information, which is then transformed by the transform and quantization unit 211, including transforming the residual information from the pixel domain to the transform domain, and quantizing the obtained transform coefficients to further reduce the bit rate. The intra estimation unit 212 and the intra prediction unit 213 are used to perform intra prediction on the to-be-encoded block, for example, to determine the intra prediction mode used to encode the to-be-encoded block. The motion compensation unit 214 and the motion estimation unit 215 are used to perform inter prediction encoding of the to-be-encoded block relative to one or more blocks in one or more reference frames to provide temporal prediction information. The motion estimation unit 215 is used to estimate the motion vector, which can estimate the motion of the to-be-encoded block, and then the motion compensation unit 214 performs motion compensation based on the motion vector. After the intra prediction mode is determined, the intra prediction unit 213 is also used to provide the selected intra prediction data to the encoding unit 219, and the motion estimation unit 215 also sends the calculated determined motion vector data to the encoding unit 219. In addition, the inverse transform and inverse quantization unit 216 is used for reconstruction of the to-be-encoded block, and the reconstructed residual block is obtained in the pixel domain, the blocking artifact is removed by the filter control analysis unit 217 and the filter unit 218, and then the reconstructed residual block is added to a predictive block in one of the frames of the decoded picture buffer 210 to generate a reconstructed video encoding block. The encoding unit 219 is used to encode various encoding parameters and quantized transform coefficients, and in the CABAC-based encoding algorithm, the context content can be based on the adjacent encoded block, which can be used to encode information indicating the determined intra prediction mode, and output the code stream of the video data. The decoded picture buffer 210 is used to store the reconstructed video encoding block for prediction reference. As the video encoding proceeds, new reconstructed video encoding blocks are continuously generated, which are all stored in the decoded picture buffer 210.

[0015] The video decoder 22 corresponding to the video encoder 21 has the following structure: Figure 2B As shown, it includes: a decoding unit 221, an inverse transform and inverse quantization unit 222, an intra-frame prediction unit 223, a motion compensation unit 224, a filtering unit 225, and a decoded image buffer unit 226, etc. The decoding unit 221 can perform header information decoding and CABAC decoding, and the filtering unit 225 can perform deblocking filtering and SAO filtering. The input video signal is processed... Figure 2A After encoding, the video signal bitstream is output; this bitstream is input to the video decoder 22, first passing through the decoding unit 221 to obtain the decoded transform coefficients; these transform coefficients are then processed by the inverse transform and inverse quantization unit 222 to generate residual blocks in the pixel domain; the intra-frame prediction unit 223 can be used to generate prediction data for the current decoded block based on the determined intra-frame prediction mode and data from previously decoded blocks in the current frame or image; the motion compensation unit 224 determines the prediction information for the current decoded block by analyzing motion vectors and other associated syntax elements, and uses this information... Predictive information is used to generate a predictive block for the current decoded block being decoded; a decoded video block is formed by summing the residual block from the inverse transform and inverse quantization unit 222 with the corresponding predictive block generated by the intra-prediction unit 223 or the motion compensation unit 224; the decoded video block is passed through the filtering unit 225 to remove block artifacts, thereby improving video quality; then the decoded video block is stored in the decoded image buffer unit 226, which stores reference images for subsequent intra-prediction or motion compensation, and is also used for the output display of the video signal.

[0016] Based on this, the technical solution of this application will be further described in detail below with reference to the accompanying drawings and embodiments. The information processing method provided in the embodiments of this application can be applied to both the video encoder 21 and the video decoder 22, and the embodiments of this application do not specifically limit it in this regard.

[0017] This application provides an information processing method applied to an electronic device. The function implemented by this method can be achieved by the processor in the electronic device calling program code. Of course, the program code can be stored in a computer storage medium. It can be seen that the electronic device includes at least a processor and a storage medium.

[0018] Figure 3A This is a schematic diagram illustrating the implementation flow of the information processing method in an embodiment of this application, as shown below. Figure 3A As shown, the method includes steps S301 to S304: S301. Determine the original pixel set of the block to be encoded in the current video frame; It should be noted that the to-be-encoded block refers to an image region in a current video frame that needs to be encoded, and the type of the to-be-encoded block is different for different encoding processing, and the content contained in the original pixel set is also different. For example, when the encoding processing is prediction encoding, the to-be-encoded block is an image block including a luminance component and a chrominance component, and the reconstructed pixels outside the to-be-encoded block can be determined as the original pixel set, that is, the reference pixels used when the to-be-encoded block is prediction encoded, but the content of the original pixel set is not limited, which is only an example; when the encoding processing is transform encoding, the to-be-encoded block is a residual block including residual values, that is, a residual block output after prediction encoding, and the pixels in the residual block can be determined as the original pixel set, but the content of the original pixel set is not limited, which is only an example; when the encoding processing is entropy encoding, the to-be-encoded block is a coefficient block including transform coefficients, that is, a coefficient block output after transform encoding, and the pixels in the coefficient block can be determined as the original pixel set, but the content of the original pixel set is not limited, which is only an example.

[0019] S302, attribute information of each original pixel in the original pixel set is determined. It can be understood that when the original pixel set is the reconstructed pixels outside the to-be-encoded block, the attribute information of each original pixel in the original pixel set is the position information, luminance component, chrominance component, etc. of the reconstructed pixels; when the original pixel set is the pixels in the residual block, the attribute information of each original pixel in the original pixel set is the position information, residual value, etc. of the pixels in the residual block; and when the original pixel set is the pixels in the coefficient block, the attribute information of each original pixel in the original pixel set is the position information of part of the pixels in the coefficient block, transform coefficient, etc.

[0020] S303, original pixels with attribute information satisfying a filtering condition are selected to obtain a target pixel set. S304, the to-be-encoded block is encoded based on the attribute information of each original pixel in the target pixel set.

[0021] It should be noted that the type of the encoding processing is not limited here, for example, the encoding processing can be one of prediction encoding, transform encoding and entropy encoding.

[0022] In the embodiments of the present application, before the encoding processing of the to-be-encoded block, part of the original pixels (i.e., the target pixel set) are selected from the original pixel set of the to-be-encoded block, and then the to-be-encoded block is encoded based on the attribute information of the part of the original pixels (instead of all the original pixels in the original pixel set), so as to reduce the complexity of the encoding processing, reduce the time spent in the encoding processing, and further improve the fluency of the user watching the video.

[0023] It should be noted that the information processing method provided in the embodiments of the present application actually improves the traditional video encoding technology (for example, video encoding standards H.264, H.265, etc.), that is, the concept of subset is added on the basis of the traditional video encoding technology. For example, before the prediction encoding of the to-be-encoded block, part of the original pixels (i.e., the subset) are selected from the original pixel set (including all the reconstructed pixels adjacent to the to-be-encoded block), and then the part of the original pixels is subjected to down-sampling processing or filtering processing, instead of the original pixel set, so that the complexity of the prediction encoding can be reduced and the encoding efficiency can be improved.

[0024] The embodiments of the present application provide another information processing method, Figure 3B For the implementation flowchart of another information processing method of the embodiments of the present application, as shown in Figure 3B The method includes steps S311 to S314: S311, determining an original pixel set of a to-be-encoded block in a current video frame, each original pixel in the original pixel set being a reconstructed pixel outside the to-be-encoded block; It can be understood that at least one reconstructed pixel outside the to-be-encoded block is determined as an original pixel in the original pixel set. Generally, the reconstructed pixel outside the to-be-encoded block is referred to as a reference pixel, because when the to-be-encoded block is predicted, the attribute information (such as luminance component, chrominance component, etc.) of the reconstructed pixel is needed to predict the to-be-predicted pixel in the to-be-encoded block. Therefore, the reconstructed pixel is a pixel point that has been predicted. For example, Figure 3C The original pixel set can include all the reconstructed pixels in the upper row region 31 of the to-be-encoded block 30, or the original pixel set can also include all the reconstructed pixels in the left column region 32 of the to-be-encoded block 30. Generally, the original pixel set includes the pixel identifier of the reconstructed pixel, for example, the pixel number, etc.

[0025] S312, determining attribute information of each reconstructed pixel in the original pixel set; For example, the position information, the luminance component, the chrominance component, and other attribute information of each reconstructed pixel in the original pixel set can be obtained based on the video data of the current video frame.

[0026] In S313, the reconstructed pixels whose attribute information meets the screening condition are selected to obtain a target pixel set. Generally, the screening condition is used to screen the reconstructed pixels that are strongly related to the to-be-encoded block, so as to obtain the target pixel set. It should be noted that the attribute information of the reconstructed pixels is included in the target pixel set.

[0027] In S314, the to-be-encoded block is predicted and encoded based on the attribute information of each reconstructed pixel in the target pixel set.

[0028] It can be understood that the prediction encoding is one processing step in the encoding process, mainly uses the spatial or temporal correlation of adjacent pixels to predict the pixel being encoded at present by using the transmitted pixel, and then encodes and transmits the difference (i.e., prediction error) between the predicted value and the real value. For example, in the prediction encoding, the attribute information of each reconstructed pixel in the target pixel set is used to perform down-sampling processing or filtering processing on the target pixel set; for another example, the attribute information of each reconstructed pixel in the target pixel set is used to determine the prediction model of the to-be-encoded block.

[0029] In other embodiments, the to-be-encoded block can also be transformed and encoded or entropy encoded based on the attribute information of each reconstructed pixel in the target pixel set.

[0030] It can be understood that the transform encoding is another processing step in the encoding process, mainly is to form data (coefficients) in the transform domain by using certain transform (such as discrete cosine transform, discrete sine transform, Hadamard transform, etc.) on the image described in the spatial domain, so as to change the data distribution and reduce the amount of effective data. For example, in the transform encoding, the attribute information of each reconstructed pixel in the target pixel set is used to derive the information of the transform encoding mode. The prediction encoding and the transform encoding are two different compression encoding methods. If the two methods are combined together, a hybrid encoding is formed, and in the hybrid encoding framework, the entropy encoding is a processing step after the prediction encoding and the transform encoding, which is a variable length coding (VLC) and can further improve the compression efficiency of the hybrid encoding. In the entropy encoding, the attribute information of each reconstructed pixel in the target pixel set can be used to derive the information of the entropy encoding context model.

[0031] In the embodiments of the present application, the reconstructed pixels with attribute information satisfying the screening condition are selected from the original pixel set of the to-be-encoded block to obtain a target pixel set; and the to-be-encoded block is predicted encoded, transform encoded or entropy encoded based on the attribute information of each reconstructed pixel in the target pixel set. In this way, the to-be-encoded block is encoded based on the attribute information of the selected part of the original pixels (rather than all the original pixels in the original pixel set), which can reduce the computational complexity of the encoding process, thereby improving the video encoding efficiency and further improving the fluency of the user watching the video.

[0032] The embodiments of the present application provide another information processing method, Figure 3D For the implementation flowchart of another information processing method of the embodiments of the present application, as shown in Figure 3D The method includes steps S321 to S324: S321, at least one to-be-encoded pixel in a to-be-encoded block in a current video frame is determined as an original pixel to obtain an original pixel set; It should be noted that the attribute information of the to-be-encoded pixels in the to-be-encoded block is different in the encoding process at different stages. For example, steps S321 to S324 perform prediction encoding, and the attribute information of the to-be-encoded pixels includes luminance components, chrominance components, position information, etc. For another example, steps S321 to S324 perform transform encoding, and the attribute information of the to-be-encoded pixels includes residual values output after prediction encoding, position information, etc., that is, the to-be-encoded block is a residual block. For another example, steps S321 to S324 perform entropy encoding, and the attribute information of the to-be-encoded pixels includes transform coefficients output after transform encoding, position information, etc., that is, the to-be-encoded block is a coefficient block.

[0033] S322, the attribute information of each original pixel in the original pixel set is determined; S323, the original pixels with attribute information satisfying the screening condition are selected to obtain a target pixel set; For example, in transform encoding, part of the to-be-encoded pixels in the residual block can be sampled based on the position information of each to-be-encoded pixel in the residual block, and the sampled part of the to-be-encoded pixels is determined as the target pixel set. For another example, in transform encoding, the to-be-encoded pixels with residual values greater than a preset threshold are selected from the residual block to obtain a target pixel set. For another example, in entropy encoding, part of the to-be-encoded pixels in the coefficient block can be sampled based on the position information of each to-be-encoded pixel in the coefficient block, and the sampled part of the to-be-encoded pixels is determined as the target pixel set.

[0034] S324, performing prediction encoding, transform encoding or entropy encoding on the to-be-encoded block based on the attribute information of each reconstructed pixel in the target pixel set.

[0035] In the embodiments of the present application, only part of the to-be-encoded pixels in the to-be-encoded block are processed, for example, prediction encoding, transform encoding or entropy encoding, instead of all the to-be-encoded pixels in the to-be-encoded block, so that the complexity of the encoding process can be effectively reduced, the data processing speed can be improved, the encoding efficiency can be improved, and the fluency of the user watching the video can be improved.

[0036] The embodiments of the present application provide another information processing method, Figure 4 For the implementation flowchart of another information processing method of the embodiments of the present application, as shown in the figure, Figure 4 The method comprises steps S401 to S406: S401, determining N reconstructed pixels in a preset range in which the to-be-encoded block is located in the current video frame as the original pixel set; wherein N is a preset integer greater than or equal to 1; For example, the reconstructed pixels in the reference row or the reference column adjacent to the to-be-encoded block are determined as the original pixel set. Here, the number of the reference row and the reference column is not limited, which can be one or more reference rows, or one or more reference columns. As shown in the figure, Figure 3C The reconstructed pixels in the last row area 31 of the to-be-encoded block 30 can be determined as the original pixel set, or the reconstructed pixels in the left column area 32 of the to-be-encoded block 30 can also be determined as the original pixel set. It should be noted that the value of N can be pre-set, for example, N = the side length of the to-be-encoded block + n, n is an integer greater than or equal to 0, and the side length of the to-be-encoded block can be represented by the number of pixels.

[0037] It should be further noted that step S401 is actually an implementation example of step S301 in the above embodiments.

[0038] S402, determining the attribute information of each reconstructed pixel in the original pixel set; S403, determining the relevance of each reconstructed pixel to the to-be-encoded block based on the attribute information of the reconstructed pixels in the original pixel set; Understandably, the correlation is used to characterize the degree of association between the reconstructed pixel and the block to be encoded (or the pixel to be predicted in the block to be encoded). For example, the positional relationship between the reconstructed pixel in the original pixel set and the block to be encoded can be determined, and this positional relationship can be defined as the correlation between the reconstructed pixel and the block to be encoded. In other embodiments, reconstructed pixels whose image components are within a preset range can be selected from the original pixel set as target pixels to obtain the target pixel set; wherein the image components are luminance components or chrominance components. For example, the preset range is... ,in, and These refer to the maximum and minimum values ​​of the image components in the original pixel set, respectively. The stronger the correlation between the reconstructed pixel and the block to be encoded, the more similar the attribute information of the reconstructed pixel is to the attribute information of the predicted pixels in the block to be encoded. Therefore, based on the correlation between the reconstructed pixel and the block to be encoded, reliable reconstructed pixels can be quickly and effectively selected as the reconstructed pixels in the target pixel set. Furthermore, when selecting reconstructed pixels that meet the screening criteria based on correlation, some reconstructed pixels with low correlation are filtered out. Therefore, the reconstructed pixels in the resulting target pixel set are all pixels strongly correlated with the block to be encoded. Thus, predictive encoding of the block to be encoded is performed based on the attribute information of each reconstructed pixel in the target pixel set, improving the robustness of the algorithm.

[0039] S404. Select the reconstructed pixels whose correlation meets the filtering conditions to obtain the target pixel set; For example, assuming the distance between the reconstructed pixel and the block to be encoded is used to characterize the correlation between them, then reconstructed pixels with a distance greater than or equal to a preset distance threshold can be selected as reconstructed pixels in the target pixel set. It should be noted that steps S403 and S404 are actually an implementation example of step S303 in the above embodiments.

[0040] S405. Preprocess the attribute information of each reconstructed pixel in the target pixel set to obtain the preprocessed target pixel set. Understandably, if only some reference pixels (i.e., reconstructed pixels in the target pixel set) are preprocessed before predictive coding of the block to be coded, and predictive coding of the block to be coded is performed based on the preprocessed results, the complexity of preprocessing can be reduced, thereby reducing the computational complexity of predictive coding and improving video smoothness.

[0041] For example, the preprocessing could be downsampling, where only the attribute information of each reconstructed pixel in the target pixel set is downsampled, rather than the original pixel set. Understandably, the purpose of downsampling is to unify the resolution among the multiple image components of the reconstructed pixel, i.e., to make the multiple image components of the reconstructed pixel have the same size in the spatial domain. Therefore, downsampling only a portion of the reconstructed pixels reduces the number of downsampling operations, thereby improving the coding efficiency of predictive coding.

[0042] Alternatively, the preprocessing can also be filtering (e.g., interpolation filtering), that is, filtering the attribute information of each reconstructed pixel in the target pixel set to obtain a filtered target pixel set. Similarly, compared to filtering the original pixel set, this method only filters the attribute information of each reconstructed pixel in the target pixel set, instead of filtering all reconstructed pixels in the original pixel set. This reduces the number of filtering operations and increases the filtering speed.

[0043] S406. Based on the attribute information of each reconstructed pixel in the preprocessed target pixel set, predictive coding is performed on the block to be encoded.

[0044] It should be noted that steps S405 and S406 are actually an implementation example of step S304 in the above embodiments.

[0045] This application provides yet another information processing method. Figure 5A This is a schematic diagram illustrating the implementation flow of another information processing method according to an embodiment of this application, as shown below. Figure 5A As shown, the method includes steps S501 to S506: S501. Determine the original pixel set of the block to be encoded in the current video frame, wherein each reference pixel in the original pixel set is a reconstructed pixel outside the block to be encoded; S502. Determine the attribute information of each reconstructed pixel in the original pixel set; S503. Select the reconstructed pixels whose attribute information meets the filtering conditions to obtain the target pixel set; S504. The attribute information of each reconstructed pixel in the target pixel set is downsampled to obtain the target pixel set after downsampling. S505. Based on the attribute information of each reconstructed pixel in the target pixel set after downsampling, a prediction model is constructed. The prediction model is used to characterize the prediction relationship between multiple image components of the pixel to be encoded in the block to be encoded. It should be noted that the prediction model can be a linear model or a nonlinear model. In this embodiment, the structure of the prediction model is not specifically limited. Based on the attribute information of each reconstructed pixel in the target pixel set after downsampling, a prediction model can be fitted so that multiple image components of the pixels to be encoded in the block conform to the prediction relationship represented in the prediction model.

[0046] For example, suppose the prediction model is as shown in formula (1): (1); In the formula, and Here, Y represents the luminance component, and C' represents the predicted chrominance component. Based on the luminance and chrominance components of multiple reconstructed pixels in the downsampled target pixel set, the model parameters can be quickly derived. and The values ​​are used to obtain a prediction model, and the chromaticity components of the pixels to be predicted in the block to be encoded are then predicted using this model. For example, the maximum and minimum values ​​of the luminance components are found from the downsampled target pixel set, and then the model parameters are determined based on these values. and The value of this is such that, compared to determining model parameters based on the original pixel set... and The former value allows for more efficient acquisition of model parameter values ​​for the prediction model, thereby improving data processing speed. Furthermore, the information processing method provided in this application is beneficial for improving coding efficiency, especially when the filtering conditions are reasonable, the improvement in coding efficiency is more significant.

[0047] In other embodiments, steps S504 and S505 can be replaced by: filtering the attribute information of each reconstructed pixel in the target pixel set to obtain a filtered target pixel set; and constructing a prediction model based on the filtered target pixel set, wherein the prediction model is used to characterize the prediction relationship between multiple image components of the pixels to be encoded in the block to be encoded. Steps S505 and S506 are actually an implementation example of step S406 in the above embodiments.

[0048] S506. Based on the prediction model, predictive coding is performed on the block to be encoded.

[0049] In other embodiments, for step S505, a prediction model is constructed based on the downsampled target pixel set. This prediction model characterizes the prediction relationships between multiple image components of the pixels to be encoded in the block to be encoded, such as...Figure 5B As shown, this can be achieved through the following steps S5051 and S5052: S5051. Based on the chromaticity component and luminance component of each reconstructed pixel in the target pixel set after downsampling, determine the parameter values ​​of the first parameter and the second parameter of the prediction model. Of course, in other embodiments, the parameter values ​​of the first parameter and the second parameter of the prediction model can also be determined based on the chromaticity component and the luminance component of each reconstructed pixel in the filtered target pixel set. Compared with determining the parameter values ​​of the first parameter and the second parameter based on the filtered original pixel set, the former has lower computational complexity.

[0050] S5052. Construct the prediction model based on the parameter values ​​of the first parameter and the second parameter.

[0051] In this application embodiment, a concept of a reference pixel subset (i.e., the target pixel set described in the above embodiments) is proposed. In predictive coding, reference pixels are often used to predict the current coding block (i.e., the block to be encoded as described in step S311 above). Assuming that reference points in certain regions of the reference pixels are more effective for predicting the current coding block, a reference pixel subset consisting of these points is used to predict the current coding block. This reduces complexity without sacrificing or even improving coding performance. Therefore, in this application embodiment, by selecting appropriate reference pixel samples before any processing step (e.g., downsampling and filtering) before predicting the current coding block, a reference pixel subset is obtained. Predictive coding is then performed based on this subset, which improves the coding efficiency of predictive coding.

[0052] In predictive coding, reference pixels are frequently used to predict the current coding block. For example, available reference pixels in the upper and left regions of the current coding block are used to predict it. These reference pixels typically consist of one or more columns to the left of the current coding block and one or more rows to the top of it. These pixels are generally reconstructed pixels, i.e., pixels that have already undergone predictive coding. Sometimes, it is also necessary to downsample these pixels to form reference pixels.

[0053] For example, see Figure 3C As shown, the current block to be encoded is a 2N*2N encoding block. The original pixel set of this encoding block includes the 2N reconstructed pixels adjacent to its upper side and the 2N reconstructed pixels adjacent to its left side.

[0054] For a given coding block, the importance and relevance of each reference pixel are different. Furthermore, reference pixels in close proximity to the current coding block may have a similar impact on the prediction of the current coding block. In other words, some pixels in the vicinity of the current coding block may be very effective in prediction, while others are detrimental. By using only selected, relatively important reference pixels as a subset of reference pixels (i.e., the target pixel set) and using this subset to predict the current coding block, good prediction results with lower computational complexity can be obtained. In this embodiment, when selecting reference pixels, a subset of reference pixels is constructed based on factors such as the importance and relevance of each reference pixel. Using this subset to predict the current coding block reduces computational complexity while improving coding performance.

[0055] For example, original neighboring reference pixels (i.e., reference pixels in the original pixel set) can be obtained from the left or top region of the current coding block; then, pixels that meet the conditions can be selected from multiple original neighboring reference pixels according to the position and / or pixel features (such as intensity), to obtain a subset of reference pixels; after obtaining the subset of reference pixels, the subset can be used in any process before predicting the current coding block, such as downsampling or filtering the subset. Another example is the use of the subset in the Cross-Component Linear Model (CCLM) prediction method. Understandably, the Cross-Component Linear Model (CCLM) is a coding tool in H.266 / VVC. This model uses the reconstructed luminance (Y) to predict the corresponding chrominance (C). For example, the chrominance value C' of the current coding block can be derived using the following linear model formula (2).

[0056] (2); Here, parameters and The value can be derived from the brightness and chromaticity of the adjacent reference pixels.

[0057] In CCLM, certain points in the original reference region can be selected. Then, a subset of reference pixels formed using these points is further processed. For example, downsampling is performed using the reference pixel subset, and the maximum and minimum values ​​are found from the downsampled reference pixel subset. Parameters are then determined based on these maximum and minimum values. and The value is determined by downsampling a subset of reference pixels. This reduces the number of downsampling operations compared to downsampling the original pixel set. Furthermore, the maximum and minimum values ​​can be found more quickly from the downsampled reference pixel subset, allowing for faster determination of the prediction model and thus improving data processing speed. Of course, if the selection criteria are appropriate, the coding efficiency of predictive encoding will be improved. In other embodiments, using a subset of reference pixels allows for faster parameter derivation compared to the original pixel set. and The value is then used to predict the current coded block using the obtained prediction model, thereby improving the data processing speed.

[0058] In the embodiments of this application, video coding is performed by using a subset of reference pixels in any processing prior to predicting the current coding block, which reduces the complexity of video coding and improves the robustness of the algorithm.

[0059] Based on the foregoing embodiments, this application provides an information processing device, which includes the included units and the modules included in each unit, which can be implemented by a processor in an electronic device; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a central processing unit (CPU), microprocessor (MPU), digital signal processor (DSP) or field programmable gate array (FPGA), etc.

[0060] Figure 6A This is a schematic diagram of the composition structure of the information processing device according to an embodiment of this application, such as... Figure 6A As shown, the device 6 includes an original pixel set determination module 61, an attribute information determination module 62, a target pixel selection module 63, and an encoding processing module 64, wherein: the original pixel set determination module 61 is configured to determine the original pixel set of the block to be encoded in the current video frame; the attribute information determination module 62 is configured to determine the attribute information of each original pixel in the original pixel set; the target pixel selection module 63 is configured to select original pixels whose attribute information meets the filtering conditions to obtain a target pixel set; and the encoding processing module 64 is configured to perform encoding processing on the block to be encoded based on the attribute information of each original pixel in the target pixel set.

[0061] In other embodiments, the original pixel set determination module 61 is configured to determine at least one reconstructed pixel outside the block to be encoded as an original pixel in the original pixel set; the encoding processing module 64 is configured to perform predictive encoding, transform encoding or entropy encoding on the block to be encoded based on the attribute information of each reconstructed pixel in the target pixel set.

[0062] In other embodiments, the original pixel set determination module 61 is configured to determine at least one pixel to be encoded within the block to be encoded as an original pixel in the original pixel set; the encoding processing module 64 is configured to perform predictive encoding, transform encoding, or entropy encoding on the target pixel set based on the attribute information of each pixel to be encoded in the target pixel set.

[0063] In other embodiments, such as Figure 6B As shown, the target pixel selection module 63 includes: a correlation determination unit 631, configured to determine the correlation between each reconstructed pixel and the block to be encoded based on the attribute information of the reconstructed pixels in the original pixel set; and a selection unit 632, configured to select reconstructed pixels whose correlation satisfies the filtering conditions to obtain a target pixel set.

[0064] In other embodiments, the correlation determination unit 631 is configured to determine the positional relationship between the reconstructed pixels in the original pixel set and the block to be encoded; and to determine the positional relationship as the correlation between the reconstructed pixels and the block to be encoded.

[0065] In other embodiments, the target pixel selection module 63 is configured to select reconstructed pixels whose image components are within a preset range from the original pixel set as target pixels, thereby obtaining the target pixel set.

[0066] In other embodiments, the image component is a luminance component or a chrominance component.

[0067] In other embodiments, such as Figure 6C As shown, the encoding processing module 64 includes: a preprocessing unit 641 configured to preprocess the attribute information of each reconstructed pixel in the target pixel set to obtain a preprocessed target pixel set; and a prediction encoding unit 642 configured to perform prediction encoding on the block to be encoded based on the attribute information of each reconstructed pixel in the preprocessed target pixel set.

[0068] In other embodiments, the preprocessing unit 641 is configured to perform downsampling processing on the attribute information of each reconstructed pixel in the target pixel set to obtain a downsampled target pixel set; or, it is configured to perform filtering processing on the attribute information of each reconstructed pixel in the target pixel set to obtain a filtered target pixel set.

[0069] In other embodiments, such as Figure 6DAs shown, the predictive coding unit 641 includes: a model building subunit 6411, configured to build a predictive model based on the attribute information of each reconstructed pixel in the target pixel set after the downsampling process or the filtering process, wherein the predictive model is used to characterize the predictive relationship between multiple image components of the pixel to be encoded in the block to be encoded; and a predictive coding subunit 6412 configured to perform predictive coding on the block to be encoded based on the predictive model.

[0070] In other embodiments, the model construction subunit 6411 is configured to determine the parameter values ​​of the first parameter and the second parameter of the prediction model based on the chromaticity component and luminance component of each reconstructed pixel in the target pixel set after the downsampling process or the filtering process; and to construct the prediction model based on the parameter values ​​of the first parameter and the second parameter.

[0071] In other embodiments, the original pixel set determination module 61 is configured to determine N reconstructed pixels within a preset range where the block to be encoded is located as the original pixel set; wherein N is a preset integer greater than or equal to 1.

[0072] In other embodiments, the original pixel set determination module 61 is configured to determine the reconstructed pixels in the reference row or reference column adjacent to the block to be encoded as the original pixel set.

[0073] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0074] It should be noted that, in the embodiments of this application, if the above-described information processing method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a mobile phone, tablet computer, personal computer, personal digital assistant, navigator, digital phone, video phone, television, sensor device, server, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0075] This application provides an electronic device, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements the steps in the information processing method provided in the above embodiments.

[0076] This application provides a computer-readable storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the steps in the information processing method provided in the above embodiments.

[0077] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0078] It should be noted that, Figure 7 This is a schematic diagram of the hardware entity of the electronic device in the embodiment of this application, such as... Figure 7 As shown, the electronic device 700 includes a memory 701 and a processor 702. The memory 701 stores a computer program that can run on the processor 702. When the processor 702 executes the program, it implements the steps in the information processing method provided in the above embodiments.

[0079] It should be noted that the memory 701 is configured to store instructions and applications executable by the processor 702, and can also cache data to be processed or already processed by the processor 702 and various modules in the electronic device 700 (e.g., image data, audio data, voice communication data and video communication data), which can be implemented by flash memory or random access memory (RAM).

[0080] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0081] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0082] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0083] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0084] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0085] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0086] Alternatively, if the integrated units described above in this application are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a mobile phone, tablet computer, personal computer, personal digital assistant, navigator, digital phone, video phone, television, sensor device, server, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, magnetic disks, or optical disks.

[0087] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0088] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

[0089] Industrial applicability In this embodiment, original pixels whose attribute information meets the filtering conditions are selected from the original pixel set of the block to be encoded to obtain a target pixel set; the block to be encoded is encoded based on the attribute information of each original pixel in the target pixel set; thus, encoding the block to be encoded based on the attribute information of a selected portion of the original pixels can reduce the computational complexity of the encoding process, thereby improving video encoding efficiency and improving the smoothness of video viewing for users.

Claims

1. An information processing method applied to an encoder, the method comprising: Determine at least one reference sample for the block to be encoded in the current video frame; wherein the reference sample is a reconstructed sample outside the block to be encoded; Based on the attribute information of the reference samples, at least one sample is selected from the at least one reference sample to obtain the target sample; wherein, the attribute information includes positional relationship, luminance component and chrominance component; The target sample is filtered based on its attribute information to obtain a filtered target sample. Based on the attribute information of the filtered target sample, the parameter values ​​of the first parameter and the second parameter of the prediction model are determined. The prediction model is used to characterize the prediction relationship between two image components, namely, the luminance component and the chrominance component. Based on the prediction model, predictive coding is performed on the block to be encoded.

2. The method according to claim 1, characterized in that, The step of filtering the target sample based on its attribute information to obtain a filtered target sample includes: The target sample is filtered based on the attribute information of the reference sample in the target sample to obtain the filtered target sample.

3. The method according to claim 1, characterized in that, The at least one reference sample includes: at least one row of reference samples adjacent to the upper side of the block to be encoded, and / or at least one column of reference samples adjacent to the left side of the block to be encoded.

4. The method according to claim 3, characterized in that, The step of selecting at least one sample from the at least one reference sample based on the attribute information of the reference sample to obtain the target sample includes: Based on the attribute information of the reference sample, the positional relationship between the at least one reference sample and the block to be encoded is determined; Based on the described positional relationship, the target sample is obtained.

5. The method according to claim 3, characterized in that, The step of selecting at least one sample from the at least one reference sample based on the attribute information of the reference sample to obtain the target sample includes: Based on the attribute information, select a portion of reference samples from at least one row of reference samples adjacent to the upper side of the block to be encoded, and / or select a portion of reference samples from at least one column of reference samples adjacent to the left side of the block to be encoded. The target sample includes the selected reference sample.

6. The method according to claim 5, characterized in that, The step of selecting a portion of reference samples from at least one row of reference samples adjacent to the upper side of the block to be encoded, based on the attribute information, includes: Based on the attribute information, at least one row of reference samples adjacent to the upper side of the block to be encoded is downsampled to obtain partial reference samples; The step of selecting a portion of reference samples from at least one column of reference samples adjacent to the left side of the block to be encoded based on the attribute information includes: Based on the attribute information, at least one column of reference samples adjacent to the left side of the block to be encoded is downsampled to obtain partial reference samples.

7. The method according to claim 2, characterized in that, The input to the filtering process includes the reference sample and one or more adjacent reference samples.

8. The method according to any one of claims 1 to 7, characterized in that, The step of determining the parameter values ​​of the first parameter and the second parameter of the prediction model based on the attribute information of the filtered target sample includes: Based on the chromaticity and luminance components of the reference sample in the filtered target sample, the parameter values ​​of the first parameter and the second parameter of the prediction model are determined.

9. An information processing method applied to a decoder, the method comprising: Determine at least one reference sample for the current processing block to be decoded; wherein the reference sample is a reconstructed sample outside the processing block; Based on the attribute information of the reference samples, at least one sample is selected from the at least one reference sample to obtain the target sample; wherein, the attribute information includes positional relationship, luminance component and chrominance component; The target sample is filtered based on its attribute information to obtain a filtered target sample. Based on the attribute information of the filtered target sample, the parameter values ​​of the first parameter and the second parameter of the prediction model are determined. The prediction model is used to characterize the prediction relationship between two image components, namely, the luminance component and the chrominance component. Based on the prediction model, the predicted value of the processing block is determined; The reconstructed value of the processing block is determined based on the predicted value and the residual value of the processing block.

10. The method according to claim 9, characterized in that, The step of filtering the target sample based on its attribute information to obtain a filtered target sample includes: The target sample is filtered based on the attribute information of the reference sample in the target sample to obtain the filtered target sample.

11. The method according to claim 9, characterized in that, The at least one reference sample includes: at least one row of reference samples adjacent to the upper side of the processing block, and / or at least one column of reference samples adjacent to the left side of the processing block.

12. The method according to claim 11, characterized in that, The step of selecting at least one sample from the at least one reference sample based on the attribute information of the reference sample to obtain the target sample includes: Based on the attribute information of the reference sample, the positional relationship between the at least one reference sample and the processing block is determined; Based on the described positional relationship, the target sample is obtained.

13. The method according to claim 11, characterized in that, The step of selecting at least one sample from the at least one reference sample based on the attribute information of the reference sample to obtain the target sample includes: Based on the attribute information, select a portion of reference samples from at least one row of reference samples adjacent to the top side of the processing block, and / or select a portion of reference samples from at least one column of reference samples adjacent to the left side of the processing block. The target sample includes the selected reference sample.

14. The method according to claim 13, characterized in that, The step of selecting a portion of reference samples from at least one row of reference samples adjacent to the upper side of the processing block based on the attribute information includes: Based on the attribute information, at least one row of reference samples adjacent to the upper side of the processing block is downsampled to obtain partial reference samples; The step of selecting a portion of reference samples from at least one column of reference samples adjacent to the left side of the processing block based on the attribute information includes: Based on the attribute information, at least one column of reference samples adjacent to the left side of the processing block is downsampled to obtain a partial reference sample.

15. The method according to claim 10, characterized in that, The input to the filtering process includes the reference sample and one or more adjacent reference samples.

16. The method according to any one of claims 9 to 15, characterized in that, The step of determining the parameter values ​​of the first parameter and the second parameter of the prediction model based on the attribute information of the filtered target sample includes: Based on the chromaticity and luminance components of the reference sample in the filtered target sample, the parameter values ​​of the first parameter and the second parameter of the prediction model are determined.

17. A computer-readable storage medium having a computer program and a bitstream stored thereon, wherein the computer program, when executed by a processor, implements the steps of the information processing method of any one of claims 1 to 8 to generate the bitstream.

Images