Pixel filtering method and device, electronic equipment and computer program product
By splitting the filtered data into sub-data and performing step-by-step shifting operations, the problem of high width of intermediate data bits in pixel filtering is solved, improving computational efficiency and parallelism, and achieving more efficient video encoding and decoding processing.
Patent Information
- Application Number
- CN202511240708.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-01
- Publication Date
- 2025-11-18
AI Technical Summary
During video encoding and decoding, the high bit width requirement of intermediate data during pixel filtering leads to low computational efficiency and fails to maximize the advantages of vector registers in processing parallel data.
By splitting the weighted first filtered data into multiple sub-data and performing step-by-step shift operations, the bit width of the intermediate results is limited to match the input and output. Combined with the bit width characteristics of the vector register, the parallelism and throughput of the operation are improved.
While ensuring filtering accuracy, it effectively reduces the intermediate data bit width requirement, significantly improves pixel filtering operation efficiency, and enhances the parallelism and throughput of the operation.
Smart Images
Figure CN120980236A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of video encoding and decoding technology, and more specifically, to a pixel filtering method, a pixel filtering device, an electronic device, and a computer program product. Background Technology
[0002] Video encoding and decoding is the process of compressing and decompressing video data, mainly involving computationally intensive operations such as motion estimation, transform coding, and entropy coding. To improve the processing efficiency of these stages, the ARMv8 (ARM architecture version 8) SIMD (Single Instruction Multiple Data) instruction set, through its single instruction multiple data parallel processing characteristics, can achieve a 2 to 4 times performance improvement in assembly optimizations of modules such as color space conversion, pixel interpolation, discrete cosine transform, and loop filtering, significantly improving the video playback experience and power consumption of mobile devices.
[0003] SIMD design involves data bit width processing. However, when implementing pixel filtering, the intermediate data being calculated usually exceeds the bit width of the input data. Due to the limited bit width of the vector register, the advantage of the vector register in processing parallel data cannot be maximized, resulting in low computational efficiency.
[0004] Therefore, there is an urgent need in this field for a pixel filtering method that can effectively reduce the intermediate data bit width requirement and improve the pixel filtering operation efficiency while ensuring filtering accuracy.
[0005] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention
[0006] The purpose of this disclosure is to provide a pixel filtering method, pixel filtering device, electronic device and computer program product, which can at least to a certain extent reduce the intermediate data bit width requirement and improve the pixel filtering operation efficiency while ensuring filtering accuracy.
[0007] According to a first aspect of this disclosure, a pixel filtering method is provided, comprising:
[0008] Obtain the target pixel before filtering and the reference pixel corresponding to the target pixel;
[0009] Based on the weighted result of the target pixel and the reference pixel and the first shift parameter, the first filtered data corresponding to the target pixel is obtained;
[0010] The first filtered data is divided into multiple sub-data, and each sub-data is shifted according to the second shift parameter to obtain the second filtered data corresponding to the target pixel, wherein the first shift parameter is greater than the second shift parameter;
[0011] The filtering result of the target pixel is obtained based on the second filtering data corresponding to the target pixel.
[0012] In one exemplary embodiment of this disclosure, before splitting the first filtered data into multiple sub-data, the method further includes:
[0013] Based on the number of target pixels and reference pixels and the first shift parameter, it is determined whether the first filtered data meets the splitting condition;
[0014] If the first filtered data meets the splitting condition, then the first filtered data is split into multiple sub-data.
[0015] If the first filtered data does not meet the splitting condition, the filtering result of the target pixel is obtained directly based on the first filtered data.
[0016] In an exemplary embodiment of this disclosure, determining whether the first filtered data satisfies the splitting condition based on the number of the target pixels and the reference pixels and the first shift parameter includes:
[0017] The first divisor is obtained based on the first shift parameter;
[0018] If the number of target pixels and reference pixels is less than or equal to the first divisor, then the first filtered data is determined to satisfy the splitting condition.
[0019] In one exemplary embodiment of this disclosure, obtaining the first filtered data corresponding to the target pixel based on the weighted result of the target pixel and the reference pixel and the first shift parameter includes:
[0020] Obtain the filtering coefficients of the target pixel, and weight the target pixel and the reference pixel according to the filtering coefficients to obtain the weighted result of the target pixel and the reference pixel;
[0021] The weighted result of the target pixel and the reference pixel is shifted according to the first shift parameter to obtain the first filtered data corresponding to the target pixel.
[0022] In one exemplary embodiment of this disclosure, splitting the first filtered data into multiple sub-data includes:
[0023] Based on the unsigned half-add instruction, the unsigned rounded half-add instruction, and the bit selection instruction, the first filtered data is split into multiple sub-data.
[0024] In one exemplary embodiment of this disclosure, obtaining the filtering result of the target pixel based on the second filtering data corresponding to the target pixel includes:
[0025] Based on the pixel value of the target pixel and a preset constant value, the maximum and minimum filter values of the target pixel are obtained.
[0026] If the calculation result of the second filtered data is less than or equal to the minimum filtered value, then the minimum filtered value is used as the filtering result of the target pixel;
[0027] If the calculation result of the second filtered data is greater than or equal to the maximum filtered value, then the maximum filtered value is used as the filtering result of the target pixel;
[0028] If the calculation result of the second filtered data is greater than the minimum filtered value and less than the maximum filtered value, then the calculation result of the second filtered data is the filtering result of the target pixel.
[0029] In one exemplary embodiment of this disclosure, the method further includes:
[0030] Obtain the preset bit width of the vector register used for pixel filtering;
[0031] The number of target pixels to be filtered simultaneously is determined based on the preset bit width of the vector register and the bit width of the target pixel.
[0032] According to a second aspect of this disclosure, a pixel filtering device is provided, comprising:
[0033] The target pixel acquisition module is configured to acquire the target pixel before filtering and the reference pixel corresponding to the target pixel;
[0034] The filtered data determination module is configured to perform a process of obtaining first filtered data corresponding to the target pixel based on the weighted result of the target pixel and the reference pixel and a first shift parameter;
[0035] The filtered data splitting module is configured to split the first filtered data into multiple sub-data, and perform a shift operation on each of the sub-data according to a second shift parameter to obtain the second filtered data corresponding to the target pixel, wherein the first shift parameter is greater than the second shift parameter;
[0036] The filtering result determination module is configured to execute the second filtering data corresponding to the target pixel to obtain the filtering result of the target pixel.
[0037] In one exemplary embodiment of this disclosure, the pixel filtering device further includes a splitting condition determination module, the splitting condition determination module comprising:
[0038] The splitting condition judgment unit is configured to determine whether the first filtered data meets the splitting condition based on the number of the target pixel and the reference pixel and the first shift parameter.
[0039] The first filtered data splitting unit is configured to split the first filtered data into multiple sub-data if the first filtered data meets the splitting condition;
[0040] The first filtering result determination unit is configured to perform the following operation: if the first filtering data does not meet the splitting condition, then directly obtain the filtering result of the target pixel based on the first filtering data.
[0041] In one exemplary embodiment of this disclosure, the splitting condition determination unit includes:
[0042] The first divisor determination unit is configured to perform the operation of obtaining the first divisor based on the first shift parameter;
[0043] The pixel count determination unit is configured to determine that if the number of the target pixel and the reference pixel is less than or equal to the first divisor, the first filtered data satisfies the splitting condition.
[0044] In one exemplary embodiment of this disclosure, the filtered data determination module includes:
[0045] The weighted result determination unit is configured to perform the following operations: obtain the filtering coefficients of the target pixel, and weight the target pixel and the reference pixel according to the filtering coefficients to obtain the weighted result of the target pixel and the reference pixel;
[0046] The weighted result shifting unit is configured to perform a shifting operation on the weighted result of the target pixel and the reference pixel according to the first shifting parameter to obtain the first filtered data corresponding to the target pixel.
[0047] In one exemplary embodiment of this disclosure, the filtered data splitting module includes:
[0048] The split instruction execution unit is configured to execute unsigned half-add instructions, unsigned rounded half-add instructions, and bit selection instructions to split the first filtered data into multiple sub-data.
[0049] In one exemplary embodiment of this disclosure, the filtering result determination module includes:
[0050] The filtering boundary value determination unit is configured to perform the operation of determining the maximum and minimum filtering values of the target pixel based on the pixel value of the target pixel and a preset constant value.
[0051] The minimum filter value determination unit is configured to perform the following: if the calculation result of the second filter data is less than or equal to the minimum filter value, then use the minimum filter value as the filtering result of the target pixel.
[0052] The maximum filter value determination unit is configured to perform the following: if the calculation result of the second filter data is greater than or equal to the maximum filter value, then the maximum filter value is used as the filtering result of the target pixel.
[0053] The calculation result determination unit is configured to perform the following: if the calculation result of the second filtered data is greater than the minimum filter value and less than the maximum filter value, then the calculation result of the second filtered data is the filtering result of the target pixel.
[0054] In one exemplary embodiment of this disclosure, the pixel filtering device further includes a parallel filtering module, the parallel filtering module comprising:
[0055] The bit width acquisition unit is configured to acquire the preset bit width of the vector register used for pixel filtering;
[0056] The parallel quantity determination unit is configured to determine the number of target pixels to be simultaneously filtered based on the preset bit width of the vector register and the bit width of the target pixel.
[0057] According to a third aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the pixel filtering method described in any of the preceding claims.
[0058] According to a fourth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the pixel filtering method described in any of the preceding claims.
[0059] The exemplary embodiments disclosed herein can have the following beneficial effects:
[0060] In the pixel filtering method of the exemplary embodiments of this disclosure, by splitting the weighted first filtered data into multiple sub-data and performing step-by-step shifting operations, the bit width of the intermediate result is limited to be consistent with the input and output, so that the intermediate result does not need to be expanded for storage. While ensuring the filtering accuracy, the bit width requirement of the intermediate data is effectively reduced. Combined with the bit width characteristics of the vector register, the parallelism of the operation and the throughput of the operation are improved, and the pixel filtering operation efficiency is significantly improved.
[0061] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0062] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.
[0063] Figure 1 A schematic flowchart of a pixel filtering method according to an exemplary embodiment of the present disclosure is shown;
[0064] Figure 2 A flowchart illustrating a sample implementation of this disclosure is shown to determine whether the first filtered data meets the splitting conditions;
[0065] Figure 3 A schematic flowchart illustrating an exemplary embodiment of this disclosure of obtaining a filtering result from second filtered data is shown.
[0066] Figure 4 A block diagram of a pixel filtering apparatus according to an exemplary embodiment of the present disclosure is shown;
[0067] Figure 5 A schematic diagram of the structure of a computer system suitable for implementing the embodiments of the present disclosure is shown. Detailed Implementation
[0068] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.
[0069] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein.
[0070] The following exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of the specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.
[0071] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.
[0072] In some relevant embodiments, taking the VVC (Versatile Video Coding) deblocking filter with short taps on the luminance component as an example, the filtering calculation formula for a set of pixels is as follows:
[0073] p0' = Clip(p0-3t) c p0+3t c ,(p2+2p1+2p0+2q0+q1+4)>>3)
[0074] p1' = Clip(p1-2t) c p1+2t c ,(p2+p1+p0+q0+2)>>2)
[0075] p2'=Clip(p2-t c p2+t c ,(2p3+3p2+p1+p0+q0+4)>>3)
[0076] q0' = Clip(q0 - 3t) c ,q0+3t c ,(q2+2q1+2q0+2p0+p1+4)>>3)
[0077] q1' = Clip(q1 - 2t) c ,q1+2t c ,(q2+q1+q0+p0+2)>>2)
[0078] q2' = Clip(q2-t) c ,q2+t c ,(2q3+3q2+q1+q0+p0+4)>>3)
[0079] Where Clip is the clipping instruction, t c Let p3, p2, p1, p0, q0, q1, q2, q3 be constants, and p0', p1', p2', q0', q1', q2' be consecutive pixels before filtering. The key calculations involve the summation and right shift of the third term in Clip. The first two terms of Clip, according to standard specifications, limit the minimum and maximum values of the third term's calculation result.
[0080] When implementing the above logic using SIMD code, the summation is performed first, followed by shifting. Since the input is 8 bits of pixels, the accumulated result exceeds the representation range of an 8-bit unsigned number. Therefore, the intermediate result needs to be stored using 16 bits, calculated using the ARMv8 128-bit vector register. This allows for the same filtering operation on 8 pixels at a time. Thus, the drawback of the above scheme is that the direct accumulation method does not maximize the advantage of using vector registers to process parallel data.
[0081] To address the aforementioned issues, this example implementation first provides a pixel filtering method. (Reference) Figure 1 As shown, the pixel filtering method described above may include the following steps:
[0082] Step S110. Obtain the target pixel before filtering and the reference pixel corresponding to the target pixel.
[0083] Step S120. Based on the weighted result of the target pixel and the reference pixel and the first shift parameter, obtain the first filtered data corresponding to the target pixel.
[0084] Step S130. Divide the first filtered data into multiple sub-data, and perform a shift operation on each sub-data according to the second shift parameter to obtain the second filtered data corresponding to the target pixel, wherein the first shift parameter is greater than the second shift parameter.
[0085] Step S140. Obtain the filtering result of the target pixel based on the second filtering data corresponding to the target pixel.
[0086] In the pixel filtering method of the exemplary embodiments of this disclosure, by splitting the weighted first filtered data into multiple sub-data and performing step-by-step shifting operations, the bit width of the intermediate result is limited to be consistent with the input and output, so that the intermediate result does not need to be expanded for storage. While ensuring the filtering accuracy, the bit width requirement of the intermediate data is effectively reduced. Combined with the bit width characteristics of the vector register, the parallelism of the operation and the throughput of the operation are improved, and the pixel filtering operation efficiency is significantly improved.
[0087] Below, in conjunction with Figures 2 to 3 The steps described above in this example implementation will be explained in more detail.
[0088] In step S110, the target pixel before filtering and the reference pixel corresponding to the target pixel are obtained.
[0089] In this example implementation, the target pixel refers to the center pixel of the image region to be filtered. This can be achieved by reading the pixel value at specified coordinates from the video frame buffer, serving as the core processing object for the filtering calculation. The reference pixel refers to the adjacent pixels that have a spatial relationship with the target pixel. Specifically, it can be implemented using the set of pixels on the left and right sides of the pixel block containing the target pixel, providing the context information required for the filtering calculation.
[0090] In step S120, the first filtered data corresponding to the target pixel is obtained based on the weighted result of the target pixel and the reference pixel and the first shift parameter.
[0091] In this example implementation, the filtering coefficients of the target pixel can be obtained, and the target pixel and the reference pixel can be weighted according to the filtering coefficients to obtain the weighted result of the target pixel and the reference pixel; then, the weighted result of the target pixel and the reference pixel can be shifted according to the first shift parameter to obtain the first filtered data corresponding to the target pixel.
[0092] The filtering coefficients are used to adjust the weight ratio of target pixels and reference pixels in the filtering process. They can be implemented using fixed values or dynamically calculated values; for example, in deblocking filtering, they can be set to coefficients related to the block boundary distance. The weighted result is the calculation result of a linear combination of target pixels and reference pixels according to the preset filtering coefficients, used to fuse multi-pixel information to generate intermediate data. The first shift parameter is the displacement amount used for initial data bit width compression, which can be implemented using an arithmetic right shift instruction.
[0093] In this example implementation, before splitting the first filtered data into multiple sub-data, it can be determined whether the first filtered data meets the splitting condition before proceeding with subsequent steps. Analysis reveals that for pixels with an input of 8 bits, the accumulated result, after shifting, falls within the range of [0, 255], consistent with the input data range. Theoretically, formulas satisfying this characteristic can be split into step-by-step shift operations, and each operation involves 8 bits of input and output data. With a 128-bit vector register, the same filtering operation can be performed on 16 pixels simultaneously, effectively improving the parallelism of instruction-level operations.
[0094] Based on the above analysis, for unsigned long operations consisting of addition, multiplication, and multi-level shifts, if the output and input ranges are consistent, the operation can be broken down into a combination of successive shifts. For example... Figure 2 As shown, determining whether the first filtered data meets the splitting conditions can specifically include the following steps:
[0095] Step S210. Based on the number of target pixels and reference pixels and the first shift parameter, determine whether the first filtered data meets the splitting condition.
[0096] In this example implementation, a first divisor can be obtained based on a first shift parameter. If the number of target pixels and reference pixels is less than or equal to the first divisor, then the first filtered data is determined to meet the splitting condition.
[0097] The first divisor refers to the threshold calculated based on the shift parameter, specifically obtained by left-shifting the value 1 by the shift parameter. For example, when the shift parameter is 3, the first divisor is 8. This feature transforms the shift parameter into a quantifiable judgment benchmark, providing a mathematical basis for the splitting conditions. The number of target pixels and reference pixels refers to the total amount of original data participating in the weighted calculation, specifically achieved by counting the number of valid pixels in adjacent pixel blocks. This feature is used to measure the data size of the intermediate calculation results, and determines whether it meets the register bit width capacity by comparing it with the first divisor.
[0098] Step S220. If the first filtered data meets the splitting condition, then the first filtered data is split into multiple sub-data.
[0099] During the filtering process, a first divisor is generated based on the binary characteristics of the shift parameter. This divisor represents the maximum data volume threshold that the register width can accommodate. When the total number of target pixels and reference pixels does not exceed this threshold, it indicates that the data size of the intermediate calculation result is within the effective processing range of the register width. In this case, performing a splitting operation can avoid data overflow. For example, when the first shift parameter is 3, the first divisor is calculated to be 8. If the total number of pixels currently being processed is 5, then the splitting condition is met. This mechanism ensures that the splitting operation is only performed within the range allowed by the register width by dynamically associating the shift parameter with the number of pixels.
[0100] Step S230. If the first filtered data does not meet the splitting conditions, the filtering result of the target pixel is obtained directly based on the first filtered data.
[0101] If the splitting condition is not met, the first filtered data will not be split, and the filtering result of the target pixel will be calculated directly according to the calculation formula of the first filtered data.
[0102] In this example implementation, under the hardware condition of fixed vector register bit width, a dynamic judgment mechanism is constructed by establishing the correlation between the number of pixels and the shift parameters. Before data splitting, the characteristics of the calculation scenario are automatically evaluated, and the splitting operation is only performed on data that meets the parallelization conditions, thus eliminating unnecessary calculation steps.
[0103] In step S130, the first filtered data is divided into multiple sub-data, and each sub-data is shifted according to the second shift parameter to obtain the second filtered data corresponding to the target pixel, wherein the first shift parameter is greater than the second shift parameter.
[0104] In this example implementation, the first filtered data can be split into multiple sub-data based on unsigned half-addition instructions, unsigned rounding half-addition instructions, and bit selection instructions.
[0105] Using the three AMRv8 instructions—uhadd (Unsigned Halving Add), urhadd (Unsigned Rounding Halving Add), and bit (Bit Insert if True)—unsigned long arithmetic operations consisting of addition, multiplication, and shift can be broken down step by step. All three instructions have the same input and output bit width. uhadd represents addition averaging, which is done by directly shifting right by one bit without rounding. urhadd represents averaging with rounding, which adds one to the addition result and then shifts right by one bit. bit represents mask assignment, which assigns the value of the first or second register to the destination register based on whether the mask is 0 or 1. The calculation formulas for these three instructions are as follows:
[0106] uhadd(m0,m1)=(m0+m1)>>1;
[0107] urhadd(m0,m1)=(m0+m1+1)>>1;
[0108] bit(flag,m0,m1)=flag? m0:m1;
[0109] The following is a simple example illustrating the feasibility of breaking down long operations with consistent input and output bit widths into stages and storing intermediate results in 8-bit increments:
[0110] Calculate (m1+m2+m3+m4)>>2, where m1, m2, m3, and m4 are all unsigned 8-bit numbers.
[0111] The calculation steps of the original scheme are as follows:
[0112] 1. Calculate m1+m2+m3+m4. The result needs to be stored in 16 bits. 128 bits / 16 bits = 8 pixels can be calculated in parallel at one time.
[0113] 2. Shift the 16-bit result right by 2 bits and convert it to 8 bits for storage.
[0114] In this example implementation, by splitting the original expression, the intermediate result can be represented using only 8 bits, allowing for parallel calculation of 128 bits / 8 bits = 16 pixels at a time. The splitting method is as follows:
[0115] Original expression = ((m1+m2+m3+m4)>>1)>>1 = [(m1+m2)>>1+(m3+m4)>>1+offset]>>1 (where offset = (m1+m2)&(m3+m4)&0x01)
[0116] The intermediate results do not need to be represented by 16 bits throughout the process, which can double the number of pixels calculated at one time compared to the original scheme, thus speeding up the calculation process.
[0117] Based on the conversion strategy in the above example, the complete splitting logic of the short tap strong filtering of the deblock luminance component in the relevant embodiments can be implemented. For ease of representation, m0...m7 can be used to represent p3, p2, p1, p0, q0, q1, q2, q3 in the relevant embodiments, and m1'...m6' can be used to represent the filtered results p0', p1', p2', q0', q1'q2', respectively.
[0118] First, calculate (m1+m2+m3+m4)>>1. The decomposition logic is as follows:
[0119] -->[(m1+m2)>>1+(m3+m4)>>1+offset0], where offset0=(m1+m2)&(m3+m4)&0x01.
[0120] In this example implementation, the suffix _uh indicates the use of the uhadd instruction, _urh indicates the urhadd instruction, offset?x:y indicates the use of the bit instruction, and m12, m12_uh, etc. are variable names used to distinguish intermediate results. The representation can be deduced from the first occurrence, and will not be elaborated further. For each pixel in the relevant embodiments, the specific method for splitting the corresponding first filtered data into second filtered data is as follows:
[0121] 1. Calculate m1' = (2*m0 + 3*m1 + m2 + m3 + m4 + 4) >> 3. The decomposition logic is as follows:
[0122] -->[m0+m1+2+(m1+m2+m3+m4)>>1]>>2
[0123] -->[(m0+m1+m12_uh+m34_uh+offset0)>>1+1]>>1
[0124] -->[(m0+m1)>>1+(m12_uh+m34_uh+offset0)>>1+offset1+1]>>1, where
[0125] offset1=(m0+m1)&(m12_uh+m34_uh+offset0)&0x01
[0126] (m12_uh+m34_uh+offset0)>>1=bit(offset0,urhadd(m12_uh,m34_uh),uhadd(m12_uh,m34_uh))
[0127] 2. Calculate m2' = (m1 + m2 + m3 + m4 + 2) >> 2, the decomposition logic is as follows:
[0128] -->[(m1+m2+m3+m4)>>1+1]>>1
[0129] -->(m12_uh+m34_uh+offset0+1)>>1
[0130] -->m2'=offset0? ((m12_uh+m34_uh)>>1)+1:(m12_uh+m34_uh+1)>>1
[0131] 3. Calculate m3' = (m1 + 2*m2 + 2*m3 + 2*m4 + m5 + 4) >> 3. The decomposition logic is as follows:
[0132] -->[(m1+m5)>>1+m2+m3+m4+2]>>2
[0133] -->[m15_uh+m2+m3+m4+2]>>2
[0134] -->[(m15_uh+m2)>>1+(m3+m4)>>1+offset3+1]>>1, where
[0135] offset3=((m15_uh+m2)&(m3+m4)&0x01;
[0136] -->m3'=offset3? ((m34_uh+m152_uh)>>1)+1:(m34_uh+m152_uh+1)>>1
[0137] 4. Calculate m4' = (m2 + 2*m3 + 2*m4 + 2*m5 + m6 + 4) >> 3. The decomposition logic is as follows:
[0138] -->[(m2+m6)>>1+m3+m4+m5+2]>>2
[0139] -->[m26_uh+m3+m4+m5+2]>>2
[0140] -->[(m26_uh+m5)>>1+(m3+m4)>>1+offset4+1]>>1, where
[0141] offset4=((m26_uh+m5)&(m3+m4)&0x01;
[0142] -->m4'=offset4? ((m34_uh+m265_uh)>>1)+1:(m34_uh+m265_uh+1)>>1
[0143] 5. Calculate m5' = (m3 + m4 + m5 + m6 + 2) >> 2, the decomposition logic is as follows:
[0144] -->[(m3+m4+m5+m6)>>1+1]>>1
[0145] -->(m34_uh+m56_uh+offset5+1)>>1, where
[0146] offset5 = m56 & m34 & 0x01;
[0147] -->m5'=offset5? ((m34_uh+m56_uh)>>1)+1:(m34_uh+m56_uh+1)>>1
[0148] 6. Calculate m6' = (m3 + m4 + m5 + 3 * m6 + 2 * m7 + 4) >> 3. The decomposition logic is as follows:
[0149] -->[m6+m7+2+(m3+m4+m5+m6)>>1]>>2
[0150] -->[(m6+m7+m34_uh+m56_uh+offset5)>>1+1]>>1
[0151] -->[(m6+m7)>>1+(m34_uh+m56_uh+offset5)>>1+offset6+1]>>1
[0152] offset6 = (m6 + m7) & (m34_uh + m56_uh + offset5) & 0x01, where
[0153] -->m6'=offset6? ((m67_uh+m3_6_mask)>>1)+1:(m67_uh+m3_6_mask+1)>>1
[0154] The above splitting is not the only splitting logic. In order to reduce the amount of computation after splitting, the splitting needs to ensure that the number of basic items after splitting is as small as possible. In addition, when writing the actual assembly code, attention should be paid to the arrangement of instructions to reduce pipeline conflicts.
[0155] In this example implementation, data bit width conversion is achieved through instruction combination, which effectively solves the problem of limited parallelism caused by the high bit width register occupied by the intermediate filtering results. This allows for the simultaneous processing of twice the number of pixel data without changing hardware resources, thus doubling the data parallelism.
[0156] Continue to refer to Figure 1 As shown, in step S140, the filtering result of the target pixel is obtained based on the second filtering data corresponding to the target pixel.
[0157] Finally, the filtering result of the target pixel can be calculated based on the split second filtering data. The intermediate data used in the calculation of the second filtering data has the same bit width as the input and output data, thus avoiding the parallelism limitation problem caused by intermediate results occupying high-bit-width registers.
[0158] In this example implementation, as Figure 3As shown, the filtering result of the target pixel is obtained based on the second filtering data corresponding to the target pixel, which may include the following steps:
[0159] Step S310. Based on the pixel value of the target pixel and the preset constant value, obtain the maximum and minimum filter values of the target pixel.
[0160] The maximum filter value refers to the upper limit of the allowed filtering result for a pixel value, which can be achieved by adding the pixel value of the target pixel to a preset constant value. The minimum filter value refers to the lower limit of the allowed filtering result for a pixel value, which can be achieved by subtracting the preset constant value from the pixel value of the target pixel. The preset constant value is a value set according to encoding standards or experience, used to control the dynamic range expansion of the filtering result.
[0161] Step S320. If the calculation result of the second filtered data is less than or equal to the minimum filtered value, then the minimum filtered value is used as the filtering result of the target pixel.
[0162] If the calculation result of the second filtering data is less than or equal to the minimum filtering value, the minimum filtering value of the target pixel is directly used as the filtering result of the target pixel.
[0163] Step S330. If the calculation result of the second filtered data is greater than or equal to the maximum filtered value, then the maximum filtered value is used as the filtering result of the target pixel.
[0164] If the calculation result of the second filtering data is greater than or equal to the minimum filtering value, the maximum filtering value of the target pixel is directly used as the filtering result of the target pixel.
[0165] Step S340. If the calculation result of the second filter data is greater than the minimum filter value and less than the maximum filter value, then the calculation result of the second filter data is the filtering result of the target pixel.
[0166] If the calculation result of the second filtering data is between the maximum and minimum filtering values, then the filtering result of the target pixel is obtained by calculating based on the second filtering data.
[0167] In this example implementation, by establishing a dynamic range constraint mechanism based on the original pixel values, the results can be forced to remain within the effective range while ensuring the accuracy of the filtering calculation. This effectively solves the problem of output distortion caused by the filtering calculation results exceeding the effective dynamic range of the pixels. It achieves adaptive control of the filtering intensity while maintaining the original pixel features, preventing the loss of detail and noise amplification caused by excessive smoothing, and ensuring that the pixel values after filtering are always within the physically representable numerical range.
[0168] In this example implementation, the preset bit width of the vector register used for pixel filtering can also be obtained, and the number of target pixels to be filtered simultaneously can be determined based on the preset bit width of the vector register and the bit width of the target pixel.
[0169] The preset bit width of the vector register refers to the total storage capacity of the vector registers provided by the hardware platform. This parameter determines the total amount of data that can be carried in a single operation. The bit width of the target pixel refers to the number of binary bits occupied by a single pixel data during the calculation process, which is specifically determined by the image encoding format. This parameter limits the minimum storage space required for each pixel processing unit. By dividing the total capacity of the vector register by the storage requirement of a single pixel, the upper limit of the number of pixels that can be processed in a single parallel operation can be accurately calculated, thereby maximizing the parallelism of data processing under hardware storage constraints. For example, in 16-bit storage mode, a 128-bit register can only carry the operation data of 8 pixels. This solution increases the data carrying capacity to 16 pixels under the same hardware conditions by dynamically calculating the parallel processing volume, enabling a single vector instruction to process twice the number of pixel units, significantly improving instruction-level parallelism efficiency.
[0170] It should be noted that although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.
[0171] Furthermore, this disclosure also provides a pixel filtering device. (See reference) Figure 4 As shown, the pixel filtering device may include a target pixel acquisition module 410, a filter data determination module 420, a filter data splitting module 430, and a filter result determination module 440. Wherein:
[0172] The target pixel acquisition module 410 is configured to acquire the target pixel before filtering and the reference pixel corresponding to the target pixel.
[0173] The filtered data determination module 420 is configured to perform a first filtered data corresponding to the target pixel based on the weighted result of the target pixel and the reference pixel and the first shift parameter;
[0174] The filtered data splitting module 430 is configured to split the first filtered data into multiple sub-data, and perform a shift operation on each sub-data according to the second shift parameter to obtain the second filtered data corresponding to the target pixel, wherein the first shift parameter is greater than the second shift parameter;
[0175] The filtering result determination module 440 is configured to execute the second filtering data corresponding to the target pixel to obtain the filtering result of the target pixel.
[0176] In some exemplary embodiments of this disclosure, a pixel filtering device provided in this disclosure may further include a splitting condition judgment module, which may include a splitting condition judgment unit, a first filtered data splitting unit, and a first filtered result determination unit. Wherein:
[0177] The splitting condition judgment unit is configured to determine whether the first filtered data meets the splitting condition based on the number of target pixels and reference pixels and the first shift parameter.
[0178] The first filtered data splitting unit is configured to split the first filtered data into multiple sub-data if the first filtered data meets the splitting conditions;
[0179] The first filtering result determination unit is configured to perform the following operation: if the first filtering data does not meet the splitting conditions, then directly obtain the filtering result of the target pixel based on the first filtering data.
[0180] In some exemplary embodiments of this disclosure, the splitting condition determination unit may include a first divisor determination unit and a pixel count determination unit. Wherein:
[0181] The first divisor determination unit is configured to perform the operation of obtaining the first divisor based on the first shift parameter;
[0182] The pixel count determination unit is configured to determine that if the number of target pixels and reference pixels is less than or equal to the first divisor, the first filtered data satisfies the splitting condition.
[0183] In some exemplary embodiments of this disclosure, the filtered data determination module 420 may include a weighted result determination unit and a weighted result shifting unit. Wherein:
[0184] The weighted result determination unit is configured to perform the operation of obtaining the filtering coefficients of the target pixel, and weighting the target pixel and the reference pixel according to the filtering coefficients to obtain the weighted result of the target pixel and the reference pixel;
[0185] The weighted result shifting unit is configured to perform a shifting operation on the weighted result of the target pixel and the reference pixel according to the first shifting parameter to obtain the first filtered data corresponding to the target pixel.
[0186] In some exemplary embodiments of this disclosure, the filtered data splitting module 430 may include a splitting instruction execution unit configured to execute an unsigned half-addition instruction, an unsigned rounding half-addition instruction, and a bit selection instruction to split the first filtered data into multiple sub-data.
[0187] In some exemplary embodiments of this disclosure, the filtering result determination module 440 may include a filtering boundary value determination unit, a minimum filtering value determination unit, a maximum filtering value determination unit, and a calculation result determination unit. Wherein:
[0188] The filtering boundary value determination unit is configured to perform the operation of determining the maximum and minimum filtering values of the target pixel based on the pixel value of the target pixel and a preset constant value.
[0189] The minimum filter value determination unit is configured to perform the following: if the calculation result of the second filter data is less than or equal to the minimum filter value, then the minimum filter value is used as the filtering result of the target pixel.
[0190] The maximum filter value determination unit is configured to perform the following: if the calculation result of the second filter data is greater than or equal to the maximum filter value, then the maximum filter value is used as the filtering result of the target pixel.
[0191] The calculation result determination unit is configured to perform the following action: if the calculation result of the second filtered data is greater than the minimum filter value and less than the maximum filter value, then the calculation result of the second filtered data is the filtering result of the target pixel.
[0192] In some exemplary embodiments of this disclosure, a pixel filtering device provided in this disclosure may further include a parallel filtering module, which may include a bit width acquisition unit and a parallel quantity determination unit. Wherein:
[0193] The bit width acquisition unit is configured to acquire the preset bit width of the vector register used for pixel filtering;
[0194] The parallel quantity determination unit is configured to determine the number of target pixels to be filtered simultaneously based on the preset bit width of the vector register and the bit width of the target pixel.
[0195] The specific details of each module / unit in the above pixel filtering device have been described in detail in the corresponding method embodiment section, and will not be repeated here.
[0196] Figure 5 A schematic diagram of the structure of a computer system suitable for implementing the embodiments of the present disclosure is shown.
[0197] It should be noted that, Figure 5 The computer system 500 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.
[0198] like Figure 5As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 502 or programs loaded from storage section 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for system operation. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.
[0199] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 510 as needed so that computer programs read from it can be installed into storage section 508 as needed.
[0200] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs various functions defined in the system of this disclosure.
[0201] Exemplary embodiments of this disclosure also provide a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the pixel filtering method described above.
[0202] In one embodiment, the computer program product can be a tangible product containing a computer program, such as a computer-readable storage medium storing the computer program. The readable storage medium can be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory, hard disk drive (HDD), solid-state drive (SSD), etc. For example, the computer program product can be implemented as a non-volatile storage medium storing the computer program, such as read-only memory, NAND flash memory, etc.
[0203] In one implementation, the computer program product can be an intangible product containing a computer program. For example, the computer program product can be implemented as a virtual digital product, such as an executable file, installation package, or other digital file storing the computer program.
[0204] Computer program code can be written in one or more programming languages. Examples of programming languages include C, Java, and C++. Program code can execute entirely on the user's computing device, partially on the user's computing device, or as a standalone software package. It can also execute partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, such as a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via an internet connection provided by a mobile network operator).
[0205] Computer programs can be carried or transmitted via signals such as electricity, magnetism, light, electromagnetic radiation, and infrared radiation. Electronic devices can convert the signals carrying computer programs into digital signals, thereby running the computer programs. When a computer program runs on an electronic device, its code is used to cause the electronic device to execute (more specifically, the processor of the electronic device to execute) the method steps of various exemplary embodiments of this disclosure, such as the pixel filtering method described above.
[0206] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0207] It should be noted that although several modules for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided and embodied by multiple modules.
[0208] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.
[0209] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.
Claims
1. A pixel filtering method, characterized in that, include: Obtain the target pixel before filtering and the reference pixel corresponding to the target pixel; Based on the weighted result of the target pixel and the reference pixel and the first shift parameter, the first filtered data corresponding to the target pixel is obtained; The first filtered data is divided into multiple sub-data, and each sub-data is shifted according to the second shift parameter to obtain the second filtered data corresponding to the target pixel, wherein the first shift parameter is greater than the second shift parameter; The filtering result of the target pixel is obtained based on the second filtering data corresponding to the target pixel.
2. The pixel filtering method according to claim 1, characterized in that, Before splitting the first filtered data into multiple sub-data, the method further includes: Based on the number of target pixels and reference pixels and the first shift parameter, it is determined whether the first filtered data meets the splitting condition; If the first filtered data meets the splitting condition, then the first filtered data is split into multiple sub-data. If the first filtered data does not meet the splitting condition, the filtering result of the target pixel is obtained directly based on the first filtered data.
3. The pixel filtering method according to claim 2, characterized in that, The step of determining whether the first filtered data meets the splitting condition based on the number of target pixels and reference pixels and the first shift parameter includes: The first divisor is obtained based on the first shift parameter; If the number of target pixels and reference pixels is less than or equal to the first divisor, then the first filtered data is determined to satisfy the splitting condition.
4. The pixel filtering method according to claim 1, characterized in that, The step of obtaining the first filtered data corresponding to the target pixel based on the weighted result of the target pixel and the reference pixel and the first shift parameter includes: Obtain the filtering coefficients of the target pixel, and weight the target pixel and the reference pixel according to the filtering coefficients to obtain the weighted result of the target pixel and the reference pixel; The weighted result of the target pixel and the reference pixel is shifted according to the first shift parameter to obtain the first filtered data corresponding to the target pixel.
5. The pixel filtering method according to claim 1, characterized in that, The step of splitting the first filtered data into multiple sub-data includes: Based on the unsigned half-add instruction, the unsigned rounded half-add instruction, and the bit selection instruction, the first filtered data is split into multiple sub-data.
6. The pixel filtering method according to claim 1, characterized in that, The step of obtaining the filtering result of the target pixel based on the second filtering data corresponding to the target pixel includes: Based on the pixel value of the target pixel and a preset constant value, the maximum and minimum filter values of the target pixel are obtained. If the calculation result of the second filtered data is less than or equal to the minimum filtered value, then the minimum filtered value is used as the filtering result of the target pixel; If the calculation result of the second filtered data is greater than or equal to the maximum filtered value, then the maximum filtered value is used as the filtering result of the target pixel; If the calculation result of the second filtered data is greater than the minimum filtered value and less than the maximum filtered value, then the calculation result of the second filtered data is the filtering result of the target pixel.
7. The pixel filtering method according to claim 1, characterized in that, The method further includes: Obtain the preset bit width of the vector register used for pixel filtering; The number of target pixels to be filtered simultaneously is determined based on the preset bit width of the vector register and the bit width of the target pixel.
8. A pixel filtering device, characterized in that, include: The target pixel acquisition module is configured to acquire the target pixel before filtering and the reference pixel corresponding to the target pixel; The filtered data determination module is configured to perform a process of obtaining first filtered data corresponding to the target pixel based on the weighted result of the target pixel and the reference pixel and a first shift parameter; The filtered data splitting module is configured to split the first filtered data into multiple sub-data, and perform a shift operation on each of the sub-data according to a second shift parameter to obtain the second filtered data corresponding to the target pixel, wherein the first shift parameter is greater than the second shift parameter; The filtering result determination module is configured to execute the second filtering data corresponding to the target pixel to obtain the filtering result of the target pixel.
9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the pixel filtering method as described in any one of claims 1 to 7.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the pixel filtering method as described in any one of claims 1 to 7.