An image processing method, system, electronic device and storage medium
By segmenting JPEG images and extracting multi-dimensional features, and dynamically adjusting the power parameters of each image block, the problem of unbalanced power consumption and performance in traditional JPEG encoding chips is solved, achieving a more efficient encoding process.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHANDONG YUNHAI GUOCHUANG CLOUD COMPUTING EQUIP IND INNOVATION CENT CO LTD
- Filing Date
- 2026-04-30
- Publication Date
- 2026-06-05
Smart Images

Figure CN122152097A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computer technology, and in particular to an image processing method, system, electronic device, and storage medium. Background Technology
[0002] JPEG (Joint Photographic Experts Group) video compression is a commonly used international standard for compressing continuous-tone still images (including grayscale and color images). The image to be compressed can be in any color space, and users can adjust the compression ratio to achieve or approach top-tier compression performance in the industry, while maintaining good resolution and image quality. Therefore, JPEG image compression is essential when storing and transmitting large numbers of images.
[0003] The core of JPEG image compression is JPEG image encoding, which involves multiple processing stages, such as Discrete Cosine Transform (DCT), quantization, and entropy coding. Each stage consumes certain hardware resources and power. Traditional JPEG encoding chips typically use a fixed power supply (such as fixed voltage and fixed frequency) for power management, which cannot dynamically adjust according to the complexity of the image content. However, image content is often diverse, and the computational complexity of encoding varies greatly for different image content. A fixed power supply leads to wasted power when processing simple content, while insufficient power may affect encoding quality or speed when processing complex content. Therefore, achieving the optimal balance between power consumption and performance in JPEG image encoding has become a challenging problem to solve. Summary of the Invention
[0004] This disclosure provides an image processing method, system, electronic device, and storage medium to at least solve the above-mentioned technical problems existing in the prior art.
[0005] According to a first aspect of this disclosure, an image processing method is provided, comprising: dividing an image to be processed into blocks to obtain multiple target image blocks; performing multi-dimensional feature extraction on the target image blocks to obtain feature parameters of the target image blocks; the feature parameters characterizing the content complexity of the target image blocks; determining target power parameters required for processing the target image blocks based on the feature parameters; the target power parameters corresponding to each target image block being the same or different; and encoding the corresponding target image blocks based on the target power parameters.
[0006] According to a second aspect of this disclosure, an image processing system is provided, comprising: a feature extraction module, configured to divide an image to be processed into blocks to obtain multiple target image blocks; to perform multi-dimensional feature extraction on the target image blocks to obtain feature parameters of the target image blocks; the feature parameters characterizing the content complexity of the target image blocks; a power control module, configured to determine target power parameters required for processing the target image blocks based on the feature parameters; the target power parameters corresponding to each target image block may be the same or different; a parameter adjustment module, configured to adjust the power parameters of an image encoding module to the target power parameters; and an image encoding module, configured to perform encoding processing on the corresponding target image blocks based on the target power parameters.
[0007] According to a third aspect of this disclosure, an electronic device is provided, comprising: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the methods described in this disclosure.
[0008] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described in this disclosure.
[0009] This disclosure discloses an image processing method, system, electronic device, and storage medium. It divides the image to be processed into blocks, extracts multi-dimensional feature parameters representing the complexity of each target image block, and determines matching target power parameters for each target image block based on these multi-dimensional feature parameters. Then, it performs encoding processing on the target image block based on the target power parameters. This establishes a fine-grained dynamic mapping mechanism between image content features and hardware power consumption status. It can supply power on demand according to the real-time complexity of the image block, avoiding the over-power supply problem when processing simple image content under traditional fixed power supply mode, significantly reducing power waste and improving power utilization efficiency. It also provides sufficient computing power for processing complex image content, effectively avoiding a decrease in encoding speed or quality, and achieving a dynamic balance between encoding power consumption and encoding performance.
[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description
[0011] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which: In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.
[0012] Figure 1 A flowchart illustrating an image processing method according to an embodiment of this disclosure is shown. Figure 1 ; Figure 2 A flowchart illustrating an image processing method according to an embodiment of this disclosure is shown. Figure 2 ; Figure 3 A flowchart of an existing image processing method is shown; Figure 4 A schematic diagram of the structure of an image processing system according to an embodiment of the present disclosure is shown. Figure 1 ; Figure 5 A schematic diagram of the structure of an image processing system according to an embodiment of the present disclosure is shown. Figure 2 ; Figure 6 A schematic diagram of the structure of an image processing system according to an embodiment of the present disclosure is shown. Figure 3 ; Figure 7 A schematic diagram of the composition structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation
[0013] To make the objectives, features, and advantages of this disclosure more apparent and understandable, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.
[0014] Figure 1 A flowchart illustrating an image processing method according to an embodiment of this disclosure is shown. Figure 1 ,like Figure 1 As shown, an image processing method includes: Step S101: The image to be processed is divided into blocks to obtain multiple target image blocks.
[0015] In this embodiment, the image to be processed is an image that needs to be JPEG encoded and compressed, which may include grayscale images and color images. The block processing performed on the image to be processed is a fixed-size uniform block, specifically according to the standard processing requirements of JPEG encoding. For example, the image to be processed is divided into multiple 8×8 pixel image blocks, which are the target image blocks. During the block processing, the entire image to be processed is divided in a row-by-row and column-by-column order. If the number of pixel rows and columns of the image to be processed is not an integer multiple of 8, zero-padding can be performed on the image edges before completing the block processing, ensuring that each target image block is a standard 8×8 pixel matrix, which is compatible with the hardware operation requirements of the subsequent JPEG encoding stage.
[0016] Step S102: Perform multi-dimensional feature extraction on the target image block to obtain the feature parameters of the target image block.
[0017] In this embodiment, multi-dimensional feature extraction is performed independently for each target image patch. The extracted feature parameters are used to characterize the complexity of the target image patch's content. These feature parameters can be quantified numerical parameters, with different values directly corresponding to the level of complexity of the target image patch's content. Multi-dimensional feature extraction is implemented using hardware circuitry, with computation completed by a dedicated feature extraction unit. The extraction process forms a pipeline with subsequent JPEG encoding steps, ensuring real-time feature extraction. Furthermore, all feature extraction operations are implemented using basic logic such as hardware shifting, addition, and convolution, adapting to chip-level hardware execution environments.
[0018] Step S103: Based on the feature parameters, determine the target power parameters required for processing the target image patch.
[0019] In this embodiment, the extracted feature parameters can be used as input, and by comparing and matching them with preset parameter thresholds, the target power parameters corresponding to the feature parameters are output. The target power parameters are power supply parameters used to control JPEG encoding, and include at least two core parameters: power supply voltage and operating frequency. These parameters directly affect the core hardware module of JPEG encoding, determining the computing power and power consumption level of the core hardware module. The target power parameters corresponding to each target image block may be the same or different. Since the feature parameters of different target image blocks may differ, i.e., the image content complexity may vary, each target image block can be matched with different target power parameters. If the feature parameters of multiple target image blocks are consistent, i.e., the image content complexity is the same, then the target power parameters matched by these multiple target image blocks are the same.
[0020] Step S104: Based on the target power parameters, encode the corresponding target image block.
[0021] In this embodiment, based on the received target power parameters including the power supply voltage and operating frequency, a complete JPEG encoding process can be performed on the corresponding target image block. The encoding process follows the JPEG international standard, including all stages such as discrete cosine transform, quantization, and entropy coding. Each target image block completes independent encoding under its corresponding target power parameters. During the encoding process, the power supply status can also be adjusted in real time according to the target power parameters of the next target image block, achieving seamless switching of power parameters during the encoding process and ensuring that each target image block completes encoding under power parameters that match the complexity of its content.
[0022] Figure 3 A flowchart illustrating an existing image processing method is shown, such as... Figure 3 As shown, in existing image processing methods, the process begins with the original image (such as an RGB color space image), which is first converted to the YCbCr color space (a luminance-chrominance separation color space used for digital image and video processing) through hue transformation (lossless processing). Then, sampling operations (lossy processing) are performed. After completing the initial color space and pixel sampling processing, the image enters the core compression stage in 8×8 image blocks. First, Discrete Cosine Transform (DCT) is performed on the image blocks. DCT is lossless processing. Then, quantization processing (lossy processing) is performed. After quantization, the subsequent encoding stage is entered, which sequentially performs entropy encoding and encoding table-based encoding operations. All encoding stages are lossless operations, ensuring the accuracy of image restoration. Finally, based on all the above processing steps, a bitstream data that can be stored and transmitted is generated, and the corresponding compressed output file is output. The image processing stage involves multiple processing steps, such as DCT, quantization, and entropy coding. Each step requires certain hardware resources and power. However, existing image processing methods typically use a fixed power supply (such as fixed voltage and fixed frequency) for power management, which cannot be dynamically adjusted according to the complexity of the image content.
[0023] The image processing method disclosed herein uses image blocks as the smallest processing unit. After dividing the image into blocks, feature parameters are extracted and target power parameters are matched. Encoding is then completed under the appropriate power parameters, breaking away from the fixed voltage and fixed frequency power supply mode of traditional JPEG encoding. This method achieves dynamic allocation of power parameters based on image content. Low-power parameters are matched to simple image blocks, avoiding power waste caused by excessive power supply to hardware resources. High-performance power parameters are matched to complex image blocks, ensuring that encoding speed and quality do not decrease due to insufficient power. This achieves the optimal balance between power consumption and performance in the overall JPEG encoding process, significantly improving the energy efficiency of the JPEG encoding system.
[0024] Figure 2 A flowchart illustrating an image processing method according to an embodiment of this disclosure is shown. Figure 2 ,like Figure 2 As shown, step S102, "performing multi-dimensional feature extraction on the target image patch to obtain the feature parameters of the target image patch," includes: Step S201: Determine the edge features of the target image block based on the grayscale change rate between adjacent pixels within the target image block.
[0025] In this embodiment, the grayscale change rate between adjacent pixels is a quantified representation of the difference in grayscale value between any pixel and its neighboring pixels within the target image block. Edge features characterize the degree of change in edges and / or textures within the target image block. A larger grayscale change rate indicates a more significant abrupt change in grayscale value at the pixel, corresponding to denser edges or textures in the image; a smaller grayscale change rate indicates no significant abrupt change in grayscale value at the pixel, representing smoother edges and textures. The determination of edge features can be implemented based on the convolution operation of the Sobel operator. By performing horizontal and vertical convolution operations on the 3×3 neighboring pixels of each pixel within the target image block, the grayscale change rate of each pixel is obtained. Then, by statistically averaging the grayscale change rates of all pixels, the edge feature parameters of the entire target image block are obtained.
[0026] Step S202: Determine the grayscale distribution characteristics of the target image block based on the statistical dispersion of pixel values within the target image block.
[0027] In this embodiment, the statistical dispersion of pixel values within the target image block is a statistical indicator describing the degree to which pixel grayscale values deviate from the overall average grayscale value. This statistical dispersion can be quantified by calculating the variance, standard deviation, and mean absolute deviation of the pixel values. For example, regarding variance, a larger variance indicates higher statistical dispersion and a more dispersed distribution of pixel grayscale values; a smaller variance indicates lower statistical dispersion and a more concentrated distribution of pixel grayscale values. The grayscale distribution feature is directly characterized by this statistical dispersion and is a single quantified numerical parameter used to characterize the uniformity of grayscale distribution within the target image block. A larger grayscale distribution feature value indicates a more uneven distribution of grayscale values within the image block, with multiple pixels exhibiting different grayscale levels, resulting in higher complexity of the image block's content; a smaller value indicates a more uniform distribution of grayscale values within the image block, with most pixels' grayscale values close to the overall average, resulting in lower complexity of the image block's content.
[0028] Step S203: Determine the frequency domain characteristics of the target image block based on the proportion of signal components in the target image block that are higher than a preset frequency threshold.
[0029] In this embodiment, signal components in the target image patch that are above a preset frequency threshold are considered special high-frequency components. The proportion of these signal components is the ratio of the number of special high-frequency components to the total number of components in the domain coefficient matrix. The frequency domain feature is directly characterized by the proportion of the signal components. The frequency domain feature characterizes the detail richness of the target image patch; that is, the larger the frequency domain feature value, the more special high-frequency components are in the image patch, and the richer the image details and the more complex the texture. The smaller the value, the fewer special high-frequency components are in the image patch, and the less detailed the image, and the smoother the content.
[0030] In this disclosure, multi-dimensional feature extraction is specifically divided into three dimensions: edge features, grayscale distribution features, and frequency domain features. This achieves a comprehensive and multi-dimensional quantitative representation of the complexity of target image blocks. Compared to single-dimensional feature extraction, it can more accurately and comprehensively reflect the actual content complexity of image blocks, avoiding image content classification bias caused by single-feature judgment. The feature extraction process for all three dimensions is implemented based on hardware-level computational logic and can be carried out in parallel, deeply integrated with the JPEG encoding process, ensuring real-time feature extraction and hardware execution efficiency. Simultaneously, each dimension of features is a quantified numerical parameter, forming a precise numerical match with subsequent classification rules. This provides a reliable and accurate basis for the dynamic allocation of power parameters, further improving the matching degree between power allocation and image content complexity, thereby more effectively achieving a balance between power consumption and performance.
[0031] In another embodiment, step S201, "determining the edge features of the target image block based on the grayscale change rate between adjacent pixels within the target image block," includes: Perform neighborhood convolution operation on the pixels within the target image block to obtain the horizontal gradient component in the horizontal direction and the vertical gradient component in the vertical direction for each pixel. Based on the horizontal and vertical gradient components, the local gradient value of each pixel is determined using the target magnitude estimation logic. The average of the local gradient values of all pixels within the target image block is determined as the overall gradient value of the target image block; the overall gradient value characterizes the edge features of the target image block. The horizontal gradient component is used to characterize the grayscale change rate of a pixel in the horizontal direction, while the vertical gradient component is used to characterize the grayscale change rate of a pixel in the vertical direction.
[0032] In this embodiment, neighborhood convolution is performed on pixels within the target image block. First, a horizontal convolution kernel is defined. Convolution kernel in the vertical direction In one example,
[0033]
[0034] Then, for the 3×3 neighborhood matrix of each pixel within the target image patch , respectively with , Perform convolution operations to obtain the horizontal gradient components. and vertical gradient components ,in,
[0035]
[0036] Horizontal gradient components The quantization represents the rate of grayscale change of a pixel in the horizontal direction; a larger value indicates a more pronounced abrupt change in grayscale in the horizontal direction. The vertical gradient component... The quantization characterizes the rate of grayscale change of a pixel in the vertical direction. The larger the value, the more obvious the grayscale change in the vertical direction. Both gradient components are non-negative integer values, and the absolute value operation is achieved by comparing the sign bit in hardware.
[0037] In this embodiment, the target amplitude estimation logic is a gradient amplitude approximation calculation rule adapted to the hardware circuit. It abandons the traditional complex square and square root operations, estimating the gradient amplitude only through hardware shifting, addition, and subtraction operations, significantly improving hardware execution efficiency. The core of this estimation logic is to calculate the local gradient value G of each pixel using an approximate formula, that is,
[0038] because Multiplication and division operations can be performed using bit shifting to obtain the local gradient values of each pixel. The specific calculation formula is as follows:
[0039] in, To be The binary value is shifted left by 3 bits, which is equivalent to ;Gy<<2 is the value of Gy<<2 The binary value is shifted left by 2 bits, which is equivalent to ;()>>4 means shifting the binary value of the calculation result within the parentheses to the right by 4 bits, which is equivalent to dividing the result by 16. The horizontal gradient component of each pixel. and vertical gradient components After substituting the target amplitude estimation logic, the calculated result is the local gradient value of the pixel. The local gradient value represents the overall grayscale change rate of a single pixel. The larger the value, the more obvious the edge and the more drastic the texture change at the pixel.
[0040] In this embodiment, all pixels within the target image block obtain their corresponding local gradient values through the above steps, denoted as G1, G2, ..., Gn (for example, for an 8×8 image block, n is 64). Hardware accumulation is performed on all local gradient values to obtain a sum of G1 + G2 + ... + Gn. Then, a hardware right shift operation is performed on this sum. For an 8×8 image block, the sum is shifted 6 bits to the right, which represents the overall gradient value of the current target image block. This shift operation is equivalent to dividing the sum by 64, and the result is the average of the local gradient values of all pixels. This average value is defined as the overall gradient value of the target image patch. The overall gradient value characterizes the edge features of the target image patch, and its magnitude is positively correlated with the overall degree of change in edges and textures within the target image patch. The larger the overall gradient value, the denser the edges and the more dramatic the texture changes within the image patch; the smaller the overall gradient value, the less obvious the edges and the smoother the texture changes within the image patch.
[0041] In this disclosure, the process of determining edge features is refined and broken down. Horizontal and vertical gradient components are obtained through Sobel operator convolution. Local gradient values are then calculated using hardware-friendly target amplitude estimation logic. Finally, the overall gradient value is obtained through statistical averaging, achieving accurate and efficient quantization of edge features of target image blocks. All operations are implemented using basic logic such as hardware shifting, addition, and subtraction, eliminating complex square, square root, and floating-point multiplication and division operations. This significantly improves hardware execution efficiency and real-time performance, fully adapting to the chip-level JPEG encoding hardware implementation requirements.
[0042] In another embodiment, based on the horizontal and vertical gradient components, the local gradient value of each pixel is determined using target magnitude estimation logic, including: Shift the vertical gradient components all to the left by 3 bits to obtain the first intermediate number; Shift the vertical gradient components to the left by 2 bits to obtain the second intermediate number; The difference between the sum of the first and second intermediate numbers and the vertical gradient component is determined as the third intermediate number; Shift the third intermediate number 4 bits to the right to obtain the fourth intermediate number; The sum of the horizontal gradient component and the fourth intermediate number is used to determine the local gradient value of the pixel.
[0043] The method for determining the local gradient value of each pixel in this embodiment is similar to the formula. Correspondingly, for the vertical gradient component Shifting the entire array 3 bits to the left yields the first median, which is the... For vertical gradient components Shifting the entire array two positions to the left yields the second median, which is the second median. ; Sum of the first and second intermediates and the vertical gradient component The difference is determined as the third median, which is the third median. Shifting the third intermediate number four positions to the right yields the fourth intermediate number, which is the... Finally, the horizontal gradient components will be... The sum of the first and fourth intermediate values is used to determine the local gradient value of the pixel. , that is .
[0044] In this embodiment, To be The binary value is shifted left by 3 bits, which is equivalent to ;Gy<<2 is the value of Gy<<2 The binary value is shifted left by 2 bits, which is equivalent to The sum of the two is Subtract Later obtained And then , equivalent to Divide by 16, which is the fourth middle number. Finally, the horizontal gradient components are... Adding it to the fourth middle number, we get , that is .
[0045] In this disclosure, by decomposing the target amplitude estimation logic into specific hardware shift and addition / subtraction steps, a pure hardware circuit for gradient amplitude approximation calculation is implemented. This eliminates the complex square and square root operations in traditional gradient amplitude calculation, while transforming floating-point multiplication and division operations into simple shift and basic addition / subtraction operations. This significantly reduces the resource consumption of the hardware circuit, improves the calculation speed and execution efficiency, and fully adapts to the real-time computing requirements of JPEG encoding chips. Each step is an integer operation, with no precision loss in the floating-point conversion process. Furthermore, by approximating 0.7 using 11 / 16, high hardware efficiency is achieved while ensuring estimation accuracy. This allows the gradient amplitude calculation to be performed synchronously with the image encoding pipeline processing without adding additional encoding latency.
[0046] In another embodiment, step S202, "determining the grayscale distribution characteristics of the target image block based on the statistical dispersion of pixel values within the target image block," includes: The average gray value of all pixels within the target image block is determined as the baseline gray value; Determine the deviation between the grayscale value of each pixel within the target image block and the reference grayscale value; The squared deviation values of all pixels within the target image block are summed to obtain the summation result; The accumulated result is shifted 6 bits to the right to obtain the variance of pixel values within the target image block; the variance characterizes the statistical dispersion of the target image block, and the statistical dispersion is used to reflect the gray-level distribution characteristics of the target image block.
[0047] In this embodiment, the grayscale values of all pixels within the target image block are first accumulated. For example, for an 8×8 pixel target image block, its pixel matrix is denoted as Data[i][j] (i,j∈[0,7]). The grayscale values of pixels in the target image block are first accumulated row by row to obtain the row accumulation value RowSum[k] (k∈[0,7]) of 8 pixels in each row. The row accumulation is performed by grouping and adding, dividing the 8 pixels in each row into 4 groups, adding 2 pixels in each group, and then summing them up. This reduces the hardware pressure of a single addition operation.
[0048] in, It refers to 8 pixels per row.
[0049] Then, the cumulative values of the 8 rows are summed column-to-column to obtain the total cumulative value Sum of the 64 pixel grayscale values. Finally, a hardware shift operation is performed on the total cumulative value Sum, shifting it 6 bits to the right. That is, the base gray value u = Sum >> 6. This shift operation is equivalent to dividing the total cumulative value Sum by 64, and the result is the base gray value.
[0050] In this embodiment, the deviation value is the difference between the gray value of each pixel in the target image block and the reference gray value u. This calculation process is performed independently for the 64 pixels in the 8×8 pixel matrix. Each pixel completes the subtraction operation between the gray value and the reference gray value through a hardware subtractor to obtain the corresponding deviation value. The deviation value can be a positive integer, a negative integer or 0, which respectively represent that the gray value of the pixel is higher than, lower than or equal to the reference gray value.
[0051] In this embodiment, after obtaining the deviation values of all pixels in the target image block, the deviation value of each pixel is squared, and the squared deviation values of all pixels are accumulated to obtain the accumulated result. Finally, the accumulated result is shifted 6 bits to the right by hardware, which is equivalent to dividing the accumulated result by 64 to obtain the average value of the squared deviation values of 64 pixels, that is, the variance of the pixel values in the target image block.
[0052] In one example, the variance of pixel values within a target image patch can be calculated based on the following formula:
[0053] in, Let be the variance of pixel values within the target image patch, where i represents the i-th row in the pixel matrix of the target image patch, and j represents the j-th column in the pixel matrix of the target image patch. This represents the grayscale value of a single pixel at the i-th row and j-th column position in the pixel matrix of the target image block, where u is the baseline grayscale value.
[0054] In this embodiment, variance is the core parameter characterizing the statistical dispersion of pixel values within a target image patch. A larger variance value indicates a greater overall deviation between the pixel grayscale value and the baseline grayscale value, resulting in higher statistical dispersion and a more uneven grayscale distribution within the image patch. Conversely, a smaller variance value indicates a smaller overall deviation between the pixel grayscale value and the baseline grayscale value, resulting in lower statistical dispersion and a more uniform grayscale distribution within the image patch. This variance value is directly used as a grayscale distribution feature parameter of the target image patch and participates in subsequent image patch classification.
[0055] In this disclosure, by breaking down the process of determining grayscale distribution characteristics into four specific hardware implementation steps—calculation of baseline grayscale value, determination of deviation value, accumulation of squared deviation values, and calculation of variance by shifting—precise quantification of the statistical dispersion of pixel values in the target image block is achieved. The variance is directly used as a grayscale distribution characteristic parameter, which can truly and intuitively reflect the uniformity of grayscale distribution within the image block, providing an accurate and reliable basis for subsequent image block classification and dynamic allocation of power parameters.
[0056] In another embodiment, step S203, "determining the frequency domain characteristics of the target image block based on the proportion of signal components in the target image block that are higher than a preset frequency threshold," includes: The target image patch is transformed and encoded to obtain the frequency domain coefficient matrix; The set of coefficients located within a predetermined row and column range at the edge of the frequency domain coefficient matrix is defined as the target component region; The ratio of the number of non-zero coefficients in the target component region to the total number of coefficients in the frequency domain coefficient matrix is determined as the signal component proportion; the signal component proportion characterizes the frequency domain features of the target image patch. The target component region includes coefficients within a preset number of rows near the bottom edge of the matrix, coefficients within a preset number of columns near the right edge of the matrix, a preset number of coefficients in the center row of the matrix near the right edge of the matrix, and a preset number of coefficients in the center column of the matrix near the bottom edge of the matrix.
[0057] In this embodiment, the transform encoding performed on the target image block can be the Discrete Cosine Transform (DCT) specified by the JPEG encoding standard. The DCT transform converts the target image block from a pixel grayscale matrix in the spatial domain to a coefficient matrix in the frequency domain. This frequency domain coefficient matrix has the same size as the pixel matrix of the original target image block. Each coefficient in the matrix corresponds to a signal component of a different frequency in the image. Low-frequency coefficients correspond to the overall contour information of the image, while high-frequency coefficients correspond to the details and texture information of the image. The coefficients in the frequency domain coefficient matrix are quantized values, which can be positive, negative integers, or 0. Their magnitude represents the intensity of the corresponding frequency signal component. In one example, the frequency domain coefficient matrix includes the corresponding frequency coordinates (u,v) in the following order (8×8 coefficient matrix): (0,0)→(0,1)→(1,0)→(2,0)→(1,1)→(0,2)→(0,3)→(1,2)→ →(7,7).
[0058] In this embodiment, the preset row and column range is a hardware configuration parameter pre-set based on the Zig-Zag scanning path of JPEG encoding and the distribution pattern of high-frequency components in the image. The default preset row and column range is the high-frequency component distribution area of the adaptive frequency domain coefficient matrix, i.e., the target component area. In one example, the target component area includes coefficients within a preset number of rows near the bottom edge of the matrix, coefficients within a preset number of columns near the right edge of the matrix, a preset number of coefficients in the center row of the matrix near the right edge of the matrix, and a preset number of coefficients in the center column of the matrix near the bottom edge of the matrix. For example, for an 8×8 frequency domain coefficient matrix, the target component area may include: all coefficients in rows 5-7 (u=5,6,7) and columns 5-7 (v=5,6,7); coefficients in the 4th row of the matrix with column number v>4, i.e., coefficients with u=4 and v=5,6,7; and coefficients in the 4th column of the matrix with row number u>4, i.e., coefficients with v=4 and u=5,6,7.
[0059] In this embodiment, a hardware non-zero determination operation is first performed on all coefficients within the target component region. A hardware comparator compares each coefficient with 0, and the number of coefficients with non-zero values is counted, denoted as N_hf. This statistical process is pipelined with the coefficient filtering process of the target component region. After filtering, the non-zero statistics are performed immediately, and parallel determination of multiple coefficients is supported. For an 8×8 frequency domain coefficient matrix, the total number of coefficients is fixed at 64, and the signal component proportion is the ratio of N_hf to 64. In the hardware implementation, the number of non-zero coefficients, N_hf, can be directly used as a quantization parameter of the frequency domain feature in subsequent classification, without the need for division. The value of N_hf is positively correlated with the actual signal component proportion, fully representing the proportion of high-frequency signal components. This signal component proportion (N_hf) is the frequency domain feature parameter of the target image block. The larger the value, the more high-frequency signal components the target image block has, resulting in richer image details and more complex textures; the smaller the value, the fewer high-frequency signal components the target image block has, resulting in less image details and smoother content.
[0060] In this disclosure, by breaking down the process of determining frequency domain features into three specific steps—obtaining the frequency domain matrix through DCT transformation, defining the target component region, and statistically analyzing the proportion of non-zero coefficients—the accurate quantification of the proportion of high-frequency signal components in the target image block is achieved. Using this proportion as a frequency domain feature parameter can realistically and intuitively reflect the richness of detail in the image block, providing an accurate and reliable basis for subsequent image block classification and dynamic allocation of power parameters.
[0061] In another embodiment, step S103, "determining the target power parameters required for processing the target image patch based on the feature parameters," includes: Based on feature parameters, the classification result of the target image patch is determined; the classification result includes smooth patch, edge patch, mixed patch and texture patch; In response to the target image block being a smooth block, the target power parameters are determined as the first power parameters; In response to the target image patch being an edge patch, the target power parameters are determined as the second power parameters; In response to the target image block being a mixed block, the target power parameters are determined as the third power parameters; In response to the target image block being a texture block, the target power parameters are determined to be the fourth power parameters; Among them, from the first power supply parameter to the fourth power supply parameter, the power supply voltage and power supply frequency become increasingly larger.
[0062] In this embodiment, based on the edge features (overall gradient value) in the feature parameters The classification results of the target image blocks are determined by the gray-scale distribution characteristics (variance σ²) and frequency domain characteristics (number of high-frequency non-zero coefficients N_hf). The classification results are divided into four categories: smooth blocks, edge blocks, mixed blocks, and texture blocks, which correspond to four levels of image content from simple to complex. Smooth blocks are the image blocks with the smoothest content and the lowest complexity. Texture blocks are the image blocks with the richest texture, the most complex details, and the highest complexity. Mixed blocks and edge blocks are intermediate types with complexity between the two. These four classification results can be transmitted in hardware through a 2-bit binary signal Class[1:0], corresponding to Class=00 (smooth block), Class=01 (edge block), Class=10 (mixed block), and Class=11 (texture block), respectively, which facilitates the rapid identification and matching of the power management control unit.
[0063] In this embodiment, for the smooth block determination signal (Class=00), the first power supply parameter is directly matched and output. This parameter is a pre-configured combination of low-power power supply parameters, adapted to the low computational complexity encoding requirements of smooth blocks. The first power supply parameter specifically includes a power supply voltage control signal and a frequency control signal. The default configuration is that the voltage control signal Vctrl=000, corresponding to an actual power supply voltage of 0.8V, and the frequency control signal Fctrl=0011, corresponding to an actual operating frequency of 100MHz. This parameter is the combination of the lowest power supply voltage and the lowest operating frequency among the four types of power supply parameters. It can minimize hardware power consumption and avoid power waste caused by excessive power supply while meeting the smooth block encoding operation requirements.
[0064] In this embodiment, the edge block determination signal (Class=01) uses a combination of medium to low power supply parameters adapted to the computational complexity of edge blocks, i.e., the second power supply parameters. Its supply voltage and operating frequency are higher than the first power supply parameters, satisfying the slightly higher computational complexity requirements of edge blocks due to the presence of edge information. The default configuration of the second power supply parameters is a voltage control signal Vctrl=010, corresponding to an actual supply voltage of 1.0V, and a frequency control signal Fctrl=0100, corresponding to an actual operating frequency of 150MHz. This parameter balances power consumption and performance while ensuring edge block encoding speed and quality.
[0065] In this embodiment, for the hybrid block determination signal (Class=10), a third power supply parameter is output. The third power supply parameter is a combination of medium to high power supply parameters adapted to the computational complexity of the hybrid block. Its supply voltage and operating frequency are higher than the second power supply parameter, meeting the higher computational complexity requirements of the hybrid block due to the coexistence of edge and high-frequency information. The default configuration of the third power supply parameter is a voltage control signal Vctrl=100, corresponding to an actual supply voltage of 1.2V, and a frequency control signal Fctrl=0110, corresponding to an actual operating frequency of 200MHz. This parameter can provide sufficient hardware computing power for the encoding operation of the hybrid block, ensuring that the encoding process does not experience a decrease in speed or quality. At the same time, compared with the highest level of power supply parameters, a certain amount of power consumption control space is still reserved.
[0066] In this embodiment, for the texture block determination signal (Class=11), a fourth power supply parameter is output. This fourth power supply parameter is the highest-level power supply parameter combination adapted to the computational complexity of the texture block. Its power supply voltage and operating frequency are the highest among the four types of parameters, meeting the highest computational complexity requirements caused by the rich texture and complex details of the texture block. The default configuration of the fourth power supply parameter is a voltage control signal Vctrl=110, corresponding to an actual power supply voltage of 1.8V, and a frequency control signal Fctrl=1000, corresponding to an actual operating frequency of 300MHz. This parameter can provide sufficient hardware computing power for the entire encoding operation of the texture block, including DCT transformation, quantization, and entropy coding, ensuring that the encoding speed and encoding quality of complex image blocks meet the preset standards and avoiding encoding anomalies caused by insufficient power.
[0067] In this embodiment, the first to fourth power parameters correspond sequentially according to the classification order of smooth blocks, edge blocks, mixed blocks, and texture blocks. Their power supply voltage and operating frequency show a step-by-step increasing pattern, which perfectly matches the increasing pattern of the complexity of the image block content. That is, the more complex the image content, the higher the power supply voltage and operating frequency configured for the encoding hardware, and the stronger the hardware computing power, which can fully meet the encoding computing needs of image blocks with different levels of complexity.
[0068] In this disclosure, by first converting feature parameters into image patch classification results, and then matching tiered power parameters based on the classification results, a precise and linear match between the complexity of image content and the power parameters of JPEG encoding hardware is achieved. This breaks the traditional fixed power supply mode and realizes refined dynamic power allocation on an image patch-by-image basis. The tiered, incremental power parameter settings allow simple image patches to be matched with low-power power parameters, avoiding power waste caused by excessive hardware power supply, while complex image patches are matched with high-computation power parameters, ensuring encoding speed and encoding quality. This fundamentally solves the problems of low power efficiency and the inability to balance performance and power consumption in existing technologies.
[0069] In another embodiment, the feature parameters of the target image block include edge features, gray-scale distribution features, and frequency-domain features; based on the feature parameters, determining the classification result of the target image block includes: In response to the edge feature being less than the first threshold and the gray-scale distribution feature being less than the second threshold, determining that the target image block is a smooth block; In response to the edge feature being greater than or equal to the first threshold, the gray-scale distribution feature being greater than the second threshold, and the frequency-domain feature being greater than or equal to the third threshold, determining that the target image block is a texture block; In response to the edge feature being greater than or equal to the first threshold, the gray-scale distribution feature being less than or equal to the second threshold, and the frequency-domain feature being greater than or equal to the third threshold, determining that the target image block is a mixed block; In response to the target image block not belonging to a smooth block, a texture block, or a mixed block, determining that the target image block is an edge block.
[0070] In this embodiment, the first threshold corresponding to the edge feature is the gradient threshold Th_grad, and the second threshold corresponding to the gray-scale distribution feature is the variance threshold Th_var. These two thresholds are hardware parameters pre-configured in the classification decision unit, with the default configuration being Th_grad = 32 and Th_var = 128, and both can be dynamically modified through the configuration register Config_REG[7:0]. The determination condition for a smooth block needs to simultaneously satisfy <Th_grad and σ² < Th_var, and neither condition can be missing. Among them <Th_grad represents that the overall gradient value of the target image block is small, there are no obvious edge and texture changes in the image, and the gray-scale change rate is low; σ² < Th_var represents that the pixel gray-scale values of the target image block have a low degree of dispersion, the gray-scale distribution is highly uniform, and there are no obvious gray-scale differences. When both conditions are satisfied, it indicates that the target image block is the simplest smooth area in terms of content, and the classification decision unit determines it as a smooth block and outputs the corresponding classification signal Class = 00.
[0071] In this embodiment, the third threshold corresponding to the frequency-domain feature is the high-frequency component threshold Th_hf. This threshold is a preset hardware parameter of the classification decision unit, with the default configuration being Th_hf = 8, and it supports dynamic adjustment through the configuration register. The determination condition for a texture block needs to simultaneously satisfy ≥Th_grad, σ² > Th_var, N_hf ≥ Th_hf, and the three sub-conditions. Among them ≥Th_grad indicates that the image patch has drastic edge and texture changes and a high grayscale change rate; σ²>Th_var indicates that the image patch has extremely uneven grayscale distribution and a large number of pixels with different grayscale levels; N_hf≥Th_hf indicates that the image patch has rich high-frequency components and a lot of detailed information. When all three conditions are met, it indicates that the target image patch is a high-complexity region with rich texture and complex details. The classification decision unit classifies it as a texture patch and outputs the classification signal Class=11.
[0072] In this embodiment, the determination criteria for the mixed block must be met simultaneously. The three sub-conditions are: ≥Th_grad, σ²≤Th_var, and N_hf≥Th_hf. ≥Th_grad and N_hf≥Th_hf indicate that the image patch has obvious edge variations and rich high-frequency details, exhibiting characteristics of a complex image patch; while σ²≤Th_var indicates that the image patch has low pixel grayscale value dispersion and relatively uniform grayscale distribution, exhibiting characteristics of a simple image patch. This type of image patch is a mixed region where edges, details, and uniform grayscale distribution coexist, and is therefore classified as a mixed patch. After classifying it as a mixed patch, the classification decision unit outputs the corresponding classification signal Class=10. The encoding computation complexity of this type of image patch is between that of edge patches and texture patches, and it will subsequently be matched with moderately high power parameters.
[0073] In this embodiment, edge blocks are considered a fallback in the classification process. If the feature parameter combination of a target image block does not meet any of the criteria for smooth blocks, textured blocks, or mixed blocks, it is classified as an edge block. These image blocks possess only obvious edge information, lacking rich high-frequency details or uneven grayscale distribution. Their encoding computation complexity falls between that of smooth blocks and mixed blocks. After classifying an image block as an edge block, the classification decision unit outputs a classification signal Class=01, which will then be matched with moderately low power parameters.
[0074] In this disclosure, a clear judgment logic based on the combination of feature parameter thresholds is used to achieve accurate classification of target image blocks. The image blocks are divided into four categories: smooth blocks, edge blocks, mixed blocks, and texture blocks, which correspond to four levels of content complexity from low to high. This provides a clear and specific judgment basis for the subsequent matching of dynamic power parameters, ensuring accurate matching between power allocation and image content complexity.
[0075] In another embodiment, an image processing method further includes: Determine the proportions of smooth blocks, edge blocks, blended blocks, and texture blocks within a preset number of frames; In response to a smooth block proportion exceeding the fourth threshold, the corresponding adjustment values for the first and second thresholds are increased. In response to a texture block proportion exceeding the fifth threshold, the corresponding adjustment values for the second and third thresholds are reduced. In response to the proportion of mixed blocks exceeding the sixth threshold, the corresponding adjustment values for the first and third thresholds are increased; In response to the smooth block proportion being less than the seventh threshold and the texture block proportion being less than the eighth threshold, the corresponding adjustment values for the first and second thresholds are decreased, and the corresponding adjustment value for the third threshold is increased.
[0076] In this embodiment, the preset frame number is the number of frames in the statistical window pre-configured by the hardware, with a default configuration of 100 consecutive frames. This value can be dynamically adjusted through the configuration register to adapt to different encoding speeds and image content change rhythms. The statistical object is all target image blocks obtained after each frame of image is processed into blocks. All target image blocks within a consecutive preset frame number are counted according to four categories: smooth blocks, edge blocks, blended blocks, and texture blocks, and the cumulative number of each category is obtained. The proportion of each type of image block is calculated as the ratio of the cumulative number of the corresponding type of image block to the total number of all image blocks within the preset frame number, denoted as the smooth block proportion. Edge block ratio Mixed block ratio Texture block ratio .
[0077] In this embodiment, when the statistically obtained smooth block proportion When the threshold value exceeds the fourth threshold, it indicates that the overall content of the current input image is too smooth, and a large number of image blocks are identified as smooth blocks. At this point, the first threshold (gradient threshold Th_grad) and the second threshold (variance threshold Th_var) are increased by the corresponding adjustment values. The adjustment values are pre-configured fixed step sizes (such as 2, 4, etc.), which can be configured through hardware registers. After increasing the first and second thresholds, the criteria for identifying smooth blocks become more stringent. Some smooth blocks that were originally at the decision boundary will be re-identified as edge blocks, thus balancing the proportion of various image blocks and avoiding the problem of insufficient computing power when encountering sudden complex image blocks, which would result in the encoding hardware operating at low power and low frequency for extended periods due to an excessively high proportion of smooth blocks.
[0078] In this embodiment, when the statistically obtained texture block proportion When the threshold exceeds the fifth threshold, it indicates that the current input image content has rich texture and complex details, with a large number of image blocks being identified as texture blocks. This causes the encoding hardware to operate at high power consumption and high frequency for extended periods, resulting in unnecessary power waste. In this case, the second threshold (variance threshold Th_var) and the third threshold (high-frequency component threshold Th_hf) are reduced by a preset fixed step size. Reducing the second and third thresholds makes the texture block identification criteria more stringent. Texture blocks that were originally at the boundary will be reclassified as mixed blocks or edge blocks, reducing the proportion of texture blocks and making the encoding hardware's operation more closely match the actual complexity of the image content. This reduces overall power consumption while maintaining encoding quality.
[0079] In this embodiment, when the statistically obtained proportion of mixed blocks When the value exceeds the sixth threshold, it indicates that the current input image content has a mixed feature of edges and details, with a large number of image blocks being identified as mixed blocks. This causes the encoding hardware to operate in a medium-to-high power consumption frequency range, lacking flexible dynamic adjustment. In this case, the first threshold (gradient threshold Th_grad) and the third threshold (high-frequency component threshold Th_hf) are increased by a preset fixed step size. Increasing the first and third thresholds makes the criteria for identifying mixed blocks more stringent. Some mixed blocks that were originally at the boundary will be reclassified as edge blocks or texture blocks, dispersing the proportion of mixed blocks and making the distribution of various image blocks more reasonable, thus improving the fineness of dynamic power parameter allocation.
[0080] In this embodiment, when the smooth block proportion Less than the seventh threshold, and the proportion of texture blocks When the threshold value is less than the eighth threshold, it indicates that the current input image content lacks both a large number of smooth areas and a large number of texture-rich areas. The image blocks are mainly concentrated in edge blocks and mixed blocks, and the classification results show a problem of intermediate clustering. This reflects that the current classification threshold setting is unreasonable, resulting in an overly narrow judgment range for simple and complex image blocks. At this time, the adjustment values of the first threshold (gradient threshold Th_grad) and the second threshold (variance threshold Th_var) are decreased, while the adjustment value of the third threshold (high-frequency component threshold Th_hf) is increased. All adjustment values are preset fixed step sizes. Decreasing the first and second thresholds can expand the judgment range of smooth blocks, classifying some boundary edge blocks as smooth blocks; increasing the third threshold can expand the judgment range of texture blocks, classifying some boundary mixed blocks as texture blocks. This makes the proportion of various types of image blocks more balanced, allowing the dynamic adjustment of power parameters to cover more power consumption frequency ranges, and further optimizing the balance between power consumption and performance.
[0081] In one example, the first threshold (gradient threshold Th_grad), the second threshold (high-frequency component threshold Th_hf), and the third threshold can be adjusted using the following Table 1: Table 1
[0082] In this disclosure, by adding a threshold adaptive adjustment step based on the classification ratio of image blocks in a preset number of frames, a closed-loop feedback system for classification determination and threshold adjustment is constructed. This enables the classification threshold to be dynamically optimized according to the actual input image content features, solving the classification deviation problem that is prone to occur when fixed thresholds are faced with diverse and differentiated image content. This significantly improves the accuracy and adaptability of image block classification, and makes the dynamic allocation of power parameters more in line with the actual image encoding computation complexity requirements.
[0083] In another embodiment, an image processing method further includes: In response to changes in configuration parameters in hardware registers, at least one of the first threshold, second threshold, third threshold, first power supply parameter, second power supply parameter, third power supply parameter, and fourth power supply parameter is adjusted; The hardware registers include a threshold register for storing classification thresholds, a voltage mapping register for storing the supply voltage in the power supply parameters, and a frequency mapping register for storing the supply frequency in the power supply parameters.
[0084] In this embodiment, changes to the configuration parameters of the hardware registers include modifications and writes made by the user through chip debugging interfaces, external control circuits, etc., to the values stored in the registers. When a change in the configuration parameters in the registers is detected, the corresponding hardware circuit will immediately read the new parameter values and, according to the functional definition of the registers, synchronously update and adjust one or more of the first threshold, second threshold, third threshold, and various power parameters. The first threshold is the gradient threshold Th_grad, the second threshold is the variance threshold Th_var, and the third threshold is the high-frequency component threshold Th_hf, corresponding to the three core quantization indicators for image block classification. The first to fourth power parameters are combinations of power supply voltage and operating frequency for matching smooth blocks, edge blocks, mixed blocks, and texture blocks, respectively, all of which are core parameters for the encoding hardware operation. The parameter adjustment process is automatically completed by the register read / write logic of the hardware circuit, without complex calculations. The adjusted parameters are directly loaded into the calculation logic and immediately participate in subsequent image block classification and power parameter matching, ensuring the real-time performance of the parameter adjustment. For example, if a user writes a new gradient threshold value of 40 to the threshold register using a debugging tool, the hardware circuit will immediately adjust the first threshold from the default 32 to 40 after detecting the change in configuration parameters. Subsequent image block classification decisions will all be performed using the new first threshold.
[0085] In this embodiment, the hardware register is a dedicated digital register integrated into the hardware architecture of the JPEG encoded dynamic power distribution system. It adopts a read-write storage logic design, supports online writing and reading of parameters, and the bit width of the register is customized according to the numerical range of the stored parameters to ensure the accuracy and integrity of parameter storage. The various registers have clear functions and correspond to different control parameters, including threshold register, voltage mapping register Vctrl, and frequency mapping register Fctrl. At the same time, other functional registers can be expanded according to actual application needs to form a systematic parameter storage and control system. The threshold register is a dedicated register for storing classification thresholds. It can be further subdivided into gradient threshold register, variance threshold register, and high-frequency component threshold register, which independently store the first threshold Th_grad, the second threshold Th_var, and the third threshold Th_hf, respectively. It supports independent configuration of a single threshold and synchronous configuration of multiple thresholds. The voltage mapping register Vctrl is a dedicated register for storing power supply voltage control signals. It has a bit width of 3 bits (Vctrl[2:0]). Different combinations of binary values correspond to different actual power supply voltages. The register stores the voltage control signal values corresponding to smooth blocks, edge blocks, blend blocks, and texture blocks, respectively. It can independently adjust the power supply voltage matched to a single image block and can also synchronously adjust multiple voltage parameters. The frequency mapping register Fctrl is a dedicated register for storing operating frequency control signals. It has a bit width of 4 bits (Fctrl[3:0]). Different combinations of binary values correspond to different actual operating frequencies. The register stores the frequency control signal values corresponding to the four types of image blocks, respectively. It supports independent configuration of single or multiple frequency parameters.
[0086] In this disclosure, by adding a step of online configuration of hardware register parameters, a dual parameter control system of automatic adjustment and manual configuration is constructed. This not only retains the intelligent adjustment capability of the adaptive parameter optimization algorithm, but also makes up for the lack of adaptability of pure automatic adjustment in special application scenarios, which greatly improves the flexibility, adaptability and engineering practicality of the image processing method.
[0087] Figure 4 A schematic diagram of the structure of an image processing system according to an embodiment of the present disclosure is shown. Figure 1 ,like Figure 4 As shown, an image processing system includes: The feature extraction module 10 is used to divide the image to be processed into blocks to obtain multiple target image blocks; to perform multi-dimensional feature extraction on the target image blocks to obtain feature parameters of the target image blocks; the feature parameters characterize the content complexity of the target image blocks. The feature extraction module 10 can also be called the image content analysis module ICAC.
[0088] The power control module 20 is used to determine the target power parameters required for processing the target image block based on the feature parameters; the target power parameters corresponding to each target image block may be the same or different. The power control module 20 may also be referred to as the power management control module (PMC).
[0089] The parameter adjustment module 30 is used to adjust the power supply parameters of the image encoding module 40 to the target power supply parameters. The parameter adjustment module 30 can also be referred to as the Dynamic Voltage and Frequency Adjustment Module (DVFS).
[0090] Image encoding module 40 is used to encode corresponding target image blocks based on target power parameters. Parameter adjustment module 30 can also be referred to as the JPEG encoding core module.
[0091] Figure 5 A schematic diagram of the structure of an image processing system according to an embodiment of the present disclosure is shown. Figure 2 ,like Figure 5 As shown, the feature extraction module 10 in the image processing system includes an image segmentation unit, a feature extraction unit, and a classification decision unit. The image segmentation unit is used to segment the image to be processed into multiple target image blocks. The feature extraction unit is used to extract multi-dimensional features from the target image blocks to obtain feature parameters of the target image blocks. The classification decision unit is used to determine the classification result of the target image blocks based on the feature parameters. The classification results include smooth blocks, edge blocks, mixed blocks, and texture blocks. It should be emphasized that the classification of the target image blocks can be performed in the classification decision unit of the feature extraction module 10 or in the power control module 20.
[0092] Figure 6 A schematic diagram of the structure of an image processing system according to an embodiment of the present disclosure is shown. Figure 3 ,like Figure 6 As shown, in the feature extraction module 10 of the image processing system, after the input image data is divided into multiple image blocks of 8×8 pixels, a dual-port RAM cache structure is adopted, containing two 8×8 pixel buffers (BufA and BufB), and pipelined processing is achieved through ping-pong operation. The depth of each RAM is 64, and the bit width is selected according to the pixel specification. For example, in RGB888 format, the bit width is 24 bits. When the first group (one 8×8 pixel group) of images is input, it is first cached in BufA; when the second group of images is input, it is cached in BufB, and at the same time, the image data in BufA is read out for subsequent feature extraction units; when the third group of images is input, the first group of images has been read from BufA and feature extraction has been completed. At this time, the third group of images is cached in BufA, and at the same time, the second group of image data in BufB is read out for subsequent feature extraction units, and so on to achieve ping-pong operation. Figure 6The input control is responsible for receiving raw image data, performing format adaptation, timing synchronization, and rate control, and accurately caching the data to BufA or BufB to provide a stable data source for subsequent processing; the output control is responsible for reading image block data from BufA or BufB and outputting it.
[0093] In another embodiment, the feature extraction module 10 is further configured to: The edge features of the target image block are determined based on the gray-level change rate between adjacent pixels within the target image block; the edge features characterize the degree of change of edges and / or texture in the target image block. Based on the statistical dispersion of pixel values within the target image block, the gray-level distribution characteristics of the target image block are determined; the gray-level distribution characteristics characterize the uniformity of gray-level distribution in the target image block. The frequency domain features of the target image block are determined based on the proportion of signal components in the target image block that are higher than a preset frequency threshold; the frequency domain features characterize the detail richness of the target image block.
[0094] In another embodiment, the feature extraction module 10 is further configured to: Perform neighborhood convolution operation on the pixels within the target image block to obtain the horizontal gradient component in the horizontal direction and the vertical gradient component in the vertical direction for each pixel. Based on the horizontal and vertical gradient components, the local gradient value of each pixel is determined using the target magnitude estimation logic. The average of the local gradient values of all pixels within the target image block is determined as the overall gradient value of the target image block; the overall gradient value characterizes the edge features of the target image block. The horizontal gradient component is used to characterize the grayscale change rate of a pixel in the horizontal direction, while the vertical gradient component is used to characterize the grayscale change rate of a pixel in the vertical direction.
[0095] In another embodiment, the feature extraction module 10 is further configured to: Shift the vertical gradient components all to the left by 3 bits to obtain the first intermediate number; Shift the vertical gradient components to the left by 2 bits to obtain the second intermediate number; The difference between the sum of the first and second intermediate numbers and the vertical gradient component is determined as the third intermediate number; Shift the third intermediate number 4 bits to the right to obtain the fourth intermediate number; The sum of the horizontal gradient component and the fourth intermediate number is used to determine the local gradient value of the pixel.
[0096] In another embodiment, the feature extraction module 10 is further configured to: The average gray value of all pixels within the target image block is determined as the baseline gray value; Determine the deviation between the grayscale value of each pixel within the target image block and the reference grayscale value; The squared deviation values of all pixels within the target image block are summed to obtain the summation result; The accumulated result is shifted 6 bits to the right to obtain the variance of pixel values within the target image block; the variance characterizes the statistical dispersion of the target image block, and the statistical dispersion is used to reflect the gray-level distribution characteristics of the target image block.
[0097] In another embodiment, the feature extraction module 10 is further configured to: The target image patch is transformed and encoded to obtain the frequency domain coefficient matrix; The set of coefficients located within a predetermined row and column range at the edge of the frequency domain coefficient matrix is defined as the target component region; The ratio of the number of non-zero coefficients in the target component region to the total number of coefficients in the frequency domain coefficient matrix is determined as the signal component proportion; the signal component proportion characterizes the frequency domain features of the target image patch. The target component region includes coefficients within a preset number of rows near the bottom edge of the matrix, coefficients within a preset number of columns near the right edge of the matrix, a preset number of coefficients in the center row of the matrix near the right edge of the matrix, and a preset number of coefficients in the center column of the matrix near the bottom edge of the matrix.
[0098] In another embodiment, the power control module 20 is further configured to: Based on feature parameters, the classification result of the target image patch is determined; the classification result includes smooth patch, edge patch, mixed patch and texture patch; In response to the target image block being a smooth block, the target power parameters are determined as the first power parameters; In response to the target image patch being an edge patch, the target power parameters are determined as the second power parameters; In response to the target image block being a mixed block, the target power parameters are determined as the third power parameters; In response to the target image block being a texture block, the target power parameters are determined to be the fourth power parameters; Among them, from the first power supply parameter to the fourth power supply parameter, the power supply voltage and power supply frequency become increasingly larger.
[0099] In another embodiment, the power control module 20 is further configured to: In response to the edge features being less than a first threshold and the grayscale distribution features being less than a second threshold, the target image block is determined to be a smooth block; In response to edge features being greater than or equal to a first threshold, grayscale distribution features being greater than a second threshold, and frequency domain features being greater than or equal to a third threshold, the target image block is determined to be a texture block; In response to edge features being greater than or equal to a first threshold, grayscale distribution features being less than or equal to a second threshold, and frequency domain features being greater than or equal to a third threshold, the target image block is determined to be a mixed block; In response to the fact that the target image patch does not belong to a smooth patch, texture patch, or blended patch, the target image patch is determined to be an edge patch.
[0100] In another embodiment, an image processing system further includes an adjustment module for: Determine the proportions of smooth blocks, edge blocks, blended blocks, and texture blocks within a preset number of frames; In response to a smooth block proportion exceeding the fourth threshold, the corresponding adjustment values for the first and second thresholds are increased. In response to a texture block proportion exceeding the fifth threshold, the corresponding adjustment values for the second and third thresholds are reduced. In response to the proportion of mixed blocks exceeding the sixth threshold, the corresponding adjustment values for the first and third thresholds are increased; In response to the smooth block proportion being less than the seventh threshold and the texture block proportion being less than the eighth threshold, the corresponding adjustment values for the first and second thresholds are decreased, and the corresponding adjustment value for the third threshold is increased.
[0101] In another embodiment, the adjustment module is further configured to: In response to changes in configuration parameters in hardware registers, at least one of the first threshold, second threshold, third threshold, first power supply parameter, second power supply parameter, third power supply parameter, and fourth power supply parameter is adjusted; The hardware registers include a threshold register for storing classification thresholds, a voltage mapping register for storing the supply voltage in the power supply parameters, and a frequency mapping register for storing the supply frequency in the power supply parameters.
[0102] According to embodiments of this disclosure, this disclosure also provides an electronic device and a readable storage medium.
[0103] Figure 7 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0104] like Figure 7As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in ROM 802 or a computer program loaded into RAM 803 from storage unit 808. RAM 803 can also store various programs and data required for the operation of the electronic device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. I / O interface 805 is also connected to bus 804. Here, ROM is read-only memory, RAM is random access memory, and I / O interface is input / output interface.
[0105] Multiple components in electronic device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of displays, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows electronic device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0106] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as an image processing method. For example, in some embodiments, an image processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of an image processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform an image processing method by any other suitable means (e.g., by means of firmware).
[0107] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0108] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0109] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0110] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).
[0111] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.
[0112] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.
[0113] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this disclosure can be achieved, and this is not limited herein.
[0114] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.
[0115] The above description is merely a specific embodiment of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.
Claims
1. An image processing method, characterized in that, The method includes: The image to be processed is divided into blocks to obtain multiple target image blocks; Multi-dimensional feature extraction is performed on the target image block to obtain the feature parameters of the target image block; the feature parameters characterize the content complexity of the target image block. Based on the aforementioned feature parameters, the target power parameters required for processing the target image block are determined; the target power parameters corresponding to each target image block may be the same or different. Based on the target power parameters, the corresponding target image blocks are encoded.
2. The method according to claim 1, characterized in that, The step of performing multi-dimensional feature extraction on the target image patch to obtain the feature parameters of the target image patch includes: The edge features of the target image block are determined based on the grayscale change rate between adjacent pixels within the target image block; the edge features characterize the degree of change in edges and / or textures within the target image block. Based on the statistical dispersion of pixel values within the target image block, the grayscale distribution characteristics of the target image block are determined; the grayscale distribution characteristics characterize the uniformity of grayscale distribution in the target image block. The frequency domain features of the target image block are determined based on the proportion of signal components above a preset frequency threshold in the target image block; the frequency domain features characterize the detail richness of the target image block.
3. The method according to claim 2, characterized in that, Determining the edge features of the target image block based on the grayscale change rate between adjacent pixels within the target image block includes: Perform neighborhood convolution operation on the pixels within the target image block to obtain the horizontal gradient component in the horizontal direction and the vertical gradient component in the vertical direction for each pixel. Based on the horizontal and vertical gradient components, the local gradient values of each pixel are determined using the target magnitude estimation logic. The average local gradient values of all pixels within the target image block are determined as the overall gradient value of the target image block; the overall gradient value characterizes the edge features of the target image block. The horizontal gradient component is used to characterize the grayscale change rate of the pixel in the horizontal direction, and the vertical gradient component is used to characterize the grayscale change rate of the pixel in the vertical direction.
4. The method according to claim 3, characterized in that, The step of determining the local gradient value of each pixel based on the horizontal and vertical gradient components and using target magnitude estimation logic includes: The vertical gradient components are shifted left by 3 bits to obtain the first intermediate number; The vertical gradient component is shifted left by 2 bits to obtain the second intermediate number; The difference between the sum of the first and second intermediate numbers and the vertical gradient component is determined as the third intermediate number; The third intermediate number is shifted 4 bits to the right to obtain the fourth intermediate number; The sum of the horizontal gradient component and the fourth intermediate number is determined as the local gradient value of the pixel.
5. The method according to claim 2, characterized in that, Determining the grayscale distribution characteristics of the target image block based on the statistical dispersion of pixel values within the target image block includes: The average gray value of all pixels within the target image block is determined as the baseline gray value; Determine the deviation between the grayscale value of each pixel in the target image block and the reference grayscale value; The squared deviation values corresponding to all pixels within the target image block are summed to obtain the summation result. The accumulated result is shifted 6 bits to the right to obtain the variance of the pixel values within the target image block; the variance characterizes the statistical dispersion of the target image block, and the statistical dispersion is used to reflect the gray-level distribution characteristics of the target image block.
6. The method according to claim 2, characterized in that, The step of determining the frequency domain characteristics of the target image block based on the proportion of signal components in the target image block that are higher than a preset frequency threshold includes: The target image block is transformed and encoded to obtain a frequency domain coefficient matrix; The set of coefficients located within a predetermined row and column range at the edge of the frequency domain coefficient matrix is determined as the target component region; The ratio of the number of non-zero coefficients in the target component region to the total number of coefficients in the frequency domain coefficient matrix is determined as the signal component proportion; the signal component proportion characterizes the frequency domain features of the target image block. The target component region includes coefficients within a preset number of rows near the bottom edge of the matrix, coefficients within a preset number of columns near the right edge of the matrix, a preset number of coefficients in the center row of the matrix near the right edge of the matrix, and a preset number of coefficients in the center column of the matrix near the bottom edge of the matrix.
7. The method according to claim 1, characterized in that, The step of determining the target power parameters required for processing the target image patch based on the feature parameters includes: Based on the feature parameters, the classification result of the target image patch is determined; the classification result includes smooth patch, edge patch, mixed patch and texture patch; In response to the target image block being a smooth block, the target power parameters are determined to be the first power parameters; In response to the target image block being an edge block, the target power parameter is determined to be a second power parameter; In response to the target image block being a mixed block, the target power parameters are determined to be third power parameters; In response to the target image block being a texture block, the target power parameter is determined to be a fourth power parameter; Among them, from the first power supply parameter to the fourth power supply parameter, the power supply voltage and power supply frequency become increasingly larger.
8. The method according to claim 7, characterized in that, The feature parameters include edge features, grayscale distribution features, and frequency domain features; determining the classification result of the target image patch based on the feature parameters includes: In response to the edge feature being less than a first threshold and the grayscale distribution feature being less than a second threshold, the target image block is determined to be a smooth block; In response to the edge feature being greater than or equal to a first threshold, the grayscale distribution feature being greater than a second threshold, and the frequency domain feature being greater than or equal to a third threshold, the target image block is determined to be a texture block; In response to the edge feature being greater than or equal to a first threshold, the grayscale distribution feature being less than or equal to a second threshold, and the frequency domain feature being greater than or equal to a third threshold, the target image block is determined to be a mixed block; In response to the fact that the target image block does not belong to the smooth block, texture block, and blend block, the target image block is determined to be an edge block.
9. The method according to claim 8, characterized in that, The method further includes: Determine the proportions of smooth blocks, edge blocks, blended blocks, and texture blocks within a preset number of frames; In response to the smooth block ratio being greater than the fourth threshold, the corresponding adjustment values for the first and second thresholds are increased; In response to the texture block proportion being greater than the fifth threshold, the corresponding adjustment values for the second and third thresholds are reduced; In response to the fact that the proportion of the mixed block is greater than the sixth threshold, the corresponding adjustment values of the first threshold and the third threshold are increased; In response to the smooth block proportion being less than a seventh threshold and the texture block proportion being less than an eighth threshold, the corresponding adjustment values for the first threshold and the second threshold are decreased, and the corresponding adjustment value for the third threshold is increased.
10. The method according to claim 8, characterized in that, The method further includes: In response to changes in configuration parameters in hardware registers, at least one of the first threshold, second threshold, third threshold, first power parameter, second power parameter, third power parameter, and fourth power parameter is adjusted; The hardware registers include a threshold register for storing classification thresholds, a voltage mapping register for storing power supply voltage in the power supply parameters, and a frequency mapping register for storing power supply frequency in the power supply parameters.
11. An image processing system, characterized in that, The system includes: The feature extraction module is used to divide the image to be processed into blocks to obtain multiple target image blocks; to perform multi-dimensional feature extraction on the target image blocks to obtain feature parameters of the target image blocks; the feature parameters characterize the content complexity of the target image blocks; A power control module is used to determine the target power parameters required for processing the target image block based on the feature parameters; the target power parameters corresponding to each target image block may be the same or different; A parameter adjustment module is used to adjust the power parameters of the image encoding module to the target power parameters; The image encoding module is used to encode the corresponding target image blocks based on the target power parameters.
12. The system according to claim 11, characterized in that, The feature extraction module is also used for: The edge features of the target image block are determined based on the grayscale change rate between adjacent pixels within the target image block; the edge features characterize the degree of change in edges and / or textures within the target image block. Based on the statistical dispersion of pixel values within the target image block, the grayscale distribution characteristics of the target image block are determined; the grayscale distribution characteristics characterize the uniformity of grayscale distribution in the target image block. The frequency domain features of the target image block are determined based on the proportion of signal components above a preset frequency threshold in the target image block; the frequency domain features characterize the detail richness of the target image block.
13. The system according to claim 12, characterized in that, The feature extraction module is also used for: Perform neighborhood convolution operation on the pixels within the target image block to obtain the horizontal gradient component in the horizontal direction and the vertical gradient component in the vertical direction for each pixel. Based on the horizontal and vertical gradient components, the local gradient values of each pixel are determined using the target magnitude estimation logic. The average local gradient values of all pixels within the target image block are determined as the overall gradient value of the target image block; the overall gradient value characterizes the edge features of the target image block. The horizontal gradient component is used to characterize the grayscale change rate of the pixel in the horizontal direction, and the vertical gradient component is used to characterize the grayscale change rate of the pixel in the vertical direction.
14. The system according to claim 13, characterized in that, The feature extraction module is also used for: The vertical gradient components are shifted left by 3 bits to obtain the first intermediate number; The vertical gradient component is shifted left by 2 bits to obtain the second intermediate number; The difference between the sum of the first and second intermediate numbers and the vertical gradient component is determined as the third intermediate number; The third intermediate number is shifted 4 bits to the right to obtain the fourth intermediate number; The sum of the horizontal gradient component and the fourth intermediate number is determined as the local gradient value of the pixel.
15. The system according to claim 12, characterized in that, The feature extraction module is also used for: The average gray value of all pixels within the target image block is determined as the baseline gray value; Determine the deviation between the grayscale value of each pixel in the target image block and the reference grayscale value; The squared deviation values corresponding to all pixels within the target image block are summed to obtain the summation result. The cumulative result is shifted 6 bits to the right to obtain the variance of the pixel values within the target image block. The variance characterizes the statistical dispersion of the target image patch, and the statistical dispersion is used to reflect the gray-level distribution characteristics of the target image patch.
16. The system according to claim 12, characterized in that, The feature extraction module is also used for: The target image block is transformed and encoded to obtain a frequency domain coefficient matrix; The set of coefficients located within a predetermined row and column range at the edge of the frequency domain coefficient matrix is determined as the target component region; The ratio of the number of non-zero coefficients in the target component region to the total number of coefficients in the frequency domain coefficient matrix is determined as the signal component proportion; the signal component proportion characterizes the frequency domain features of the target image block. The target component region includes coefficients within a preset number of rows near the bottom edge of the matrix, coefficients within a preset number of columns near the right edge of the matrix, a preset number of coefficients in the center row of the matrix near the right edge of the matrix, and a preset number of coefficients in the center column of the matrix near the bottom edge of the matrix.
17. The system according to claim 11, characterized in that, The power control module is also used for: Based on the feature parameters, the classification result of the target image patch is determined; the classification result includes smooth patch, edge patch, mixed patch and texture patch; In response to the target image block being a smooth block, the target power parameters are determined to be the first power parameters; In response to the target image block being an edge block, the target power parameter is determined to be a second power parameter; In response to the target image block being a mixed block, the target power parameters are determined to be third power parameters; In response to the target image block being a texture block, the target power parameter is determined to be a fourth power parameter; Among them, from the first power supply parameter to the fourth power supply parameter, the power supply voltage and power supply frequency become increasingly larger.
18. The system according to claim 17, characterized in that, The power control module is also used for: In response to the edge feature being less than a first threshold and the grayscale distribution feature being less than a second threshold, the target image block is determined to be a smooth block; In response to the edge feature being greater than or equal to a first threshold, the grayscale distribution feature being greater than a second threshold, and the frequency domain feature being greater than or equal to a third threshold, the target image block is determined to be a texture block; In response to the edge feature being greater than or equal to a first threshold, the grayscale distribution feature being less than or equal to a second threshold, and the frequency domain feature being greater than or equal to a third threshold, the target image block is determined to be a mixed block; In response to the fact that the target image block does not belong to the smooth block, texture block, and blend block, the target image block is determined to be an edge block.
19. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method of any one of claims 1-10.
20. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-10.