A data processing method, electronic equipment, storage medium and chip

Abstract

The application provides a data processing method, an electronic device, a storage medium and a chip. The method can process all data in P*P pixel values in parallel in the process of color space conversion processing in encoding performed by the electronic device. Compared with a related scheme of processing all data in P*P pixel values in series, the processing speed and the processing efficiency of the electronic device in processing data can be improved, thereby reducing the waiting time of a user when the user uses the electronic device to encode data and improving user experience.

Description

Technical Field

[0001] This application relates to the field of data processing, and more particularly to a data processing method, electronic device, storage medium, and chip. Background Technology

[0002] Image encoding and image decoding are indispensable technologies in image processing. They make image storage and transmission more efficient, and also provide convenience for image editing and processing.

[0003] The proposed solution involves serially processing image data for encoding and decoding. However, this serial processing can slow down the processing speed of electronic devices, resulting in longer waiting times for users when encoding and decoding images, thus impacting the user experience. Summary of the Invention

[0004] This application provides a data processing method, an electronic device, a storage medium, and a chip, which can improve the processing speed and efficiency of electronic devices for processing data, thereby reducing the waiting time for users when encoding data using electronic devices and improving the user experience.

[0005] Firstly, a data processing method is provided, which is applied to an electronic device, and the method includes:

[0006] Multiple initial datasets are obtained, each initial dataset comprising P×P initial data points. Each initial data point includes at least one of a first pixel value, a second pixel value, and DCT coefficients. The first pixel value includes at least one first color component of a first color space, and the second pixel value includes at least one second color component of a second color space. The DCT coefficients represent the frequency of the second color component, where P is an integer greater than 1. The multiple initial datasets are then encoded to obtain multiple target datasets, each target dataset comprising P×P target data points. Each target data point includes at least one of a second pixel value, DCT coefficients, and quantized DCT coefficients. The encoding process for each initial dataset includes: encoding at least a portion of the P×P initial data points in the initial dataset in parallel to obtain P×P target data points in the target dataset. The encoding process includes at least one of a first color space conversion, a discrete cosine transform, and quantization.

[0007] It should be understood that the first color space is the RGB color space mentioned in the following embodiments, and the second color space is the YCbCr color space mentioned in the following embodiments. The RGB color space includes at least the R component, G component, and B component, which are first color components, and the YCbCr color space includes at least the Y component, Cb component, and Cr component, which are second color components. The first pixel value is a pixel value composed of the R component, G component, and B component, and the second pixel value is a pixel value composed of the Y component, Cb component, and Cr component. The first color space conversion process is the process of converting pixel values ​​from the RGB color space to the YCbCr color space.

[0008] When the initial data is a first pixel value, the target data is a second pixel value, and the encoding process is a first color space conversion process, the P×P initial data can include P×P first pixel values. Since the first pixel value is a pixel value composed of R components, G components, and B components, the P×P first pixel values ​​can include P×P R components, P×P G components, and P×P B components. The P×P target data can include P×P second pixel values. Since the second pixel value is a pixel value composed of Y components, Cb components, and Cr components, the P×P second pixel values ​​include P×P Y components, P×P Cb components, and P×P Cr components. Therefore, multiple initial datasets can refer to the (M / P)×(N / P) data subsets A1, B1, and C1 mentioned in the following embodiments, where each data subset A1 includes P×P R components, each data subset B1 includes P×P G components, and each data subset C1 includes P×P B components. Multiple target datasets can refer to the (M / P)×(N / P) data subsets A2, B2, and C2 mentioned in the following embodiments, where each data subset A2 includes P×P Y components, each data subset B2 includes P×P Cb components, and each data subset C2 includes P×P Cb components.

[0009] In the implementation, obtaining multiple initial datasets can refer to obtaining (M / P)×(N / P) data subsets A1, (M / P)×(N / P) data subsets B1 and (M / P)×(N / P) data subsets C1 mentioned in the following embodiments.

[0010] Encoding multiple initial datasets to obtain multiple target datasets can refer to: performing color space conversion on (M / P)×(N / P) data subsets A1, B1, and C1 to obtain (M / P)×(N / P) data subsets A2, B2, and C2. For example, this process can be referred to in the embodiments below. Figure 7 The process of color space conversion is shown.

[0011] Encoding at least a portion of P×P initial data in the initial dataset in parallel to obtain P×P target data in the target dataset can, for example, mean that the process of performing a first color space conversion on an initial dataset includes: substituting each R component of P×P R components in a data subset A1, the G component in the corresponding data subset B1, and the B component in the corresponding data subset C1 into Formula 1 mentioned in the following embodiments for parallel calculation to obtain P×P Y components in data subset A2, P×P Cb components in data subset B2, and P×P Cr components in data subset C2. In other words, for P×P first pixel values, all data in their P×P R components, P×P G components, and P×P B components can be processed in parallel for color space conversion. The process of performing a first color space conversion on other initial data can refer to the above process. For example, this process can also refer to the following embodiments. Figure 24 The process of parallel computing is shown.

[0012] When the initial data is the second pixel value, the target data is DCT coefficients, and the encoding process is discrete cosine transform, the initial data can refer to a second color component (such as the Y component, Cb component, or Cr component) of the second pixel value. P×P initial data can refer to P×P second color components, and P×P target data can refer to P×P DCT coefficients. Therefore, multiple initial datasets can refer to the Q data subsets A3, J data subsets B3, and L data subsets C3 mentioned in the following embodiments. Each data subset A3 includes P×P Y components, each data subset B3 includes P×P Cb components, and each data subset C3 includes P×P Cr components. Multiple target datasets can refer to the G data subsets D mentioned in the following embodiments, each data subset D including P×P DCT coefficients.

[0013] In the implementation, obtaining multiple initial datasets can refer to obtaining the Q data subsets A3, J data subsets B3, and L data subsets C3 mentioned in the following embodiments.

[0014] Encoding multiple initial datasets to obtain multiple target datasets can refer to performing discrete cosine transform on Q data subsets A3, J data subsets B3, and L data subsets C3 to obtain G data subsets D.

[0015] Encoding at least a portion of P×P initial data in the initial dataset in parallel to obtain P×P target data in the target dataset can mean, for example, that the P×P second color components are organized as a two-dimensional array, meaning that the P×P second color components include P rows and P columns of second color components. For any one of the Q data subsets A3, J data subsets B3, and L data subsets C3 containing P×P second color components, the P rows of second color components can be computed in parallel, or the P columns of second color components can be computed in parallel, to obtain P×P DCT coefficients. For example, if the P×P second color components include 8 rows and 8 columns, totaling 64 Y components, the 8 rows of Y components can be subjected to Discrete Cosine Transform (DCT) in parallel, and the 8 Y components in each row can be subjected to DCT in parallel or serially, thus obtaining 8×8 DCT coefficients. In other words, for the P×P second color components, some data can be subjected to DCT in parallel, while other data can be subjected to DCT in parallel or serially.

[0016] When the initial data consists of DCT coefficients, the target data consists of quantized DCT coefficients, and the encoding process is quantization, the P×P initial data can include P×P DCT coefficients, and the P×P target data can include P×P quantized DCT coefficients. Multiple initial datasets can refer to the G data subsets D mentioned in the following embodiments, where each subset D includes P×P DCT coefficients. Multiple target datasets can refer to the G data subsets E mentioned in the following embodiments, where each subset E includes P×P quantized DCT coefficients.

[0017] In implementation, obtaining multiple initial datasets can refer to obtaining the G data subsets D mentioned in the following examples.

[0018] Encoding multiple initial datasets to obtain multiple target datasets can refer to quantizing G data subsets D to obtain G data subsets E.

[0019] Encoding at least a portion of P×P initial data points in the initial dataset in parallel to obtain P×P target data points in the target dataset can mean that, for each data subset D, the P×P DCT coefficients included in subset D are quantized in parallel to obtain P×P quantized DCT coefficients. In other words, for the P×P DCT coefficients, all P×P data points are quantized in parallel.

[0020] In the process of performing first color space conversion, discrete cosine transform, or quantization on data, the relevant schemes process the data sequentially. For example, in the process of performing first color space conversion on P×P first pixel values ​​to obtain P×P second pixel values, the scheme first performs the first color space conversion on the first of the P×P first pixel values ​​to obtain one second pixel value, and then performs the first color space conversion on the second of the P×P first pixel values ​​to obtain another second pixel value, and so on sequentially to obtain P×P second pixel values. As another example, in the process of performing discrete cosine transform on P×P second color components to obtain P×P DCT coefficients, the scheme first performs discrete cosine transform on the first of the P×P second color components, and then performs discrete cosine transform on the second of the P×P second color components, and so on sequentially to obtain P×P DCT coefficients. For example, in the process of quantizing P×P DCT coefficients to obtain P×P quantized DCT coefficients, the relevant scheme first performs quantization on the first DCT coefficient among the P×P DCT coefficients to obtain a quantized DCT coefficient, then performs quantization on the second DCT coefficient to obtain a quantized DCT coefficient, and so on in a serial manner to obtain P×P quantized DCT coefficients.

[0021] Because the relevant solutions process data serially, the data processing speed is slow and the processing efficiency is low. Furthermore, the slow processing speed may lead to high CPU utilization (CPU utilization is the ratio of CPU processing time to total processing time; since the processing speed of electronic devices is slow (mainly due to the CPU processing data within the electronic device), the CPU processing time is long, resulting in high CPU utilization). Since CPU utilization is directly proportional to the power consumption of electronic devices (higher CPU utilization means higher power consumption), this also leads to higher power consumption in electronic devices.

[0022] In this embodiment, since the electronic device can encode at least a portion of P×P initial data in the initial dataset in parallel to obtain P×P target data in the target dataset, and the encoding process includes at least one of first color space conversion processing, discrete cosine transform processing, and quantization processing, the electronic device can process all or part of the P×P initial data in parallel during the execution of the first color space conversion processing, and / or discrete cosine transform processing, and / or quantization processing. Compared with related schemes that process all the P×P initial data serially, the processing speed and efficiency of the electronic device can be improved, thereby reducing the waiting time for users when using the electronic device to encode data and improving the user experience.

[0023] Furthermore, by increasing the data processing speed of electronic devices, CPU utilization is reduced, thereby reducing the power consumption of electronic devices.

[0024] In conjunction with the first aspect, in one possible implementation of the first aspect, the electronic device is configured with a processor, the processor having at least one two-dimensional register, the two-dimensional register being composed of multiple one-dimensional registers; and, at least a portion of P×P initial data in the initial dataset is encoded in parallel to obtain P×P target data in the target dataset, including:

[0025] The processor reads P×P initial data from at least one two-dimensional register in parallel; the processor encodes at least a portion of the read P×P initial data in parallel to obtain P×P target data.

[0026] It should be understood that a one-dimensional register refers to a storage unit that can hold a single data bit or a group of data bits, while a two-dimensional register refers to a register that can store a two-dimensional array. Two-dimensional registers provide electronic devices with multi-dimensional data storage and access capabilities, and can store more data than one-dimensional registers. Two-dimensional registers can provide hardware support for parallel computation of data. For example, because two-dimensional registers can store more data, the processor can access and process this data simultaneously, thereby achieving data-level parallelism. Furthermore, two-dimensional registers can significantly improve data processing throughput and support the simultaneous parallel processing of multiple data sets.

[0027] For more information on this implementation method, please refer to [link / reference]. Figure 14 The processes shown in S434 and S435 can also be referenced. Figure 25 The processes shown in S455 and S456 can also be referred to Figure 33 The processes S464 and S465 are shown.

[0028] In this embodiment of the application, the electronic device is equipped with at least one two-dimensional register. Since the two-dimensional register can provide hardware support for parallel computing of data, the electronic device can read P×P initial data in parallel from at least one two-dimensional register through the processor; and encode at least a portion of the read P×P initial data in parallel through the processor to obtain P×P target data, thereby improving the processing speed and efficiency of the electronic device in processing data.

[0029] In conjunction with the first aspect, in one possible implementation of the first aspect, the electronic device is further configured with internal memory; and, before reading P×P initial data in parallel from at least one two-dimensional register by the processor, the method further includes:

[0030] The processor loads P×P initial data from internal memory into at least one two-dimensional register in parallel.

[0031] For details on the implementation process of this method, please refer to [link / reference]. Figure 14 The process shown in S433 can also be referenced. Figure 25 The process shown in S454 can also be referred to Figure 33 The process of S463 is shown.

[0032] It should be understood that the process of an electronic device loading data from memory into registers is as follows: the CPU accesses memory and writes the data from memory into the register. A related approach is to load data from memory into registers serially. For example, the CPU first accesses a piece of data in memory, writes that data into the register, then accesses another piece of data in memory and writes that data into the register again. Therefore, the CPU needs to access memory more times.

[0033] Frequent memory access by electronic devices can lead to several problems: 1. Decreased CPU performance. With frequent memory accesses, the CPU needs to constantly read data from memory, increasing its workload and causing performance degradation. 2. Increased power consumption. Memory access is one of the most power-intensive operations in electronic devices; frequent accesses increase power consumption. 3. Increased memory access latency. Memory access latency refers to the time delay between the CPU issuing a memory access request and receiving a response. Excessive memory accesses require the memory controller to process more requests, potentially causing queuing and increasing latency. These problems are particularly pronounced with large datasets such as high-resolution images.

[0034] In this embodiment, the electronic device can load P×P initial data from internal memory into at least one two-dimensional register in parallel. That is, the electronic device can write P×P initial data stored in memory to at least one two-dimensional register with fewer memory accesses. Compared to related schemes that serially load data from memory into registers, the number of memory accesses can be reduced. Because the number of memory accesses is reduced, CPU performance can be improved, power consumption of the electronic device can be reduced, and memory access latency can be reduced. Furthermore, the reduced number of memory accesses can alleviate memory bandwidth pressure to some extent.

[0035] In conjunction with the first aspect, in one possible implementation of the first aspect, the initial data is a first pixel value, the target data is a second pixel value, and the encoding process is a first color space conversion process; and, at least a portion of P×P initial data in the initial dataset is encoded in parallel to obtain P×P target data in the target dataset, including:

[0036] The first color space conversion is performed in parallel on the P×P first pixel values ​​in the initial dataset to obtain the P×P second pixel values ​​in the target dataset.

[0037] For example, please refer to Figure 24 , Figure 24 The P×P first pixel values ​​include 8×8 R components, 8×8 G components, and 8×8 B components. Figure 24 Each rectangle shown represents a first pixel value.

[0038] Please refer to Figure 24 In formula (a), the CPU can substitute the data in each rectangle into Y in formula one. i =0.299×R i +0.578×G i +0.114×B i We obtain a data subset A2, which includes 8×8=64 Y components from A2-11(Y11) to A2-11(Y88). Figure 24 The operations on the multiple rectangles shown in (a) are parallel operations.

[0039] CPU is executing Figure 24 While the operation is shown in (a), please refer to... Figure 24 In formula (b), the data in each rectangle can also be substituted into Cb in formula one in parallel. i = -0.1687 × R i -0.3313×G i +0.5×B i+128, resulting in a data subset B2, which includes 8×8=64 Cb components from B2-11(Cb11) to B2-11(Cb88). Figure 24 The operations within the multiple rectangles shown in (b) are parallel operations.

[0040] CPU is executing Figure 24 While the operation is shown in (a), please refer to... Figure 24 In formula (c), the data in each rectangle can also be substituted into Cr in formula one in parallel. i =0.5×R i -0.4187×G i -0.0813×B i +128, resulting in a data subset C2, which includes 8×8=64 Cr components from C2-11(Cr11) to C2-11(Cr88). Figure 24 The operations within the multiple rectangles shown in (c) are parallel operations.

[0041] The aforementioned 64 Y components, 64 Cb components, and 64 Cr components constitute 64 second pixel values.

[0042] The relevant solutions process data serially during the first color space conversion; for example, first... Figure 24 In the example (a), the data within the first rectangle is calculated, and then the data within the second rectangle is calculated, thus achieving serial calculation.

[0043] In this embodiment, the first color space conversion process can be performed in parallel on the P×P first pixel values ​​in the initial dataset to obtain the P×P second pixel values ​​in the target dataset. Since the data is processed in parallel, the processing speed and efficiency of the color space conversion process can be improved.

[0044] In conjunction with the first aspect, in one possible implementation of the first aspect, the initial data are second color components, the second color components being at least one of the second color components in the second pixel values; the target data are DCT coefficients; the encoding process is discrete cosine transform processing; P×P initial data are P×P second color components; the data organization of the P×P second color components is a two-dimensional array, the two-dimensional array including P rows and P columns of second color components; and, at least a portion of the P×P initial data in the initial dataset is encoded in parallel to obtain P×P target data in the target dataset, including:

[0045] Perform parallel Discrete Cosine Transform (DCT) on the P rows of the second color components in the initial dataset to obtain P×P DCT coefficients in the target dataset; or,

[0046] The second color component of column P in the initial dataset is subjected to discrete cosine transform in parallel to obtain P×P DCT coefficients in the target dataset.

[0047] In implementation, for example, please refer to Figure 27 Assume the initial dataset contains 8×8 Y components, and the data organization of the 8×8 Y components is a two-dimensional array. For example, this two-dimensional data could refer to... Figure 27 The matrix A shown contains 8 rows and 8 columns, totaling 64 Y components. The electronic device can perform discrete cosine transform (DCT) on 8 rows of Y components in parallel. The 8 Y components in each row can be processed in parallel or serially to obtain 8×8 DCT coefficients.

[0048] For example, please refer to Figure 26 The electronic device can perform discrete cosine transform (DCT) on 8 columns of Y components in 8 rows and 8 columns in parallel. The 8 Y components in each column can be processed in parallel or serially to obtain 8×8 DCT coefficients.

[0049] In this embodiment, the second color components in rows P of the initial dataset can be processed in parallel using Discrete Cosine Transform (DCT) to obtain P×P DCT coefficients in the target dataset; or the second color components in columns P of the initial dataset can be processed in parallel using DCT to obtain P×P DCT coefficients in the target dataset. Since the data is processed in parallel, the processing speed and efficiency of DCT can be improved.

[0050] In conjunction with the first aspect, in one possible implementation of the first aspect, the second color components in the P rows of the initial dataset are subjected to discrete cosine transform in parallel to obtain P×P DCT coefficients in the target dataset, including:

[0051] Based on the P rows of second color components and P columns of DCT transform coefficients from the initial dataset, a vector outer product operation is performed in parallel, and the results of the vector outer product operation are accumulated to obtain P×P DCT coefficients. These P×P DCT transform coefficients are preset coefficients used for discrete cosine transform processing. The data organization of the P×P DCT transform coefficients is a two-dimensional array, including P rows and P columns of DCT transform coefficients. A vector outer product operation is performed between the second color component in each row and the corresponding column of DCT transform coefficients from the P columns.

[0052] Perform parallel Discrete Cosine Transform (DCT) on the second color components of columns P in the initial dataset to obtain P×P DCT coefficients in the target dataset, including:

[0053] The vector outer product operation is performed in parallel on the second color component of column P and the DCT transform coefficients of row P of the P×P DCT transform coefficients in the initial dataset, and the results of the vector outer product operation are accumulated to obtain P×P DCT coefficients. In this case, the vector outer product operation is performed on the second color component of each column and the DCT transform coefficients of the row P corresponding to the second color component of each row.

[0054] In implementation, for example, please refer to Figure 27 Assume the initial dataset contains 8×8 Y components, and the data organization of the 8×8 Y components is a two-dimensional array. For example, this two-dimensional data could refer to... Figure 27 The matrix A shown contains 64 Y components in 8 rows and 8 columns. Assume that the P×P DCT transform coefficients include 8×8 DCT transform coefficients, and the data organization of the 8×8 DCT transform coefficients is a two-dimensional array. For example, this two-dimensional data could refer to... Figure 27 The matrix B shown contains 64 DCT transform coefficients in 8 rows and 8 columns. The electronic device can perform a vector outer product operation in parallel with the 8 rows of Y components and the 8 columns of DCT transform coefficients in the 8 rows and 8 columns of the DCT transform coefficients, and then accumulate the results of the vector outer product operation to obtain the 8 rows and 8 columns of DCT coefficients.

[0055] For example, please refer to Figure 26 The electronic device can perform a vector outer product operation in parallel with the 8 columns of Y components in the 8x8 Y component and the 8 rows of DCT transform coefficients in the 8x8 DCT transform coefficient, and then accumulate the results of the vector outer product operation to obtain the 8x8 DCT coefficient.

[0056] The proposed solution involves performing discrete cosine transform on P×P second color components and P×P DCT transform coefficients using matrix multiplication. However, this method involves a large number of calculations and has high computational complexity.

[0057] This application embodiment can convert matrix multiplication into vector outer product accumulation. For example, it can perform vector outer product operations in parallel based on the P rows of second color components and the P columns of DCT transform coefficients in the P×P DCT transform coefficients in the initial dataset, and accumulate the results of the vector outer product operations to obtain P×P DCT coefficients. Alternatively, it can perform vector outer product operations in parallel based on the P columns of second color components and the P rows of DCT transform coefficients in the P×P DCT transform coefficients in the initial dataset, and accumulate the results of the vector outer product operations to obtain P×P DCT coefficients. By accumulating vector outer products, the number of calculations can be reduced and the computational complexity can be reduced compared to performing discrete cosine transform through matrix multiplication.

[0058] In conjunction with the first aspect, in one possible implementation of the first aspect, the electronic device includes a processor and internal memory, wherein the P×P second color components are stored in the internal memory in a row-wise manner, and the P×P DCT transform coefficients are also stored in the internal memory in a row-wise manner; and before performing a vector outer product operation in parallel based on the P rows of second color components and the P columns of DCT transform coefficients in the P×P DCT transform coefficients in the initial dataset, and before accumulating the results of the vector outer product operation to obtain the P×P DCT coefficients, the method further includes:

[0059] The processor changes the storage method of the P×P DCT transform coefficients in internal memory from row-based to column-based; and...

[0060] Before performing a parallel vector outer product operation on the second color components in column P of the initial dataset and the DCT transform coefficients in rows P of the P×P DCT transform coefficients, and before summing the results of the vector outer product operation to obtain the P×P DCT coefficients, the method also includes:

[0061] The processor changes the storage method of the P×P second color components in the internal memory from row storage to column storage.

[0062] For more information on this implementation method, please refer to [link / reference]. Figure 29 The implementation method shown will not be elaborated here.

[0063] In related schemes, data is usually stored in memory in a row-based manner, and column data is not stored contiguously. Such schemes require multiple accesses to memory to obtain column data from P×P DCT transform coefficients or column data from P×P second color components.

[0064] In this embodiment, the processor can convert the storage method of P×P DCT transform coefficients in internal memory from row-based storage to column-based storage, or the processor can convert the storage method of P×P second color components in internal memory from row-based storage to column-based storage. Thus, when column data of the P×P DCT transform coefficients or P×P second color components is needed, the column data is stored contiguously in memory. The electronic device can obtain P columns of DCT transform coefficients or P columns of second color components with fewer memory accesses. By reducing the number of memory accesses, CPU performance can be improved, power consumption of the electronic device can be reduced, and memory access latency can be lowered.

[0065] In conjunction with the first aspect, in one possible implementation of the first aspect, the initial data are DCT coefficients, the target data are quantized DCT coefficients, and the encoding process is quantization processing; and, at least a portion of P×P initial data in the initial dataset are encoded in parallel to obtain P×P target data in the target dataset, including:

[0066] The P×P DCT coefficients in the initial dataset are quantized in parallel to obtain the P×P quantized DCT coefficients in the target dataset.

[0067] For example, please refer to Figure 37 , Figure 37 The data within the first row of rectangles shown includes 8×8 DCT coefficients from an initial dataset, as well as 8×8 quantization coefficients. The electronic device can quantize the data within each rectangle in the first row (e.g., by division), and all rectangles in the first row are quantized simultaneously and in parallel, resulting in 8×8 quantized DCT coefficients.

[0068] In this embodiment, P×P DCT coefficients in the initial dataset can be quantized in parallel to obtain P×P quantized DCT coefficients in the target dataset. Since the data is processed in parallel, the processing speed and efficiency of the quantization process can be improved.

[0069] In conjunction with the first aspect, in one possible implementation of the first aspect, multiple initial datasets are encoded to obtain multiple target datasets, including:

[0070] Based on multiple encoding processes, multiple initial datasets are encoded into multiple target datasets. Each encoding process is used to encode a portion of the initial datasets in parallel into a portion of the target datasets.

[0071] It should be understood that encoding processing includes at least one of the following: first color space conversion processing, discrete cosine transform processing, and quantization processing. Therefore, multiple encoding processing refers to, for example, multiple first color space conversion processing, or, for example, multiple encoding processing refers to discrete cosine transform processing, etc.

[0072] It should also be understood that due to the large volume of image data (e.g., multiple initial datasets contain a large amount of image data), electronic devices may not be able to encode all the image data at once. Therefore, electronic devices can encode multiple initial datasets multiple times, with each encoding process involving a portion of the initial dataset processed in parallel. For example, if multiple initial datasets include eight initial datasets, the electronic device can encode these eight initial datasets in four separate processes, processing two initial datasets each time, and these two initial datasets are processed in parallel.

[0073] In this embodiment of the application, multiple initial datasets can be encoded into multiple target datasets based on multiple encoding processes. Each encoding process is used to encode a portion of the initial datasets into a portion of the target datasets in parallel. Since each encoding process is a parallel processing of a portion of the target datasets, the processing speed of electronic devices for data processing can be improved.

[0074] In conjunction with the first aspect, in one possible implementation of the first aspect, multiple initial datasets are encoded into multiple target datasets based on multiple encoding processes, including:

[0075] Determine the number of initial datasets that can be processed in each encoding process; based on the number of initial datasets that can be processed in each encoding process and the multiple initial datasets, determine the total number of encoding processes; based on the total number of encoding processes, perform cyclical encoding on the multiple initial datasets to obtain multiple target datasets; wherein, each cyclical processing process includes: performing parallel encoding on a portion of the initial datasets to obtain a portion of the target datasets.

[0076] For more information on this implementation method, please refer to [link / reference]. Figure 14 ,or Figure 25 ,or Figure 33 The implementation process shown will not be repeated here.

[0077] In conjunction with the first aspect, in one possible implementation of the first aspect, the electronic device includes multiple processors; and multiple initial datasets are encoded to obtain multiple target datasets, including:

[0078] Multiple initial datasets are divided into multiple first data groups, each first data group including at least one initial dataset. The number of first data groups matches the number of processors, and the amount of data in each first data group is the same.

[0079] Multiple processors encode multiple first data groups in parallel to obtain multiple target datasets. Each processor encodes at least one first data group, and each processor processes the same number of first data groups.

[0080] For example, this implementation can refer to the following embodiment, which states that "K1 data groups H1 can be divided into multiple data groups H11, the number of data groups H11 matches the number of CPUs, each data group H11 includes at least P×P R components, P×P G components corresponding to the P×P R components, and P×P B components corresponding to the P×P R components, and the data volume of each data group H11 is the same. Each CPU in the multiple CPUs can process at least one data group H11 in parallel to obtain K1 data subsets A2, K1 data subsets B2, and K1 data subsets C2. Multiple CPUs process multiple data groups H11 in parallel, and each CPU processes the same amount of data." This will not be elaborated further here.

[0081] It should be understood that when there are many initial datasets, the large amount of data may cause an imbalance in the load on the multiple processors in a multi-core processor electronic device.

[0082] In this embodiment, multiple initial datasets can be divided to ensure that each processor processes the same number of the first data groups. This balances the workload of each processor, ensuring optimal utilization and preventing some processors from being overloaded while others remain idle, thereby improving the overall computing efficiency of the electronic device. Load balancing also reduces the risk of individual processors failing due to overload, thus improving the reliability and stability of the entire electronic device. Furthermore, load balancing reduces the use of high-power processors among multiple processors, thereby lowering the overall energy consumption of the electronic device.

[0083] In conjunction with the first aspect, in one possible implementation of the first aspect, the logical size of the two-dimensional register is variable.

[0084] It should be understood that the logical size of a two-dimensional register can be varied based on the amount of data. For example, when the amount of data is large, the logical size of the two-dimensional register can be changed by increasing the width and / or depth of the original register. When the amount of data is small, the logical size can be changed by decreasing the width and / or depth of the original register. This adapts to the amount of data, allowing electronic devices to flexibly change the logical size of the two-dimensional register based on the data volume, enabling them to process data with varying amounts of data.

[0085] Secondly, a data processing method is provided for use in electronic devices, the method comprising:

[0086] Multiple target datasets are acquired, each target dataset comprising P×P target data points. Each target data point includes at least one of a second pixel value, DCT coefficients, and quantized DCT coefficients. The second pixel value includes at least one second color component in a second color space, and the DCT coefficients characterize the frequency of the second color component. P is an integer greater than 1. The multiple target datasets are then decoded to obtain multiple initial datasets, each initial dataset comprising P×P initial data points. Each initial data point includes at least one of a first pixel value, a second pixel value, and DCT coefficients. The first pixel value includes at least one first color component in a first color space. The decoding process for each target dataset includes: performing parallel decoding on at least a portion of the P×P target data points in the target dataset to obtain P×P initial data points in the initial dataset. The decoding process includes at least one of second color space conversion, inverse discrete cosine transform, and inverse quantization.

[0087] It should be understood that the first color space is the RGB color space mentioned in the embodiments below, and the second color space is the YCbCr color space mentioned in the embodiments below. The RGB color space includes at least the R component, G component, and B component, etc., as first color components, and the YCbCr color space includes at least the Y component, Cb component, and Cr component, etc., as second color components. The first pixel value is a pixel value composed of the R component, G component, and B component, etc., and the second pixel value is a pixel value composed of the Y component, Cb component, and Cr component, etc. The second color space conversion process is the process of converting pixel values ​​from the YCbCr color space to the RGB color space. The inverse discrete cosine transform process refers to the inverse process of the discrete cosine transform process, and the inverse quantization process refers to the inverse process of the quantization process.

[0088] When the target data consists of quantized DCT coefficients, the initial data consists of DCT coefficients, and the decoding process is inverse quantization, the P×P target data can include P×P quantized DCT coefficients, and the P×P initial data can include P×P DCT coefficients. Multiple target datasets can refer to the G data subsets E mentioned in the following embodiments, where each data subset E includes P×P quantized DCT coefficients. Multiple initial datasets can refer to the G data subsets D mentioned in the following embodiments, where each data subset D includes P×P DCT coefficients.

[0089] In implementation, obtaining multiple target datasets can refer to obtaining the G data subsets E mentioned in the following examples.

[0090] Decoding multiple target datasets to obtain multiple initial datasets can refer to performing inverse quantization on G data subsets E to obtain G data subsets D.

[0091] Decoding at least a portion of P×P target data points in the target dataset in parallel to obtain P×P initial data points in the initial dataset can mean that, for each data subset E, the P×P quantized DCT coefficients included in subset E are inversely quantized in parallel to obtain P×P DCT coefficients. In other words, for the P×P quantized DCT coefficients, all P×P data points are inversely quantized in parallel.

[0092] When the target data consists of DCT coefficients, the initial data consists of a second color component (such as a Y component, Cb component, or Cr component) in the second pixel value, and the decoding process is inverse discrete cosine transform, the P×P target data can include P×P DCT coefficients, and the P×P initial data can include P×P second color components. Multiple target datasets can refer to the G data subsets D mentioned in the following embodiments, where each data subset D includes P×P DCT coefficients. Multiple initial datasets can refer to the Q data subsets A3, J data subsets B3, and L data subsets C3 mentioned in the following embodiments, where each data subset A3 includes P×P Y components, each data subset B3 includes P×P Cb components, and each data subset C3 includes P×P Cr components.

[0093] In implementation, obtaining multiple target datasets can refer to obtaining the G data subsets D mentioned in the following examples.

[0094] Decoding multiple target datasets to obtain multiple initial datasets can mean decoding G data subsets D to obtain Q data subsets A3, J data subsets B3, and L data subsets C3.

[0095] Decoding at least a portion of P×P target data in the target dataset in parallel to obtain P×P initial data in the initial dataset can refer to, for example, the P×P DCT coefficients being organized as a two-dimensional array, meaning that the P×P DCT coefficients include P rows and P columns of DCT coefficients. For each of the G data subsets D, the P rows of DCT coefficients in the P×P DCT coefficients in subset D can be computed in parallel, or the P columns of DCT coefficients can be computed in parallel, to obtain P×P second color components. For example, if the P×P DCT coefficients include 8 rows and 8 columns, totaling 64 DCT coefficients, the 8 rows of DCT coefficients can be subjected to inverse discrete cosine transform in parallel. The 8 DCT coefficients in each row can be subjected to inverse discrete cosine transform in parallel or serially, thus obtaining 8×8 second color components. These 8×8 second color components can refer to one of the Q data subsets A3, J data subsets B3, and L data subsets C3. In other words, for P×P DCT coefficients, some of the data can be subjected to inverse discrete cosine transform in parallel, while the other part can be subjected to inverse discrete cosine transform in parallel or serially.

[0096] When the target data is the second pixel value, the initial data is the first pixel value, and the decoding process is the second color conversion process, the P×P initial data can include P×P first pixel values. Since the first pixel value is a pixel value composed of R components, G components, and B components, the P×P first pixel values ​​can include P×P R components, P×P G components, and P×P B components. The P×P target data can include P×P second pixel values. Since the second pixel value is a pixel value composed of Y components, Cb components, and Cr components, the P×P second pixel values ​​include P×P Y components, P×P Cb components, and P×P Cr components. Therefore, multiple target datasets can refer to the (M / P)×(N / P) data subsets A2, B2, and C2 mentioned in the following embodiments, where each data subset A2 includes P×P Y components, each data subset B2 includes P×P Cb components, and each data subset C2 includes P×P Cb components. Multiple initial datasets can refer to the (M / P)×(N / P) data subsets A1, B1, and C1 mentioned in the following embodiments, where each data subset A1 includes P×P R components, each data subset B1 includes P×P G components, and each data subset C1 includes P×P B components.

[0097] In implementation, obtaining multiple target datasets can refer to obtaining (M / P)×(N / P) data subsets A2, (M / P)×(N / P) data subsets B2, and (M / P)×(N / P) data subsets C2 mentioned in the following embodiments.

[0098] Decoding multiple target datasets to obtain multiple initial datasets can refer to performing a second color space conversion on (M / P)×(N / P) data subsets A2, (M / P)×(N / P) data subsets B2 and (M / P)×(N / P) data subsets C2 to obtain (M / P)×(N / P) data subsets A1, (M / P)×(N / P) data subsets B1 and (M / P)×(N / P) data subsets C1.

[0099] Decoding at least a portion of P×P target data in the target dataset in parallel to obtain P×P initial data in the initial dataset can refer to, for example, the process of performing a second color space conversion on a target dataset includes: substituting each Y component of the P×P Y components in a data subset A2, the Cb component in the corresponding data subset B2, and the Cr component in the corresponding data subset C2 into the following formula eleven for parallel calculation to obtain P×P R components in data subset A1, P×P G components in data subset B1, and P×P B components in data subset C1. In other words, for P×P second pixel values, all data in their P×P Y components, P×P Cb components, and P×P Cr components can be processed in parallel for color space conversion. The process of performing a second color space conversion on other target data can refer to the above process.

[0100] The relevant schemes process the data serially during the second color space conversion, inverse discrete cosine transform, or inverse quantization. For example, in the process of inverse quantization of P×P quantized DCT coefficients to obtain P×P DCT coefficients, the relevant schemes first perform inverse quantization on the first quantized DCT coefficient to obtain one DCT coefficient, then perform inverse quantization on the second quantized DCT coefficient to obtain another DCT coefficient, and so on serially to obtain P×P quantized DCT coefficients.

[0101] For example, in the process of performing inverse discrete cosine transform on P×P DCT coefficients to obtain P×P second color components, the relevant scheme first performs inverse discrete cosine transform on the first DCT coefficient among the P×P DCT coefficients, and then performs inverse discrete cosine transform on the second DCT coefficient, and so on in a serial manner to obtain P×P second color components.

[0102] For example, in the process of converting P×P second pixel values ​​to a second color space to obtain P×P first pixel values, the relevant scheme first performs a second color space conversion based on the first second pixel value among the P×P second pixel values ​​to obtain a first pixel value, and then performs a second color space conversion based on the second second pixel value among the P×P second pixel values ​​to obtain a first pixel value. This serial method is used to obtain P×P first pixel values.

[0103] Because the relevant solutions process data serially, the data processing speed is slow and the processing efficiency is low. Furthermore, the slow processing speed may lead to high CPU utilization (CPU utilization is the ratio of CPU processing time to total processing time; since the processing speed of electronic devices is slow (mainly due to the CPU processing data within the electronic device), the CPU processing time is long, resulting in high CPU utilization). Since CPU utilization is directly proportional to the power consumption of electronic devices (higher CPU utilization means higher power consumption), this also leads to higher power consumption in electronic devices.

[0104] In this embodiment, since the electronic device can perform parallel decoding processing on at least a portion of P×P target data in the target dataset to obtain P×P initial data in the initial dataset, and the decoding processing includes at least one of second color space conversion processing, inverse discrete cosine transform processing, and inverse quantization processing, the electronic device can process all or part of the data in the P×P target data in parallel during the execution of second color space conversion processing, and / or inverse discrete cosine transform processing, and / or inverse quantization processing. Compared with related schemes that perform serial processing on all the data in the P×P target data, the processing speed and efficiency of the electronic device can be improved, thereby reducing the waiting time for users when using the electronic device to decode data and improving the user experience.

[0105] Furthermore, by increasing the data processing speed of electronic devices, CPU utilization is reduced, thereby reducing the power consumption of electronic devices.

[0106] In conjunction with the second aspect, in one possible implementation of the second aspect, the electronic device is configured with a processor, the processor having at least one two-dimensional register, the two-dimensional register being composed of multiple one-dimensional registers; and, at least a portion of P×P target data in the target dataset is decoded in parallel to obtain P×P initial data in the initial dataset, including:

[0107] The processor reads P×P target data in parallel from at least one two-dimensional register; the processor then decodes at least a portion of the read P×P target data in parallel to obtain P×P initial data.

[0108] It should be understood that a one-dimensional register refers to a storage unit that can hold a single data bit or a group of data bits, while a two-dimensional register refers to a register that can store a two-dimensional array. Two-dimensional registers provide electronic devices with multi-dimensional data storage and access capabilities, and can store more data than one-dimensional registers. Two-dimensional registers can provide hardware support for parallel computation of data. For example, because two-dimensional registers can store more data, the processor can access and process this data simultaneously, thereby achieving data-level parallelism. Furthermore, two-dimensional registers can significantly improve data processing throughput and support the simultaneous parallel processing of multiple data sets.

[0109] For more information on this implementation method, please refer to [link / reference]. Figure 39 The processes shown in S524 and S525 can also be referred to Figure 40 The processes shown in S535 and S536 can also be referred to Figure 41 The processes S554 and S555 are shown.

[0110] In this embodiment of the application, the electronic device is equipped with at least one two-dimensional register. Since the two-dimensional register can provide hardware support for parallel computing of data, the electronic device can read P×P target data in parallel from at least one two-dimensional register through the processor; and the processor can perform parallel decoding processing on at least a portion of the read P×P target data to obtain P×P initial data, thereby improving the processing speed and efficiency of the electronic device in processing data.

[0111] In conjunction with the second aspect, in one possible implementation of the second aspect, the electronic device is further configured with internal memory; and, before the processor performs parallel decoding processing on at least a portion of the read P×P target data to obtain P×P initial data, the method further includes:

[0112] The processor loads P×P target data in parallel from internal memory.

[0113] For more information on this implementation method, please refer to [link / reference]. Figure 39 The process shown in S523 can also be referred to Figure 40 The process shown in S534 can also be referred to Figure 41 The process of S553 is shown.

[0114] It should be understood that the process of an electronic device loading data from memory into registers is as follows: the CPU accesses memory and writes the data from memory into the register. A related approach is to load data from memory into registers serially. For example, the CPU first accesses a piece of data in memory, writes that data into the register, then accesses another piece of data in memory and writes that data into the register again. Therefore, the CPU needs to access memory more times.

[0115] Frequent memory access by electronic devices can lead to several problems: 1. Decreased CPU performance. With frequent memory accesses, the CPU needs to constantly read data from memory, increasing its workload and causing performance degradation. 2. Increased power consumption. Memory access is one of the most power-intensive operations in electronic devices; frequent accesses increase power consumption. 3. Increased memory access latency. Memory access latency refers to the time delay between the CPU issuing a memory access request and receiving a response. Excessive memory accesses require the memory controller to process more requests, potentially causing queuing and increasing latency. These problems are particularly pronounced with large datasets such as high-resolution images.

[0116] In this embodiment, the electronic device can load P×P target data from internal memory into at least one two-dimensional register in parallel. In other words, the electronic device can write P×P target data stored in memory to at least one two-dimensional register with fewer memory accesses. Compared to related schemes that serially load data from memory into registers, the number of memory accesses can be reduced. Because the number of memory accesses is reduced, CPU performance can be improved, power consumption of the electronic device can be reduced, and memory access latency can be reduced. Furthermore, the reduced number of memory accesses can alleviate memory bandwidth pressure to some extent.

[0117] In conjunction with the second aspect, in one possible implementation of the second aspect, the target data are quantized DCT coefficients, the initial data are DCT coefficients, and the decoding process is inverse quantization; and, at least a portion of P×P target data in the target dataset are decoded in parallel to obtain P×P initial data in the initial dataset, including:

[0118] The P×P quantized DCT coefficients in the target dataset are dequantized to obtain the P×P DCT coefficients in the initial dataset.

[0119] This implementation method can be referenced. Figure 37 This implementation method is similar to Figure 37 The difference is that, Figure 37The first row of rectangles shown includes one DCT coefficient and one quantization coefficient within each rectangle, and a division operation is performed within each rectangle for quantization. In this implementation, each rectangle includes one quantized DCT coefficient and one quantization coefficient, and a multiplication operation is performed within each rectangle for inverse quantization.

[0120] In this embodiment, P×P quantized DCT coefficients in the target dataset can be dequantized to obtain P×P DCT coefficients in the initial dataset. Since the data is processed in parallel, the processing speed and efficiency of the dequantization process can be improved.

[0121] In conjunction with the second aspect, in one possible implementation of the second aspect, the target data are DCT coefficients, the initial data are at least one of the second color components in the second pixel values, the decoding process is inverse discrete cosine transform processing, the P×P target data are P×P DCT coefficients, the data organization of the P×P DCT coefficients is a two-dimensional array, the two-dimensional array includes P rows and P columns of DCT coefficients; and, at least a portion of the P×P target data in the target dataset is decoded in parallel to obtain P×P initial data in the initial dataset, including:

[0122] Perform inverse discrete cosine transform on the P rows of DCT coefficients in the target dataset in parallel to obtain P×P second color components in the initial dataset; or, perform inverse discrete cosine transform on the P columns of DCT coefficients in the target dataset in parallel to obtain P×P second color components in the initial dataset.

[0123] This implementation method can be referenced. Figure 26 and Figure 27 This implementation method is similar to Figure 26 and Figure 27 The difference is that, Figure 26 and Figure 27 This involves performing a Discrete Cosine Transform (DCT) on the second color component to obtain the DCT coefficients. This implementation method is... Figure 26 and Figure 27 The inverse process is the process of performing an inverse discrete cosine transform on the DCT coefficients to obtain the second color component. In other words, assuming... Figure 27 The matrix A shown contains 64 DCT coefficients in 8 rows and 8 columns. The electronic device can perform inverse discrete cosine transform on the 8 rows of DCT coefficients in parallel. The 8 DCT coefficients in each row can be processed in parallel or serially to obtain 8×8 Y components.

[0124] Assumption Figure 26The matrix A shown contains 64 DCT coefficients in 8 rows and 8 columns. The electronic device can perform inverse discrete cosine transform on 8 columns of DCT coefficients in parallel. The 8 DCT coefficients in each column can be processed in parallel or serially to obtain 8×8 Y components.

[0125] In this embodiment, the inverse discrete cosine transform (ICD) can be performed in parallel on the P rows of DCT coefficients in the target dataset to obtain P×P second color components in the initial dataset; or the inverse discrete cosine transform (ICD) can be performed in parallel on the P columns of DCT coefficients in the target dataset to obtain P×P second color components in the initial dataset. Since the data is processed in parallel, the processing speed and efficiency of the inverse discrete cosine transform can be improved.

[0126] In conjunction with the second aspect, in one possible implementation of the second aspect, the P rows of DCT coefficients in the target dataset are subjected to inverse discrete cosine transform in parallel to obtain P×P second color components in the initial dataset, including:

[0127] Based on the P rows of DCT coefficients and P columns of inverse DCT transform coefficients from the P×P inverse DCT transform coefficients in the target dataset, a vector outer product operation is performed in parallel, and the results of the vector outer product operation are accumulated to obtain P×P second color components. The P×P inverse DCT transform coefficients are preset coefficients used for inverse discrete cosine transform processing. The P×P inverse DCT transform coefficients are organized as a two-dimensional array, including P rows and P columns of inverse DCT transform coefficients. A vector outer product operation is performed between each row of DCT coefficients and the corresponding column of inverse DCT transform coefficients from the P columns.

[0128] The P columns of DCT coefficients in the target dataset are processed in parallel with inverse discrete cosine transform to obtain P×P second color components in the initial dataset, including:

[0129] The vector outer product operation is performed in parallel on the P columns of DCT coefficients and the P rows of inverse DCT transform coefficients in the target dataset, and the results of the vector outer product operation are accumulated to obtain P×P second color components. In this case, the vector outer product operation is performed on the DCT coefficients in each column and the inverse DCT transform coefficients in the P rows of inverse DCT transform coefficients corresponding to the DCT coefficients in each row.

[0130] This implementation method can be referenced. Figure 26 and Figure 27 This implementation method is similar to Figure 26 and Figure 27 The difference is that, Figure 26 and Figure 27The process involves performing a discrete cosine transform on 8×8 Y components and 8×8 DCT transform coefficients to obtain 8×8 DCT coefficients. However, the embodiment of this application performs a discrete cosine transform on 8×8 DCT coefficients and 8×8 inverse DCT transform coefficients to obtain 8×8 Y components.

[0131] For example, in this embodiment, the 8 rows of DCT coefficients and the 8 columns of inverse DCT transform coefficients may be subjected to a vector outer product operation in parallel, and the results of the vector outer product operation may be accumulated to obtain an 8-row, 8-column Y component. Alternatively, in this embodiment, the 8 columns of DCT coefficients and the 8 rows of inverse DCT transform coefficients may be subjected to a vector outer product operation in parallel, and the results of the vector outer product operation may be accumulated to obtain an 8-row, 8-column Y component.

[0132] The proposed approach involves performing inverse discrete cosine transform on P×P DCT coefficients and P×P inverse DCT transform coefficients using matrix multiplication. However, this method involves a large number of calculations and has high computational complexity.

[0133] This application embodiment can convert matrix multiplication into vector outer product accumulation. For example, it can perform vector outer product operations in parallel based on the P rows of DCT coefficients and the P columns of inverse DCT transform coefficients in the P×P inverse DCT transform coefficients in the target dataset, and accumulate the results of the vector outer product operations to obtain P×P second color components. Alternatively, it can perform vector outer product operations in parallel based on the P columns of DCT coefficients and the P rows of inverse DCT transform coefficients in the P×P inverse DCT transform coefficients in the target dataset, and accumulate the results of the vector outer product operations to obtain P×P second color components. By accumulating vector outer products, the number of calculations can be reduced and the computational complexity can be reduced compared to performing discrete cosine transform through matrix multiplication.

[0134] In conjunction with the second aspect, in one possible implementation of the second aspect, the electronic device includes a processor and internal memory. The P×P DCT coefficients are stored row-wise in the internal memory, and the P×P inverse DCT transform coefficients are also stored row-wise in the internal memory. Furthermore, before performing a vector outer product operation in parallel on the P rows of DCT coefficients and the P columns of inverse DCT transform coefficients in the P×P inverse DCT transform coefficients of the target dataset, and before accumulating the results of the vector outer product operation to obtain the P×P second color components, the method further includes:

[0135] The processor changes the storage method of the P×P inverse DCT transform coefficients in internal memory from row-based storage to column-based storage; and...

[0136] Before performing a vector outer product operation on the P columns of DCT coefficients and P rows of inverse DCT transform coefficients in the target dataset in parallel, and before summing the results of the vector outer product operation to obtain the P×P second color components, the method also includes:

[0137] The processor converts the storage method of P×P DCT coefficients in internal memory from row storage to column storage.

[0138] For more information on this implementation method, please refer to [link / reference]. Figure 29 The implementation method shown will not be elaborated here.

[0139] In related schemes, data is usually stored in memory in a row-based manner, and column data is not stored contiguously. Such schemes require multiple accesses to memory to obtain column data from P×P inverse DCT transform coefficients, or column data from P×P DCT coefficients.

[0140] In this embodiment, the processor can convert the storage method of the P×P inverse DCT transform coefficients in internal memory from row-based storage to column-based storage, or vice versa. This way, when column data of the P×P inverse DCT transform coefficients or the P×P DCT coefficients are needed, the column data is stored contiguously in memory. The electronic device can obtain the P columns of inverse DCT transform coefficients or the P columns of DCT coefficients with fewer memory accesses. By reducing the number of memory accesses, CPU performance can be improved, power consumption of the electronic device can be reduced, and memory access latency can be lowered.

[0141] In conjunction with the second aspect, in one possible implementation of the second aspect, the target data is the second pixel value, the initial data is the first pixel value, and the decoding process is a second color space conversion process; and, at least a portion of P×P target data in the target dataset is decoded in parallel to obtain P×P initial data in the initial dataset, including:

[0142] The second color space conversion is performed in parallel on the P×P second pixel values ​​in the target dataset to obtain the P×P first pixel values ​​in the initial dataset.

[0143] This implementation method can be referenced. Figure 24 The implementation shown is similar to... Figure 24 The difference is that, Figure 24 Each rectangle shown includes the R, G, and B components of the first pixel value, and the calculation is performed by substituting the data within each rectangle into Formula 1. In this implementation, it is possible to... Figure 24 The data within each rectangle is replaced with the Y, Cb, and Cr components of the second pixel value. Then, during the calculation, the replaced data within each rectangle is substituted into Formula 11 for calculation.

[0144] In this embodiment, the second color space conversion process can be performed in parallel on P×P second pixel values ​​in the target dataset to obtain P×P first pixel values ​​in the initial dataset. Since the data is processed in parallel, the processing speed and efficiency of the color space conversion process can be improved.

[0145] In conjunction with the second aspect, in one possible implementation of the second aspect, multiple target datasets are encoded to obtain multiple initial datasets, including:

[0146] Based on multiple decoding processes, multiple target datasets are decoded into multiple initial datasets. Each decoding process is used to decode a portion of the target datasets in parallel into a portion of the initial datasets.

[0147] It should be understood that decoding processing includes second color space conversion, inverse quantization processing, and inverse discrete cosine transform processing. Therefore, multiple decoding processes can refer to, for example, multiple second color space conversions and multiple inverse quantization processes.

[0148] It should also be understood that due to the large volume of image data (e.g., multiple target datasets contain a large amount of image data), electronic devices may not be able to decode all the image data at once. Therefore, electronic devices can decode multiple target datasets multiple times, with each decoding process involving a portion of the target dataset being processed in parallel. For example, if multiple target datasets include eight target datasets, the electronic device can decode these eight target datasets in four separate processes, processing two target datasets each time, and these two target datasets are processed in parallel.

[0149] In this embodiment of the application, multiple target datasets can be decoded into multiple initial datasets based on multiple decoding processes. Each decoding process is used to decode a portion of the target datasets in the multiple target datasets into a portion of the initial datasets in parallel. Since each decoding process is a parallel processing of a portion of the target datasets, the processing speed of electronic devices for processing data can be improved.

[0150] In conjunction with the second aspect, in one possible implementation of the second aspect, multiple target datasets are decoded into multiple initial datasets based on multiple decoding processes, including:

[0151] Determine the number of partial target datasets that can be processed in each decoding process; based on the number of partial target datasets that can be processed in each decoding process and the multiple target datasets, determine the total number of decoding processes; based on the total number of decoding processes, perform cyclic processing on the decoding of multiple target datasets to obtain multiple initial datasets; wherein, each cyclic processing process includes: performing parallel decoding processing on partial target datasets to obtain partial initial datasets.

[0152] For more information on this implementation method, please refer to [link / reference]. Figure 39 ,or Figure 40 ,or Figure 41 The implementation process shown will not be repeated here.

[0153] In conjunction with the second aspect, in one possible implementation of the second aspect, multiple target datasets are encoded to obtain multiple initial datasets, including:

[0154] Multiple target datasets are divided into multiple second data groups, each of which includes at least one target dataset. The number of second data groups matches the number of processors, and the amount of data in each second data group is the same.

[0155] Multiple processors decode multiple second data groups in parallel to obtain multiple initial datasets. Each processor is used to decode at least one second data group, and each processor processes the same number of second data groups.

[0156] For example, this implementation can refer to the following embodiment, which mentions that "K4 P×P quantized DCT coefficients and K4 P×P quantized coefficients can be divided into multiple data groups H14. The number of data groups H14 matches the number of CPUs. Each data group H14 includes at least P×P quantized DCT coefficients and P×P quantized coefficients, and the amount of data in each data group H14 is the same. Each CPU in the multiple CPUs can process at least one data group H14 in parallel to obtain K4 P×P DCT coefficients. Multiple CPUs process multiple data groups H14 in parallel, and the amount of data processed by each CPU is the same." This will not be elaborated further here.

[0157] It should be understood that when there is a large amount of data in multiple target datasets, the large amount of data in multiple target datasets may cause an imbalance in the load of multiple processors in a multi-core processor electronic device.

[0158] In this embodiment, multiple target datasets can be divided into smaller, equal groups, ensuring that each processor processes the same number of data sets. This balances the workload of each processor, ensuring optimal utilization and preventing overloaded processors from idling while others remain idle, thus improving the overall computational efficiency of the electronic device. Load balancing also reduces the risk of individual processor failures due to overload, enhancing the reliability and stability of the entire electronic device. Furthermore, it reduces the use of high-power processors, thereby lowering the overall energy consumption of the electronic device.

[0159] In conjunction with the second aspect, in one possible implementation of the second aspect, the logical size of the two-dimensional register is variable.

[0160] It should be understood that the logical size of a two-dimensional register can be varied based on the amount of data. For example, when the amount of data is large, the logical size of the two-dimensional register can be changed by increasing the width and / or depth of the original register. When the amount of data is small, the logical size can be changed by decreasing the width and / or depth of the original register. This adapts to the amount of data, allowing electronic devices to flexibly change the logical size of the two-dimensional register based on the data volume, enabling them to process data with varying amounts of data.

[0161] Thirdly, an electronic device is provided for performing the method provided in the first aspect. Specifically, the electronic device may include a processing unit for performing any possible implementation of the first or second aspect.

[0162] Fourthly, an electronic device is provided, comprising: one or more processors; one or more memories; wherein the one or more memories store one or more computer programs, the one or more computer programs including instructions that, when executed by the one or more processors, cause the electronic device to perform the method in any possible implementation of the first or second aspect described above.

[0163] Fifthly, a computer-readable storage medium is provided, including computer instructions that, when executed on an electronic device, cause the electronic device to perform the method described in the first or second aspect.

[0164] A sixth aspect provides a chip, including a memory for storing instructions; and a processor for retrieving and executing the instructions from the memory, causing an electronic device on which the chip is mounted to perform the method described in the first or second aspect above. Attached Figure Description

[0165] Figure 1 This is a schematic diagram of the hardware structure of an electronic device 100 provided in an embodiment of this application.

[0166] Figure 2 This is a schematic diagram of the software architecture of an electronic device 100 provided in an embodiment of this application.

[0167] Figure 3 This is a schematic diagram of a scenario provided in an embodiment of this application.

[0168] Figure 4 This is a schematic diagram of image encoding provided in an embodiment of this application.

[0169] Figure 5 This is a schematic diagram of an original image provided in an embodiment of this application.

[0170] Figure 6 This is a schematic diagram of a partitioning process provided in an embodiment of this application.

[0171] Figure 7 This is a schematic diagram of a color space conversion provided in an embodiment of this application.

[0172] Figure 8 This is a schematic diagram of a 4:4:4 chromaticity sampling format provided in an embodiment of this application.

[0173] Figure 9 This is a schematic diagram of a 4:2:2 chromaticity sampling format provided in an embodiment of this application.

[0174] Figure 10 This is a schematic diagram of a 4:2:0 chromaticity sampling format provided in an embodiment of this application.

[0175] Figure 11 This is a schematic diagram of a discrete cosine transform provided in an embodiment of this application.

[0176] Figure 12 This is a schematic diagram of a quantization process provided in an embodiment of this application.

[0177] Figure 13 This is a schematic diagram of an encoding process provided in an embodiment of this application.

[0178] Figure 14 This is a schematic diagram of a color space conversion provided in an embodiment of this application.

[0179] Figure 15 This is a schematic diagram of a two-dimensional register provided in an embodiment of this application.

[0180] Figure 16 This is a schematic diagram of writing a data subset A1 into memory, provided in an embodiment of this application.

[0181] Figure 17 This refers to a schematic diagram provided in an embodiment of this application, showing how to write multiple data subsets A1 into memory.

[0182] Figure 18 This is a schematic diagram of a parallel loading of a data subset A provided in an embodiment of this application.

[0183] Figure 19 This is a schematic diagram illustrating the loading of data from memory into a two-dimensional register, as provided in an embodiment of this application.

[0184] Figure 20 This is another schematic diagram of loading data from memory into a two-dimensional register provided in an embodiment of this application.

[0185] Figure 21 This is another schematic diagram of loading data from memory into a two-dimensional register provided in an embodiment of this application.

[0186] Figure 22 This is a schematic diagram illustrating another method of loading data from memory into a variable two-dimensional register in parallel, as provided in an embodiment of this application.

[0187] Figure 23 This is a schematic diagram of parallel data reading provided in an embodiment of this application.

[0188] Figure 24 This is a schematic diagram of a parallel data processing embodiment provided in this application.

[0189] Figure 25 This is a schematic diagram of a discrete cosine transform provided in an embodiment of this application.

[0190] Figure 26 This is a schematic diagram of an external vector accumulation provided in an embodiment of this application.

[0191] Figure 27 This is a schematic diagram of another vector-outside accumulation provided in the embodiments of this application.

[0192] Figure 28 This is a schematic diagram of reading column data provided in an embodiment of this application.

[0193] Figure 29 This is a schematic diagram illustrating the conversion from row-based storage to column-based storage provided in an embodiment of this application.

[0194] Figure 30 This is a schematic diagram of parallel data loading provided in an embodiment of this application.

[0195] Figure 31 This is a schematic diagram of another parallel data reading method provided in an embodiment of this application.

[0196] Figure 32 This is a schematic diagram of a parallel processing method provided in an embodiment of this application.

[0197] Figure 33 This is a schematic diagram of a quantization process provided in an embodiment of this application.

[0198] Figure 34 This is a schematic diagram of element-by-element division provided in an embodiment of this application.

[0199] Figure 35 This is a schematic diagram of another parallel data loading method provided in an embodiment of this application.

[0200] Figure 36 This is a schematic diagram of another parallel data reading method provided in an embodiment of this application.

[0201] Figure 37 This is a schematic diagram of another parallel processing method provided in an embodiment of this application.

[0202] Figure 38 This is a schematic diagram of image decoding provided in an embodiment of this application.

[0203] Figure 39 This is a schematic diagram of inverse quantization provided in an embodiment of this application.

[0204] Figure 40 This is a schematic diagram of an inverse discrete cosine transform provided in an embodiment of this application.

[0205] Figure 41 This is a schematic diagram of a color space conversion provided in an embodiment of this application.

[0206] Figure 42 This is a schematic diagram illustrating the implementation process of image encoding provided in an embodiment of this application.

[0207] Figure 43 This is a schematic diagram illustrating the implementation process of image decoding provided in an embodiment of this application. Detailed Implementation

[0208] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0209] Image encoding and image decoding are indispensable technologies in image processing. They make image storage and transmission more efficient, and also provide convenience for image editing and processing.

[0210] The proposed solution involves serially processing image data for encoding and decoding. However, this serial processing can slow down the processing speed of electronic devices, resulting in longer waiting times for users when encoding and decoding images, thus impacting the user experience.

[0211] To address the aforementioned issues, this application provides a data processing method that can process image data in parallel. Compared to serial image data processing, this method can improve the processing speed of electronic devices, thereby reducing the waiting time for users when encoding and decoding images using electronic devices and enhancing the user experience.

[0212] The data processing method provided in this application is applied to electronic devices. Electronic devices can also be called terminals, user equipment (UE), mobile stations (MS), mobile terminals (MT), etc. Electronic devices can be mobile phones, cameras, tablets, laptops, desktop computers, computers with wireless transceiver capabilities, virtual reality (VR) electronic devices, augmented reality (AR) electronic devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical surgery, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, and so on. The embodiments of this application do not limit the specific technologies or device forms used in the electronic devices.

[0213] To better understand the embodiments of this application, the structure of the electronic device in the embodiments of this application will be described below using a mobile phone as an example.

[0214] For example, Figure 1 A schematic diagram of the electronic device 100 is shown. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 130, antenna 1, antenna 2, a mobile communication module 140, a wireless communication module 150, an audio module 160, a speaker 160A, a receiver 160B, a microphone 160C, a headphone jack 160D, a sensor module 170, a camera 180, a display screen 190, etc. The sensor module 170 may include a pressure sensor 170A, a touch sensor 170B, etc.

[0215] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0216] Processor 110 may include one or more processing units, such as a central processing unit (CPU), application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0217] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0218] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0219] The processor 110 can also be configured with two-dimensional registers. Two-dimensional registers are registers that can store two-dimensional arrays. Two-dimensional registers can provide electronic devices with multi-dimensional data storage and access capabilities. Two-dimensional registers can be regarded as storage units composed of multiple one-dimensional registers. Two-dimensional registers can also be called two-dimensional matrix registers, matrix registers, array registers, etc.

[0220] In this embodiment of the application, the processor 110 can utilize two-dimensional registers to perform parallel processing of image data in the color space conversion step, discrete cosine transform step, and quantization step during the image encoding process, and also utilize two-dimensional registers to perform parallel processing of image data in the inverse quantization step, inverse discrete cosine transform step, and color space conversion step during the image decoding process.

[0221] The wireless communication function of electronic device 100 can be implemented through antenna 1, antenna 2, mobile communication module 140, wireless communication module 150, modem processor, and baseband processor.

[0222] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 140, and antenna 2 is coupled to wireless communication module 150, so that electronic device 100 can communicate with networks and other devices through wireless communication technology.

[0223] Electronic device 100 implements display functions through GPU, display screen 190, and application processor.

[0224] The display screen 190 is used to display images, videos, etc.

[0225] Electronic device 100 can perform shooting functions through ISP, camera 180, video codec, GPU, display 190 and application processor.

[0226] The ISP is used to process data fed back from the camera 180 degrees.

[0227] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the electronic device 100.

[0228] Internal memory 130 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 130. Internal memory 130 may include a program storage area and a data storage area.

[0229] Electronic device 100 can implement audio functions such as music playback and recording through audio module 160, speaker 160A, receiver 160B, microphone 160C, headphone jack 160D, and application processor.

[0230] Pressure sensor 170A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, pressure sensor 170A may be disposed on display screen 190. When a touch operation is applied to display screen 190, electronic device 100 detects the intensity of the touch operation based on pressure sensor 170A.

[0231] Touch sensor 170B, also known as a "touch panel," can be located on display screen 190. The touch sensor 170B and display screen 190 together form a touchscreen, also known as a "touch screen." Touch sensor 170B is used to detect touch operations applied to or near it, such as clicks and swipes.

[0232] This concludes the introduction to the hardware structure of electronic device 100. It is understandable that... Figure 1 The components included in the hardware structure shown do not constitute a specific limitation on the electronic device 100. The electronic device 100 may have more or fewer components than shown in the figure, may combine two or more components, or may have different component configurations. Figure 1 The various components shown can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0233] In addition, an operating system runs on top of these components. Examples include Apple's iOS, Google's Android, and Microsoft's Windows. Applications can be installed and run on this operating system.

[0234] The operating system of electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses the layered architecture Android system as an example to exemplify the software structure of electronic device 100.

[0235] Figure 2 This is a schematic diagram of the software structure of an electronic device provided in an embodiment of this application.

[0236] like Figure 2 As shown, electronic devices can adopt a layered architecture, dividing the software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the software layers of the software structure are divided from top to bottom as follows: application layer, application framework layer, native layer, native interface layer, and hardware layer, etc.

[0237] The application layer, or application layer for short, can include a series of application packages, such as gallery apps, camera apps, browsers, and social applications. When these application packages are run, they can access the various modules provided by the application framework layer through application programming interfaces (APIs) and execute corresponding business logic.

[0238] The application framework layer may include image processing modules, etc. These image processing modules can refer to image format modules (ImageFormat), encoders (ImageEncoder), decoders (ImageDecoder), bitmap factories (BitmapFactory), etc., and are used to encode image data. The image processing modules are also used to decode the encoded image data.

[0239] In other embodiments, the image processing module may also be located in the Java layer, which may be located in the application layer or in other layers, such as the application framework layer, etc. This application embodiment does not limit this.

[0240] The native interface layer, also known as the Java Native Interface (JNI) layer, serves as a connector to the upper layers (such as...). Figure 2 The application framework layer (or Java layer) shown is a bridge between the local layer and the application framework layer, enabling the upper layer to call various modules in the local layer and execute corresponding business logic.

[0241] The Native Layer includes image processing components and libraries. Image processing components may include encoders (ImageEncoder), decoders (ImageDecoder), and bitmap factories (BitmapFactory), while image processing libraries may include library A (libjpeg-turbo) and library B (libjpeg). The image processing components are used to call the image processing libraries to encode and decode image data.

[0242] In other embodiments, the image processing components and image processing libraries may also be located in other layers, such as the Hardware Abstraction Layer (HAL).

[0243] The hardware layer includes components such as the CPU and sensors.

[0244] It should be noted that, Figure 2 The layers in the illustrated software architecture and the components contained in each layer do not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include more layers than illustrated, such as a system library (FWK LIB) layer and a kernel layer. Furthermore, each layer may include more or fewer components than illustrated, which is not limited in this application.

[0245] It is understood that, in order to implement the data processing methods in the embodiments of this application, electronic devices include hardware and / or software modules that perform various functions. Based on the algorithmic steps of the examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments.

[0246] It should be noted that although the embodiments of this application are illustrated using the Android system as an example, the basic principles are equally applicable to electronic devices based on operating systems such as iOS or Windows.

[0247] The above embodiments describe the hardware structure and software architecture of electronic devices. The following embodiments describe the application scenarios of the data processing method provided in this application.

[0248] The data processing method provided in this application can be applied to scenarios involving image encoding or decoding.

[0249] For example, the above application scenario could be Scenario 1, where a user takes pictures or videos using an electronic device and previews or views them. In this scenario, the electronic device can encode the captured pictures or videos using the data processing method provided in this application embodiment and store the encoded pictures or videos. Then, the electronic device can decode the encoded pictures or videos using the data processing method provided in this application embodiment, allowing the user to preview or view the captured pictures or videos.

[0250] For example, the above scenario could be scenario two, which is an application scenario where a user views pictures or videos in a gallery application. In this scenario, the electronic device can decode the encoded pictures or videos stored in the gallery application using the data processing method provided in the embodiments of this application so that the user can view the pictures or videos.

[0251] For example, the above scenario could be Scenario 3, which is an application scenario where a user sends or receives images or videos in a social application (which can also be called a messaging application, instant messaging application, chat application, etc.). In this scenario, before the user sends images or videos through the social application, the electronic device encodes the images or videos using the data processing method provided in this application embodiment. Then, the user sends the encoded image or video file through the social application. After the electronic device receives the encoded images or videos through the social application, it can decode the encoded images or videos using the data processing method provided in this application embodiment to display the images or videos on the social application.

[0252] For example, the above scenario could be scenario four, which is an application scenario where a user browses images or videos in a browser. In this scenario, after the electronic device decodes the encoded images or videos using the data processing method provided in this application embodiment, the browser can display the images or videos for the user to browse.

[0253] For example, the above scenario could be scenario five, which is an application scenario where a user edits images or videos in an image editing application (this application is used to perform editing operations such as trimming and cropping on images or videos). In this scenario, the electronic device needs to decode the encoded images or videos using the data processing method provided in the embodiments of this application to restore the original image data, after which the user can edit the original image data corresponding to the images or videos.

[0254] It should be noted that the application scenarios for electronic devices to encode or decode images include, but are not limited to, scenarios one through five mentioned above. The following uses scenario one as an example to illustrate the application scenarios of encoding and decoding.

[0255] For example, please refer to Figure 3 , Figure 3 This is a schematic diagram of a scenario provided in an embodiment of this application. Figure 3 This explanation uses a mobile phone as an example.

[0256] Figure 3 The user interface 310 shown in (a) includes a "camera" icon 311, which the user can tap. In response to the user tapping icon 311, the phone displays... Figure 3Interface 320 is shown in (b) above. Users can perform shooting operations on interface 320; for example, a user can click on control 321 on interface 320. In response to the user clicking control 321, the mobile phone first captures the subject 323 in the viewfinder 322, then encodes the original image data using the data processing method provided in this embodiment, stores the encoded image data, and finally generates and displays a thumbnail based on the encoded image data. Figure 3 Interface 330 is shown in (c) in the figure, and interface 330 includes thumbnail 331.

[0257] In addition, users can also click on thumbnail 331. In response to this click, the phone can decode the encoded thumbnail using the data processing method provided in this embodiment to display it. Figure 3 (d) shows interface 340, which includes the decoded image 341.

[0258] The image encoding mentioned above refers to the process of converting raw image data into a more compact format. Because raw image data is characterized by its large volume and redundant information, image encoding typically involves compressing the raw image data. This reduces redundant information and thus reduces the amount of data.

[0259] Electronic devices use image encoding to compress and encode raw image data into more compact formats, including Joint Photographic Experts Group (JPEG), JPEG 2000, Advanced Video Coding for Generic Audiovisual Services (H.264 / AVC), and WebP. Image decoding is the reverse process of encoding, that is, the process of decoding and decompressing the compressed and encoded image data to restore the original image data.

[0260] The image encoding process includes processing steps such as color space conversion, discrete cosine transform, quantization, and encoding of image data. The following embodiments take JPEG format as an example. First, the overall process of image encoding is introduced, and then the steps of color space conversion, discrete cosine transform, and quantization performed using the data processing method provided in the embodiments of this application are introduced.

[0261] The overall process of image encoding

[0262] For example, please refer to Figure 4 , Figure 4This is a schematic diagram of image encoding provided in an embodiment of this application. It illustrates the process of compressing and encoding raw image data to convert it into JPEG format. This process can be simply referred to as JPEG compression encoding, and it includes steps S41 to S47.

[0263] S41, the electronic device acquires the dataset corresponding to the original image. The dataset corresponding to the original image includes M×N pixel values. Each pixel value includes color component 1 of color space 1. Color component 1 includes color component 11, color component 12 and color component 13. That is, the dataset corresponding to the original image includes dataset A, dataset B and dataset C. Dataset A includes M×N color component 11, dataset B includes M×N color component 12 and dataset C includes M×N color component 13, where M and N are integers greater than 0.

[0264] It should be understood that the dataset corresponding to the original image refers to the collection of pixel values ​​in the uncompressed and unencoded original image. The number of pixel values ​​in the dataset corresponding to the original image is M×N, which can also be understood as the resolution of the original image being M×N, where M and N are integers greater than 0. M can be equal to N, or M can be greater than or less than N; this application does not limit this. This application uses M=N as an example for illustrative explanation. For example, please refer to... Figure 5 , Figure 5 This is a schematic diagram of an original image provided in an embodiment of this application. Figure 5 (a) shows the original image, which includes 32 pixel values ​​in the horizontal direction and 32 pixel values ​​in the vertical direction. Figure 5 The original image shown in (a) has a resolution of 32 pixels (Px) × 32px. Figure 5 In this context, M = N = 32, which is based on... Figure 5 The dataset corresponding to the original image obtained in (a) includes 32×32=1024 pixel values.

[0265] It should also be understood that a pixel's color space is a mathematical model used to describe and represent the colors of an image. It defines a set of numerical values ​​for colors that can be used to accurately represent and reproduce colors on digital devices. A color space is typically defined by three or more parameters (usually three), which can be color intensity, hue, and saturation, among others. Common color spaces include the Red-Green-Blue (RGB) color space, the Luminance-Chrominance (YUV) color space, and the Luminance-Blue-Chrominance-Red-Chrominance (YCbCr) color space. In simple terms, a pixel value is composed of parameters within a color space.

[0266] because Figure 4 This illustrates the JPEG compression encoding process. JPEG compression encoding requires converting the original image from the RGB color space to the YCbCr color space. Therefore, color space 1 in step S41 can refer to the RGB color space. The color component 1 of color space 1 refers to the three color components in the RGB color space. These three color components 1 can include a red component (for ease of description, this red component is called color component 11), a green component (color component 12), and a blue component (color component 13). That is, each pixel value in step S41 includes the three color components 1: red (R component), green (G component), and blue (B component). For example, as shown... Figure 5 As shown in (b), the dataset corresponding to the original image contains 32×32=1024 pixel values, and each pixel value includes R component, G component and B component.

[0267] Of course, in other embodiments, each pixel value may include more or fewer color components than three. For example, each pixel value may also include an alpha component, which is used to represent the transparency of the color.

[0268] Similarly, it can be understood that since each pixel value includes R, G, and B components, the dataset corresponding to the original image acquired by the electronic device can include dataset A, dataset B, and dataset C. Dataset A includes M×N color components 11, dataset B includes M×N color components 12, and dataset C includes M×N color components 13. For example, as... Figure 5 As shown in (c), dataset A contains 32×32 R components, dataset B contains 32×32 G components, and dataset C contains 32×32 B components.

[0269] In implementation, for example, electronic devices respond to user input. Figure 3 The operation of clicking control 321 in the interface 320 shown allows the image sensor in the electronic device to capture the subject 323. The captured image data is then processed by an image signal processor to obtain M×N R components, M×N B components, and M×N G components corresponding to the subject 323. The image signal processor then transmits these M×N R components, M×N B components, and M×N G components to the CPU, enabling the CPU to obtain the dataset corresponding to the original image. This application embodiment does not limit the implementation method of the electronic device obtaining the dataset corresponding to the original image.

[0270] S42, the electronic device divides the M×N color components 11 in dataset A into (M / P)×(N / P) data subsets A1, each containing P×P color components 11; divides the M×N color components 12 in dataset B into (M / P)×(N / P) data subsets B1, each containing P×P color components 12; divides the M×N color components 13 in dataset C into (M / P)×(N / P) data subsets C1, each containing P×P color components 13; where P is an integer greater than 0, P is less than M, and P is less than N.

[0271] It should be understood that the JPEG compression encoding process requires steps such as Discrete Cosine Transform (DCT) and quantization, as mentioned below. Since the computational complexity of DCT and quantization is relatively high, to reduce the computational complexity in subsequent steps, the electronic device needs to partition datasets A, B, and C from the original image's dataset. Dataset A is divided into multiple subsets A1, dataset B into multiple subsets B1, and dataset C into subset C1. This reduces the computational complexity of subsequent steps. The implementation process of the partitioning process in step S42 is not limited in this embodiment.

[0272] It should also be understood that, in order to reduce computational complexity while ensuring visual coherence of the image, each subset of data after division may include 8×8 color components, i.e., P = 8 in step S42. Of course, in other embodiments, P may be other values, and this application embodiment does not limit this.

[0273] For example, please refer to Figure 6 , Figure 6This is a schematic diagram of a partitioning process provided in an embodiment of this application. The electronic device partitions the M×N color components 11 in dataset A to obtain (M / P)×(N / P) data subsets A1, where each data subset A1 includes P×P color components 11. This can mean that the electronic device... Figure 6 (a) shows the 32×32 R components in dataset A, which are then partitioned to obtain the following: Figure 6 (b) shows (32 / 8) × (32 / 8) data subsets A1, each containing 8 × 8 R components. The electronic device partitions the M × N color components I2 in dataset B to obtain (M / P) × (N / P) data subsets B1, where each subset B1 contains P × P color components I2. This can refer to the electronic device... Figure 6 (a) shows the 32×32 G components in dataset B, which are then partitioned to obtain the following: Figure 6 (b) shows (32 / 8) × (32 / 8) data subsets B1, each containing 8 × 8 B components. The electronic device partitions the M × N color components 13 in dataset C to obtain (M / P) × (N / P) data subsets C1, where each subset C1 contains P × P color components 13. This can refer to the electronic device... Figure 6 The dataset C shown in (a) is divided into 32×32 B components, resulting in the following: Figure 6 (b) shows (32 / 8) × (32 / 8) data subsets C1, each of which contains 8 × 8 B components.

[0274] It should be noted that when M×N is not divisible by P×P, interpolation is required in the image data to make M×N divisible by P×P.

[0275] Step S42 can also be simply understood as the electronic device dividing the original image into multiple image blocks, each image block including 8×8 pixel values, and subsequent steps can perform compression encoding processing based on each image block.

[0276] S43, the electronic device performs color space conversion processing on (M / P)×(N / P) data subsets A1, (M / P)×(N / P) data subsets B1 and (M / P)×(N / P) data subsets C1 to obtain (M / P)×(N / P) data subsets A2, (M / P)×(N / P) data subsets B2 and (M / P)×(N / P) data subsets C2. Each data subset A2 includes P×P color components 21 of color space 2, each data subset B2 includes P×P color components 23 of color space 2, and each data subset C2 includes P×P color components 23 of color space 2. Color components 21, 22 and 23 refer to color components 2 in color space 2.

[0277] It should be understood that the principle of JPEG compression encoding is to distinguish between important and unimportant data in the original image data, and then reduce or remove the unimportant data. This reduces redundant information in the original image data and achieves the purpose of "compression".

[0278] In JPEG compression encoding, to distinguish between important and unimportant data in the original image data, the color space of the original image (e.g., the RGB color space mentioned in the previous example) needs to be converted to the YCbCr color space. This is because human vision has the following visual characteristics: changes in brightness and darkness in an image are more easily perceived by the human eye than changes in color. The YCbCr color space is a color space derived based on this visual characteristic. The YCbCr color space includes a luminance component Y (Y component), a blue color difference component Cb (Cb component), and a red color difference component Cr (Cr component). The Y component determines the brightness and darkness of the image, while the Cb and Cr components determine the color. Thus, after the pixel values ​​in the original image data are converted to the YCbCr color space, the Y component can be considered important data, while the Cb and Cr components can be considered unimportant data, allowing for the distinction between important and unimportant data in the original image data.

[0279] It should also be understood that in step S43, color component 2 of color space 2 can refer to the YcbCr color component in the YCbCr color space. Color component 21 refers to the Y component, color component 22 refers to the Cb component, and color component 23 refers to the Cr component.

[0280] Similarly, it can be understood that the electronic device can substitute any color component 11 in any data subset A1 of (M / P)×(N / P) data subsets A1, color component 12 corresponding to any color component 11 in (M / P)×(N / P) data subsets B1, and color component 13 corresponding to any color component 11 in (M / P)×(N / P) data subsets C1 into the following formula 1 to obtain a color component 21 in (M / P)×(N / P) data subsets A2, a color component 22 in (M / P)×(N / P) data subsets B2, and a color component 23 in (M / P)×(N / P) data subsets C2, so as to obtain (M / P)×(N / P) data subsets A2, (M / P)×(N / P) data subsets B2, and (M / P)×(N / P) data subsets C2.

[0281]

[0282] Among them, R i It refers to any color component 11 (e.g., the R component) in a (M / P)×(N / P) subset A1 of data, G i This refers to the data in (M / P)×(N / P) subsets A2 that are related to R. i The corresponding color component 12 (e.g., G component), B i This refers to the data in (M / P)×(N / P) subsets A3 that are related to R. i The corresponding color component is 13 (e.g., the B component). Y i It refers to a color component 21 (e.g., the Y component) in a (M / P) × (N / P) subset A2 of data, Cb i It refers to a color component 22 (e.g., the Cb component) in a (M / P) × (N / P) subset of data B2. i It refers to a color component 23 (e.g., the Cr component) in a (M / P)×(N / P) subset of data C2.

[0283] For example, please refer to Figure 7 , Figure 7 This is a schematic diagram illustrating a color space conversion provided in an embodiment of this application. The electronic device can convert the color space based on the above formula one. Figure 7In Figure (a), the (32 / 8) × (32 / 8) data subsets A1, B1, and C1 are converted from the RGB color space to the YCbCr color space, resulting in (32 / 8) × (32 / 8) data subsets A2, B2, and C2. Each data subset A2 includes 8 × 8 Y components, each data subset B2 includes 8 × 8 Cb components, and each data subset C2 includes 8 × 8 Cr components.

[0284] The process of obtaining a data subset A2, a data subset B2, and a data subset C2 from a data subset A1, a corresponding data subset B1, and a corresponding data subset C1 can be as follows: For example, an electronic device can... Figure 7 Substituting the 8×8 R components in the first row and first column of the 4×4 data subset A1, the 8×8 G components in the first row and first column of the 4×4 data subset B1, and the 8×8 B components in the first row and first column of the 4×4 data subset C1 into Formula 1 above, we obtain... Figure 7 (b) shows the 8×8 R components in the first row and first column of the 4×4 data subset A2, the 8×8 G components in the first row and first column of the 4×4 data subset B2, and the 8×8 B components in the first row and first column of the 4×4 data subset C2. The 4×4 data subsets A2, B2, and C2 are obtained by this method.

[0285] For details on the specific implementation of step S43, please refer to the following embodiment, which will not be elaborated here.

[0286] S44, the electronic device performs chromaticity sampling on (M / P)×(N / P) data subsets A2, (M / P)×(N / P) data subsets B2 and (M / P)×(N / P) data subsets C2 to obtain Q data subsets A3, J data subsets B3 and L data subsets C3; each data subset A3 includes P×P color components 21, each data subset B3 includes P×P color components 22, and each data subset C3 includes P×P color components 23, where Q, J, and L are integers greater than 0.

[0287] As can be seen from the above embodiments, the principle of JPEG compression encoding is to distinguish between important and unimportant data in the original image data, and then reduce or remove the unimportant data. The electronic device can distinguish between important data (for example, important data can refer to the luminance component Y) and unimportant data (unimportant data can refer to the blue color difference component Cb and the red color difference component Cr) through S43, and the electronic device can reduce the amount of data for the Cb and Cr components through S43.

[0288] It should be understood that chroma sampling refers to downsampling the Cb and Cr components in the YCbCr color space during the JPEG compression encoding process, in order to reduce the amount of image data and improve compression efficiency.

[0289] It should also be understood that chroma sampling formats include 4:4:4, 4:2:2, and 4:2:0. The chroma sampling format determines the number of data units included in the Minimum Coded Unit (MCU), which refers to the smallest block of data encoded in an image. An MCU consists of one or more 8x8 data units. For example, with a chroma sampling format of 4:4:4, the MCU includes one 8x8 data unit. With a chroma sampling format of 4:2:2, the MCU includes two 8x8 data units. With a chroma sampling format of 4:2:0, the MCU includes four 8x8 data units.

[0290] The 4:4:4 format refers to a sampling format in which the Y, Cb, and Cr components are not downsampled; that is, the number of Y, Cb, and Cr components remains unchanged. For an example, please refer to [link / reference needed]. Figure 8 , Figure 8 This is a schematic diagram of a 4:4:4 chromaticity sampling format provided in an embodiment of this application. Figure 8 Figure (a) shows that when the chroma sampling format is 4:4:4, an MCU includes one data unit, and one data unit includes 8x8 pixel values. Since each pixel value can be composed of Y, Cb, and Cr components after color space conversion via S43, an MCU can include... Figure 8 (a) shows the 8x8 Y components, 8x8 Cb components, and 8x8 Cr components. The 8x8 Y components can refer to... Figure 7 (b) shows one of the (32 / 8) × (32 / 8) data subsets A2 containing 8 × 8 Y components. The 8 x 8 Cb components can refer to... Figure 7 (b) shows one of the (32 / 8) × (32 / 8) data subsets B2 containing 8 × 8 Cb components. The 8x8 Cr components can refer to... Figure 7(b) shows one of the (32 / 8) × (32 / 8) data subsets C2 containing 8 × 8 Cr components.

[0291] like Figure 8 As shown in (a), the electronic device performs chroma sampling on data subsets A2, B2, and C2 in a 4:4:4 format, which yields... Figure 8 (a) shows data subsets A3, B3, and C3. Data subset A3 contains the same number of Y components as data subset A2, data subset B3 contains the same number of Cb components as data subset B2, and data subset C3 contains the same number of Cr components as data subset C2.

[0292] based on Figure 8 The sampling method shown in (a) involves the electronic device performing chroma sampling on (M / P)×(N / P) data subsets A2, B2, and C2 in a 4:4:4 format, resulting in Q data subsets A3, J data subsets B3, and L data subsets C3. This can be referenced. Figure 8 (b) In this context, for example, electronic devices in a 4:4:4 format... Figure 8 In diagram (b), the (32 / 8) × (32 / 8) data subsets A2, B2, and C2 shown are subjected to chromatic sampling to obtain (32 / 8) × (32 / 8) data subsets A3, B3, and C3. Figure 8 As can be seen from (b), when the chroma sampling format is 4:4:4, the number of data subsets A3 is the same as the number of data subsets A2, therefore Q = (M / P) × (N / P) in step S44. The number of data subsets B3 is the same as the number of data subsets B2, therefore J = (M / P) × (N / P) in step S44. The number of data subsets C3 is the same as the number of data subsets C2, therefore L = (M / P) × (N / P) in step S44.

[0293] The 4:2:2 format refers to a sampling format where the number of Y components remains constant, while the number of Cb and Cr components in the horizontal direction is reduced by half. For an example, please refer to [reference needed]. Figure 9 , Figure 9 This is a schematic diagram of a 4:2:2 chromaticity sampling format provided in an embodiment of this application. Figure 9Figure (a) shows that when the chroma sampling format is 4:2:2, an MCU includes two data units, each containing 8x8 pixel values. Since each pixel value can be composed of Y, Cb, and Cr components after color space conversion via S43, an MCU can include... Figure 9 Figure (a) shows two data subsets A2, B2, and C2, where each subset A2 contains 8x8 Y components, each subset B2 contains 8x8 Cb components, and each subset C2 contains 8x8 Cr components. The two data subsets A2 can refer to... Figure 7 (b) shows two horizontally adjacent data subsets A2 out of the (32 / 8) × (32 / 8) data subsets A2. The two data subsets B2 can refer to... Figure 7 (b) shows two horizontally adjacent data subsets B2 among the (32 / 8) × (32 / 8) data subsets B2. The two data subsets C2 can refer to... Figure 7 (b) shows two horizontally adjacent data subsets C2 among the (32 / 8) × (32 / 8) data subsets C2.

[0294] like Figure 9 As shown in (a), the electronic device performs chroma sampling on two data subsets A2, B2, and C2 in a 4:2:2 format, which yields... Figure 9 Figure (a) shows two data subsets A3, one data subset B3, and one data subset C3. The number of Y components included in the two data subsets A3 remains unchanged compared to the two data subsets A2. The number of Cb components in the one data subset B3 is halved in the horizontal direction compared to the two data subsets B2, while remaining unchanged in the vertical direction. The number of Cr components in the one data subset C3 is halved in the horizontal direction compared to the two data subsets C2, while remaining unchanged in the vertical direction.

[0295] based on Figure 9 The sampling method shown in (a) involves the electronic device performing chromaticity sampling on (M / P)×(N / P) data subsets A2, B2, and C2 in a 4:2:2 format, resulting in Q data subsets A3, J data subsets B3, and L data subsets C3. (See reference...) Figure 9 (b) In this context, for example, electronic devices in a 4:2:2 format... Figure 9In (b) of the diagram, chromatic sampling of the (32 / 8) × (32 / 8) data subsets A2, B2, and C2 yields (32 / 8) × (32 / 8) data subsets A3, B3, and C3. Figure 9 As can be seen from (b), when the chroma sampling format is 4:2:2, the number of data subsets A3 is the same as the number of data subsets A2, therefore Q in step S44 = (M / P) × (N / P). The number of data subsets B3 is half that of data subset B2, therefore J in step S44 = (M / P) × (N / P) × (1 / 2). The number of data subsets C3 is half that of data subset C2, therefore L in step S44 = (M / P) × (N / P) × (1 / 2).

[0296] The 4:2:0 format refers to a sampling format where the number of Y components remains constant, while the number of Cb components in both the horizontal and vertical directions is reduced by half, and the number of Cr components in both the horizontal and vertical directions is also reduced by half. For an example, please refer to... Figure 10 , Figure 10 This is a schematic diagram of a 4:2:0 chromaticity sampling format provided in an embodiment of this application. Figure 10 Figure (a) shows that in a chroma sampling format of 4:2:0, an MCU includes 4 data units, each containing 8x8 pixel values. Since each pixel value can be composed of Y, Cb, and Cr components after color space conversion via S43, an MCU can include... Figure 9 Figure (a) shows four data subsets A2, B2, and C2, where each subset A2 contains 8x8 Y components, each subset B2 contains 8x8 Cb components, and each subset C2 contains 8x8 Cr components. The four data subsets A2 can refer to... Figure 7 (b) shows four data subsets A2 that are adjacent in both the horizontal and vertical directions among the (32 / 8) × (32 / 8) data subsets A2. These four data subsets B2 can refer to... Figure 7 (b) shows four data subsets B2 that are adjacent in both the horizontal and vertical directions among the (32 / 8) × (32 / 8) data subsets B2. These four data subsets C2 can refer to... Figure 7 (b) shows four data subsets C2 that are adjacent in both the horizontal and vertical directions among the (32 / 8) × (32 / 8) data subsets C2.

[0297] like Figure 10 As shown in (a), the electronic device performs chroma sampling on four data subsets A2, B2, and C2 in a 4:2:0 format, which yields... Figure 10 Figure (a) shows four data subsets A3, one data subset B3, and one data subset C3. The number of Y components included in the four data subsets A3 remains unchanged compared to the four data subsets A2. The number of Cb components in the one data subset B3 is reduced by half in both the horizontal and vertical directions compared to the four data subsets B2. The number of Cr components in the one data subset C3 is reduced by half in both the horizontal and vertical directions compared to the four data subsets C2.

[0298] based on Figure 10 The sampling method shown in (a) involves the electronic device performing chromatic sampling on (M / P)×(N / P) data subsets A2, B2, and C2 in a 4:2:0 format, resulting in Q data subsets A3, J data subsets B3, and L data subsets C3. This can be referenced. Figure 10 (b) In this context, for example, electronic devices use a 4:2:0 format. Figure 10 In diagram (b), the (32 / 8) × (32 / 8) data subsets A2, B2, and C2 shown are subjected to chromatic sampling, resulting in (32 / 8) × (32 / 8) data subsets A3, B3, and C3. Figure 10 As can be seen from (b) above, when the chroma sampling format is 4:2:0, the number of data subsets A3 is the same as the number of data subsets A2, therefore Q in step S44 = (M / P) × (N / P). The number of data subsets B3 is 1 / 4 of the number of data subsets B2, therefore J in step S44 = (M / P) × (N / P) × (1 / 4). The number of data subsets C3 is 1 / 2 of the number of data subsets C2, therefore L in step S44 = (M / P) × (N / P) × (1 / 4).

[0299] S45, the electronic device performs discrete cosine transform on Q data subsets A3, J data subsets B3 and L data subsets C3 to obtain G = Q + J + L data subsets D; each data subset D includes P × P DCT coefficients.

[0300] It should be understood that the Discrete Cosine Transform (DCT) can transform image data from the spatial domain to the frequency domain; for example, it can transform... Figure 10(b) shows that the 8×8 Y components (spatial domain) in a data subset A3 are converted into 8×8 frequency coefficients (frequency domain).

[0301] After discrete cosine transform (DCT), image data can be converted into frequency coefficients of varying degrees, which helps subsequent steps to effectively compress the data. This is because DCT converts image data into low-frequency, mid-frequency, and high-frequency coefficients. Low-frequency coefficients represent the basic shape and color gradations of the image, primarily determining the overall brightness and coarse details. Therefore, low-frequency coefficients are highly important to human vision, and more are typically retained in subsequent quantization steps to preserve the image's main structure and color. Mid-frequency coefficients represent medium-level details such as texture and edges, significantly affecting local details and textures. Therefore, moderately retaining mid-frequency coefficients in subsequent quantization steps can achieve effective compression while maintaining image quality. High-frequency coefficients represent subtle details and noise in the image, such as minute textures and color variations. Therefore, high-frequency coefficients have less visual impact and are easily ignored by the human eye. Consequently, in subsequent quantization processes, these high-frequency coefficients can be significantly simplified or even zeroed out, achieving higher compression rates with almost no impact on visual quality.

[0302] It should also be understood that the frequency coefficients mentioned above are often referred to as DCT coefficients.

[0303] Similarly, it can be understood that the process by which an electronic device performs discrete cosine transform on Q data subsets A3, J data subsets B3, and L data subsets C3 to obtain G = Q + J + L data subsets D can be referenced. Figure 11 . Figure 11 This is a schematic diagram of a discrete cosine transform provided in an embodiment of this application. For example... Figure 11 As shown, the electronic device performs a discrete cosine transform on (32 / 8) × (32 / 8) data subsets A3 to obtain (32 / 8) × (32 / 8) data subsets D. The electronic device performs a discrete cosine transform on (32 / 8) × (32 / 8) × (1 / 4) data subsets B3 to obtain (32 / 8) × (32 / 8) × (1 / 4) data subsets D. The electronic device performs a discrete cosine transform on (32 / 8) × (32 / 8) × (1 / 4) data subsets B3 to obtain (32 / 8) × (32 / 8) × (1 / 4) data subsets D. Performing Discrete Cosine Transform (DCT) on 32 / 8)×(1 / 4) data subsets C3 yields (32 / 8)×(32 / 8)×(1 / 4) data subsets D, resulting in a total of G=(32 / 8)×(32 / 8)+(32 / 8)×(32 / 8)×(1 / 4)+(32 / 8)×(32 / 8)×(1 / 4) data subsets D, with each data subset D containing 8×8 DCT coefficients.

[0304] It should be noted that Figure 11The Q data subsets A3, J data subsets B3, and L data subsets C3 shown are data subsets obtained by the electronic device through color sampling in a 4:2:0 format. After the electronic device performs color sampling in other formats (e.g., 4:4:4, 4:2:2, etc.) to obtain the Q data subsets A3, J data subsets B3, and L data subsets C3, the method for performing a discrete cosine transform on them can be found in [reference needed]. Figure 11 This will not be elaborated upon here.

[0305] For details on the specific implementation of step S45, please refer to the following embodiment, which will not be elaborated here.

[0306] S46, the electronic device quantizes G data subsets D to obtain G data subsets E, each data subset E including P×P quantized DCT coefficients.

[0307] It should be understood that quantization refers to the process of mapping the DCT coefficients included in each data subset D to a finite dataset. For example, if a data subset D contains P×P DCT coefficients, after quantization, data subset D is mapped to a finite dataset, which can be called data subset E. Data subset E contains P×P quantized DCT coefficients. DCT coefficients represent the frequency components of an image. Quantization reduces the precision of these coefficients. Because the precision of the DCT coefficients is reduced, DCT coefficients can be stored in less space, thereby achieving the purpose of data compression.

[0308] It should also be understood that the process by which electronic devices quantize G data subsets D to obtain G data subsets E, each of which includes P×P quantized DCT coefficients, can be found in [reference needed]. Figure 12 . Figure 12 This is a schematic diagram of a quantization process provided in an embodiment of this application. For example... Figure 12 As shown, the process of quantizing G data subsets D by the electronic device includes: the electronic device quantizes (32 / 8)×(32 / 8)+(32 / 8)×(32 / 8)×(1 / 4)+(32 / 8)×(32 / 8)×(1 / 4) data subsets D to obtain (32 / 8)×(32 / 8)+(32 / 8)×(32 / 8)×(1 / 4)+(32 / 8)×(32 / 8)×(1 / 4) data subsets E, and the 8×8 DCT coefficients in each data subset D are quantized into 8×8 quantized DCT coefficients.

[0309] For details on the specific implementation of step S46, please refer to the following embodiment, which will not be elaborated here.

[0310] S47, the electronic device encodes G data subsets E to obtain G sets of encoded data, where P×P quantized DCT coefficients in each data subset E are encoded into a set of encoded data.

[0311] It should be understood that JPEG compression primarily uses Huffman coding for entropy encoding. Huffman coding assigns a unique binary code to each input data. Frequently occurring symbols are assigned short codes, while infrequently occurring symbols are assigned long codes, thereby reducing the overall encoding length. This application embodiment does not limit the implementation method of the encoding process in step S47.

[0312] It should also be understood that a data subset E contains P×P quantized DCT coefficients. After Huffman coding, these P×P quantized DCT coefficients can be encoded into a set of coded data. For example, a set of coded data could be the string "91CF FEA57F D1 BF CF FA45". Therefore, an electronic device can obtain G sets of coded data by encoding G data subsets E.

[0313] For example, please refer to Figure 13 , Figure 13 This is a schematic diagram of an encoding process provided in an embodiment of this application. For example... Figure 13 As shown, the electronic device encodes G = 24 data subsets E, where each of the 8×8 quantized DCT coefficients in each data subset E is encoded into a set of encoded data, resulting in a total of 24 sets of encoded data.

[0314] It should be noted that the sequence numbers of steps S41 to S47 do not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. For example, step S42 can be executed after S44 and before S45.

[0315] It should also be noted that the steps S41 to S47 described above are merely a simple description of the overall compression coding process involved in the embodiments of this application, and should not constitute any limitation on the overall compression coding process involved in the embodiments of this application. In other embodiments, more or fewer steps than those described in S41 to S47 may be included. For example, a zig-zag scan step may be included after S46 and before S47, which is used to rearrange the quantized DCT coefficients to optimize coding efficiency.

[0316] The processing steps S41 to S47 described above are essentially data processing steps of an electronic device. This data processing process can be simply summarized as follows: for example, the CPU in an electronic device loads data from memory into a register, then the CPU reads data from the register, and finally the CPU processes the read data. The following embodiments, in conjunction with the data processing process of an electronic device, will introduce the specific implementation of the color space conversion involved in step S43 above.

[0317] Color space conversion

[0318] For example, please refer to Figure 14 , Figure 14 This is a schematic diagram of color space conversion provided in an embodiment of this application. It illustrates the implementation method of color space conversion during JPEG compression encoding, including steps S431 to S438.

[0319] S431, the CPU determines the number K1 of data groups H1 that can be loaded by at least one two-dimensional register based on the bit width of data group H1 and the logical size of at least one two-dimensional register. Data group H1 consists of one data subset A1, one data subset B1 corresponding to one data subset A1, and one data subset C1 corresponding to one data subset A1.

[0320] It should be understood that a register is a high-speed storage device in electronic devices used to store instructions, data, and addresses. Registers are categorized into one-dimensional registers and two-dimensional registers. A one-dimensional register refers to a storage unit that can hold a single data bit or a group of data bits; it can also be called a vector register, etc. A two-dimensional register refers to a register that can store a two-dimensional array. Two-dimensional registers provide electronic devices with multi-dimensional data storage and access capabilities. A two-dimensional register can be viewed as a storage unit composed of multiple one-dimensional registers; it can also be called a two-dimensional matrix register, matrix register, array register, etc.

[0321] The logical size of a two-dimensional register refers to its depth and width. The depth refers to the number of rows in a two-dimensional array that the register can store. For example, please refer to... Figure 15 , Figure 15 This is a schematic diagram of a two-dimensional register provided in an embodiment of this application. Figure 15 In this context, a two-dimensional register consists of 16 one-dimensional registers. Figure 15 If each row in the array represents a two-dimensional register, then the depth of the two-dimensional register is 16, meaning that the two-dimensional register can store 16 rows of data.

[0322] Here, width refers to the bit width of each one-dimensional register. For example, Figure 15The 16 one-dimensional registers shown each have a bit width of 128 bits, therefore the two-dimensional register also has a bit width of 128 bits. Bit width refers to the amount of data an electronic device can transmit at one time; simply put, it is the width of the data that an electronic device can transmit at one time.

[0323] The width can also be understood as the number of columns of two-dimensional data that a two-dimensional register can store. For example, assuming that the bit width of a data item in a two-dimensional array is 8 bits, then... Figure 15 The illustrated two-dimensional register can store (128 / 8 bits = 16) data points from a two-dimensional array in each row, and the register can store 16 columns of data. Alternatively, assuming that the bit width of a data point in a two-dimensional array is 16 bits, then the two-dimensional register can store (128 / 16 bits = 8) data points from a two-dimensional array in each row, and the register can store 8 columns of data.

[0324] It should also be understood that, combined with the above formula one and Figure 7 It can be seen that electronic devices need to convert color spaces when performing color space conversion. Figure 7 Substituting the 8×8 R components in the first row and first column of the 4×4 data subset A1, the 8×8 G components in the first row and first column of the 4×4 data subset B1, and the 8×8 B components in the first row and first column of the 4×4 data subset C1 into Formula 1 above, we can obtain... Figure 7 (b) shows the 8×8 R components in the first row and first column of the 4×4 data subset A2, the 8×8 G components in the first row and first column of the 4×4 data subset B2, and the 8×8 B components in the first row and first column of the 4×4 data subset C2. Data subsets A1, B1, and C3 correspond one-to-one. Therefore, when performing color space conversion, the CPU needs to read data subset A1, its corresponding data subset B1, and its corresponding data subset C1 from the two-dimensional register before performing the color space conversion. For ease of description, in this embodiment, a data subset A1, its corresponding data subset B1, and its corresponding data subset C1 are referred to as a data group H1.

[0325] Similarly, the bit width of data group H1 refers to the sum of the total bit width of the P×P color components 11 in data subset A1, the total bit width of the P×P color components 12 in data subset B1, and the total bit width of the P×P color components 13 in data subset C1. For example, assuming that the bit width of one color component 11 (R component) is 8 bits, the bit width of one color component 12 (G component) is 8 bits, the bit width of one color component 13 (B component) is 8 bits, and P = 8, then the total bit width of 8×8 R components is 8×8×8 bits, the total bit width of 8×8 G components is 8×8×8 bits, and the total bit width of 8×8 B components is 8×8×8 bits. Therefore, the bit width of data group H1 is 8×8×8 bits + 8×8×8 bits + 8×8×8 bits. In implementation, for example, in step S41 above, the electronic device can transmit M×N R components, M×N B components, and M×N G components to the CPU through the image signal processor so that the CPU can obtain the dataset corresponding to the original image. During the process of the image signal processor transmitting M×N R components, M×N B components, and M×N G components to the CPU, the image signal sensor can also transmit the bit width of each R component (color component 11), the bit width of each G component (color component 12), and the bit width of each B component (color component 13) to the CPU. In this way, in step S431, the CPU can calculate the bit width of the data group H1 based on the bit width of each color component 11, the bit width of each G color component 12, and the bit width of each B color component 13.

[0326] Similarly, it's understandable that due to the large amount of image data and the limited logical size of the two-dimensional register, it might not be possible to load all the image data at once. In other words, the CPU cannot read all the image data from the two-dimensional register at once for color space conversion. Based on the above, this embodiment first determines the number K1 data groups H1 that can be loaded into the two-dimensional register at one time, based on the logical size of the two-dimensional register and the bit width of the data group H1. Then, it performs color space conversion on all the image data through a loop process. In each loop, the CPU can load K1 data groups H1 from memory into the two-dimensional register, then read K1 data groups H1 from the two-dimensional register, and finally perform color space conversion on the read K1 data groups H1.

[0327] In this embodiment of the application, the CPU can determine the number K1 of data groups H1 that can be loaded into at least one two-dimensional register based on the following method.

[0328] Method 1

[0329] In Method 1, at least one two-dimensional register includes a two-dimensional register, and the logical size of the two-dimensional register remains unchanged, that is, the depth and width of the two-dimensional register do not change.

[0330] In implementation, the CPU can determine the number K1 of data groups H1 that can be loaded into the two-dimensional register using the following formula:

[0331]

[0332] For example, assuming the logic size of the two-dimensional register is 48×128bit=6144bit, and the bit width of data group H1 is 8×8×8bit+8×8×8bit+8×8×8bit=1536bit, then the quantity K1 obtained based on Formula 2 is 6144 / 1536=4.

[0333] In some embodiments, the quantity K1 obtained by the CPU through Formula 2 is a non-integer. The CPU also needs to perform floor rounding on the non-integer, that is, regardless of the fractional part, the fractional part is directly discarded and only the integer part is retained.

[0334] For example, assuming the logic size of the two-dimensional register is 16×128bit=2048bit, and the bit width of data group H1 is 8×8×8bit+8×8×8bit+8×8×8bit=1536bit, then the quantity K1 obtained based on Formula 2 is 2048 / 1536=1.33. After rounding, K1=1.

[0335] Method 2

[0336] In Method 2, at least one two-dimensional register includes multiple two-dimensional registers, and the logical size of each two-dimensional register remains unchanged.

[0337] In Method 2, when the CPU determines the number K1 of data groups H1 that can be loaded from multiple two-dimensional registers, there are two cases. Case 1 is to load one color component 2 for each two-dimensional register. For example, the multiple two-dimensional registers include three two-dimensional registers: one for loading the R component, one for loading the G component, and one for loading the B component.

[0338] Scenario 2 involves loading multiple color components 2 for each two-dimensional register. For example, multiple two-dimensional registers include three two-dimensional registers, and each of these three two-dimensional registers can be loaded with R component, G component and B component.

[0339] In scenario one, the CPU can determine the number K1 of data sets H1 that can be loaded from multiple two-dimensional registers using the following formula three:

[0340]

[0341] For example, suppose multiple two-dimensional registers include three two-dimensional registers, one for loading the R component, one for loading the G component, and one for loading the B component. The logic size of these three two-dimensional registers is 16×128bit=2048bit, and the bit width of data group H1 is 8×8×8bit+8×8×8bit+8×8×8bit=1536bit. Then, based on formula 3, the quantity K1=(2048*3) / 1536=4.

[0342] It should be noted that in some embodiments, the quantity K1 obtained by the CPU through Formula 3 is not an integer. The CPU also needs to perform floor rounding on the non-integer, that is, regardless of the decimal part, the decimal part is directly discarded and only the integer part is retained.

[0343] In scenario two, the CPU can calculate the number of data groups H1 that a single two-dimensional register can load based on Formula 2 above, and then multiply this number by the number of two-dimensional registers to obtain the number K1 of data groups H1 that multiple two-dimensional registers can load. For example, if multiple two-dimensional registers include three two-dimensional registers, and the number of data groups H1 that a single two-dimensional register can load, obtained through Formula 2, is 1, then the number K1 of data groups H1 that multiple two-dimensional registers can load is 3.

[0344] Method 3

[0345] In method three, at least one two-dimensional register includes a two-dimensional register, and the logical size of the two-dimensional register is variable, meaning that the width and / or depth of the two-dimensional register can change. For example, the original two-dimensional register is... Figure 15The diagram shows 16 one-dimensional registers, each with a bit width of 128 bits. Therefore, the original two-dimensional register's logical size is 16 × 128 bits = 2048 bits. Assuming the bit width of each one-dimensional register remains unchanged, but the number of one-dimensional registers increases from 16 to 24, the depth of the two-dimensional register changes, and the logical size of the two-dimensional register after the change is 24 × 128 bits = 3072 bits. Assuming the bit width of each one-dimensional register changes from 128 bits to 256 bits, and the number of one-dimensional registers remains unchanged, the width of the two-dimensional register changes, and the logical size of the two-dimensional register after the change is 16 × 256 bits = 4096 bits. Assuming the bit width of each one-dimensional register changes from 128 bits to 256 bits, and the number of one-dimensional registers changes from 16 to 24, both the depth and width of the two-dimensional register change, and the logical size of the two-dimensional register after the change is 24 × 256 bits = 6144 bits. Among them, two-dimensional registers with variable logical dimensions can also be called variable two-dimensional registers, variable rectangular registers, variable array registers, expandable two-dimensional registers, scalable two-dimensional registers, etc. One-dimensional registers with variable bit widths can also be called variable vector registers, expandable vector registers, scalable vector registers, etc.

[0346] It should be understood that the logical size of the two-dimensional register can be varied based on the total image data (for example, the total image data can be obtained from (M / P)×(N / P) data subsets A1, B1, and C1 obtained by S42). For instance, when the total image data is large, the logical size of the two-dimensional register can be changed by increasing the width and / or depth of the original register. When the total image data is small, the logical size can be changed by decreasing the width and / or depth of the original register. This adapts to the amount of image data, allowing electronic devices to flexibly change the logical size of the two-dimensional register based on the amount of image data, enabling the electronic device to flexibly process images with different data volumes.

[0347] In implementation, when the logic size of the variable two-dimensional register changes to a fixed logic size, the CPU can determine the number K1 of data groups H1 that the variable two-dimensional register can load using Formula 2 above. For example, assuming the logic size of the variable two-dimensional register is 24 × 128 bits = 3072 bits, and the bit width of data group H1 is 8 × 8 × 8 bits + 8 × 8 × 8 bits + 8 × 8 × 8 bits = 1536 bits, then the number K1 obtained based on Formula 2 is 3072 / 1536 = 2.

[0348] Method 4

[0349] In method four, at least one two-dimensional register comprises multiple two-dimensional registers, and the logical size of each of the multiple two-dimensional registers is variable. For example, the original logical size of the three two-dimensional registers is... Figure 15 The 16 rows × 16 columns shown are 16 × 128 bits = 2048 bits. The logic size of the three two-dimensional registers after the changes is 24 rows × 16 columns, which is 24 × 128 bits = 3072 bits.

[0350] In Method 4, when the CPU determines the number K1 of data groups H1 that can be loaded by multiple variable two-dimensional registers, it can also be divided into Case 1 and Case 2 as described above.

[0351] In scenario one, when the logic size of the variable two-dimensional registers changes to a fixed logic size, the CPU can determine the number K1 of data groups H1 that can be loaded by the multiple variable two-dimensional registers using Formula 3 above. For example, assuming that the logic size of the three variable two-dimensional registers is 24 × 128 bits = 3072 bits, and the bit width of data group H1 is 8 × 8 × 8 bits + 8 × 8 × 8 bits + 8 × 8 × 8 bits = 1536 bits, then the number K1 obtained based on Formula 3 is (3072 * 3) / 1536 = 6.

[0352] In case two, when the logic size of the variable two-dimensional register is changed to a fixed logic size, the CPU can calculate the number of data groups H1 that a single variable two-dimensional register can load using the method described above in method three. Then, multiplying this number by the number of variable two-dimensional registers, the CPU can obtain the number K1 of data groups H1 that multiple variable two-dimensional registers can load.

[0353] S432, the CPU determines the total number of iterations of loop processing 1 based on the number of data groups H1 K1 and (M / P)×(N / P) data subsets A1, (M / P)×(N / P) data subsets B1 and (M / P)×(N / P) data subsets C1. Each iteration of loop processing 1 is used to perform color space conversion on data subsets A1, B1 and C1 in K1 data groups H1.

[0354] As can be seen from the above embodiments, this application embodiment requires color space conversion of all image data through loop processing 1. Before loop processing, the CPU can determine the total number of loop processing 1 through step S432.

[0355] In implementation, the CPU can determine the total number of times loop 1 is processed using the following formula:

[0356]

[0357] Where (M / P)×(N / P) refers to the number of data subsets A1, B1, or C1, and (M / P)×(N / P) can also be understood as the total number of data groups H1 in all image data. All image data includes (M / P)×(N / P) data subsets A1, (M / P)×(N / P) data subsets B1, and (M / P)×(N / P) data subsets C1. K1 refers to the number of data groups H1 that can be loaded by at least one two-dimensional register.

[0358] For example, suppose all image data refers to Figure 7 In (a) shown, there are 4×4 data subsets A1, B1, and C1. Therefore, the total image data includes 4×4 = 16 data groups H1. Assuming that at least one two-dimensional register can load 4 data groups H1, the number K1 is 4. Based on Formula 4 above, the total number of loops is 16 / 4 = 4.

[0359] It should be noted that in some embodiments, the total number of loop iterations obtained by the CPU using Formula 4 is not an integer. The CPU also needs to round up the non-integer, that is, regardless of the decimal part, the integer part is incremented by one. For example, suppose that all the image data includes 4 × 4 = 16 data groups H1. Assuming the quantity K1 is 3, then based on Formula 4 above, the total number of loop iterations is 16 / 3 = 5.3, and after rounding up, the total number of loop iterations is 6.

[0360] It should be understood that after the CPU determines the total number of iterations of loop processing 1 through step S432, it begins loop processing and can perform the operations required for each loop through the following steps S433 to S435.

[0361] S433: The CPU loads K1 data sets H1 from memory in parallel into at least one two-dimensional register.

[0362] It should be understood that after the CPU in the electronic device executes the above step S42 and obtains (M / P)×(N / P) data subsets A1, (M / P)×(N / P) data subsets B1, and (M / P)×(N / P) data subsets C1, the CPU usually writes (M / P)×(N / P) data subsets A1, (M / P)×(N / P) data subsets B1, and (M / P)×(N / P) data subsets C1 into memory for storage in row-major order.

[0363] For example, please refer to Figure 16 , Figure 16This is a schematic diagram of writing a data subset A1 into memory, provided in an embodiment of this application. Figure 16 The data subset A1-11 shown in the figure can refer to Figure 7 (a) shows one of the (32 / 8) × (32 / 8) data subsets A1.

[0364] For ease of description, Figure 7 The (32 / 8) × (32 / 8) data subsets A1 shown in (a) can be considered as data subsets A1 arranged in 4 rows and 4 columns, with each data subset A1 having 8 rows and 8 columns and a total of 64 R components. Figure 16 The diagram shows that A1-11 represents Figure 7 (a) shows the data subset A1 in the first row and first column of the 4-row, 4-column subset A1. A1-11(R11) refers to the first row and first column of the 8-row, 8-column R component included in the data subset A1-11, and A1-11(R12) refers to the first row and second column of the 8-row, 8-column R component included in the data subset A1-11. The meanings of other identifiers (such as A1-11(R48), A1-11(R88)) can be found in A1-11(R11) and A1-11(R12).

[0365] The CPU writing the data subset A1-11 into memory in row-major order means, for example... Figure 16 As shown, the CPU first writes A1-11(R11) to A1-11(R18), which are included in the first row of the data subset A1-11, into memory. Then, it writes A1-11(R21) to A1-11(R28), which are included in the second row, into memory. In this way, all 8 rows and 8 columns of the R components in the data subset A1-11 are written into memory. That is to say, each row of data in the data subset A1-11 is stored contiguously.

[0366] The process by which the CPU writes (M / P)×(N / P) data subsets A1 into memory in row-major order can be found in [reference]. Figure 17 . Figure 17 This refers to a schematic diagram provided in an embodiment of this application, showing how to write multiple data subsets A1 into memory. Figure 17 The process of writing to memory shown can be understood as writing... Figure 7 Figure (a) illustrates the process of writing (32 / 8) × (32 / 8) data subsets A1 into memory. Similarly, for ease of description, Figure 17 The (32 / 8)×(32 / 8) data subsets A1 shown can be considered as data subsets A1 arranged in 4 rows and 4 columns, with each data subset A1 having 8 rows and 8 columns and a total of 64 R components. Figure 17In this context, A1-11 represents the data subset A1 in the first row and first column of a 4-row, 4-column subset A1, and A1-12 represents the data subset A1 in the first row and second column of a 4-row, 4-column subset A1. The meanings of other identifiers (such as A1-32 and A1-44) can be found in A1-11 and A1-12. A1-11(R11) refers to the first row and first column of the 8-row, 8-column R component included in the data subset A1-11, and A1-12(R11) refers to the first row and first column of the 8-row, 8-column R component included in the data subset A1-12. The meanings of other identifiers (such as A1-32(R12) and A1-44(R12)) can be found in A1-11(R11) and A1-12(R11).

[0367] Figure 17 In this process, the CPU first writes the data subsets A1-11(R11) to A1-11(R88), A1-12(R11) to A2-11(R88), A1-13(R11) to A2-11(R88), and A1-14(R11) to A2-11(R88) from the first row of the (32 / 8)×(32 / 8) data subset A1 into memory. Then, in the same manner, the CPU sequentially writes the data subsets A from the second, third, and fourth rows into memory. The process of the CPU writing each data subset A into memory can be found in [reference needed]. Figure 16 The writing process is illustrated. That is, given multiple data subsets with multiple rows and columns, the CPU can write and store each data subset in memory in row-major order, and each row of data within each subset is stored contiguously in memory. The method of writing and storing multiple data subsets in memory as described in the following embodiments can be found in [reference needed]. Figure 16 and Figure 17 The method will not be elaborated further.

[0368] It should be understood that the CPU, through Figure 16 and Figure 17 The writing process shown writes (M / P)×(N / P) data subsets A1, (M / P)×(N / P) data subsets B1, and (M / P)×(N / P) data subsets C1 into memory. After the CPU executes the above steps S431 and S432, it can load K1 data sets H1 from memory into at least one two-dimensional register in parallel.

[0369] It should also be understood that each piece of data stored in memory has a unique memory address, and the CPU can access that data in memory based on its memory address. For example, please refer to... Figure 17The memory stores each data in (32 / 8)×(32 / 8) data subsets A1. Each data subset A1 includes 8×8 data, meaning that a total of 1024 data are stored in the memory. The CPU can access the data A1-31(R11) located in the first row and first column of the data subset A1 in the (32 / 8)×(32 / 8) data subsets A1 based on the memory address A1-31(R11).

[0370] The process of the CPU loading data into the two-dimensional register can be as follows: the CPU accesses the data in memory through the memory address of the data, then reads the data and loads it into the two-dimensional register.

[0371] The method by which the CPU loads a subset of data A1 from memory in parallel into the two-dimensional register can be found in [reference]. Figure 18 . Figure 18 This is a schematic diagram illustrating the parallel loading of a data subset A according to an embodiment of this application. For example, Figure 18 Data stored in memory and Figure 16 The data stored in them is the same. Figure 18 The memory stores 64 R components from the data subset A1-11, each R component having a bit width of 8 bits. Figure 18 The logic dimensions of the two-dimensional register can be referenced. Figure 15 For example, a two-dimensional register has a depth of 16 rows and a width of 128 bits, with each row capable of loading 128 / 8 bits = 16 R components. Figure 18 As shown, when the CPU loads the 64 R components A1-11(R11) to A1-11(R88) stored in memory, it can access the memory through the memory addresses of these 64 R components, then read these 64 R components from memory, and then load these 64 R components into the two-dimensional register.

[0372] because Figure 18 The illustrated two-dimensional register can load 16 R components per row. The CPU can load the 16 components A1-11 (R11) to A1-11 (R28) in parallel into the first row of the two-dimensional register, load the 16 R components A1-11 (R31) to A1-11 (R48) in parallel into the second row, load the 16 components A1-11 (R51) to A1-11 (R68) in parallel into the third row, and load the 16 components A1-11 (R71) to A1-11 (R88) in parallel into the fourth row. These four loading operations are performed in parallel. In the following embodiments, the method for loading multiple data subsets in parallel can be referred to... Figure 18 The method shown.

[0373] It is understandable that at least one two-dimensional register can include one or more two-dimensional registers, and the logical size of the two-dimensional register can be variable or immutable. Based on the different numbers and variability of the logical size of the two-dimensional registers, the CPU will load K1 data sets H1 from memory into the two-dimensional registers in parallel in different ways. The different loading methods are described below through methods five through eight.

[0374] Method 5

[0375] In Method 5, at least one two-dimensional register includes a two-dimensional register, and the logical size of the two-dimensional register remains unchanged.

[0376] In implementation, the CPU loads the K1 data subsets A1, B1, and C1 included in the K1 data sets H1 into a two-dimensional register in parallel.

[0377] For example, please refer to Figure 19 , Figure 19 This is a schematic diagram illustrating the loading of data from memory into a two-dimensional register, as provided in an embodiment of this application. Figure 19 The memory shown can store Figure 7 (a) shows (32 / 8) × (32 / 8) data subsets A1, (32 / 8) × (32 / 8) data subsets B1, and (32 / 8) × (32 / 8) data subsets C1, which means... Figure 19 The memory shown stores (32 / 8) × (32 / 8) = 16 data subsets A1, totaling 16 × 8 × 8 = 1024 R components, as well as 1024 G components and 1024 B components. The bit width of each R component, each G component, and each B component is 8 bits. The meaning of the identifiers B1-11 (G11) and C1-11 (B11) can be found in the above embodiment.

[0378] Figure 19 The logical dimensions of the two-dimensional register shown in the figure are... Figure 15 The illustrated two-dimensional registers all have the same logical size, and this logical size is immutable. For example, a two-dimensional register has a depth of 16 rows and a width of 128 bits. Each row can load 128 / 8 bits = 16 R components, or 16 G components, or 16 B components. The CPU is based on... Figure 19The logical size of the shown two-dimensional register and the number of data groups H1 that can be loaded into the two-dimensional register, determined by method one in step S431 above, are K1 = (16 * 128 bits) / (8 × 8 × 8 bits + 8 × 8 × 8 bits + 8 × 8 × 8 bits) = 1. The total number of times the CPU performs loop processing 1, determined by step S432 above, is ((M / P) * (N / P)) / K1 = ((32 / 8) * (32 / 8)) / 1 = 16 times.

[0379] The process of the CPU performing each loop processing can include, for example, when the CPU performs the first loop processing 1, a data group H1 includes... Figure 19 The data subsets shown in memory include A1-11 (R11) to A1-11 (R88), totaling 8×8=64 R components; B1-11 (G11) to B1-11 (G88), totaling 8×8=64 R components; and C1-11 (B11) to C1-11 (B88), totaling 8×8=64 B components. The CPU loads A1-11 (R11) to A1-11 (R88), B1-11 (G11) to B1-11 (G88), and C1-11 (B11) to C1-11 (B88) in parallel into a two-dimensional register. For example, the CPU loads A1-11 (R11) to A1-11 (R88) in parallel into rows 1 to 4 of the two-dimensional register; the CPU loads B1-11 (G11) to B1-11 (G88) in parallel into rows 5 to 8 of the two-dimensional register; and the CPU loads C1-11 (B11) to C1-11 (B88) in parallel into rows 9 to 12 of the two-dimensional register. These 12 loading operations are performed in parallel. The method by which the CPU loads data from memory into the two-dimensional register during other loop processing can be referenced. Figure 19 The loading method is shown.

[0380] Method Six

[0381] In Method 6, at least one two-dimensional register includes multiple two-dimensional registers, and the logical size of each of the multiple two-dimensional registers remains unchanged.

[0382] In method six, when the CPU loads K1 data sets H1 from memory in parallel into the two-dimensional registers, there are two scenarios. Scenario one involves loading one color component 2 for each two-dimensional register. For example, multiple two-dimensional registers may include three registers: one for loading the R component, one for loading the G component, and one for loading the B component. Scenario two involves loading multiple color components 2 for each two-dimensional register. For example, multiple two-dimensional registers may include three registers, where each register can load the R, G, and B components.

[0383] In scenario one, the CPU can load K1 data subsets A1, B1, and C1 from K1 data sets H1 into multiple two-dimensional registers in parallel. These two-dimensional registers include two-dimensional register 1, two-dimensional register 2, and two-dimensional register 3. The CPU loads K1 data subsets A1 into two-dimensional register 1 in parallel, K1 data subsets B1 into two-dimensional register 2 in parallel, and K1 data subsets C1 into two-dimensional register 3 in parallel.

[0384] For example, please refer to Figure 20 , Figure 20 This is another schematic diagram of loading data from memory into a two-dimensional register provided in an embodiment of this application. Figure 20 The data stored in memory shown is... Figure 19 The data stored in memory is the same, so we will not go into details here. Figure 20 The diagram shows three two-dimensional registers: Register 1, Register 2, and Register 3. These three registers have the same logical size: a depth of four rows and a width of 128 bits. Each row can load 128 / 8 bits = 16 R components, or 16 G components, or 16 B components. The CPU is based on... Figure 20 Based on the logical dimensions of the shown two-dimensional registers and Case 1 of Method 2 in step S431 above, the number of data groups H1 that can be loaded by the three two-dimensional registers is determined to be K1 = ((4*128bit) + (4*128bit) + (4*128bit)) / (8×8×8bit + 8×8×8bit + 8×8×8bit) = 1. The total number of times the CPU performs loop processing 1 based on step S432 above is determined to be ((M / P)*(N / P)) / K1 = ((32 / 8)*(32 / 8)) / 1 = 16 times.

[0385] Each time the CPU performs loop processing 1, it can include: for example, when the CPU performs the first loop processing 1, a data set H1 includes... Figure 20The data subsets shown in the memory include A1-11 (R11) to A1-11 (R88), which contain a total of 8×8=64 R components; B1-11 (G11) to B1-11 (G88), which contain a total of 8×8=64 R components; and C1-11 (B11) to C1-11 (B88), which contain a total of 8×8=64 B components.

[0386] The CPU loads A1-11 (R11) to A1-11 (R88), B1-11 (G11) to B1-11 (G88), and C1-11 (B11) to C1-11 (B88) in parallel into three two-dimensional registers. For example, the CPU loads A1-11 (R11) to A1-11 (R88) in parallel into rows 1 to 4 of two-dimensional register 1, loads B1-11 (G11) to B1-11 (G88) in parallel into rows 1 to 4 of two-dimensional register 2, and loads C1-11 (B11) to C1-11 (B88) in parallel into rows 1 to 4 of two-dimensional register 3.

[0387] In scenario two, the CPU can load K1 data subsets A1, B1, and C1 from K1 data sets H1 into multiple two-dimensional registers in parallel. These registers include two-dimensional register 1, two-dimensional register 2, and two-dimensional register 3. The CPU loads K1 / 3 data subsets A1, B1, and C1 into two-dimensional register 1 in parallel; loads them into two-dimensional register 2 in parallel; and loads them into two-dimensional register 3 in parallel. K1 / 3 is a positive integer.

[0388] For example, please refer to Figure 21 , Figure 21 This is another schematic diagram of loading data from memory into a two-dimensional register provided in an embodiment of this application. Figure 21 The data stored in memory shown is... Figure 19 The data stored in memory is the same, so we will not go into details here. Figure 21 The diagram shows three two-dimensional registers: Register 1, Register 2, and Register 3. These three registers have the same logical size: 12 rows deep and 128 bits wide. Each row can load 128 / 8 bits = 16 R components, or 16 G components, or 16 B components. The CPU is based on... Figure 21The logical size of the two-dimensional register shown is the same as case two in method two of step S431 above. That is, after calculating the number of data groups H1 that a single two-dimensional register can load based on formula two above, multiplying it by the number of two-dimensional registers, we can get the number of data groups H1 that multiple two-dimensional registers can load: K1 = ((12*128bit) / (8×8×8bit+8×8×8bit+8×8×8bit))*3 = 3. The total number of times the CPU performs loop processing 1 based on step S432 above is ((M / P)*(N / P)) / K1 = ((32 / 8)*(32 / 8)) / 3 = 6 times.

[0389] Each time the CPU performs loop processing 1, it can include, for example, when the CPU performs the first loop processing 1, one of the three data groups H1 includes: Figure 21 The memory shown includes A1-11(R11) to A1-11(R88), B1-11(G11) to B1-11(G88), and C1-11(B11) to C1-11(B88). A data group H1 includes... Figure 21 The memory shown includes A1-12 (R11) to A1-12 (R88), B1-12 (G11) to B1-12 (G88), and C1-12 (B11) to C1-12 (B88). A data group H1 includes... Figure 21 The memory shown includes A1-13(R11) to A1-13(R88), B1-13(G11) to B1-13(G88), and C1-13(B11) to C1-13(B88).

[0390] The CPU can load data from these three data sets into three two-dimensional registers in parallel. For example, Figure 21The CPU shown loads A1-11 (R11) to A1-11 (R88) in parallel into rows 1 to 4 of two-dimensional register 1, loads A1-12 (R11) to A1-12 (R88) in parallel into rows 5 to 8 of two-dimensional register 1, and loads A1-13 (R11) to A1-13 (R88) in parallel into rows 9 to 12 of two-dimensional register 1. The CPU loads B1-11 (G11) to B1-11 (G88) in parallel into rows 1 to 4 of two-dimensional register 2, loads B1-12 (G11) to B1-12 (G88) in parallel into rows 5 to 8 of two-dimensional register 2, and loads B1-13 (G11) to B1-13 (G88) in parallel into rows 9 to 12 of two-dimensional register 2. The CPU loads C1-11 (B11) to C1-11 (B88) in parallel into rows 1 to 4 of two-dimensional register 3, loads C1-12 (B11) to C1-12 (B88) in parallel into rows 5 to 8 of two-dimensional register 3, and loads C1-13 (B11) to C1-13 (B88) in parallel into rows 9 to 12 of two-dimensional register 3.

[0391] Method Seven

[0392] In mode seven, at least one two-dimensional register includes a variable two-dimensional register, and the logical size of the two-dimensional register is variable.

[0393] In implementation, when the logic size of the variable two-dimensional register is changed to a fixed logic size, the CPU loads the K1 data subsets A1, B1, and C1 included in the K1 data groups H1 into the variable two-dimensional register in parallel.

[0394] For example, please refer to Figure 22 , Figure 22 This is a schematic diagram illustrating another method of loading data from memory into a variable two-dimensional register in parallel, as provided in an embodiment of this application. Figure 22 Data stored in memory and Figure 21 The data stored in memory is the same, which will not be elaborated further here. Assuming... Figure 22 The variable two-dimensional register shown originally had a two-dimensional logic size of 16×128 bits. After the logic size of the variable two-dimensional register was changed, it became... Figure 22 The figure shown is 24×128bit. The CPU is based on... Figure 22The changed logic size of the variable two-dimensional register shown, and the number of data groups H1 that can be loaded in parallel by the variable two-dimensional register determined by method three in step S431 above, are K1 = (24 * 128 bits) / (8 × 8 × 8 bits + 8 × 8 × 8 bits + 8 × 8 × 8 bits) = 2. The total number of times the CPU performs loop processing 1 based on step S432 above is ((M / P) * (N / P)) / K1 = ((32 / 8) * (32 / 8)) / 2 = 8 times.

[0395] Each time the CPU performs loop processing 1, it can include, for example, when the CPU performs the first loop processing 1, one of the two data groups H1 includes: Figure 22 The memory shown includes A1-11(R11) to A1-11(R88), B1-11(G11) to B1-11(G88), and C1-11(B11) to C1-11(B88). Another data group H1 includes... Figure 22 The memory shown includes A1-12(R11) to A1-12(R88), B1-12(G11) to B1-12(G88), and C1-12(B11) to C1-12(B88).

[0396] The CPU can load the data from these two data sets H1 in parallel into a variable two-dimensional register. For example, the CPU loads A1-11 (R11) to A1-11 (R88) and A1-12 (R11) to A1-12 (R88) in parallel into rows 1 to 8 of the variable two-dimensional register. It loads B1-11 (R11) to B1-11 (R88) and B1-12 (R11) to B1-12 (R88) in parallel into rows 9 to 16 of the variable two-dimensional register. It loads C1-11 (R11) to C1-11 (R88) and C1-12 (R11) to C1-12 (R88) in parallel into rows 17 to 24 of the variable two-dimensional register.

[0397] Method Eight

[0398] In mode eight, at least one two-dimensional register comprises multiple two-dimensional registers, and the logical size of each two-dimensional register is variable.

[0399] In method eight, when the CPU loads K1 data sets H1 from memory in parallel into the variable two-dimensional registers, there are two cases. Case one involves loading one color component 2 in parallel for each variable two-dimensional register. For example, multiple variable two-dimensional registers may include three variable two-dimensional registers: one for loading the R component in parallel, one for loading the G component in parallel, and one for loading the B component in parallel. Case two involves loading multiple color components 2 in parallel for each variable two-dimensional register. For example, multiple variable two-dimensional registers may include three variable two-dimensional registers, where each of these three variable two-dimensional registers can load the R, G, and B components in parallel.

[0400] In scenario one, the CPU can load K1 data subsets A1, B1, and C1 from K1 data sets H1 into multiple variable two-dimensional registers in parallel. These variable two-dimensional registers include variable two-dimensional register 1, variable two-dimensional register 2, and variable two-dimensional register 3. The CPU loads K1 data subsets A1 into variable two-dimensional register 1 in parallel, K1 data subsets B1 into variable two-dimensional register 2 in parallel, and K1 data subsets C1 into variable two-dimensional register 3 in parallel.

[0401] For example, please refer to Figure 20 Assuming Figure 20 The original logic size of the variable two-dimensional registers 1, 2, and 3 shown was 6 × 128 bits, or 6 rows and 16 columns. After the logic size of the variable two-dimensional registers 1, 2, and 3 was changed, it became... Figure 20 The variable two-dimensional register shown, i.e., the logic size of variable two-dimensional registers 1, 2, and 3 after changes is 4 × 128 bits, that is, 4 rows and 16 columns, then the CPU can... Figure 20 The parallel loading method shown loads K1 data subsets A1, B1 and C1 from K1 data groups H1 into variable two-dimensional registers 1, 2 and 3 in parallel, which will not be described in detail here.

[0402] In scenario two, the CPU can load K1 data subsets A1, B1, and C1 from K1 data sets H1 in parallel into multiple variable two-dimensional registers. These variable two-dimensional registers include variable two-dimensional register 1, variable two-dimensional register 2, and variable two-dimensional register 3. The CPU loads K1 / 3 data subsets A1, B1, and C1 in parallel into variable two-dimensional register 1; loads them in parallel into variable two-dimensional register 2; and loads them in parallel into variable two-dimensional register 3. K1 / 3 is an integer greater than 0.

[0403] For example, please refer to Figure 21 Assuming Figure 21 The original logic size of the variable two-dimensional registers 1, 2, and 3 shown was 6 × 128 bits, or 6 rows and 16 columns. After the logic size of the variable two-dimensional registers 1, 2, and 3 was changed, it became... Figure 21 The variable two-dimensional register shown, i.e., the logic size of variable two-dimensional registers 1, 2, and 3 after changes, is 12 × 128 bits, that is, 12 rows and 16 columns. Then the CPU can... Figure 21 The parallel loading method shown loads K1 data subsets A1, B1 and C1 from K1 data groups H1 into variable two-dimensional registers 1, 2 and 3 in parallel, which will not be described in detail here.

[0404] It should also be understood that in step S433 above, the CPU can access memory according to the memory address of each data in the K1 data groups H1 through Single Instruction Multiple Data (SIMD) operation, and load the K1 data groups H1 from memory into at least one two-dimensional register in parallel.

[0405] SIMD is a computer processing technique that allows a single CPU instruction to operate on multiple data elements simultaneously. This technique can significantly improve performance when processing large amounts of data because multiple data elements can be processed in parallel without requiring multiple independent instructions. For example, SIMD supports accessing memory once based on the memory addresses of multiple data elements, loading multiple data elements into registers in parallel, and reading multiple vectors or matrices from registers in parallel, where each vector or matrix contains multiple data elements; as well as processing multiple vectors or matrices in parallel, and processing multiple data elements within a vector or matrix in parallel. The processor in this application embodiment includes dedicated SIMD hardware, such as Advanced RISC Machine Scalable Vector Extension 2 (ARM SVE2) and Advanced RISC Machine Scalable Matrix Extension (ARM SME), which are specifically designed to perform SIMD operations. SIMD technology can be implemented using special instruction sets (e.g., ARM SVE2 instruction set, ARM SME instruction set) and registers (e.g., the two-dimensional registers provided in this application embodiment).

[0406] S434, the CPU reads K1 data groups H1 in parallel from at least one two-dimensional register.

[0407] Because the SMID operation supports parallel reading of multiple data sets, the CPU can use the SMID operation to read multiple data sets (K1) from at least one two-dimensional register in parallel, including data in data groups (H1). For an example, please refer to... Figure 23 , Figure 23 This is a schematic diagram of parallel data reading provided in an embodiment of this application. Figure 23 Therefore, step S433 is through Figure 19 The example illustrates how data is loaded from memory into a two-dimensional register in parallel.

[0408] Figure 19 In this context, the number of data sets H1, K1, is 1. The CPU loads one data set H1 from memory in parallel into rows 1 to 12 of the two-dimensional register. The CPU can then... Figure 19 The first row of the shown two-dimensional register was read. Figure 23 The values ​​A1-11(R11), ..., A1-11(R18), A1-11(R21), ..., A1-11(R28) shown are read from the second row of the two-dimensional register. Figure 23 The following are shown: A1-11(R11), ..., A1-11(R18), A1-11(R21), ..., A1-11(R28), CPU from... Figure 19 The data read from rows 3 to 12 of the shown two-dimensional register can be referenced. Figure 23 This will not be elaborated further here. Each row of data mentioned above was read in parallel.

[0409] It is understandable that the data read from lines 1 to 4 can be considered as the data in a subset A1 of a data group H1, the data read from lines 5 to 8 can be considered as the data in a subset B1 of a data group H1, and the data read from lines 9 to 12 can be considered as the data in a subset C1 of a data group H1.

[0410] It should be noted that Figure 23 Each piece of data shown is read simultaneously and in parallel from a two-dimensional register.

[0411] S435, the CPU performs parallel processing on K1 data groups H1, including K1 data subsets A1, K1 data subsets B1 and K1 data subsets C1, to obtain K1 data subsets A2, K1 data subsets B2 and K1 data subsets C2. Each data subset A2 includes P×P color components 21, each data subset B2 includes P×P color components 22, and each data subset C2 includes P×P color components 23.

[0412] Since the SMID operation supports parallel processing of multiple data sets, in its implementation, the CPU can substitute each R component from K1 data subsets A1, the G component corresponding to each R component from K1 data subsets B1, and the B component corresponding to each R component from K1 data subsets C1 into Formula 1 for parallel computation, resulting in K1 data subsets A2, B2, and C2. Each data subset A2 includes 8×8 Y components, each data subset B2 includes 8×8 Cb components, and each data subset C2 includes 8×8 Cr components.

[0413] It should be understood that the R component, the G component corresponding to the R component, and the B component corresponding to the R component refer to, for example, Figure 7In (a), the (32 / 8) × (32 / 8) data subsets A1 can be considered as 4-row, 4-column data subsets A1, each containing an 8-row, 8-column R component. Similarly, the (32 / 8) × (32 / 8) data subsets B1 can be considered as 4-row, 4-column data subsets B1, each containing an 8-row, 8-column G component. The (32 / 8) × (32 / 8) data subsets C1 can be considered as 4-row, 4-column data subsets C1, each containing an 8-row, 8-column B component. The R component in the first row, first column of subset A1 corresponds to the G component in the first row, first column of subset B1, and the B component in the first row, first column of subset C1. The R component in the 8th row and 1st column of the data subset A1 in the 4th row and 1st column corresponds to the G component in the 8th row and 1st column of the data subset B1 in the 4th row and 1st column, and corresponds to the B component in the 8th row and 1st column of the data subset C1 in the 4th row and 1st column. The correspondence of other RGB values ​​can be referred to the above description, and will not be repeated here.

[0414] Each R component in K1 data subsets A1, the G component corresponding to each R component in K1 data subsets B1, and the B component corresponding to each R component in K1 data subsets C1 refer to, for example, please refer to Figure 24 , Figure 24 This is a schematic diagram of a parallel data processing embodiment provided in this application. Figure 24 The data read by the CPU from at least one two-dimensional register and Figure 23 Same as in [the text]. Figure 24 In the middle, K1 = 1, Figure 24 In the diagram, the G component corresponding to A1-11(R11) is B1-11(G11), and the B component corresponding to A1-11(R11) is C1-11(B11). That is to say, Figure 24 Each matrix box shown contains an R component, a G component corresponding to the R component, and a B component corresponding to the R component.

[0415] It should also be understood that since the CPU can access data in memory based on its memory address and load that data into a two-dimensional register, when the CPU reads data from the two-dimensional register and processes it, the CPU has already determined which of the multiple data sets stored in memory that the data refers to. For example, when the CPU reads data from the two-dimensional register... Figure 24 After showing A1-11(R11), determine that A1-11(R11) refers to the memory stored in memory. Figure 7In Figure (a), the R component in the first row and first column of the (32 / 8) × (32 / 8) data subsets A1 is shown. Based on this, the CPU can substitute each R component in the K1 data subsets A1 read from the two-dimensional register, the G component corresponding to each R component in the K1 data subsets B1, and the B component corresponding to each R component in the K1 data subsets C1 into Formula 1 for parallel computation.

[0416] For example, please refer to Figure 24 In formula (a), the CPU can substitute the data in each rectangle into Y in formula one. i =0.299×R i +0.578×G i +0.114×B i We obtain a data subset A2, which includes 8×8=64 Y components from A2-11(Y11) to A2-11(Y88). Figure 24 The operations within the multiple rectangles shown in (a) are parallel operations.

[0417] CPU is executing Figure 24 While the operation is shown in (a), please refer to... Figure 24 In formula (b), the data in each rectangle can also be substituted into Cb in formula one in parallel. i = -0.1687 × R i -0.3313×G i +0.5×B i +128, resulting in a data subset B2, which includes 8×8=64 Cb components from B2-11(Cb11) to B2-11(Cb88). Figure 24 The operations within the multiple rectangles shown in (b) are parallel operations.

[0418] CPU is executing Figure 24 While the operation is shown in (a), please refer to... Figure 24 In formula (c), the data in each rectangle can also be substituted into Cr in formula one in parallel. i =0.5×R i -0.4187×G i -0.0813×B i +128, resulting in a data subset C2, which includes 8×8=64 Cr components from C2-11(Cr11) to C2-11(Cr88). Figure 24 The operations within the multiple rectangles shown in (c) are parallel operations.

[0419] It should be noted that the implementation of step S435 above is illustrated using the example of the CPU reading K1 data groups H1 from at least one two-dimensional register as a single data group H1. When the CPU reads K1 data groups H1 from the two-dimensional register as multiple data groups H1, the method for parallel processing of multiple data groups H1 can refer to the method for parallel processing of a single data group H1. For example, if the K1 data groups H1 are two data groups H1, the CPU can process the first data group H1 in parallel using the same method as for processing a single data group H1, and the second data group H1 in parallel using the same method as for processing a single data group H1, and these two data groups H1 are processed in parallel.

[0420] It should also be noted that when there are K1 data groups H1, the large amount of data in multiple data groups H1 may cause an imbalance in the load of the multiple CPUs in a multi-core CPU electronic device. For example, if the multi-core CPU includes CPU1 and CPU2, due to the large amount of data, CPU1 may process 3 / 4 of the data in multiple data groups H1, while CPU2 may process 1 / 4 of the data in multiple data groups H1, resulting in an unbalanced load.

[0421] To balance the load across multiple CPUs, in some embodiments, K1 data groups H1 can be divided into multiple data groups H11. The number of data groups H11 matches the number of CPUs. Each data group H11 includes at least P×P R components, P×P G components corresponding to the P×P R components, and P×P B components corresponding to the P×P R components. The amount of data in each data group H11 is the same. Each CPU can process at least one data group H11 in parallel to obtain K1 data subsets A2, B2, and C2. Multiple CPUs process multiple data groups H11 in parallel, with each CPU processing the same amount of data.

[0422] It should be understood that electronic devices can partition data through load balancing modules, such as a scheduler. Matching the number of data groups H11 with the number of CPUs means that the number of data groups H11 is an integer multiple of the number of CPUs. For example, if there are 2 CPUs, the number of data groups H11 can be a positive multiple of 2 (e.g., 4). Similarly, if there are 3 CPUs, the number of data groups H11 can be a positive multiple of 3 (e.g., 6).

[0423] Each data group H11 includes at least P×P R components, P×P G components corresponding to the P×P R components, and P×P B components corresponding to the P×P R components. The same data size for each data group H11 means that, assuming K1 = 8 and there are 4 CPUs, the 8 data groups H1 can be divided into 4 data groups H11. Each data group H11 includes 2 data groups H1, and each data group H1 includes 8×8 R components, 8×8 G components corresponding to the 8×8 R components, and 8×8 B components corresponding to the 8×8 R components. The data size of these 4 data groups H11 is the same. For explanations regarding the G components corresponding to the R components and the B components corresponding to the R components, please refer to the above embodiment; they will not be repeated here.

[0424] Each of the multiple CPUs can process at least one data group H11 in parallel. Multiple CPUs processing multiple data groups H11 in parallel, with each CPU processing the same amount of data, means that, assuming K1 = 8 and CPU = 4, the 8 data groups H1 can be divided into 4 data groups H11, each data group H11 comprising 2 data groups H1. Each of the 4 CPUs can process 2 data groups H1 in parallel, and these 4 CPUs process the data simultaneously. Furthermore, since each CPU processes the same number of data groups H1, the amount of data processed by each CPU is the same. For details on how a CPU can process one or more data groups H11 in parallel, please refer to the above embodiment; it will not be repeated here.

[0425] In this embodiment, because the electronic device can partition the processing so that each CPU in a multi-core CPU processes the same amount of data, the workload of each CPU can be balanced. Load balancing ensures the utilization of each CPU, preventing some CPUs from being overloaded while others are idle, thereby improving the overall computing efficiency of the electronic device. Load balancing reduces the risk of individual CPUs failing due to overload, thus improving the reliability and stability of the entire electronic device. Furthermore, load balancing reduces the use of high-power CPUs among multiple CPUs, thereby lowering the overall energy consumption of the electronic device.

[0426] S436, the CPU determines whether the current number of iterations of loop 1 is equal to the total number of iterations of loop 1.

[0427] It should be understood that the number of times the current loop processes 1 means, for example, that the number of times the current loop processes 1 is 1 when it processes 1 for the first time, and the number of times the current loop processes 1 is 2 when it processes 1 for the second time, and so on.

[0428] It should also be understood that if the number of iterations of the current loop process 1 equals the total number of iterations of loop process 1, then the current loop process 1 is the last iteration of loop process 1, and loop process 1 needs to be terminated after the current iteration of loop process 1 is completed. If the number of iterations of the current loop process 1 is less than the total number of iterations of loop process 1, then the current loop process 1 is not the last iteration of loop process 1, and loop process 1 needs to be executed again after the current iteration of loop process 1 is completed.

[0429] In implementation, for example, the CPU can use a counter to record the number of times loop 1 is processed. For instance, the counter might record 1 for the first iteration of loop 1, 2 for the second iteration, and so on. The CPU can determine whether the current iteration of loop 1 equals the total number of iterations. For example, if the total number of iterations is 6, and the counter records 4 iterations, then the CPU determines that the current iteration of loop 1 does not equal the total number of iterations.

[0430] S437, if the CPU determines that the current number of iterations of loop 1 is less than the total number of iterations of loop 1, it returns to execute steps S433 to S435.

[0431] It should be understood that steps S433 to S435 can perform the operations required for each loop.

[0432] S438, if the CPU determines that the number of times loop 1 is processed in the current loop is equal to the total number of times loop 1 is processed, then the loop ends to obtain (M / P)×(N / P) data subsets A2, (M / P)×(N / P) data subsets B2 and (M / P)×(N / P) data subsets C2.

[0433] For example, after the CPU finishes processing loop 1, the resulting (M / P)×(N / P) data subsets A2, B2, and C2 can be referenced. Figure 7 (b) shows (32 / 8) × (32 / 8) data subsets A2, (32 / 8) × (32 / 8) data subsets B2 and (32 / 8) × (32 / 8) data subsets C2.

[0434] In the relevant schemes, the CPU processes data serially during color space conversion. For example, the CPU serially loads data from memory into registers, serially reads data from registers, and serially processes the read data.

[0435] It should be understood that the process of the CPU loading data from memory into registers is as follows: the CPU accesses memory and writes the data from memory into the register. Because related schemes load data from memory into registers serially, for example, the CPU first accesses a piece of data in memory, writes that data into the register, then accesses another piece of data in memory and writes that data into the register again, thus requiring the CPU to access memory a relatively large number of times.

[0436] Frequent CPU memory accesses can lead to several problems: 1. Decreased CPU performance. With frequent memory accesses, the CPU needs to constantly read data from memory, increasing its workload and causing performance degradation. 2. Increased power consumption in electronic devices. Memory access operations are among the most power-intensive operations in electronic devices; frequent accesses increase power consumption. 3. Increased memory access latency. Memory access latency refers to the time delay between the CPU issuing a memory access request and receiving a memory response. Excessive memory accesses require the memory controller to process more requests, potentially causing request queuing and increasing latency. These problems are particularly pronounced for images with large datasets (such as high-resolution images).

[0437] In this embodiment, the CPU can load multiple data items from memory into a two-dimensional register in parallel. For example, the CPU can access multiple data items in memory at once via an S433 microcontroller and then write the data items into the register in parallel. Compared to related schemes that load data serially from memory into registers, the number of memory accesses can be reduced. Because the number of memory accesses is reduced, CPU performance can be improved, power consumption of electronic devices can be reduced, and memory access latency can be lowered. Furthermore, reducing the number of memory accesses can alleviate memory bandwidth pressure to some extent.

[0438] Because the proposed solution involves the CPU serially reading data from registers and processing it serially using serial instructions (e.g., the CPU NEON instruction), such as first serially reading a set of R, G, and B components from the registers and performing color space conversion on these components, and then serially reading another set of R, G, and B components and performing the same color space conversion again, the CPU's color space conversion processing is slow and inefficient. Furthermore, this slow processing speed can lead to higher CPU utilization (CPU utilization is the ratio of CPU processing time to total processing time; slower processing speed results in longer processing time and higher utilization), and since CPU utilization is directly proportional to the power consumption of electronic devices (higher CPU utilization leads to higher power consumption), the overall power consumption of the electronic device is also higher.

[0439] In this embodiment, the CPU can read multiple data items from registers in parallel and process them. For example, the CPU reads multiple data items from registers in parallel via S435 and processes them in parallel via S436 to achieve color space conversion. Therefore, compared to related schemes that read data from registers serially and process the data serially, the processing speed and efficiency of the CPU in color space conversion can be improved. Furthermore, because the CPU processing speed is improved, CPU utilization is reduced, thereby reducing the power consumption of the electronic device.

[0440] The above embodiments describe Figure 4 The specific implementation of color space conversion involved in step S43 is shown below. The following embodiments illustrate this. Figure 4 The specific implementation of the discrete cosine transform involved in step S45 will be introduced.

[0441] Discrete Cosine Transform

[0442] For example, please refer to Figure 25 , Figure 25 This is a schematic diagram of a discrete cosine transform provided in an embodiment of this application. It illustrates the implementation of the discrete cosine transform during JPEG compression encoding, including steps S451 to S459.

[0443] S451, the CPU changes the storage method of Q data subsets A3, J data subsets B3 and L data subsets C3 in memory from row storage to column storage, or the CPU changes the storage method of P×P DCT transform coefficients in memory from row storage to column storage.

[0444] It should be understood that, typically, the discrete cosine transform of each 8×8 color component (e.g., the Y component) in JPEG compression encoding can be expressed as a matrix multiplication operation as shown in Equation 5 below:

[0445] Matrix C = Matrix A × Matrix B. Formula 5

[0446] In this matrix, matrix A is an 8×8 matrix containing color components (e.g., the Y component). Matrix B is an 8×8 matrix containing DCT transform coefficients, which are pre-defined coefficients used for performing discrete cosine transform. Matrix C is a matrix containing 8×8 DCT coefficients obtained by performing a discrete cosine transform on an 8×8 Y component.

[0447] Considering that matrix multiplication has a large number of calculations and high computational complexity, in order to reduce the number of calculations and reduce computational complexity, this embodiment converts matrix multiplication into the accumulation of vector outer products when performing discrete cosine transform on each 8×8 color component 2 (e.g., the Y component).

[0448] For example, please refer to Figure 26 , Figure 26 This is a schematic diagram of an out-of-vector accumulation provided in an embodiment of this application. Assume the CPU needs to perform... Figure 26 This demonstrates a discrete cosine transform of an 8×8 Y component. Typically, a CPU performs this operation via matrix multiplication, for example... Figure 26 Multiplying matrix A and matrix B as shown yields matrix C, which is then used for discrete cosine transform. The data included in matrices A, B, and C can be found in the above embodiment, and will not be repeated here.

[0449] It should be understood that each element in matrix C, obtained through matrix multiplication, is the dot product of the i-th row of matrix A and the j-th row of matrix B. For example, Figure 26 The DCT coefficient 1 in matrix C shown is the dot product of the first row element of matrix A and the first column element of matrix B. For example, DCT coefficient 11 = Y11 × transform coefficient 11 + Y12 × transform coefficient 21 + Y13 × transform coefficient 31 + Y14 × transform coefficient 41 + Y15 × transform coefficient 51 + Y16 × transform coefficient 61 + Y17 × transform coefficient 71 + Y18 × transform coefficient 81. As can be seen from the above, obtaining an element in matrix C through matrix multiplication requires 8 multiplication operations and 7 addition operations. Since matrix C has 64 elements, obtaining matrix C through matrix multiplication requires 8 × 64 = 512 multiplication operations and 7 × 64 = 448 addition operations, for a total of 512 + 448 = 960 calculations. Therefore, performing Discrete Cosine Transform (DCT) through matrix multiplication involves a large number of calculations and has high computational complexity.

[0450] This application's embodiments can convert matrix multiplication operations into the accumulation of vector outer products. For example, Figure 26 The diagram illustrates the outer product operation performed by combining the first column of matrix A (which can be considered a vector) with the first row of matrix B (which can also be considered a vector), and the second column of matrix A (which can be considered a vector) with the second row of matrix B (which can also be considered a vector). This process is repeated for each of the eight rows of matrix A with each of the eight columns of matrix B. Finally, the results of these outer product operations are summed. As can be seen, obtaining matrix C through the summation of vector outer products requires eight outer product operations and one summation operation, totaling nine calculations. Therefore, using the summation of vector outer products reduces the number of calculations and computational complexity compared to performing discrete cosine transform through matrix multiplication.

[0451] It should be understood that the result of the cross product of vectors is a matrix. For example, if vector a is a 1×8 vector and vector b is an 8×1 vector, then the cross product of vectors a and b is an 8×8 matrix. The element in the i-th row and j-th column of this matrix is ​​the product of the i-th element of vector a and the j-th element of vector b. For example, the element in the first row and first column of this matrix is ​​the product of the first element of vector a and the first element of vector b.

[0452] Figure 26 This illustrates the outer accumulation operation performed on each column of matrix A and the corresponding row of matrix B. In some embodiments, the outer accumulation operation can also be performed on each row of matrix A and the corresponding column of matrix B. For an example, please refer to [reference needed]. Figure 27 , Figure 27 This is a schematic diagram of another vector-outside accumulation provided in the embodiments of this application.

[0453] It should also be understood that Figure 26 and Figure 27 This illustrates the process by which the CPU performs a discrete cosine transform on an 8×8 Y component using external accumulation operations. The CPU executes... Figure 4After step S44 in the overall encoding process shown, Q data subsets A3, J data subsets B3, and L data subsets C3 are obtained. Each data subset A3 includes P×P color components 21 (e.g., 8×8 Y components), each data subset B3 includes P×P color components 22 (e.g., 8×8 Cb components), and each data subset C3 includes P×P color components 23 (e.g., 8×8 Cr components). In step S45, the CPU performs discrete cosine transform on each of the Q data subsets A3, J data subsets B3, and L data subsets C3 obtained in step S44 through external accumulation and addition operations.

[0454] In this embodiment of the application, the reason why the CPU needs to change the storage method of Q data subsets A3, J data subsets B3, and L data subsets C3 in memory from row storage to column storage, or the CPU needs to change the storage method of P×P DCT transform coefficients in memory from row storage to column storage, is as follows:

[0455] pass Figure 26 It is known that the CPU requires each row of data in matrix A and each column of data in matrix B for external accumulation and addition operations. This data needs to be loaded from memory into a two-dimensional register, and then read from the register by the CPU. However, memory typically stores data in row-major order. For example... Figure 16 As shown, when storing data in memory, the first row of data A1-11(R11) to A1-11(R18) of the data subset A1-11 is stored first, followed by the second row of data A1-11(R21) to A1-11(R28), and so on. In other words, the data in the data subset A1-11 (which can be considered a matrix) stored in memory is stored contiguously. Typically, the CPU can access memory once to read the contiguously stored data and load it into a two-dimensional register according to its storage order in memory. For example... Figure 18 The illustrated two-dimensional register loads the first 16 consecutive data items from memory in its first row, the 17th to 32nd consecutive data items from memory in its second row, and so on. The CPU can read data from the two-dimensional register to obtain data. For example, from... Figure 18 The sequence of data read from the first row of the two-dimensional register shown is A1-11(R11), ..., A1-11(R18), A1-11(R21), ..., A1-11(R28).

[0456] As can be seen from the above, if data is stored contiguously in memory, then the CPU's access to memory to load data and its reading from the two-dimensional register will also be arranged contiguously in rows. Figure 26 The external accumulation operation shown requires the CPU to read data from a two-dimensional register, such as... Figure 26 The column data shown is not stored contiguously in memory, requiring the CPU to access memory multiple times. Each data point in the column is sequentially loaded into a two-dimensional register through these multiple memory accesses, allowing the CPU to read the column data from the register. For an example, please refer to [reference needed]. Figure 28 , Figure 28 This is a schematic diagram of reading column data provided in an embodiment of this application. Figure 28 The data stored in memory shown is Figure 26 The data in matrix A is stored in memory in a row-based format. When the CPU needs to obtain... Figure 26 To retrieve the data for the first column of matrix A, the process involves first accessing memory once to load Y11 from memory into a two-dimensional register. Then, another memory access is performed to load Y21 from memory into the two-dimensional register. This process is repeated eight times. After eight memory accesses, the eight data points Y11, Y21, ..., Y81 can be loaded into the two-dimensional register. The CPU can then read these eight data points from the register, thus obtaining the data for the first column of matrix A. The methods for obtaining the data for the other columns of matrix A can be found in [reference needed]. Figure 28 .

[0457] Figure 28 The example shown requires multiple memory accesses to retrieve column data. Figure 26 The example shown illustrates the external accumulation operation. Similarly, when the external accumulation operation is performed using... Figure 27 When performed in the manner shown, it also suffers from the problem of needing to access memory multiple times to retrieve column data.

[0458] Frequent CPU memory accesses can lead to several problems: 1. Decreased CPU performance. 2. Increased power consumption of electronic devices. 3. Increased memory access latency. Therefore, to reduce the number of memory accesses, embodiments of this application employ... Figure 26 Before the external accumulation operation shown, the storage method of Q data subsets A3, J data subsets B3, and L data subsets C3 in memory can be changed from row storage to column storage. (This is achieved through...) Figure 27 Before the external accumulation operation shown, the storage method of the P×P DCT transform coefficients in memory can be changed from row storage to column storage.

[0459] For example, please refer to Figure 29 , Figure 29 This is a schematic diagram illustrating the conversion from row-based storage to column-based storage provided in an embodiment of this application. Figure 29The logical size of the illustrated two-dimensional register is 8 rows and 8 columns. For an explanation of the logical size of the two-dimensional register, please refer to the above embodiment; it will not be repeated here. Of course, in other embodiments, the two-dimensional register can have other logical sizes; here, 8 rows and 8 columns are used as an example. The data stored in memory 1 can be any of the Q data subsets A3, comprising 8×8 Y components. These 8×8 Y components are stored contiguously in memory in row-based storage. The CPU can sequentially read these 8×8 Y components from memory and load them horizontally into different rows of the two-dimensional register. Then, the CPU sequentially reads the data from different columns of the two-dimensional register vertically and writes the read data into memory. In this way, the 8×8 Y components can be stored contiguously in memory in column-based storage, for example, referring to the data stored in memory 2.

[0460] The CPU's method for converting the storage of Q data subsets A3, J data subsets B3, and L data subsets C3 in memory from row-based to column-based storage, and the method for converting the storage of P×P DCT transform coefficients in memory from row-based to column-based storage, can be found in [reference needed]. Figure 29 The method shown in the diagram will not be repeated here.

[0461] It should also be understood that the above-mentioned conversion of the memory storage method from row-based storage to column-based storage can also be called a transpose operation.

[0462] S452, the CPU determines the number K2 of P×P color components 2 that can be loaded in at least one two-dimensional register based on the bit width of P×P color components 2, the bit width of P×P DCT transform coefficients and the logic size of at least one two-dimensional register.

[0463] It should be understood that P×P color components can refer to 8×8 Y components, 8×8 Cb components, or 8×8 Cr components. Assuming that the bit width of one Y component, one Cb component, and one Cr component is each 8 bits, then the bit width of P×P color components is 8×8×8 bits. Assuming that the bit width of one DCT transform coefficient is 8 bits, then the bit width of P×P DCT transform coefficients is 8×8×8 bits. For an explanation of at least one two-dimensional register and its logical size, please refer to the above embodiment; it will not be repeated here.

[0464] It should also be understood that, due to the large amount of image data and the limited logical size of the two-dimensional register, it may not be possible to load all the image data at once. In other words, the CPU cannot read all the image data from the two-dimensional register at once to perform a discrete cosine transform. Based on the above, this embodiment first determines the number K2 of P×P color components that the two-dimensional register can load at one time, based on the logical size of the two-dimensional register, the bit width of the P×P color components, and the bit width of the P×P DCT transform coefficients. Then, it performs a discrete cosine transform on all the image data through a loop process. In each loop process, the CPU can load K2 P×P color components from memory into the two-dimensional register, then read K2 P×P color components from the two-dimensional register, and finally perform a discrete cosine transform on the read K2 P×P color components using an out-of-vector accumulation method.

[0465] As can be seen from the above examples, Figure 25 The illustrated discrete cosine transform (DCT) process involves a total of Q data subsets A3, J data subsets B3, and L data subsets C3. The Q data subsets A3 can include Q P×P color components (e.g., Y components), the J data subsets B3 can include J P×P color components (e.g., Cb components), and the L data subsets C3 can include P×P color components (e.g., Cr components). In other words, the total image data includes G = Q + J + L P×P color components. When performing the DCT, the CPU needs to perform out-of-vector accumulation on each of the G P×P color components and the P×P DCT transform coefficients. For example, in one loop, the CPU needs to perform out-of-vector accumulation on K2 P×P color components and K2 P×P DCT transform coefficients. The memory stores G P×P color components and 1 P×P DCT transform coefficient. Therefore, in order for the CPU to perform out-of-vector accumulation and summation of K² P×P color components and K² P×P DCT transform coefficients in one loop, the CPU needs to load one P×P DCT transform coefficient from memory into a two-dimensional register, and read P×P DCT transform coefficients from the two-dimensional register multiple times (e.g., K² times). In other words, during each loop, the two-dimensional register can load K² P×P color components and one P×P DCT transform coefficient.

[0466] Based on the above analysis, in implementation, regardless of whether the number of two-dimensional registers is one or more, and whether the logical size of the two-dimensional registers is variable or immutable, the CPU can determine the number K2 of P×P color components that can be loaded at one time by at least one two-dimensional register using the following formula:

[0467]

[0468] In some embodiments, the quantity K2 obtained by the CPU through Formula 6 is a non-integer. The CPU also needs to perform floor rounding on the non-integer, that is, regardless of the fractional part, the fractional part is directly discarded and only the integer part is retained.

[0469] S453, the CPU determines the total number of iterations of loop processing 2 based on the number of P×P color components 2 K2 and Q data subsets A3, J data subsets B3 and L data subsets C3. Each iteration of loop processing 2 is used to perform discrete cosine transform on K2 P×P color components 2.

[0470] In implementation, the CPU can determine the total number of iterations for loop 2 using the following formula:

[0471]

[0472] Where Q is the number of data subsets A3, J is the number of data subsets B3, L is the number of data subsets C3, and K2 is the number of P×P color components that can be loaded into at least one two-dimensional register at a time.

[0473] In other embodiments, the total number of loop iterations obtained by the CPU using Formula 7 is not an integer. The CPU also needs to perform floor operations on the non-integer, that is, regardless of the decimal part, the integer part is incremented by one.

[0474] It should be understood that after the CPU determines the total number of iterations of loop processing 2 through step S453, it begins loop processing and can perform the operations required for each loop through the following steps S454 to S456.

[0475] In S454, the CPU loads K2 P×P color components from memory in parallel into at least one two-dimensional register, and loads P×P DCT transform coefficients from memory in parallel into two-dimensional registers.

[0476] It should be understood that the CPU can access memory once based on the memory addresses of K2 P×P color components and P×P DCT transform coefficients through SIMD operations, and load K2 P×P color components from memory into at least one two-dimensional register in parallel, and load P×P DCT transform coefficients into two-dimensional registers in parallel.

[0477] In the implementation, for example, please refer to Figure 30 , Figure 30 This is a schematic diagram of parallel data loading provided in an embodiment of this application.

[0478] Figure 30The example used is P×P=8×8. Figure 30 The 8×8 DCT transform coefficients shown are stored in memory in column storage. Assuming that the bit width of one DCT transform coefficient is 8 bits, then the bit width of 8×8 DCT transform coefficients is 8×8×8 bits = 512 bits. Figure 30 The 4×4 data subset A3, 2×2 data subset B3, and 2×2 data subset C3 stored in memory can refer to the data processed by the CPU. Figure 4 The chromaticity sampling process in step S44 yields Q data subsets A3, J data subsets B3, and L data subsets C3. For example, the data in the 4×4 data subset A3 can refer to... Figure 11 The data in the (32 / 8) × (32 / 8) data subsets A3, each containing 8 × 8 Y components, with each Y component having a bit width of 8 bits, would have a bit width of 8 × 8 × 8 bits = 512 bits for the 8 × 8 Y components. The data in the 2 × 2 data subsets B3 could refer to... Figure 11 The data included in the (32 / 8)×(32 / 8)×(1 / 4) data subsets B3, each data subset B3 includes 8×8 Cb components. Assuming the bit width of one Cb component is 8 bits, then the bit width of 8×8 Cb components is 8×8×8 bits = 512 bits. The data in the 2×2 data subsets C3 can refer to... Figure 11 The data included in the (32 / 8)×(32 / 8)×(1 / 4) data subsets C3, each data subset C3 includes 8×8 Cr components. Assuming that the bit width of one Cr component is 8 bits, then the bit width of 8×8 Cr components is 8×8×8 bits = 512 bits.

[0479] Figure 30 Data subsets A3, B3, and C3 are stored in memory in a row-based manner. For an explanation of row-based and column-based storage, please refer to the above embodiment, which will not be repeated here.

[0480] Assumption Figure 30 The two-dimensional register has a depth of 16 rows and a width of 128 bits. Each row can load 128 / 8 bits = 16 Y components, or Cb components, or Cr components, or DCT transform coefficients. The number of P×P color components 2 that the CPU can load in at least one two-dimensional register, as determined by step S452, is K2 ((16×128)-512) / 512 = 3. The total number of loop processing operations 2 determined by the CPU in step S453 is ((4×4)+(2×2)+(2×2)) / 3 = 8.

[0481] The CPU's process of performing each loop can include, for example, during the first loop, the CPU can determine the memory addresses of the 8×8 DCT transform coefficients and... Figure 30 The memory shown includes 8×8 DCT transform coefficients stored in columns, such as transform coefficients 11 to 88, which are loaded into rows 1 to 4 of the two-dimensional register, and these 64 DCT transform coefficients are loaded in parallel. Based on the memory addresses of the three data subsets A3 (A3-11(Y11) to A3-11(Y88), A3-12(Y11) to A3-12(Y88), and A3-13(Y11) to A3-13(Y88), three data subsets A3 from the 4×4 data subsets A3 stored in memory (e.g., A3-11(Y11) to A3-11(Y88), A3-12(Y11) to A3-12(Y88), and A3-13(Y11) to A3-13(Y88)) are loaded into rows 5 to 16 of the two-dimensional register. These three data subsets A3 are loaded in parallel. In other words, all 16 rows of data in the two-dimensional register are loaded in parallel. The method by which the CPU loads data from memory into the two-dimensional register during other loop processing can be referenced. Figure 30 The loading method is shown. Figure 30 The meanings of the markings A3-11(Y11), B3-11(Cb11), etc. shown can be referred to the above embodiments, and will not be repeated here.

[0482] It should be noted that the method by which the CPU loads data from memory into the two-dimensional registers varies depending on whether there is one or more two-dimensional registers, and whether the logical size of the two-dimensional registers is variable or immutable. (Refer to...) Figure 30 The loading method shown is that the CPU loads 8×8 DCT transform coefficients from memory into a two-dimensional register, and loads K2 8×8 color components into the two-dimensional register, and the above loading operations are performed in parallel.

[0483] It should also be noted that Figure 30 This illustrates the parallel data loading method in step S451 when the CPU changes the storage method of the P×P DCT transform coefficients in memory from row-based storage to column-based storage. In step 451, when the CPU changes the storage method of the Q data subsets A3, J data subsets B3, and L data subsets C3 in memory from row-based storage to column-based storage (i.e., the Q data subsets A3, J data subsets B3, and L data subsets C3 are stored in columns, and the P×P DCT transform coefficients are stored in rows), the CPU's parallel data loading method can be found in [reference needed]. Figure 30 The method shown is not repeated here.

[0484] S455, the CPU reads K2 P×P color components in parallel from at least one two-dimensional register, and reads K2 P×P DCT transform coefficients from the two-dimensional register.

[0485] Because the SMID operation supports parallel reading of multiple data, the CPU can use the SMID operation to read multiple data items comprising K² P×P color components from at least one two-dimensional register in parallel. For an example, please refer to... Figure 31 , Figure 31 This is a schematic diagram of another parallel data reading method provided in an embodiment of this application. Figure 31 Therefore, step S454 is through Figure 30 The example illustrates how data is loaded from memory into a two-dimensional register in parallel.

[0486] Figure 30 The diagram shows that the CPU can load at least one 2D register with a capacity of K2 of 3, representing the number of P×P color components 2. The CPU loads 3 P×P color components 2 from memory in parallel to rows 5 through 16 of the 2D register. The CPU can... Figure 30 The fifth line of the shown two-dimensional register was read. Figure 31 The values ​​A31-11(Y11), ..., A31-11(Y18), A31-11(Y21), ..., A31-11(Y28) shown are read from the 6th row of the two-dimensional register. Figure 31 The following are shown: A31-11(Y31), ..., A31-11(Y38), A31-11(Y41), ..., A31-11(Y48). The CPU starts from... Figure 30 The data read from rows 7 to 16 of the shown two-dimensional register can be referenced. Figure 31 This will not be elaborated further here. Each row of data mentioned above was read in parallel.

[0487] It is understandable that the data read from lines 5 to 8 can be considered as data in 1 P×P color component 2, the data read from lines 9 to 12 can be considered as data in 1 P×P color component 2, and the data read from lines 13 to 16 can be considered as data in 1 P×P color component 2, for a total of 3 P×P color components 2.

[0488] The CPU can also read K2 P×P DCT transform coefficients from the two-dimensional register, with each reading of P×P DCT transform coefficients being performed in parallel.

[0489] It should be understood that during each loop, the CPU needs to perform out-of-vector accumulation and summation on K² P×P color components and K² P×P DCT transform coefficients. However, in the S454, the CPU loads one P×P DCT transform coefficient from memory. Therefore, in order for the CPU to perform out-of-vector accumulation and summation on K² P×P color components and K² P×P DCT transform coefficients in one loop, the CPU needs to read K² P×P DCT transform coefficients from a two-dimensional register. For an example, please refer to... Figure 30 As can be seen from the above examples, Figure 30 As shown, K2 = 3. The CPU loads the 8×8 DCT transform coefficients from memory into rows 1 to 4 of the two-dimensional register. The CPU can read the data from rows 1 to 4 of the two-dimensional register three times. For example, the data read by the CPU from row 1 of the two-dimensional register the first time can be referenced. Figure 31 The transformation coefficients 11, ..., 81 shown, and the data read by the CPU from rows 2 to 4 of the two-dimensional register for the first time, can be referenced. Figure 31 The data read by the CPU from rows 1 to 4 of the two-dimensional register for the second and third times can be found in [reference]. Figure 31 It should be understood that because the SMID operation supports parallel reading of multiple data, the data read by the CPU from rows 1 to 4 of the two-dimensional register each time is read in parallel.

[0490] S456, the CPU processes K2 P×P color components and K2 P×P DCT transform coefficients in parallel to obtain K2 P×P DCT coefficients.

[0491] Since SMID operation supports parallel processing of multiple data and multiple vectors, the CPU can perform an outer product operation on the P rows of each of the K2 P×P color components and the P columns of each of the K2 P×P DCT transform coefficients, and then perform an accumulation operation to obtain P×P DCT coefficients, thus obtaining K2 P×P DCT coefficients.

[0492] In this process, each row of color component 2 is multiplied by its corresponding column of DCT transform coefficients. The multiplication of each P×P color component 2 in the P rows of color components 2 with each P×P DCT transform coefficient in the P columns of DCT transform coefficients is performed in parallel.

[0493] It should be understood that since the CPU can access data in memory based on its memory address and load that data into a two-dimensional register, when the CPU reads data from the two-dimensional register and processes it, the CPU has already determined which of the multiple data sets stored in memory that the data refers to. For example, when the CPU reads data from the two-dimensional register... Figure 30 The transform coefficient 11 shown in the diagram refers to the DCT transform coefficient in the first row and first column of the 8×8 DCT transform coefficients stored in memory. For example, the CPU reads from a two-dimensional register... Figure 30 As shown in the diagram, A3-11(Y11) refers to the Y component of the first row and first column of the first data subset A3, which is one of the 4×4 data subsets A3 stored in memory. In other words, since the CPU can access data in memory based on its memory address and load that data into a two-dimensional register, when the CPU reads data from the two-dimensional register and processes it, it knows which data belongs to which subset A3, which data belongs to the same row within subset A3, and which data belongs to the same column among the P×P DCT coefficients. Based on this, the CPU can perform an outer product operation on the same row of data in subset A3 and the corresponding column of the P×P DCT coefficients. That is, the CPU can perform an outer product operation on each row of color component 2 and its corresponding column of DCT transform coefficients.

[0494] For example, please refer to Figure 32 , Figure 32 This is a schematic diagram of a parallel processing method provided in an embodiment of this application. Figure 32 The data in is from the CPU Figure 30 The data read from the two-dimensional register is shown. When the CPU reads transform coefficients 11, ..., 81 from the two-dimensional register, it can determine that transform coefficients 11, ..., 81 belong to the first column of the 8×8 DCT coefficients. When the CPU reads A3-11(Y11), ..., A3-11(Y18) from the two-dimensional register, it can determine that A3-11(Y11), ..., A3-11(Y18) belong to the first row of the 4×4 data subset A3. The CPU can perform an outer product operation on the eight Y components belonging to the first row of the 4×4 data subset A3 and the eight transform coefficients belonging to the first column of the 8×8 DCT coefficients corresponding to the first row. The first row corresponds to the first column. The correspondence between other rows and columns can be found in [reference needed]. Figure 32 ,for example Figure 32The data shown, A3-12(Y21), ..., A3-12(Y28), are the data in the second row of the data subset A3, which is in the first row and second column of the 4×4 data subset A3. The transformation coefficients 12, ..., 82 are the data in the second column of the 8×8 DCT coefficients. The data in the second row corresponds to the data in the second column.

[0495] In implementation, the CPU processes K² P×P color components and K² P×P DCT transform coefficients in parallel to obtain K² P×P DCT coefficients. The process can be as follows:

[0496] For example, the CPU can... Figure 30 The 5th to 8th rows of the shown two-dimensional register were read. Figure 30 The 8 rows of Y components in the first row and first column of the 4×4 data subset A3 shown are included. Figure 32 The second row of the six-row matrix boxes shows the data in the eight matrix boxes, with each rectangle containing a row of Y components.

[0497] from Figure 30 The first reading of rows 1 to 4 of the shown two-dimensional register Figure 30 The diagram shows 8 columns of DCT transform coefficients out of an 8×8 DCT transform coefficient set. These 8 columns of DCT transform coefficients include... Figure 32 The data in the first row of the six-row matrix boxes shown are from the eight matrix boxes, with each rectangle containing a column of DCT transform coefficients.

[0498] The CPU can perform an outer product operation on each of the 8 rows of Y components and the corresponding column of DCT transform coefficients from the 8 columns of DCT transform coefficients. Then, it accumulates the results of the outer products of each row of Y components and their corresponding column of DCT transform coefficients to obtain... Figure 32 The diagram shows a total of 8 × 8 = 64 DCT coefficients, from D-11 (DCT11) to D-11 (DCT88). The outer product operation is performed in parallel on the above 8 rows of Y components and 8 columns of DCT transform coefficients.

[0499] At the same time, the CPU can also perform parallel processing from... Figure 30 The 9th to 12th rows of the shown two-dimensional register were read. Figure 30 The 8 rows of Y components in the first row and second column of the 4×4 data subset A3 shown are included. Figure 32 The fourth row of the six-row matrix boxes shows the data in eight matrix boxes, each rectangle containing one row of the Y component. And the CPU from... Figure 30 The second read of rows 1 to 4 of the shown two-dimensional register Figure 30The diagram shows 8 columns of DCT transform coefficients out of an 8×8 DCT transform coefficient set. These 8 columns of DCT transform coefficients include... Figure 32 The data in the 8 rectangles shown in the 3rd row of the 6-row matrix frame is illustrated, with each rectangle containing a column of DCT transform coefficients. The CPU can also perform a parallel outer product operation on these 8 rows of Y components and these 8 columns of DCT transform coefficients, based on the method described above, to obtain... Figure 32 The D-12(DCT11) to D-12(DCT88) shown contain a total of 8×8=64 DCT coefficients.

[0500] At the same time, the CPU can also perform parallel processing from... Figure 30 Lines 13 to 16 of the shown two-dimensional register were read. Figure 30 The 8 rows of Y components in the 1st row and 3rd column of the 4×4 data subset A3 shown are included. Figure 32 The data shown in the 6th row of the matrix boxes comprises 8 matrix boxes, each rectangle containing one row of the Y component. And the CPU from... Figure 30 The third read from rows 1 to 4 of the shown two-dimensional register Figure 30 The diagram shows 8 columns of DCT transform coefficients out of an 8×8 DCT transform coefficient set. These 8 columns of DCT transform coefficients include... Figure 32 The data in the 8 rectangles shown in the 5th row of the 6-row matrix frame is illustrated; each rectangle includes a column of DCT transform coefficients. The CPU can also perform a parallel outer product operation on these 8 rows of Y components and these 8 columns of DCT transform coefficients, based on the method described above, to obtain... Figure 32 The D-13(DCT11) to D-13(DCT88) shown contain a total of 8×8=64 DCT coefficients.

[0501] It should be understood that the above Figure 32 This example illustrates the process of parallel processing of K2 P×P color components and K2 P×P DCT transform coefficients to obtain K2 P×P DCT coefficients, using K2=3 as an example. When K2 equals other integers, the implementation method can refer to the above process and will not be repeated here. It should also be understood that the meanings of identifiers such as D-12 (DCT11) and D-12 (DCT88) can be referred to the above embodiments and will not be repeated here. For example, D-12 (DCT11) can refer to... Figure 11 The DCT coefficients in the first row and second column of the data subset D shown are from the (32 / 8)×(32 / 8) data subset D.

[0502] Similarly, it can be understood that the color component 2 (e.g., the Y component) in the above row can be considered as a 1×8 vector. For example, a 1×8 vector can be referenced... Figure 27The first row of matrix A shown is a vector, containing 1 row and 8 columns of data. One column of DCT transform coefficients can be considered an 8×1 vector. For example, an 8×1 vector can be referenced... Figure 27 The vector in the first column of matrix B shown consists of 8 rows and 1 column of data. The outer product operation between the Y component in the first row and the DCT transform coefficients in the first column can be understood as a calculation based on the outer product operation of vectors. The result of the outer product operation is a matrix; for example, the outer product of a 1×8 vector and an 8×1 vector results in an 8×8 matrix. The element in the i-th row and j-th column of this matrix is ​​the product of the i-th element of the Y component in the first row and the j-th element of the DCT transform coefficients in the first column. For example, the element in the first row and first column of this matrix is ​​the product of the first element of the 1×8 vector and the first element of the 8×1 vector; the element in the first row and second column of this matrix is ​​the product of the first element of the 1×8 vector and the second element of the 8×1 vector, and so on. In other words, performing an outer product operation between one row of Y components and one column of DCT transform coefficients requires multiplying each Y component in one row of Y components (e.g., one row includes 8 Y components) with all the data in one column of DCT transform coefficients (e.g., one column includes 8 DCT transform coefficients), for a total of 64 multiplication operations.

[0503] In this embodiment, the outer product operation of each row color component 2 and each column DCT transform coefficient can be performed in parallel. For example, the above 64 multiplication operations are executed in parallel at the same time.

[0504] In some embodiments, the outer product operation between each row of color components 2 and each column of DCT transform coefficients can also be performed serially. For example, the CPU can perform serial calculations using a multi-level nested loop structure. This multi-level nested loop structure includes three nested loops: the outer loop iterates through the data in each row of Y components, the middle loop iterates through each column of DCT transform coefficients, and the inner loop executes calculation logic based on the iterated data, such as performing multiplication operations, thereby achieving serial calculations. For example, one row of Y components includes A3-11(Y11), A3-11(Y12), ..., A3-11(Y17), A3-11(Y18), a total of 8 Y components; one column of DCT transform coefficients includes DCT coefficient 11, DCT coefficient 21, ..., DCT coefficient 71, DCT coefficient 81, a total of 8 DCT coefficients. When the outer loop iterates to the first data A3-11(Y11) in the Y component of row 1, the middle loop iterates through the eight DCT transform coefficients in column 1, and the inner loop multiplies A3-11(Y11) with each of the eight DCT transform coefficients. Then, when the outer loop iterates to the second data A3-11(Y12) in the Y component of row 1, the middle loop again iterates through the eight DCT transform coefficients in column 1, and the inner loop multiplies A3-11(Y12) with each of the eight DCT transform coefficients, until the outer loop has iterated through all the data in column 1 of the Y component.

[0505] It should be noted that the implementation of step S456 above is illustrated by performing vector-outside accumulation and addition operations on each row of K2 P×P color components and each column of K2 P×P DCT transform coefficients. Figure 27 The diagram illustrates the method of vector extrapolation. The CPU performs vector extrapolation on each column of the K2 P×P color components and each row of the K2 P×P DCT transform coefficients (i.e., ... Figure 26 The implementation of the vector outer product (addition) shown can refer to the above implementation. For example, the CPU can perform an outer product operation on the P columns of color components 2 in each of the K2 P×P color components 2 and the P rows of DCT transform coefficients in each of the K2 P×P DCT transform coefficients, and then perform an accumulation operation to obtain P×P DCT coefficients, thus obtaining K2 P×P DCT coefficients. Specifically, the outer product operation is performed on each column of color component 2 and its corresponding row of DCT transform coefficients. The P columns of color components 2 in each of the P×P color components 2 and the P rows of DCT transform coefficients in each of the P×P DCT transform coefficients are performed concurrently during the outer product operation.

[0506] It should be understood that when the data volume of K² P×P color components and K² P×P DCT transform coefficients is large, the workload of the multiple CPUs in a multi-core CPU electronic device becomes unbalanced due to the large data volume. For example, in a multi-core CPU including CPU1 and CPU2, due to the large data volume, CPU1 processes 3 / 4 of the data of K² P×P color components and K² P×P DCT transform coefficients, while CPU2 processes 1 / 4 of the data, resulting in an unbalanced workload.

[0507] To balance the load across multiple CPUs, in some embodiments, the K² P×P color components and K² P×P DCT transform coefficients can be divided into multiple data groups H12. The number of data groups H12 matches the number of CPUs. Each data group H12 includes at least P×P color components and P×P DCT transform coefficients, and the amount of data in each data group H12 is the same. Each CPU can process at least one data group H12 in parallel to obtain K² P×P DCT coefficients. Multiple CPUs process multiple data groups H12 in parallel, with each CPU processing the same amount of data.

[0508] It should be understood that electronic devices can partition data using load balancing modules, such as a scheduler. Matching the number of data groups H12 with the number of CPUs means that the number of data groups H12 is an integer multiple of the number of CPUs.

[0509] Each data set H12 includes at least P×P color components 2 and P×P DCT transform coefficients. The same amount of data in each data set H12 means that, assuming K2=4, P=8, and the number of CPUs is 4, then 4 8×8 color components 2 and 4 8×8 DCT transform coefficients can be divided into 4 data sets H12. Each data set H12 includes 1 8×8 color component 2 and 1 8×8 DCT transform coefficient, and the amount of data in these 4 data sets H12 is the same.

[0510] Each of the multiple CPUs can process at least one data set H12 in parallel. Multiple CPUs processing multiple data sets H12 in parallel, with each CPU processing the same amount of data, means that each of the four CPUs can process the 8×8 color components and 8×8 DCT transform coefficients included in one data set H12 in parallel, and these four CPUs process the data simultaneously. Furthermore, since each CPU processes the same number of data sets H12, the amount of data processed by each CPU is the same. For details on how CPUs can process one or more data sets H12 in parallel, please refer to the above embodiment; further details will not be repeated here.

[0511] In this embodiment, because the electronic device can partition the processing so that each CPU in a multi-core CPU processes the same amount of data, the workload of each CPU can be balanced. Load balancing ensures the utilization of each CPU, preventing some CPUs from being overloaded while others are idle, thereby improving the overall computing efficiency of the electronic device. Load balancing reduces the risk of individual CPUs failing due to overload, thus improving the reliability and stability of the entire electronic device. Furthermore, load balancing reduces the use of high-power CPUs among multiple CPUs, thereby lowering the overall energy consumption of the electronic device.

[0512] S457, the CPU determines whether the current number of iterations of loop 2 is equal to the total number of iterations of loop 2.

[0513] The implementation method for this step can be found in the above embodiment, and will not be repeated here.

[0514] S458, if the CPU determines that the current number of iterations of loop 2 is less than the total number of iterations of loop 2, it returns to steps S454 to S456.

[0515] It should be understood that steps S454 to S456 can perform the operations required for each loop.

[0516] S459, if the CPU determines that the current number of times loop 2 is processed is equal to the total number of times loop 2 is processed, then the loop ends to obtain G data subsets D, each data subset D including P×P DCT coefficients.

[0517] For example, after the loop processing ends, the resulting G data subsets D can be referenced. Figure 11 The G data subsets D are shown.

[0518] The proposed solution involves performing discrete cosine transform on Q data subsets A3, J data subsets B3, and L data subsets C3 using matrix multiplication. However, performing discrete cosine transform via matrix multiplication involves a large number of calculations and has high computational complexity.

[0519] In this embodiment, the storage method of Q data subsets A3, J data subsets B3, and L data subsets C3 in memory can be changed from row storage to column storage through the above step S451, or the CPU can change the storage method of P×P DCT transform coefficients in memory from row storage to column storage. Then, in step S456, discrete cosine transform can be performed on the K2 P×P color components and K2 P×P DCT transform coefficients that need to be processed in each loop through vector outer product accumulation operation. The number of calculations can be reduced and the computational complexity can be reduced by accumulating vector outer products compared with performing discrete cosine transform through matrix multiplication operation.

[0520] Furthermore, when performing discrete cosine transform on K2 P×P color components and K2 P×P DCT transform coefficients through vector-outside accumulation and addition operations in the embodiments of this application, column data of K2 P×P color components (e.g., a column of data refers to a column of Y components mentioned in the above embodiments) or column data of K2 P×P DCT transform coefficients (e.g., a column of data refers to a column of DCT transform coefficients mentioned in the above embodiments) are required. However, the relevant solutions store data in memory in a row-based storage manner, and the column data is not stored contiguously. The relevant solutions require multiple accesses to memory to obtain the column data of K2 P×P color components or the column data of K2 P×P DCT transform coefficients.

[0521] In this embodiment, step S451 transforms the storage method of Q data subsets A3, J data subsets B3, and L data subsets C3 in memory from row-based storage to column-based storage, or the CPU transforms the storage method of P×P DCT transform coefficients in memory from row-based storage to column-based storage. Thus, the P×P color component 2 column data or K2 P×P DCT transform coefficient column data required for processing are stored contiguously in memory. The CPU can obtain the K2 P×P color component 2 column data or K2 P×P DCT transform coefficient column data with fewer memory accesses. By reducing the number of memory accesses, CPU performance can be improved, power consumption of electronic devices can be reduced, and memory access latency can be lowered.

[0522] Furthermore, the relevant scheme involves the CPU serially reading data from registers and processing that data serially. For example, it first reads a set of data from the register, including 8×8 Y components and 8×8 DCT transform coefficients, and then performs calculations on this set of data using matrix multiplication. Then it reads another set of data from the register, also including 8×8 Y components and 8×8 DCT transform coefficients, and performs calculations on this set of data using matrix multiplication, thus performing serial calculations. Therefore, the CPU's processing speed for discrete cosine transforms in this scheme is slow and inefficient. Moreover, the slow CPU processing speed may lead to high CPU utilization (CPU utilization is the ratio of CPU processing time to total processing time; slower CPU processing speed leads to longer CPU processing time, resulting in higher CPU utilization). Since CPU utilization is directly proportional to the power consumption of electronic devices (higher CPU utilization means higher power consumption), this also results in higher power consumption for the electronic devices.

[0523] In this embodiment, data can be read from the register in parallel via step S455, and the read data can be processed in parallel via step S456. Therefore, compared to related schemes that read data from the register serially and process the read data serially, the processing speed and efficiency of the CPU performing discrete cosine transform can be improved. Furthermore, since the CPU processing speed is improved, the CPU utilization is reduced, thereby reducing the power consumption of the electronic device.

[0524] The above embodiments describe Figure 4 The following examples illustrate the specific implementation of the discrete cosine transform involved in step S45. Figure 4 The specific implementation of the quantization process involved in step S46 will be introduced.

[0525] Quantification

[0526] For example, please refer to Figure 33 , Figure 33 This is a schematic diagram of a quantization process provided in an embodiment of this application. It illustrates the implementation of quantization processing during JPEG compression encoding, including steps S461 to S468.

[0527] S461, the CPU determines the number K3 of P×P DCT coefficients that can be loaded into at least one two-dimensional register based on the bit width of P×P DCT coefficients, the bit width of P×P quantization coefficients, and the logic size of at least one two-dimensional register.

[0528] It should be understood that the quantization of each 8×8 DCT coefficient in JPEG compression encoding can typically be represented by the matrix operation shown in the following formula:

[0529]

[0530] In this matrix, matrix C contains 8×8 DCT coefficients obtained through Discrete Cosine Transform (DCT). Matrix D contains 8×8 quantization coefficients, which are preset coefficients used for quantization, such as those in a preset quantization table. Matrix E contains 8×8 quantized DCT coefficients obtained by quantizing the original 8×8 DCT coefficients.

[0531] In Formula 8 above, matrix E is obtained by dividing matrix C and matrix D element-wise (element-wise division is also called the Hadamard Division). Each element in matrix E is the quotient of the corresponding element in matrix C and matrix D. For example, please refer to... Figure 34 , Figure 34 This is a schematic diagram of element-by-element division provided in an embodiment of this application. Figure 34 In matrix E, the quantized coefficients represent the quantized DCT coefficients. Quantized coefficient 11 is the quotient of DCT coefficient 11 in matrix C and quantized coefficient 11 in matrix D, and quantized coefficient 12 is the quotient of DCT coefficient 12 in matrix C and quantized coefficient 12 in matrix D.

[0532] In the relevant scheme, the CPU performs serial quantization processing on each 8×8 DCT coefficient. For example, Figure 34 The CPU shown obtains the quantized coefficient 11 in matrix E based on the quotient of DCT coefficient 11 in matrix C and quantization coefficient 11 in matrix D. Then, it obtains the quantized coefficient 12 in matrix E based on the quotient of DCT coefficient 12 in matrix C and quantization coefficient 12 in matrix D. By using this serial processing method, 8×8 quantized DCT coefficients can be obtained.

[0533] Considering that serial processing involves a large number of calculations and high computational complexity, for example, Figure 34 As shown, the CPU needs to divide the 64 elements in matrix C by the 64 elements in matrix D in turn, for a total of 64 division operations, in order to obtain the elements in matrix E.

[0534] The embodiments of this application can reduce the number of calculations by using parallel processing, thereby reducing computational complexity. For example, please refer to... Figure 34In this embodiment, matrix C can be expanded into a one-dimensional array 1, and matrix D can be expanded into a one-dimensional array 2. A one-dimensional array can also be understood as a vector. Performing parallel division operations between each element of one-dimensional array 1 and each element of one-dimensional array 2 yields a one-dimensional array 3. The data in one-dimensional array 3 is the same as that in matrix E. This parallel processing reduces the number of calculations (e.g., from 64 to 1), thus reducing computational complexity and improving processing speed and efficiency.

[0535] It should also be understood that the general process of CPU quantization is as follows: the CPU loads data from memory into registers, then the CPU reads data from the registers, and finally the CPU performs quantization processing on the read data.

[0536] Figure 33 The quantization process shown is for Figure 4 The diagram shows that after performing discrete cosine transform in step S45, G data subsets D are obtained and then quantized. Each data subset D includes P×P DCT coefficients.

[0537] Because the image data volume (e.g., the image data consists of G subsets D) is large, and the logical size of the two-dimensional register is limited, it may not be possible to load all the image data at once. In other words, the CPU cannot read all the image data from the two-dimensional register at once for quantization processing. Based on the above reasons, this embodiment first determines the number K3 of P×P DCT coefficients that the two-dimensional register can load at one time, based on the logical size of the two-dimensional register, the bit width of the P×P DCT coefficients, and the bit width of the P×P quantization coefficients. Then, it performs quantization processing on all the image data through a loop. In each loop, the CPU loads K3 P×P DCT coefficients from memory into the two-dimensional register, then reads K3 P×P DCT coefficients from the two-dimensional register, and finally performs quantization processing on the read K3 P×P DCT coefficients.

[0538] As can be seen from the above embodiments, when the CPU performs quantization processing, it needs to divide each of the P×P DCT coefficients in the G P×P DCT coefficients element-wise with the P×P quantization coefficients. For example, in one loop, the CPU needs to divide K3 P×P DCT coefficients element-wise with K3 P×P quantization coefficients. The memory stores G P×P DCT coefficients and 1 P×P quantization coefficient. Therefore, in order for the CPU to perform element-wise division of K3 P×P DCT coefficients with K3 P×P quantization coefficients in one loop, the CPU needs to load 1 P×P quantization coefficient from memory into a two-dimensional register, and read P×P quantization coefficients from the two-dimensional register multiple times (e.g., K3 times). In other words, in each loop, the two-dimensional register can load K3 P×P DCT coefficients and 1 P×P quantization coefficient.

[0539] Based on the above analysis, in implementation, regardless of whether the number of two-dimensional registers is one or more, and whether the logic size of the two-dimensional registers is variable or immutable, the CPU can determine the number K3 of P×P DCT coefficients that can be loaded at one time for at least one two-dimensional register using the following formula:

[0540]

[0541] In some embodiments, the quantity K3 obtained by the CPU through Formula 9 is a non-integer. The CPU also needs to perform floor rounding on the non-integer, that is, regardless of the decimal part, the decimal part is directly discarded and only the integer part is retained.

[0542] S462, the CPU determines the total number of loop processing 3 based on the number of P×P DCT coefficients K3 and the G data subsets D. Each loop processing 3 is used to quantize the K3 P×P DCT coefficients.

[0543] In implementation, the CPU can determine the total number of iterations for loop 2 using the following formula:

[0544]

[0545] In the S463, the CPU loads K3 P×P DCT coefficients from memory in parallel into at least one two-dimensional register, and loads P×P quantization coefficients from memory in parallel into two-dimensional registers.

[0546] For details on how to implement this step, please refer to the implementation of S454; it will not be elaborated here.

[0547] Exemplary, exemplary, please refer to Figure 35 , Figure 35 This is a schematic diagram of another parallel data loading method provided in an embodiment of this application.

[0548] Figure 35 The example used is P×P=8×8. Figure 35 The 8×8 quantization coefficients shown are stored in memory in rows. Assuming that the bit width of one quantization coefficient is 8 bits, then the bit width of 8×8 quantization coefficients is 8×8×8 bits = 512 bits. Figure 35 The 4×4 data subset D, 2×2 data subset D, and 2×2 data subset D stored in memory can refer to the data processed by the CPU. Figure 4 The discrete cosine transform process in step S45 yields Q, J, and L data subsets D1, D2, and D3 respectively. For example, the data in the 4×4 data subset D1 could refer to... Figure 12 The data in the (32 / 8) × (32 / 8) data subsets D includes data where each subset D contains 8 × 8 DCT coefficients. Assuming each DCT coefficient has a bit width of 8 bits, then the bit width of 8 × 8 DCT coefficients is 8 × 8 × 8 bits = 512 bits. The data in the 2 × 2 data subsets D can refer to... Figure 11 The data included in the (32 / 8)×(32 / 8)×(1 / 4) data subsets D, each data subset D includes 8×8 DCT coefficients. Assuming the bit width of one DCT coefficient is 8 bits, then the bit width of 8×8 Cb is 8×8×8 bits = 512 bits. The data in the 2×2 data subsets D can refer to... Figure 11 The data included in the (32 / 8)×(32 / 8)×(1 / 4) data subsets D, each data subset D includes 8×8 DCT coefficients. Assuming that the bit width of a DCT coefficient is 8 bits, then the bit width of 8×8 Cr is 8×8×8 bits = 512 bits. Figure 35 The data subset D is stored in memory in a row-based manner. For an explanation of row-based storage, please refer to the above embodiment, which will not be repeated here.

[0549] Assumption Figure 35 The two-dimensional register has a depth of 16 rows and a width of 128 bits. Each row can load 128 / 8 bits = 16 DCT coefficients, or quantization coefficients. The number of P×P DCT coefficients that the CPU can load in at least one two-dimensional register, K3, as determined by step S461, is ((16×128)-512) / 512 = 3. The total number of loop processes 3 determined by the CPU in step S462 is ((4×4)+(2×2)+(2×2)) / 3 = 8.

[0550] The CPU's process of performing each loop can include, for example, during the first loop, the CPU can determine the memory addresses of the 8×8 quantization coefficients. Figure 35The memory shown includes 8×8 quantization coefficients stored in rows, such as DCT coefficients 11 to DCT coefficients 88, which are loaded into rows 1 to 4 of the two-dimensional register, and these 64 quantization coefficients are loaded in parallel. Based on the memory addresses of the three data subsets D—D-11 (DCT11) to D-11 (DCT88), D-12 (DCT11) to D-12 (DCT88), and D-13 (DCT11) to D-13 (DCT88)—three data subsets D from the 4×4 data subsets D stored in memory (e.g., D-11 (DCT11) to D-11 (DCT88), D-12 (DCT11) to D-12 (DCT88), and D-13 (DCT11) to D-13 (DCT88)) are loaded into rows 5 to 16 of the two-dimensional register. The data in these three data subsets D is loaded in parallel. In other words, all 16 rows of data in the two-dimensional register are loaded in parallel. The method by which the CPU loads data from memory into the two-dimensional register during other loop processing can be referenced. Figure 35 The loading method is shown. Figure 35 The meanings of the identifiers D-13 (DCT11) to D-13 (DCT88) shown can be referred to the above embodiments, and will not be repeated here.

[0551] It should be noted that the method by which the CPU loads data from memory into the two-dimensional registers varies depending on whether there is one or more two-dimensional registers, and whether the logical size of the two-dimensional registers is variable or immutable. (Refer to...) Figure 35 The loading method shown is that the CPU loads 8×8 quantization coefficients from memory into a two-dimensional register, and loads K3 8×8 DCT coefficients into a two-dimensional register, and the above loading operations are performed in parallel.

[0552] In S464, the CPU reads K3 P×P DCT coefficients in parallel from at least one two-dimensional register, and reads K3 P×P quantization coefficients from the two-dimensional register.

[0553] For details on how to implement this step, please refer to the implementation of S455; it will not be elaborated here.

[0554] For example, please refer to Figure 36 , Figure 36 This is a schematic diagram of another parallel data reading method provided in an embodiment of this application. Figure 36 Therefore, step S463 is through Figure 35 The example illustrates how data is loaded from memory into a two-dimensional register in parallel.

[0555] Figure 35The diagram shows that the CPU can load at least one P×P DCT coefficients (K3) into at least one two-dimensional register during one loop processing cycle. The CPU loads three P×P DCT coefficients from memory in parallel into rows 5 through 16 of the two-dimensional register. The CPU can... Figure 35 The fifth line of the shown two-dimensional register was read. Figure 36 The D-11 (DCT coefficient 11), ..., D-11 (DCT coefficient 18), D-11 (DCT coefficient 21), ..., D-11 (DCT coefficient 28) shown are read from the 6th row of the two-dimensional register. Figure 36 The figures shown are D-11 (DCT coefficient 31), ..., D-11 (DCT coefficient 38), D-11 (DCT coefficient 41), ..., D-11 (DCT coefficient 48). The CPU starts from... Figure 35 The data read from rows 7 to 16 of the shown two-dimensional register can be referenced. Figure 36 This will not be elaborated further here. Each row of data mentioned above was read in parallel.

[0556] It is understandable that the data read from lines 5 to 8 can be considered as data from 1 P×P DCT coefficients, the data read from lines 9 to 12 can be considered as data from 1 P×P DCT coefficients, and the data read from lines 13 to 16 can be considered as data from 1 P×P DCT coefficients, for a total of 3 P×P DCT coefficients.

[0557] The CPU can also read P×P quantization coefficients from the two-dimensional register K3 times, with each reading of P×P quantization coefficients being done in parallel.

[0558] It should be understood that during each loop, the CPU needs to perform out-of-vector accumulation and summation on K3 P×P DCT coefficients and K3 P×P quantization coefficients. However, in the S463, the CPU loads 1 P×P quantization coefficient from memory. Therefore, in order for the CPU to perform pixel-by-pixel division of K3 P×P DCT coefficients and K3 P×P quantization coefficients during one loop, the CPU needs to read K3 P×P quantization coefficients from a two-dimensional register. For an example, please refer to... Figure 35 As can be seen from the above examples, Figure 35 As shown, K3 = 3. The CPU loads 8×8 quantization coefficients from memory into rows 1 to 4 of the two-dimensional register. The CPU can read data from rows 1 to 4 of the two-dimensional register three times. For example, the data read by the CPU from row 1 of the two-dimensional register the first time can be referenced. Figure 36 The quantization coefficients shown are 11, ..., 18, 21, ..., 28. The data read by the CPU from rows 2 to 4 of the two-dimensional register for the first time can be referenced. Figure 36 The data read by the CPU from rows 1 to 4 of the two-dimensional register for the second and third times can be found in [reference]. Figure 36 It should be understood that because the SMID operation supports parallel reading of multiple data, the data read by the CPU from rows 1 to 4 of the two-dimensional register each time is read in parallel.

[0559] In the S465, the CPU processes K3 P×P DCT coefficients and K3 P×P quantization coefficients in parallel to obtain K3 P×P quantized DCT coefficients.

[0560] Since SMID operation supports parallel processing of multiple data, the CPU can perform a division operation on each of the K3 P×P DCT coefficients and the quantization coefficient corresponding to each DCT coefficient in the K3 P×P quantization coefficients to obtain K3 P×P quantized DCT coefficients.

[0561] It should be understood that the quantization coefficient corresponding to the DCT coefficient means, for example, that among the 8×8 DCT coefficients and 8×8 quantization coefficients, the DCT coefficient in the first row and first column corresponds to the quantization coefficient in the first row and first column, the DCT coefficient in the second row and second column corresponds to the quantization coefficient in the second row and second column, and each DCT coefficient has a unique corresponding quantization coefficient.

[0562] It should also be understood that since the CPU can access data in memory based on the memory address of the data and load the data into the two-dimensional register, when the CPU reads data from the two-dimensional register and processes it, the CPU has already determined which of the multiple data stored in memory the data refers to. Based on this, the CPU can perform a division operation on each of the K3 P×P DCT coefficients and the quantization coefficient corresponding to each DCT coefficient among the K3 P×P quantization coefficients.

[0563] For example, please refer to Figure 37 , Figure 37 This is a schematic diagram of another parallel processing method provided in an embodiment of this application. Figure 37 The data in the rectangle shown is... Figure 36 The data shown is the same, with each matrix frame including a DCT coefficient and its corresponding quantization coefficient. The CPU can simultaneously process... Figure 37 The data in the multiple matrix frames shown are divided to obtain three 8×8 quantized DCT coefficients. For example, one quantized DCT coefficient includes E-11 (quantized coefficient 11) to E-11 (quantized coefficient 88). One quantized DCT coefficient includes E-12 (quantized coefficient 11) to E-12 (quantized coefficient 88). One quantized DCT coefficient includes E-13 (quantized coefficient 11) to E-13 (quantized coefficient 88). Figure 37 The quantization coefficients refer to the quantized DCT coefficients. The meanings of the designations E-11 (quantized coefficient 11) and E-12 (quantized coefficient 88) can be found in the examples above and will not be repeated here. For example, E-11 could refer to… Figure 12 The data subset E in the first row and first column of the (32 / 8) × (32 / 8) data subsets E shown, E-11 (quantized coefficient 11), can refer to... Figure 12 The quantized DCT coefficients are shown in the first row and first column of the first data subset E of the (32 / 8)×(32 / 8) data subset E.

[0564] It should be understood that when the data volume is large, with K3 P×P DCT coefficients and K3 P×P quantization coefficients, the workload of the multiple CPUs in a multi-core CPU electronic device becomes unbalanced due to the large data volume. For example, in a multi-core CPU including CPU1 and CPU2, due to the large data volume, CPU1 processes 3 / 4 of the K3 P×P DCT coefficients and K3 P×P quantization coefficients, while CPU2 processes 1 / 4, resulting in an unbalanced workload.

[0565] To balance the load across multiple CPUs, in some embodiments, the K3 P×P DCT coefficients and K3 P×P quantization coefficients can be divided into multiple data groups H13. The number of data groups H13 matches the number of CPUs. Each data group H13 includes at least P×P DCT coefficients and P×P quantization coefficients, and the amount of data in each data group H13 is the same. Each CPU can process at least one data group H13 in parallel to obtain K3 P×P quantized DCT coefficients. Multiple CPUs process multiple data groups H13 in parallel, with each CPU processing the same amount of data.

[0566] It should be understood that electronic devices can partition data using load balancing modules, such as a scheduler. Matching the number of data groups H13 with the number of CPUs means that the number of data groups H13 is an integer multiple of the number of CPUs.

[0567] Each data set H13 includes at least P×P DCT coefficients and P×P quantization coefficients. The same amount of data in each data set H13 means that, assuming K3=4, P=8, and the number of CPUs is 4, then 4 8×8 DCT coefficients and 4 8×8 quantization coefficients can be divided into 4 data sets H13. Each data set H13 includes 1 8×8 DCT coefficient and 1 8×8 quantization coefficient, and the amount of data in these 4 data sets H13 is the same.

[0568] Each of the multiple CPUs can process at least one data set H13 in parallel. Multiple CPUs processing multiple data sets H13 in parallel, with each CPU processing the same amount of data, means that each of the four CPUs can process the 8×8 DCT coefficients and 8×8 quantization coefficients included in one data set H13 in parallel, and these four CPUs process the data simultaneously. Furthermore, since each CPU processes the same number of data sets H13, the amount of data processed by each CPU is the same. For details on how CPUs can process one or more data sets H13 in parallel, please refer to the above embodiment; it will not be repeated here.

[0569] In this embodiment, because the electronic device can partition the processing so that each CPU in a multi-core CPU processes the same amount of data, the workload of each CPU can be balanced. Load balancing ensures the utilization of each CPU, preventing some CPUs from being overloaded while others are idle, thereby improving the overall computing efficiency of the electronic device. Load balancing reduces the risk of individual CPUs failing due to overload, thus improving the reliability and stability of the entire electronic device. Furthermore, load balancing reduces the use of high-power CPUs among multiple CPUs, thereby lowering the overall energy consumption of the electronic device.

[0570] S466, the CPU determines whether the current number of times loop 3 is processed is equal to the total number of times loop 3 is processed.

[0571] For details on how to implement this step, please refer to the above example.

[0572] S467, if the CPU determines that the current number of iterations of loop 3 is less than the total number of iterations of loop 3, it returns to execute steps S463 to S465.

[0573] For details on how to implement this step, please refer to the above example.

[0574] S468, if the CPU determines that the current number of times loop 3 is processed is equal to the total number of times loop 3 is processed, then the loop ends to obtain G data subsets E, each of which includes P×P quantized DCT coefficients.

[0575] For example, the data obtained after the CPU finishes processing a loop can be used as a reference. Figure 12 The G data subsets E are shown.

[0576] In the relevant scheme, the CPU performs serial quantization processing on each 8×8 DCT coefficient. For example, Figure 34The CPU, as shown, obtains the quantized coefficient 11 in matrix E by quotienting the DCT coefficient 11 in matrix C and the quantization coefficient 11 in matrix D. Then, it obtains the quantized coefficient 12 in matrix E by quotienting the DCT coefficient 12 in matrix C and the quantization coefficient 12 in matrix D, and so on. This serial processing method can yield 8×8 quantized DCT coefficients. However, serial processing suffers from a high number of calculations and high computational complexity.

[0577] In this embodiment of the application, quantization can be performed in parallel processing via step S465. For example, Figure 37 The multiple matrix frames shown can perform division operations simultaneously in parallel to achieve quantization. Compared to serial quantization, the embodiments of this application can reduce the number of calculations, thereby reducing computational complexity and improving the processing speed and efficiency of quantization. Furthermore, due to the increased CPU processing speed, CPU utilization is reduced, thus lowering the power consumption of the electronic device.

[0578] Furthermore, because the relevant scheme processes each 8×8 DCT coefficient serially, the CPU accesses memory numerous times when acquiring data. For example, Figure 34 The CPU, as shown, first accesses memory once to obtain the DCT coefficient 11 in matrix C and the quantization coefficient 11 in matrix D. Based on the quotient of DCT coefficient 11 and quantization coefficient 11, it obtains the quantized coefficient 11 in matrix E. Then, the CPU accesses memory again to obtain the DCT coefficient 12 in matrix C and the quantization coefficient 12 in matrix D. Based on the quotient of DCT coefficient 12 and quantization coefficient 12, it obtains the quantized coefficient 12 in matrix E, and so on. It can be seen that this scheme involves a relatively large number of memory accesses.

[0579] In this embodiment, the CPU can load multiple data items from memory into a two-dimensional register in parallel. For example, the CPU accesses multiple data items in memory at once via S463, then writes these multiple data items into the register in parallel, and then reads and processes the multiple data items in the register via S464 and S465. Compared to related solutions, this reduces the number of memory accesses. Because the number of memory accesses is reduced, CPU performance can be improved, power consumption of electronic devices can be reduced, and memory access latency can be lowered.

[0580] The above embodiments describe the overall process of image encoding, including steps such as color space conversion, discrete cosine transform, and quantization. The following embodiments describe the image decoding process. First, the overall process of image decoding is described, and then the steps of inverse quantization, inverse discrete cosine transform, and color space conversion performed using the data processing method provided in this application are described.

[0581] It should be understood that image decoding is the reverse process of encoding, that is, the process of restoring the original image data after decoding and decompression of compressed and encoded image data. The meanings of relevant terms used in the following image decoding embodiments can be found in the explanations of those terms in the image encoding embodiments above, and will not be repeated here.

[0582] The overall process of image decoding

[0583] For example, please refer to Figure 38 , Figure 38 This is a schematic diagram of image decoding provided in an embodiment of this application. It illustrates the process of restoring the original image from a JPEG format image after decoding and decompression. This process can be simply referred to as JPEG decoding and decompression, and includes steps S51 to S57.

[0584] S51, the electronic device decodes the G groups of encoded data to obtain G data subsets E. Each group of encoded data is decoded into a data subset E, and each data subset E includes P×P quantized DCT coefficients.

[0585] It should be understood that the implementation process of this step is the reverse process of the encoding process in step S47. The embodiments of this application do not limit the implementation method of this step.

[0586] For example, please refer to Figure 13 After the electronic device performs decoding, it can... Figure 13 The 24 sets of encoded data shown are decoded into G = 16 + 4 + 4 data subsets E. Each data subset E includes 8 × 8 quantized DCT coefficients.

[0587] S52, the electronic device performs inverse quantization on G data subsets E to obtain G data subsets D, each of which includes P×P DCT coefficients.

[0588] It should be understood that the implementation process of this step is the reverse process of the quantization process in step S46.

[0589] For example, please refer to Figure 12 Electronic devices can Figure 12 The G data subsets E shown are inversely quantized to obtain G data subsets D. Each data subset D contains 8×8 DCT coefficients.

[0590] For details on the specific implementation of this step, please refer to the following example, which will not be elaborated here.

[0591] S53, the electronic device performs inverse discrete cosine transform on G data subsets D to obtain Q data subsets A3, J data subsets B3 and L data subsets C3. Each data subset A3 includes P×P color components 21, each data subset B3 includes P×P color components 22, and each data subset C3 includes P×P color components 23.

[0592] It should be understood that the implementation process of this step is the inverse process of the discrete cosine transform in step S45.

[0593] For example, please refer to Figure 11 Electronic devices can Figure 11 The inverse discrete cosine transform of the data subset D shown in G yields Q = (32 / 8) × (32 / 8) data subsets A3, J = (32 / 8) × (32 / 8) × (1 / 4) data subsets B3, and L = (32 / 8) × (32 / 8) × (1 / 4) data subset C3.

[0594] For details on the specific implementation of this step, please refer to the following example, which will not be elaborated here.

[0595] S54, the electronic device performs inverse chromaticity sampling on Q data subsets A3, J data subsets B3 and L data subsets C3 to obtain (M / P)×(N / P) data subsets A2, (M / P)×(N / P) data subsets B2 and (M / P)×(N / P) data subsets C2. Each data subset A2 includes P×P color components 21, each data subset B2 includes P×P color components 22, and each data subset C2 includes P×P color components 23.

[0596] It should be understood that the implementation process of this step is the reverse process of the chromaticity sampling process in step S44.

[0597] In practice, electronic devices can perform inverse chromaticity sampling through upsampling or interpolation methods, and this application does not limit this.

[0598] As shown in step S44 above, the chroma sampling formats include 4:4:4, 4:2:2, and 4:2:0. When the chroma sampling format is 4:4:4, the number of data subsets A3 is the same as the number of data subsets A2; therefore, the CPU does not need to perform inverse chroma sampling processing on the 4:4:4 format. In the chroma sampling formats 4:2:2 or 4:2:0, the number of data subsets A3 is reduced relative to the number of data subsets A2; therefore, the electronic device needs to perform inverse chroma sampling processing to make the number of data subsets A3 the same as the number of data subsets A2.

[0599] This application uses inverse chroma sampling processing as an example to illustrate the 4:2:0 format. For example, please refer to... Figure 10 In (b), the electronic device can perform inverse chromaticity sampling on Q = (32 / 8) × (32 / 8) data subsets A3, J = (32 / 8) × (32 / 8) × (1 / 4) data subsets B3, and L = (32 / 8) × (32 / 8) × (1 / 4) data subsets C3 to obtain (M / P) × (N / P) = (32 / 8) × (32 / 8) data subsets A2, (M / P) × (N / P) = (32 / 8) × (32 / 8) data subsets B2 and (M / P) × (N / P) = (32 / 8) × (32 / 8) data subsets C2. Each data subset A2 includes 8 × 8 Y components, each data subset B2 includes 8 × 8 Cb components, and each data subset C2 includes 8 × 8 Cb components.

[0600] S55, the electronic device performs color space conversion processing on (M / P)×(N / P) data subsets A2, (M / P)×(N / P) data subsets B2 and (M / P)×(N / P) data subsets C2 to obtain (M / P)×(N / P) data subsets A1, (M / P)×(N / P) data subsets B1 and (M / P)×(N / P) data subsets C1. Each data subset A1 includes P×P color components 11, each data subset B1 includes P×P color components 12, and each data subset C1 includes P×P color components 13.

[0601] It should be understood that the color space conversion in step S43 is to convert the RGB color space to the YCbCr color space. The color space conversion in step S55 is to convert the YCbCr color space back to the RGB color space. For example, please refer to... Figure 7 The electronic device performs color space conversion processing on (32 / 8)×(32 / 8) data subsets A2, (32 / 8)×(32 / 8) data subsets B2 and (32 / 8)×(32 / 8) data subsets C2 to obtain (32 / 8)×(32 / 8) data subsets A1, (32 / 8)×(32 / 8) data subsets B1 and (32 / 8)×(32 / 8) data subsets C1. Each data subset A1 includes 8×8 R components, each data subset B1 includes 8×8 G components, each data subset C1 includes 8×8 B components, each data subset A2 includes 8×8 Y components, each data subset B2 includes 8×8 Cb components, and each data subset C2 includes 8×8 Cr components.

[0602] S56, the electronic device merges (M / P)×(N / P) data subsets A1 to obtain M×N color components 11, merges (M / P)×(N / P) data subsets B1 to obtain M×N color components 12, and merges (M / P)×(N / P) data subsets C1 to obtain M×N color components 13, thus obtaining the dataset corresponding to the original image.

[0603] It should be understood that, since the dataset corresponding to the original image needs to be divided into multiple 8×8 image blocks during JPEG encoding, the multiple 8×8 image blocks need to be merged during JPEG decoding to obtain the dataset corresponding to the original image. The implementation method of this process is not limited in the embodiments of this application.

[0604] The above embodiments describe the overall process of image decoding. The following embodiments describe the specific implementation of step S52, inverse quantization, in the above process.

[0605] Inverse quantization

[0606] For example, please refer to Figure 39 , Figure 39 This is a schematic diagram of inverse quantization provided in an embodiment of this application. It illustrates an implementation method of inverse quantization during JPEG decoding, including steps S521 to S528.

[0607] S521, the CPU determines the number K4 of P×P quantized DCT coefficients that can be loaded into at least one two-dimensional register based on the bit width of P×P quantized DCT coefficients, the bit width of P×P quantized coefficients, and the logic size of at least one two-dimensional register.

[0608] The difference between this step and step S461 above is that step S461 determines the number K3 of P×P DCT coefficients that can be loaded into at least one two-dimensional register based on the bit width of P×P DCT coefficients, the bit width of P×P quantization coefficients, and the logic size of at least one two-dimensional register, while step S521 determines the number K4 of P×P quantized DCT coefficients that can be loaded into at least one two-dimensional register based on the bit width of P×P quantized DCT coefficients, the bit width of P×P quantization coefficients, and the logic size of at least one two-dimensional register.

[0609] Typically, the bit width of P×P DCT coefficients is the same as the bit width of P×P quantized DCT coefficients. For example, these two bit widths are usually 8×8×8 bits = 512 bits. Therefore, the number K4 of P×P quantized DCT coefficients that the CPU determines can be loaded into at least one two-dimensional register is the same as the number K3 of P×P DCT coefficients that the CPU determines can be loaded into at least one two-dimensional register. Therefore, the implementation method of step 521 can refer to the implementation method of step 461, and will not be repeated here.

[0610] S522, the CPU determines the total number of loop processing 4 based on the number of P×P quantized DCT coefficients K4 and the G data subsets E. Each loop processing 4 is used to perform inverse quantization processing on K4 P×P quantized DCT coefficients.

[0611] The difference between this step and step S462 above is that in step S462, the CPU determines the total number of iterations for loop processing 3 based on the number of P×P DCT coefficients K3 and the G data subsets D, while in step S462, the CPU determines the total number of iterations for loop processing 4 based on the number of quantized P×P DCT coefficients K4 and the G data subsets E.

[0612] Since the number of G data subsets E and G data subsets D are the same, and K3 and K4 are the same, the implementation of this step can refer to the implementation of step S462 above, and will not be repeated here.

[0613] The S523 CPU loads K4 P×P quantized DCT coefficients from memory in parallel into at least one two-dimensional register, and loads P×P quantized coefficients from memory in parallel into two-dimensional registers.

[0614] The difference between this step and step S463 above is that step S463 is that the CPU loads K3 P×P DCT coefficients from memory in parallel to at least one two-dimensional register, while step S523 is that the CPU loads K4 P×P quantized DCT coefficients from memory in parallel to at least one two-dimensional register.

[0615] The method for storing multiple DCT coefficients in memory is usually the same as storing multiple quantized DCT coefficients, both using row-based storage. Therefore, the implementation of step S523 can refer to the implementation of step S463, and will not be repeated here.

[0616] S524: The CPU reads K4 P×P quantized DCT coefficients in parallel from at least one two-dimensional register, and reads K4 P×P quantized coefficients from the two-dimensional register.

[0617] The implementation method for this step can be found in step S464, and will not be repeated here.

[0618] The S525 CPU processes K4 P×P quantized DCT coefficients and K4 P×P quantized coefficients in parallel to obtain K4 P×P DCT coefficients.

[0619] The difference between this step and step S465 is that in step S465, the CPU divides each DCT coefficient from the K3 P×P DCT coefficients by the corresponding quantization coefficient from the K3 P×P quantization coefficients to obtain K4 P×P quantized DCT coefficients. In step S525, the CPU multiplies each quantized DCT coefficient from the K4 P×P quantized DCT coefficients by the corresponding quantization coefficient from the K4 P×P quantization coefficients to obtain K4 P×P DCT coefficients. For example, please refer to... Figure 37 In step S465, the CPU performs parallel operations on D-11 (DCT11) ÷ quantization coefficient 11 = E-11 (quantized coefficient 11) and D-11 (DCT18) ÷ quantization coefficient 18 = E-11 (quantized coefficient 18). In step S525, the CPU performs parallel operations on E-11 (quantized coefficient 11) × quantization coefficient 11 = D-11 (DCT11) and E-11 (quantized coefficient 18) × quantization coefficient 18 = D-11 (DCT18).

[0620] Therefore, the implementation of step S525 can refer to the implementation of step S465. For example, the CPU can replace the K3 P×P DCT coefficients in step S465 with K4 P×P quantized DCT coefficients, and replace the division operation with the multiplication operation, so as to obtain the calculation result of step S525.

[0621] It should be understood that in some implementations of step S465, the K3 P×P DCT coefficients and K3 P×P quantization coefficients can also be divided to achieve load balancing. Similarly, in some implementations of step S525, the K4 P×P quantized DCT coefficients and K4 P×P quantization coefficients can also be divided to achieve load balancing.

[0622] For example, K4 P×P quantized DCT coefficients and K4 P×P quantized coefficients can be divided into multiple data groups H14. The number of data groups H14 matches the number of CPUs. Each data group H14 includes at least P×P quantized DCT coefficients and P×P quantized coefficients, and the amount of data in each data group H14 is the same. Each CPU in the multiple CPUs can process at least one data group H14 in parallel to obtain K4 P×P DCT coefficients. Multiple CPUs process multiple data groups H14 in parallel, and each CPU processes the same amount of data. For a detailed implementation of this method, please refer to the above embodiment, which will not be repeated here.

[0623] S526, the CPU determines whether the current number of times loop 4 is processed is equal to the total number of times loop 4 is processed.

[0624] The implementation method for this step can be found in the above embodiment, and will not be repeated here.

[0625] S527, if the CPU determines that the current number of times loop 4 is processed is less than the total number of times loop 4 is processed, then return to execute steps S523 to S525.

[0626] The implementation method for this step can be found in the above embodiment, and will not be repeated here.

[0627] S528, if the CPU determines that the current number of times loop 4 is processed is equal to the total number of times loop 4 is processed, then the loop ends to obtain G data subsets D, each of which includes P×P DCT coefficients.

[0628] For example, G data subsets D can be... Figure 12 The diagram shows G = 4 × 4 + 4 + 4 = 24 data subsets D. Each data subset D includes 8 × 8 DCT coefficients.

[0629] The implementation method for this step can be found in the above embodiment, and will not be repeated here.

[0630] It should be understood that the technical effects achieved by the above-mentioned inverse quantization process can be referred to the technical effects achieved by the quantization process in the above embodiments, and will not be repeated here.

[0631] The above embodiments describe the implementation of inverse quantization. The following embodiments describe the specific implementation of step S53, inverse discrete cosine transform, in the overall image decoding process.

[0632] Inverse Discrete Cosine Transform

[0633] For example, please refer to Figure 40 , Figure 40This is a schematic diagram of an inverse discrete cosine transform provided in an embodiment of this application. It illustrates the implementation of the inverse discrete cosine transform during JPEG decoding, including steps S531 to S539.

[0634] S531, the CPU changes the storage method of the G data subsets D in memory from row storage to column storage, or the CPU changes the storage method of the P×P DCT inverse transform coefficients in memory from row storage to column storage.

[0635] As illustrated in the examples above, the Discrete Cosine Transform (DCT) of each 8×8 color component (e.g., the Y component) in JPEG compression encoding can typically be represented as a matrix multiplication operation: Matrix C = Matrix A × Matrix B. Here, Matrix A is a matrix containing 8×8 color components (e.g., the Y component). Matrix B is a matrix containing 8×8 DCT transform coefficients, which are preset coefficients used for the DCT. Matrix C is a matrix containing the 8×8 DCT coefficients obtained by performing a DCT on an 8×8 Y component.

[0636] In JPEG decompression and decoding, the inverse discrete cosine transform (ICT) of each 8×8 DCT coefficient can also be represented as a matrix multiplication operation: Matrix A = Matrix C × Matrix B1. Here, matrix C is a matrix containing the 8×8 DCT coefficients, and matrix B1 is a matrix containing 8×8 inverse DCT transform coefficients, which are pre-defined coefficients used for the ICT. Matrix A is a matrix containing the 8×8 color components (e.g., the Y component) obtained by performing the ICT transform on an 8×8 DCT coefficient.

[0637] In the above embodiments, JPEG compression encoding requires converting matrix multiplication operations into the accumulation of vector outer products. That is, it requires performing an outer product accumulation operation on each column of matrix A with the corresponding row of matrix B, or on each row of matrix A with the corresponding column of matrix B. Similarly, in this embodiment, JPEG decoding also requires converting matrix multiplication operations into the accumulation of vector outer products. That is, it requires performing an outer product accumulation operation on each column of matrix C with the corresponding row of matrix B1, or on each row of matrix C with the corresponding column of matrix B1.

[0638] Therefore, based on the same reason stated in step S451 above for changing the storage method from row-based storage to column-based storage, this embodiment of the application needs to change the storage method of the G data subsets D in memory from row-based storage to column-based storage, or the CPU needs to change the storage method of the P×P DCT inverse transform coefficients in memory from row-based storage to column-based storage. The implementation method for changing the storage method can be found in step S451, and will not be repeated here.

[0639] S532, the CPU determines the number K5 of P×P DCT coefficients that can be loaded into at least one two-dimensional register based on the bit width of P×P DCT coefficients, the bit width of P×P DCT inverse transform coefficients, and the logic size of at least one two-dimensional register.

[0640] The difference between this step and step S452 is that in step S452, the CPU determines the number K2 of P×P color components 2 that can be loaded in at least one two-dimensional register based on the bit width of P×P color components 2, the bit width of P×P DCT transform coefficients, and the logical size of at least one two-dimensional register. In step S532, the CPU determines the number K5 of P×P DCT coefficients that can be loaded in at least one two-dimensional register based on the bit width of P×P DCT coefficients, the bit width of P×P DCT inverse transform coefficients, and the logical size of at least one two-dimensional register.

[0641] Since the bit width of P×P color components 2 is usually the same as the bit width of P×P DCT coefficients, for example, both are 8×8×8 bits = 512 bits, and the bit width of P×P DCT transform coefficients is usually the same as the bit width of P×P inverse DCT transform coefficients, for example, both are 512 bits, the number of P×P DCT coefficients K5 that can be loaded at one time by at least one two-dimensional register is the same as the number of P×P color components 2 K2 that can be loaded by at least one two-dimensional register. Therefore, the implementation method of step S532 can refer to the implementation method of step S452, and will not be repeated here.

[0642] S533, the CPU determines the total number of loop processes 5 based on the number of P×P DCT coefficients K5 that can be loaded in at least one two-dimensional register and the G data subsets D. Each loop process 5 is used to perform inverse discrete cosine transform on the K5 P×P DCT coefficients.

[0643] The implementation method for this step can refer to the implementation method of step S453.

[0644] The S534 CPU loads K5 P×P DCT coefficients from memory in parallel into at least one two-dimensional register, and loads P×P inverse DCT transform coefficients from memory in parallel into two-dimensional registers.

[0645] The implementation method for this step can refer to the implementation method of step S454.

[0646] The S535 CPU reads K5 P×P DCT coefficients in parallel from at least one two-dimensional register, and reads K5 P×P inverse DCT transform coefficients from the two-dimensional register.

[0647] The implementation method for this step can refer to the implementation method of step S455.

[0648] The S536 CPU processes K5 P×P DCT coefficients and K5 P×P inverse DCT transform coefficients in parallel to obtain K5 P×P color components.

[0649] In implementation, the CPU performs an outer product operation on the P rows of DCT coefficients from each of the K5 P×P DCT coefficients and the P columns of inverse DCT coefficients from each of the K5 P×P inverse DCT coefficients, and then sums the results to obtain P×P color components 2, thus obtaining K5 P×P color components 2. Specifically, the outer product operation is performed on each row of DCT coefficients and its corresponding column of inverse DCT coefficients. The outer product operation is performed in parallel on the P rows of DCT coefficients and the P columns of inverse DCT coefficients from each of the P×P DCT coefficients. The implementation method of this process can be referred to in step S456, and will not be elaborated here.

[0650] It should also be understood that when the data volume of K5 P×P DCT coefficients and K5 P×P inverse DCT transform coefficients is large, the large data volume leads to an uneven load on the multiple CPUs in a multi-core CPU electronic device. For example, in a multi-core CPU including CPU1 and CPU2, due to the large data volume, CPU1 processes 3 / 4 of the K5 P×P DCT coefficients and K5 P×P inverse DCT transform coefficients, while CPU2 processes 1 / 4, resulting in an uneven load.

[0651] To balance the load across multiple CPUs, in some embodiments, the K5 P×P DCT coefficients and K5 P×P inverse DCT transform coefficients can be divided into multiple data groups H15. The number of data groups H15 matches the number of CPUs. Each data group H15 includes at least P×P DCT coefficients and P×P inverse DCT transform coefficients, and the amount of data in each data group H15 is the same. Each CPU can process at least one data group H15 in parallel to obtain K5 P×P color components. Multiple CPUs process multiple data groups H15 in parallel, with each CPU processing the same amount of data. The implementation details of this process can be found in the embodiments described above and will not be repeated here.

[0652] S537, the CPU determines whether the current number of times loop 5 is processed is equal to the total number of times loop 5 is processed.

[0653] The implementation method for this step can be found in the above embodiment and will not be repeated here.

[0654] S538, if the CPU determines that the current number of times loop 5 is processed is less than the total number of times loop 5 is processed, then return to execute steps S534 to S536.

[0655] The implementation method for this step can be found in the above embodiment, and will not be repeated here.

[0656] S539, if the CPU determines that the current number of times loop 5 is processed is equal to the total number of times loop 5 is processed, then the loop ends to obtain Q data subsets A3, J data subsets B3 and L data subsets C3. Each data subset A3 includes P×P color components 21, each data subset B3 includes P×P color components 22, and each data subset C3 includes P×P color components 23.

[0657] The implementation method for this step can be found in the above embodiment and will not be repeated here.

[0658] It should be understood that the technical effects achieved by the inverse discrete cosine transform described above can be referred to the technical effects achieved by the discrete cosine transform in the above embodiments, and will not be repeated here.

[0659] The above embodiments describe the implementation of the inverse discrete cosine transform. The following embodiments describe the specific implementation of the color space conversion in step S55 of the overall image decoding process.

[0660] Color space conversion

[0661] For example, please refer to Figure 41 , Figure 41This is a schematic diagram of color space conversion provided in an embodiment of this application. It illustrates the implementation method of color space conversion during JPEG decoding, including steps S551 to S558.

[0662] S551, the CPU determines the number K6 of data groups H2 that can be loaded by at least one two-dimensional register based on the bit width of data group H2 and the logic size of at least one two-dimensional register. Data group H2 consists of a data subset A2, a data subset B2 corresponding to one data subset A2, and a data subset C2 corresponding to one data subset A2.

[0663] It should be understood that the above text Figure 14 The color space conversion in the illustrated embodiment is a conversion from the RGB color space to the YCbCr color space. Figure 41 The color space conversion shown is the conversion from the YCbCr color space to the RGB color space.

[0664] The meanings of data subset A2, data subset B2 corresponding to data subset A2, and data subset C2 corresponding to data subset A2 can be found in the explanation of data subset A1, data subset B1 corresponding to data subset A1, and data subset C1 corresponding to data subset A1 in S431. They will not be repeated here.

[0665] The implementation method for this step can be referred to the implementation method of step S431, and will not be repeated here.

[0666] S552, the CPU determines the total number of iterations of loop processing 6 based on the number K6 of data groups H2 that can be loaded by at least one two-dimensional register, and (M / P)×(N / P) data subsets A2, (M / P)×(N / P) data subsets B2, and (M / P)×(N / P). Each iteration of loop processing 6 is used to perform color space conversion on data subsets A2, B2, and C2 in the K6 data groups H2. Each data subset A2 includes P×P color components 21, each data subset B2 includes P×P color components 22, and each data subset C2 includes P×P color components 23.

[0667] The implementation method for this step can be referred to the above embodiment, and will not be repeated here.

[0668] In the S553, the CPU loads K6 data sets H2 from memory in parallel into at least one two-dimensional register.

[0669] The implementation method for this step can be referred to the above embodiment, and will not be repeated here.

[0670] The S554 CPU reads K6 data groups H2 in parallel from at least one two-dimensional register.

[0671] The implementation method for this step can be referred to the above embodiment, and will not be repeated here.

[0672] S555, the CPU performs parallel processing on K6 data groups H2, including K6 data subsets A2, B2, and C2, to obtain K6 data subsets A1, B1, and C1. Each data subset A1 includes P×P color components 11, each data subset B1 includes P×P color components 12, and each data subset C1 includes P×P color components 13.

[0673] In implementation, the CPU can substitute each Y component in the K6 data subsets A2, the Cb component corresponding to each Y component in the K6 data subsets B2, and the Cr component corresponding to each Y component in the K6 data subsets C2 into the following formula eleven for parallel calculation, to obtain the K6 data subsets A1, B1, and C1. Each data subset A1 includes 8×8 R components, each data subset B1 includes 8×8 G components, and each data subset C1 includes 8×8 Cb components.

[0674]

[0675] Among them, R i It refers to any R component in the K6 data subsets A1, G i B refers to any G component in the K6 data subsets B1, where B i It refers to any B component in the K6 data subset C1. i It refers to any Y component in the K6 data subset A2, Cb i It refers to any Cb component in the K6 data subset B2, Cr i It refers to any Cr component in the K6 data subset C2.

[0676] The implementation method of the above process can be referred to the implementation method of step S435, which will not be repeated here.

[0677] It should also be noted that when K6 data groups H2 are actually multiple data groups H2, the large amount of data in multiple data groups H2 may cause an imbalance in the load of the multiple CPUs in a multi-core CPU electronic device. For example, if the multi-core CPU includes CPU1 and CPU2, due to the large amount of data, CPU1 may process 3 / 4 of the data in multiple data groups H2, while CPU2 may process 1 / 4 of the data in multiple data groups H2, resulting in an unbalanced load.

[0678] To balance the load across multiple CPUs, in some embodiments, the K6 data groups H2 can be divided into multiple data groups H21. The number of data groups H21 matches the number of CPUs. Each data group H21 includes at least P×P Y components, P×P Cb components corresponding to the P×P Y components, and P×P Cr components corresponding to the P×P Y components. The amount of data in each data group H21 is the same. Each CPU can process at least one data group H21 in parallel to obtain K6 data subsets A1, B1, and C1. Multiple CPUs process multiple data groups H21 in parallel, with each CPU processing the same amount of data. The implementation of this process can be found in the embodiments described above, and will not be repeated here.

[0679] S556, the CPU determines whether the current number of times loop 6 is processed is equal to the total number of times loop 6 is processed.

[0680] The implementation method for this step can be found in the above embodiment, and will not be repeated here.

[0681] S557, if the CPU determines that the current number o...

Claims

1. A data processing method, characterized in that, Applied to electronic devices, the method includes: Obtain multiple initial datasets, each initial dataset comprising P×P initial data points, each initial data point comprising at least one of a first pixel value, a second pixel value, and DCT coefficients, wherein the first pixel value comprises at least one first color component of a first color space, the second pixel value comprises at least one second color component of a second color space, and the DCT coefficients characterize the frequency of the second color component, where P is an integer greater than 1. Encoding multiple initial datasets yields multiple target datasets, each target dataset comprising P×P target data points, wherein each target data point includes at least one of the second pixel value, the DCT coefficients, and quantized DCT coefficients; wherein the encoding process for each of the multiple initial datasets includes: At least P×P initial data in the initial dataset are encoded in parallel to obtain P×P target data in the target dataset. The encoding process includes at least one of first color space conversion processing, discrete cosine transform processing, and quantization processing.

2. The method according to claim 1, characterized in that, The electronic device is configured with a processor, the processor having at least one two-dimensional register, the two-dimensional register being composed of a plurality of one-dimensional registers; and the step of encoding at least a portion of P×P initial data in the initial dataset in parallel to obtain P×P target data in the target dataset includes: The processor reads P×P initial data in parallel from at least one of the two-dimensional registers; The processor performs parallel encoding processing on at least P×P initial data to obtain P×P target data.

3. The method according to claim 2, characterized in that, The electronic device is also configured with internal memory; and, prior to the parallel reading of P×P initial data from at least one of the two-dimensional registers by the processor, the method further includes: The processor loads P×P initial data from the internal memory into at least one of the two-dimensional registers in parallel.

4. The method according to any one of claims 1 to 3, characterized in that, The initial data is the first pixel value, the target data is the second pixel value, and the encoding process is a first color space conversion process; and, the parallel encoding process of at least P×P initial data in the initial dataset to obtain P×P target data in the target dataset includes: The first color space conversion is performed in parallel on the P×P first pixel values ​​in the initial dataset to obtain the P×P second pixel values ​​in the target dataset.

5. The method according to any one of claims 1 to 4, characterized in that, The initial data is the second color component, which is at least one of the second color components in the second pixel value. The target data is the DCT coefficients. The encoding process is discrete cosine transform processing. The P×P initial data are P×P second color components, and the data organization of the P×P second color components is a two-dimensional array, which includes P rows and P columns of the second color components. The parallel encoding process of at least a portion of the P×P initial data in the initial dataset to obtain the P×P target data in the target dataset includes: Perform parallel Discrete Cosine Transform (DCT) on the P rows of the second color components in the initial dataset to obtain P×P DCT coefficients in the target dataset; or, The second color component in column P of the initial dataset is subjected to discrete cosine transform in parallel to obtain P×P DCT coefficients in the target dataset.

6. The method according to claim 5, characterized in that, The step of performing a parallel discrete cosine transform on the P rows of the second color components in the initial dataset to obtain P×P DCT coefficients in the target dataset includes: Based on the P rows of the second color components and the P columns of DCT transform coefficients in the initial dataset, a vector outer product operation is performed in parallel, and the results of the vector outer product operation are accumulated to obtain P×P DCT coefficients. The P×P DCT transform coefficients are preset coefficients used for discrete cosine transform processing. The data organization of the P×P DCT transform coefficients is a two-dimensional array, including P rows and P columns of DCT transform coefficients. A vector outer product operation is performed on the second color components in each row and the corresponding column of DCT transform coefficients in each of the P columns. The step of performing a parallel discrete cosine transform on the second color components in columns P of the initial dataset to obtain P×P DCT coefficients in the target dataset includes: Based on the second color component in column P and the DCT transform coefficients in row P of the P×P DCT transform coefficients in the initial dataset, a vector outer product operation is performed in parallel, and the results of the vector outer product operation are accumulated to obtain P×P DCT coefficients. In this case, the vector outer product operation is performed on the second color component in each column and the DCT transform coefficients in row P that correspond to the second color component in each row.

7. The method according to claim 6, characterized in that, The electronic device includes a processor and an internal memory. The P×P second color components are stored in the internal memory in a row-by-row manner, and the P×P DCT transform coefficients are also stored in the internal memory in a row-by-row manner. And, before performing a vector outer product operation in parallel on the DCT transform coefficients in the P rows of the second color component and the P columns of the P×P DCT transform coefficients in the initial dataset, and before accumulating the results of the vector outer product operation to obtain the P×P DCT coefficients, the method further includes: The processor converts the storage method of the P×P DCT transform coefficients in the internal memory from row-based storage to column-based storage; and... Before performing a vector outer product operation in parallel on the second color components in column P of the initial dataset and the DCT transform coefficients in row P of the P×P DCT transform coefficients, and before summing the results of the vector outer product operation to obtain the P×P DCT coefficients, the method further includes: The processor converts the storage method of the P×P second color components in the internal memory from row storage to column storage.

8. The method according to any one of claims 1 to 7, characterized in that, The initial data are the DCT coefficients, the target data are the quantized DCT coefficients, and the encoding process is quantization processing; and the parallel encoding processing of at least P×P initial data in the initial dataset to obtain P×P target data in the target dataset includes: The P×P DCT coefficients in the initial dataset are quantized in parallel to obtain the P×P quantized DCT coefficients in the target dataset.

9. The method according to any one of claims 1 to 8, characterized in that, The process of encoding multiple initial datasets to obtain multiple target datasets includes: Based on multiple encoding processes, multiple initial datasets are encoded into multiple target datasets. Each encoding process is used to encode a portion of the initial datasets in parallel into a portion of the target datasets.

10. The method according to claim 9, characterized in that, The process of encoding multiple initial datasets into multiple target datasets based on multiple encoding processes includes: Determine the amount of the initial dataset that can be processed in each encoding process; The total number of encoding processes is determined based on the number of portions of the initial dataset that can be processed in each encoding process and the multiple initial datasets. Based on the total number of encoding processes, the encoding of multiple initial datasets is performed cyclically to obtain multiple target datasets; wherein each cyclic processing includes: The initial dataset is encoded in parallel to obtain a portion of the target dataset.

11. The method according to any one of claims 1 to 10, characterized in that, The electronic device includes multiple processors; and the encoding process of the multiple initial datasets to obtain multiple target datasets includes: The multiple initial datasets are divided into multiple first data groups, each first data group including at least one of the initial datasets, the number of first data groups matching the number of processors, and the amount of data in each first data group being the same; Multiple processors encode multiple first data groups in parallel to obtain multiple target datasets. Each processor encodes at least one first data group, and each processor processes the same number of first data groups.

12. The method according to any one of claims 2 to 11, characterized in that, The logical size of the two-dimensional register is variable.

13. A data processing method, characterized in that, Applied to electronic devices, the method includes: Acquire multiple target datasets, the target datasets including P×P target data, the target data including at least one of second pixel value, DCT coefficient and quantized DCT coefficient, wherein the second pixel value includes at least one second color component of the second color space, the DCT coefficient represents the frequency of the second color component, and P is an integer greater than 1; Decoding multiple target datasets yields multiple initial datasets, each initial dataset comprising P×P initial data points. Each initial data point includes at least one of a first pixel value, a second pixel value, and DCT coefficients. The first pixel value includes at least one first color component of a first color space. The process of decoding each of the multiple target datasets includes: At least P×P target data in the target dataset are decoded in parallel to obtain P×P initial data in the initial dataset. The decoding process includes at least one of second color space conversion processing, inverse discrete cosine transform processing, and inverse quantization processing.

14. The method according to claim 13, characterized in that, The electronic device is equipped with a processor, the processor having at least one two-dimensional register, the two-dimensional register being composed of a plurality of one-dimensional registers; and the parallel decoding processing of at least a portion of P×P target data in the target dataset to obtain P×P initial data in the initial dataset includes: The processor reads P×P target data in parallel from at least one of the two-dimensional registers; The processor performs parallel decoding on at least P×P of the target data to obtain P×P of the initial data.

15. The method according to claim 14, characterized in that, The electronic device is further configured with an internal memory; and, before the processor performs parallel decoding processing on at least a portion of the P×P target data to obtain P×P initial data, the method further includes: The processor loads P×P target data in parallel from the internal memory.

16. The method according to any one of claims 13 to 15, characterized in that, The target data is the quantized DCT coefficients, the initial data is the DCT coefficients, and the decoding process is inverse quantization; and the parallel decoding process of at least P×P target data in the target dataset to obtain P×P initial data in the initial dataset includes: The P×P quantized DCT coefficients in the target dataset are dequantized to obtain the P×P DCT coefficients in the initial dataset.

17. The method according to any one of claims 13 to 16, characterized in that, The target data is the DCT coefficients, the initial data is at least one of the second color components in the second pixel value, the decoding process is inverse discrete cosine transform processing, the P×P target data are P×P DCT coefficients, the data organization form of the P×P DCT coefficients is a two-dimensional array, the two-dimensional array includes the DCT coefficients in P rows and P columns; And, the step of decoding at least P×P target data in the target dataset in parallel to obtain P×P initial data in the initial dataset includes: The DCT coefficients in the P rows of the target dataset are subjected to inverse discrete cosine transform in parallel to obtain the P×P second color components in the initial dataset. or, The DCT coefficients in columns P of the target dataset are subjected to inverse discrete cosine transform in parallel to obtain P×P second color components in the initial dataset.

18. The method according to claim 17, characterized in that, The process of performing inverse discrete cosine transform on the P rows of DCT coefficients in the target dataset in parallel to obtain P×P second color components in the initial dataset includes: Based on the P rows of DCT coefficients and P columns of inverse DCT transform coefficients in the target dataset, a vector outer product operation is performed in parallel, and the results of the vector outer product operation are accumulated to obtain P×P second color components. The P×P inverse DCT transform coefficients are preset coefficients used for inverse discrete cosine transform processing. The P×P inverse DCT transform coefficients are organized as a two-dimensional array, including P rows and P columns of inverse DCT transform coefficients. A vector outer product operation is performed between each row of DCT coefficients and one column of inverse DCT transform coefficients corresponding to each row of DCT coefficients in the P columns. The DCT coefficients in columns P of the target dataset are subjected to inverse discrete cosine transform in parallel to obtain P×P second color components in the initial dataset, including: Based on the DCT coefficients in column P and the inverse DCT transform coefficients in row P of the P×P inverse DCT transform coefficients in the target dataset, a vector outer product operation is performed in parallel, and the results of the vector outer product operation are accumulated to obtain P×P second color components. In this case, the vector outer product operation is performed on the DCT coefficients in each column and the inverse DCT transform coefficients in row P that correspond to the DCT coefficients in each row.

19. The method according to claim 18, characterized in that, The electronic device includes a processor and an internal memory. The P×P DCT coefficients are stored in the internal memory in a row-by-row manner, and the P×P inverse DCT transform coefficients are also stored in the internal memory in a row-by-row manner. And, before performing a vector outer product operation on the inverse DCT transform coefficients in P rows of the target dataset and P columns of the P×P inverse DCT transform coefficients in parallel, and before accumulating the results of the vector outer product operation to obtain P×P second color components, the method further includes: The processor converts the storage method of the P×P inverse DCT transform coefficients in the internal memory from row-based storage to column-based storage; and... Before performing a vector outer product operation on the inverse DCT transform coefficients in rows P of the P columns of the DCT coefficients and P×P inverse DCT transform coefficients in the target dataset, and before summing the results of the vector outer product operation to obtain P×P second color components, the method further includes: The processor converts the storage method of the P×P DCT coefficients in the internal memory from row storage to column storage.

20. The method according to any one of claims 13 to 19, characterized in that, The target data is the second pixel value, the initial data is the first pixel value, and the decoding process is a second color space conversion process; and, the parallel decoding process of at least P×P target data in the target dataset to obtain P×P initial data in the initial dataset includes: The second color space conversion is performed in parallel on the P×P second pixel values ​​in the target dataset to obtain the P×P first pixel values ​​in the initial dataset.

21. The method according to any one of claims 13 to 20, characterized in that, The encoding process of multiple target datasets yields multiple initial datasets, including: Based on multiple decoding processes, multiple target datasets are decoded into multiple initial datasets. Each decoding process is used to decode a portion of the multiple target datasets into a portion of the initial datasets in parallel.

22. The method according to claim 21, characterized in that, The process of decoding multiple target datasets into multiple initial datasets based on multiple decoding processes includes: Determine the number of portions of the target dataset that can be processed in each decoding process; The total number of decoding processes is determined based on the number of portions of the target dataset that can be processed in each decoding process and the multiple target datasets. Based on the total number of decoding processes, the decoding of multiple target datasets is performed cyclically to obtain multiple initial datasets; wherein each cyclic processing includes: The target dataset is decoded in parallel to obtain a portion of the initial dataset.

23. The method according to any one of claims 13 to 22, characterized in that, The encoding process of multiple target datasets yields multiple initial datasets, including: The multiple target datasets are divided into multiple second data groups, each second data group includes at least one of the target datasets, the number of second data groups matches the number of processors, and the amount of data in each second data group is the same; Multiple processors perform decoding processing on multiple second data groups in parallel to obtain multiple initial datasets. Each processor is used to decode at least one second data group, and each processor processes the same number of second data groups.

24. The method according to any one of claims 2 to 23, characterized in that, The logical size of the two-dimensional register is variable.

25. An electronic device, characterized in that, The electronic device includes: One or more processors, and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method as claimed in claims 1 to 12, or any one of claims 13 to 24.

26. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the one or more processors being configured to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1 to 12, or 13 to 24.

27. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on an electronic device, cause the electronic device to perform the method as claimed in any one of claims 1 to 12, or 13 to 24.

Images