A pixel sampling device, method and image compression system

Abstract

The application relates to the technical field of image processing, and discloses a pixel sampling device, a pixel sampling method and an image compression system. The device comprises a first address generator, a prediction adjustment unit, a private cache unit and a downsampling unit. The first address generator is used for initiating a read request to a target image in an external memory according to a preset initial address and a termination address. The read request is used for reading all target data blocks of the target image. The private cache unit is used for caching high-frequency data blocks and corresponding pixel frequency information in a historical image and a prediction period value. The prediction adjustment unit is used for predicting whether a current target data block is the same as the high-frequency data blocks in the private cache unit according to a current prediction period value, and determining the target data block or the high-frequency data blocks as a to-be-converted data block according to a prediction result. The prediction period value is determined based on a prediction result of predicting a previous target data block. The downsampling unit is used for performing color space conversion and chroma information downsampling on the to-be-converted data block to obtain a result data block. Based on the device, a hardware sampling process of image compression can be realized, the occupation of system resources is reduced, and the image compression efficiency can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a pixel sampling device, method and image compression system. BACKGROUND

[0002] The human eye is more sensitive to luminance information than to chrominance information, and more sensitive to low-frequency information than to high-frequency information. Image compression is based on the visual characteristics of the human eye, and data that the human eye is not sensitive to is reduced or removed. JPEG (Joint Photographic Experts Group) is a widely used image file format. The compression of JPEG is based on lossy compression. The process first converts the RGB chrominance information to the YCbCr color space. YCbCr is a derivative version of the YUV color space. In this color space, Y is the luminance component, Cb is the blue chrominance component, and Cr is the red chrominance component. In order to reduce the large amount of redundant information contained in the chrominance channel, the chrominance information of the image data is also down-sampled, usually from the original YUV 4:4:4 format to 4:2:2 or 4:2:0 format. Then, the luminance and chrominance information in the image spatial domain is transformed to the frequency domain by DCT (Discrete Cosine Transform), the luminance information is quantized in detail, the color information is quantized coarsely, the low-frequency information is quantized in detail, and the high-frequency information is quantized coarsely. Finally, the quantized results are variable-length encoded to achieve the purpose of compressing the picture and ensure that the picture quality perceived by the human eye does not decrease significantly.

[0003] The current implementation of JPEG is mostly software implementation. The color space conversion and sampling and DCT (Discrete Cosine Transform) processing are implemented based on the operating system. The current problem is that software execution of JPEG image compression occupies a large amount of CPU (Central Processing Unit) computing resources and occupies many software threads. In the case of tight computing resources, the computing efficiency is reduced. In particular, in fields with strong timeliness, such as AI computing, server cluster computing, supercomputing, etc., the computing resources are greatly consumed, and the efficiency of image compression processing is severely reduced. In addition, the optimization of the image frequency domain in the JPEG compression process is in the DCT stage after sampling, but a large amount of high-frequency data is still retained in the sampling stage. Frequent access to high-frequency data in the sampling process causes a great access pressure on the bus, occupies too many bus resources, and causes the delay to rise, resulting in low efficiency of image compression. SUMMARY

[0004] Therefore, the present application aims to provide a pixel sampling device, method and image compression system to reduce system resource occupation and improve the processing efficiency of image compression.

[0005] To achieve the above-mentioned purpose, the technical scheme of the present application is as follows:

[0006] The first aspect of the embodiment of the present application provides a pixel sampling device connected with an external memory through a system bus, comprising:

[0007] a first address generator, a prediction adjustment unit, a private cache unit and a down-sampling unit;

[0008] The first address generator is configured to initiate a read request to a target image in the external memory according to a preset initial address and a termination address; the read request is configured to read all target data blocks of the target image;

[0009] The private cache unit is configured to cache high-frequency data blocks and corresponding pixel frequency information in a history image, and a prediction period value;

[0010] The prediction adjustment unit is configured to predict whether a current target data block is the same as a high-frequency data block in the private cache unit according to a current prediction period value; and determine the target data block or the high-frequency data block as a to-be-converted data block according to a prediction result; the prediction period value is determined based on a prediction result of predicting a previous target data block;

[0011] The down-sampling unit is configured to perform color space conversion and chroma information down-sampling on the to-be-converted data block to obtain a result data block.

[0012] Optionally, the pixel sampling device further comprises at least one bus read interface, a bus arbitrator and a first cache module;

[0013] The first address generator is further configured to sequentially access each bus read interface and initiate a corresponding read request to the external memory;

[0014] The bus arbitrator is configured to adjust a write order of target data blocks returned by each bus read interface according to protocol handshake signals returned by each bus read interface; and sequentially write each target data block to the first cache module according to the adjusted write order;

[0015] The first cache module is configured to cache all target data blocks in the target image.

[0016] Optionally, the pixel sampling device further comprises a second cache module configured to cache to-be-converted data blocks;

[0017] Before predicting the pixel frequency information of the target data block according to the prediction period value, the prediction adjustment unit is further configured to perform the following steps: determining, according to a DCT feedback result of a previous historical image, high-frequency data blocks in the historical image, wherein a ratio of a maximum pixel frequency value to a minimum pixel frequency value in the high-frequency data blocks is greater than or equal to a first threshold value; adding a mark to all the high-frequency data blocks and storing the mark in the private cache unit, wherein the mark comprises a frame position, a storage address and a high-frequency label number;

[0018] The prediction of the pixel frequency information of the target data block according to the prediction period value specifically comprises:

[0019] Based on the high-frequency label number, the corresponding historical data block is obtained from the private cache unit;

[0020] The prediction period value is read from the private cache unit to determine whether the current prediction period value is greater than or equal to a second threshold value;

[0021] In the case that the prediction period value is greater than or equal to the second threshold value, the pixel frequency information of the target data block is predicted to be consistent with the historical data block;

[0022] The historical data block is written into the second cache module as a prediction result.

[0023] Optionally, the prediction adjustment unit is configured to determine a data block to be converted, and specifically comprises:

[0024] The target data block is compared with the historical data block to determine whether the pixel frequency information is consistent;

[0025] In the case that the pixel frequency information is consistent, the prediction data block stored in the second cache module is taken as the data block to be converted;

[0026] In the case that the pixel frequency information is inconsistent, the target data block is written into the second cache module as the data block to be converted, and the prediction data block is overwritten.

[0027] Optionally, the prediction adjustment unit is further configured to perform the following steps:

[0028] In the case that the pixel frequency information is consistent, the current prediction period value is increased by 1, and the target data block is added with the same high-frequency label number as the prediction data block;

[0029] In the case that the pixel frequency information is inconsistent, the current prediction period value is decreased by 1.

[0030] Optionally, the prediction adjustment unit is further configured to perform the following steps:

[0031] in case that the predicted period value is less than the second threshold, predicting that the target data block is inconsistent with the pixel frequency information of the historical data block;

[0032] writing the target data block directly into the second cache module.

[0033] Optionally, the prediction adjustment unit is further configured to adjust the access order of all the to-be-converted data blocks in the second cache module, so that the to-be-converted data block without the high-frequency tag number is accessed preferentially.

[0034] Optionally, the downsampling unit comprises:

[0035] a calculation module configured to perform color space conversion on the to-be-converted data block by using a binary algorithm to obtain to-be-sampled data; the calculation module comprises a shift register, an 8-bit multiplier, an 8-bit complement adder / subtracter and a register;

[0036] a sampling memory configured to buffer the to-be-sampled data, wherein each two to-be-sampled data are stored in a combined form;

[0037] a sampling module comprising a read operation module and a second address generator; the second address generator is configured to generate a plurality of first addresses based on the second size; the read operation module is configured to perform chroma information downsampling on the to-be-sampled data block in the sampling memory according to each first address to obtain a result data block.

[0038] According to a second aspect of the embodiment of the present application, a pixel sampling method is provided, based on the pixel sampling device provided in the first aspect of the embodiment of the present application, comprising:

[0039] initiating a read request to a target image in an external memory according to a pre-set initial address and a termination address by using the first address generator, and reading all target data blocks of the target image;

[0040] predicting whether the current target data block is the same as the high-frequency data block in the private cache unit according to the current prediction period value; and determining the target data block or the high-frequency data block as the to-be-converted data block according to the prediction result; the prediction period value is determined based on the prediction result of the previous target data block;

[0041] performing color space conversion and chroma information downsampling on the to-be-converted data block to obtain a result data block.

[0042] According to a third aspect of the embodiment of the present application, an image compression system is provided, comprising:

[0043] The pixel sampling device provided in the first aspect of the embodiments of the present application, wherein the downsampling unit further comprises a third cache module configured to store the result data block.

[0044] The compression unit is configured to acquire the result data block from the third cache module, and perform DCT transformation, quantization and encoding compression to obtain a compressed data block.

[0045] The DCT feedback result of all target data blocks of the target image obtained through DCT transformation is sent to a prediction adjustment module in the pixel sampling device.

[0046] According to the fourth aspect of the embodiments of the present application, a computer readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the pixel sampling method according to the second aspect of the embodiments of the present application are implemented.

[0047] According to the fifth aspect of the embodiments of the present application, an electronic device is provided, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the pixel sampling method according to the second aspect of the embodiments of the present application are implemented.

[0048] The pixel sampling device provided in the present application is connected with an external memory through a system bus. A first address generator initiates a read request to the external memory according to the storage address of the target image in the external memory, and reads the data of the target image in the form of a target data block. A prediction adjustment unit acquires the target data block, and predicts whether the current target data block is the same as the high-frequency data block in the last historical image stored in the private cache unit based on the current prediction period value. If the current target data block is the same as the high-frequency data block in the last historical image, the historical image data block in the private cache is replaced by the target data block and output to the downsampling data block for processing. Since the pixels in the image are continuous, in the case of multiple prediction hits (i.e., high prediction period value), the probability that the next target data block is the same as the high-frequency data block stored in the private cache is higher. Reading the data block from the private cache and inputting it to the downsampling unit can improve the timeliness of compressed image processing.

[0049] The present application realizes color space conversion and downsampling in hardware, thereby reducing the frequency of CPU accessing the bus and reducing the occupation of system resources. By predicting the next target data block in advance, in the case of prediction hit of high-frequency data, the downsampling unit can directly process the high-frequency data block in the private cache, thereby reducing the time delay caused by frequent access to the external memory through the system bus, improving the image sampling efficiency, and further improving the overall compression efficiency of the image. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0051] Figure 1 is a schematic diagram of a pixel sampling device according to an embodiment of the present application;

[0052] Figure 2 is a state jump schematic diagram of a state machine controller according to an embodiment of the present application;

[0053] Figure 3 is a work flow diagram of a prediction adjustment unit according to an embodiment of the present application;

[0054] Figure 4 is a schematic diagram of a down-sampling unit according to an embodiment of the present application;

[0055] Figure 5 is a flow chart of a pixel sampling method according to an embodiment of the present application;

[0056] Figure 6 is a schematic diagram of an image compression system according to an embodiment of the present application. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0058] It should be understood that the "one embodiment" or "an embodiment" mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily mean the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner.

[0059] In various embodiments of the present application, it should be understood that the size of the serial number of the following processes does not mean the execution order, and the execution order of the processes should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0060] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The description of the exemplary embodiments is intended to apply to any embodiment of the application, unless specified otherwise. It is noted that the same elements in different drawings are denoted by the same reference numerals. The following exemplary embodiments described are not representative of all embodiments consistent with the present application. Rather, they are merely examples of apparatuses and methods consistent with some aspects as detailed herein.

[0061] It should be noted that the embodiments and features of the embodiments in the present application can be combined with each other without conflict.

[0062] The present application realizes an efficient pixel sampling scheme for JPEG by hardware modeling with Verilog. The present embodiment is based on FPGA (Field-Programmable Gate Array) and describes the implementation process of the RGB image compression JPEG scheme. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0063] Figure 1 is a schematic diagram of a pixel sampling device according to an embodiment of the present application. The device is connected to an external memory through a system bus and includes a first address generator 101, a prediction adjustment unit 102, a private cache unit 103, and a down-sampling unit 104.

[0064] The first address generator 101 is configured to initiate a read request to a target image in the external memory according to a pre-set initial address and a termination address; the read request is configured to read all target data blocks of the target image.

[0065] The private cache unit 103 is configured to cache high-frequency data blocks and corresponding pixel frequency information in a history image, and a prediction period value.

[0066] The prediction adjustment unit 102 is configured to predict whether a current target data block is the same as a high-frequency data block in the private cache unit according to a current prediction period value; and determine the target data block or the high-frequency data block as a to-be-converted data block according to a prediction result; the prediction period value is determined based on a prediction result of predicting a previous target data block.

[0067] The down-sampling unit 104 is configured to perform color space conversion on the to-be-converted data block, and down-sample chroma information to obtain a result data block.

[0068] In the embodiment, the pixel sampling device is modeled based on verilog. The 32-bit RGB image data is stored in the external memory in advance. The pixel sampling device generates a read request through the first address generator and accesses the external memory through the system bus to request reading of the target image data in the external memory. In the embodiment, the target image data is RGB image data. The target data block is read in the form of an 8x8 size data block (i.e., a target data block) and is predicted by the prediction adjustment unit. The feedback result of the DCT stage of the previous image (historical image) is stored in the private cache unit. The prediction adjustment unit predicts the target data block according to the information of the high-frequency data block in the previous image and the current prediction period value, and predicts whether the current target data block is the same as the high-frequency data block in the previous historical image stored in the private cache unit. If the current target data block is the same as the high-frequency data block in the previous historical image, the historical image data block in the private cache is replaced with the target data block and is output to the down-sampling data block for processing. The prediction period value changes according to the prediction result of each prediction of different data blocks. Since the pixels in the image are continuous, the prediction result of the previous data block will affect the prediction period value and further affect the current data block prediction result.

[0069] After the high-frequency data block in the target image is predicted by the prediction adjustment unit, in the case of a prediction hit (i.e., correct prediction) of the target data block, the down-sampling unit can directly process the corresponding high-frequency data block in the historical image stored in the private cache unit, without reading the target data block from the external memory each time, thereby reducing the bus read-write frequency of accessing the high-frequency data, improving the efficiency of sampling the target image, and improving the timeliness of compressing the JPRG.

[0070] As an embodiment of the present application, the pixel sampling device further comprises at least one bus read interface, a bus arbiter, and a first cache module.

[0071] The first address generator is further configured to sequentially access each bus read interface and initiate a corresponding read request to the external memory.

[0072] The bus arbiter is configured to adjust the write order of the target data blocks returned by each bus read interface according to the protocol handshake signals returned by each bus read interface, and sequentially write each target data block to the first cache module according to the adjusted write order.

[0073] The first cache module is configured to cache all target data blocks in the target image.

[0074] In the embodiment, the running state of each module in the pixel sampling device is controlled by the state machine controller. Figure 2is a state jump schematic diagram of the state machine controller in an embodiment of the present application. As shown in Figure 2 The state machine mainly has eight states, which are as follows:

[0075] (1) In the initialization address stage, the state machine controller controls the first address generator to configure the initial address and the terminal address of the internal image storage position. After the configuration is completed, the work is enabled, and the next state is jumped to;

[0076] (2) In the pixel data reading stage, the read address and the read request are generated through the first read address generator, and the read data (RGB image data) is obtained through the bus. In the embodiment, the system bus is based on the AMBA bus protocol, and the interface adopts the AHB bus interface protocol part in the AMBA protocol to ensure the communication compatibility of modeling. After the read request is sent, the next state is jumped to;

[0077] (3) The RGB image data is written into the first cache module (FIFO-0) in the form of a block (data block) to wait for reading, and after the target data block cache is completed, the next state is jumped to;

[0078] (4) In the high-frequency prediction stage, the prediction adjustment unit performs high-frequency prediction on the target data block. If it is predicted that the target data block is the same as the high-frequency data block in the private cache unit, the target data block is marked, and the high-frequency data block that has been cached is replaced with the target data block and written into the second cache module (FIFO-1) as a to-be-converted data block for subsequent processing. In addition, the prediction adjustment unit also adjusts the order of the data blocks processed by the downsampling unit, and processes the data blocks with balanced pixel frequencies preferentially, and arranges the high-frequency data blocks at the back. After the adjustment is completed, the next state is jumped to;

[0079] (5) The second cache module stores all to-be-converted data blocks that need to be color space converted and downsampled, and sends them to the downsampling module in the order of the cache queue. After all the data blocks are sent, the next state is jumped to;

[0080] (6) The downsampling module performs color space conversion on the to-be-converted data block to obtain YCbCr data. After the color space conversion is completed, the next state is jumped to;

[0081] (7) The downsampling module performs chroma information downsampling operation on the YCbCr data to remove part of the chroma information and reduce the storage space occupied by the image data. After the downsampling is completed, the next state is jumped to;

[0082] (8) After the downsampling is completed, the YCbCr image data is output.

[0083] The number of bus read interfaces for accessing the external memory in the embodiment can be set according to actual conditions, and four bus read interfaces (interface 0, interface 1, interface 2, and interface 3) are taken as an example for description in the embodiment. The first address generator sequentially accesses the system bus read interfaces based on the initial address accumulation mode, sends a read request to the external memory, and the external memory returns response data (i.e., the target image). In an embodiment, when multiple read requests or read data occur simultaneously, the execution sequence is arbitrated by the bus arbitration module.

[0084] In the embodiment, the bus arbiter arbitrates the order of writing the target data block into the first cache module (FIFO-0) according to the protocol handshake signals fed back by each bus read interface. Specifically, a polling arbitration mode is adopted, the return ID in the data responded by the external memory is read, and the target data block is sequentially written into the first cache module according to the order of sending the read request. In the embodiment, the first cache module is built based on the BLOCK RAM device of FPGA, and internally includes a write data control module, a read data control module, a Gray code conversion module, a Gray code synchronization module, a full / empty signal generation module, and a data storage module (dual-port RAM). The parameter characteristics of the first cache module are: 32 bits of bit width, 4096 of depth, 1 read interface, and 1 write interface.

[0085] In the case where multiple bus read interfaces simultaneously have response data, the bus arbiter arbitrates the writing order of each response data, so that the target data block is sequentially written into the first cache unit according to the requested order, ensuring that the processing order of the target data block is correct and avoiding errors in the subsequent processing process.

[0086] As an embodiment of the present application, the pixel sampling device further includes a second cache module for caching the data block to be converted.

[0087] Before predicting the pixel frequency information of the target data block according to the prediction period value, the prediction adjustment unit is further configured to perform the following steps: determining a high-frequency data block in the historical image according to the DCT feedback result of the last historical image; the ratio of the maximum pixel frequency to the minimum pixel frequency in the high-frequency data block is greater than or equal to a first threshold value; adding a mark to all high-frequency data blocks and storing the mark in the private cache unit; the mark includes a frame position, a storage address, and a high-frequency label number;

[0088] The prediction of the pixel frequency information of the target data block according to the prediction period value specifically includes:

[0089] Based on the high-frequency label number, the corresponding historical data block is obtained from the private cache unit.

[0090] reading a prediction period value from the private cache unit, and determining whether the current prediction period value is greater than or equal to a second threshold value;

[0091] in the case where the prediction period value is greater than or equal to the second threshold value, predicting that the pixel frequency information of the target data block is consistent with that of the historical data block;

[0092] writing the historical data block as a prediction result into the second cache module.

[0093] Figure 3 is a work flow chart of a prediction adjustment unit in an embodiment of the present application. As shown in Figure 3 the prediction adjustment unit in the embodiment acquires feedback results of a previous historical image in a DCT stage before predicting a current target data block, and adjusts the processing mode of the data block according to the frequency domain feedback results of the DCT stage of JPEG. The private cache unit is responsible for recording the pixel frequency evaluation of each data block in the picture, and marking the corresponding position and address of the picture. Specifically, high frequency data blocks are selected from the historical picture, and the steepness of the data block is determined according to the appearance frequency of the pixel with the highest appearance frequency in the data block and the appearance frequency of the pixel with the lowest appearance frequency, i.e., whether it is a high frequency data block is determined. In the embodiment, the first threshold value is used to determine whether it is a high frequency data block. If the ratio of the pixel frequency with the highest frequency to the pixel frequency with the lowest frequency in the data block is greater than or equal to the first threshold value, it is determined to be a high frequency data block. In actual application, the first threshold value of the prediction period value can be set according to the demand, for example, set to 3.

[0094] The high frequency data block is stored in the private cache space and marked. The marking content and data structure are shown in Table 1, including the frame position of the data block in the image to which it belongs, the storage space address of the data block and the corresponding high frequency label number.

[0095] Table 1

[0096] Frame position Storage space address High frequency tag number

[0097] Among them, the high frequency label number occupies 0-7 bits; the storage space address occupies 8-39 bits; and the frame position occupies 40-55 bits.

[0098] In the prediction of the target data block, the prediction period value is read, and if the prediction period value is greater than or equal to the second threshold value, it indicates that the current target data block is consistent with the pixel frequency information of the corresponding high-frequency data block stored in the private cache unit. On this basis, the corresponding historical data block is read from the private cache unit as the prediction result according to the high-frequency tag number, and the cached data block is used to replace the target data block and written into the second cache module, thereby accelerating the sampling efficiency of the image and reducing the delay caused by frequent access to the bus to obtain the target data block. For example, in the case that the last target data block hits the historical data block of the high-frequency tag number 2, and the prediction period value is higher than the second threshold value, the current target data block is likely to hit the historical data block of the high-frequency tag number 3. Therefore, the corresponding historical data block is found based on the high-frequency tag number 3 as the prediction result of the current target data block.

[0099] As an embodiment of the present application, the prediction adjustment unit is used to determine the to-be-converted data block, and specifically includes:

[0100] Comparing the target data block with the historical data block to determine whether the pixel frequency information is consistent;

[0101] In the case that the pixel frequency information comparison is consistent, the prediction data block stored in the second cache module is taken as the to-be-converted data block;

[0102] In the case that the pixel frequency information comparison is inconsistent, the target data block is written into the second cache module as the to-be-converted data block, and the prediction data block is overwritten.

[0103] In the embodiment, after the target data block prediction hits the high-frequency data block and the prediction result is output, in order to ensure the correctness of the target image processed by the downsampling module, the data block of the prediction result is compared with the real target data block, so as to verify whether the prediction result is correct.

[0104] In the embodiment, after the prediction result is written into the second cache module, the target data block is compared with the prediction result (historical data block), so as to verify whether the prediction result of the target data block is correct. In the case that the pixel frequency information comparison of the two is consistent, it is determined that the prediction result is correct, that is, the prediction result (prediction data block) written into the second cache module can be directly used for processing by the downsampling module. According to the high-frequency tag number of the prediction data block, the target data block is also marked, and is combined with the target data block and written into the private cache unit.

[0105] If the pixel frequency information comparison is inconsistent, it indicates a prediction error, in which case the target data block is written into the second cache module for refreshing, to overwrite the prediction result (prediction data block) in the second cache module, to ensure the correctness of the image data processed by the subsequent downsampling module. In this embodiment, the efficiency of downsampling is improved by predicting the target data block in advance, and the prediction result is verified after being written into the second cache module, to ensure the correctness of the sampled image data.

[0106] As an embodiment of the present application, the prediction adjustment unit is further configured to perform the following steps:

[0107] In the case of consistent pixel frequency information comparison, the current prediction period value is increased by 1; the target data block is added with the same high-frequency tag number as the prediction data block;

[0108] In the case of inconsistent pixel frequency information comparison, the current prediction period value is decreased by 1.

[0109] In this embodiment, the prediction period value is adjusted based on the comparison result of each prediction result and the real data block, to improve the correctness of the prediction result when predicting the next data block, to reduce the number of repeated writing and overwriting of the second cache unit, and to further improve the image sampling efficiency.

[0110] Specifically, in the case of consistent pixel frequency information comparison, i.e. the prediction result of the target data block is correct, the current prediction period value is increased by 1, to increase the hit probability when predicting the next target data block. That is, the probability of predicting the next data block is improved, which indicates that in the case of continuous multiple prediction hits of high-frequency data, the next data block is more likely to be high-frequency data.

[0111] Correspondingly, in the case of inconsistent pixel frequency information comparison, i.e. the prediction result of the target data block is incorrect, the current prediction period value is decreased by 1, to decrease the hit probability when predicting the next target data block. That is, the probability of predicting the next data block is reduced, which indicates that after the current prediction error, the next target data block has a certain probability of not being high-frequency data.

[0112] As an embodiment of the present application, the prediction adjustment unit is further configured to perform the following steps:

[0113] In the case that the prediction period value is less than the second threshold value, it is predicted that the pixel frequency information of the target data block is inconsistent with the historical data block;

[0114] The target data block is directly written into the second cache module.

[0115] In the embodiment, when the predicted period value does not reach the second threshold value, it indicates that the prediction of the data block before the current target data block does not hit, and due to the continuity of pixels in the image, the probability of the current target data block hitting high-frequency data is small. In this case, the prediction of the pixel frequency information of the target data block and the historical data block is inconsistent, that is, the current target data block is not high-frequency data, and therefore the target data block is normally written into the second cache module and waits for the processing of the downsampling module.

[0116] As an embodiment of the present application, the prediction adjustment unit is further configured to adjust the access order of all the data blocks to be converted in the second cache module, so that the data blocks without the high-frequency label number are preferentially accessed.

[0117] In the embodiment, the prediction adjustment unit further adjusts the reading order of each data block according to whether the target data block in the second cache module has the high-frequency label number. Specifically, the data blocks to be converted with the high-frequency label number (high-frequency data) are arranged at the rear, and the data blocks without the high-frequency label number are preferentially processed. Since the high-frequency data needs to be removed in the subsequent DCT conversion stage in the JPRG image compression process, by adjusting the processing order, the subsequent centralized processing of the high-frequency data can be facilitated, and the processing efficiency is further improved.

[0118] As an embodiment of the present application, the downsampling unit comprises:

[0119] The calculation module is configured to perform color space conversion on the data block to be converted by using a binary algorithm to obtain the data to be sampled; the calculation module comprises a shift register, an 8-bit multiplier, an 8-bit complement adder / subtracter and a register;

[0120] The sampling memory is configured to buffer the data to be sampled, wherein each two data to be sampled are stored in a combined form.

[0121] The sampling module comprises a read operation module and a second address generator; the second address generator is configured to generate a plurality of first addresses based on the second size; and the read operation module is configured to perform chroma information downsampling on the data block to be sampled in the sampling memory according to each first address to obtain a result data block.

[0122] In the JPEG compression process, conversion from the RGB color space to the YCbCr color space needs to use the following conversion formula:

[0123] Y = 0.299000R + 0.587000G + 0.114000B

[0124] Cb = -0.168736R - 0.331264G + 0.500002B

[0125] Cr = 0.500000R - 0.418688G - 0.081312B

[0126] wherein Y is the luminance component, Cb is the blue chroma component, Cr is the red chroma component, R is the red component in the RGB image, G is the green component in the RGB image, and B is the blue component in the RGB image. In current software compression schemes, floating point calculation and subtraction calculation are involved. Since a floating point number is represented by 32 bits, a large amount of floating point calculation will cause excessive system power consumption, greatly occupy computing resources and result in low image compression efficiency. In the embodiment, in order to further reduce the occupation of computing resources and improve the sampling efficiency, the downsampling unit is implemented based on a binary hardware mode to realize color space conversion and downsampling.

[0127] Figure 4 is a schematic diagram of the downsampling unit in an embodiment of the present application. As shown in Figure 4 , the downsampling unit comprises:

[0128] a shift register comprising calculation parameters (such as the coefficient "0.299000" in the conversion formula described above) for inputting calculation of Y component, Cb component and Cr component;

[0129] 3 sets of calculation modules, each set of calculation modules comprising: 3 eight-bit binary multipliers (whose inputs are the arrays of R, G and B components of the data block respectively), 3 eight-bit complement adders and 3 registers, for realizing YCbCr color space conversion;

[0130] a sampling memory (RAM) for storing the converted YCbCr data;

[0131] a second address generator for generating the first address of the upper left corner of each block according to the converted YCbCr data in the size of 2x2 block, and a read operation module for reading YCbCr data from the RAM for downsampling according to the data block address generated by the second address generator. In the embodiment, 4:2:0 mode is used for downsampling to reduce the storage space occupied by the image as much as possible.

[0132] The downsampling unit further comprises an output buffer module, which is a FIFO buffer for buffering the image data after downsampling.

[0133] In the embodiment, the downsampling unit realizes the color space conversion and chroma information downsampling operation based on the eight-bit binary calculation mode. Compared with the picture compression scheme using software to perform 32-bit floating point calculation, the present scheme greatly reduces the occupation of computing resources, reduces the computing power consumption and improves the image compression efficiency.

[0134] In an embodiment, the second address generator can generate a corresponding first address data according to different data block sizes. It is worth noting that the size of the data block needs to be able to divide the size of 2x2. For example, the second address generator can be configured to generate a first address based on a data block size of 4x4 size, and the read operation module reads YCbCr data of a corresponding size from the RAM based on the size. Taking the sizes of 2x2 and 4x4 as examples, by appropriately expanding the data block size on which the second address generator generates the first address, the frequency of the read operation module accessing the RAM is reduced, the system resource occupation is further reduced, and the image sampling efficiency is improved.

[0135] Based on the same inventive concept, an embodiment of the present application provides a pixel sampling method. Referring to Figure 5 , Figure 5 is a flowchart of the pixel sampling method according to an embodiment of the present application. As shown in Figure 5 , the method comprises:

[0136] S1: initiating a read request to a target image in an external memory through a first address generator according to a pre-set initial address and a termination address, and reading all target data blocks of the target image;

[0137] S2: predicting whether a current target data block is the same as a high-frequency data block in a private cache unit according to a current prediction period value, and determining the target data block or the high-frequency data block as a to-be-converted data block according to a prediction result; the prediction period value is determined based on a prediction result of predicting a previous target data block;

[0138] S3: performing color space conversion and chroma information downsampling on the to-be-converted data block to obtain a result data block.

[0139] As an embodiment of the present application, reading all target data blocks of the target image comprises:

[0140] accessing each bus read interface in sequence through the first address generator, and initiating a corresponding read request to the external memory;

[0141] adjusting the write order of the target data blocks returned by each bus read interface according to a protocol handshake signal returned by each bus read interface; and writing each target data block into the first cache module in sequence according to the adjusted write order.

[0142] As an embodiment of the present application, before predicting whether the current target data block is the same as the high-frequency data block in the private cache unit, it further comprises:

[0143] determining high-frequency data blocks in the historical image according to a DCT feedback result of a previous historical image, wherein a ratio of a maximum pixel frequency to a minimum pixel frequency in the high-frequency data blocks is greater than or equal to a first threshold value;

[0144] adding a mark to all the high-frequency data blocks and storing the mark in the private cache unit, wherein the mark comprises a frame position, a storage address and a high-frequency label number.

[0145] As an embodiment of the present application, predicting whether the target data block is identical to the high-frequency data block in the private cache unit comprises:

[0146] acquiring a corresponding historical data block from the private cache unit based on the high-frequency label number;

[0147] reading a prediction period value from the private cache unit and judging whether the current prediction period value is greater than or equal to a second threshold value;

[0148] in a case where the prediction period value is greater than or equal to the second threshold value, predicting that pixel frequency information of the target data block is identical to that of the historical data block;

[0149] writing the historical data block as a prediction result into a second cache module.

[0150] As an embodiment of the present application, determining the target data block or the high-frequency data block as a data block to be converted according to the prediction result comprises:

[0151] comparing the target data block with the historical data block to judge whether the pixel frequency information is identical;

[0152] in a case where the pixel frequency information is identical, storing a prediction data block in the second cache module as the data block to be converted;

[0153] in a case where the pixel frequency information is not identical, writing the target data block as the data block to be converted into the second cache module and covering the prediction data block.

[0154] As an embodiment of the present application, the method further comprises:

[0155] in a case where the pixel frequency information is identical, increasing the current prediction period value by 1 and adding the same high-frequency label number to the target data block as to the prediction data block;

[0156] in a case where the pixel frequency information is not identical, decreasing the current prediction period value by 1.

[0157] As an embodiment of the present application, the method further comprises:

[0158] If the predicted period value is less than the second threshold, the predicted pixel frequency information of the target data block is inconsistent with that of the historical data block;

[0159] The target data block is written directly to the second cache module.

[0160] As one embodiment of this application, the method further includes:

[0161] Adjust the access order of all data blocks to be converted in the second cache module so that data blocks without high-frequency tag numbers are accessed first.

[0162] As one embodiment of this application, the data block to be converted is subjected to color space conversion and chromaticity information downsampling to obtain a result data block, including:

[0163] The calculation module uses a binary algorithm to perform color space conversion on the data block to be converted, obtains the data to be sampled, and writes it into the sampling memory;

[0164] Multiple starting addresses are generated using a second address generator based on a second size;

[0165] The sampling module downsamples the chromaticity information of the data block to be sampled in the sampling memory according to each starting address to obtain the result data block.

[0166] Based on the same inventive concept, one embodiment of this application provides an image compression system. (Reference) Figure 6 , Figure 6 This is a schematic diagram of an image compression system proposed in an embodiment of this application. Figure 6 As shown, the system includes:

[0167] As provided in the pixel sampling device in the above embodiments, the downsampling unit further includes a third cache module for storing the result data block;

[0168] The compression unit is used to obtain the result data block from the third cache module, and perform DCT transformation, quantization and encoding compression to obtain a compressed data block;

[0169] The DCT feedback results of all target data blocks of the target image obtained by DCT transformation are sent to the prediction adjustment module in the pixel sampling device.

[0170] In this embodiment, the first buffer module of the pixel sampling device ( Figure 6 FIFO-0), second cache module ( Figure 6 FIFO-1 and the third cache module ( Figure 6 Both FIFO-2 and FIFO-2 adopt the FIFO structure.

[0171] Based on the same inventive concept, one embodiment of the present application provides a readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of the pixel sampling method according to any one of the above embodiments of the present application.

[0172] Based on the same inventive concept, one embodiment of the present application provides an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the pixel sampling method according to any one of the above embodiments of the present application when executing the computer program.

[0173] As to the method in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments of the device, and will not be described in detail here.

[0174] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0175] For the method embodiments, in order to simply describe, they are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the action sequence described, because according to the present application, some steps can be performed in other sequences or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and components involved are not necessarily required by the present application.

[0176] Those skilled in the art should know that the embodiments of the present application can be provided as a method, device, or computer program product. Therefore, the embodiments of the present application can be in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0177] The embodiments of the present application are described with reference to the flowchart illustrations and / or block diagrams of the methods, terminal devices (systems) and computer program products according to the embodiments of the present application. It is understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing terminal devices to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal devices, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0178] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal devices to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0179] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal devices, such that a series of operational steps are performed on the computer or other programmable terminal devices to create a computer implemented process so that the instructions executed on the computer or other programmable terminal devices provide steps for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0180] Although the preferred embodiments of the present application have been described, those skilled in the art will be able to make additional modifications and variations to these embodiments once they have the benefit of the foregoing description. Accordingly, it is intended that the present application embrace all such modifications and variations as fall within the scope of the embodiments of the present application.

[0181] Finally, it is to be understood that the phraseology or terminology such as "first" and "second" etc. used herein is merely intended to differentiate one entity or operation from another entity or operation, without necessarily requiring or implying any actual such relationship or order between such entities or operations. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0182] The pixel sampling device, method and image compression system provided by the present application are described in detail above, and the principles and implementation manners of the present application are described by using specific examples. The above description of the embodiments is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation manners and application ranges can be changed according to the idea of the present application. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A pixel sampling device, characterized by, The system bus is connected with an external memory, including a first address generator, a prediction adjustment unit, a private cache unit and a downsampling unit; The first address generator is configured to initiate a read request to a target image in the external memory according to a preset initial address and a termination address; the read request is configured to read all target data blocks of the target image; The private cache unit is configured to cache high-frequency data blocks and corresponding pixel frequency information in historical images, and prediction period values; The prediction adjustment unit is configured to determine high-frequency data blocks in a historical image according to a DCT feedback result of a previous historical image; a ratio of a maximum pixel frequency value to a minimum pixel frequency value in the high-frequency data blocks is greater than or equal to a first threshold value; a mark is added to all the high-frequency data blocks and stored in the private cache unit; the mark includes a frame position, a storage address and a high-frequency label number; the prediction adjustment unit is configured to predict whether a current target data block is the same as the high-frequency data blocks in the private cache unit according to a current prediction period value; the target data block or the high-frequency data blocks is determined as a to-be-converted data block according to a prediction result; the prediction period value is determined based on a prediction result of predicting a previous target data block; The downsampling unit is configured to perform color space conversion on the to-be-converted data block and downsampling of chroma information to obtain a result data block; The downsampling unit includes: A calculation module is configured to perform color space conversion on the to-be-converted data block by using a binary algorithm to obtain to-be-sampled data; the calculation module includes a shift register, an 8-bit multiplier, an 8-bit complement adder / subtractor and a register; A sampling memory is configured to cache the to-be-sampled data, wherein each two to-be-sampled data are stored in a combined form; A sampling module includes a read operation module and a second address generator; the second address generator is configured to generate a plurality of first addresses based on a second size; the read operation module is configured to perform downsampling of chroma information on a to-be-sampled data block in the sampling memory according to each first address to obtain a result data block.

2. The pixel sampling device of claim 1, wherein, Further including at least one bus read interface, a bus arbiter and a first cache module; The first address generator is further configured to sequentially access each bus read interface and initiate a corresponding read request to the external memory; The bus arbiter is configured to adjust a write order of target data blocks returned by each bus read interface according to protocol handshake signals returned by each bus read interface; Each target data block is sequentially written into the first cache module according to the adjusted write order; The first cache module is configured to cache all target data blocks in the target image.

3. The pixel sampling device of claim 1, wherein, Further including a second cache module configured to cache the to-be-converted data block; The pixel frequency information of the target data block is predicted according to the prediction period value, specifically including: Based on the high-frequency label number, corresponding historical data blocks are obtained from the private cache unit; A prediction period value is read from the private cache unit to determine whether the current prediction period value is greater than or equal to a second threshold value; In a case that the predicted period value is greater than or equal to a second threshold value, it is predicted that the target data block is consistent with the pixel frequency information of the historical data block; The historical data block is written into the second cache module as a prediction result.

4. The pixel sampling device of claim 3, wherein, The prediction adjustment unit is configured to determine a data block to be converted, and specifically includes: The target data block is compared with the historical data block to determine whether the pixel frequency information is consistent; In a case that the pixel frequency information is consistent, the prediction data block stored in the second cache module is taken as the data block to be converted; In a case that the pixel frequency information is inconsistent, the target data block is written into the second cache module as the data block to be converted, and the prediction data block is overwritten.

5. The pixel sampling device of claim 4, wherein, The prediction adjustment unit is further configured to perform the following steps: In a case that the pixel frequency information is consistent, the current prediction period value is increased by 1, and the target data block is added with the same high-frequency label number as the prediction data block; In a case that the pixel frequency information is inconsistent, the current prediction period value is decreased by 1.

6. The pixel sampling device of claim 3, wherein, The prediction adjustment unit is further configured to perform the following steps: In a case that the prediction period value is less than the second threshold value, it is predicted that the target data block is inconsistent with the pixel frequency information of the historical data block; The target data block is directly written into the second cache module.

7. The pixel sampling device of claim 3, wherein, The prediction adjustment unit is further configured to adjust the access order of all data blocks to be converted in the second cache module, so that the data block to be converted without a high-frequency label number is preferentially accessed.

8. A method of pixel sampling, the method comprising: The pixel sampling device according to any one of claims 1-7, comprising: A first address generator is configured to initiate a read request to a target image in an external memory according to a preset initial address and a termination address, and read all target data blocks of the target image; According to the current prediction period value, it is predicted whether the current target data block is the same as a high-frequency data block in the private cache unit; according to the prediction result, the target data block or the high-frequency data block is determined as a data block to be converted; the prediction period value is determined based on the prediction result of the previous target data block; The data block to be converted is subjected to color space conversion and chroma information downsampling to obtain a result data block.

9. An image compression system characterized by, The pixel sampling device according to any one of claims 1-7, wherein the downsampling unit further comprises a third cache module configured to store the result data block; A compression unit is configured to obtain the result data block from the third cache module, and perform DCT transformation, quantization and encoding compression to obtain a compressed data block; The DCT feedback result of all target data blocks of the target image obtained by DCT transformation is sent to a prediction adjustment module in the pixel sampling device. The computer program is executed by the processor to implement the steps in the method of claim 8.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The processor executes the computer program to implement the steps in the method of claim 8.

11. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps in the method of claim 8.

Images