Image data solid-state storage method and system

Abstract

A solid-state image data storage method compresses image data when writing and decompresses it when reading. The image data compression process involves inputting image data, segmenting the image, performing a fast DCT transform, performing threshold quantization, and obtaining compressed data. The image data decompression process involves inputting compressed data, segmenting the data, performing a fast inverse DCT transform, and obtaining a decompressed image. This method compresses image data at the master control level, enabling high-speed storage. This method maximizes the amount of image data written into limited NAND storage, extending the NAND's lifespan.

Description

Technical Field

[0001] The present invention relates to the field of storage, and in particular to a solid-state storage method and system for image data for image storage. Background Art

[0002] With the advancement of science and technology, especially in aerospace and machine vision, the digital images captured by high-speed digital cameras, whether for spaceborne, shipborne, or industrial measurement, urgently require a large-capacity, high-speed, and highly reliable image storage system. With the emergence and widespread use of NAND Flash, NAND Flash-based SSDs, with their higher reliability, better performance, and lower energy consumption, can meet the storage needs of high-speed digital cameras.

[0003] Due to NAND Flash process limitations, bad blocks (factory bad blocks and newly added bad blocks) may occur during shipment and use. When NAND Flash is shipped from the factory, the manufacturer allows for a certain number of bad blocks, typically between 2% and 5%, and marks them as factory bad blocks. Furthermore, Flash devices have a limited number of erase and write cycles. After exceeding these limits, certain bits will become unable to flip (from "0" to "1"), creating an irreversible error. This block then becomes an invalid block, or a newly added bad block. The NAND Flash datasheet specifies that a bad block cannot be erased or written to.

[0004] Therefore, it is very necessary to study how to use limited NAND Flash cells to store as much image data as possible and extend the service life of NAND Flash. Compressing image data before storing it in NAND Flash is a very effective method. Summary of the Invention

[0005] To address the shortcomings of the existing technology, the present invention provides a solid-state image data storage method and system that compresses image data at the master control level to achieve high-speed image data storage, writing as much image data as possible into limited NAND memory space and extending the life of the NAND.

[0006] In order to solve the above technical problems, the present invention adopts a technical solution: a solid-state storage method for image data, which performs data compression when writing image data and decompresses the data when reading image data; the data compression during writing is specifically:

[0007] S11) The SSD master receives and parses the Host data write command and applies for pre-compression write buffer space;

[0008] S12) The SSD master controller starts DMA to move the image data to be written into the pre-compression write buffer space;

[0009] S13), starting the compression module to compress the image data cached in the write buffer before compression, and at the same time applying for a new write buffer space, storing the compressed image data in the new buffer, and releasing the write buffer space before compression after all compression is completed;

[0010] S14), the SSD master controller starts NFC to write the compressed image data in the cache into NAND;

[0011] The data decompression during reading is as follows:

[0012] S21) The SSD master receives and parses the Host data read command, and applies for read buffer space before decompression;

[0013] S22) The SSD master controller starts NFC and moves the data to be read from the NAND into the read buffer space before decompression;

[0014] S23), starting the decompression module to decompress the image data stored in the read buffer before decompression, and at the same time applying for a new read buffer space, storing the decompressed image data in the new read buffer space, and releasing the read buffer space before decompression after all decompression is completed;

[0015] S24) The SSD master controller starts DMA to transfer the decompressed image data in the read buffer space to the Host.

[0016] Furthermore, the image compression process is:

[0017] S131), dividing the image data to be compressed into several sub-blocks of 8*8 size;

[0018] S132) Perform fast DCT transformation on each segmented sub-block. The fast DCT transformation formula is as follows:

[0019] F(u,v)=Qf(x,y)Q T ,

[0020]

[0021] Among them, Q is the fast DCT transform matrix, Q T is the transposed matrix of Q, f(x,y) is the original data before compression,

[0022] F(u,v) is the result after fast DCT transformation;

[0023] S133) Perform threshold selection on the coefficients after the fast DCT transformation, sort the 64 coefficients after the sub-block transformation from large to small, and retain the first ten coefficients. The retention or removal of the remaining 54 coefficients is determined according to the following formula:

[0024] Th=64*δ-10,

[0025] Where δ is the image compression ratio, i.e., δ = compressed image data size / original image data size, and Th is the number of the remaining 54 coefficients selected in descending order;

[0026] S134) Save the compressed DCT coefficients into a buffer.

[0027] Furthermore, the value range of δ is [5 / 32, 1).

[0028] Furthermore, the image decompression process is:

[0029] S231), dividing the compressed image data into several sub-blocks of 8*8 size;

[0030] S232), perform fast inverse DCT transform on each word block, the fast inverse DCT transform formula is:

[0031] f′(x,y)=PF′(u,v)P T ,

[0032]

[0033] Among them, P is the inverse matrix of the fast DCT transform matrix, P T is the transposed matrix of P, f′(x,y) is the decompressed image data, and F′(u,v) is the result after fast DCT transformation screening;

[0034] S233) Save the decompressed image data into the read buffer.

[0035] The present invention also discloses an image data solid-state storage system, including an SSD interface layer, a flash memory conversion layer, a bus controller, a write buffer cache, a compression module, a DMA, an NFC, a read buffer cache, a decompression module, a NAND controller, and a NAND;

[0036] The SSD interface layer is connected to the host and is used to receive and parse data write commands and data read commands sent by the host;

[0037] The flash translation layer is connected to the SSD interface layer and is used to apply for pre-compression write buffer space and pre-decompression read buffer space based on the parsed command information;

[0038] The flash translation layer is connected to the bus controller, which is also connected to the write buffer cache, DMA, NFC, and read buffer cache. The bus controller implements data transmission between the flash translation layer and the write buffer cache, DMA, NFC, and read buffer cache.

[0039] The write buffer cache is connected to the compression module, which compresses the data temporarily stored in the write buffer cache and writes it to NAND after compression is completed;

[0040] The read buffer cache is connected to the decompression module, which decompresses the data temporarily stored in the read buffer cache;

[0041] This system is used to execute the above method.

[0042] Beneficial effects of the present invention: The present invention provides a high-speed solid-state storage method and system for image data, which realizes high-speed compressed storage of image data from the main control level, can set the image compression ratio according to the specific usage scenario, adapt to the usage requirements of different scenarios, and can write as much image data as possible in the limited NAND space, effectively reducing the number of erase and write times of the block in the NAND and extending the service life of the NAND. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a schematic diagram of the storage system;

[0044] Figure 2 Flowchart when compressing data;

[0045] Figure 3 Flowchart for data reading and decompression;

[0046] Figure 4 This is a flowchart of image compression;

[0047] Figure 5 Flowchart for decompressing an image. DETAILED DESCRIPTION

[0048] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0049] Example 1

[0050] This embodiment discloses a solid-state storage system for image data, such as Figure 1 As shown, it includes SSD interface layer, flash conversion layer, bus controller, write buffer cache, compression module, DMA, NFC, read buffer cache, decompression module, NAND controller, and NAND;

[0051] The SSD interface layer is connected to the host and is used to receive and parse data write commands and data read commands sent by the host;

[0052] The flash translation layer is connected to the SSD interface layer and is used to apply for pre-compression write buffer space and pre-decompression read buffer space based on the parsed command information;

[0053] The flash translation layer is connected to the bus controller, which is also connected to the write buffer cache, DMA, NFC, and read buffer cache. The bus controller implements data transmission between the flash translation layer and the write buffer cache, DMA, NFC, and read buffer cache.

[0054] The write buffer cache is connected to the compression module, which compresses the data temporarily stored in the write buffer cache and writes it to NAND after compression is completed;

[0055] The read buffer cache is connected to the decompression module, and the decompression module decompresses the data temporarily stored in the read buffer cache.

[0056] This system performs data compression when writing image data. The operation steps of the writing compression system are as follows: Figure 2 , specifically including:

[0057] S101: The SSD master controller receives and parses the Host data write command and applies for pre-compression write buffer space.

[0058] Specifically, after the SSD master control interface layer receives the data write command sent by the Host, it parses the command information carried in the command and then passes it to the flash translation layer. After receiving the parsed command information, the flash translation layer applies for pre-compression write buffer space based on the logical address size through the bus controller.

[0059] S102: The SSD master controller starts DMA to move the image data to be written into the pre-compression write buffer space.

[0060] Specifically, after the flash translation layer applies for sufficient pre-compression write buffer space, it feeds back the buffer address to the SSD interface layer. After the SSD interface layer configures the buffer and address information, it starts DMA through the bus controller to move the image data to be written from the host to the pre-compression write buffer.

[0061] S103: Start the compression module to compress the image data cached in the pre-compression write buffer, and apply for a new write buffer space at the same time, store the compressed image data in the new buffer, and release the pre-compression write buffer space after all compression is completed.

[0062] Specifically, after starting the compression module, the image data in the buffer before compression is used as input. After compression, the compressed image data is output and stored in a new cache buffer. After all compression is completed, the buffer space before compression is released. Figure 4 , as follows:

[0063] The first step is to divide the image data to be compressed into several sub-blocks of 8*8 size;

[0064] The second step is to perform fast DCT transform on each sub-block. The fast DCT transform formula is as follows:

[0065] F(u,v)=Qf(x,y)Q T ,

[0066]

[0067] Among them, Q is the fast DCT transform matrix, Q T is the transposed matrix of Q, f(x,y) is the original data before compression, and F(u,v) is the result after fast DCT transformation.

[0068] The main data information of an image is stored in the low-frequency part of the image, while some edge details and noise are stored in the high-frequency part of the image. After the fast DCT transform, the low-frequency part is mainly concentrated in the upper left corner of the DCT energy spectrum, with large low-frequency coefficients, while the high-frequency part is mainly distributed in the lower right corner of the DCT energy spectrum, with less information. When compressing an image, in order to ensure that the image data can be reproduced after decompression, as much low-frequency information as possible is retained and high-frequency information is discarded.

[0069] The third step is to select the threshold value of the coefficients after the fast DCT transformation. To ensure that the image information can be restored during decompression, the 64 coefficients after the sub-block transformation are sorted from large to small and the first ten coefficients are retained. The retention of the remaining 54 coefficients is determined according to the following formula:

[0070] Th=64*δ-10,

[0071] Wherein, δ is the image compression ratio, i.e., δ = compressed image data size / original image data size. It is an adjustable parameter that is adjusted according to the application scenario and has a value range of [5 / 32, 1). Th is the number of the remaining 54 coefficients selected in descending order, and the remaining coefficients are set to 0.

[0072] Among the 64 coefficients after the transformation, in descending order, except for the first ten coefficients that must be retained and the remaining 54 coefficients to be retained determined by the threshold, the remaining coefficients are the remaining coefficients and are set to 0.

[0073] Save the compressed DCT coefficients to buffer.

[0074] S104 : The SSD master controller starts NFC to write the compressed image data in the cache into the NAND.

[0075] Specifically, after data compression is completed, the SSD master configures the buffer and PBA address information, starts NFC through the bus controller, and writes the compressed image data cached in the buffer into the NAND of the corresponding address.

[0076] This embodiment performs data decompression when reading image data, and the reading and decompression system operates as follows: Figure 3 , specifically including:

[0077] S201: The SSD master controller receives and parses the Host data read command, and applies for pre-decompression read buffer space.

[0078] Specifically, after the SSD master control interface layer receives the data read command sent by the Host, it parses the command information carried in the command and then passes it to the flash translation layer. After receiving the parsed command information, the flash translation layer applies for pre-decompression read buffer space based on the PBA size through the bus controller.

[0079] S202: The SSD master controller starts NFC to move the data to be read from the NAND into the read buffer space before decompression.

[0080] Specifically, after the flash translation layer applies for sufficient pre-decompression read buffer space, it feeds back the buffer address to the SSD interface layer. After the SSD interface layer configures the buffer and PBA address information, it activates NFC through the bus controller and moves the image data to be written from NAND to the pre-decompression read buffer.

[0081] S203, starting the decompression module to decompress the image data stored in the read buffer before decompression, and at the same time applying for a new read buffer space, storing the decompressed image data in the new read buffer space, and releasing the read buffer space before decompression after all decompression is completed.

[0082] Specifically, after starting the decompression module, the image data in the read buffer before decompression is used as input. After decompression, the decompressed image data is output and stored in a new read buffer. After all decompression is completed, the read buffer space before decompression is released. Figure 5 , as follows:

[0083] The first step is to divide the image data to be compressed into several sub-blocks of 8*8 size;

[0084] The second step is to perform fast inverse DCT transform on each sub-block. The fast inverse DCT transform formula is as follows

[0085] f′(x,y)=PF′(u,v)P T ,

[0086]

[0087] Among them, P is the inverse matrix of Q, P T is the transposed matrix of P, f′(x,y) is the decompressed image data, and F′(u,v) is the result after fast DCT transformation screening.

[0088] Save the decompressed image data to the read buffer.

[0089] S204 , the SSD master controller starts DMA to transmit the decompressed image data in the read buffer space to the Host.

[0090] Specifically, after data decompression is completed, the SSD master configures the decompressed read buffer information, starts DMA through the bus controller, and transmits the decompressed image data cached in the read buffer to the host.

[0091] Example 2

[0092] This embodiment discloses a solid-state storage method for image data. This method realizes high-speed compressed storage of image data from the master control level, that is, data compression is performed when image data is written and data decompression is performed when image data is read. Figure 2 、 3 As shown, the data compression during writing is as follows:

[0093] S11) The SSD master receives and parses the Host data write command and applies for pre-compression write buffer space;

[0094] S12) The SSD master controller starts DMA to move the image data to be written into the pre-compression write buffer space;

[0095] S13), starting the compression module to compress the image data cached in the write buffer before compression, and at the same time applying for a new write buffer space, storing the compressed image data in the new buffer, and releasing the write buffer space before compression after all compression is completed;

[0096] S14), the SSD master controller starts NFC to write the compressed image data in the cache into NAND;

[0097] The data decompression during reading is as follows:

[0098] S21) The SSD master receives and parses the Host data read command, and applies for read buffer space before decompression;

[0099] S22) The SSD master controller starts NFC and moves the data to be read from the NAND into the read buffer space before decompression;

[0100] S23), starting the decompression module to decompress the image data stored in the read buffer before decompression, and at the same time applying for a new read buffer space, storing the decompressed image data in the new read buffer space, and releasing the read buffer space before decompression after all decompression is completed;

[0101] S24) The SSD master controller starts DMA to transfer the decompressed image data in the read buffer space to the Host.

[0102] like Figure 4 、 5 As shown in FIG, it is a flow chart of the method for compressing and decompressing image data. The specific process is the same as that in Example 1 and will not be described again in the present invention.

[0103] The method and system described in the present invention achieve high-speed compressed storage of image data from the master control level. The image compression ratio can be set according to the specific usage scenario to meet the usage requirements of different scenarios. As much image data as possible can be written into the limited NAND space, the number of times blocks in the NAND are erased and written is effectively reduced, thereby extending the service life of the NAND.

[0104] The English of the present invention is explained below:

[0105] SSD: solid-state drive;

[0106] DMA: Direct Memory Access;

[0107] NFC: NAND Flash controller;

[0108] Buffer: cache;

[0109] Lba: logical address;

[0110] Pba: physical address;

[0111] Host: host.

[0112] What has been described above are only the basic principles and preferred embodiments of the present invention. Improvements and substitutions made by those skilled in the art based on the present invention fall within the protection scope of the present invention.

Claims

1. A solid-state storage method for image data, characterized in that: This method performs data compression when writing image data and decompresses data when reading image data; The data compression during writing is as follows: S11) The SSD master receives and parses the Host data write command and applies for pre-compression write buffer space; S12) The SSD master controller starts DMA to move the image data to be written into the pre-compression write buffer space; S13), starting the compression module to compress the image data cached in the write buffer before compression, and at the same time applying for a new write buffer space, storing the compressed image data in the new write buffer space, and releasing the write buffer space before compression after all compression is completed; S14), the SSD master controller starts NFC to write the compressed image data in the cache into NAND; The data decompression during reading is as follows: S21) The SSD master receives and parses the Host data read command and applies for read buffer space before decompression; S22) The SSD master controller starts NFC and moves the data to be read from NAND into the read buffer space before decompression; S23), starting the decompression module to decompress the image data stored in the read buffer before decompression, and at the same time applying for a new read buffer space, storing the decompressed image data in the new read buffer space, and releasing the read buffer space before decompression after all decompression is completed; S24) The SSD master controller starts DMA to transfer the decompressed image data in the read buffer space to the Host; The image compression process is: S131), dividing the image data to be compressed into several sub-blocks of 8*8 size; S132), perform fast DCT transformation on each sub-block of the segmentation, and the fast DCT transformation formula is as follows: ， ， in, is the fast DCT transform matrix, for The transposed matrix of is the original data before compression, is the result after fast DCT transformation; S133), threshold selection is performed on the coefficients after the fast DCT transformation, and the 64 coefficients after the sub-block transformation are sorted from large to small, and the first ten coefficients are retained. The retention or removal of the remaining 54 coefficients is determined according to the following formula: ， in, is the image compression ratio, that is , Th is the number of the remaining 54 coefficients selected in descending order; S134), save the compressed DCT coefficients into a buffer.

2. The solid-state storage method for image data according to claim 1, wherein: The value range is [5 / 32, 1).

3. The solid-state storage method for image data according to claim 1, wherein: The process of image decompression is: S231), dividing the compressed image data into several sub-blocks of 8*8 size; S232), perform fast inverse DCT transform on each word block, the fast inverse DCT transform formula is: ， ， in, is the inverse matrix of the fast DCT transform matrix, is the transposed matrix of P, is the decompressed image data, It is the result after fast DCT transformation screening; S233), save the decompressed image data into the read buffer.

4. A solid-state image data storage system, characterized in that: Includes SSD interface layer, flash conversion layer, bus controller, write buffer cache, compression module, DMA, NFC, read buffer cache, decompression module, NAND controller, NAND; The SSD interface layer is connected to the host and is used to receive and parse data write commands and data read commands sent by the host; The flash translation layer is connected to the SSD interface layer and is used to apply for pre-compression write buffer space and pre-decompression read buffer space based on the parsed command information; The flash translation layer is connected to the bus controller, which is also connected to the write buffer cache, DMA, NFC, and read buffer cache. The bus controller implements data transmission between the flash translation layer and the write buffer cache, DMA, NFC, and read buffer cache. The write buffer cache is connected to the compression module, which compresses the data temporarily stored in the write buffer cache and writes it to NAND after compression is completed; The read buffer cache is connected to the decompression module, which decompresses the data temporarily stored in the read buffer cache; The system is used to execute the method described in any one of claims 1 to 3.

Images