Data storage method and device

Abstract

The invention relates to the field of data storage, in particular to a data storage method and device. The data storage method comprises the following steps: (1) calling and caching target data to be stored; (2) dividing the target data into an ID segment, an information segment, a general data segment and a special data segment; and (3) performing special storage on a special data segment, performing general storage on other data segments, and establishing a corresponding relationship between the ID segment and the other three data segments. According to the method, centrality judgment is carried out on data according to the characteristics of centrality data, different data reading and coordinate access sequences are adopted for image data with transverse centrality, longitudinal centrality or overall centrality, so that similar data can be better read in similar periods, the storage space is reduced, and the storage efficiency is improved. And the data security is also improved.

Description

Technical Field

[0001] This invention relates to the field of data storage, and specifically to a method and apparatus for storing image data with a certain degree of centrality. Background Technology

[0002] With the development of the information age, a large amount of data needs to be stored. Data processing capabilities, data transmission bandwidth, data encryption technology and data storage space have become several major bottlenecks restricting the development of digital technology.

[0003] Many data sets exhibit certain patterns, such as centrality. Furthermore, these data often require long-term storage, especially CT and MRI data, which typically occupy significant storage space. Moreover, these data are frequently regenerated during periodic review, requiring even longer storage times, leading to exceptionally large storage requirements. Therefore, effectively reducing the storage space required for image data storage will significantly alleviate the pressure on image data storage. Summary of the Invention

[0004] To address the current problems with data storage, this invention aims to provide a data storage method that both reduces data storage space and achieves better lossless data preservation.

[0005] While researching medical data, the inventors discovered that the data that occupies a relatively large amount of space is not some basic text data, but some special data, such as image data. Image data often differs greatly from other image data. These image data often have better centrality, with core area data concentrated in a specific area, while peripheral data often have greater similarity.

[0006] With the development of high-definition imaging, the demand for storage space is increasing exponentially.

[0007] The technical solution of the present invention to address this problem is as follows:

[0008] A data storage method, the method comprising the following steps:

[0009] (1) Retrieve and cache the target data to be stored;

[0010] (2) Divide the target data into ID segment, information segment, general data segment and special data segment. The ID segment includes data that has a unique identification function for the stored target; the information segment includes data that indicates other information about the stored target besides the stored target ID; the general data segment includes general diagnosis and treatment data of the stored target; the special data segment includes at least image data with lateral centrality, longitudinal centrality or global centrality.

[0011] (3) Special data segments are stored in a special way, other data segments are stored in a general way, and the correspondence between the ID segment and the other three data segments is established;

[0012] (3.1) For special data segments containing images, first determine whether the gradient change of the image belongs to a total gradient change, a horizontal gradient change, or a vertical gradient change.

[0013] The horizontally centered image is rotated 90 degrees to convert it into a vertically centered image for processing; for any data image, the image pixel values ​​are read according to different image types; for vertically centered images and images without a center, the pixel values ​​are extracted row by row; for images with total-to-part variation, the pixel values ​​are extracted around the image; the read values ​​are cached in a buffer and processed separately.

[0014] (3.2) After reading a predetermined number of pixel values, find the maximum and minimum values ​​from each of the read pixel values, and determine whether the difference between the maximum and minimum values ​​of each read pixel exceeds the interval threshold. If the number of consecutive pixel values ​​whose difference between the maximum and minimum values ​​does not exceed the interval threshold exceeds the set number, then the pixels between the start and end pixels of the consecutive pixels are set as a pixel leading region. If the number of consecutive pixel values ​​whose difference between the maximum and minimum values ​​does not exceed the interval threshold does not exceed the set number, then the first pixel that causes the difference between the maximum and minimum values ​​to exceed the interval threshold and its preceding pixels are defined as coordinate leading region pixels. Starting from the next pixel, repeat the above steps to determine whether it belongs to a pixel leading region. In this way, the image is divided into pixel leading regions and coordinate leading regions.

[0015] (3.3) For the pixel leader area, the data storage format is: pixel leader identifier, leader pixel value area, pixel coordinate area, pixel deviation value; for the coordinate leader area, the data storage format is: coordinate leader identifier, leader coordinate value area, sequential pixel value area.

[0016] Preferably, the method further includes: (3.4) When storing information segments, comparing information segment data of the same ID storage target at different times, determining whether the information segment data registered at different times are completely the same, using the latest information segment as the information segment data basis, if they are completely the same, establishing a mapping from historical information segment data to the information segment data basis, if they are not completely the same, determining whether the difference in information segment data exceeds a predetermined proportion, if it exceeds the predetermined proportion, recording the information segment data and using it as the second information segment basis data, otherwise, recording the same fields, different fields, and the intervals of different fields to form a mapping from different fields to the information segment data basis;

[0017] (3.5) For the ID segment and the general data segment, lossless compression is performed using the DEFLATE algorithm;

[0018] Preferably, firstly, a horizontal gradient change judgment is performed. Several rows of data in the image data are extracted at intervals, and the changing trend of the absolute pixel value of each row of data is judged. If, in each row of pixels, the average gradient value of pixels on one side of the middle pixel is greater than a positive threshold, and the average gradient value of pixels on the other side is less than a negative threshold, or vice versa, then the image is judged to have horizontal centrality. Then, a vertical gradient change judgment is performed. Several columns of data in the image data are extracted at intervals, and the changing trend of the absolute pixel value of each column of data is judged. If, in each column of pixels, the average gradient value of pixels on one side of the middle pixel is greater than a positive threshold, and the average gradient value of pixels on the other side is less than a negative threshold, or vice versa, then the image is judged to have vertical centrality. If an image has both horizontal and vertical centrality, it is classified as a total-part variation image. If an image has horizontal centrality but no vertical centrality, it is classified as a horizontal gradient change image. If an image has vertical centrality but no horizontal centrality, it is classified as a vertical gradient change image. Images that do not meet any of the above criteria are classified as images without centrality.

[0019] Preferably, for a pixel leader region, the minimum, maximum, or average pixel value of the pixel leader region is used as the leader pixel of the pixel leader region. Then, the start coordinates and end coordinates of the pixel region are stored, and finally, the difference between each pixel and the leader pixel is stored.

[0020] Preferably, the set number is 8, 16, 32, 64, 128, or 256.

[0021] Preferably, the method further includes determining whether the current patient ID in the read patient data contains historical data in the database. If it contains historical data, it determines whether it contains a special data segment of the same type. If it does, it retrieves the historical storage architecture of the special data segment and determines the new data storage method according to the historical storage architecture of the special data segment.

[0022] Preferably, the method further includes: when storing data, constructing a master data table to store ID data and the start and end position information of data reading for three data segments: information segment, general data segment, and special data segment.

[0023] A data storage device includes a data processing unit and a data storage unit, wherein the data processing unit is used to process data according to the method and store it in the data storage unit.

[0024] This invention addresses the characteristics of medical data by determining its centrality. Image data with different lateral, longitudinal, or overall centrality are read and accessed in different orders, allowing similar data to be retrieved more effectively at similar times. This reduces storage space and improves data security. This is because the specific nature of the storage method means that data will become garbled if not read according to the predetermined procedure, thus enhancing data security. Attached Figure Description

[0025] Figure 1 This is a flowchart of the data storage method of the present invention;

[0026] Figure 2 A comparison of the compression ratios achievable by the compression method of this invention with existing compression methods for different groups of data;

[0027] Figure 3 This is an example of the image gradient calculation process. Detailed Implementation

[0028] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0029] The data storage method in this embodiment includes the following steps:

[0030] (1) Retrieve and summarize the target data to be stored (taking medical data as an example). Due to the large amount of data, it is impossible to retrieve all the target data at once. Therefore, the target data for different time periods can be retrieved in batches and processed in batches.

[0031] (2) Divide the target data into ID segment, information segment, general data segment and special data segment. Data that has a unique identification function for the stored target is divided into ID segment. No processing is performed on the ID segment data. The ID segment can include a general ID or a special ID established by a specific unit. Data that indicates other information about the stored target besides the stored target ID is divided into information segment. General diagnosis and treatment data of the stored target (target population or target test subject) is divided into general data segment. Data of a single diagnosis and treatment of the stored target that exceeds a preset threshold is divided into special data segment. The special data segment is preferably medical image data with certain centrality characteristics.

[0032] (3) Data of different data segments are processed in different ways, and a mapping relationship is established between the ID segment and the other three data segments; in one implementation, when storing data, a data master table is constructed to store the ID data and the start and end position information of the data reading of the three data segments: information segment, general data segment and special data segment.

[0033] For information segments and general data segments, general storage is performed, preferably using the DEFLATE algorithm for lossless compression.

[0034] (3.3) For special data segments, mainly image data, first determine whether the image gradient change belongs to a total-partial change, a horizontal gradient change, or a vertical gradient change (e.g., ...). Figure 3 The image shown is an example of image gradient value calculation. The specific judgment method is as follows: First, a horizontal gradient change judgment is performed. Several rows of data are extracted from the image data at intervals. The trend of the absolute pixel value change in each row is judged. If, in each row of pixels, on one side of the middle pixel (or several central pixels, taking their average), the average gradient value (and / or median, the same below) of the pixels is positive and greater than γ, and on the other side the average gradient value is negative and less than -γ (a preset gradient threshold, for example, set to...). (G h G is the maximum pixel value in that row. l If the minimum pixel value is α (a proportional coefficient, which can be set to 3-7), and S is the total number of pixels in the row, then the row is considered central. If the number of rows meeting the condition exceeds a certain proportion (e.g., 75-98%), the image is considered to have lateral centrality. Then, a vertical gradient change judgment is performed. Several columns of data are extracted from the image data at intervals, and the trend of the absolute pixel values ​​in each column is judged. If, in each column, the average gradient of pixels on one side (above or below) of the middle pixel is positive and greater than γ', and the average gradient of pixels on the other side is negative and less than -γ', then the column is considered central. If the number of columns meeting the condition exceeds a certain proportion, the image is considered to have vertical centrality. If an image has both lateral and vertical centrality, it is classified as a total-to-partial change image. If an image has lateral centrality but no vertical centrality, it is classified as a lateral gradient change image. If an image has vertical centrality but no lateral centrality, it is classified as a vertical gradient change image. Images that do not meet any of the above criteria are classified as non-central images.

[0035] When storing images, the horizontally centered image is first rotated 90 degrees to convert it into a vertically centered image, so that the horizontally and vertically centered images can be processed uniformly.

[0036] For vertically centered images (including images transformed from horizontally centered images) and non-centralized images, the pixel values ​​of the image are extracted row by row; for images with total-total variation, a wraparound pixel value extraction is performed, that is, the image is converted into a quasi-polar coordinate system for processing.

[0037] For any data image, first mark the image type and reading method, then divide the image into two different regions, the coordinate leading region and the pixel leading region, and store them separately.

[0038] Image pixel values ​​are read according to different image types, and the read image pixels are cached in a buffer for separate processing. Here, we take longitudinally centered images as an example, such as X-ray images of the tibia, longitudinal CT or X-ray images of the spine. Such images often have strong regional characteristics, with a central band of high brightness and generally lower brightness in the lateral areas. Because of these distinct regional features, a targeted storage method can significantly reduce storage space.

[0039] Specifically, for a vertically centered image, starting from the first and last rows, pixels are read inwards from the top and bottom respectively, and the pixel values ​​are read sequentially into the buffer. The maximum and minimum values ​​are then searched among the read pixel values. It is determined whether the difference between the maximum and minimum values ​​of each read pixel exceeds an interval threshold. If the number of consecutive pixel values ​​whose maximum and minimum differences do not exceed the interval threshold exceeds a set number (e.g., 8, 16, 32, 64, 128, etc.), then the area covered by the starting and ending pixels at the top (the part read downwards from the first row of the image) and the starting and ending pixels at the bottom (the part read downwards from the last row of the image) is set as the first pixel leading area. If the number of pixels does not exceed a set threshold, the first pixel whose difference between the maximum and minimum values ​​exceeds the interval threshold, along with its preceding pixel, is defined as a coordinate leading region pixel. Starting from the next pixel, the above steps are repeated to determine whether it belongs to a pixel leading region, until the difference between more than a set number of pixels does not exceed the set threshold. After determining the first pixel leading region, starting from the pixel after the upper termination pixel and the pixel after the lower termination pixel, pixels are read from the buffer, and it is determined whether the difference between subsequent pixels and the pixels read in this batch exceeds the interval threshold. If the number of consecutive pixel values ​​that do not exceed the interval threshold exceeds a set number, this area is called the second pixel leading region. This process is repeated to divide the image into various pixel leading regions and coordinate leading regions. For each image, the horizontal and vertical coordinates of the image, the image reading method, and the number of pixel leading regions and coordinate leading regions are first identified. Then, the two types of data are stored separately. For the pixel leading region, the data storage format is: pixel leading identifier, leading pixel area, pixel coordinate area, and pixel deviation value. Add a pixel leader identifier, and then use the minimum, maximum, or average pixel value of the pixel leader region as the leader pixel of the pixel leader region. Next, store the start and end coordinates of the pixel region in the pixel coordinate region. Since the image type is a vertically centered image, the start and end coordinates include the start and end coordinates of the pixel leader region on the first side of the image (e.g., the top) within the leader pixel range, and the start and end coordinates of the pixel leader region on the second side of the image (e.g., the bottom) within the leader pixel range. Sequentially store the difference between each pixel in this segment and the leader pixel in the pixel deviation value region, storing each deviation value with minimal storage space.

[0040] For the coordinate leader area, the data storage format is: coordinate leader identifier, leader coordinate value area, and sequential pixel value area. The coordinate leader identifier is used to identify the subsequent predetermined number of storage blocks as the coordinate leader area. The leader coordinate value area is used to store the starting coordinates, ending coordinates, the continuation method between coordinates (i.e., the above-mentioned pixel reading method), the number of bits for storing the pixel corresponding to each coordinate, and the total number of pixels in the area. The sequential pixel value area is used to sequentially store the pixel value corresponding to each coordinate.

[0041] For images with global centrality, such as cardiac angiography images, when reading image data, clockwise is considered positive and counterclockwise is considered negative, and the order of reading and writing image data is converted to a polar coordinate-like format. That is, the reading order is (1,1), (2,1), (3,1)...(8,1), (8,2), (8,3),..., (8,8), (7,8), (6,8),...(1,8), (1,7), (1,6),...(1,2), (2,2), (3,2)..., and so on.

[0042] In a preferred implementation, the pixels are read in a clockwise direction, decreasing from the distance from the image center. Assuming the number of pixels is M*N, where M is the number of pixels horizontally and N is the number of pixels vertically, and (X, Y) are the pixel coordinates, then... Pixel values ​​are compared from largest to smallest. If the 'd' values ​​are the same, the reading order is determined by clockwise or counterclockwise angular position. Pixels are then read from the periphery to the center to determine if each pixel meets the requirements of the pixel leader region. This reading method was used for the central image data in subsequent experimental verification.

[0043] For simplicity, we'll use an 8x8 image as an example. When reading and comparing data, the pixel value at position (1,1) is read first. Then, the pixels at positions (1,8), (8,1), and (8,8) are read. That is, starting from the initial coordinate point (1,1), the image center is used as a circle, and the pixels are read in a circular fashion, from largest to smallest, according to their distance from the center. Next, (1,2), (1,7), (2,8), (7,8), (8,8), (8,2), and (7,1) are read, and so on.

[0044] In a preferred implementation, the contour center of the target image is extracted, the distance is calculated with the contour center as the center, and the pixels are read in order of their distance from the center.

[0045] Preferably, (3.4) when storing information segments, the information segment data of targets (recorded subjects) with the same ID at different times are compared to determine whether the information segment data registered at different times are completely identical. The earliest information segment is used as the information segment data basis. If they are completely identical, a mapping from historical information segment data to the information segment data basis is established. If they are not completely identical, it is determined whether the difference in information segment data exceeds a predetermined proportion. If it exceeds the predetermined proportion, the information segment data is recorded and used as the basis data for the second information segment. Otherwise, the difference fields and the range of the difference fields are recorded to form a mapping from the difference fields to the information segment data basis. Alternatively, the information data can be stored in the same way as the lossless compression in (3.5).

[0046] (3.5) For general data segments, use the DEFLATE algorithm for lossless compression.

[0047] Test Instance

[0048] To verify the method of this invention, the inventors conducted data storage tests using cardiac CT image data as an example. Two sets of cardiac CT image data from multiple patients, each 10GB, and two sets of spinal MRI data, each 10GB, were selected. These were compressed using both LZW and the method of this invention, and compression ratio tests were performed. The test results are as follows: Figure 2 As shown.

[0049] As can be seen from the figure, the compression ratio achievable by the LZW method is approximately 3:1, while the compression ratio of the method of this invention can reach approximately 4-5:1. This is significantly higher than the compression ratio of existing image storage methods.

[0050] Although the principles of the present invention have been described in detail above with reference to preferred embodiments, those skilled in the art should understand that the above embodiments are merely illustrative explanations of the implementation of the present invention and are not intended to limit the scope of the present invention. The details in the embodiments do not constitute a limitation on the scope of the present invention. Any obvious changes, such as equivalent transformations or simple substitutions, based on the technical solutions of the present invention without departing from the spirit and scope of the present invention fall within the protection scope of the present invention.

Claims

1. A data storage method, characterized in that, The method includes the following steps: (1) Retrieve and cache the target data to be stored; (2) Divide the target data into ID segment, information segment, general data segment and special data segment. The ID segment includes data that has a unique identification function for the stored target; the information segment includes data that indicates other information about the stored target besides the stored target ID. The general data segment includes the general record data of the storage target; Special data segments must include image data that exhibits lateral centrality, longitudinal centrality, or overall centrality. (3) Special data segments are stored in a special way, other data segments are stored in a general way, and the correspondence between the ID segment and the other three data segments is established; (3.1) For special data segments containing images, the gradient change of the image is first determined to be a total-to-part change, a horizontal gradient change, or a vertical gradient change, thereby classifying the image into a total-to-part image, a horizontally centered image, a vertically centered image, or a non-centered image. The horizontally centered image is rotated 90 degrees to convert it into a vertically centered image for processing; for any data image, the image pixel values ​​are read according to different image types; for vertically centered images and images without a center, the pixel values ​​are extracted row by row; for total-to-part images, the pixel values ​​are extracted around the image; the read values ​​are cached in a buffer and processed separately. (3.2) After reading a predetermined number of pixel values, find the maximum and minimum values ​​from each of the read pixel values, and determine whether the difference between the maximum and minimum values ​​of each read pixel exceeds the interval threshold. If the number of consecutive pixel values ​​whose difference between the maximum and minimum values ​​does not exceed the interval threshold exceeds the set number, then the pixels between the start and end pixels of the consecutive pixels are set as a pixel leading region. If the number of consecutive pixel values ​​whose difference between the maximum and minimum values ​​does not exceed the interval threshold does not exceed the set number, then the first pixel that causes the difference between the maximum and minimum values ​​to exceed the interval threshold and its preceding pixels are defined as coordinate leading region pixels. Starting from the next pixel, repeat the above steps to determine whether it belongs to a pixel leading region. In this way, the image is divided into pixel leading regions and coordinate leading regions. (3.3) For the pixel leader area, the data storage format is: pixel leader identifier, leader pixel value area, pixel coordinate area, pixel deviation value; for the coordinate leader area, the data storage format is: coordinate leader identifier, leader coordinate value area, sequential pixel value area.

2. The method according to claim 1, characterized in that, Also includes: (3.4) When storing information segments, compare the information segment data of the same ID storage target at different times to determine whether the information segment data registered at different times are completely the same. Use the latest information segment as the information segment data base. If they are completely the same, establish a mapping from historical information segment data to the information segment data base. If they are not completely the same, determine whether the difference in information segment data exceeds a predetermined ratio. If it exceeds the predetermined ratio, record the information segment data and use it as the second information segment base data. Otherwise, record the same fields, different fields, and the range of different fields to form a mapping from different fields to the information segment data base. (3.5) For the ID segment and the general data segment, lossless compression is performed using the DEFLATE algorithm.

3. The method according to claim 1, characterized in that, The method includes: First, horizontal gradient change is determined by extracting several rows of data from the image at intervals and judging the trend of pixel absolute values ​​in each row. If, in each row, the average gradient value of pixels on one side of the middle pixel is greater than a positive threshold, and the average gradient value of pixels on the other side is less than a negative threshold, or vice versa, then the image is determined to have horizontal centrality. Next, vertical gradient change is determined by extracting several columns of data from the image at intervals and judging the trend of pixel absolute values ​​in each column. If, in each column, the average gradient value of pixels on one side of the middle pixel is greater than a positive threshold, and the average gradient value of pixels on the other side is less than a negative threshold, or vice versa, then the image is determined to have vertical centrality. If an image has both horizontal and vertical centrality, it is classified as a total-to-partial change image. If an image has horizontal centrality but no vertical centrality, it is classified as a horizontal gradient change image. If an image has vertical centrality but no horizontal centrality, it is classified as a vertical gradient change image. Images that do not meet any of the above criteria are classified as centrality-free images.

4. The method according to claim 1, characterized in that, For a pixel leader region, the minimum, maximum, or average pixel value of the pixel leader region is used as the leader pixel of the pixel leader region. Next, the start coordinates and end coordinates of the pixel region are stored, and finally, the difference between each pixel and the leader pixel is stored.

5. The method according to claim 1, characterized in that, Step 3.2 includes: for vertically centered images, starting from the first and last rows, pixels are read from the top and bottom sides inwards, and the pixel values ​​are read sequentially into the buffer; for total-to-part images, the pixels are read in a clockwise or counterclockwise direction according to the distance from the image center from large to small.

6. The method according to claim 1, characterized in that, The set number is 8, 16, 32, 64, 128, or 256.

7. The method according to claim 1, characterized in that, The method further includes determining whether the current patient ID in the read patient data contains historical data in the database. If it contains historical data, it determines whether it contains a special data segment of the same type. If it does, it retrieves the historical storage architecture of the special data segment and determines the new data storage method according to the historical storage architecture of the special data segment.

8. The method according to claim 1, characterized in that, The method further includes: when storing data, constructing a master data table to store ID data and the start and end position information of data reading for three data segments: information segment, general data segment, and special data segment.

9. A data storage device, characterized in that, The data storage device includes a data processing unit and a data storage unit, wherein the data processing unit is used to process data according to the method of any one of claims 1-6 and store it in the data storage unit.

Images