Duplicated data deletion method for medical big data

Abstract

The invention discloses a repeated data deletion method for medical big data. The method comprises the following steps: acquiring to-be-processed medical data; performing similarity calculation on the medical data to obtain data files sorted according to the maximum similarity; and performing repeated data deletion operation according to the data file, and performing rewriting based on the maximum similarity to realize deletion of medical repeated data. According to the repeated data deletion method for the medical big data, whether the medical data are redundant or not is determined based on similarity calculation, noise data are removed at the same time, and the recovery performance and the deletion performance are improved by rewriting the data blocks based on the maximum similarity; by designing a block chain strategy and a repeated data deletion recovery algorithm, the security and integrity of the medical data are efficiently ensured; therefore, the method is suitable for the medical industry, and is high in efficiency and good in safety.

Description

technical field [0001] The invention belongs to the technical field of computers, and in particular relates to a method for removing duplicate data for medical big data. Background technique [0002] With the development of economy and technology, digitization and informatization have been widely integrated into all walks of life. The development of digitalization and informatization in the medical industry has contributed to the generation of medical big data. [0003] In addition to the large-scale, high-speed, diverse and high-value characteristics of general big data, medical big data also has the characteristics of multi-modality, redundancy, security, incompleteness and timeliness. Among them, redundancy and security have brought huge challenges to the application of medical big data. Since medical data is growing much faster than storage devices, and data storage, adequate equipment, and energy consumption are all expensive, data redundancy puts enormous pressure on...

AI Technical Summary

Problems solved by technology

[0005] However, data deduplication technology disperses data blocks in various containers, which destroys data locality and generates a lot of fragments, which affects system recovery performance and deletion performance

However, the existing data deduplication method improves the recovery performance at the expense of DER, and does not dig into the characteristics of medical data to improve the DER, recovery performance and deletion performance of the system at the same time, and it is impossible to judge which medical data has redundancy; in addition, There are methods that rely on the time attribute of the backup version when improving recovery performance, but medical data has an indeterminate time attribute

More importantly, medical data has higher compliance requirements for security and personal information protection than ordinary data, and the existing encrypted deduplication technology requires verification by a third-party organization, which cannot effectively guarantee the security of medical data

Images