Medical archive file storage optimization method
By cleaning and regression model training of medical archive data, using the difference function to evaluate the optimal model for storage optimization, and dynamically expanding the space when there is insufficient storage, the problem of cold data occupies high-speed storage in medical archive storage is solved, and storage efficiency is achieved and improve fault tolerance.
Patent Information
- Application Number
- CN202410921886.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-10
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The prior art fails to effectively distinguish data types and heat in medical file storage, resulting in cold data occupying high-speed storage space, resulting in insufficient storage space, system stuck or data loss.
By obtaining historical medical archive data, cleaning and segmenting data into training sets and test sets, pre-training regression models perform medical archive file heat prediction, using the difference function to evaluate the optimal model, optimize the data to the appropriate storage medium, and Dynamically expand temporary storage space when there is insufficient storage.
It avoids overfitting the regression model, optimizes the storage structure, reduces the occupation of cold data on high-speed storage media, improves storage efficiency and fault tolerance, and reduces the risk of system blockage and data loss.
Smart Images

Figure CN118963646A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of data storage and relates to a medical archive file storage optimization method. Background Art
[0002] In the information age, the rapid growth of data has become a norm. Enterprises, organizations, and individuals are faced with the challenge of dealing with large amounts of electronic archives. These archives may include text documents, images, videos, and other data in a variety of formats. When dealing with such a huge amount of data, effective archive management becomes essential.
[0003] In most cases, the stored data is not allocated to different storage media based on the data type and data popularity, but is all stored in the same storage medium, resulting in some cold data being stored in fast-read and write solid-state drives. For example, the data and medical records of healthy examination subjects or recovered patients make data storage unoptimized, occupying a large amount of high-speed storage space. At the same time, the problem of storage space occupation cannot be solved well. When the storage space is insufficient, data congestion will occur, causing the entire system to freeze or data loss. Summary of the invention
[0004] In order to solve the problems existing in the background technology, the present invention proposes a medical archive file storage optimization method.
[0005] In order to achieve the above object, the present invention provides a method for optimizing storage of medical archive files, comprising the following steps:
[0006] Acquire historical medical archive file storage data from a medical archive information storage device, perform data cleaning on the historical medical archive file storage data, and divide the pre-processed historical medical archive file storage data into a training set and a test set;
[0007] Pre-training a number of regression models, using the training set to train each regression model for predicting the heat of medical archive files, and using the test set to test the trained regression model, and recording the test results of each regression model;
[0008] Setting a unified difference function to calculate the difference value of the test data of each regression model, evaluating different regression models, and obtaining the optimal regression model;
[0009] Input the file data to be stored into the optimal regression model to optimize the storage of medical file information, and output the heat prediction result, divide the heat prediction result into predicted heat levels, and move the files to be managed to the corresponding solid-state hard disk, mechanical hard disk or magnetic tape based on the predicted heat level;
[0010] When the storage capacity of the medical file information storage device is insufficient, temporary storage space is added to the medical file information storage device using the memory of the user machine according to the user machine that needs to store.
[0011] Further, in response to detecting new medical archive file storage data, the new medical archive file data is added to the historical medical archive file storage data so that the historical medical archive file storage data is updated with the new medical archive file data;
[0012] Based on the updated historical medical archive file storage data, the unified difference function is reused to calculate the difference of the test data of each regression model.
[0013] Furthermore, the acquired historical medical archive file storage data is cleaned including removing null values, replacing abnormal values, processing invalid values and extracting features, and the historical medical archive file storage data after data cleaning is divided into 80% training set and 20% test set.
[0014] Furthermore, the process of training several regression models and selecting the optimal regression model includes:
[0015] Initializing the parameters of each of the regression models, allocating the same number of training sets to each of the initialized regression models for archival information storage optimization training, testing each regression model that has completed the first training, and recording the test data;
[0016] Perform iterative training for each regression model to optimize archival information storage, and use the difference function to evaluate each regression model after each iterative training. Use the difference function to calculate the difference value of the regression model to the test data, and rank the regression models according to the difference value. The smaller the difference value, the higher the ranking of the regression model.
[0017] The training is completed after all the historical medical archive file storage data in the training set are allocated, and the regression model that is finally ranked first is taken as the optimal regression model, and the optimal regression model is monitored regularly using updated data.
[0018] Further, setting a unified difference function to calculate the difference value of the test data of each regression model includes:
[0019] Get the average results of the test data analyzed by different regression models:
[0020]
[0021] Where x represents the number of regression models, represents the prediction result of the i-th regression model for the data y, It represents the average results of the analysis of data y by different regression models;
[0022] Set the measurement function MEAP of the regression model evaluation score. According to the value of the measurement function MEAP, judge the training effect of the regression model. The expression is:
[0023]
[0024] in, And y n ≠0;
[0025] like And y n =0 is recorded as 0;
[0026] In the above formula, s represents the total amount of test data. Represents the predicted value of the nth data analysis, y n Represents the actual observation value of the nth data;
[0027] The smaller the value of the measurement standard function MEAP is, the better the corresponding regression model training effect is.
[0028] Furthermore, the adding of the memory of the user machine as a temporary storage space of the medical archive information storage device includes:
[0029] When the remaining storage space of the medical archive information storage device is less than the set value, and the user has a need to upload the patient's medical archive to the medical archive information storage device, the medical archive file storage device chooses to add a new temporary storage space;
[0030] Divide space on the current user machine to expand the capacity of the medical archive information storage device, control access rights to the space divided on the user machine, hide the newly divided space in the user machine, and add the newly divided space on the user machine as a temporary storage space to the medical archive file storage system;
[0031] The user machine uploads the marking information of the patient's medical file and the user machine number to the medical file information storage device, and the patient's medical file is stored in the temporary storage space.
[0032] Further, the medical archive information storage device reads the medical archive data from the temporary storage space after the capacity is expanded, and the medical archive file storage device transmits the data in the temporary storage space to the medical archive information storage device;
[0033] After the transmission is completed, the temporary storage space deletes the temporary data stored in itself, modifies the pointer of the data stored in the temporary storage space in the medical archive information storage device, and modifies the permission of the temporary storage space to occupy the storage space in the user machine.
[0034] Compared with the prior art, the present invention has the following beneficial effects:
[0035] The present invention calculates the difference values of different regression models by using a unified difference function, thereby avoiding the overfitting phenomenon of a certain regression model and ensuring the fairness of the calculation of the difference values of the regression models. The medical archive file storage optimization method provided by the present invention also supports dynamic expansion of storage space. The temporary storage space only occupies a small amount of memory of the user machine to maintain the normal operation of the entire system, reducing the probability of system blockage and the possibility of data loss. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 It is a schematic diagram of the overall process of the present invention;
[0037] Figure 2 It is a schematic diagram of the temporary storage space adding process according to an embodiment of the present invention. DETAILED DESCRIPTION
[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0039] The invention proposes a medical archive file storage optimization method.
[0040] like Figure 1 As shown, the technical solution adopted by the present invention includes the following steps:
[0041] First, extract the data to be optimized for storage. Obtain historical medical archive file storage data from the medical archive information storage device, perform data cleaning on the historical medical archive file storage data, and divide the preprocessed historical medical archive file storage data into a training set and a test set. Data cleaning of the historical medical archive file storage data can ensure that no outliers affect the training of the regression model in subsequent calculations, and ensure the accuracy of the regression model in subsequent calculations of medical archive information.
[0042] Then the historical data is trained and tested using the regression model. Several regression models are pre-trained, and each regression model is trained to predict the popularity of medical archive files using the training set, and the trained regression model is tested using the test set, and the test data of each regression model is recorded. There are two or more pre-trained regression models to ensure the selectivity of subsequent regression models and improve the fault tolerance of the medical archive file storage optimization method.
[0043] Perform a unified evaluation on different regression models. Set a unified difference function to calculate the difference value of the test data of each regression model, and take the regression model with the smallest difference value as the optimal regression model. Use a unified difference function to calculate the difference when different regression models process the same data. It should be noted that the difference function used here is different from the cost function used in the regression model. This can avoid overfitting and optimize the screening of regression models.
[0044] Use the optimal regression model to optimize data storage. Input the file data to be stored into the optimal regression model to optimize the storage of medical archive information, output the heat prediction results, and classify the heat prediction results into predicted heat levels. Move the files to be managed to the corresponding solid-state drive, mechanical hard drive or magnetic tape based on the predicted heat level. Reduce the space occupied by cold data in high-speed storage media, increase the storage and transmission speed of hot data, and optimize the storage structure of the entire medical archive file.
[0045] At the same time, when the above steps are executed, when the storage capacity of the medical file information storage device is insufficient, temporary storage space is added to the medical file information storage device using the memory of the user machine according to the user machine that needs to be stored, so that the fault tolerance rate of the method is improved.
[0046] When the medical archive file is updated, in response to detecting new medical archive file storage data, the new medical archive file storage data is added to the historical medical archive file storage data so that the historical medical archive file storage data is updated with the new medical archive file storage data.
[0047] Based on the updated historical medical archive file storage data, the unified difference function is reused to calculate the difference of the test data of each regression model. When the medical archive file is updated, the regression model is updated synchronously to avoid the original regression model being unable to complete the storage optimization operation of the current medical archive information storage device due to file update problems.
[0048] The acquired historical medical archive file storage data is cleaned, including removing null values, replacing abnormal values, processing invalid values and extracting features. The cleaned historical medical archive file storage data is divided into 80% training set and 20% test set.
[0049] Through data cleaning, the data that cannot be processed by the regression model is reduced, and the stability and accuracy of the regression model are improved.
[0050] Among them, the historical medical file information storage data includes the examination and detection data of the examination object, such as thyroid ultrasound examination images, breast ultrasound examination images, head CT images, blood routine examination and other information, and also includes the storage location, data type and data size of the above-mentioned examination and detection data.
[0051] The process of training several regression models and selecting the best regression model includes:
[0052] Parameters of each of the regression models are initialized, and the same number of training sets are allocated to each of the initialized regression models for archival information storage optimization training. Each regression model that has completed the first training is tested and the test data is recorded.
[0053] Each regression model is iteratively trained for archival information storage optimization, and each regression model after each iterative training is evaluated using the difference function. The difference function is used to calculate the difference value of the regression model to the test data, and the regression models are ranked according to the difference value. The smaller the difference value, the higher the ranking of the regression model.
[0054] The training is completed after all the historical medical archive file storage data in the training set are allocated. The regression model that is finally ranked first is taken as the optimal regression model. The optimal regression model is monitored regularly using updated data to monitor the impact of the updated data on the optimal regression model. If the updated data makes the current regression model unable to meet the demand, multiple regression models are retrained and the optimal regression model is ranked again.
[0055] Performing a unified evaluation on different regression models. Setting a unified difference function to calculate the difference value of the test data of each regression model includes the following steps:
[0056] First, get the average results of the test data analyzed by different regression models:
[0057]
[0058] Where x represents the number of regression models, represents the prediction result of the i-th regression model for the data y, Represents the average results of the analysis of data y using different regression models.
[0059] Calculate To make predictions for different regression models, we can avoid the overfitting of a certain regression model, which will affect the subsequent use of the overfitting model and make it unable to cope with other data.
[0060] Then, the measurement function MEAP of the regression model evaluation score is set. According to the value of the measurement function MEAP, the training effect of the regression model is judged. The expression is:
[0061]
[0062] in, And y n ≠0;
[0063] like And y n =0 is recorded as 0;
[0064] In the above formula, s represents the total amount of test data. Represents the predicted value of the nth data analysis, y n Represents the actual observation value of the nth data.
[0065] By calculation and n The difference of each model for the single data analysis is obtained. The smaller the value of the measurement standard function MEAP is, the better the corresponding regression model training effect is. The regression model with the smallest MEAP value is selected to optimize the storage of medical file information.
[0066] During the execution of the above steps, when the storage capacity of the medical archive information storage device is insufficient, temporary storage space is added to the medical archive information storage device using the memory of the user machine according to the user machine that needs to store.
[0067] When the remaining storage space of the medical file information storage device is less than a set value, for example, the set value is 16 GB, and the user has a need to upload the patient's medical file to the medical file information storage device, the medical file file storage device chooses to add new temporary storage space.
[0068] A certain temporary storage space, for example, 20GB, is allocated on the current user machine to expand the capacity of the medical archive information storage device, and the temporary storage space on the user machine is connected to the medical archive information storage device through a distributed architecture. And the access rights to the temporary storage space allocated on the user machine are controlled, and the newly allocated space is hidden in the user machine. Avoid data leakage caused by private access by users. Add the newly allocated space of the user machine as a temporary storage space to the medical archive file storage system. In this way, the fault tolerance performance of the medical archive information storage device can be achieved, and the failure of the method to be executed due to insufficient internal space can be reduced.
[0069] At the same time, when a user machine that has been bound to a temporary storage space subsequently uploads a patient's medical file, it only needs to upload the patient's medical file tag information and the user machine number to the medical file information storage device. When the patient's medical file is stored in a temporary storage space, retaining the tag information and the host number can bind the user machine number and the patient's medical file as a pointer, which is convenient for retrieval when extracting information later.
[0070] When the capacity of the medical archive information storage device is expanded, the medical archive data is read from the temporary storage space, and the medical archive file storage device transmits the data in the temporary storage space to the medical archive information storage device.
[0071] When the data in the temporary storage space is transmitted to the medical archive information storage device, the temporary storage space deletes the temporary data stored in itself, modifies the pointer of the data stored in the temporary storage space in the medical archive information storage device, and modifies the permission of the temporary storage space to occupy the storage space in the user machine. Modifying the pointer of the data stored in the temporary storage space in the medical archive information storage device can avoid the inability to retrieve the file due to the change of the file location, and at the same time modify the permission of the temporary storage space to occupy the storage space, and return the right to use the storage space to the user machine, without causing any negative impact on the user machine.
[0072] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for optimizing storage of medical archive files, characterized in that: The following steps are involved: Acquire historical medical archive file storage data from a medical archive information storage device, perform data cleaning on the historical medical archive file storage data, and divide the pre-processed historical medical archive file storage data into a training set and a test set; Pre-training a number of regression models, using the training set to train each regression model for predicting the heat of medical archive files, and using the test set to test the trained regression model, and recording the test results of each regression model; Setting a unified difference function to calculate the difference value of the test data of each regression model, evaluating different regression models, and obtaining the optimal regression model; Input the file data to be stored into the optimal regression model to optimize the storage of medical file information, and output the heat prediction result, divide the heat prediction result into predicted heat levels, and move the files to be managed to the corresponding solid-state hard disk, mechanical hard disk or magnetic tape based on the predicted heat level; When the storage capacity of the medical file information storage device is insufficient, temporary storage space is added to the medical file information storage device using the memory of the user machine according to the user machine that needs to store.
2. A method for optimizing storage of medical archive files according to claim 1, characterized in that: In response to detecting new medical archive file storage data, adding the new medical archive file data to the historical medical archive file storage data so that the historical medical archive file storage data is updated with the new medical archive file data; Based on the updated historical medical archive file storage data, the unified difference function is reused to calculate the difference of the test data of each regression model.
3. A medical archive file storage optimization method according to claim 1, characterized in that: The acquired historical medical archive file storage data is cleaned, including removing null values, replacing abnormal values, processing invalid values and extracting features. The cleaned historical medical archive file storage data is divided into 80% training set and 20% test set.
4. A method for optimizing storage of medical archive files according to claim 1, characterized in that: The process of training several regression models and selecting the best regression model includes: Initializing the parameters of each of the regression models, allocating the same number of training sets to each of the initialized regression models for archival information storage optimization training, testing each regression model that has completed the first training, and recording the test data; Perform iterative training for each regression model to optimize archival information storage, and use the difference function to evaluate each regression model after each iterative training. Use the difference function to calculate the difference value of the regression model to the test data, and rank the regression models according to the difference value. The smaller the difference value, the higher the ranking of the regression model. The training is completed after all the historical medical archive file storage data in the training set are allocated, and the regression model that is finally ranked first is taken as the optimal regression model, and the optimal regression model is monitored regularly using updated data.
5. A method for optimizing storage of medical archive files according to claim 1, characterized in that: Setting a unified difference function to calculate the difference value of each test data of the regression model includes: Get the average results of the test data analyzed by different regression models: Where x represents the number of regression models, represents the prediction result of the i-th regression model for the data y, It represents the average results of the analysis of data y by different regression models; Set the measurement function MEAP of the regression model evaluation score. According to the value of the measurement function MEAP, judge the training effect of the regression model. The expression is: in, And y n ≠0; like And y n =0 is recorded as 0; In the above formula, s represents the total amount of test data. Represents the predicted value of the nth data analysis, y n Represents the actual observation value of the nth data; The smaller the value of the measurement standard function MEAP is, the better the corresponding regression model training effect is.
6. A method for optimizing storage of medical archive files according to claim 1, characterized in that: The memory of the user machine is added as temporary storage space of the medical archive information storage device, including: When the remaining storage space of the medical archive information storage device is less than the set value, and the user has a need to upload the patient's medical archive to the medical archive information storage device, the medical archive file storage device chooses to add a new temporary storage space; Divide space on the current user machine to expand the capacity of the medical archive information storage device, control access rights to the space divided on the user machine, hide the newly divided space in the user machine, and add the newly divided space on the user machine as a temporary storage space to the medical archive file storage system; The user machine uploads the marking information of the patient's medical file and the user machine number to the medical file information storage device, and the patient's medical file is stored in the temporary storage space.
7. A method for optimizing storage of medical archive files according to claim 6, characterized in that: After the capacity of the medical archive information storage device is expanded, the medical archive data is read from the temporary storage space, and the medical archive file storage device transmits the data of the temporary storage space to the medical archive information storage device; After the transmission is completed, the temporary storage space deletes the temporary data stored in itself, modifies the pointer of the data stored in the temporary storage space in the medical archive information storage device, and modifies the permission of the temporary storage space to occupy the storage space in the user machine.
Citation Information
Patent Citations
Data storage method, first mobile terminal, storage node and storage system
CN105635214A
Power system short-term load probability forecasting method, device and system
CN109214605A
Capacity expansion method and device for storage system
CN110333824A
Short-term wind power prediction method based on IJS-SVR model
CN114707421A
Medical data security storage method based on popularity grading strategy
CN117195291A
