Data processing method and related device
By standardizing, structured and desensitizing the original data, the problem of inconsistent data formats between different institutions is solved, and efficient data sharing and secure sharing are achieved.
Patent Information
- Application Number
- CN202411958454.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-27
- Publication Date
- 2025-04-25
AI Technical Summary
The use of different data formats between electronic systems between different institutions leads to reduced data sharing efficiency, especially when information exchange and collaborative work efficiency between medical institutions is affected.
By standardizing and structuring the original data, the effective data is mapped into the standardized template, the entity and its association relationship are extracted, structured data is generated, and desensitized for sharing.
It unifies the data formats between different institutions, improves data sharing efficiency, breaks down information barriers, and ensures data security and privacy.
Smart Images
Figure CN119884028A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a data processing method and related devices. Background Art
[0002] With the rapid development of electronic information technology, electronic data is playing an increasingly important role in various fields due to its characteristics of easy management and storage.
[0003] Currently, many institutions use electronic systems to manage and store electronic data. However, electronic systems of different institutions often use different data formats to manage and store electronic data. This difference leads to a decrease in data sharing efficiency. Therefore, a solution is urgently needed to improve data sharing efficiency. Summary of the invention
[0004] Based on the above problems, the present application provides a data processing method and related devices.
[0005] The embodiments of the present application disclose the following technical solutions:
[0006] A first aspect: The present application provides a data processing method, including:
[0007] According to the type of valid data extracted from the original data, the valid data is mapped into a standardized template to obtain standardized data;
[0008] According to the entities extracted from the standardized data and the association relationships between the entities, the standardized data is structured to obtain structured data;
[0009] The structured data is shared.
[0010] In a possible implementation, mapping the valid data to a standardized template according to the type of the valid data extracted from the original data to obtain the standardized data includes:
[0011] Classifying the original data according to the valid data extracted from the original data to obtain the category of the original data;
[0012] According to the category of the original data and the type of valid data extracted from the original data, the valid data is mapped to a standardized template corresponding to the category of the original data to obtain standardized data.
[0013] In a possible implementation, mapping the valid data to a standardized template according to the type of the valid data extracted from the original data to obtain the standardized data includes:
[0014] According to the type of valid data extracted from the original data, the valid data is mapped into a standardized template to obtain sub-standardized data;
[0015] Determining an information integrity ratio of the sub-standardized data based on a ratio of the data volume of the sub-standardized data to the data volume of the original data;
[0016] When the information integrity ratio is greater than or equal to the integrity threshold, normalized data is obtained.
[0017] In a possible implementation manner, the sharing of the structured data includes:
[0018] Desensitizing the sensitive information identified from the structured data to obtain processed data;
[0019] The processed data is shared.
[0020] In a possible implementation, performing desensitization processing on the sensitive information identified from the structured data to obtain processed data includes:
[0021] De-identifying the identification information identified from the structured data; and / or,
[0022] According to the desensitization rules, the sensitive words identified from the structured data are desensitized to obtain processed data.
[0023] In a possible implementation manner, the de-identifying the identification information identified from the structured data includes:
[0024] The identification information identified from the structured data is replaced with the encrypted flag information or preset replacement information.
[0025] In a possible implementation manner, the de-identifying the identification information identified from the structured data includes:
[0026] When the identification information identified from the structured data is an image, identifying text information in the image to obtain the text information in the image;
[0027] The image is de-identified according to the text information in the image and the desensitization strategy.
[0028] In a possible implementation, the step of performing structured processing on the standardized data according to the entities extracted from the standardized data and the associations between the entities to obtain structured data includes:
[0029] Performing data cleaning on the standardized data to obtain cleaned data;
[0030] Entities extracted from the cleaned data are standardized based on professional terminology to obtain standardized entities;
[0031] The standardized entities and the relationships between the standardized entities are mapped to a structured data model to obtain structured data.
[0032] In a possible implementation, after the standardized data is structured according to the entities extracted from the standardized data and the associations between the entities to obtain the structured data, the method further includes:
[0033] Performing integrity verification on the structured data to obtain an integrity ratio of the structured data;
[0034] In a case where it is determined that the integrity ratio of the structured data is greater than or equal to a ratio threshold, the structured data is stored.
[0035] A second aspect: The embodiment of the present application provides a data processing device, including:
[0036] Standardized units, structured units, and shared units;
[0037] The standardization unit is used to map the valid data to a standardized template according to the type of the valid data extracted from the original data to obtain standardized data;
[0038] The structuring unit is used to perform structural processing on the standardized data according to the entities extracted from the standardized data and the association relationships between the entities to obtain structured data;
[0039] The sharing unit is used to share the structured data.
[0040] In a possible implementation manner, the standardization unit is specifically used to:
[0041] Classifying the original data according to the valid data extracted from the original data to obtain the category of the original data;
[0042] According to the category of the original data and the type of valid data extracted from the original data, the valid data is mapped to a standardized template corresponding to the category of the original data to obtain standardized data.
[0043] In a possible implementation manner, the standardization unit is specifically used to:
[0044] According to the type of valid data extracted from the original data, the valid data is mapped into a standardized template to obtain sub-standardized data;
[0045] Determining an information integrity ratio of the sub-standardized data based on a ratio of the data volume of the sub-standardized data to the data volume of the original data;
[0046] When the information integrity ratio is greater than or equal to the integrity threshold, normalized data is obtained.
[0047] In a possible implementation, the sharing unit is specifically configured to:
[0048] Desensitizing the sensitive information identified from the structured data to obtain processed data;
[0049] The processed data is shared.
[0050] In a possible implementation, the sharing unit is specifically configured to:
[0051] De-identifying the identification information identified from the structured data; and / or,
[0052] According to the desensitization rules, the sensitive words identified from the structured data are desensitized to obtain processed data.
[0053] In a possible implementation, the sharing unit is specifically configured to:
[0054] The identification information identified from the structured data is replaced with the encrypted flag information or preset replacement information.
[0055] In a possible implementation, the sharing unit is specifically configured to:
[0056] When the identification information identified from the structured data is an image, identifying text information in the image to obtain the text information in the image;
[0057] The image is de-identified according to the text information in the image and the desensitization strategy.
[0058] In a possible implementation, the structural unit is specifically used to:
[0059] Performing data cleaning on the standardized data to obtain cleaned data;
[0060] Entities extracted from the cleaned data are standardized based on professional terminology to obtain standardized entities;
[0061] The standardized entities and the relationships between the standardized entities are mapped to a structured data model to obtain structured data.
[0062] In a possible implementation, the device further includes a verification unit;
[0063] The verification unit is specifically used for:
[0064] Performing integrity verification on the structured data to obtain an integrity ratio of the structured data;
[0065] In a case where it is determined that the integrity ratio of the structured data is greater than or equal to a ratio threshold, the structured data is stored.
[0066] A third aspect: An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a data processing method as described in the first aspect above are implemented.
[0067] Compared with the prior art, this application has the following beneficial effects:
[0068] The embodiment of the present application provides a data processing method and related devices, which obtain standardized data by mapping the valid data to a standardized template according to the type of valid data extracted from the original data; perform structured processing on the standardized data according to the entities extracted from the standardized data and the association relationship between the entities to obtain structured data; and share the structured data. In the embodiment of the present application, the format of the original data is unified by standardizing and structuring the original data, which can break the information barriers between different institutions and effectively improve the efficiency of data sharing between different institutions. BRIEF DESCRIPTION OF THE DRAWINGS
[0069] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0070] Figure 1 A flowchart of a data processing method provided in an embodiment of the present application;
[0071] Figure 2 A flowchart of a data processing method provided in an embodiment of the present application;
[0072] Figure 3 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0073] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0074] Currently, many institutions use electronic systems to manage and store electronic data. However, the electronic systems of different institutions often use different data formats to manage and store electronic data. This difference leads to reduced data sharing efficiency.
[0075] Taking medical institutions as an example, many medical institutions use electronic systems to manage and store electronic data. However, since different medical institutions use different electronic systems, there are problems such as poor interoperability and inconsistent data formats between different electronic systems, which makes data sharing between different medical institutions difficult to achieve, affecting information exchange and collaborative work efficiency among medical staff.
[0076] Based on this, the embodiment of the present application provides a data processing method and related devices, by mapping the valid data to a standardized template according to the type of valid data extracted from the original data, to obtain standardized data; according to the entities extracted from the standardized data and the association relationship between the entities, the standardized data is structured to obtain structured data. After obtaining the structured data, the structured data can be shared. In the embodiment of the present application, by standardizing and structuring the original data, the format of the original data is unified, which can break the information barriers between different institutions and effectively improve the efficiency of data sharing between different institutions.
[0077] It should be noted that the user information and data involved in the embodiments of the present application (including but not limited to original data, data used for analysis, stored data, displayed data, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.
[0078] Combine the following Figure 1 A data processing method provided in an embodiment of the present application is introduced. Figure 1 A flowchart of a data processing method provided in an embodiment of the present application includes S101-S103.
[0079] S101. Map the valid data to a standardized template according to the type of the valid data extracted from the original data to obtain standardized data.
[0080] In the embodiments of the present application, the original data is introduced as a medical record document as an example.
[0081] Medical records are records of the occurrence, development, and outcome of a patient's disease, as well as the process of medical activities such as examination, diagnosis, and treatment. They are also medical health records of patients that are written in the prescribed format and requirements by summarizing, organizing, and comprehensively analyzing the collected information.
[0082] Medical records can have a variety of formats and types, for example: paper medical records are traditional handwritten or printed medical records; electronic medical records (EMR) are digitized medical records that are easy to store, retrieve and share; structured medical records are medical records that store medical record information in the form of data fields, which is convenient for data analysis and processing; unstructured medical records are medical records that store medical record information in free text form, such as medical staff's notes or reports.
[0083] Among them, structured medical records can exist in the form of electronic medical records; unstructured medical records can be paper medical records or electronic medical records.
[0084] As the core of medical data, medical records play an irreplaceable role in improving the quality of medical services, promoting medical progress, and protecting the rights and interests of patients. With the development of electronic information technology, many medical institutions use electronic systems to manage and store electronic medical records. However, the data format and structure of electronic medical records supported by different electronic systems may be inconsistent, resulting in reduced data sharing efficiency.
[0085] In the embodiment of the present application, the collected raw data in various data formats can be parsed to extract valid data from the raw data, wherein the valid data may include but is not limited to information such as text and images in the raw data.
[0086] In a possible implementation, the category of the original data can be obtained by classifying the original data according to the valid data extracted from the original data. According to the category of the original data and the type of the valid data extracted from the original data, the valid data is mapped to a standardized template corresponding to the category of the original data to obtain standardized data.
[0087] Exemplarily, the data format of the medical record document in the embodiment of the present application includes but is not limited to text file format (TXT), JavaScript Object Notation (JSON), extensible markup language (XML), etc. By parsing the collected medical record documents in various data formats, valid data in the medical record documents can be extracted.
[0088] By classifying the medical record document according to the valid data extracted from the medical record document, the category of the medical record document can be obtained. For example, the category of the medical record document may include but is not limited to admission record, discharge record, daily course of illness, first course of illness, first course of illness by a senior physician, operation record, death record, etc.
[0089] It can be understood that in the embodiment of the present application, since medical records can come from multiple different electronic systems, the method of classifying medical records based on valid data can be determined according to actual conditions. For example, the method of classifying medical records based on valid data can include but is not limited to three methods: a classification method based on template coding, a classification method based on topic (Topic) and a classification method based on document content.
[0090] Among them, the classification method based on template coding and the classification method based on topic can be realized through mapping and regularization; the classification method based on document content can be realized by training the classification model based on the random forest algorithm.
[0091] In an embodiment of the present application, after determining the category of the medical record document, the valid data can be mapped to a standardized template corresponding to the category of the medical record document according to the category of the medical record document and the type of valid data extracted from the medical record document to obtain standardized data.
[0092] For example, in the embodiment of the present application, the standardized template may be a Hypertext Markup Language (HTML) template. As the basis for building web pages, HTML has the characteristics of simple and intuitive syntax, cross-platform compatibility, and easy maintenance and updating.
[0093] In one example, the HTML template can be pre-designed according to the requirements. In the process of designing the HTML template, the standard structure and style of various types of medical records can be defined, including but not limited to the header, body, footer and other parts of various types of medical records.
[0094] For example, when designing the HTML template corresponding to the admission record, the titles such as chief complaint, current medical history, past medical history, menstrual and marital history, family history, physical examination, diagnosis, signature, etc. can be arranged and displayed in order to form the HTML template of the admission record.
[0095] According to the category of the medical record document and the type of valid data extracted from the medical record document, the valid data is mapped to a standardized template corresponding to the category of the medical record document to obtain standardized data.
[0096] Exemplarily, when the category of the medical record document is an admission record, the types of valid data may include but are not limited to the chief complaint, current medical history, past medical history, menstrual marital history, family history, etc. Taking the type of valid data as the chief complaint as an example, the chief complaint content extracted from the medical record document is mapped to the position corresponding to the chief complaint title in the HTML template of the admission record. Similarly, when the type of valid data is the current medical history, the current medical history content extracted from the medical record document is mapped to the position corresponding to the current medical history title in the HTML template of the admission record. By analogy, after mapping each valid data in the admission record to the HTML template of the admission record, the standardized data corresponding to the admission record can be obtained.
[0097] In a possible implementation, when the original data is text content, natural language processing (NLP) word segmentation can be performed on the text content, and chapter information in the extracted text content is used as valid data.
[0098] In a possible implementation, in order to ensure the accuracy of the standardized data, the quality of the standardized data may be evaluated according to the calculated information integrity ratio.
[0099] In the embodiment of the present application, according to the type of valid data extracted from the original data, the valid data is mapped to the standardized template to obtain sub-standardized data. Based on the ratio of the data volume of the sub-standardized data to the data volume of the original data, the information integrity ratio of the sub-standardized data is determined; when the information integrity ratio is greater than or equal to the integrity threshold, the standardized data is obtained.
[0100] For example, the information integrity ratio (IIR) can be expressed by formula (1) as follows:
[0101] IIR = the amount of sub-normalized data / the amount of original data (1)
[0102] When the information integrity ratio is 1, it means that the process of converting the original data into the sub-normalized data is lossless; when the information integrity ratio is less than 1, it means that the process of converting the original data into the sub-normalized data is lossy.
[0103] In one example, when the information integrity ratio is greater than or equal to the integrity threshold, it indicates that the data loss is small, and the sub-standardized data can be determined as the standardized data; when the information integrity ratio is less than the integrity threshold, it indicates that the data loss is large. At this time, the valid data can be re-extracted and mapped to the corresponding standardized template, and the information integrity ratio can be recalculated until the information integrity ratio is greater than or equal to the integrity threshold. The sub-standardized data at this time can be determined as the standardized data, or manual participation can be performed to improve the quality of the standardized data.
[0104] In another example, the quality of the standardized data can be evaluated by using the content loss ratio (CLR), which can be expressed as follows using formula (2):
[0105] CLR=1-IIR (2)
[0106] The content loss rate is used to indicate the amount of data lost in the process of converting raw data into standardized data. When the content loss rate is greater than the loss rate threshold, it means that the data loss is large. At this time, the valid data can be re-extracted and mapped to the corresponding standardized template, and the content loss rate can be recalculated until the content loss rate is less than or equal to the loss rate threshold. The sub-standardized data at this time is determined as standardized data, or manual intervention is required to improve the quality of standardized data. When the content loss rate is less than or equal to the loss rate threshold, it means that the data loss is small, and the sub-standardized data can be directly determined as standardized data.
[0107] It is understandable that in the embodiment of the present application, the numerical value of the integrity threshold is not specifically limited and can be set according to actual conditions.
[0108] S102: According to the entities extracted from the standardized data and the association relationships between the entities, the standardized data is structured to obtain structured data.
[0109] In a possible implementation, S102 may be implemented by steps 21 to 23.
[0110] Step 21: Perform data cleaning on the standardized data to obtain cleaned data.
[0111] The process of data cleaning of standardized data may include but is not limited to removing duplicate records, correcting erroneous data, unifying data formats, etc.
[0112] Step 22: extract entities from the cleaned data, and standardize the entities based on professional terminology to obtain standardized entities.
[0113] In the embodiment of the present application, the entities extracted from the cleaned data may include but are not limited to disease names, drug names, dosages, etc. After the entities are extracted, the extracted entities may be standardized into a medical terminology set or coding system.
[0114] For example, the same disease may have multiple different names, and to facilitate subsequent data sharing, they may be standardized based on professional terminology.
[0115] Step 23: Map the standardized entities and the relationships between the standardized entities to a structured data model to obtain structured data.
[0116] After obtaining the standardized entities, the associations between the standardized entities can be determined, such as the association between patients and diagnoses, and the association between drug names and dosages. The standardized entities and the associations between the standardized entities are mapped to a structured data model, and data fields are created for each standardized entity to obtain structured data.
[0117] In a possible implementation, to ensure the accuracy and completeness of structured data, after the standardized entities and the associations between the standardized entities are mapped to the structured data model, the structured data may be integrity verified to obtain the integrity ratio of the structured data.
[0118] When it is determined that the integrity ratio of the structured data is greater than or equal to the ratio threshold, the structured data is stored. In the embodiment of the present application, the verified structured data is stored in a suitable data storage system, which can facilitate subsequent search, query, and analysis processes.
[0119] It is understandable that the numerical value of the ratio threshold in the embodiment of the present application is not specifically limited and can be set according to actual conditions.
[0120] S103: Share the structured data.
[0121] In the embodiment of the present application, the structured data has a unified format, and after obtaining the structured data, the structured data can be directly shared among various organizations. By sharing the structured data, the information barriers between different organizations can be broken, effectively improving the efficiency of data sharing between different organizations.
[0122] In a possible implementation, in order to improve the security of data sharing, after obtaining structured data, sensitive information identified from the structured data may be desensitized to obtain processed data, and the processed data may be shared.
[0123] Exemplarily, desensitizing the sensitive information identified from the structured data includes de-identifying the identification information identified from the structured data; and / or desensitizing the sensitive words identified from the structured data according to the desensitization rules to obtain processed data. The desensitization rules include but are not limited to the Health Insurance Portability and Accountability Act and data classification requirements.
[0124] When it is determined that identification information exists in the structured data by identifying the structured data, the identification information identified from the structured data may be replaced with encrypted flag information or preset replacement information.
[0125] For example, the identification information may be a personal identifier in a medical record, such as a name, telephone number, etc., and the identified personal identifier may be replaced with the encrypted identification information or the preset replacement information. For example, the name identified in the medical record may be replaced with "Name 1", "Name 2", etc.; if the identified telephone number is 12345678901, the telephone number identified in the medical record may be replaced with "123*****901".
[0126] In one example, when the identification information identified from the structured data is an image, text information in the image can be identified to obtain the text information in the image; and the image can be de-identified based on the text information in the image and a desensitization strategy.
[0127] For example, the structured data may include an electronic signature in a picture format, a logo of a medical institution, etc. By recognizing the image based on a neural network model algorithm, text information in the image can be obtained.
[0128] The text information is compared with the doctor's name and medical institution name in the standardized template or outpatient visit record to determine whether the image is the doctor's electronic signature or the medical institution's logo.
[0129] In the case where it is determined that the image is a doctor's electronic signature or a logo of a medical institution, the image may be de-identified based on a corresponding desensitization strategy.
[0130] For example, if the image is determined to be the electronic signature of a doctor of a first medical institution or the logo of the first medical institution, and the structured data is used for data sharing within the first medical institution, then the image does not need to be de-identified. If the structured data is used for data sharing between the first medical institution and the second medical institution, the image can be de-identified.
[0131] In an embodiment of the present application, the sensitive words identified from the structured data may be desensitized according to a desensitization rule to obtain processed data. The desensitization rule includes at least one of the health insurance portability and liability act and data classification requirements as an example for introduction.
[0132] In one possible implementation, sensitive words in structured data may be identified based on the Health Insurance Portability and Accountability Act so as to be desensitized.
[0133] Among them, the Health Insurance Portability and Accountability Act (HIPAA) is an important health insurance law that stipulates the protection standards, security requirements, data breach notification rules, and electronic transaction standards for personal health information. The implementation of this law is of great significance for protecting the privacy and security of personal health information, promoting the sharing and interoperability of medical information, and improving the quality and efficiency of medical care.
[0134] For example, HIPAA stipulates the protection standards for personal health information, including the collection, use, disclosure and storage of information. Without the individual's authorization, covered entities shall not use protected health information (PHI) for marketing purposes or disclose it to third parties.
[0135] In another possible implementation, sensitive words in structured data may be identified based on data classification requirements so as to perform desensitization processing on them.
[0136] Data classification requirements are used to indicate the classification of various data. For example, in the process of identifying sensitive words from the structured data based on the data classification requirements and desensitizing the words, if the disease level is determined to be level one, it is desensitized, and if the disease level is determined to be level two, it is not desensitized. In the case where the data classification requirements indicate that the level of disease A is level one and the level of disease B is level two, disease A can be desensitized and disease B can be left untreated.
[0137] It should be noted that the embodiments of the present application do not specifically limit how to classify data, and it can be set according to actual needs. The above is only an example.
[0138] In another possible implementation, the embodiments of the present application may comprehensively consider the Health Insurance Portability and Accountability Act and data classification requirements, identify sensitive words from the structured data based on the two, and perform desensitization processing on the structured data to obtain processed data.
[0139] In summary, in the embodiments of the present application, the format of the original data is unified by standardizing, structuring and desensitizing the original data, which can break the information barriers between different institutions and effectively improve the efficiency of data sharing between different institutions.
[0140] By adopting HTML templates, the management process of medical records is simplified and the need for data storage is reduced. At the same time, by desensitizing structured data, the security and privacy of data are guaranteed, facilitating data sharing between different institutions.
[0141] For ease of understanding, the following Figure 2 A data processing method provided in an embodiment of the present application is generally introduced. Figure 2 A flowchart of a data processing method provided in an embodiment of the present application.
[0142] Taking the original medical record as an example, the original medical record is parsed to extract valid data from the medical record, determine the type of the medical record based on the valid data, and determine the HTML template corresponding to the medical record type from multiple HTML templates based on the type of the medical record.
[0143] The valid data is mapped to the HTML template corresponding to the type of the medical record document. In this process, the valid data can be processed by NLP based on the knowledge base to obtain chapter information, and the chapter information is mapped to the HTML template to obtain sub-standardized data.
[0144] The sub-standardized data is subjected to HTML verification, that is, based on the ratio of the data volume of the sub-standardized data to the data volume of the original data, the information integrity ratio of the sub-standardized data is determined; when the information integrity ratio is greater than or equal to the integrity threshold, the standardized data is obtained.
[0145] The standardized data is cleaned to obtain cleaned data. Based on the knowledge base, entity recognition is performed on the cleaned data, and the entities are standardized based on professional terms to obtain standardized entities.
[0146] After obtaining the standardized entities, relationships are extracted from the standardized entities based on the knowledge base, the associations between the standardized entities are determined, and the standardized entities and the associations between the standardized entities are mapped to a structured data model to obtain structured data.
[0147] After obtaining the structured data, the structured data is verified. If the data verification is qualified, the structured data is stored; if the data verification is unqualified, the data is cleaned again, and manual intervention is required until qualified structured data is obtained.
[0148] Desensitizing the structured data to obtain processed data, wherein the desensitizing process includes de-identifying identification information identified from the structured data; and / or desensitizing sensitive words identified from the structured data in accordance with at least one of the Health Insurance Portability and Accountability Act and data classification requirements.
[0149] After the processed data is obtained, the processed data can be shared among different medical institutions. In the embodiment of the present application, the format of the medical record documents is unified, the security and privacy of the medical record documents are ensured, and the efficiency of data sharing can be improved while ensuring data security by standardizing, structuring and desensitizing the original medical record documents.
[0150] The present application provides a data processing device, see Figure 3 , this figure is a structural schematic diagram of a data processing device provided in an embodiment of the present application. Its specific implementation method is consistent with the implementation method and the technical effect achieved in the embodiment of the above method, and some contents will not be repeated here.
[0151] The present embodiment of the application provides a data processing device 1100, including:
[0152] Standardized unit 1101, structured unit 1102 and shared unit 1103;
[0153] The standardization unit 1101 is used to map the valid data to a standardized template according to the type of the valid data extracted from the original data to obtain standardized data;
[0154] The structuring unit 1102 is used to perform structuring processing on the standardized data according to the entities extracted from the standardized data and the association relationships between the entities to obtain structured data;
[0155] The sharing unit 1103 is used to share the structured data.
[0156] In a possible implementation manner, the standardization unit is specifically used to:
[0157] Classifying the original data according to the valid data extracted from the original data to obtain the category of the original data;
[0158] According to the category of the original data and the type of valid data extracted from the original data, the valid data is mapped to a standardized template corresponding to the category of the original data to obtain standardized data.
[0159] In a possible implementation manner, the standardization unit is specifically used to:
[0160] According to the type of valid data extracted from the original data, the valid data is mapped into a standardized template to obtain sub-standardized data;
[0161] Determining an information integrity ratio of the sub-standardized data based on a ratio of the data volume of the sub-standardized data to the data volume of the original data;
[0162] When the information integrity ratio is greater than or equal to the integrity threshold, normalized data is obtained.
[0163] In a possible implementation, the sharing unit is specifically configured to:
[0164] Desensitizing the sensitive information identified from the structured data to obtain processed data;
[0165] The processed data is shared.
[0166] In a possible implementation, the sharing unit is specifically configured to:
[0167] De-identifying the identification information identified from the structured data; and / or,
[0168] According to the desensitization rules, the sensitive words identified from the structured data are desensitized to obtain processed data.
[0169] In a possible implementation, the sharing unit is specifically configured to:
[0170] The identification information identified from the structured data is replaced with the encrypted flag information or preset replacement information.
[0171] In a possible implementation, the sharing unit is specifically configured to:
[0172] When the identification information identified from the structured data is an image, identifying text information in the image to obtain the text information in the image;
[0173] The image is de-identified according to the text information in the image and the desensitization strategy.
[0174] In a possible implementation, the structural unit is specifically used to:
[0175] Performing data cleaning on the standardized data to obtain cleaned data;
[0176] Entities extracted from the cleaned data are standardized based on professional terminology to obtain standardized entities;
[0177] The standardized entities and the relationships between the standardized entities are mapped to a structured data model to obtain structured data.
[0178] In a possible implementation, the device further includes a verification unit and a storage unit;
[0179] The verification unit is used to perform integrity verification on the structured data to obtain an integrity ratio of the structured data;
[0180] The storage unit is used to store the structured data when it is determined that the integrity ratio of the structured data is greater than or equal to a ratio threshold.
[0181] The embodiment of the present application provides a data processing device that standardizes and structures the original data to unify the format of the original data, which can break the information barriers between different institutions and effectively improve the efficiency of data sharing between different institutions. At the same time, by desensitizing the structured data, the security and privacy of the data are guaranteed, which facilitates data sharing between different institutions.
[0182] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of a data processing method as described above are implemented.
[0183] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A data processing method, characterized in that: include: According to the type of valid data extracted from the original data, the valid data is mapped into a standardized template to obtain standardized data; According to the entities extracted from the standardized data and the association relationships between the entities, the standardized data is structured to obtain structured data; The structured data is shared.
2. The method according to claim 1, characterized in that The step of mapping the valid data to a standardized template according to the type of the valid data extracted from the original data to obtain the standardized data includes: Classifying the original data according to the valid data extracted from the original data to obtain the category of the original data; According to the category of the original data and the type of valid data extracted from the original data, the valid data is mapped to a standardized template corresponding to the category of the original data to obtain standardized data.
3. The method according to claim 1, characterized in that The step of mapping the valid data to a standardized template according to the type of the valid data extracted from the original data to obtain the standardized data includes: According to the type of valid data extracted from the original data, the valid data is mapped into a standardized template to obtain sub-standardized data; Determining an information integrity ratio of the sub-standardized data based on a ratio of the data volume of the sub-standardized data to the data volume of the original data; When the information integrity ratio is greater than or equal to the integrity threshold, normalized data is obtained.
4. The method according to any one of claim 1, characterized in that: The sharing of the structured data comprises: Desensitizing the sensitive information identified from the structured data to obtain processed data; The processed data is shared.
5. The method according to claim 4, characterized in that The desensitizing the sensitive information identified from the structured data to obtain processed data includes: De-identifying the identification information identified from the structured data; and / or, According to the desensitization rules, the sensitive words identified from the structured data are desensitized to obtain processed data.
6. The method according to claim 5, characterized in that The de-identifying the identification information identified from the structured data includes: The identification information identified from the structured data is replaced with the encrypted flag information or preset replacement information.
7. The method according to claim 5, characterized in that The de-identifying the identification information identified from the structured data includes: When the identification information identified from the structured data is an image, identifying text information in the image to obtain the text information in the image; The image is de-identified according to the text information in the image and the desensitization strategy.
8. The method according to claim 1, characterized in that The step of performing structural processing on the standardized data according to the entities extracted from the standardized data and the association relationships between the entities to obtain structured data includes: Performing data cleaning on the standardized data to obtain cleaned data; Entities extracted from the cleaned data are standardized based on professional terminology to obtain standardized entities; The standardized entities and the relationships between the standardized entities are mapped to a structured data model to obtain structured data.
9. The method according to any one of claims 1 to 8, characterized in that: After the standardized data is structured according to the entities extracted from the standardized data and the associations between the entities to obtain the structured data, the method further includes: Performing integrity verification on the structured data to obtain an integrity ratio of the structured data; In a case where it is determined that the integrity ratio of the structured data is greater than or equal to a ratio threshold, the structured data is stored.
10. A data processing device, characterized in that: include: Standardized units, structured units, and shared units; The standardization unit is used to map the valid data to a standardized template according to the type of the valid data extracted from the original data to obtain standardized data; The structuring unit is used to perform structural processing on the standardized data according to the entities extracted from the standardized data and the association relationships between the entities to obtain structured data; The sharing unit is used to share the structured data.