A data pool construction method and system suitable for artificial intelligence Chinese medicine syndrome differentiation modeling
By extracting and preprocessing traditional Chinese medicine clinical records in real time, calculating cosine similarity and dynamically adjusting the extraction speed, the problems of data redundancy and low storage efficiency in traditional Chinese medicine syndrome differentiation are solved, and data quality and model generalization ability are improved.
Patent Information
- Application Number
- CN202411273164.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-12
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-09-12
AI Technical Summary
During the traditional Chinese medicine diagnosis process, there are problems such as data dispersion, inconsistent format, low information extraction efficiency and high data repetition rate, resulting in low data redundancy and low storage efficiency. It is difficult for the existing technology to effectively evaluate data when processing similar or repeated clinical records. Similarity.
By extracting and preprocessing clinical records from the traditional Chinese medicine clinical record database in real time, using natural language processing technology to calculate cosine similarity, deleting or adjusting the number of diagnosis, dynamically adjusting the extraction speed and response time, and numerical processing combined with the bag of words model to form suitable for Data pool for artificial intelligence traditional Chinese medicine dialectical modeling.
Improve the timeliness and consistency of data, reduce data redundancy, enhance data readability and analytics, improve system flexibility and responsiveness, avoid overfitting problems in model training, and improve Data quality and generalization capabilities of models are achieved.
Smart Images

Figure CN118779293B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data pool construction, and in particular to a data pool construction method and system suitable for artificial intelligence Chinese medicine syndrome differentiation modeling. Background Art
[0002] With the rapid development of artificial intelligence technology, especially its increasing application in the field of medical health, the demand for high-quality, structured data is becoming increasingly urgent. The background of this solution is based on this demand, aiming to solve the problems of data dispersion, inconsistent format, low information extraction efficiency and high data duplication rate in the process of traditional Chinese medicine differentiation. In the field of traditional Chinese medicine, despite the rich knowledge of ancient books and clinical practice experience, how to efficiently convert these unstructured data into a format that can be used for artificial intelligence model learning is still a challenge.
[0003] The patent document with publication number CN117591678A discloses a data management system for constructing a knowledge graph of ancient Chinese medical books, which includes: a data acquisition unit, an information storage unit, a graph processing unit, and a fusion display unit. The system combines structured data with unstructured content by automatically processing knowledge content text, independently constructed knowledge graph models, and extraction channels. It can be seen that the system focuses on the structuring and graphing of knowledge, and is not comprehensive enough for the deep processing and analysis of data, such as the application of natural language processing technology, and lacks a comprehensive evaluation and deduplication mechanism for data similarity, especially for clinical records, which contain a large amount of similar or repeated information. In addition, the prior art often causes the data pool to occupy too much memory when processing a large number of similar or repeated clinical records, resulting in data redundancy and low storage efficiency due to insufficient data similarity evaluation. Summary of the invention
[0004] To this end, the present invention provides a data pool construction method and system suitable for artificial intelligence Chinese medicine syndrome differentiation modeling, so as to overcome the problems of data redundancy and low storage efficiency caused by insufficient data similarity evaluation in the prior art.
[0005] To achieve the above-mentioned purpose, the present invention provides a data pool construction method suitable for artificial intelligence TCM syndrome differentiation modeling, comprising:
[0006] Extracting clinical records of several targets in real time from a preset TCM clinical record database at a preset extraction speed, and preprocessing the clinical records to obtain several standardized clinical records;
[0007] Obtain the recording time of each standardized clinical record, make a judgment based on all the recording times and a preset time range, extract keywords for the diagnosis results of the preset diagnosis times in the standardized clinical record according to the judgment result, calculate the cosine similarity of all the extracted keywords, make a judgment based on the cosine similarity and the preset standard similarity, delete the corresponding standardized clinical record according to the judgment result, or correct the diagnosis times, extract keywords for the diagnosis results of the corrected diagnosis times in the standardized clinical record until the corresponding standardized clinical record is deleted according to the comparison result;
[0008] Obtain the number of deletions of the standardized clinical records corresponding to the deletions, and adjust the extraction speed according to the number of deletions and the preset standard number of deletions;
[0009] Obtaining the time from extracting clinical records of a number of targets at a time to obtaining a number of standardized clinical records, and correcting the adjusted extraction speed according to a preset number of responses, duration, and a preset standard response duration to obtain a corrected extraction speed;
[0010] All standardized clinical records extracted and preprocessed from the TCM clinical record database at a modified extraction speed in real time are integrated to obtain and output a data pool suitable for artificial intelligence TCM syndrome differentiation modeling.
[0011] Further, when the recording time of the standardized clinical record is within a preset time range, it is determined that keywords need to be extracted.
[0012] Furthermore, calculating the cosine similarity of all extraction results includes:
[0013] All keywords are converted into numerical vectors through the bag-of-words model, all numerical vectors are stored in a matrix, and the cosine similarity between every two numerical vectors in the matrix is calculated.
[0014] Furthermore, when the cosine similarity is greater than a preset standard similarity, and the difference between the cosine similarity and the standard similarity is greater than a preset standard similarity difference, the previous standardized clinical record is deleted.
[0015] Furthermore, when the cosine similarity is greater than a preset standard similarity and the difference between the cosine similarity and the standard similarity is less than or equal to a preset standard similarity difference, the number of diagnostic times is reduced and adjusted according to the difference between the cosine similarity and the standard similarity, the standard similarity difference and a preset diagnostic time coefficient.
[0016] Furthermore, when the number of deletions is greater than the maximum value of the standard deletion number range, the extraction speed is adjusted to be reduced according to the number of deletions, the maximum value of the standard deletion number range and a preset extraction speed coefficient.
[0017] Furthermore, when the deletion quantity is less than the minimum value of the standard deletion quantity range, the extraction speed is adjusted according to the deletion quantity, the minimum value of the standard deletion quantity range and a preset extraction speed coefficient.
[0018] Furthermore, the ratio of the sum of the response times to the response times is calculated to obtain the average response time. When the average response time is greater than the standard response time, the adjusted extraction speed is reduced according to the average response time, the standard response time and a preset correction coefficient.
[0019] Furthermore, when the average response time is less than or equal to the standard response time, the adjusted extraction speed is reduced according to the average response time, the standard response time and a preset correction coefficient.
[0020] A data pool construction system suitable for artificial intelligence TCM syndrome differentiation modeling is constructed based on the above-mentioned data pool construction method suitable for artificial intelligence TCM syndrome differentiation modeling, comprising:
[0021] An extraction module is used to extract clinical records of several targets in real time from a preset TCM clinical record database at a preset extraction speed, and pre-process the clinical records to obtain several standardized clinical records;
[0022] a processing module connected to the extraction module, for obtaining the recording time of each standardized clinical record, making a judgment based on all the recording times and a preset time range, extracting keywords from the diagnosis results of the preset diagnosis times in the standardized clinical record according to the judgment result, calculating the cosine similarity of all the extracted keywords, making a judgment based on the cosine similarity and the preset standard similarity, deleting the corresponding standardized clinical record according to the judgment result, or correcting the diagnosis times, extracting keywords from the diagnosis results of the corrected diagnosis times in the standardized clinical record until the corresponding standardized clinical record is deleted according to the comparison result;
[0023] An adjustment module, connected to the processing module, for obtaining the deletion number of the standardized clinical records corresponding to the deletion, and adjusting the extraction speed according to the deletion number and a preset standard deletion number;
[0024] A correction module, connected to the processing module and the adjustment module respectively, for obtaining the time duration from extracting a plurality of target clinical records at a time to obtaining a plurality of standardized clinical records, and correcting the adjusted extraction speed according to a preset number of responses, time duration and a preset standard response time duration to obtain a corrected extraction speed;
[0025] The integration output module is connected to the processing module and the correction module respectively, and is used to integrate all standardized clinical records extracted and pre-processed from the TCM clinical record database in real time at the corrected extraction speed, obtain a data pool suitable for artificial intelligence TCM syndrome differentiation modeling and output it.
[0026] Compared with the prior art, the beneficial effect of the present invention is that, through real-time extraction and standardized processing, the timeliness and consistency of data are ensured, and the natural language processing technology is used to accurately replace terms, which enhances the readability and analyzability of data, so that the artificial intelligence system can more accurately understand and process TCM clinical records. By calculating the cosine similarity and deleting records or adjusting the number of diagnoses according to the standard similarity, data redundancy is effectively reduced. In addition, the mechanism of dynamically adjusting the extraction speed and response time ensures that the data pool can adapt to different processing needs and real-time requirements, improves the flexibility and responsiveness of the system, and effectively solves the problem of poor model training effect and low prediction accuracy caused by data redundancy and low storage efficiency due to insufficient data similarity evaluation.
[0027] Furthermore, by limiting the time range and similarity threshold, data redundancy is effectively reduced, and the relevance and novelty of data are enhanced. The mechanism of dynamically adjusting the extraction speed and response time further improves the real-time and adaptability of the data pool.
[0028] Furthermore, the efficiency and accuracy of data processing are improved through numerical processing. The numerical vectorization of the bag-of-words model allows text data to be effectively processed by machine learning algorithms, while the calculation of cosine similarity provides a quantitative means for identifying and processing similar texts. It helps to identify and delete duplicate or highly similar records in the data pool, thereby optimizing data quality and reducing data redundancy.
[0029] Furthermore, by deleting highly similar records, we not only reduce data redundancy, but also avoid overfitting problems that may occur during model training. It also helps improve the generalization ability of the model because the diversity in the dataset is enhanced.
[0030] Furthermore, by reducing the number of diagnoses, it is possible to avoid duplication of information due to the high similarity between records, while retaining key diagnostic information, enhancing the practicality of the data and the depth of clinical records, and helping to improve the overall quality of the data pool.
[0031] Furthermore, by reducing the speed when too many deletions are made, rapid loss of data and potential loss of information is avoided.
[0032] Furthermore, increasing the speed when deleting insufficient data ensures timely replenishment and diversity of data, and guarantees the stability and continuous updating of the data pool.
[0033] Furthermore, by monitoring and adjusting the average response time, we ensure that the response speed of the data pool construction process is always maintained at a level that satisfies users. When the actual response time exceeds expectations, we reduce the extraction speed to provide the system with more time to process each record.
[0034] Furthermore, by appropriately reducing the extraction speed when the average response time meets or is lower than the standard, it is possible to avoid waste of resources due to excess processing capacity while maintaining the smoothness of data processing and the response speed of the system.
[0035] Furthermore, through automated and intelligent processes, the efficiency of data pool construction and the consistency of data quality are ensured. The modular design improves the flexibility and maintainability of the system, allowing each link to be independently adjusted and optimized according to actual conditions. By dynamically adjusting the extraction speed and the number of diagnoses, the system can adaptively respond to changes in data similarity and processing time, reducing data redundancy. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 This is a flow chart of a method for constructing a data pool suitable for artificial intelligence TCM syndrome differentiation modeling in this embodiment;
[0037] Figure 2 Extract the decision logic diagram of keyword decision for this embodiment;
[0038] Figure 3 The decision logic diagram for deleting the standardized clinical record decision for this embodiment;
[0039] Figure 4 This is a schematic diagram of a data pool construction system suitable for artificial intelligence TCM syndrome differentiation modeling in this embodiment. DETAILED DESCRIPTION
[0040] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0041] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.
[0042] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.
[0043] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.
[0044] See also Figure 1 As shown, it is a flow chart of the data pool construction method applicable to artificial intelligence TCM syndrome differentiation modeling in this embodiment;
[0045] This embodiment provides a data pool construction method suitable for artificial intelligence TCM syndrome differentiation modeling, including:
[0046] Extracting a number of target clinical records from a preset TCM clinical record database in real time at a preset extraction speed, replacing the terms and formats of the extraction results according to a preset term library through natural language processing technology, and standardizing the data format of the replaced extraction results to obtain a number of standardized clinical records;
[0047] Obtain the recording time of each standardized clinical record, make a judgment based on all the recording times and a preset time range, extract keywords for the diagnosis results of the preset diagnosis times in the standardized clinical record according to the judgment result, calculate the cosine similarity of all the extracted keywords, make a judgment based on the cosine similarity and the preset standard similarity, delete the corresponding standardized clinical record according to the judgment result, or correct the diagnosis times, extract keywords for the diagnosis results of the corrected diagnosis times in the standardized clinical record until the corresponding standardized clinical record is deleted according to the comparison result;
[0048] Obtain the number of deletions of the standardized clinical records corresponding to the deletions, and adjust the extraction speed according to the number of deletions and the preset standard number of deletions;
[0049] Obtaining the time from extracting clinical records of a number of targets at a time to obtaining a number of standardized clinical records, and correcting the adjusted extraction speed according to a preset number of responses, duration, and a preset standard response duration to obtain a corrected extraction speed;
[0050] All standardized clinical records extracted and preprocessed from the TCM clinical record database at a modified extraction speed in real time are integrated to obtain and output a data pool suitable for artificial intelligence TCM syndrome differentiation modeling.
[0051] The preset TCM clinical record database is a database for storing TCM clinical data, including basic information of the target, clinical record information, and diagnosis results;
[0052] Natural language processing technology is a series of technologies used to process and analyze natural language data, including text analysis, semantic understanding, machine translation, etc. In TCM syndrome differentiation modeling, NLP technology can help extract and understand text information in clinical records. In this embodiment, the BERT model is used to understand the complex text in TCM clinical records and extract key information.
[0053] The preset terminology library is a database containing professional terms of traditional Chinese medicine, which is used to help the NLP system correctly identify and process TCM-related vocabulary. The terminology library can include disease names, symptom descriptions, treatment methods, etc. The terminology library in this embodiment includes at least the terms "qi deficiency", "damp heat", and "acupuncture".
[0054] Data format standardization is the process of converting data from different sources and formats into a unified format for easy processing and analysis. This includes standardizing data fields, data types, and data structures.
[0055] In this embodiment, the date format of all clinical records is unified into "YYYY-MM-DD", and all Chinese medicine names are unified into standard names, such as "Ginseng" is unified into "Radix Ginseng".
[0056] The keywords of the diagnosis result are keywords or phrases related to the diagnosis result extracted from the clinical record. These keywords can help quickly identify and understand the diagnosis information. In this embodiment, the keywords extracted from a TCM diagnosis record may include "excessive liver fire", "weak spleen and stomach", and "qi stagnation and blood stasis".
[0057] Cosine similarity is a similarity measure for measuring the angle between two vectors. In text analysis, it is often used to calculate the similarity between text vectors to identify similar text or documents. In this embodiment, the bag-of-words model is used to convert the text into a vector, and then the cosine similarity of two TCM clinical records is calculated to determine whether they describe similar diagnostic results. A vocabulary is constructed for all words in TCM clinical records, and each word is assigned a unique index. Each TCM clinical record is converted into a vector. When using the bag-of-words model, the dimension of each vector corresponds to a word in the vocabulary, and the value of the vector usually represents the frequency of occurrence (word frequency) or word frequency-inverse document frequency of the word in the text. The cosine similarity is further calculated by calculating the dot product and modulus of the vector.
[0058] The preset extraction speed refers to the rate at which data is extracted from the TCM clinical record database. It depends on the amount of data, processing power, storage capacity, and real-time requirements. It is usually set to a few records per second to hundreds of records per second. In this embodiment, it is set to extract 10 records per second. Ensure that the data flow is stable and the system's processing capacity is not overloaded, while meeting real-time requirements.
[0059] The preset time range refers to the time span of clinical records considered in the data pool. It depends on the research purpose, the timeliness of the data, and the analysis requirements. It is usually set to the last year, three years, or longer. In this embodiment, it is set to records within the last five years. Ensure the timeliness and relevance of the data while reducing the impact of outdated data.
[0060] The pre-defined number of diagnoses refers to the number of diagnoses considered when analyzing each clinical record.
[0061] Depends on the complexity of clinical records and the diversity of diagnoses. Usually set to 1 to 3 main diagnoses. In this embodiment, it is set to consider the first two main diagnoses for each record. Focusing on the main diagnoses improves the relevance of the data and the accuracy of the analysis.
[0062] The preset standard similarity is a threshold for judging the similarity between records. It depends on the diversity and redundancy of the data. The cosine similarity is usually set between 0.7 and 0.9. In this embodiment, it is set to 0.85. It effectively identifies and deletes highly similar records, reduces data redundancy, and improves data quality.
[0063] The preset standard deletion number is the preset number of records to be deleted when deduplicating data. It depends on the size of the data pool and the deduplication goal. It is usually set to 5% to 10% of the total number of records in the data pool. In this embodiment, it is set to 8% of the total number of records in the data pool. Control the number of deletions to avoid excessive deduplication and maintain data diversity.
[0064] The preset number of responses is the number of times the system responds during data extraction and processing. It depends on the processing capacity of the system and the needs of the user. It is usually set to several to dozens of times per hour according to the system performance. In this embodiment, it is set to respond 10 times per hour to ensure the response speed and processing efficiency of the system to meet user needs.
[0065] The standard response time is the time limit for the system response, which depends on the system performance and the user's requirements for real-time performance. It is usually set to a few seconds to a few minutes. In this embodiment, it is set to no more than 30 seconds per response to ensure the system's response speed and improve the user experience.
[0066] In the process of building a data pool suitable for TCM syndrome differentiation modeling based on artificial intelligence, the target clinical records are first extracted from the TCM clinical record database in real time at a preset extraction speed, and the extracted records are standardized in terms and formats using natural language processing technology and a preset terminology library to ensure the consistency of the data format. Subsequently, these standardized clinical records are integrated to form a structured data pool, providing a rich and accurate data foundation for TCM syndrome differentiation modeling. In addition, by analyzing the record time and combining it with the preset time range, natural language processing technology is used to extract keywords and calculate cosine similarity to identify and delete or adjust records with high similarity and optimize the quality of the data pool. At the same time, the extraction speed is dynamically adjusted according to the number of deleted records and the response time to ensure that the construction of the data pool is both efficient and meets real-time requirements.
[0067] Through real-time extraction and standardized processing, the timeliness and consistency of the data are ensured. The natural language processing technology is used to accurately replace the terms, which enhances the readability and analyzability of the data, enabling the artificial intelligence system to more accurately understand and process TCM clinical records. By calculating the cosine similarity and deleting the records or adjusting the number of diagnoses according to the standard similarity, data redundancy is effectively reduced. In addition, the mechanism of dynamically adjusting the extraction speed and response time ensures that the data pool can adapt to different processing needs and real-time requirements, and improves the flexibility and responsiveness of the system.
[0068] Please continue reading Figure 2 As shown, it is a decision logic diagram for extracting keyword decisions in this embodiment;
[0069] Specifically, when the recording time of the standardized clinical record is within a preset time range, it is determined that keywords need to be extracted.
[0070] Clinical records are extracted from the TCM clinical record database in real time at a preset speed, and the records are standardized using natural language processing technology combined with a preset terminology library to ensure the uniformity of the data format. Then, under the condition that the record time meets the preset time range, keywords are extracted and cosine similarity is calculated, so that records with high similarity can be deleted or adjusted, and finally all standardized clinical records are integrated to form a data pool.
[0071] By limiting the time range and similarity threshold, data redundancy is effectively reduced and the relevance and novelty of data are enhanced. The mechanism of dynamically adjusting the extraction speed and response time further improves the real-time and adaptability of the data pool.
[0072] Specifically, calculating the cosine similarity of all extraction results includes: converting all keywords into numerical vectors through a bag-of-words model, storing all numerical vectors in a matrix, and calculating the cosine similarity between every two numerical vectors in the matrix.
[0073] Based on the determination that keywords need to be extracted, the method uses the bag-of-words model to convert the extracted keywords into numerical vectors and store these vectors in a matrix. Subsequently, the cosine similarity between any two numerical vectors in the matrix is calculated to quantify the similarity between the texts.
[0074] The efficiency and accuracy of data processing are improved through numerical processing. The numerical vectorization of the bag-of-words model allows text data to be effectively processed by machine learning algorithms, while the calculation of cosine similarity provides a quantitative means for identifying and processing similar texts. It helps to identify and delete duplicate or highly similar records in the data pool, thereby optimizing data quality and reducing data redundancy.
[0075] Please continue reading Figure 3 As shown, it is a decision logic diagram for the decision of deleting standardized clinical records in this embodiment;
[0076] Specifically, when the cosine similarity is greater than a preset standard similarity, and the difference between the cosine similarity and the standard similarity is greater than a preset standard similarity difference, the previous standardized clinical record is deleted.
[0077] During the data pool construction process, when the calculated cosine similarity not only exceeds the preset standard similarity, but also the difference with the standard similarity is greater than the preset standard similarity difference, the system will automatically identify and delete the previously extracted standardized clinical records.
[0078] By deleting highly similar records, we not only reduce data redundancy, but also avoid overfitting problems that may occur during model training. It also helps improve the generalization ability of the model because the diversity in the data set is enhanced.
[0079] Specifically, when the cosine similarity is greater than the preset standard similarity and the difference between the cosine similarity and the standard similarity is less than or equal to the preset standard similarity difference, the number of diagnosis times is reduced and adjusted according to the difference between the cosine similarity and the standard similarity, the standard similarity difference and the preset diagnosis time coefficient.
[0080] In the data pool construction, when the calculated cosine similarity exceeds the preset standard similarity and the difference with the standard similarity is less than or equal to the preset standard similarity difference, the system will adjust the records in the data set by reducing the number of diagnoses accordingly based on the specific difference between the cosine similarity and the standard similarity, the standard similarity difference and the preset diagnosis number coefficient.
[0081] By reducing the number of diagnoses, it is possible to avoid duplication of information due to the high similarity between records, while retaining key diagnostic information, enhancing the practicality of the data and the depth of clinical records, and helping to improve the overall quality of the data pool.
[0082] Specifically, when the number of deletions is greater than the maximum value of the standard deletion number range, the extraction speed is adjusted to be reduced according to the number of deletions, the maximum value of the standard deletion number range, and a preset extraction speed coefficient.
[0083] In the dynamic management of the data pool, the system monitors the number of deleted standardized clinical records. When the number of deleted records exceeds the preset maximum standard deletion number, the system will reduce the extraction speed based on the excess number, the maximum standard deletion number and the extraction speed factor to reduce excessive deletion of data.
[0084] By reducing the speed when too many deletions are made, rapid loss of data and potential loss of information is avoided.
[0085] Specifically, when the number of deletions is less than the minimum value of the standard deletion number range, the extraction speed is adjusted according to the number of deletions, the minimum value of the standard deletion number range, and a preset extraction speed coefficient.
[0086] On the contrary, if the number of deletions is lower than the minimum standard deletion number, the system will increase the extraction speed to improve the efficiency of data collection and ensure that the data pool can continuously obtain an appropriate amount of new data.
[0087] Increasing the speed when deletion is insufficient ensures timely replenishment and diversity of data, and guarantees the stability and continuous updating of the data pool.
[0088] The preset extraction speed factor is a parameter used to adjust the speed of extracting clinical records from the database, which depends on a variety of factors, including the size of the database, the complexity of the data, the processing power of the system, the storage capacity, the network bandwidth, and the urgency of the data pool construction. It is usually determined based on system testing and demand analysis, and is a value between 0.5 and 2, where 1 represents the original or unadjusted speed. In this embodiment, it is assumed that the extraction speed factor is set to 1.5, which means that the system will extract data at a faster rate than the original speed. The speed of data collection is accelerated to cope with situations where a large amount of data is urgently needed for model training.
[0089] Specifically, the ratio of the sum of the response times to the response times is calculated to obtain the average response time. When the average response time is greater than the standard response time, the adjusted extraction speed is reduced according to the average response time, the standard response time and a preset correction coefficient.
[0090] The correction factor is a value used to fine-tune the extraction speed, which is based on the real-time performance of the system and the preset performance standards. It depends on the system performance and data complexity, and is usually between 0.5 and 2, where 1 means no adjustment. In this embodiment, it is assumed that the correction factor is set to 0.8. This means that when it is detected that the average response time exceeds the standard, the extraction speed will be reduced by 20%. When the system response time exceeds expectations, the processing time of each record can be increased by reducing the extraction speed, thereby improving the accuracy and processing quality of the data.
[0091] During the data pool construction process, the system will calculate the total response time, that is, the total time from extracting clinical records to completing standardized processing, and then calculate the ratio of this total time to the number of responses to obtain the average response time. If the average response time exceeds the preset standard response time, the system will reduce the extraction speed accordingly based on the difference between the average response time and the standard response time and a preset correction factor to ensure the efficiency of the data processing process.
[0092] By monitoring and adjusting the average response time, we ensure that the response speed of the data pool construction process is always maintained at a user-satisfactory level. When the actual response time exceeds the expected level, we can reduce the extraction speed to provide the system with more time to process each record.
[0093] Specifically, when the average response time is less than or equal to the standard response time, the corrected and adjusted extraction speed is reduced according to the average response time, the standard response time and a preset correction coefficient.
[0094] During the data pool construction and management process, the system will continuously track and calculate the average response time, which is the average time from data extraction to processing completion. If the average response time is less than or equal to the preset standard response time, the system will adjust the extraction speed accordingly based on the specific difference between the average response time and the standard response time, as well as a preset correction factor, to optimize the overall data flow and processing efficiency.
[0095] By appropriately reducing the extraction speed when the average response time meets or is lower than the standard, resource waste due to excess processing capacity can be avoided while maintaining the smoothness of data processing and the response speed of the system.
[0096] Please continue reading Figure 4 As shown, it is a schematic diagram of a data pool construction system suitable for artificial intelligence TCM syndrome differentiation modeling in this embodiment;
[0097] A data pool construction system suitable for artificial intelligence TCM syndrome differentiation modeling is constructed based on the above-mentioned data pool construction method suitable for artificial intelligence TCM syndrome differentiation modeling, comprising:
[0098] An extraction module is used to extract clinical records of several targets in real time from a preset TCM clinical record database at a preset extraction speed, and pre-process the clinical records to obtain several standardized clinical records;
[0099] a processing module connected to the extraction module, for obtaining the recording time of each standardized clinical record, making a judgment based on all the recording times and a preset time range, extracting keywords from the diagnosis results of the preset diagnosis times in the standardized clinical record according to the judgment result, calculating the cosine similarity of all the extracted keywords, making a judgment based on the cosine similarity and the preset standard similarity, deleting the corresponding standardized clinical record according to the judgment result, or correcting the diagnosis times, extracting keywords from the diagnosis results of the corrected diagnosis times in the standardized clinical record until the corresponding standardized clinical record is deleted according to the comparison result;
[0100] An adjustment module, connected to the processing module, for obtaining the deletion number of the standardized clinical records corresponding to the deletion, and adjusting the extraction speed according to the deletion number and a preset standard deletion number;
[0101] A correction module, connected to the processing module and the adjustment module respectively, for obtaining the time duration from extracting a plurality of target clinical records at a time to obtaining a plurality of standardized clinical records, and correcting the adjusted extraction speed according to a preset number of responses, time duration and a preset standard response time duration to obtain a corrected extraction speed;
[0102] The integration output module is connected to the processing module and the correction module respectively, and is used to integrate all standardized clinical records extracted and pre-processed from the TCM clinical record database in real time at the corrected extraction speed, obtain a data pool suitable for artificial intelligence TCM syndrome differentiation modeling and output it.
[0103] This system realizes the construction of the data pool for TCM syndrome differentiation modeling through a series of modular components. The extraction module is responsible for real-time data extraction from the TCM clinical record database; the processing module performs natural language processing on the extracted data, and uses the preset terminology library for term replacement and data format standardization; the integration output module integrates and outputs standardized clinical records as a data pool; the adjustment module is responsible for determining the record time, extracting keywords, calculating cosine similarity, and deleting records or adjusting the number of diagnoses based on the similarity results; at the same time, the adjustment module also adjusts the extraction speed based on the number of deletions; the correction module dynamically adjusts the extraction speed based on the response time and preset standards to ensure the efficiency and response speed of data processing.
[0104] Through automated and intelligent processes, the efficiency of data pool construction and the consistency of data quality are ensured. The modular design improves the flexibility and maintainability of the system, allowing each link to be independently adjusted and optimized according to actual conditions. By dynamically adjusting the extraction speed and the number of diagnoses, the system can adaptively respond to changes in data similarity and processing time, reducing data redundancy.
[0105] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.
[0106] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A method for constructing a data pool suitable for artificial intelligence TCM syndrome differentiation modeling, characterized in that: include: Extracting clinical records of several targets in real time from a preset TCM clinical record database at a preset extraction speed, and preprocessing the clinical records to obtain several standardized clinical records; Obtaining the recording time of each standardized clinical record, making a determination based on all the recording times and a preset time range, extracting keywords from the diagnosis results of a preset number of diagnoses in the standardized clinical record based on the determination result, calculating the cosine similarity of all the extracted keywords, performing a comparison based on the cosine similarity and a preset standard similarity, deleting the corresponding standardized clinical record based on the comparison result, or correcting the number of diagnoses, extracting keywords from the diagnosis results of the corrected number of diagnoses in the standardized clinical record until deleting the corresponding standardized clinical record based on the comparison result; Obtain the number of deletions of the standardized clinical records corresponding to the deletions, and adjust the extraction speed according to the number of deletions and the preset standard number of deletions; Obtaining the time from extracting clinical records of a number of targets at a time to obtaining a number of standardized clinical records, and correcting the adjusted extraction speed according to a preset number of responses, duration, and a preset standard response duration to obtain a corrected extraction speed; Integrate all standardized clinical records extracted and preprocessed from the TCM clinical record database in real time at the modified extraction speed to obtain and output a data pool suitable for artificial intelligence TCM syndrome differentiation modeling; The pre-defined number of diagnoses refers to the number of diagnoses considered when analyzing each clinical record.
2. The method for constructing a data pool suitable for artificial intelligence TCM syndrome differentiation modeling according to claim 1, characterized in that: When the recording time of the standardized clinical record is within a preset time range, it is determined that keywords need to be extracted.
3. The method for constructing a data pool suitable for artificial intelligence TCM syndrome differentiation modeling according to claim 2, characterized in that: Calculating the cosine similarity of all extraction results includes: All keywords are converted into numerical vectors through the bag-of-words model, all numerical vectors are stored in a matrix, and the cosine similarity between every two numerical vectors in the matrix is calculated.
4. The method for constructing a data pool suitable for artificial intelligence TCM syndrome differentiation modeling according to claim 3, characterized in that: When the cosine similarity is greater than the preset standard similarity, and the difference between the cosine similarity and the standard similarity is greater than the preset standard similarity difference, the previous standardized clinical record is deleted.
5. The method for constructing a data pool suitable for artificial intelligence TCM syndrome differentiation modeling according to claim 4, characterized in that: When the cosine similarity is greater than the preset standard similarity and the difference between the cosine similarity and the standard similarity is less than or equal to the preset standard similarity difference, the diagnosis times are reduced and adjusted according to the difference between the cosine similarity and the standard similarity, the standard similarity difference and the preset diagnosis times coefficient.
6. The method for constructing a data pool suitable for artificial intelligence TCM syndrome differentiation modeling according to claim 5, characterized in that: When the deletion quantity is greater than the maximum value of the standard deletion quantity range, the extraction speed is adjusted according to the deletion quantity, the maximum value of the standard deletion quantity range and the preset extraction speed coefficient.
7. The method for constructing a data pool suitable for artificial intelligence TCM syndrome differentiation modeling according to claim 6, characterized in that: When the deletion quantity is less than the minimum value of the standard deletion quantity range, the extraction speed is adjusted according to the deletion quantity, the minimum value of the standard deletion quantity range and the preset extraction speed coefficient.
8. The method for constructing a data pool suitable for artificial intelligence TCM syndrome differentiation modeling according to claim 7, characterized in that: Calculate the sum of the response times, calculate the ratio of the sum of the response times to the response times, and get the average response time. When the average response time is greater than the standard response time, reduce the corrected extraction speed based on the average response time, the standard response time and a preset correction coefficient.
9. The method for constructing a data pool suitable for artificial intelligence TCM syndrome differentiation modeling according to claim 8, characterized in that: When the average response time is less than or equal to the standard response time, the corrected and adjusted extraction speed is reduced according to the average response time, the standard response time and a preset correction coefficient.
10. A data pool construction system suitable for artificial intelligence TCM syndrome differentiation modeling, constructed based on the data pool construction method suitable for artificial intelligence TCM syndrome differentiation modeling according to any one of claims 1 to 9, characterized in that: include: An extraction module is used to extract clinical records of several targets in real time from a preset TCM clinical record database at a preset extraction speed, and pre-process the clinical records to obtain several standardized clinical records; a processing module connected to the extraction module, for obtaining the recording time of each standardized clinical record, making a determination based on all the recording times and a preset time range, extracting keywords from the diagnosis results of the preset diagnosis times in the standardized clinical record according to the determination result, calculating the cosine similarity of all the extracted keywords, comparing the cosine similarity with the preset standard similarity, deleting the corresponding standardized clinical record according to the comparison result, or correcting the diagnosis times, extracting keywords from the diagnosis results of the corrected diagnosis times in the standardized clinical record until the corresponding standardized clinical record is deleted according to the comparison result; An adjustment module, connected to the processing module, for obtaining the deletion number of the standardized clinical records corresponding to the deletion, and adjusting the extraction speed according to the deletion number and a preset standard deletion number; A correction module, connected to the processing module and the adjustment module respectively, for obtaining the time duration from extracting a plurality of target clinical records at a time to obtaining a plurality of standardized clinical records, and correcting the adjusted extraction speed according to a preset number of responses, time duration and a preset standard response time duration to obtain a corrected extraction speed; An integration output module, connected to the processing module and the correction module respectively, for integrating all standardized clinical records extracted and preprocessed from the TCM clinical record database in real time at the corrected extraction speed, obtaining a data pool suitable for artificial intelligence TCM syndrome differentiation modeling and outputting the data; The pre-defined number of diagnoses refers to the number of diagnoses considered when analyzing each clinical record.
Citation Information
Patent Citations
Data management system for constructing traditional Chinese medicine ancient book knowledge graph
CN117591678A
Visual intelligent system and equipment for realizing digitization and physical archive management
CN116934285A
Breast cancer clinical data analysis diagnosis and treatment platform based on artificial intelligence large language model
CN118016280A
Data processing method and system based on cloud computing
CN118152124A
Extensible data deduplication system and method
US20110231362A1
