A medical data storage, processing and analysis intelligent platform based on Python language

Abstract

The present application relates to the technical field of medical information processing, and particularly relates to a medical data storage, processing and analysis intelligent platform based on Python language, which comprises a data acquisition module, a determination module, a word segmentation module, a labeling module, an adjustment module, a conversion module, a dimension reduction module, a correction module, a generation module, a storage module and an analysis module. The present application significantly improves the processing efficiency and accuracy of medical document data through an automatic text processing and analysis process, and the dynamic adjustment of the segmentation granularity and the dimension reduction process can flexibly cope with the complexity of different texts, ensuring the accuracy of data processing. Combined with the analysis capability of the mixed Gaussian model, the platform can extract effective information from massive medical literature, providing strong support for subsequent medical research and decision-making, and effectively solving the problems of low medical audit efficiency and inability to quickly respond to real-time medical data changes caused by reliance on manual intervention and lagging rule updates.

Description

Technical Field

[0001] This invention relates to the field of medical information processing technology, and in particular to an intelligent platform for medical data storage, processing and analysis based on the Python language. Background Technology

[0002] With the continuous development of medical informatization, the amount of data generated in the medical field is exploding, especially the accumulation of textual information such as medical literature, clinical records, and medical records. Traditional manual analysis and processing methods are no longer sufficient to cope with the rapid growth and complexity of massive amounts of data, and cannot effectively extract the potential medical knowledge and diagnostic information within them. At the same time, the increasing precision of medical research and clinical decision-making places higher demands on data analysis. How to extract valuable information from a large amount of medical texts and conduct effective data storage, processing, and analysis has become an important issue in the process of medical informatization.

[0003] Chinese patent application publication number CN108648810A discloses a data processing method and apparatus for medical review. The method includes the following steps: S01, responding to a modification operation triggered by a modification operation entry point, a review logic configuration interface is invoked and displayed, wherein the modification operation entry point for modifying medical review rules is preset; S02, when the review logic configuration interface receives a first modification request instruction, a first modification window for editing and modifying preset original medical review rules is provided; when the review logic configuration interface receives a second modification request instruction, a second modification window for creating new medical review rules is provided; S03, the edited medical review rule is received in the first modification window, or the newly created medical review rule is received in the second modification window, and the received medical review rule is used as the current new medical review rule.

[0004] Therefore, this method emphasizes the manual editing and modification of the review rules interface, relies on manual input and editing, is easily affected by human factors, has low operational efficiency, and is difficult to automate the processing of large-scale data. In practical applications, the modification and adjustment of medical review rules need to be made by combining a large amount of medical literature, medical record data and real-time data. Traditional manual modification methods cannot effectively handle such large-scale and complex data. Summary of the Invention

[0005] To address this, the present invention provides an intelligent platform for medical data storage, processing, and analysis based on the Python language. This platform overcomes the problems of low efficiency in medical review and inability to quickly respond to real-time changes in medical data in existing technologies due to reliance on manual intervention and delayed rule updates. This is achieved through automated data processing, intelligent analysis, and dynamic rule adjustment.

[0006] To achieve the above objectives, this invention provides an intelligent platform for medical data storage, processing, and analysis based on the Python language, comprising:

[0007] The data acquisition module is used to collect text data in real time during the medical document input process;

[0008] A determination module, which is connected to the data acquisition module, is used to determine several complex texts based on the text data;

[0009] The word segmentation module is connected to the determination module and is used to segment each complex text with a preset word segmentation tool and a preset segmentation granularity to form several phrases.

[0010] The annotation module, which is connected to the word segmentation module, is used to annotate the part of speech of each phrase according to a preset annotation tool, thereby forming several annotated phrases;

[0011] An adjustment module, which is connected to the annotation module, is used to adjust the preset segmentation granularity according to the grammatical consistency of each annotated phrase at any adjacent time within a preset adjustment period.

[0012] A conversion module, which is connected to the annotation module, is used to convert each of the annotation phrases according to a preset word vector model to form several word vectors;

[0013] A dimensionality reduction module, which is connected to the transformation module, is used to reduce the dimensionality of each word vector by a preset number of dimensionality reductions based on the dimensionality of all word vectors within a preset determination time period, thereby forming several dimensionality-reduced vectors.

[0014] A correction module, which is connected to the dimensionality reduction module, is used to correct a preset number of dimensionality reductions based on the semantic similarity and preset standard similarity of any adjacent dimensionality reduction vectors.

[0015] A generation module, which is connected to the transformation module and the dimensionality reduction module respectively, is used to generate several sentence vectors based on all the word vectors and the dimensionality reduction vectors;

[0016] A storage module, connected to the generation module, is used to associate and store the sentence vector and the labeled phrase in a preset database;

[0017] An analysis module, connected to the storage module, is used to analyze the sentence vectors and the labeled phrases according to a preset Gaussian mixture model, and generate an analysis report.

[0018] Furthermore, the determining module includes:

[0019] A vocabulary recognition unit is used to recognize different words in the text data to form a number of different words, and to recognize the total vocabulary in the text data to form a number of total words;

[0020] A complexity calculation unit, connected to the word recognition unit, is used to calculate the ratio of the number of different words to the total number of words to form the text complexity.

[0021] A distribution calculation unit, which is connected to the complexity calculation unit, is used to calculate the standard deviation of the text complexity within a preset historical time period to form a complexity distribution degree;

[0022] A distribution degree comparison unit, which is connected to the distribution calculation unit, is used to compare the complex distribution degree with the preset standard distribution degree to form a distribution comparison result;

[0023] A determination unit, connected to the distribution degree comparison unit, is used to determine a number of complex texts based on the distribution comparison result, the text complexity, and the comparison result of the preset standard complexity.

[0024] Furthermore, the word segmentation module includes:

[0025] A stop word recognition unit is used to identify stop words in the complex text and form a number of stop words;

[0026] A segmentation determination unit, which is connected to the stop word recognition unit, is used to determine whether to perform segmentation based on a preset stop word list and each stop word, thereby forming a segmentation determination result;

[0027] The word segmentation unit is connected to the segmentation determination unit and is used to segment each complex text according to the segmentation determination result, the preset segmentation granularity and the preset word segmentation tool to form several phrases.

[0028] Furthermore, the adjustment module includes:

[0029] A sorting unit is used to sort the labeled phrases according to their parts of speech to form a part-of-speech sequence;

[0030] A consistency calculation unit, which is connected to the sorting unit, is used to calculate the cosine similarity of the part-of-speech sequence at any adjacent time to form a grammatical consistency score.

[0031] An adjustment unit, connected to the consistency calculation unit, is used to adjust the preset segmentation granularity according to the syntax consistency.

[0032] Furthermore, the adjustment unit includes:

[0033] The consistency comparison subunit is used to compare the syntactic consistency with the preset standard consistency to form a consistency comparison result;

[0034] An adjustment subunit, which is connected to the consistency comparison subunit, is used to adjust the preset segmentation granularity based on the consistency comparison result, the preset standard consistency, and the preset adjustment coefficient.

[0035] Furthermore, the dimensionality reduction module includes:

[0036] A complexity calculation unit is used to calculate the variance of the dimension to form the complexity.

[0037] A complex fluctuation calculation unit, which is connected to the complexity calculation unit, is used to calculate the standard deviation of the complexity to form a complex fluctuation value;

[0038] A dimensionality reduction unit, connected to the complex fluctuation calculation unit, is used to reduce the dimensionality of each word vector according to the complex fluctuation value, forming several dimensionality reduction vectors.

[0039] Furthermore, the dimensionality reduction unit includes:

[0040] The dimension reduction comparison subunit is connected to the complex fluctuation calculation unit to compare the complex fluctuation value with the preset standard complex fluctuation value to form a dimension reduction comparison result.

[0041] A dimensionality reduction operation subunit, which is connected to the dimensionality reduction comparison subunit, is used to reduce the dimensionality of each word vector according to the dimensionality reduction comparison result, a preset dimensionality reduction model, and a preset dimensionality reduction quantity, to form several dimensionality reduction vectors.

[0042] Furthermore, the correction module includes:

[0043] A similarity calculation unit is used to calculate the cosine similarity of the reduced-dimensional vectors to form semantic similarity;

[0044] A correction unit, connected to the similarity calculation unit, is used to correct the preset dimensionality reduction number based on the relative deviation between the semantic similarity and the preset standard similarity, and a preset correction coefficient.

[0045] Furthermore, the generation module includes:

[0046] A combination unit is used to combine all the word vectors and each of the dimensionality-reduced vectors by weighted averaging to form several temporary sentence vectors;

[0047] A generation unit, connected to the combination unit, is used to perform regularization processing on each of the temporary sentence vectors according to a preset post-processing algorithm to form several sentence vectors.

[0048] Furthermore, the analysis module includes:

[0049] The analysis unit is used to cluster the sentence vectors according to the preset Gaussian mixture model to form clustering results;

[0050] An extraction unit, connected to the analysis unit, is used to extract the part-of-speech distribution features of the labeled phrases to form feature extraction results;

[0051] An output unit, which is connected to the analysis unit and the extraction unit respectively, is used to output an analysis report based on the clustering results and the feature extraction results.

[0052] Compared with existing technologies, the beneficial effects of this invention are that it significantly improves the processing efficiency and accuracy of medical document data through automated text processing and analysis processes. The dynamic adjustment of segmentation granularity and dimensionality reduction process can flexibly cope with the complexity of different texts, ensuring the accuracy of data processing. Combined with the analytical capabilities of Gaussian mixture models, the platform can extract effective information from massive amounts of medical literature, providing strong support for subsequent medical research and decision-making. At the same time, the platform's database storage and report generation functions enhance the accessibility and visualization of data, which is helpful for data mining and knowledge discovery in the medical field. It effectively solves the problems of low efficiency in medical review and inability to quickly respond to real-time changes in medical data due to reliance on manual intervention and lagging rule updates.

[0053] Furthermore, by comprehensively considering the lexical composition and complexity distribution of the text, it is possible to ensure that when identifying complex text, the lexical diversity and structural complexity of the text can be accurately captured, rather than relying solely on a single indicator.

[0054] Furthermore, through multi-level processing of the word segmentation module, interference from stop words can be effectively avoided, making subsequent text analysis more accurate. Stop word recognition and reasonable segmentation judgment ensure that meaningless components in the text do not participate in subsequent analysis, improving the quality and efficiency of word segmentation. Preset segmentation granularity and word segmentation tools further optimize the segmentation effect, ensuring that the generated phrases have high semantic coherence.

[0055] Furthermore, dynamically adjusting the segmentation granularity based on grammatical consistency helps optimize the accuracy and effectiveness of word segmentation. The complexity of grammatical structures may vary across different texts. Adjusting the segmentation granularity based on the part-of-speech consistency between adjacent time steps can flexibly address changes in grammatical structure within the text, ensuring that semantic information is not lost during segmentation. This effectively avoids over-segmentation or under-segmentation issues caused by fixed granularity, thereby improving the overall accuracy and effectiveness of text processing.

[0056] Furthermore, by combining grammatical consistency with preset standard consistency, the adjustment unit can precisely control the adjustment of word segmentation granularity, making it more consistent with the actual grammatical structure of the text. This adjustment mechanism not only improves the grammatical adaptability of word segmentation and avoids over-segmentation or under-segmentation, but also flexibly adjusts according to the complexity of the actual text, improving the accuracy and robustness of text processing.

[0057] Furthermore, the diversity and stability of word vectors are measured by computational complexity and fluctuation values, thereby enabling targeted dimensionality reduction. Dimensionality reduction reduces the impact of redundant information while preserving the semantic features of word vectors, improving the efficiency and accuracy of data processing.

[0058] Furthermore, through precise coordination of dimensionality reduction comparison and dimensionality reduction operation, the dimensionality reduction process can be optimized according to the complexity and volatility of word vectors, avoiding excessive loss of information. By comparing with preset standards, the system can automatically adjust the dimensionality reduction strategy to achieve more efficient semantic representation and better computational performance.

[0059] Furthermore, by dynamically adjusting the number of dimensionality reduction values ​​based on the deviation between semantic similarity and a preset standard similarity, the dimensionality reduction operation is ensured to retain semantic information while further reducing the loss of redundant data. Adjusting the number of dimensionality reduction values ​​can improve the quality of word vectors, enhance the accuracy and consistency of semantic representation, and thus improve the effectiveness of subsequent analysis and model training.

[0060] Furthermore, by combining weighted average with word vectors and dimensionality-reduced vectors, sentence vectors can comprehensively reflect the semantic information in the text. Regularization improves the stability and consistency of the generated sentence vectors, ensuring the normalization of the vectors and avoiding the adverse effects of excessively large or small values ​​on subsequent analysis, thus helping to improve the accuracy and robustness of the model.

[0061] Furthermore, sentence vector clustering using a Gaussian mixture model effectively groups similar sentences into the same category, helping to reveal the underlying structures and patterns in the text. Combined with the extraction of part-of-speech distribution features, this further enhances the ability to analyze the deep grammatical structure of the text. The final analysis report provides accurate data support and decision-making basis for subsequent text processing, classification, or applications. Attached Figure Description

[0062] Figure 1 This is a schematic diagram of the intelligent platform for medical data storage, processing, and analysis based on the Python language in this embodiment;

[0063] Figure 2 This embodiment defines the determination logic diagram for determining complex text by the determining unit;

[0064] Figure 3 This embodiment provides a logic diagram for determining the preset segmentation granularity of the adjusted subunit.

[0065] Figure 4 This is a logic diagram for determining the dimensionality reduction operation subunit in this embodiment. Detailed Implementation

[0066] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0067] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0068] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0069] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0070] Please see Figure 1 As shown, it is a schematic diagram of the intelligent platform for medical data storage, processing and analysis based on the Python language in this embodiment;

[0071] This embodiment provides an intelligent platform for medical data storage, processing, and analysis based on the Python language, including:

[0072] The data acquisition module is used to collect text data in real time during the medical document input process;

[0073] A determination module, which is connected to the data acquisition module, is used to determine several complex texts based on the text data;

[0074] The word segmentation module is connected to the determination module and is used to segment each complex text with a preset word segmentation tool and a preset segmentation granularity to form several phrases.

[0075] The annotation module, which is connected to the word segmentation module, is used to annotate the part of speech of each phrase according to a preset annotation tool, thereby forming several annotated phrases;

[0076] An adjustment module, which is connected to the annotation module, is used to adjust the preset segmentation granularity according to the grammatical consistency of each annotated phrase at any adjacent time within a preset adjustment period.

[0077] A conversion module, which is connected to the annotation module, is used to convert each of the annotation phrases according to a preset word vector model to form several word vectors;

[0078] A dimensionality reduction module, which is connected to the transformation module, is used to reduce the dimensionality of each word vector by a preset number of dimensionality reductions based on the dimensionality of all word vectors within a preset determination time period, thereby forming several dimensionality-reduced vectors.

[0079] A correction module, which is connected to the dimensionality reduction module, is used to correct a preset number of dimensionality reductions based on the semantic similarity and preset standard similarity of any adjacent dimensionality reduction vectors.

[0080] A generation module, which is connected to the transformation module and the dimensionality reduction module respectively, is used to generate several sentence vectors based on all the word vectors and the dimensionality reduction vectors;

[0081] A storage module, connected to the generation module, is used to associate and store the sentence vector and the labeled phrase in a preset database;

[0082] An analysis module, connected to the storage module, is used to analyze the sentence vectors and the labeled phrases according to a preset Gaussian mixture model, and generate an analysis report.

[0083] The data acquisition module is based on natural language processing (NLP) technology. It extracts relevant data in real time by parsing structured and unstructured text in medical documents to support subsequent analysis.

[0084] Word vectors are a technique that maps words to low-dimensional, dense vectors in a vector space to represent semantic information between words.

[0085] Preset word segmentation tools are algorithms used to segment complex text into words or phrases. Depending on the language characteristics of the text and the application scenario, they generally use standard word segmentation libraries, depending on the complexity of the corpus. In this embodiment, the Jieba word segmentation tool is used, which has high accuracy and wide applicability, and is suitable for Chinese medical text processing.

[0086] The preset segmentation granularity refers to the degree to which text is cut into multiple small units (such as phrases) during word segmentation. It depends on the structure of the text and the needs of analysis, and is usually set between 2 and 5 words. In this embodiment, it is set to 2 to 4 words to ensure that the segmented phrases can represent medical terms or concepts with practical significance, and to avoid excessive splitting into meaningless word units.

[0087] Preset annotation tools are tools or models used to annotate words or phrases in text with part-of-speech or grammatical tags. Depending on the language used and the task requirements, general annotation tools such as spaCy or NLTK in Python are usually used and set to include the main part-of-speech categories. In this embodiment, spaCy is used for part-of-speech tagging, which can quickly process complex grammatical structures in medical texts.

[0088] Syntactic consistency refers to the consistency of syntactic structure between phrases within adjacent time steps. It depends on the syntactic structure of the text and the accuracy of the annotation tools. It is generally evaluated by calculating the syntax tree structure of adjacent phrases and is usually set between 0 and 1. In this embodiment, it is set to 0.85, which can ensure the stability of word segmentation and annotation and help improve the accuracy of text analysis.

[0089] The preset word vector model is used to transform each word in the text into a low-dimensional dense vector. Depending on the corpus used and the task requirements, the Word2Vec model or the GloVe model is generally used, with word vectors ranging from 300 to 768 dimensions. This embodiment uses the GloVe word vector model, set to 300 dimensions, which can balance computational efficiency and representational power while ensuring model performance, making it suitable for handling the ambiguity of medical texts.

[0090] The preset dimensionality reduction number refers to the number of dimensions reduced during the dimensionality reduction operation. It is used to reduce computational complexity and redundant information. It depends on the dimension of the word vector and the requirements of the target task. It is usually set between 30 and 100 dimensions. In this embodiment, it is set to 50 dimensions, which can retain sufficient semantic information, reduce computational burden, and improve the efficiency of subsequent model processing.

[0091] Semantic similarity measures the semantic similarity between two word vectors, and depends on the quality of the word vectors and the characteristics of the target task.

[0092] The preset judgment duration refers to the time range or data length considered when performing word vector dimensionality reduction or other analyses. It determines the time frame for data processing and judgment, and depends on the frequency of data changes, processing requirements, and the real-time requirements of the algorithm. Generally, the preset judgment duration should be set according to the specific application scenario, typically ranging from a few seconds to a few minutes. In this embodiment, it is set to 30 seconds, which balances the real-time nature of data processing with the computational burden of the system, avoiding excessive data noise due to an overly short time window, while also ensuring that the system can respond sensitively to rapidly changing data streams.

[0093] The preset standard similarity refers to the standard value used to judge the similarity of word vectors after dimensionality reduction. It depends on the needs of the application scenario, especially the semantic consistency requirements, and is usually set between 0.85 and 0.95. In this embodiment, it is set to 0.9, which can ensure that the word vectors maintain a high semantic similarity after dimensionality reduction, which is helpful for subsequent analysis and modeling.

[0094] Preset Gaussian mixture models (GMMs) are tools used for probabilistic distribution modeling, which can help analyze multimodal data and estimate the distribution characteristics of the data, depending on the type and complexity of the data distribution.

[0095] First, the data acquisition module collects input medical text data in real time. Next, the identification module identifies complex text based on the text data and segments it into phrases using the word segmentation module. The annotation module tags the phrases with part-of-speech tags, and the adjustment module further adjusts the segmentation granularity based on grammatical consistency. Subsequently, the transformation module converts the annotated phrases into word vectors, the dimensionality reduction module reduces the dimensionality of the word vectors, and the correction module adjusts the number of dimensionality reductions based on semantic similarity. The generation module combines the word vectors and the dimensionality-reduced vectors to generate sentence vectors and stores these vectors and phrases in a database. Finally, the analysis module applies a Gaussian mixture model to analyze the stored data and generate a detailed analysis report; all of the above is implemented using the Python language.

[0096] The automated text processing and analysis workflow significantly improves the efficiency and accuracy of medical document data processing. The dynamic adjustment of segmentation granularity and dimensionality reduction process can flexibly cope with the complexity of different texts, ensuring the accuracy of data processing. Combined with the analytical capabilities of Gaussian mixture models, the platform can extract effective information from massive amounts of medical literature, providing strong support for subsequent medical research and decision-making. At the same time, the platform's database storage and report generation functions enhance the accessibility and visualization of data, which is helpful for data mining and knowledge discovery in the medical field. It effectively solves the problems of low efficiency in medical review and inability to quickly respond to real-time changes in medical data due to reliance on manual intervention and lagging rule updates.

[0097] Please continue reading. Figure 2 As shown, this is the determination logic diagram of the determination unit for determining complex text in this embodiment;

[0098] Specifically, the determining module includes:

[0099] A vocabulary recognition unit is used to recognize different words in the text data to form a number of different words, and to recognize the total vocabulary in the text data to form a number of total words;

[0100] A complexity calculation unit, connected to the word recognition unit, is used to calculate the ratio of the number of different words to the total number of words to form the text complexity.

[0101] A distribution calculation unit, which is connected to the complexity calculation unit, is used to calculate the standard deviation of the text complexity within a preset historical time period to form a complexity distribution degree;

[0102] A distribution degree comparison unit, which is connected to the distribution calculation unit, is used to compare the complex distribution degree with the preset standard distribution degree to form a distribution comparison result;

[0103] A determination unit, connected to the distribution degree comparison unit, is used to determine a number of complex texts when the distribution comparison result shows that the complexity degree is greater than the preset standard distribution degree and the text complexity is greater than the preset standard complexity.

[0104] The preset standard distribution degree refers to a standard value for the distribution of text complexity set based on historical data or experience. It depends on the average structural complexity of the text type and domain, and is usually set within the standard deviation range of complexity variation in text, between 0.3 and 0.5. In this embodiment, it is set to 0.4 to capture the fluctuation range of common structures and terminology distributions in medical documents, enabling the system to identify text that exceeds the normal complexity range.

[0105] The preset standard complexity is a value set by analyzing the average complexity of historical datasets, depending on the frequency and diversity of vocabulary usage in domain-specific texts. This value can be set based on the vocabulary distribution of the training data, typically between 0.2 and 0.6. In this embodiment, it is set to 0.5, which effectively distinguishes between common, concise text and complex, terminology-dense text based on the analysis results of vocabulary usage diversity in medical documents.

[0106] First, the vocabulary recognition unit analyzes the input text data to identify different words and all words in the text, thus forming the different words and the total vocabulary. Next, the complexity calculation unit calculates the ratio between the different words and the total vocabulary to obtain the text complexity. Then, the distribution calculation unit uses the standard deviation of text complexity in historical data to calculate the complexity distribution degree, and the distribution degree comparison unit compares the calculated distribution degree with a preset standard distribution degree to form a distribution comparison result. Finally, the determination unit judges based on the comparison result that when the text complexity is greater than the standard complexity and the complexity distribution degree exceeds the standard distribution degree, it is identified as complex text and these texts are determined to undergo further processing.

[0107] By comprehensively considering the lexical composition and complexity distribution of a text, it is possible to accurately capture the lexical diversity and structural complexity of the text when identifying complex texts, rather than relying solely on a single indicator.

[0108] Specifically, the word segmentation module includes:

[0109] A stop word recognition unit is used to identify stop words in the complex text and form a number of stop words;

[0110] A segmentation determination unit, which is connected to the stop word recognition unit, is used to determine whether to perform segmentation based on a preset stop word list and each stop word, thereby forming a segmentation determination result;

[0111] The word segmentation unit is connected to the segmentation determination unit and is used to segment each complex text according to the preset segmentation granularity and the preset word segmentation tool when the segmentation determination result is to perform segmentation, thereby forming several phrases.

[0112] A pre-defined stop word list is a collection of common words without actual semantic meaning, typically including pronouns, conjunctions, and particles. The size depends on the text type and language model, and is usually based on frequently occurring, meaningless words from a large corpus. The size of the stop word list generally varies depending on the text length and processing objectives, typically ranging from several hundred to several thousand words. In this embodiment, the stop word list is set to approximately 500 words to suit the characteristics of medical texts, ensuring that meaningless words are removed without missing key information. By appropriately setting the size of the stop word list, computational load can be effectively reduced, the efficiency of subsequent text analysis can be improved, and excessive segmentation errors can be avoided, ensuring accuracy in the text processing process.

[0113] First, the stop word recognition unit identifies stop words in the text. Based on a pre-defined stop word list, each stop word is evaluated to determine whether segmentation is necessary. If segmentation is required, the text is segmented; otherwise, the segmentation is skipped. Finally, the word segmentation unit performs the actual segmentation operation on the complex text based on the segmentation results, the pre-defined segmentation granularity, and the word segmentation tool, dividing it into several phrases for subsequent processing.

[0114] Through multi-level processing by the word segmentation module, interference from stop words can be effectively avoided, making subsequent text analysis more accurate. Stop word recognition and reasonable segmentation judgment ensure that meaningless components in the text do not participate in subsequent analysis, improving the quality and efficiency of word segmentation. Preset segmentation granularity and word segmentation tools further optimize the segmentation effect, ensuring that the generated phrases have high semantic coherence.

[0115] Specifically, the adjustment module includes:

[0116] A sorting unit is used to sort the labeled phrases according to their parts of speech to form a part-of-speech sequence;

[0117] A consistency calculation unit, which is connected to the sorting unit, is used to calculate the cosine similarity of the part-of-speech sequence at any adjacent time to form a grammatical consistency score.

[0118] An adjustment unit, connected to the consistency calculation unit, is used to adjust the preset segmentation granularity according to the syntax consistency.

[0119] First, the sorting unit sorts the labeled phrases according to their parts of speech, forming a part-of-speech sequence. Then, the consistency calculation unit calculates the cosine similarity between adjacent part-of-speech sequences to quantify the grammatical consistency between parts of speech. Finally, the adjustment unit dynamically adjusts the preset segmentation granularity based on the calculated grammatical consistency, ensuring that the word segmentation operation retains sufficient grammatical information while avoiding excessive or insufficient refinement.

[0120] Dynamically adjusting the segmentation granularity based on grammatical consistency helps optimize the accuracy and effectiveness of word segmentation. The complexity of grammatical structures may vary across different texts. Adjusting the segmentation granularity based on the consistency of part-of-speech tags at adjacent time points can flexibly address changes in grammatical structure within the text, ensuring that semantic information is not lost during segmentation. This effectively avoids over-segmentation or under-segmentation issues caused by fixed granularity, thereby improving the overall accuracy and effectiveness of text processing.

[0121] Please continue reading. Figure 3 As shown, it is the determination logic diagram of the adjustment subunit adjusting the preset segmentation granularity in this embodiment;

[0122] Specifically, the adjustment unit includes:

[0123] The consistency comparison subunit is used to compare the syntactic consistency with the preset standard consistency to form a consistency comparison result;

[0124] An adjustment subunit, connected to the consistency comparison subunit, is used to adjust the preset segmentation granularity based on the relative deviation between the syntax consistency and the preset standard consistency and a preset adjustment coefficient when the consistency comparison result is that the syntax consistency is less than the preset standard consistency. The adjusted preset segmentation granularity is positively correlated with the relative deviation between the syntax consistency and the preset standard consistency.

[0125] The preset standard consistency degree refers to a reference value set when adjusting the granularity of word segmentation. It is the minimum standard for measuring grammatical consistency and depends on the grammatical complexity of the text and the processing objectives. It is usually set between 0.7 and 0.9. In this embodiment, it is set to 0.85 to balance the fine granularity of text segmentation with grammatical consistency, so as to ensure the accuracy and readability of text processing. By reasonably setting the standard consistency degree, excessive splitting or merging can be avoided, thereby ensuring that the processed text still maintains its original grammatical structure.

[0126] The preset adjustment coefficient is a parameter used to adjust the granularity of word segmentation. It is dynamically adjusted based on the comparison results of grammatical consistency and depends on the specific application scenario and processing requirements. It is usually set between 0.1 and 0.5. In this embodiment, it is set to 0.2 to appropriately widen the range of granularity adjustment when the grammatical consistency is low. This allows the text segmentation to adapt to complex grammatical structures without being overly refined, thus affecting processing efficiency. By setting an appropriate adjustment coefficient, the flexibility of the system can be optimized, ensuring optimal word segmentation results in various text types.

[0127] The specific adjustment process is accomplished through two sub-units. First, the consistency comparison sub-unit compares the calculated syntactic consistency with a preset standard consistency, generating a consistency comparison result. Then, the adjustment sub-unit dynamically adjusts the segmentation granularity based on the consistency comparison result, the standard consistency, and preset adjustment coefficients to ensure the accuracy and flexibility of the segmentation operation. The preset standard consistency and adjustment coefficients provide the system with a basis for flexibly adjusting the granularity, allowing the segmentation granularity to be optimized according to the syntactic consistency of the actual text.

[0128] By combining grammatical consistency with preset standard consistency, the adjustment unit can precisely control the adjustment of word segmentation granularity, making it more consistent with the actual grammatical structure of the text. This adjustment mechanism not only improves the grammatical adaptability of word segmentation and avoids over-segmentation or under-segmentation, but also flexibly adjusts according to the complexity of the actual text, improving the accuracy and robustness of text processing.

[0129] Specifically, the dimensionality reduction module includes:

[0130] A complexity calculation unit is used to calculate the variance of the dimension to form the complexity.

[0131] A complex fluctuation calculation unit, which is connected to the complexity calculation unit, is used to calculate the standard deviation of the complexity to form a complex fluctuation value;

[0132] A dimensionality reduction unit, connected to the complex fluctuation calculation unit, is used to reduce the dimensionality of each word vector according to the complex fluctuation value, forming several dimensionality reduction vectors.

[0133] First, the complexity calculation unit calculates the variance of the word vector dimension to quantify the complexity of the word vector. Next, the complexity fluctuation calculation unit calculates the standard deviation of the complexity to obtain the complexity fluctuation value, which measures the volatility of the word vector. Finally, the dimensionality reduction unit reduces the dimensionality of the word vector based on the complexity fluctuation value, generating the dimensionality-reduced word vector.

[0134] By measuring the diversity and stability of word vectors through computational complexity and fluctuation values, targeted dimensionality reduction can be achieved. Dimensionality reduction reduces the impact of redundant information while preserving the semantic features of word vectors, thus improving the efficiency and accuracy of data processing.

[0135] Please continue reading. Figure 4 As shown, it is the logic diagram for determining the dimensionality reduction of the dimensionality reduction operation subunit in this embodiment;

[0136] Specifically, the dimensionality reduction unit includes:

[0137] The dimension reduction comparison subunit is connected to the complex fluctuation calculation unit to compare the complex fluctuation value with the preset standard complex fluctuation value to form a dimension reduction comparison result.

[0138] A dimensionality reduction operation subunit, which is connected to the dimensionality reduction comparison subunit, is used to reduce the dimensionality of each word vector according to a preset dimensionality reduction model and a preset dimensionality reduction quantity when the dimensionality reduction comparison result is that the complex fluctuation value is greater than the preset standard complex fluctuation value, thereby forming a number of dimensionality reduction vectors.

[0139] A preset dimensionality reduction model is a predefined mathematical or statistical model used to compress high-dimensional data into low-dimensional data. Its value depends on the dimensionality reduction objective (such as maximizing data interpretability, preserving key features, or improving computational efficiency) and the dimensionality reduction algorithm used (such as PCA and t-SNE). Typically, different models are selected based on task requirements. For example, PCA is usually used for linear dimensionality reduction, while t-SNE is suitable for non-linear dimensionality reduction. In this embodiment, PCA is chosen as the preset dimensionality reduction model to preserve as much of the key information as possible through linear transformation, effectively reducing dimensionality, lowering computational burden, and preserving data variability as much as possible.

[0140] The preset dimensionality reduction number refers to the desired target dimension after reducing the dimensionality of the word vectors. This number typically depends on the number of features in the data, computational efficiency, and model complexity. It is usually set between 10% and 30% of the original dimension to maintain information integrity as much as possible while reducing computational complexity. In this embodiment, the preset dimensionality reduction number is set to 100 dimensions, reducing the 300-dimensional vector of each word to 100 dimensions. This approach preserves semantic information and key data features as much as possible, effectively reducing computational load and storage requirements while maintaining sufficient accuracy in semantic expression. This significantly reduces processing complexity and accelerates analysis without excessively losing key information.

[0141] The word vectors undergo precise dimensionality reduction using two sub-units. First, the dimensionality reduction comparison sub-unit calculates the complexity fluctuation value of the word vectors and compares it with a preset standard complexity fluctuation value to obtain the dimensionality reduction comparison result. Next, the dimensionality reduction operation sub-unit applies the corresponding dimensionality reduction operation to reduce the dimensionality of the word vectors based on the dimensionality reduction comparison result, the preset dimensionality reduction model, and the number of dimensionality reductions, generating the dimensionality-reduced word vectors.

[0142] By precisely combining dimensionality reduction comparison and dimensionality reduction operations, the dimensionality reduction process can be optimized according to the complexity and volatility of word vectors, avoiding excessive loss of information. By comparing with preset standards, the system can automatically adjust the dimensionality reduction strategy to achieve more efficient semantic representation and better computational performance.

[0143] Specifically, the correction module includes:

[0144] A similarity calculation unit is used to calculate the cosine similarity of the reduced-dimensional vectors to form semantic similarity;

[0145] A correction unit, connected to the similarity calculation unit, is used to correct the preset dimensionality reduction number based on the relative deviation between the semantic similarity and the preset standard similarity and a preset correction coefficient. The relative deviation between the semantic similarity and the preset standard similarity is positively correlated with the corrected preset dimensionality reduction number.

[0146] The preset correction coefficient is a parameter used to control the sensitivity and adjustment magnitude during the dimensionality reduction correction process. It determines the strength of the correction operation based on the deviation between the predetermined standard similarity and the calculated actual semantic similarity. This depends on the specific application requirements, such as the need for more accurate semantic matching or the tolerance for some semantic loss. It is typically set between 0 and 1; in this embodiment, it is set to 0.1 to ensure semantic accuracy while avoiding over-adjustment and preventing the dimensionality reduction operation from being too aggressive, leading to the loss of useful information.

[0147] First, the cosine similarity between the dimensionality-reduced vectors is calculated to obtain the semantic similarity of each vector. Then, the correction unit adjusts the preset number of dimensionality reductions based on the relative deviation between the calculated semantic similarity and the preset standard similarity, combined with a preset correction coefficient. The adjusted number of dimensionality reductions better matches the actual semantic consistency, thereby optimizing the dimensionality reduction results.

[0148] By dynamically adjusting the number of dimensionality reduction values ​​based on the deviation between semantic similarity and a preset standard similarity, the dimensionality reduction operation is ensured to retain semantic information while further reducing the loss of redundant data. Adjusting the number of dimensionality reduction values ​​can improve the quality of word vectors, enhance the accuracy and consistency of semantic representation, and thus improve the effectiveness of subsequent analysis and model training.

[0149] Specifically, the generation module includes:

[0150] A combination unit is used to combine all the word vectors and each of the dimensionality-reduced vectors by weighted averaging to form several temporary sentence vectors;

[0151] A generation unit, connected to the combination unit, is used to perform regularization processing on each of the temporary sentence vectors according to a preset post-processing algorithm to form several sentence vectors.

[0152] The pre-processing algorithm refers to the algorithm that further processes the generated temporary sentence vectors. Its purpose is to optimize the quality of the sentence vectors, improve their representational power, and ensure they are better suited for subsequent tasks and analysis. This depends on task requirements, data characteristics, and computational resources. This embodiment uses L2 normalization, which ensures that the generated sentence vectors have the same magnitude, thereby avoiding the impact of differences in sentence vector lengths on similarity calculation.

[0153] First, the combination unit performs a weighted average combination of all word vectors and dimensionality-reduced vectors to generate temporary sentence vectors. Then, the generation unit applies a pre-defined post-processing algorithm to regularize these temporary sentence vectors, ensuring that the generated sentence vectors are within a certain range and satisfy semantic consistency and computational stability, thus forming the final sentence vectors.

[0154] By combining weighted averages with word vectors and dimensionality-reduced vectors, sentence vectors can comprehensively reflect the semantic information in the text. Regularization improves the stability and consistency of the generated sentence vectors, ensuring the normalization of the vectors and avoiding the adverse effects of excessively large or small values ​​on subsequent analysis, thus helping to improve the accuracy and robustness of the model.

[0155] Specifically, the analysis module includes:

[0156] The analysis unit is used to cluster the sentence vectors according to the preset Gaussian mixture model to form clustering results;

[0157] An extraction unit, connected to the analysis unit, is used to extract the part-of-speech distribution features of the labeled phrases using a conditional random field (CRF) model, thereby forming a feature extraction result;

[0158] An output unit, which is connected to the analysis unit and the extraction unit respectively, is used to output an analysis report based on the clustering results and the feature extraction results.

[0159] The analysis unit uses a pre-defined Gaussian mixture model to cluster the generated sentence vectors, identifying the similarities between different sentence vectors and generating clustering results. Next, the extraction unit extracts the part-of-speech distribution features of the tagged phrases in the sentences, generating feature extraction results. Finally, the output unit generates and outputs an analysis report based on the clustering and feature extraction results, providing a deeper understanding and analysis of the text.

[0160] Sentence vector clustering using a Gaussian mixture model effectively groups similar sentences into the same category, helping to reveal the underlying structure and patterns in the text. Combining this with the extraction of part-of-speech distribution features further enhances the ability to analyze the deep grammatical structure of the text. The final analysis report provides accurate data support and decision-making basis for subsequent text processing, classification, or applications.

[0161] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0162] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A smart platform for medical data storage, processing, and analysis based on the Python language, characterized in that: include: The data acquisition module is used to collect text data in real time during the medical document input process; A determination module, which is connected to the data acquisition module, is used to determine several complex texts based on the text data; The word segmentation module is connected to the determination module and is used to segment each complex text with a preset word segmentation tool and a preset segmentation granularity to form several phrases. The annotation module, which is connected to the word segmentation module, is used to annotate the part of speech of each phrase according to a preset annotation tool, thereby forming several annotated phrases; An adjustment module, which is connected to the annotation module, is used to adjust the preset segmentation granularity according to the grammatical consistency of each annotated phrase at any adjacent time within a preset adjustment period. A conversion module, which is connected to the annotation module, is used to convert each of the annotation phrases according to a preset word vector model to form several word vectors; A dimensionality reduction module, which is connected to the transformation module, is used to reduce the dimensionality of each word vector by a preset number of dimensionality reductions based on the dimensionality of all word vectors within a preset determination time period, thereby forming several dimensionality-reduced vectors. A correction module, which is connected to the dimensionality reduction module, is used to correct a preset number of dimensionality reductions based on the semantic similarity and preset standard similarity of any adjacent dimensionality reduction vectors. A generation module, which is connected to the transformation module and the dimensionality reduction module respectively, is used to generate several sentence vectors based on all the word vectors and the dimensionality reduction vectors; A storage module, connected to the generation module, is used to associate and store the sentence vector and the labeled phrase in a preset database; An analysis module, connected to the storage module, is used to analyze the sentence vectors and the labeled phrases according to a preset Gaussian mixture model and generate an analysis report; The determining module includes: A vocabulary recognition unit is used to recognize different words in the text data to form a number of different words, and to recognize the total vocabulary in the text data to form a number of total words; A complexity calculation unit, connected to the word recognition unit, is used to calculate the ratio of the number of different words to the total number of words to form the text complexity. A distribution calculation unit, which is connected to the complexity calculation unit, is used to calculate the standard deviation of the text complexity within a preset historical time period to form a complexity distribution degree; A distribution degree comparison unit, which is connected to the distribution calculation unit, is used to compare the complex distribution degree with the preset standard distribution degree to form a distribution comparison result; A determination unit, connected to the distribution degree comparison unit, is used to determine a number of complex texts based on the distribution comparison result, the text complexity, and the comparison result of the preset standard complexity. The adjustment module includes: A sorting unit is used to sort the labeled phrases according to their parts of speech to form a part-of-speech sequence; A consistency calculation unit, which is connected to the sorting unit, is used to calculate the cosine similarity of the part-of-speech sequence at any adjacent time to form a grammatical consistency score. An adjustment unit, connected to the consistency calculation unit, is used to adjust the preset segmentation granularity according to the syntax consistency. The adjustment unit includes: The consistency comparison subunit is used to compare the syntactic consistency with the preset standard consistency to form a consistency comparison result; An adjustment subunit, connected to the consistency comparison subunit, is used to adjust the preset segmentation granularity based on the consistency comparison result, the preset standard consistency, and a preset adjustment coefficient; the dimensionality reduction module includes: A complexity calculation unit is used to calculate the variance of the dimension to form the complexity. A complex fluctuation calculation unit, which is connected to the complexity calculation unit, is used to calculate the standard deviation of the complexity to form a complex fluctuation value; A dimensionality reduction unit, which is connected to the complex fluctuation calculation unit, is used to reduce the dimensionality of each word vector according to the complex fluctuation value to form several dimensionality reduction vectors. The dimensionality reduction unit includes: The dimension reduction comparison subunit is connected to the complex fluctuation calculation unit to compare the complex fluctuation value with the preset standard complex fluctuation value to form a dimension reduction comparison result. A dimensionality reduction operation subunit, which is connected to the dimensionality reduction comparison subunit, is used to reduce the dimensionality of each word vector according to the dimensionality reduction comparison result, a preset dimensionality reduction model and a preset dimensionality reduction quantity, to form several dimensionality reduction vectors; The correction module includes: A similarity calculation unit is used to calculate the cosine similarity of the reduced-dimensional vectors to form semantic similarity; A correction unit, connected to the similarity calculation unit, is used to correct the preset dimensionality reduction number based on the relative deviation between the semantic similarity and the preset standard similarity, and a preset correction coefficient.

2. The intelligent platform for medical data storage, processing, and analysis based on Python language according to claim 1, characterized in that, The word segmentation module includes: A stop word recognition unit is used to identify stop words in the complex text and form a number of stop words; A segmentation determination unit, which is connected to the stop word recognition unit, is used to determine whether to perform segmentation based on a preset stop word list and each stop word, thereby forming a segmentation determination result; The word segmentation unit is connected to the segmentation determination unit and is used to segment each complex text according to the segmentation determination result, the preset segmentation granularity and the preset word segmentation tool to form several phrases.

3. The intelligent platform for medical data storage, processing, and analysis based on Python language according to claim 2, characterized in that, The generation module includes: A combination unit is used to combine all the word vectors and each of the dimensionality-reduced vectors by weighted averaging to form several temporary sentence vectors; A generation unit, connected to the combination unit, is used to perform regularization processing on each of the temporary sentence vectors according to a preset post-processing algorithm to form several sentence vectors.

4. The intelligent platform for medical data storage, processing, and analysis based on Python language according to claim 3, characterized in that, The analysis module includes: The analysis unit is used to cluster the sentence vectors according to the preset Gaussian mixture model to form clustering results; An extraction unit, connected to the analysis unit, is used to extract the part-of-speech distribution features of the labeled phrases to form feature extraction results; An output unit, which is connected to the analysis unit and the extraction unit respectively, is used to output an analysis report based on the clustering results and the feature extraction results.

Images