Hospital operation multi-modal data processing method and system based on deep learning
By using deep learning methods to process textual and numerical data in hospital operations, and by employing a weighted fusion of word vectors and numerical labels, the problem of decision-making ambiguity caused by large differences between textual and numerical data is solved, thus achieving more accurate operational management.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 浙江融健科技有限公司
- Filing Date
- 2026-01-26
- Publication Date
- 2026-05-12
AI Technical Summary
In hospital operation and management, textual and numerical data have large differences in distribution, scale and semantics. Simply splicing them together cannot effectively capture their interaction relationships, resulting in unclear management decision-making basis.
By using a deep learning-based approach, natural language processing models are used to output word vectors and numerical label vectors. TF-IDF and cosine similarity are combined to adjust the importance of data, and weighted fusion of text and numerical data is performed. Finally, a neural network is used to output the fused vector, providing a decision reference.
It achieves deep integration of text and numerical data, providing more comprehensive operational decision-making information, highlighting key cost indicators and abnormal data, and improving the accuracy of management decisions.
Smart Images
Figure CN121583483B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of hospital operation management technology. More specifically, this invention relates to a method and system for multimodal data processing in hospital operations based on deep learning. Background Technology
[0002] With the development of artificial intelligence, multimodal learning has become a research hotspot. In the field of hospital operation and management, multimodal data processing integrates and analyzes data from different sources and types, such as financial reports, resource usage records, cost data, and operation logs.
[0003] In multimodal data processing scenarios in hospital operations, it is often necessary to combine textual data, such as operational reports and financial reports, with numerical data, such as cost indicators and resource utilization indicators, for analysis. Early fusion methods directly connected (concatenated) textual and numerical data, ignoring the significant differences in distribution, scale, and semantics between them. Simple concatenation could not effectively capture the interactions between them, leading to unclear decision-making criteria for management. Summary of the Invention
[0004] To address the technical problem that the large differences in distribution, scale, and semantics between textual and numerical data make simple splicing ineffective in capturing their interaction relationships, thus leading to ambiguous decision-making basis for managers, this invention provides solutions in the following aspects.
[0005] In the first aspect, a deep learning-based multimodal data processing method for hospital operations includes:
[0006] The hospital management database retrieves textual and numerical data related to operations. The textual data includes operational and financial reports, while the numerical data includes revenue, costs, and human resource efficiency. The numerical data also includes numerical labels.
[0007] The text data and numerical data labels are input into the trained natural language processing model, which outputs the word vector of each word in the text data and the word vector of each label in the numerical data. The TF-IDF of each word in the text data is calculated, and the corresponding TF-IDF is adjusted based on the similarity between word vectors in the text data to obtain the text data importance of the word. The numerical data importance of each label is calculated using the relative offset of the corresponding numerical values of the labels.
[0008] For each word vector in the text data and each label word vector in the numerical data, the correlation between the word and the label is calculated by combining the importance of the corresponding text data and the importance of the numerical data; based on the correlation, weighted fusion on the text side and weighted fusion on the numerical side are performed respectively to obtain the overall text vector and numerical vector; nonlinear fusion is then applied to the text vector and numerical vector to obtain the final fused vector.
[0009] The final fusion vector is input into the trained neural network, and the output includes cost risk and operational efficiency assessments, providing decision-making references for managers based on the output results.
[0010] Preferably, the natural language processing model is a word vector model.
[0011] Preferably, the process of obtaining the importance of the textual data of the words includes:
[0012] Calculate the absolute value of the cosine similarity between word vectors in the text data, and use the mean of all the absolute values of cosine similarity in the text data as an adjustment factor; use the product of the adjustment factor and the TF-IDF as the text data importance of the word.
[0013] Preferably, the process of obtaining the numerical importance of the label data includes:
[0014] The normal range of values for each label is obtained based on historical numerical data statistics;
[0015] When the value of the label is within the normal range, the relative offset of the label is set to 0; when the value of the label is not within the normal range, the maximum value among the difference between the lower limit of the normal range corresponding to each label and the label value and the upper limit of the normal range corresponding to the label is used as the relative offset of the label.
[0016] Calculate the ratio of the relative offset degree to the difference between the upper and lower limits of the normal range, and use this ratio as the numerical importance of the label.
[0017] Preferably, the process of obtaining the relevance between the word and the tag includes:
[0018] Select any word from the text data as the first target and any label as the second target. Calculate the absolute value of the cosine similarity between the word vectors of the first target and the word vectors of the second target, and label it as the first factor. Also, calculate the normalized value of the product of the text data importance of the first target and the numerical data importance of the second target, and label it as the second factor.
[0019] The product of the first factor and the second factor is used as the correlation between the first objective and the second objective.
[0020] Preferably, the process of obtaining the overall text vector includes:
[0021] Iterate through all the second targets, calculate the sum of the normalized values of the correlation between all the second targets and the first target; calculate the product of the sum of the normalized values of the correlation between all the second targets and the first target and the importance of the text data of the first target, and iterate through all the first targets to accumulate the results of the product to obtain the overall text vector.
[0022] Preferably, the process of obtaining the overall numerical vector includes:
[0023] Iterate through all first targets, calculate the sum of the normalized values of the correlation between all first targets and second targets; calculate the product of the sum of the normalized values of the correlation between all first targets and second targets and the numerical importance of the second target, and iterate through all second targets to accumulate the product results to obtain the overall numerical vector.
[0024] Secondly, a deep learning-based multimodal data processing system for hospital operations includes a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the deep learning-based multimodal data processing method for hospital operations described in any one of the claims is implemented.
[0025] The beneficial effects of this invention are:
[0026] This invention acquires textual data (operational reports, financial reports) and numerical data (revenue, costs, and human resource efficiency) from a hospital management database, and outputs word vectors using natural language processing models. Combined with importance calculation and correlation analysis, it achieves deep fusion of textual and numerical data, overcoming the shortcomings of simple concatenation. This allows different types of data to complement each other, providing more comprehensive information for operational decision-making. Specifically, by calculating the cosine similarity between word vectors to obtain an adjustment factor, and adjusting the TF-IDF based on this factor, the importance of each word in the textual data can be more accurately reflected. The invention also considers the impact of semantic relationships between words on word importance, ensuring that important words are more fully represented in subsequent analysis. Furthermore, by statistically obtaining the normal range of values corresponding to each label based on historical numerical data, and calculating the importance of numerical data based on the relative deviation of the label values from the normal range, the invention highlights values that deviate from the normal range. These values are often significant for cost control and resource optimization, helping managers to focus more on key cost indicators. Attached Figure Description
[0027] Figure 1 This is a flowchart of steps S1-S4 in the deep learning-based multimodal data processing method for hospital operations according to an embodiment of the present invention.
[0028] Figure 2This is a structural block diagram of a hospital operation multimodal data processing system based on deep learning, according to an embodiment of the present invention. Detailed Implementation
[0029] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.
[0030] Reference Figure 1 The deep learning-based multimodal data processing method for hospital operations includes steps S1-S4, as detailed below:
[0031] S1: Obtain textual and numerical data of operations from the hospital management database. The textual data includes operational reports and financial reports, while the numerical data includes revenue, costs, and human resource efficiency. The numerical data also includes numerical labels.
[0032] In one embodiment, based on the database account and permissions provided by the hospital, the user logs into the hospital management database through a secure channel, associates the operation table with the department ID or project ID, and extracts text data such as project descriptions (e.g., department descriptions of operational status), historical operation records, financial analysis reports, and operation reports, as well as numerical data such as revenue, costs, and human resource efficiency, specifically such as human resource costs, equipment costs, energy consumption, material utilization rate, budget execution rate, and cost overrun rate, etc., with each numerical data corresponding to a label, i.e., the specific value.
[0033] S2: Input the labels of text data and numerical data into the trained natural language processing model, and output the word vector of each word in the text data and the word vector of each label in the numerical data; calculate the TF-IDF of each word in the text data, and adjust the corresponding TF-IDF based on the similarity between word vectors in the text data to obtain the text data importance of the word; calculate the numerical data importance of each label using the relative offset of the corresponding numerical values of the labels.
[0034] After acquiring textual and numerical data on hospital operations, it is necessary to integrate and analyze these two types of data to support operational decision-making. Directly concatenating these two types of data will result in problems such as uneven dimensionality and semantic mismatch, because textual data is in the form of natural language, while numerical data is simply numbers, making direct correlation and analysis difficult.
[0035] Therefore, converting the labels of numerical data into vectors places them in the same semantic space as the word vectors obtained from processed text data. For example, if the text data mentions "cost overrun," and the numerical data contains the label "cost overrun rate" along with specific values, converting the "cost overrun rate" label into a vector allows for semantic comparison and association with the word vector of "cost overrun," thus better integrating the two types of data and providing a foundation for subsequent analysis.
[0036] In one embodiment, after performing conventional processing operations such as word segmentation and removal of invalid words on the text data, the text data and the labels of the numerical data are respectively input into a trained natural language processing model, such as a word vector model, and the corresponding word vectors are output.
[0037] Operational documents contain many technical terms and rare words, which are of great importance for accurately analyzing operational status.
[0038] Term Frequency (TF) reflects the frequency of a word's occurrence in a given text. A word appearing frequently is likely closely related to the condition or symptom described in the text. For example, in an operations report on budget execution, the word "overspending" might appear frequently, indicating a high correlation with the department's operational status. Inverse Document Frequency (IDF) is used to reduce the influence of common words and highlight rare but crucial ones. For instance, common words like "of" and "is" appear frequently in all texts, but they are not meaningful for judging operational status. Conversely, rare operational terms like "resource idleness" and "abnormal cost-benefit ratio," while appearing less frequently, are crucial for accurately analyzing operational status. IDF assigns lower weights to common words and higher weights to rare words, allowing for a greater focus on words important for operational decisions when analyzing text.
[0039] TF-IDF (Term Frequency-Inverse Document Frequency) combines term frequency and inverse document frequency to more accurately measure the relative importance of words in text data. By calculating TF-IDF values, words that appear frequently in a specific text but are relatively rare in the entire document collection can be identified; these words often contain key information about operational status.
[0040] It should be noted that since the above TF-IDF calculation method is existing technology, the specific calculation content will not be described in detail here.
[0041] While TF-IDF can measure the relative importance of words in text data, it only considers the frequency of word occurrences and neglects word meaning. For example, "cost overrun" and "budget overflow" are semantically very similar, but TF-IDF might treat them as different words and calculate their importance separately, leading to inaccurate importance calculations. To address the issue of TF-IDF ignoring semantics, it needs to be improved by incorporating the similarity between word vectors.
[0042] In one embodiment, the absolute value of the cosine similarity between word vectors in the text data is calculated. The mean of the absolute values of all cosine similarities in the text data is used as an adjustment factor. The calculated adjustment factor is multiplied by the TF-IDF of the word calculated above to adjust and obtain the text data importance of each word in the text data.
[0043] Through the above adjustments, cosine similarity measures the directional similarity between two vectors. In natural language processing, it measures the semantic similarity between two word vectors. Calculating the mean of the absolute values of cosine similarity between word vectors in text data comprehensively considers the semantic associations between words. Furthermore, this mean is used as an adjustment factor and multiplied by TF-IDF, ensuring that the importance adjustment considers not only the frequency of individual words and document frequency, but also their position within the entire text semantic network. For words with similar semantics but different TF-IDF values, the adjusted importance reflects the actual importance of the word in the text, highlighting the importance of rare operational terms and reducing interference from redundant text data.
[0044] Furthermore, for numerical data, by calculating the importance of numerical data labels, the most critical numerical indicators for judging operational status can be identified. For example, when analyzing cost control, indicators such as labor costs, equipment costs, and energy consumption are of high importance; while when analyzing resource efficiency, material utilization rate and budget execution rate are more prominent. Managers can use these highly important numerical indicators to more accurately judge the hospital's operational status.
[0045] In one embodiment, the importance of the numerical data of each label is calculated by using the relative offset of the corresponding numerical values of the labels.
[0046] Specifically, historical data values for each label are extracted from the hospital management database. For a single label, the corresponding historical data are arranged in ascending order, and specific quantiles are calculated to determine the upper and lower limits of the normal range. For example, the 5th percentile is selected as the lower limit, and the 95th percentile as the upper limit, so that data in between are considered normal.
[0047] Then, the deviation of the label value from the normal range is analyzed. When the label value is within the normal range, the relative offset of the label is set to 0. When the label value is not within the normal range, the maximum value among the difference between the lower limit of the normal range corresponding to each label and the label value and the upper limit of the corresponding normal range is calculated as the relative offset of the label.
[0048] Then, the ratio of the difference between the upper and lower limits of the relative offset and the normal range is calculated, and this ratio is used as the numerical importance of the label.
[0049] By defining the normal range, we can quickly identify which labels have values that deviate from the normal range. Furthermore, by setting the denominator, we can normalize the values and map the relative deviations to a relatively uniform scale, making the deviations of different labels comparable.
[0050] The closer the relative offset is to 0, the lower the importance of the corresponding tag; conversely, the greater the relative offset, the higher the importance of the corresponding tag.
[0051] Finally, based on the above operations, the textual importance of words in the textual data and the numerical importance of each label in the numerical data can be calculated.
[0052] S3: For the word vector of each word in the text data and the word vector of each label in the numerical data, calculate the correlation between the word and the label by combining the importance of the corresponding text data and the importance of the numerical data; perform weighted fusion on the text side and weighted fusion on the numerical side according to the correlation to obtain the overall text vector and numerical vector, and use non-linear fusion on the text vector and numerical vector to obtain the final fused vector.
[0053] After obtaining the importance of all textual and numerical data in S2 above, to further establish the correlation between textual words and numerical labels, the importance within each modality (i.e., within both textual and numerical data) is used as an "amplifier" to strengthen key words and key values. This approach considers both intramodal importance and intermodal relationships, which helps to more accurately perform data fusion and analytical diagnosis.
[0054] In one embodiment, for each word vector in the text data and each label vector in the numerical data, the correlation between the word and the label is calculated by combining the corresponding importance of the text data and the importance of the numerical data.
[0055] Specifically, select any word from the text data as the first target and any label as the second target. Calculate the absolute value of the cosine similarity between the word vectors of the first target and the word vectors of the second target, and label it as the first factor. Calculate the normalized value (e.g., using norm normalization) of the product of the text data importance of the first target and the numerical data importance of the second target, and label it as the second factor. The product of the first factor and the second factor is used as the correlation between the first target and the second target.
[0056] If the textual data of the first target and the numerical data of the second target are both highly important, and they are highly correlated (i.e., the absolute value of the cosine similarity between the word vectors of the first target and the word vectors of the second target is large), then the vector pairs of the first and second targets will have a high correlation in the fusion process. If one of them is less important or the absolute value of the cosine similarity between the word vectors of the first target and the word vectors of the second target is small, then the vector pairs of the first and second targets will have a low correlation in the fusion process.
[0057] The textual and numerical data importance calculated above reflects the relationship between a single word and a single label, but cannot directly support global decision-making. Therefore, we integrate the support of all numerical labels with textual words as the center, and integrate the support of all textual words with numerical labels as the center.
[0058] In other words, the fusion of the text side and the numerical side is bidirectional. Not only are text words fused according to the numerical importance of the tags, but tags are also fused according to the textual importance of the text words, which helps to capture the interaction between the two.
[0059] In one embodiment, the first and second targets selected above are still used as examples. Specifically, all second targets are iterated over, the sum of the normalized values of the correlation between all second targets and the first target is calculated, the product of the sum of the normalized values of the correlation between all second targets and the first target and the importance of the text data of the first target is calculated, and the product of the product is accumulated for all first targets to obtain the overall text vector.
[0060] Simultaneously, iterate through all first targets, calculate the sum of normalized values of the correlation between all first targets and second targets; calculate the product of the sum of normalized values of the correlation between all first targets and second targets and the numerical importance of the second target, and iterate through all second targets to accumulate the product results to obtain the overall numerical vector.
[0061] Furthermore, the overall text vector and numerical vector obtained from the above calculations are non-linearly fused to obtain the final fused vector, that is, the fusion form is as follows:
[0062]
[0063] In the formula, For the final fusion vector, This indicates that the text vector and the numerical vector mentioned above will be concatenated. This represents a nonlinear fusion function, where an MLP (Multilayer Perceptron) is used to perform a nonlinear transformation on the concatenated vectors.
[0064] Through the above operations, the final fusion vector has both semantic and numerical relevance, avoiding the situation where using only text may overlook numerical anomalies, and using only numerical values may lose operational descriptions.
[0065] S4: Input the final fusion vector into the trained neural network. The output includes cost risk and operational efficiency assessments, and provides decision-making references for managers based on the output results.
[0066] In one embodiment, the final fused vector obtained in S3 is input into a convolutional neural network. Convolutional neural networks, with their unique structure, can efficiently process complex information in operational data. Their convolutional layers extract local features from the input fused vector through a series of learnable convolutional kernels. These kernels can capture local patterns at different scales and locations in the operational data, such as identifying abnormal cost features in financial data or uncovering key problem combinations in operational reports. Pooling layers follow the convolutional layers, downsampling the feature maps output by taking local extrema (e.g., max pooling) or averages (e.g., average pooling) to reduce data dimensionality while retaining the most important feature information. This enhances the model's robustness to small changes in the input data, prevents overfitting, and extracts more representative global features.
[0067] When training convolutional neural networks, it's crucial to carefully select an appropriate loss function based on the specific task type. For classification tasks, such as identifying cost risk types, the cross-entropy loss function is a commonly used choice. The cross-entropy loss function measures the difference between the probability distribution predicted by the model and the true label probability distribution. By minimizing the cross-entropy loss, the model can continuously adjust its parameters, making the prediction results closer to reality, thus accurately identifying the type of cost risk in operations. For regression tasks, such as predicting the degree of cost overrun, the mean squared error loss function is more suitable. The mean squared error loss function calculates the average of the squares of the differences between the model's predicted values and the true values. By minimizing the mean squared error, the model can more accurately predict the severity of cost overruns, providing quantitative evidence for managers to develop optimization plans.
[0068] It should be noted that the convolutional neural network used above is existing technology, and will not be elaborated on further here.
[0069] Based on the model's output, comparisons and analyses are performed using a large-scale historical operational database. By identifying historical cases with similar operational conditions, cost characteristics, and risk profiles, the optimization process and final results of these cases are summarized. For example, if the model output recommends implementing cost control measure A for a department, the specific details of implementing measure A in similar cases can be listed, including key information such as implementation period, cost savings rate, and efficiency improvement rate. A suggestion like "Based on similar cases, we recommend implementing cost control measure A, with a cost savings rate of 15%" not only provides a specific measure selection but also offers the important indicator of the savings rate, allowing managers to more intuitively understand the effectiveness and reliability of the optimization plan.
[0070] Alternatively, when the model predicts a 25% risk of cost overruns within three months, in addition to providing the risk percentage, it is necessary to further analyze the possible types of overruns, as well as the probability and severity of each type. Simultaneously, based on this risk information, corresponding monitoring and intervention recommendations should be provided to management, such as "It is recommended to strengthen monitoring, conduct weekly cost audits, and adjust budget allocations promptly if abnormal indicators are observed."
[0071] In addition, to ensure the scientific validity and rationality of the optimization recommendations, multidisciplinary experts can be invited to review and discuss the model output and the processed recommendations.
[0072] This invention also provides a deep learning-based multimodal data processing system for hospital operations. For example... Figure 2 As shown, the system includes a processor and a memory, the memory storing computer program instructions, which, when executed by the processor, implement the deep learning-based multimodal data processing method for hospital operations according to the first aspect of the present invention.
[0073] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and will not be described in detail here.
[0074] It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept, and these all fall within the scope of protection of this invention. Therefore, the scope of protection of this patent should be determined by the appended claims.
Claims
1. A deep learning-based method for processing multimodal data on hospital operations, characterized in that, include: The hospital management database retrieves textual and numerical data related to operations. The textual data includes operational and financial reports, while the numerical data includes revenue, costs, and human resource efficiency. The numerical data also includes numerical labels. Input the labels of text data and numerical data into the trained natural language processing model, and output the word vector of each word in the text data and the word vector of each label in the numerical data. Calculate the TF-IDF of each word in the text data, and adjust the corresponding TF-IDF based on the similarity between word vectors in the text data to obtain the text data importance of the word; calculate the numerical data importance of each label using the relative offset of the corresponding label values; For each word vector in the text data and each label vector in the numerical data, the correlation between the word and the label is calculated by combining the importance of the corresponding text data and the importance of the numerical data. The process of obtaining the relevance between words and tags includes: Select any word from the text data as the first target and any label as the second target. Calculate the absolute value of the cosine similarity between the word vectors of the first target and the word vectors of the second target, and label it as the first factor. Also, calculate the normalized value of the product of the text data importance of the first target and the numerical data importance of the second target, and label it as the second factor. The product of the first factor and the second factor is taken as the correlation between the first objective and the second objective; Based on the correlation, weighted fusion is performed on both the text and numerical sides to obtain the overall text vector and numerical vector. Nonlinear fusion is then applied to the text and numerical vectors to obtain the final fused vector. The process of obtaining the overall text vector includes: Iterate through all the second targets, calculate the sum of the normalized values of the correlation between all the second targets and the first target; calculate the product of the sum of the normalized values of the correlation between all the second targets and the first target and the importance of the text data of the first target, and iterate through all the first targets to accumulate the results of the product to obtain the overall text vector. The overall process of obtaining the numerical vector includes: Iterate through all first targets, calculate the sum of the normalized values of the correlation between all first targets and second targets; calculate the product of the sum of the normalized values of the correlation between all first targets and second targets and the numerical importance of the second target, and iterate through all second targets to accumulate the product results to obtain the overall numerical vector. The final fusion vector is input into the trained neural network, and the output includes cost risk and operational efficiency assessments, providing decision-making references for managers based on the output results.
2. The method for processing multimodal hospital operation data based on deep learning according to claim 1, characterized in that, The natural language processing model selected is the word vector model.
3. The method for processing multimodal hospital operation data based on deep learning according to claim 1, characterized in that, The process of obtaining the textual data importance of the words includes: Calculate the absolute value of the cosine similarity between word vectors in the text data, and use the mean of all the absolute values of cosine similarity in the text data as an adjustment factor; use the product of the adjustment factor and the TF-IDF as the text data importance of the word.
4. The method for processing multimodal hospital operation data based on deep learning according to claim 1, characterized in that, The process of obtaining the numerical importance of the label data includes: The normal range of values for each label is obtained based on historical numerical data statistics; When the value of the label is within the normal range, the relative offset of the label is set to 0; when the value of the label is not within the normal range, the maximum value among the difference between the lower limit of the normal range corresponding to each label and the label value and the upper limit of the normal range corresponding to the label is used as the relative offset of the label. Calculate the ratio of the relative offset degree to the difference between the upper and lower limits of the normal range, and use this ratio as the numerical importance of the label.
5. A multimodal data processing system for hospital operations based on deep learning, characterized in that: include: A processor and a memory, the memory storing computer program instructions that, when executed by the processor, implement the deep learning-based multimodal data processing method for hospital operations according to any one of claims 1-4.