Medical data processing method, system and equipment based on big data and medium

Abstract

The invention discloses a medical data processing method, system and device based on big data and a medium. The method comprises the steps that medical image data and medical text data of a patient are acquired; performing feature extraction on the medical image data to obtain an image feature vector, and analyzing the medical text data to obtain a text feature matrix; inputting the image feature vector and the text feature matrix into a multi-modal fusion model, calculating an association weight of the image feature and the text feature, and obtaining a fusion feature representation; and establishing a disease prediction model according to the fusion feature representation, performing disease classification diagnosis on the patient, and generating a structured diagnosis report according to a diagnosis result. According to the method, the medical image data and the text data are deeply fused through the multi-modal fusion model, the complementary advantages of different modal data are fully utilized, and the problem of information missing possibly existing in a single data source is avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical data processing, and in particular to a medical data processing method, system, device and medium based on big data. BACKGROUND

[0002] With the rapid development of medical informatization and the wide application of big data technology, medical data presents the characteristics of large data volume, various types and complex structure. Medical data mainly includes medical image data, electronic medical record text data, examination data and other modalities. These data contain rich disease diagnosis information and patient health status information. Using artificial intelligence technology to intelligently process and analyze medical data to assist doctors in disease diagnosis and treatment decision has become an important development direction in the field of medical health. At present, medical data processing technology based on deep learning has made significant progress in disease recognition, image analysis and text mining.

[0003] However, the existing medical data processing method mostly uses single modal data for analysis, such as disease diagnosis based on medical images only, or semantic analysis of medical text only, lacking effective fusion of multi-modal medical data. This single modal processing method is easy to lead to insufficient information utilization, affecting the accuracy and reliability of diagnosis. At the same time, the existing method often ignores the semantic association and context relationship between medical terms when processing medical text, and lacks adaptive weight distribution mechanism in feature fusion, making it difficult to fully play the complementary advantages of different modal data, limiting the precision and effect of disease diagnosis. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides a medical data processing method, system, device and medium based on big data to solve the problems that the existing detection and monitoring methods are time-limited, require unit cooling preparation in advance, and have high time and material costs; at the same time, the detection results are static data, which cannot predict the wall thickness change and service life length in real time and perform corresponding visual warning.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides a medical data processing method based on big data, comprising the steps of:

[0008] obtaining medical image data and medical text data of a patient;

[0009] performing feature extraction on the medical image data to obtain an image feature vector, and analyzing the medical text data to obtain a text feature matrix;

[0010] inputting the image feature vector and the text feature matrix into a multi-modal fusion model, calculating the correlation weight of the image feature and the text feature, and obtaining a fusion feature representation;

[0011] establishing a disease prediction model according to the fusion feature representation, performing disease classification diagnosis on the patient, and generating a structured diagnosis report according to the diagnosis result.

[0012] As a preferred scheme of the medical data processing method based on big data, the step of analyzing the medical text data comprises:

[0013] The medical text data is subjected to word segmentation and standardization processing, the medical terms are converted into vector representation through word embedding technology, and a first string is obtained;

[0014] The first string is input into a position coding module, position information and context relationship are added to each medical term, and a second string containing time sequence features is generated;

[0015] The second string is subjected to deep semantic analysis, the key features and semantic associations of the medical text are extracted, and a third string subjected to semantic enhancement is output;

[0016] A probability attention prediction model is constructed through the third string, the correlation probability between the medical terms is calculated, and the text feature matrix is formed.

[0017] The beneficial effects of the preferred technical scheme are that the context relationship and time sequence features of the medical terms can be processed through the word embedding technology and the position coding module, the correlation probability between the medical terms can be calculated through the probability attention prediction model, and the semantic understanding accuracy of the medical text is improved.

[0018] As a preferred scheme of the medical data processing method based on big data, the step of inputting the image feature vector and the text feature matrix into a multi-modal fusion model comprises:

[0019] The correlation probability between the medical terms is calculated according to the probability attention prediction model, and a first probability matrix is obtained;

[0020] The semantic similarity score between adjacent medical terms is calculated, and a second probability matrix is obtained;

[0021] The spatial distance difference between the medical terms is calculated through a semantic vector space, and a third probability matrix is obtained;

[0022] The image feature vector is weighted and fused with the first probability matrix, the second probability matrix and the third probability matrix, and the fusion feature representation is output.

[0023] The beneficial effects of the preferred technical scheme are that three different probability matrices are constructed to capture the correlation probability, semantic similarity and spatial distance difference respectively, and are fused and calculated by weighting with image features, so that the complementary information of image and text data is fully utilized, and the limitations of single fusion mode are avoided.

[0024] As a preferred scheme of the medical data processing method based on big data, the step of establishing a disease prediction model according to the fusion feature representation comprises:

[0025] The fusion feature representation is input into a feature extraction layer to extract comprehensive disease feature data of the patient.

[0026] According to the comprehensive disease feature data and the historical diagnosis data feature of the patient, a disease feature vector of the patient is calculated.

[0027] A multi-classifier model is established based on the disease feature vector, including a first classifier based on image features, a second classifier based on text features and a third classifier based on fusion features.

[0028] The three prediction results are fused by weighting to obtain a disease classification probability distribution, and the disease prediction result is determined according to the disease classification probability distribution to complete the establishment of the disease prediction model.

[0029] The beneficial effects of the preferred technical scheme are that a multi-classifier model containing image, text and fusion features is established, the historical diagnosis data feature of the patient is comprehensively judged, and the stability and accuracy of disease prediction are improved through a multiple verification mechanism and a weighting fusion strategy.

[0030] As a preferred scheme of the medical data processing method based on big data, the step of performing disease classification diagnosis on the patient comprises:

[0031] The fusion feature representation of the patient to be diagnosed is input into the disease prediction model to obtain a disease classification probability distribution.

[0032] According to the disease classification probability distribution, the historical diagnosis data feature of the patient is extracted, the patient is classified in a first classification, and a preset disease category of the patient is obtained.

[0033] The historical diagnosis data feature of the patient in a preset time period is obtained, the difference between the historical diagnosis data feature and the current disease feature vector is calculated, and a historical difference value is obtained.

[0034] The historical difference value is divided into different range intervals according to a preset level, and the probability weight of the estimated disease in different range intervals is calculated.

[0035] According to the probability weight, the diagnostic probability of different diagnostic results is calculated, the diagnostic results are classified secondly, and a disease diagnostic result is obtained.

[0036] The preferred technical scheme has the beneficial effects that by analyzing the difference between the historical diagnostic data features of the patient and the current disease features, the probability weight is divided according to the level, the personalized diagnosis based on the historical data is realized, and the accuracy of disease classification is improved.

[0037] As a preferred scheme of the medical data processing method based on big data, the step of generating a structured diagnostic report according to the diagnostic result comprises:

[0038] According to the disease diagnostic result, a preset medical diagnostic template is matched to obtain a basic diagnostic report framework.

[0039] Pathological description information, treatment suggestion information and matters needing attention information related to the disease diagnostic result are retrieved from a medical knowledge base.

[0040] The image feature vector is converted into an image analysis description, and the text feature matrix is converted into a symptom summary description.

[0041] The patient basic information, image analysis description, symptom summary description, disease diagnostic result and treatment suggestion information are structured and organized.

[0042] A structured diagnostic report containing multi-modal analysis results is generated.

[0043] As a preferred scheme of the medical data processing method based on big data, the step of extracting features from the medical image data comprises:

[0044] The medical image data is standardized, and the image format and resolution parameters are unified.

[0045] The standardized medical image data is subjected to deep semantic analysis through a feature extraction layer, and spatial features and texture features of the medical image data are extracted.

[0046] The spatial features and texture features are subjected to weighted fusion calculation, and a feature representation containing spatial distance difference is generated in combination with position information.

[0047] The feature representation containing spatial distance difference is input into a semantic vector space for processing, and the image feature vector is output.

[0048] In a second aspect, the present application provides a medical data processing system based on big data, comprising:

[0049] A data acquisition module is configured to acquire medical image data and medical text data of a patient.

[0050] a multi-modal fusion module configured to input the image feature vector and the text feature matrix into a multi-modal fusion model, calculate the correlation weight of the image feature and the text feature, and obtain a fusion feature representation;

[0051] a disease prediction module configured to establish a disease prediction model according to the fusion feature representation;

[0052] a diagnosis module configured to perform disease classification diagnosis on the patient by using the disease prediction model;

[0053] a report generation module configured to generate a structured diagnosis report according to the diagnosis result.

[0054] In a third aspect, the present application provides an electronic device comprising:

[0055] a memory and a processor;

[0056] the memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions, so as to realize the steps of the medical data processing method based on big data.

[0057] In a fourth aspect, the present application provides a computer readable storage medium storing computer executable instructions, which, when executed by a processor, realize the steps of the medical data processing method based on big data.

[0058] Compared with the prior art, the present application has the following advantages:

[0059] The medical image data and the text data are deeply fused by the multi-modal fusion model, the complementary advantages of different modal data are fully utilized, and the information missing problem that may exist in a single data source is avoided. The probability attention prediction model and the deep semantic analysis technology are adopted, the semantic correlation and the context relationship between the medical terms are deeply mined, the correlation weight of the image feature and the text feature is calculated, the adaptive fusion of different modal features is realized, the disease diagnosis is more comprehensive and accurate, and the understanding depth and the diagnosis accuracy of the medical data are effectively improved.

[0060] In addition, the multi-classifier model is established and combined with the patient historical diagnosis data features, the stability and the reliability of the diagnosis result are enhanced through multiple verifications and probability weight calculation. Meanwhile, the correlation between the patient historical diagnosis data features and the current disease features is analyzed, the comprehensive judgment of the patient disease development trend is realized, the structured diagnosis report is automatically generated, the diagnosis output format is standardized, the influence of the human factors on the diagnosis result expression is reduced, the consistency and the professionalism of the diagnosis report are improved, and a more comprehensive reference basis is provided for the clinical decision. BRIEF DESCRIPTION OF DRAWINGS

[0061] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments description. Obviously, the drawings in the following description only some of the embodiments of the present application, and for those skilled in the art, without creative labor, can also obtain other drawings according to these drawings.

[0062] Figure 1 The overall flowchart of the medical data processing method based on big data according to an embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0064] Embodiment 1, refer to Figure 1 According to an embodiment of the present application, a medical data processing method based on big data is provided, comprising steps S1-S4:

[0065] S1, obtaining medical image data and medical text data of a patient;

[0066] S2, performing feature extraction on the medical image data to obtain an image feature vector, and performing analysis on the medical text data to obtain a text feature matrix;

[0067] S3, inputting the image feature vector and the text feature matrix into a multi-modal fusion model, calculating the correlation weight of the image feature and the text feature, and obtaining a fusion feature representation;

[0068] S4, establishing a disease prediction model according to the fusion feature representation, performing disease classification diagnosis on the patient, and generating a structured diagnosis report according to the diagnosis result.

[0069] It should be noted that the traditional medical data processing method often uses single modal data for analysis, such as disease diagnosis based only on medical images or only on text data. This way is easy to lead to insufficient information utilization, affecting the accuracy of diagnosis. At the same time, medical image data and text data have different data structures and feature representation methods, and direct fusion can easily cause feature conflict and information loss. Moreover, the existing method often ignores the semantic association and context relationship between medical terms when processing medical text, and lacks effective weight distribution mechanism when fusing features, making it difficult to fully play the complementary advantages of different modal data, limiting the precision and reliability of disease diagnosis.

[0070] Therefore, in view of the single modal analysis and feature fusion problems existing above, through the steps of S1-S4, effective acquisition and standardized processing of medical image data and text data are realized, high-quality feature representations are obtained through deep feature extraction and semantic analysis technology; the correlation weights of different modal features are calculated through the multi-modal fusion model to realize adaptive feature fusion, and the complementary information of image and text data is fully utilized; a disease prediction model is established based on the fused features to realize more accurate and reliable disease classification diagnosis, and a standardized diagnosis report is automatically generated to improve the intelligent level of medical diagnosis and the clinical application effect.

[0071] Embodiment 2, refer to Figure 1 For an embodiment of the present application, based on the above embodiment, a medical data processing method based on big data is provided.

[0072] In the embodiments of the present application, in step S2, the step of analyzing the medical text data includes S2.1.1-S2.1.4:

[0073] S2.1.1, the medical text data is segmented and standardized, the medical terms are converted into vector representations through word embedding technology, and a first string is obtained;

[0074] S2.1.2, the first string is input into a position coding module, position information and context relationship are added to each medical term, and a second string containing time sequence features is generated;

[0075] S2.1.3, the second string is subjected to deep semantic analysis, the key features and semantic associations of the medical text are extracted, and a third string subjected to semantic enhancement is output;

[0076] S2.1.4, a probability attention prediction model is constructed through the third string, the correlation probability between medical terms is calculated, and a text feature matrix is formed.

[0077] It should be noted that the traditional medical text processing method often uses simple keyword extraction or basic natural language processing technology, which easily ignores the complex semantic associations and context dependencies between medical terms. Medical text data has the characteristics of high density of professional terms, complex semantic hierarchy, strong correlation between terms, etc. Direct application of general text processing methods can easily cause loss of semantic information and inaccurate extraction of key features; moreover, existing methods often lack effective modeling of medical term position information and time sequence features when processing medical text, and lack quantitative analysis of the correlation probability between terms when constructing feature representations, making it difficult to fully exploit the deep semantic information of medical text and limiting the accuracy of subsequent disease diagnosis.

[0078] Therefore, in order to solve the problems of semantic understanding and feature extraction of medical text, the steps S2.1.1-S2.1.4 are used to realize the standardization preprocessing and high-quality word embedding representation of medical text data, add position information and time sequence features to medical terms through the position encoding module, enhance the structured expression of the text, extract the key features and semantic associations of medical text through deep semantic analysis technology, and construct a probability attention prediction model to quantify the association probability between medical terms, thereby forming a high-quality text feature matrix to provide accurate text feature representation for subsequent multi-modal fusion.

[0079] In the embodiments of the present application, the medical text data is segmented and standardized in step S2.1.1, and the medical terms are converted into vector representation through word embedding technology, including preprocessing the medical text to remove irrelevant symbols and stop words, segmenting the text using a medical field-specific segmenter, and unifying the term expression form through medical term standardization processing.

[0080] In an alternative embodiment, the word embedding technology in step S2.1.1 can also use a pre-trained medical field word vector model to combine the field knowledge graph to perform semantic enhancement on the medical terms, thereby improving the professionalism and accuracy of the word vector representation.

[0081] In another alternative embodiment, the standardization processing in step S2.1.1 can also include synonym merging and abbreviation expansion of medical terms, and the semantic unification of the terms is realized through a medical ontology library to avoid different expressions of the same concept affecting the subsequent analysis effect.

[0082] In the embodiments of the present application, the first string is input into the position encoding module in step S2.1.2 to add position information and context relationship to each medical term, including calculating the absolute position and relative position of the medical term in the text, generating a position encoding vector combined with the structural features of the medical document, and fusing it with the word embedding vector.

[0083] In the embodiments of the present application, the second string is subjected to deep semantic analysis in step S2.1.3 to extract the key features and semantic associations of the medical text, including processing the text representation containing position information through a multi-layer semantic analysis network, identifying the semantic dependency relationship between medical terms, and extracting key semantic features related to disease diagnosis.

[0084] It should be noted that the deep semantic analysis process gradually extracts deep semantic information of the text through a multi-layer neural network structure, which can effectively capture the complex association relationship between medical terms and avoid the limitations of traditional shallow feature extraction methods, thereby laying a foundation for constructing high-quality text feature representation.

[0085] In the embodiments of the present application, the probability attention prediction model constructed by the third string in step S2.1.4 calculates the correlation probability between medical terms, including calculating the attention weight between terms based on the semantic enhanced text representation, quantifying the contribution of different medical terms to the diagnosis result through the probability distribution, and combining the attention weight and the semantic feature to form the text feature matrix.

[0086] In an optional embodiment, the probability attention prediction model in step S2.1.4 can also combine prior knowledge in the medical knowledge graph to improve the accuracy and interpretability of the medical term correlation probability calculation through the knowledge enhanced attention mechanism.

[0087] In another optional embodiment, the probability attention prediction model constructed in step S2.1.4 can also use a multi-head attention mechanism to calculate the correlation between medical terms from different angles, and improve the expression ability and diagnosis relevance of the text feature matrix through multi-dimensional attention fusion.

[0088] In the embodiments of the present application, in step S2, the feature extraction step of the medical image data includes S2.2.1-S2.2.4:

[0089] S2.2.1, standardizing the medical image data to unify the image format and resolution parameters;

[0090] S2.2.2, performing deep semantic analysis on the standardized medical image data through a feature extraction layer to extract spatial features and texture features of the medical image data;

[0091] S2.2.3, weighted fusion calculation of spatial features and texture features, combined with position information to generate feature representation containing spatial distance difference;

[0092] S2.2.4, inputting the feature representation containing the spatial distance difference into the semantic vector space for processing to output the image feature vector.

[0093] It should be noted that the traditional medical image processing method often uses simple image preprocessing and basic feature extraction technology, which easily ignores the deep semantic information of complex spatial structures and texture features in medical images. Medical image data has the characteristics of large resolution difference, multiple imaging devices, and uneven image quality. Direct application of general image processing methods can easily cause loss of key diagnostic information and inaccurate feature representation; moreover, existing methods often lack effective fusion of spatial position information and texture features when extracting image features, and lack quantitative analysis of spatial distance difference when constructing feature representation, making it difficult to fully exploit the deep feature information of medical images, limiting the accuracy of subsequent disease diagnosis.

[0094] Therefore, in view of the existing medical image feature extraction and representation problems, through the steps of S2.2.1-S2.2.4, the standardized preprocessing and format unification of medical image data are realized, the spatial features and texture features of the image are analyzed through the deep feature extraction layer to enhance the structured expression of the image, and the feature representation containing the spatial distance difference is generated by combining the position information, and the high-quality image feature vector is formed by processing in the semantic vector space, thereby providing accurate image feature representation for subsequent multi-modal fusion.

[0095] In the embodiments of the present application, in step S2.2.1, the medical image data is standardized, the image format and resolution parameters are unified, including converting the medical images collected by different devices into a standard format, adjusting the image resolution to a preset range, and performing standardization processing on brightness, contrast and noise.

[0096] In an optional embodiment, the standardization processing in step S2.2.1 can also include image geometric correction and spatial registration, which can improve the image quality through image enhancement technology and ensure that medical images from different sources have consistent data features.

[0097] In another optional embodiment, the standardization processing in step S2.2.1 can also use an adaptive image preprocessing strategy to automatically select the optimal standardization scheme according to the image type and imaging parameters, thereby improving the pertinence and accuracy of the processing effect.

[0098] In the embodiments of the present application, in step S2.2.2, the standardized medical image data is analyzed by a feature extraction layer to extract spatial features and texture features of the medical image data, including extracting local spatial features of the image through a multi-layer convolutional network structure, and identifying texture patterns and structural information in the image through a texture analysis algorithm.

[0099] In the embodiments of the present application, in step S2.2.3, the spatial features and texture features are weighted and fused to generate a feature representation containing spatial distance differences by combining position information, including calculating the importance weight of the spatial features and texture features, analyzing the spatial relationship between the features by combining the position coordinate information in the image, and quantifying the spatial distance differences between different regions.

[0100] It should be noted that the above weighted fusion process can effectively integrate the multi-dimensional feature information of the image, and the spatial perception ability of the feature representation is enhanced by introducing the position information, thereby avoiding the limitation of ignoring the spatial relationship in the traditional feature fusion method, and providing a guarantee for constructing high-quality image feature representation.

[0101] In the embodiments of the present application, the feature representation containing the spatial distance difference in step S2.2.4 is input into the semantic vector space for processing, and an image feature vector is output, including mapping the fused feature representation to a high-dimensional semantic space, enhancing the expression ability of the feature through the transformation of the semantic vector space, and generating a standardized image feature vector.

[0102] In an optional embodiment, the semantic vector space processing in step S2.2.4 can also combine prior knowledge of medical images to enhance the semantic expression ability and diagnostic relevance of the image feature vector through knowledge-enhanced vector space transformation.

[0103] In another optional embodiment, the semantic vector space processing in step S2.2.4 can also adopt an adaptive dimension adjustment strategy to dynamically adjust the dimension of the feature vector according to the complexity of the image feature, thereby improving the computational efficiency while ensuring the feature expression ability.

[0104] In the embodiments of the present application, the step of inputting the image feature vector and the text feature matrix into the multi-modal fusion model in step S3 includes steps S3.1-S3.4:

[0105] S3.1, calculate the association probability between medical terms according to the probability attention prediction model to obtain a first probability matrix;

[0106] Specifically, the self-attention decoder is used to output a probability matrix H = [h1, h2,..., hn]T, where hi represents an output probability vector corresponding to the i-th output position, and n is the total number of medical terms. By calculating the correlation between probability vectors, the association probability distribution between medical terms is obtained. The calculation formula of the first probability matrix P1 is:

[0107]

[0108] where d is the dimension of the probability vector, the softmax function is used for normalization processing to ensure that the probability value is in the interval [0, 1], P1(i, j) is the element value of the i-th row and the j-th column in the first probability matrix, representing the association probability of term i and term j, and hi is the output probability vector corresponding to the j-th output position.

[0109] S3.2, calculate the semantic similarity score between adjacent medical terms to obtain a second probability matrix;

[0110] The cosine similarity of the output probability vector is used to calculate the semantic similarity between adjacent medical terms. For adjacent medical terms i and j, the semantic similarity calculation formula is:

[0111] Sim(i, j) = (hi · hj) / (||hi|| x ||hj||).

[0112] where Sim(i, j) is the cosine similarity score between term i and term j, ||hi||, ||hj|| are the L2-norms of the probability vectors hi and hj;

[0113] The semantic similarity scores of all adjacent terms are grouped into a second probability matrix M, where Mij represents the similarity score between term i and term j. Adjacent terms are defined as pairs of terms whose positions in the text sequence differ by no more than a window size k, which is set to 3 in this embodiment.

[0114] S3.3, calculate the spatial distance difference between medical terms in the semantic vector space to obtain a third probability matrix;

[0115] The semantic vector representation corresponding to each medical term is obtained using the Word2Vec technology. Let the semantic vector of term i be vi and the semantic vector of term j be vj. The Euclidean distance between terms in the semantic vector space is calculated as:

[0116] Dist(i, j) = ||vi-vj||2;

[0117] In order to convert the distance into a probability form, a Gaussian kernel function is used for conversion:

[0118] N(i, j) = exp(-Dist(i, j) 2 / 2σ 2 );

[0119] where Dist(i, j) is the Euclidean distance between the semantic vectors of terms i and j, and σ is the bandwidth parameter of the Gaussian kernel, which is set to 1.0 in this embodiment. The spatial distance probabilities of all term pairs form a third probability matrix N, where N(i, j) is an element in the third probability matrix, representing the probability value based on semantic distance, and exp(-Dist(i, j) 2 / 2σ 2 ) is a Gaussian kernel function that converts the distance into a probability value in the range [0, 1].

[0120] S3.4, weighted fusion calculation of image feature vector and first probability matrix, second probability matrix, third probability matrix, output fusion feature representation.

[0121] Let the image feature vector be V, with a dimension of d x 1. The image feature vector is expanded to the same dimension space as the probability matrix through linear transformation. The calculation formula of the fusion feature representation F is:

[0122]

[0123] where, represents the outer product operation of matrix and vector, and a1, a2, a3 represent the weight coefficients of the three probability matrices respectively, and β is the direct weight coefficient of the image feature. In the embodiment, a1=0.4, a2=0.3, a3=0.2, and β=0.1. The weight coefficients are obtained by cross-validation method on the training set, and the weight coefficients are optimized to maximize the discriminant performance of the fusion feature.

[0124] In the embodiment, in step S4, the step of establishing a disease prediction model according to the fusion feature representation includes S4.1.1-S4.1.4:

[0125] S4.1.1, input the fusion feature representation into a feature extraction layer to extract comprehensive disease feature data of the patient;

[0126] The feature extraction layer adopts a three-layer neural network structure for deep feature learning. The first layer maps the fusion feature representation from the original dimension to a 512-dimensional intermediate representation, and enhances the non-linear expression ability through a ReLU activation function. The second layer is further compressed to 256 dimensions to extract more abstract feature patterns. The third layer outputs 128-dimensional comprehensive disease feature data. Dropout technology is used between each layer to prevent overfitting, and the dropout rates are set to 30%, 20% and 10% respectively. This layer-by-layer dimension reduction design can effectively extract the core disease features of the patient and remove redundant information.

[0127] S4.1.2, calculate the disease feature vector of the patient according to the comprehensive disease feature data combined with the historical diagnosis data features of the patient; by obtaining the historical diagnosis records of the patient in the past year, the feature information of each diagnosis is extracted to form a historical feature sequence. A time attention mechanism is used to give different weights to the historical features at different time points, and the more recent diagnosis records have higher weights. Specifically, the time interval is converted into attention weights, the records within one month have a weight of 1.0, the records within three months have a weight of 0.8, the records within six months have a weight of 0.6, and the records within one year have a weight of 0.4. The weighted historical features are fused with the current comprehensive disease feature data, wherein the current features have a weight of 70% and the historical features have a weight of 30%, to form the final disease feature vector.

[0128] S4.1.3, establish a multi-classifier model based on the disease feature vector, including a first classifier based on the image feature, a second classifier based on the text feature, and a third classifier based on the fusion feature;

[0129] The 128-dimensional disease feature vector is segmented according to the feature source, the first 43 dimensions correspond to image-related features, the middle 43 dimensions correspond to text-related features, and the last 42 dimensions correspond to multi-modal fusion features. A special classifier is constructed for each type of feature. The image classifier focuses on identifying visual feature patterns and uses a convolutional layer residual connection structure; the text classifier focuses on semantic feature recognition and uses an attention mechanism to enhance semantic understanding; the fusion classifier processes cross-modal features and uses a gating mechanism to balance different modal information. Each classifier outputs a probability distribution of 10 disease categories. The network depth of the classifier is 3 layers, and the number of hidden layer neurons is 64, 32, and 10, respectively.

[0130] S4.1.4, the three prediction results are weighted and fused to obtain a disease classification probability distribution, the disease prediction result is judged according to the disease classification probability distribution, and the establishment of the disease prediction model is completed;

[0131] The prediction results of the three classifiers are fused by weighted average. The weight of the text classifier is set to 40%, because the medical text contains rich symptom description information; the weight of the image classifier is 35%, which provides important visual diagnostic basis; the weight of the fusion classifier is 25%, which is used as a supplement to cross-modal information. The final disease probability distribution is obtained after fusion, and the class with the highest probability is selected as the prediction result. When the highest probability is more than 80%, the diagnosis is considered reliable, when the probability is between 50% and 80%, it is marked as medium reliability, and when the probability is less than 50%, manual review is recommended. The cross-entropy loss function is used in the model training process, the Adam optimization algorithm is used, the initial learning rate is 0.001, the batch size is 32, and the training rounds are 100. The learning rate is dynamically adjusted through the validation set to prevent overfitting.

[0132] In the present embodiment, the steps of classifying and diagnosing diseases of patients include S4.2.1-S4.2.5:

[0133] S4.2.1, input the fusion feature representation of the patient to be diagnosed into the disease prediction model to obtain a disease classification probability distribution;

[0134] The 128-dimensional fusion feature representation obtained in step S4 is input into the trained disease prediction model for forward inference calculation. The disease prediction model uses an ensemble learning architecture and includes three parallel classifier branches, each branch processing different modal feature information. The model outputs a probability distribution of 10 common disease categories, including cardiovascular diseases, respiratory diseases, digestive diseases, nervous system diseases, and other major disease types. The probability distribution is normalized by a softmax function to ensure that the sum of all class probabilities is 1.0.

[0135] S4.2.2. Extracting the historical diagnosis data features of the patient according to the disease classification probability distribution, performing the first classification on the patient, and obtaining the preset disease categories of the patient;

[0136] Based on the results of the disease classification probability distribution, the historical diagnosis records of the patient within the past three years are automatically retrieved, and the historical diagnosis data features related to the current high-probability disease categories are extracted. The historical data features include key information such as past disease diagnosis results, treatment plan selection, drug use records, and abnormal indicators of examination results. Through the first classification screening by setting a probability threshold, when the prediction probability of a certain disease category exceeds 60%, the category is marked as a candidate disease category. If there are multiple candidate categories, the weight is ordered according to the historical diagnosis frequency and symptom matching degree, and the top three categories with the highest weight are selected as the preset disease categories.

[0137] S4.2.3. Obtaining the historical diagnosis data features of the patient within a preset time period, calculating the difference between the historical diagnosis data features and the current disease feature vector, and obtaining the historical difference value;

[0138] By obtaining the detailed historical diagnosis data of the patient within the past twelve months, including laboratory examination indicators, imaging examination results, and clinical symptom records, the historical diagnosis data is converted into a 128-dimensional standardized feature vector, and the difference between the current disease feature vector and the historical diagnosis data feature vector is calculated. The difference calculation uses the weighted Euclidean distance method, which gives different weights to different dimensions of features, with the image feature weight being 0.4, the text feature weight being 0.35, and the fusion feature weight being 0.25. Through the time decay function, the weight of the historical data at different time points is adjusted, with the recent data having a higher weight and the long-term data gradually decreasing, forming a historical difference vector reflecting the disease development trend.

[0139] S4.2.4. Dividing the historical difference value into different range intervals according to a preset level, and calculating the probability weight of the estimated disease in each range interval;

[0140] According to the distribution characteristics of the historical difference value, the difference value range is divided into five level intervals: significant deterioration interval (difference value > 2.0), mild deterioration interval (1.0 < difference value ≤ 2.0), stable interval (-0.5 ≤ difference value ≤ 1.0), improvement interval (-1.5 ≤ difference value < -0.5), and significant improvement interval (difference value < -1.5). Different probability weight coefficients are set for each interval, with the significant deterioration interval weight being 1.5, indicating an increased disease risk; the mild deterioration interval weight being 1.2; the stable interval weight being 1.0; the improvement interval weight being 0.8; and the significant improvement interval weight being 0.6. By statistically analyzing the distribution of each preset disease category in different intervals, the probability weight of each disease category in each interval is calculated, forming a disease-interval probability weight matrix.

[0141] S4.2.5, calculate the diagnosis probability of different diagnosis results according to the probability weight, perform second classification on the diagnosis results, and obtain the disease diagnosis result;

[0142] The disease classification probability distribution is weighted and fused with the probability weight matrix to obtain a corrected diagnosis probability. The calculation method of the corrected diagnosis probability is: multiplying the initial disease probability by the corresponding probability weight, and then performing normalization processing. Through setting a diagnosis threshold, second classification is performed, when the corrected diagnosis probability is greater than 80%, it is determined as a high-confidence diagnosis result; when the probability is between 60% and 80%, it is marked as a medium-confidence diagnosis result; when the probability is between 40% and 60%, it is marked as a low-confidence diagnosis result, and further examination is suggested; when the probability is less than 40%, it is marked as an uncertain diagnosis, which needs to be reviewed by a doctor. Finally, the complete disease diagnosis result including the diagnosis result, confidence level and recommended treatment plan is output.

[0143] It should be noted that the disease classification diagnosis process described above realizes dynamic disease risk assessment by fusing current disease characteristics and historical diagnosis information, avoiding the limitations of single time point diagnosis. Through the multi-level classification strategy and probability weight adjustment, the accuracy and reliability of the diagnosis are improved, and more comprehensive diagnosis reference information is provided for clinicians.

[0144] In an optional embodiment, when the patient is classified for the first time in step S4.2.2, the basic information characteristics of the patient, such as age, gender, family history, etc., can also be combined to adjust the disease probability by the Bayesian classification method, and the accuracy of the classification is improved.

[0145] In another optional embodiment, the grade division in step S3.2.4 can also use a dynamic threshold setting method, which adjusts the interval division standard according to the development law and clinical characteristics of different disease types, and realizes personalized risk assessment.

[0146] In the embodiments of the present application, in step S4, the step of generating a structured diagnosis report according to the diagnosis result includes S4.3.1-S4.3.5:

[0147] S4.3.1, match the preset medical diagnosis template according to the disease diagnosis result to obtain a basic diagnosis report framework;

[0148] By embedding ten standardized diagnosis report templates for major disease categories, including cardiovascular disease templates, respiratory system disease templates, digestive system disease templates, etc. Each template follows the international medical report standard format, including six main parts: patient basic information area, chief complaint and history of present illness area, examination result area, diagnosis conclusion area, treatment suggestion area and follow-up plan area. According to the disease diagnosis result obtained in step S4, then match the corresponding diagnosis template as the basic framework. When there are multiple diseases possible, by selecting the template corresponding to the disease category with the highest probability, and reserving an expansion area for multiple diagnosis results in the template, to ensure the completeness and professionalism of the report.

[0149] S4.3.2, retrieve pathological description information, treatment suggestion information and precautions information related to the disease diagnosis result from the medical knowledge base;

[0150] It is known that the medical knowledge base covers a full range of content such as etiology analysis, pathological mechanism, clinical manifestations, diagnostic criteria, treatment plan, prognosis evaluation, etc. According to the disease category and severity level of the diagnosis result, relevant medical information is accurately retrieved from the medical knowledge base. Pathological description information includes but is not limited to disease pathogenesis, histopathological changes, physiological function impact, etc. Treatment suggestion information covers drug treatment plan, surgical treatment selection, rehabilitation training plan, lifestyle adjustment, etc. Precautions information includes but is not limited to drug contraindications, dietary restrictions, exercise precautions, review time arrangement, etc. The retrieval process uses a semantic matching algorithm to ensure that the information obtained is highly relevant to the patient's specific condition.

[0151] S4.3.3, convert the image feature vector into image analysis description, and convert the text feature matrix into symptom summary description;

[0152] The image feature vector is converted into a structured image analysis description by a natural language generation model. The conversion process first identifies key abnormal indicators in the image features, such as lesion location, morphological characteristics, density changes, and boundary sharpness. Then, according to the medical imaging terminology standard, a professional image analysis report is generated, including technical quality evaluation, normal structure description, abnormal finding details, and imaging diagnosis opinion. The conversion process of the text feature matrix includes symptom keyword extraction, symptom severity assessment, symptom time series analysis, and symptom correlation judgment. Through medical terminology standardization processing, the patient's subjective symptom description is converted into an objective clinical symptom summary, including main symptoms, accompanying symptoms, symptom duration, symptom severity, etc.

[0153] S4.3.4, structure the patient's basic information, image analysis description, symptom summary description, disease diagnosis result and treatment suggestion information;

[0154] According to the requirements of the standard medical report format, various types of information are hierarchically structured and organized. The patient basic information area contains identification information such as name, gender, age, medical record number, and visit time; the image analysis description is arranged according to the examination site and method, and each examination result contains examination date, examination equipment, main findings, diagnosis opinion, etc.; the symptom summary description is organized according to categories, including circulatory system symptoms, respiratory system symptoms, digestive system symptoms, etc., and each category contains specific manifestations, duration, severity score, etc.; the disease diagnosis result is arranged according to the main diagnosis and secondary diagnosis, containing diagnosis name, diagnosis basis, confidence rating, etc.; the treatment suggestion information is sorted according to treatment priority, including immediate treatment measures, long-term treatment plan, and preventive health care suggestions.

[0155] S4.3.5, generating a structured diagnosis report containing multi-modal analysis results;

[0156] The structured medical information is finally integrated according to the standard medical report format to generate a complete diagnosis report. The report adopts a hierarchical information display method, including three main levels of executive summary, detailed analysis, and conclusion suggestion. The executive summary provides key information such as diagnosis points, main findings, and emergency level, allowing doctors to quickly understand the patient's condition; the detailed analysis section contains complete examination results, symptom analysis, diagnosis reasoning process, etc.; the conclusion suggestion section provides clear diagnosis conclusion, treatment plan selection, follow-up plan arrangement, etc. The report supports multiple output formats, including PDF document, XML structured data, HTML web format, etc., meeting the integration needs of different medical systems. At the same time, the report contains digital signature and timestamp information to ensure the security and traceability of the medical document.

[0157] It should be noted that the structured diagnosis report generation process integrates multi-modal medical data analysis results to realize the automatic conversion from raw medical data to standardized diagnosis report. The report content not only maintains medical professionalism and accuracy, but also has good readability and practicality, providing high-quality diagnosis decision support tools for clinicians.

[0158] In an alternative embodiment, when retrieving information from the medical knowledge base in step S4.3.2, the patient's personal characteristics such as age, gender, and medical history can also be combined to filter and adjust the weight of the information, ensuring the relevance and applicability of the treatment suggestions.

[0159] In another optional implementation, when the feature vector is converted into the description information in step S4.3.3, a multi-level information extraction method can also be used, including key information extraction, secondary information supplement, background information explanation, and different levels, according to different requirements of the report use scene, adjusting the information detail level and the professional term use level.

[0160] In a further implementation, the structured diagnostic report generated in step S4.3.5 can also contain a quality control module, through medical logic consistency check, diagnostic conclusion rationality verification, treatment suggestion safety evaluation, and other quality control processes, to ensure the medical accuracy and clinical safety of the report content.

[0161] In summary, through the multi-modal fusion model, the medical image data and the text data are deeply fused, the complementary advantages of different modal data are fully utilized, and the information missing problem that may exist in a single data source is avoided. The probability attention prediction model and the deep semantic analysis technology are used to deeply mine the semantic association and context relationship between medical terms, the correlation weight of the image features and the text features is calculated, the adaptive fusion of different modal features is realized, the disease diagnosis is more comprehensive and accurate, and the understanding depth and the diagnosis accuracy of the medical data are effectively improved.

[0162] In addition, the multi-classifier model is established and combined with the patient historical diagnosis data features, through multiple verifications and probability weight calculation, the stability and reliability of the diagnosis result are enhanced. At the same time, by analyzing the correlation between the patient historical diagnosis data features and the current disease features, the comprehensive judgment of the patient's disease development trend is realized, and the structured diagnostic report is automatically generated, the diagnosis output format is standardized, the influence of human factors on the diagnosis result expression is reduced, and the consistency and professionalism of the diagnosis report are improved, which provides a more comprehensive reference basis for clinical decision-making.

[0163] Embodiment 3, the above is a schematic scheme of a medical data processing method based on big data. It should be noted that the technical scheme of the medical data processing system based on big data and the technical scheme of the medical data processing method based on big data described above belong to the same concept. The technical scheme of the medical data processing system based on big data in this embodiment is not described in detail. The details can be referred to the description of the technical scheme of the medical data processing method based on big data.

[0164] The embodiment also provides a medical data processing system based on big data, which comprises:

[0165] The data acquisition module is configured to acquire medical image data and medical text data of a patient.

[0166] The multi-modal fusion module is configured to input the image feature vector and the text feature matrix into a multi-modal fusion model, calculate a correlation weight of the image feature and the text feature, and obtain a fusion feature representation.

[0167] The disease prediction module is configured to establish a disease prediction model according to the fusion feature representation.

[0168] The diagnosis module is configured to perform disease classification diagnosis on the patient by using the disease prediction model.

[0169] The report generation module is configured to generate a structured diagnosis report according to the diagnosis result.

[0170] The embodiment also provides an electronic device suitable for medical data processing based on big data, including a memory and a processor; the memory is configured to store computer executable instructions, and the processor is configured to execute the computer executable instructions to implement the medical data processing method based on big data.

[0171] The embodiment also provides a storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the medical data processing method based on big data.

[0172] The storage medium proposed in the embodiment and the medical data processing method based on big data proposed in the above embodiment belong to the same inventive concept, and the technical details not described in the embodiment can be referred to the above embodiment, and the embodiment and the above embodiment have the same beneficial effects.

[0173] Through the above description of the embodiments, those skilled in the art can clearly understand that the present application can be realized by means of software and necessary general hardware, and of course can also be realized by hardware. Based on such understanding, the technical solutions of the present application or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product can be stored in a computer readable storage medium, such as a floppy disk, a read-only memory (ROM), a random access memory (RAM), a FLASH memory, a hard disk or an optical disk, and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods of various embodiments of the present application.

[0174] It should be noted that the above examples are only used to illustrate the technical solutions of the present application but not to limit the present application. Although the present application is described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced, without departing from the spirit and scope of the technical solutions of the present application, and all of them should be covered in the scope of the claims of the present application.

Claims

1. A medical data processing method based on big data, characterized in that: Including steps: Obtain patients' medical imaging data and medical text data; Performing feature extraction on the medical image data to obtain an image feature vector, and analyzing the medical text data to obtain a text feature matrix; Inputting the image feature vector and the text feature matrix into a multimodal fusion model, calculating the association weights between the image features and the text features, and obtaining a fusion feature representation; A disease prediction model is established based on the fusion feature representation, disease classification diagnosis is performed on patients, and a structured diagnosis report is generated based on the diagnosis results.

2. The medical data processing method based on big data according to claim 1, characterized in that: The step of analyzing the medical text data includes: The medical text data is segmented and standardized, and the medical terms are converted into vector representations using word embedding technology to obtain the first string; Inputting the first character string into a position encoding module, adding position information and contextual relationships to each medical term, and generating a second character string containing time series features; Performing deep semantic analysis on the second character string to extract key features and semantic associations of the medical text, and outputting a semantically enhanced third character string; A probabilistic attention prediction model is constructed using the third character string, and the association probabilities between medical terms are calculated to form the text feature matrix.

3. The medical data processing method based on big data according to claim 2, characterized in that: The step of inputting the image feature vector and the text feature matrix into a multimodal fusion model comprises: Calculating association probabilities between medical terms according to the probabilistic attention prediction model to obtain a first probability matrix; Calculating semantic similarity scores between adjacent medical terms to obtain a second probability matrix; Calculating the spatial distance difference between medical terms through the semantic vector space to obtain a third probability matrix; Perform a weighted fusion calculation on the image feature vector, the first probability matrix, the second probability matrix, and the third probability matrix, and output the fused feature representation.

4. The medical data processing method based on big data according to claim 3, characterized in that: The steps of establishing a disease prediction model based on the fusion feature representation include: Inputting the fused feature representation into a feature extraction layer to extract comprehensive disease feature data of the patient; Calculating the patient's disease feature vector based on the comprehensive disease feature data and the patient's historical diagnostic data features; Establishing a multi-classifier model based on the disease feature vector, including a first classifier based on image features, a second classifier based on text features, and a third classifier based on fusion features; The three prediction results are weighted and fused to obtain the disease classification probability distribution. The disease prediction results are judged according to the disease classification probability distribution to complete the establishment of the disease prediction model.

5. The medical data processing method based on big data according to claim 4, characterized in that: The steps for classifying a patient's disease include: Inputting the fused feature representation of the patient to be diagnosed into the disease prediction model to obtain a disease classification probability distribution; Extracting historical diagnostic data features of the patient based on the disease classification probability distribution, performing a first classification on the patient, and obtaining a preset disease category of the patient; Obtain the patient's historical diagnostic data features within a preset time period, calculate the difference between the historical diagnostic data features and the current disease feature vector, and obtain the historical difference; Divide the historical difference into different ranges according to the preset levels, and calculate the probability weights of the estimated disease in different ranges respectively; The diagnostic probabilities of different diagnostic results are calculated according to the probability weights, and the diagnostic results are classified into the second category to obtain the disease diagnosis results.

6. The method for processing medical data based on big data according to claim 5, wherein: The steps to generate a structured diagnostic report based on the diagnostic results include: Matching the disease diagnosis results with a preset medical diagnosis template to obtain a basic diagnosis report framework; Retrieving pathological description information, treatment recommendation information, and precaution information related to the disease diagnosis result from a medical knowledge base; Converting the image feature vector into an image analysis description, and converting the text feature matrix into a symptom summary description; Structurally organize patient basic information, imaging analysis description, symptom summary description, disease diagnosis results and treatment recommendations; Generates a structured diagnostic report containing multimodal analysis results.

7. The medical data processing method based on big data according to claim 6, characterized in that: The steps for feature extraction of medical imaging data include: Standardizing the medical imaging data to unify imaging formats and resolution parameters; Perform deep semantic analysis on the standardized medical imaging data through the feature extraction layer to extract the spatial and texture features of the medical imaging data; Performing weighted fusion calculation on the spatial features and texture features, and generating a feature representation including spatial distance differences in combination with position information; The feature representation containing the spatial distance difference is input into the semantic vector space for processing, and the image feature vector is output.

8. A medical data processing system based on big data, applying the method according to any one of claims 1 to 7, characterized in that: include: A data acquisition module is used to acquire the patient's medical imaging data and medical text data; The multimodal fusion module is used to input the image feature vector and text feature matrix into the multimodal fusion model, calculate the association weights between the image features and the text features, and obtain the fusion feature representation; Disease prediction module, used to establish a disease prediction model based on fusion feature representation; The diagnosis module is used to classify and diagnose diseases of patients using disease prediction models; The report generation module is used to generate a structured diagnosis report based on the diagnosis results.

9. An electronic device comprising: memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the medical data processing method based on big data described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the medical data processing method based on big data as described in any one of claims 1 to 7.

Images