A multilingual medical information processing method and device

Abstract

The application discloses a multilingual medical information processing method and device. The scheme receives multilingual medical information of multiple modes such as text, voice, image, report and video, extracts feature vectors of each mode, aligns the feature space by means of a cross-modal attention mechanism, obtains a comprehensive medical feature vector, calls a multi-country medical specification knowledge base to generate a constraint mask matrix, carries out adaptive checking on the feature vector according to the matrix, controls data flow by means of a double-threshold decision logic combined with a three-level state machine, and adopts a SHA-256 hash chain to store evidence and trace the nodes in the whole process. The application effectively improves the processing accuracy of multilingual medical information, strictly follows different regional medical compliance rules, realizes the traceability of the whole process, and guarantees the safe and standardized processing of medical information.

Description

Technical Field

[0001] This application relates to the field of medical information technology, and in particular to a multilingual medical information processing method and apparatus. Background Technology

[0002] To achieve efficient cross-border medical information flow and enhance the cross-linguistic utilization value of medical data, standardized processing and precise conversion of multilingual medical information are necessary. For example, in cross-border medical collaboration and multi-center data sharing, the same medical concepts often have multiple language expressions, as well as differences in colloquial expressions across different regions. Failure to standardize and unify the processing of this multilingual medical information will lead to data fragmentation and information silos, severely impacting subsequent applications. For instance, in cross-border medical information exchange and multi-institutional collaboration, inconsistencies in medical information expression may cause misunderstandings, hindering accurate communication and information sharing. Furthermore, in medical data analysis, scattered and differentiated multilingual medical information makes subsequent statistical, mining, and correlation analysis difficult, significantly limiting the depth and breadth of medical data integration and utilization. In one embodiment, when medical information systems are interconnected, the mismatch of multilingual medical information may become an obstacle to data exchange and integration.

[0003] However, existing multilingual medical information processing technologies generally suffer from numerous technical bottlenecks. Most existing solutions employ general machine translation models, supporting only direct translation into everyday language. They lack a high-dimensional aligned mapping corpus for medical terminology, resulting in significant translation errors and ambiguities in specialized medical terms such as disease names, examination items, drug names, and medical procedures. For example, when faced with users using colloquial or ambiguous medical expressions with regional characteristics, general translation models often resort to mechanical literal translation, preventing recipients from obtaining standard professional medical judgments. Most commercially available medical translation products and related technologies are limited to bidirectional conversion between plain text or real-time speech, failing to support the simultaneous recognition, cross-modal parsing, and bilingual conversion of multimodal unstructured medical data such as symptom images, examination and test reports, and vital sign videos. This leads to severe gaps and loss of critical medical information when it flows between different modalities, resulting in low efficiency in cross-border medical data transfer. Furthermore, the translation function and subsequent intelligent verification function of traditional medical information systems are completely disconnected. The translated dialogue data stream cannot be directly converted or seamlessly input into the subsequent intelligent verification model, and still heavily relies on manual review and secondary transcription.

[0004] Therefore, we hope to find a method that can accurately convert multilingual medical information, uniformly analyze multimodal medical data, and verify the compliance of medical information, so as to achieve more efficient and accurate multilingual medical information processing. Summary of the Invention

[0005] The purpose of this application is to provide a multilingual medical information processing method and apparatus that can improve the accuracy of multilingual medical information processing, ensure compliance with medical standards, and enhance data traceability.

[0006] To achieve the above objectives, this application provides the following technical solution: a multilingual medical information processing method, comprising: It can receive multilingual medical information input, which includes at least one of text information, voice information, image information, report information, and video information; Multimodal feature extraction is performed on multilingual medical information input to obtain multiple feature vectors corresponding to different modalities; Multiple feature vectors are aligned in feature space using a cross-modal attention mechanism to obtain an aligned comprehensive medical feature vector. The medical standard constraint mask matrix was obtained from a multi-national medical standard knowledge base. The medical standard constraint mask matrix contains the medical information processing compliance rules of different countries and regions. Adaptive verification processing is performed on the comprehensive medical feature vector based on the medical standard constraint mask matrix to obtain the verified medical information processing result. The medical information processing results are controlled by a three-level state machine based on the dual threshold judgment logic. The entire process of multilingual medical information processing is traceable and verifiable through SHA-256 hash chains.

[0007] Optionally, receiving and processing multilingual medical information input includes: For the input speech information, extract the Mel-frequency cepstral coefficients (MFCC) features of the speech information; The Mel-Cepstral Coefficient (MFCC) features are input into an end-to-end dialect adaptive recognition model with an attention mechanism for speech recognition. Multilingual translation is performed using a translation adaptive algorithm based on the LoRA incremental fine-tuning architecture. The translation adaptive algorithm is calculated using a translation probability model, which is as follows: ; Automatic correction of colloquial medical expressions is achieved using a medical synonym map.

[0008] Optionally, multimodal feature extraction of multilingual medical information input includes: For image and report information, a deformable convolutional neural network (DCN) is used for feature extraction. For video information, a three-dimensional convolutional neural network (3D-CNN) is used to extract features by sampling dynamic keyframes. For text information and speech recognition results, word embedding is used for feature vectorization.

[0009] Optionally, feature space alignment of multiple feature vectors via a cross-modal attention mechanism includes: Text feature vectors, visual feature vectors, and structured feature vectors are projected into the same multidimensional medical feature semantic space; Cross-modal feature fusion is performed using a multi-head attention mechanism, and the feature alignment calculation formula is as follows: ; Obtain aligned comprehensive medical feature vectors after eliminating dimensional differences in heterogeneous data.

[0010] Optionally, the medical standard constraint mask matrix can be obtained from a multi-national medical standard knowledge base, including: Construct a knowledge base that includes medical information processing standards, clinical drug use rules, and drug indications and contraindications rules from multiple countries; Supports automatic incremental updates of the knowledge base in the cloud; Based on the target country or region for medical information processing, a corresponding medical standard constraint mask matrix is ​​dynamically generated. .

[0011] Optionally, adaptive verification processing of the comprehensive medical feature vector based on the medical standard constraint mask matrix includes: In the multi-head self-attention calculation process, a medical norm constraint mask matrix is ​​introduced. Perform constraint correction; Attention is calculated using a masked constraint formula, which is as follows: ; When the results of medical information processing do not conform to the medical standards of the corresponding country or region, the medical standard constraint mask matrix will be applied. The corresponding element is set to 0. When the medical information processing result conforms to medical standards, the corresponding element is set to 1.

[0012] Optionally, the three-level state machine control flow of the medical information processing results based on the dual threshold judgment logic includes: Calculate the maximum confidence level of risk conflict in the medical information processing results, max(P). Set a first security threshold θ1 and a second security threshold θ2; Based on the comparison results between the maximum confidence value of risk conflict max(P) and the first safety threshold θ1 and the second safety threshold θ2, different state transition branches are triggered.

[0013] Optionally, different state transition branches can be triggered, including: When the maximum risk conflict confidence level max(P) is less than the first safety threshold θ1, execute the low-risk release branch and automatically release the flow; When the maximum risk conflict confidence value max(P) is greater than or equal to the first safety threshold θ1 and less than the second safety threshold θ2, the risk branch is suspended during execution, the flow handle is suspended, and a risk warning pops up. When the maximum risk conflict confidence value max(P) is greater than or equal to the second security threshold θ2, execute the high-risk interception branch, lock the flow handle, and block the delivery.

[0014] Optionally, executing high-risk interception branches also includes: Automatically recommend alternative medical information processing solutions; Forced association of a dual instruction trigger between the first processing end confirmation instruction and the second review end asynchronous electronic signature instruction.

[0015] Optionally, evidence storage and traceability of all nodes in the multilingual medical information processing process can be achieved through SHA-256 hash chains, including: The initial input data, translation vector, verification results, editing trajectory, and review signature are treated as nodes in the entire lifecycle. For each flow node, a unique irreversible hash digest is generated by combining the SHA-256 algorithm with the current timestamp. The formula for calculating the hash digest is: .

[0016] Optionally, based on clinical outcome feedback data, the DPO algorithm is used to perform timed incremental fine-tuning of the underlying translation model and validation model; Achieve continuous self-optimization of the multilingual medical information processing model.

[0017] Optionally, establish a real-time bilingual interactive and collaborative channel between doctors and patients; Supports two-way real-time conversion of multilingual medical information; Enables synchronous interaction and collaborative editing during medical information processing.

[0018] This application also proposes a multilingual medical information processing device, comprising: A medical-specific multilingual dialect translation module is configured to receive multilingual medical information input and perform speech recognition and multilingual conversion processing; The multimodal medical information parsing module is configured to extract multimodal features from multilingual medical information inputs and align the feature space through a cross-modal attention mechanism to obtain a comprehensive medical feature vector. The multinational medical standards knowledge base module is configured to store and dynamically output a medical standards constraint mask matrix; The AI ​​adaptive medical information verification module is configured to perform masked attention calculation and adaptive verification processing on the comprehensive medical feature vector based on the medical standard constraint mask matrix. The real-time bilingual interactive collaboration module for doctors and patients is configured to execute a three-level state machine control flow based on dual threshold judgment logic to achieve handle locking and interactive collaboration. The medical information storage and traceability and model self-optimization module is configured to perform full-process node storage and traceability through SHA-256 hash chain, and to use the DPO algorithm for incremental fine-tuning and self-optimization of the model.

[0019] Compared to existing technologies, this application provides a multilingual medical information processing method and apparatus, which receives multilingual medical information input, including at least one of text, voice, image, report, and video information; performs multimodal feature extraction on the multilingual medical information input to obtain multiple feature vectors corresponding to different modalities; aligns the multiple feature vectors in feature space through a cross-modal attention mechanism to obtain an aligned comprehensive medical feature vector; and obtains a medical standard constraint mask matrix from a multinational medical standard knowledge base, the medical standard constraint mask matrix containing medical information processing data from different countries and regions. The system incorporates compliance rules; adaptive verification of comprehensive medical feature vectors based on medical standard constraint mask matrix to obtain verified medical information processing results; a three-level state machine control flow for the medical information processing results based on dual threshold judgment logic; and evidence storage and traceability of all nodes in the multilingual medical information processing process through SHA-256 hash chain. By using multimodal feature extraction and cross-modal alignment, medical standard constraint verification, and dual threshold judgment mechanism, the system addresses terminology ambiguity and modal fragmentation issues in multilingual medical information processing, thereby improving the accuracy of multilingual medical information processing, ensuring medical standard compliance, and enhancing data traceability. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic flowchart of a multilingual medical information processing method provided in an embodiment of this application.

[0022] Figure 2 This is a schematic diagram of a multilingual medical information processing device provided in an embodiment of this application. Detailed Implementation

[0023] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0024] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0025] like Figure 1 As shown, this application provides a multilingual medical information processing method. First, it receives multilingual medical information input, which may include at least one of text, voice, image, report, and video information. Information input can be performed by the user through manual file upload, pasting text, or real-time recording via microphone and camera. The system can be configured with different interfaces to receive these different modalities of data; for example, a text box for receiving text, a file upload interface for receiving image, report, and video files, and an audio input interface for receiving voice.

[0026] In one embodiment, multimodal feature extraction is performed on the multilingual medical information input to obtain multiple feature vectors corresponding to different modalities. For the received multilingual medical information input, traditional feature extraction methods for each modality can be employed. For example, for text information, the bag-of-words model or TF-IDF algorithm can be used to extract features; for speech information, acoustic features such as Linear Predictive Coding (LPC) or zero-crossing rate can be used; for image information, local feature descriptors such as SIFT or HOG can be used; and for video information, motion features can be extracted using inter-frame differencing or optical flow methods. These methods aim to convert the raw data into a machine-processable numerical form, i.e., feature vectors.

[0027] Subsequently, a cross-modal attention mechanism is used to align the feature vectors in their feature spaces, resulting in an aligned comprehensive medical feature vector. After obtaining feature vectors from different modalities, a general attention mechanism can be employed for feature space alignment. For example, a weight can be assigned to each modality's feature vector, and they can be fused using a weighted summation. Alternatively, a simple fully connected network can be designed to take all modality feature vectors as input and output a unified comprehensive feature vector. The purpose of this alignment is to eliminate the differences in representation between different modalities, enabling subsequent processing within a unified semantic space.

[0028] Next, a medical compliance constraint mask matrix is ​​obtained from a multinational medical compliance knowledge base. This matrix contains compliance rules for medical information processing from different countries and regions. Multinational medical compliance rules can be stored using a pre-built database or file system. When processing medical information from a specific country or region, the system can retrieve the corresponding medical compliance rules from this storage based on user selection or the information's origin. These rules are then manually or through a simple mapping logic converted into a matrix form—the medical compliance constraint mask matrix—where each element represents the applicability or importance of a particular rule.

[0029] Based on this, an adaptive verification process is performed on the comprehensive medical feature vector using the medical standard constraint mask matrix to obtain the verified medical information processing result. After obtaining the comprehensive medical feature vector, it can be subjected to simple logical operations or multiplication operations with the medical standard constraint mask matrix. For example, if an element of the mask matrix indicates that a piece of medical information does not comply with the standard, the corresponding part in the comprehensive feature vector can be marked or adjusted. The verification process can be a simple rule matching process, comparing the key information in the feature vector with the compliance rules in the mask matrix to identify potential conflicts or non-compliance and generate preliminary verification results.

[0030] In one embodiment, the medical information processing result is controlled by a three-level state machine based on dual-threshold judgment logic. The verified medical information processing result can be assigned a risk score or confidence value. The system can preset two fixed thresholds, such as a lower threshold and a higher threshold. By comparing the risk score with these two thresholds, the processing result can be divided into three levels: low risk, medium risk, and high risk. Each level corresponds to a predefined state, such as low risk (directly approved), medium risk (requiring manual review), and high risk (directly rejected).

[0031] Finally, a SHA-256 hash chain is used to preserve and trace the entire process of multilingual medical information processing. At each key stage of medical information processing, such as data reception, feature extraction, and verification completion, the data or its summary for the current stage can be recorded. These records can be sequentially hashed using the SHA-256 algorithm, with the previous hash value used as one of the inputs for the current hash calculation, thus forming a simple hash chain. This hash chain can be stored in a log file or database to provide basic evidence preservation and traceability capabilities.

[0032] In some possible embodiments, specific methods for receiving and processing multilingual medical information input are proposed, with particular optimization for speech information input. First, for the speech information input, Mel-frequency cepstral coefficients (MFCC) features are extracted. MFCC is a feature parameter widely used in speech recognition and processing. By simulating the human ear's perception of sound, it can effectively capture the spectral envelope information of the speech signal, while also exhibiting robustness to noise and speaker differences. The purpose of extracting MFCC features is to convert the original speech waveform into a more compact and recognizable digital representation, enabling efficient processing by subsequent machine learning models. This is typically achieved through steps such as pre-emphasis, framing, windowing, Fast Fourier Transform, Mel filter bank energy calculation, and Discrete Cosine Transform on the speech signal.

[0033] Subsequently, the Mel-Cepstral Coefficient (MFCC) features are input into an end-to-end dialect-adaptive recognition model with an attention mechanism for speech recognition. Speech recognition is the process of converting human speech into text. In the medical field, due to the existence of multiple languages ​​and dialects, and the highly specialized nature of medical terminology, traditional speech recognition models are insufficient. The end-to-end dialect-adaptive recognition model with an attention mechanism can directly learn the speech-to-text mapping from MFCC features and focus on the most relevant part of the input sequence to the current output through the attention mechanism, thereby improving recognition accuracy. The dialect adaptation capability enables it to handle medical speech inputs from different regions and accents, while the end-to-end design simplifies the model structure and reduces errors that may be introduced by intermediate steps. This model is typically based on deep neural network architectures, such as recurrent neural networks, long short-term memory networks, or Transformer models, and achieves dialect adaptation by introducing multi-dialect datasets or transfer learning techniques.

[0034] In one embodiment, multilingual conversion is performed using a translation adaptive algorithm based on the LoRA incremental fine-tuning architecture. The translation adaptive algorithm is calculated using a translation probability model, which is as follows: Multilingual translation involves translating text from one language into another, a process that demands extremely high accuracy and expertise in the medical field. A translation adaptation algorithm based on the LoRA (Low-Rank Adaptation) incremental fine-tuning architecture aims to efficiently and cost-effectively fine-tune a large pre-trained translation model to adapt it to specific medical domains and language pairs. LoRA updates model weights by injecting low-rank matrices into specific layers of the pre-trained model, thus achieving rapid adaptation to new tasks without modifying most of the original model's parameters. This method effectively handles the specialized vocabulary, sentence structures, and contexts unique to the medical field, improving translation accuracy and fluency. (Translation probabilistic model) Used to calculate text in a given source language At that time, target language text The probability of, where The function is used to convert the output into a probability distribution. The activation function introduces nonlinearity. and It is a weight matrix. and It may represent some kind of context or hidden state.

[0035] Finally, a medical synonym graph is used to automatically correct colloquial medical expressions. In actual doctor-patient communication, patients or doctors may use non-standard, colloquial expressions to describe their condition or diagnosis, which may lead to misunderstandings. A medical synonym graph is a knowledge graph containing professional medical terms and the relationships between their various colloquial, non-standard, or synonymous expressions. Using this graph for automatic correction can identify and replace colloquial expressions in speech recognition or translation results, standardizing them into professional medical terms, thereby improving the accuracy and standardization of medical information processing. This is typically done by constructing or acquiring a comprehensive medical synonym graph, and after the speech recognition or translation results are generated, performing lexical analysis and entity recognition on the text to identify any colloquial or non-standard medical expressions, and then querying the graph for mapping and replacement.

[0036] In some possible embodiments, specific implementation methods for multimodal feature extraction of multilingual medical information input are proposed to ensure that the features of different modal data can be captured efficiently and accurately.

[0037] Specifically, when performing multimodal feature extraction on multilingual medical information input, a deformable convolutional neural network (DCN) is used for feature extraction of image and report information. DCN is an advanced convolutional neural network whose core lies in introducing learnable offsets, allowing the sampling position of the convolutional kernel to be adaptively adjusted. This adaptive sampling capability enables DCN to better adapt to the irregular shapes, scales, and poses of lesions in medical images, as well as the complexity of unstructured or semi-structured layouts in medical reports, thus more accurately capturing key features in images and reports. For example, when analyzing medical images, DCN can accurately locate and extract features of lesion areas, even those with highly irregular shapes. For report information, DCN can treat it as a special type of image data, learning the spatial relationships between text, charts, and layouts to extract more discriminative feature vectors.

[0038] Simultaneously, for video information, a 3D convolutional neural network (3D-CNN) is employed for feature extraction based on dynamic keyframe sampling. The 3D-CNN can perform convolution operations simultaneously in both spatial and temporal dimensions, effectively capturing temporal information and motion features in medical videos (such as endoscopic examinations and surgical recordings). Combined with dynamic keyframe sampling technology, the system can intelligently identify and select the most representative and informative frames from the video for processing, rather than processing all frames uniformly. This approach not only preserves the core dynamic information of the video, such as the dynamic changes in lesion areas or organ movement patterns, but also significantly reduces computational complexity and improves processing efficiency.

[0039] Furthermore, word embedding is employed for feature vectorization processing of both text information and speech recognition results. Word embedding is a technique that maps discrete words or phrases to a low-dimensional continuous vector space, making semantically similar words closer together in the vector space. In this way, text information such as medical records and diagnostic descriptions, as well as the text results converted by the speech recognition model, can be transformed into semantically rich vector representations, thereby capturing the semantic information and contextual relationships of words. This greatly enhances the model's ability to understand medical terminology and complex medical contexts, providing high-quality semantic features for subsequent medical information processing.

[0040] In some possible embodiments, it is proposed to project text feature vectors, visual feature vectors, and structured feature vectors onto the same multidimensional medical feature semantic space. Text feature vectors refer to vectors extracted from text information and speech recognition results through methods such as word embedding, which can represent textual semantic information. For example, pre-trained models such as Word2Vec, GloVe, and BERT can be used to map words or sentences to a high-dimensional vector space. Visual feature vectors refer to vectors extracted from image and video information through models such as deformable convolutional neural networks (DCN) or three-dimensional convolutional neural networks (3D-CNN), which can represent visual content (such as lesions, imaging manifestations). Structured feature vectors refer to features with specific structure and semantics extracted from report information, such as basic patient information, diagnostic results, and treatment plans. This information usually exists in key-value pairs or tabular form and is converted into vector representations through encoding. Projecting these feature vectors from different modalities onto the same multidimensional medical feature semantic space means transforming them into a common high-dimensional space with a unified semantic interpretation through linear transformations or nonlinear mappings. The design of this space aims to bring feature vectors expressing the same or similar medical concepts in different modalities closer to each other in the space, thereby eliminating heterogeneity between modalities and laying the foundation for subsequent feature fusion.

[0041] Building upon this, cross-modal feature fusion is achieved through a multi-head attention mechanism. Multi-head attention is a mechanism that allows a model to collectively focus on information from different locations across different representation subspaces. It operates multiple attention heads in parallel, each independently learning linear projections of the query (Q), key (K), and value (V) and calculating attention weights. The outputs of all heads are then concatenated and subjected to a linear transformation. This mechanism captures the complex relationships between features from different modalities and weights and aggregates information from multiple perspectives, thus achieving a more comprehensive and robust fusion of cross-modal information. The feature alignment calculation formula is as follows: .

[0042] in, This represents the comprehensive medical feature vector obtained after alignment and fusion. It is a layer normalization operation used to stabilize the training process and accelerate convergence. It normalizes the features of each sample to make its mean 0 and variance 1, thereby reducing the internal covariate bias. It is a text feature vector. It is a multi-head attention mechanism. The query vector represents a vector composed of text feature vectors. After linear transformation matrix "Get" is used to indicate the current focus of attention. The key vector is represented by the visual feature vector. After linear transformation matrix This is obtained and used to match the query vector to determine attention weights. The value vector is represented by structured feature vectors. After linear transformation matrix The result is the information content ultimately weighted and aggregated by the attention mechanism. This formula indicates that the text feature vector... As a query, focus on visual feature vectors and structured feature vectors The provided information, through a multi-head attention mechanism, allows the model to learn the correlations between textual, visual, and structured information. Based on these correlations, the model weights the visual and structured information and then fuses the weighted information with the textual information. Ultimately, this results in an aligned, comprehensive medical feature vector that eliminates dimensional differences between heterogeneous data.

[0043] In some possible embodiments, specific methods for obtaining medical standard constraint mask matrices from multi-national medical standard knowledge bases are proposed. These methods include constructing a knowledge base containing multi-national medical information processing standards, clinical drug use rules, and drug indications and contraindications rules; supporting automatic incremental updates of the knowledge base in the cloud; and dynamically generating corresponding medical standard constraint mask matrices based on the target country or region of the medical information processing. .

[0044] Specifically, a knowledge base is constructed that includes multinational medical information processing standards, clinical drug use rules, and drug indications and contraindications rules. The aim is to establish a centralized, structured information storage system for aggregating and managing global medical compliance information. This knowledge base can be constructed using ontology, semantic networks, or relational databases, and its content covers, but is not limited to, national drug approval standards, treatment guidelines, medical record writing standards, medical device usage regulations, clinical trial ethical requirements, treatment pathways for specific diseases, drug interactions, dosage limits, and patient privacy protection regulations (such as GDPR and HIPAA). These rules are encoded into a machine-readable format, for example, represented through a rule engine or logic programming language, providing a comprehensive and authoritative basis for subsequent medical information verification.

[0045] Supporting automatic incremental updates of the knowledge base in the cloud ensures its real-time nature, accuracy, and completeness, addressing frequent changes in medical guidelines. This can be achieved in several ways. For example, the system can interface with official databases or APIs of national medical regulatory agencies, drug administrations, and professional medical associations to regularly retrieve the latest regulations, guidelines, and drug information. Simultaneously, intelligent web crawlers can be deployed to periodically scan relevant authoritative websites, identifying and extracting updated content. For complex or ambiguous rules, expert review and annotation can be combined to ensure the accuracy of updated content. Furthermore, a version control system manages each update of the knowledge base, supporting retrospective analysis and auditing. An incremental synchronization mechanism synchronizes only the changed parts, rather than a full update, to improve efficiency and reduce resource consumption.

[0046] Based on the target country or region for medical information processing, a corresponding medical standard constraint mask matrix is ​​dynamically generated. This process transforms abstract medical guidelines into a computable mathematical form, enabling constraints and guidance during subsequent feature validation. First, the system identifies the target country or region based on user input, system configuration, or metadata about the medical information itself (such as patient nationality and location of treatment). Then, it retrieves relevant medical guidelines and medication rules from a knowledge base. Finally, the retrieved rules are mapped onto various dimensions of a comprehensive medical feature vector, generating a medical guidelines constraint mask matrix. For example, if a certain feature dimension (such as the use of a certain drug) is prohibited in the target country, then the mask matrix... The element at the corresponding position may be set to 0; if a condition is met, it is set to 1. This matrix is ​​generated in real time, based on the target country or region and the latest state of the knowledge base for each processing request.

[0047] In some possible embodiments, a medical norm constraint mask matrix is ​​proposed. The specific methods for adaptively verifying comprehensive medical feature vectors include introducing a medical norm constraint mask matrix during the multi-head self-attention calculation process. Constraint corrections are performed; attention calculations are performed using a masked constraint formula, which is as follows: ; and when the results of medical information processing do not conform to the medical standards of the corresponding country or region, the medical standard constraint mask matrix will be applied. The corresponding element is set to 0. When the medical information processing result conforms to medical standards, the corresponding element is set to 1.

[0048] Specifically, this embodiment employs a multi-head self-attention mechanism when performing adaptive verification processing on the aligned comprehensive medical feature vectors. This mechanism allows the model to simultaneously focus on information from different locations while processing sequence data, and to extract features from multiple different "angles" or "representation subspaces." In this process, a medical specification constraint mask matrix is ​​introduced. Its function is to impose constraints and corrections on the computational process of the self-attention mechanism. This means It doesn't simply act as a post-processing filter, but directly participates in deep feature interactions and weight allocation, thereby guiding the model's behavior internally and making it tend to generate results that conform to the specifications. The formula for calculating attention with mask constraints is: In this formula, These represent the query, key, and value matrices, respectively. This is the scaling factor. The key is that, when calculating the attention weights, the query matrix is ​​first... AND key matrix Multiplying by the transpose and scaling yields the original attention scores. These original attention scores are then multiplied by the medical norm constraint mask matrix. Perform element-wise multiplication ( This multiplication operation makes It can directly modify the original attention score, that is, if If any element in the algorithm is 0, the corresponding attention score will be set to 0, thus preventing the model from paying attention to or utilizing non-standard information; if If any element in the matrix is ​​1, the corresponding attention score remains unchanged. Finally, the attention score after mask correction is applied. The function is normalized and compared with the value matrix. Multiplying these results yields the final attention output. (Medical regulation constraint mask matrix) The element values ​​are dynamically set based on whether the medical information processing results conform to the medical standards of the corresponding country or region. Specifically, when a certain aspect or feature of the medical information processing results does not conform to the preset medical standards, the medical standard constraint mask matrix... The corresponding element will be set to 0. This is equivalent to "closing" the attention channel for that non-compliant feature in the attention mechanism. Conversely, when a certain aspect of the medical information processing result complies with medical standards, the corresponding element will be set to 1, allowing the attention mechanism to process that feature normally. This binary setting provides the attention mechanism with a clear and direct compliance signal.

[0049] In some possible embodiments, multilingual medical information can be processed through multimodal feature extraction, cross-modal feature alignment, and adaptive verification based on a medical standard constraint mask matrix to obtain preliminary medical information processing results. However, in practical applications, due to the complexity and variability of medical information and the differences in standards across different countries and regions, a single verification process is insufficient to fully guarantee the reliability and security of the processing results. If potential risks exist in the processing results and are not effectively identified and classified, it may lead to serious medical accidents or compliance issues.

[0050] To address the aforementioned issues, this embodiment proposes a three-level state machine control flow for medical information processing results based on dual-threshold judgment logic. Specifically, this flow process includes: calculating the maximum risk conflict confidence value max(P) of the medical information processing results; setting a first safety threshold θ1 and a second safety threshold θ2; and triggering different state transition branches based on the comparison results between the maximum risk conflict confidence value max(P) and the first safety threshold θ1 and the second safety threshold θ2.

[0051] The calculation of the maximum confidence score for risk conflict in medical information processing results, max(P), refers to the risk assessment of the medical information processing results after adaptive verification, quantifying potential conflicts, inconsistencies, or non-compliance. This maximum value, max(P), measures the system's "confidence" in the potential risks of a given processing result. This can be achieved through various machine learning or statistical methods. For example, during the verification process, if the system identifies multiple potential risk points or uncertainties, each factor is assigned a confidence score, and max(P) is the highest confidence score among these risk points. Specifically, a risk assessment module can be designed. This module receives the verified medical information processing results and matches them against a pre-defined set of risk rules (e.g., drug interactions, contraindications, dosage exceeding limits, diagnostic conflicts, etc.). For each matched risk rule, its confidence score is calculated, and the maximum value among all risk confidence scores is taken as max(P).

[0052] Setting a first safety threshold θ1 and a second safety threshold θ2 are two pre-defined key values ​​used to divide the risk conflict confidence level max(P) into different risk level ranges, thereby guiding the subsequent three-level state machine control flow. These two thresholds are the core of the system's risk management strategy, directly determining how the system handles different risk levels. These two thresholds are usually determined through consultation among medical experts, compliance experts, and system engineers, and can be dynamically adjusted based on actual operational data and feedback. For example, they can be determined through historical medical accident data, expert experience, simulation tests, etc. The first safety threshold θ1 can be set as a lower risk tolerance, indicating that the processing result below this value is considered relatively safe; the second safety threshold θ2 is set as a higher risk warning line, indicating that the processing result above this value has a serious risk. These thresholds can be stored in the system's configuration database and allow administrators to configure them flexibly.

[0053] The core logic for implementing the three-level state machine control flow is to trigger different state transition branches based on the comparison result of the maximum risk conflict confidence value max(P) with the first safety threshold θ1 and the second safety threshold θ2. By comparing the calculated maximum risk conflict confidence value max(P) with the preset first safety threshold θ1 and second safety threshold θ2, the system can intelligently determine the risk level of the current medical information processing result and select the corresponding processing path accordingly, ensuring that medical information of different risk levels is processed appropriately and safely. This comparison logic can be implemented using the if-elseif-else structure in a programming language, or by explicitly defining the transition conditions and triggering actions between different states using the state machine design pattern. Each branch corresponds to a predefined processing flow, such as automatic release, manual review, and forced interception, but the specific transition actions will be further elaborated in subsequent implementations.

[0054] In some possible embodiments, triggering different state transition branches is proposed as follows: when the maximum risk conflict confidence value max(P) is less than the first security threshold θ1, a low-risk release branch is executed, and the transition is automatically allowed; when the maximum risk conflict confidence value max(P) is greater than or equal to the first security threshold θ1 and less than the second security threshold θ2, a medium-risk suspension branch is executed, the transition handle is suspended, and a risk warning is displayed; and when the maximum risk conflict confidence value max(P) is greater than or equal to the second security threshold θ2, a high-risk interception branch is executed, the transition handle is locked, and the transmission is blocked.

[0055] Specifically, when the maximum risk conflict confidence value max(P) of the medical information processing result calculated by the system is less than the preset first safety threshold θ1, it indicates that the risk of the processing result is extremely low and meets the safety standards. At this time, the system will execute the low-risk release branch, which means that the medical information processing result will be automatically released and processed without any manual intervention, directly entering the subsequent processing stage or being sent to the target system. This automatic release mechanism can be achieved by updating the data status, triggering the next automated task, or moving the data into the processed queue to ensure efficient and seamless flow.

[0056] When the maximum confidence level of the risk conflict, max(P), is greater than or equal to the first safety threshold θ1, but less than the second safety threshold θ2, the system determines that the medical information processing result has a medium risk. In this case, the system will execute a medium-risk suspension branch, specifically by suspending the workflow handle, i.e., pausing the automated progress of the current processing flow. Simultaneously, the system will display a risk warning, notifying relevant medical staff or reviewers of the medium risk information, prompting them to manually review the medical information. The workflow handle can be a unique identifier associated with the processing task, and its status is set to "suspended" or "pending review," awaiting manual confirmation or correction.

[0057] In one embodiment, when the maximum risk conflict confidence value max(P) reaches or exceeds the second security threshold θ2, the system determines that the medical information processing result is high-risk, potentially indicating serious errors or non-compliance. At this point, the system will execute a high-risk interception branch, taking the most stringent control measures, namely locking the flow handle and blocking its transmission. Locking the flow handle means that the identifier of the processing task will be permanently locked or discarded, preventing any further automated or manual processing. Blocking its transmission ensures that the high-risk medical information processing result will not be transmitted to downstream systems or end users, thereby effectively preventing potential medical accidents, compliance issues, or harm to patients.

[0058] In some possible embodiments, the steps for executing the high-risk interception branch also include: automatically recommending alternative medical information processing solutions; and a dual instruction trigger that forcibly associates the confirmation instruction from the first processing end with the asynchronous electronic signature instruction from the second review end.

[0059] Specifically, when the system identifies and blocks high-risk medical information processing results, this embodiment can automatically generate and recommend a series of alternative processing solutions that comply with medical standards to avoid process stagnation. These alternative solutions can be intelligently matched and generated based on compliance rules, clinical medication rules, and drug indications and contraindications rules stored in multinational medical standard knowledge bases, combined with historical processing data and successful cases. For example, if the original processing solution triggers high-risk blocking due to a drug contraindication, the system can recommend other alternative drug solutions with the same efficacy but no contraindications, or suggest adjusting the diagnostic description to comply with the coding standards of a specific region. This aims to provide users with immediate and feasible solutions, reducing the complexity and time cost of manual intervention.

[0060] To ensure the rigor and traceability of the decision-making process for high-risk medical information processing, this embodiment introduces a dual-instruction triggering mechanism. The first processing-end confirmation instruction is typically issued by the user or system operator who initially submitted or processed the medical information, indicating their understanding of the risks and preliminary confirmation of the recommended alternatives or further processing suggestions. The second review-end asynchronous electronic signature instruction is issued by reviewers with higher authority or professional qualifications (e.g., senior physicians, compliance experts) after independent review, providing final approval of the processing result or alternative through electronic signature. "Asynchronous" here means that the two instructions can be issued at different times, allowing reviewers ample time for independent evaluation. This dual-instruction trigger ensures shared responsibility and non-repudiation of key decisions, significantly enhancing the security, compliance, and auditability of the processing.

[0061] In some possible embodiments, a method for evidence storage and traceability of the entire process of multilingual medical information processing is proposed using a SHA-256 hash chain. Specifically, the initial input data, translation vectors, verification processing results, editing trajectory, and review signature are considered as lifecycle nodes; for each node, a unique irreversible hash digest is generated using the SHA-256 algorithm combined with the current timestamp; and the formula for calculating the hash digest is as follows: .

[0062] In this context, the "full lifecycle flow nodes" refer to the collection of key data states and operational events during the multilingual medical information processing. Specifically, initial input data refers to the raw multilingual medical information received by the system, such as patient voice descriptions, text medical records, and medical images; translation vectors refer to the intermediate representations after language model processing, which may include conversion results between different languages; verification processing results refer to the information after adaptive verification using a medical standard constraint mask matrix, serving as a crucial basis for the system to determine compliance; editing traces refer to records of manual or system modifications and additions to medical information; and review signatures refer to electronic credentials for manual or automatic system confirmation of the processing results. Defining these key points as flow nodes aims to comprehensively capture the complete evolution of medical information from input to final processing results, providing foundational data for subsequent traceability.

[0063] The SHA-256 algorithm combines the current timestamp with a rolling hash digest to generate unique, irreversible hash values. SHA-256 (Secure Hash Algorithm 256) is a widely used cryptographic hash function whose core characteristic is its ability to map data of arbitrary length to a fixed-length (256-bit) hash value. This algorithm possesses one-wayness (it's difficult to deduce the original data from the hash value) and collision resistance (it's difficult to find two different inputs that produce the same hash value), making it ideal for data integrity verification. Combining it with the current timestamp ensures that even with the same input data, the hash values ​​generated at different times will be different, further enhancing the uniqueness of the hash value. The rolling generation mechanism means that each node's hash digest contains not only the current node's data and timestamp but also the hash digest of the previous node. This chain structure ensures that any tampering with historical data will result in changes to all subsequent hash values, making it easily detectable and achieving data immutability.

[0064] The formula for calculating the hash digest is: This formula clarifies how the hash digest is generated. Among other things, This represents the hash digest of the current node n; Indicates the current node The specific data content included, such as initial input data, translation vectors, and verification processing results; Indicates the previous node The hash digest is the key to implementing a hash chain; The timestamp represents the time when the current node is processing the data; ∥ represents the data concatenation operation. By concatenating the current data, the hash value of the previous node, and the timestamp, and then performing SHA-256 calculation, it is ensured that each hash digest is closely associated with the previous node, forming an immutable chain.

[0065] In some possible embodiments, a DPO algorithm based on clinical outcome feedback data is proposed to perform timed incremental fine-tuning of the underlying translation and validation models, achieving continuous self-optimization of the multilingual medical information processing model. Clinical outcome feedback data refers to real-world data on patient treatment effects, disease progression, adverse reactions, and recovery during actual diagnosis and treatment with the assistance of a medical information processing system. This data can be obtained through various means, such as electronic medical records, patient follow-up records, clinical trial reports, medical imaging diagnostic results, and direct feedback from doctors and patients. It is a crucial basis for evaluating model performance, identifying model biases, and guiding model optimization. The DPO algorithm, or Direct Preference Optimization algorithm, focuses on directly optimizing the model to better match human preferences or a pre-defined reward function without explicitly training a reward model. In the field of medical information processing, the DPO algorithm can directly adjust the parameters of the underlying translation and validation models by utilizing the "good" and "bad" processing outcome preferences inherent in the clinical outcome feedback data. For example, when a translation or validation result provided by the model leads to a positive clinical outcome, the DPO algorithm will increase the probability of generating such an outcome; conversely, if it leads to a negative outcome, it will decrease its probability. The underlying translation model is responsible for language conversion of multilingual medical information, such as translating a patient's oral medical history from one language to another. The validation model is responsible for using a mask matrix based on medical standard constraints. A model that adaptively validates comprehensive medical feature vectors. Timed incremental fine-tuning refers to small-scale, continuous parameter updates to the deployed translation and validation models within a preset time period (e.g., weekly, monthly, or quarterly), using the latest collected clinical outcome feedback data. This incremental approach avoids the enormous computational overhead of training from scratch, while ensuring the model can promptly absorb new medical knowledge, terminology changes, or standard updates, maintaining its accuracy and adaptability in dynamic medical environments. Through this mechanism, the multilingual medical information processing model can continuously learn, adapt, and improve its performance, forming a positive feedback loop and achieving continuous self-optimization.

[0066] In some possible embodiments, a multilingual medical information processing method is proposed, which further includes: establishing a real-time bilingual interactive and collaborative channel between doctors and patients; supporting bidirectional real-time conversion of multilingual medical information; and realizing synchronous interaction and collaborative editing during the medical information processing process.

[0067] The establishment of a real-time bilingual interactive collaboration channel between doctors and patients aims to provide a platform for instant communication, allowing both parties to exchange information in real time via text, voice, or video during medical information processing. This channel can be implemented in ways including, but not limited to, WebRTC-based audio and video call modules, instant messaging (IM) systems, and shared document editing interfaces. The establishment of this channel is fundamental to achieving barrier-free communication between doctors and patients, ensuring the immediacy and interactivity of information transmission.

[0068] This system supports bidirectional real-time conversion of multilingual medical information, ensuring that doctors and patients can communicate in their respective native languages ​​during real-time bilingual interactive collaboration. The system can instantly convert one party's input into a language the other can understand. This can be achieved using an advanced neural machine translation (NMT) model, optimized in conjunction with a domain-specific medical terminology dictionary. For example, when a patient asks a question in Spanish, the system can translate it into the doctor's working language (such as Chinese) in real time, and then translate the doctor's Chinese response back into Spanish for the patient in real time. This bidirectional real-time conversion eliminates language barriers, enabling smooth communication between doctors and patients from different language backgrounds.

[0069] This technology enables synchronous interaction and collaborative editing in the medical information processing process, allowing doctors and patients to jointly view, modify, and confirm information in real time at specific stages, such as the development of medical records, diagnostic reports, or treatment plans. Implementation methods may include: real-time data synchronization technology based on the WebSocket protocol, ensuring that modifications to the same document are immediately reflected in each other's work; a version control system to record editing history; and a permission management mechanism to differentiate editing permissions between doctors and patients. For example, when a doctor is writing a diagnostic report, the patient can see the report content in real time and ask questions about parts they don't understand. The doctor can then provide immediate explanations or make corrections, and both parties jointly confirm the final medical information. This synchronous interaction and collaborative editing ensures the accuracy and transparency of medical information and enhances patient participation and trust.

[0070] It should be noted that the above description describes some embodiments of the present invention. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than that shown in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0071] Based on the same inventive concept, corresponding to any of the above embodiments, such as Figure 2As shown in the embodiments of this application, a multilingual medical information processing device is also proposed, comprising: A medical-specific multilingual dialect translation module is configured to receive multilingual medical information input and perform speech recognition and multilingual conversion processing; The multimodal medical information parsing module is configured to extract multimodal features from multilingual medical information inputs and align the feature space through a cross-modal attention mechanism to obtain a comprehensive medical feature vector. The multinational medical standards knowledge base module is configured to store and dynamically output a medical standards constraint mask matrix; The AI ​​adaptive medical information verification module is configured to perform masked attention calculation and adaptive verification processing on the comprehensive medical feature vector based on the medical standard constraint mask matrix. The real-time bilingual interactive collaboration module for doctors and patients is configured to execute a three-level state machine control flow based on dual threshold judgment logic to achieve handle locking and interactive collaboration. The medical information storage and traceability and model self-optimization module is configured to perform full-process node storage and traceability through SHA-256 hash chain, and to use the DPO algorithm for incremental fine-tuning and self-optimization of the model.

[0072] The apparatus described above is used to implement the corresponding multilingual medical information processing method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0073] The above description is merely a preferred embodiment and the technical principles employed in this application. This application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions that can be made by those skilled in the art will not depart from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of this application. The scope of this application is determined by the scope of the claims.

Claims

1. A multilingual medical information processing method, comprising: The system can receive multilingual medical information input, which includes at least one of text information, voice information, image information, report information, and video information. Multimodal feature extraction is performed on the multilingual medical information input to obtain multiple feature vectors corresponding to different modalities; The multiple feature vectors are aligned in feature space using a cross-modal attention mechanism to obtain an aligned comprehensive medical feature vector. A medical standard constraint mask matrix is ​​obtained from a multi-national medical standard knowledge base. The medical standard constraint mask matrix contains medical information processing compliance rules from different countries and regions. Based on the medical standard constraint mask matrix, the comprehensive medical feature vector is adaptively verified to obtain the verified medical information processing result. The medical information processing results are controlled and flowed through a three-level state machine based on the dual threshold determination logic. The entire process of multilingual medical information processing is traced and documented using SHA-256 hash chains.

2. The multilingual medical information processing method according to claim 1, wherein, The process of receiving and processing multilingual medical information input includes: For the input voice information, extract the Mel-frequency cepstral coefficient (MFCC) features of the voice information; The Mel-Cepstral Coefficient (MFCC) features are input into an end-to-end dialect adaptive recognition model with an attention mechanism for speech recognition. Multilingual translation is performed using a translation adaptive algorithm based on the LoRA incremental fine-tuning architecture. This algorithm employs a translation probability model for computation, which is as follows: ； Automatic correction of colloquial medical expressions is achieved using a medical synonym map.

3. The multilingual medical information processing method according to claim 1, wherein, The multimodal feature extraction of the multilingual medical information input includes: For image and report information, a deformable convolutional neural network (DCN) is used for feature extraction. For video information, a three-dimensional convolutional neural network (3D-CNN) is used to extract features by sampling dynamic keyframes. For text information and speech recognition results, word embedding is used for feature vectorization.

4. The multilingual medical information processing method according to claim 1, wherein, The medical standard constraint mask matrix obtained from a multi-national medical standard knowledge base includes: Construct a knowledge base that includes medical information processing standards, clinical drug use rules, and drug indications and contraindications rules from multiple countries; Supports automatic incremental updates of the knowledge base in the cloud; Based on the target country or region for medical information processing, a corresponding medical standard constraint mask matrix is ​​dynamically generated. .

5. The multilingual medical information processing method according to claim 1, wherein, The three-level state machine control flow of the medical information processing result based on the dual threshold determination logic includes: Calculate the maximum confidence level of the risk conflict of the medical information processing results, max(P); Set a first security threshold θ1 and a second security threshold θ2; Based on the comparison results between the maximum risk conflict confidence value max(P) and the first security threshold θ1 and the second security threshold θ2, different state transition branches are triggered.

6. The multilingual medical information processing method according to claim 5, wherein, The triggering of different state transition branches includes: When the maximum risk conflict confidence value max(P) is less than the first security threshold θ1, a low-risk release branch is executed, and the process is automatically released. When the maximum risk conflict confidence value max(P) is greater than or equal to the first safety threshold θ1 and less than the second safety threshold θ2, the risk branch is suspended during execution, the flow handle is suspended, and a risk warning pops up. When the maximum risk conflict confidence value max(P) is greater than or equal to the second security threshold θ2, a high-risk interception branch is executed, locking the flow handle and blocking the delivery.

7. The multilingual medical information processing method according to claim 6, wherein, The high-risk interception branch also includes: Automatically recommend alternative medical information processing solutions; Forced association of a dual instruction trigger between the first processing end confirmation instruction and the second review end asynchronous electronic signature instruction.

8. The multilingual medical information processing method according to claim 1 further includes: Based on clinical outcome feedback data, the DPO algorithm was used to perform timed incremental fine-tuning of the underlying translation model and validation model. Achieve continuous self-optimization of the multilingual medical information processing model.

9. The multilingual medical information processing method according to claim 1 further includes: Establish a real-time bilingual interactive and collaborative channel between doctors and patients; Supports two-way real-time conversion of multilingual medical information; Enables synchronous interaction and collaborative editing during medical information processing.

10. A multilingual medical information processing device, comprising: A medical-specific multilingual dialect translation module is configured to receive multilingual medical information input and perform speech recognition and multilingual conversion processing; The multimodal medical information parsing module is configured to extract multimodal features from the multilingual medical information input and align the feature space through a cross-modal attention mechanism to obtain a comprehensive medical feature vector. The multinational medical standards knowledge base module is configured to store and dynamically output a medical standards constraint mask matrix; The AI ​​adaptive medical information verification module is configured to perform masked attention calculation and adaptive verification processing on the comprehensive medical feature vector based on the medical standard constraint mask matrix. The real-time bilingual interactive collaboration module for doctors and patients is configured to execute a three-level state machine control flow based on dual threshold judgment logic to achieve handle locking and interactive collaboration. The medical information storage and traceability and model self-optimization module is configured to perform full-process node storage and traceability through SHA-256 hash chain, and to use the DPO algorithm for incremental fine-tuning and self-optimization of the model.

Images