Medical examination-oriented real-time voice stream key information extraction method

Abstract

A medical examination-oriented real-time voice stream key information extraction method belongs to the technical field of artificial intelligence and medical information, and is characterized in that non-sensitive medical corpus and general corpus collection are combined with a style-oriented prompt control mechanism to drive a large language model to generate diversified simulation corpus, and an extended basic corpus set in the field of medical examination is constructed; the corpora are normalized and screened, and a high-quality corpus training set is formed; constructing a probability language model oriented to the field of medical examination based on the training set; detecting the medical examination real-time voice stream, abandoning mute segments, and forming an effective voice audio set; generating a candidate transcription sequence and an acoustic probability score for each speech segment by using an acoustic decoder; performing comprehensive scoring and sorting on the candidate sequences, and outputting an optimal transcription text; and extracting the structured medical examination key information from the transcriptional text by using a large language model. The medical term and numerical value recognition accuracy is high, and reliable support is provided for electronic medical record automatic input and report generation.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and medical information technology, specifically relating to a method for extracting key information from real-time voice streams for medical examinations. Background Technology

[0002] Current medical speech recognition faces three core challenges: the high density of technical terminology and precise numerical values ​​leads to a high error rate in general speech recognition systems; real medical speech data is extremely scarce and highly privacy-sensitive, making it difficult to use for training or fine-tuning acoustic models; and the lack of end-to-end structured output capabilities prevents direct integration with examination report systems.

[0003] In recent years, some solutions have attempted to integrate voice and image information to improve report generation quality. For example, Chinese invention patent CN118866227B proposes a "method for assisting in the generation of ultrasound examination reports," which uses speech recognition to obtain the doctor's spoken content and aligns it with ultrasound image features to generate a report. Although this solution achieves multimodal fusion, its speech recognition module still directly calls a general automatic speech recognition (ASR) system without specifically modeling the high-density professional terminology and precise numerical expressions in the spoken content. This results in a high error rate in transcribing key information, making it difficult to meet the requirements of clinical rigor.

[0004] Another approach attempts to alleviate the scarcity of speech data through data augmentation. For example, Chinese invention patent application CN119694300A proposes using Generative Adversarial Networks (WGAN) to augment speech recognition training data, thereby improving the model's generalization ability in low-resource scenarios. However, the entire technical framework of this patent is geared towards general speech recognition and does not incorporate any domain knowledge guidance. The generated speech-text pairs lack the ability to model typical language structures and co-occurrence patterns of professional terms in medical examination scenarios. Therefore, even when applied to medical speech recognition, the augmented data is unlikely to effectively improve the accuracy of recognizing key diagnostic information, and essentially fails to overcome the language modeling bottleneck caused by the scarcity of medical speech data.

[0005] In summary, although some publicly available technologies exist in this field, there is a lack of end-to-end, domain-specific solutions tailored to medical examination voice scenarios. An ideal system should simultaneously possess: a special processing mechanism for high-density numerical values ​​and terminology in medical examinations; optimization based on non-privacy-sensitive corpora for extended medical domains; and a complete pipeline for end-to-end extraction of structured medical cues from free speech. Summary of the Invention

[0006] To overcome the shortcomings of existing technologies, this invention provides a method for extracting key information from real-time speech streams for medical examinations. Through four core technologies—simulated corpus generation, numerical modeling optimization, domain-extended probabilistic language model rescoring, and structured extraction from large language models—this method significantly improves the accuracy of terminology and numerical recognition, reduces reliance on real-world privacy data and cloud-based large model inference, and thus makes up for the key deficiencies in existing patent disclosures and practical applications.

[0007] The technical solution adopted by this invention to solve its technical problem is: A method for extracting key information from real-time audio streams for medical examinations includes the following steps: Step S1: By collecting non-sensitive medical domain corpora and publicly available general corpora, and using a style-guided prompting control mechanism to drive a large language model to generate diverse simulation corpora, a basic corpus for the medical examination field is constructed. Step S2: By introducing a multi-level corpus quality control mechanism, the expression at the structural and symbolic levels is unified, and the basic corpus constructed in Step S1 is standardized and screened to form a corpus training set. Step S3: Use the corpus training set formed in step S2 to train a probabilistic language model in order to construct a probabilistic language model for the field of medical examination. Step S4: Receive the real-time medical examination voice stream and perform voice detection, marking it frame by frame as "voice" or "silence / noise", discarding long silent segments, thereby forming a set of audio segments focused on voice; Step S5: Decode each segment in the set described in Step S4 using an acoustic decoder for Automatic Speech Recognition (ASR), calculate the token that may correspond to each frame in the speech, and generate several candidate transcribed sequences and their acoustic scores for each segment. Step S6: Based on the probabilistic language model (PLM) for medical examination trained in step S3, and combined with the acoustic score described in step S5, the candidate transcribed sequences are comprehensively scored and ranked to output the overall optimal transcribed text. Step S7: Using a large language model, extract verbal cues from the doctor's speech during the medical examination from the final transcribed text of Step S6.

[0008] Furthermore, in step S1, the basic corpus It consists of multiple subsets: ; in, For general speech recognition transcription corpus, For medical Q&A and medical dialogue data, For large language models Simulated corpora generated under multiple different prompting controls; The aforementioned subsets collectively form a domain-extended corpus during training, which is used to improve the model's overall language modeling ability and statistical adaptability in medical examination language.

[0009] Preferably, the prompting control mechanism is used to drive a large language model without relying on privacy-sensitive data. Generate simulated speech data that conforms to the oral patterns of medical examinations. The large language model High temperature parameters are configured during the generation process. This mechanism defines a set of style-guided cue projects: ; in, As a teaching template-based prompt, it guides the large language model by setting the role of a medical teacher. Simulate oral templates used in teaching scenarios to generate highly standardized and accurately terminological corpora; For clinical examination-type prompts, guide the large language model. Simulates the natural verbal behavior of doctors during real-time examinations, requiring the output of a complete description of the lesion including medical modifiers, with sentence structure closer to real clinical scenarios; Provides free-voice prompts to guide large language models. Generate a corpus with mixed styles that differ significantly in expression length and level of detail; The resulting simulated corpus can be formally represented as: ; in, For large language models Given the prompt Under control, high temperature parameters are configured. The generated subset of the corpus.

[0010] More preferably, in the set of prompting projects The introduction of a data placeholder strategy to guide large language models When generating the corpus, placeholders are used to represent specific medical examination and measurement data, and a large language model is established. The original generated corpus set is Each sample All contain placeholder representations. Define a replacement mapping function. ,satisfy: ; Where m is the simulated medical examination measurement value used to fill the placeholders, which follows a Gaussian distribution. .

[0011] Furthermore, in step S2, the quality control mechanism includes the following operations: Perform numeric transcription functions on Arabic numerals, Roman numerals, operators, and units in the text. Convert it to its corresponding Chinese form; Perform punctuation cleaning function on text Remove all punctuation marks and special characters, keeping only Chinese characters and uppercase Latin letters; Extending the language model dictionary (Lexicon) yields a domain-extended language model dictionary. ,in, For general domain dictionaries, For medical terminology dictionary, A dictionary of medical abbreviations; Perform word segmentation on each sentence S of the text. It is segmented into word sequences. ;in, For word sequence The word count. Also, for each sentence... Corresponding word sequence word count If the following conditions are met: or If the statement is invalid, then the statement should be removed; where, The set threshold for text length to be removed. This is an illegal character set that was set.

[0012] Preferably, the numerical expressions in the extended corpus of medical examination are decomposed. Numerals are broken down into single-character word sequences, and numerals consisting of multiple characters are removed from the corpus. A numerical decomposition mapping function is defined. Express any numerical value Convert to a sequence of corresponding single words: ; Correspondingly, from the language model dictionary All multi-character numeric words longer than one character were removed, allowing the language model to perform contextual modeling based solely on single-character sequences; the word sequence corresponding to the oral context in medical examinations was also considered. The conditional probability expressed in the correct numerical value is calculated using the chain of transition probabilities: ; in, Numerical expression Length; Numerical expression The word sequence corresponding to the external context, Numerical units in numerical expression The preceding sequence content, together with the preceding content, constitutes the context for calculating conditional probabilities.

[0013] Furthermore, in step S3, the probabilistic language model, based on the corpus training set formed in step S2, is trained to learn the linguistic statistical distribution features of the extended medical domain with the goal of minimizing the negative log-likelihood. The loss function is: ; in, For the vocabulary participating in the model's probabilistic learning, For vocabulary The word sequence corresponding to the context, For the corpus training set; probability Estimation is achieved through a smoothing strategy based on contextual diversity.

[0014] In step S4, the audio tagging and segmentation include: Define speech activity detection mapping By calculating the length as Real-time medical examination audio stream Each frame The probability of belonging to speech and a pre-set probability threshold Compare and output a frame-level binary tag sequence. : ; Identify all consecutive non-speech frame sequences satisfy When the duration of non-voice communication exceeds the preset silence threshold Then, the interval is divided at both ends and discarded to obtain a set of audio segments focused on speech after segmentation. .

[0015] The process of step S5 is to process each audio segment output in step S4. Several candidate transcribed sequences and their corresponding acoustic scores are generated using an acoustic decoder; specifically, an acoustic decoding function is defined. The input speech segments are mapped to a set of "candidate sequence-score" pairs: ; in, For the first Candidate sequences, Score its acoustics. This represents the number of candidate sequences.

[0016] In step S6, the rescoring mapping function is defined. The sequences in the candidate transcription sequence set output in step S5 are... Mapped to a comprehensive score The sequence with the highest score was selected as the final transcribed text. The overall score is composed of the acoustic score, the score of the medical examination domain probabilistic language model (PLM) obtained from step S3 training, and the parameter configuration. Joint calculation: ; in, For the medical examination field, PLM predicts the probability of this sequence. , For sequence Length, , , The pre-set weight parameters.

[0017] In step S7, the template-guided (few-shot) prompting process drives the large language model. The final transcribed text output from step S6 Extract key information from medical examinations and configure with low temperature parameters. Output a structured medical clue JSON object Specifically: ; in, A prompt containing instructions. For output template; The prompts and templates are for large language models The following constraints and instructions shall be applied: Contextual correction capability: When there are obvious recognition errors in the transcribed text, the large language model can correct them based on medical common sense; Style control: Force output concise, objective, and subjective descriptions to avoid generating vague expressions; Content focus: Only process content directly related to the findings of the medical examination, ignoring irrelevant statements such as pleasantries and operating instructions; Structural requirements: Strictly follow the preset JSON template for output, with fields including organ name, measurement value, morphological description, boundary features, echo properties, etc., to ensure seamless integration with hospital information systems (such as PACS, EMR).

[0018] The beneficial effects of this invention are mainly reflected in: 1. Improve the accuracy and robustness of medical speech recognition. Existing medical speech recognition systems mostly rely on general acoustic decoders to directly output transcription results, which is difficult to handle the frequent occurrence of precise numerical values, professional terminology, and free-flowing speech styles in medical examination scenarios. This invention constructs an extended probabilistic language model for the medical examination domain based on non-privacy simulated corpora, and performs joint acoustic-language re-scoring on ASR candidate sequences, effectively correcting homonym misidentification and numerical expression errors. Simultaneously, through quality control mechanisms such as single-character decomposition modeling and Chinese numerical transcription of numerals in the training corpus, the ability of the trained probabilistic language model to model the context of measured values ​​is further optimized. Therefore, without retraining the acoustic decoder, the recognition accuracy of key diagnostic information and the system robustness are significantly improved. 2. Reduce data dependence and training costs, and overcome privacy constraints. Traditional speech recognition requires a large amount of real patient speech for fine-tuning, facing challenges such as data scarcity, high annotation costs, and difficulties in privacy compliance. This invention drives a large language model to generate diverse simulation corpora through a style-guided prompting mechanism, and constructs a high-quality training set by combining general corpora and medical dialogue corpora. The simulation corpora employ a placeholder generation + Gaussian sampling replacement strategy to generate numerical expressions, preserving the authenticity and rationality of numerical values ​​in medical speech while completely avoiding the use of real patient data. This data strategy allows PLM training to avoid contact with any sensitive speech or text, significantly reducing the data acquisition threshold and compliance risks, while ensuring the breadth of language coverage and domain adaptability of the model. 3. Achieve end-to-end automated output from speech streams to structured medical clues. Existing solutions typically stop at speech-to-text conversion, with subsequent structuring still relying on rule engines or manual intervention, making it difficult to handle complex semantics such as omissions and pronouns. This invention introduces template-guided prompting engineering at the end of the process. Through dual constraints of instruction prompts and output templates, it drives a large language model to accurately extract key elements such as organs, measurements, and morphological descriptions from free-flowing spoken text, and outputs machine-readable structured objects that conform to the interface specifications of clinical information systems. This design achieves seamless integration of "speech, text, and structured output," providing directly usable structured examination information for downstream applications such as automatic electronic medical record entry and report generation. 4. Ensuring privacy and security throughout the entire process while maintaining deployment flexibility. The core processes of this invention, including corpus construction, PLM training, ASR rescoring, and structured extraction, do not rely on real patient speech or text. Furthermore, key models can be deployed on local edge devices or hospital servers, avoiding the uploading of sensitive data to the cloud. The entire processing pipeline meets regulatory and ethical requirements while also addressing clinical needs for low latency and high reliability, demonstrating strong feasibility and potential for widespread adoption. Attached Figure Description

[0019] Figure 1This is an overall flowchart of a method for extracting key information from real-time audio streams for medical examinations.

[0020] Figure 2 This is a flowchart of the medical simulation corpus generation and quality control process, which shows the generation, processing, and quality control of the corpus training set.

[0021] Figure 3 This is a flowchart of the acoustic decoder generating candidates and the probabilistic language model rescoring based on the probabilistic language model in the field of medical examination. It illustrates the method of integrating acoustic scores and medical examination PLM language scores to comprehensively evaluate and rank multiple candidate sequences, thereby outputting a final transcribed text with high accuracy. Detailed Implementation

[0022] The present invention will now be further described with reference to the accompanying drawings.

[0023] Reference Figures 1-3 A method for extracting key information from real-time audio streams for medical examinations, the overall process is as follows: Figure 1 As shown, this embodiment will provide a detailed description of steps S1 to S7.

[0024] The method for extracting key information from real-time audio streams for medical examinations in this embodiment includes the following steps: Step S1: Construct a basic corpus extended to the medical field. The process is as follows: First, a non-sensitive general speech recognition transcription corpus was collected. Medical Q&A Conversation Material Furthermore, based on this, a style-oriented prompt control mechanism is adopted to drive the large language model. Generate simulated corpus This mechanism defines a set of style-oriented cue engineering. Each guides a large language model High temperature parameter configuration The system generates teaching template-type, clinical examination-type, and free-narrative-type corpora to cover the linguistic features of medical oral communication in three dimensions: structural standardization, clinical authenticity, and expressive diversity. This preferred embodiment... For a pre-trained general-purpose large language model (e.g., the large language model Qwen3-8B), the resulting set of simulated corpora can be formally represented as: ; in, For large language models Given the prompt Under control, high temperature parameters are configured. The generated subset of corpus. In this embodiment, the large language model The temperature parameter is defined as [0, 2], this embodiment The value range is [1.4, 2), preferably... .

[0025] Large language model In the process of generating the simulation corpus, a data placeholder strategy is introduced to make the original generated corpus set... Includes numeric placeholders, and then uses a replacement mapping function. Replace the placeholders with those that follow a Gaussian distribution. By simulating random values ​​of medical measurements, a simulation corpus that is non-privacy-sensitive but possesses medical statistical authenticity is obtained. In this embodiment, the random value follows a Gaussian distribution. Ultimately, the three types of corpora were merged into a foundational corpus for expanding the field of medical examinations. .

[0026] Step S2: Implement corpus quality control. Perform unified structure and symbol level preprocessing, the process is as follows: First, through a numerical transcribing function based on substitution rules... Convert Arabic numerals, Roman numerals, operators, and units into their corresponding Chinese forms; Secondly, through the punctuation cleaning function Find and remove all punctuation marks and special characters, keeping only Chinese characters and uppercase Latin letters; Subsequently, the language model dictionary was expanded. ,in For general domain dictionaries, For medical terminology dictionary, A dictionary of medical abbreviations; Next, the word segmentation function is executed for each statement. Based on dictionary The maximum matching algorithm is used to segment the sequence into word sequences and calculate the number of words. If the number of words is less than a preset threshold, the sequence is considered complete. If the sample contains illegal characters, it will be removed; this embodiment preferably uses the minimum number of sequence words. ; Finally, the numerical splitting mapping function is executed on the numeral representation. This process converts all numerals into sequences of single words and removes all numerals consisting of multiple words from the dictionary, forcing the language model to model numerical representations based solely on single-word contexts. (The word sequence corresponding to the spoken context in a medical examination is then described.) The conditional probability expressed in the correct numerical value is calculated using the chain of transition probabilities: ; in, Numerical expression Length; Numerical expression The word sequence corresponding to the external context, Numerical units in numerical expression The preceding sequence content, together with the preceding content, constitutes the context for calculating conditional probabilities. This is achieved through forced splitting of numerals. This allows the model to learn only The probability of a single character in the expression significantly improves the generalization ability of numerical representation and avoids the sparsity of multi-character numerals. Its effect is manifested in... ; The above processing results in a training corpus.

[0027] Step S3: Train a probabilistic language model for the medical examination domain. The process is as follows: Based on the training set of the corpus formed in step S2, a probabilistic language model is trained with the objective of minimizing the negative log-likelihood. The loss function is: ; in, For the vocabulary participating in the model's probabilistic learning, For vocabulary The word sequence corresponding to the context, For the corpus training set; probability Estimation is performed using a smoothing strategy based on contextual diversity. In this embodiment, the probabilistic language model can adopt an n-gram structure (preferably 3-gram in this embodiment), and combine it with Kneser-Ney smoothing to estimate the conditional probability, thereby improving the estimation stability in the case of sparse data.

[0028] Step S4: Segment the real-time medical examination audio stream. The process is as follows: During a medical examination, the system receives a real-time audio stream of the medical examination. Define a speech activity detection map. By calculating the input length as Real-time audio stream Each frame The probability of belonging to speech Compared with the preset probability threshold The comparison results are used to output a frame-level binary label sequence. In this embodiment, a pre-trained speech activity detection model based on transformers is preferred for implementing the detection mapping. This is to distinguish between "speech" and "silence / noise". It identifies all consecutive non-speech frame intervals, and if their duration exceeds a preset silence threshold... Then discard that interval, forming a set of audio segments focused on speech. In this embodiment, the probability threshold The preferred setting is 0.5, the mute threshold. The preferred setting is 12 seconds, in order to reduce the consumption of subsequent steps in processing silent audio segments while ensuring the integrity of the semantic context.

[0029] Step S5: The acoustic model generates candidate transcription sequences and acoustic scores. The process is as follows: For each speech segment output in step S4 The system generates several candidate transcribed sequences and their corresponding acoustic scores through acoustic decoding. This embodiment preferably uses a CTC-based acoustic model, which predicts and merges adjacent identical tokens on a frame-by-frame basis of the input audio segment to obtain candidate sequences. Specifically, an acoustic decoding function is defined. The input speech segments are mapped to a set of "candidate sequences – scores": ; in, For the first Candidate sequences, The number of candidate sequences provides a diverse hypothesis space for subsequent reordering.

[0030] Step S6: Based on the probabilistic language model, perform candidate rescoring and optimal selection. The process is as follows: Define rescoring mapping function Using the medical domain-extended probabilistic language model trained in step S3, each candidate sequence output in step S5 is processed. Calculate its language model score And, combining the acoustic score and other configuration parameters, a comprehensive score is calculated through weighted fusion: ; in, The probability of PLM predicting this sequence in the medical examination field. , For sequence Length, , , These are pre-set weighting parameters. This embodiment preferably sets them as follows: , , The candidate sequence with the highest overall score was selected as the final transcribed text. This significantly improves the accuracy of recognizing technical terms and precise numerical values ​​without retraining the acoustic model.

[0031] Step S7, Structured Extraction and Output, the process is as follows: The final transcribed text output from step S6 Input large language model This embodiment is preferred. This is a pre-trained general-purpose large language model (e.g., the large language model Qwen3-8B). Under template-guided prompting, this large language model provides prompts based on instructions. With preset output template Dual constraints, configured with low-temperature parameters Extract information and output a structured medical clue JSON object. Specifically: ; in, A prompt containing instructions. This is the output template. In this embodiment, the large language model... The temperature parameter is defined as This embodiment The value range is (0, 0.6], preferably... .

[0032] The prompts and templates are for large language models The following constraints and instructions shall be applied: The output style is limited to being concise, objective, and free of subjective speculation; Only process content directly related to medical examination findings, ignoring irrelevant statements; Machine-readable structured objects (such as JSON format) that conform to the interface specifications of the clinical information system are generated strictly according to the template for use in subsequent processes.

[0033] This embodiment also includes a real-time speech stream key information extraction system for medical examinations, used to implement the processing steps S1 to S7 of the real-time speech stream key information extraction method for medical examinations, including: The corpus construction module collects non-sensitive general speech recognition transcription corpora. Medical Q&A Conversation Material Furthermore, based on this, a style-oriented prompt control mechanism is adopted to drive the large language model. Generate simulated corpus This embodiment is preferred. This is for pre-trained general-purpose large language models (e.g., the large language model Qwen3-8B). The mechanism defines a set of style-oriented cue engineering techniques. Each guides a large language model High temperature parameter configuration The system generates teaching template-type, clinical examination-type, and free-narration-type corpora to cover the linguistic features of medical oral communication in three dimensions: structural standardization, clinical authenticity, and expressive diversity. The resulting simulated corpus can be formally represented as follows: ; in, For large language models Given the prompt Under control, high temperature parameters are configured. The generated subset of corpus. In this embodiment, the large language model The temperature parameter is defined as [0, 2], this embodiment The value range is [1.4, 2), preferably... .

[0034] The corpus quality control module implements a corpus quality control mechanism, including: Numerical transcribing functions based on substitution rules Convert Arabic numerals, Roman numerals, operators, and units to Chinese. Punctuation cleaning function Find and remove all punctuation marks and special characters; Dictionary extension unit, extended language model dictionary ,in For general domain dictionaries, For medical terminology dictionary, A dictionary of medical abbreviations; The text segmentation and filtering unit performs dictionary-based processing on each sentence. The word segmentation function of the maximum matching algorithm Calculate the number of valid words; if it is less than a preset threshold... If the sample contains illegal characters, it will be removed; this embodiment preferably uses the minimum vocabulary threshold. ; Numerical splitting function This process converts all numeral expressions in the corpus into single-word numerical sequences and removes all numerals consisting of multiple characters from the dictionary, forcing the language model to model numerical expressions based solely on single-word contexts. (The word sequence corresponding to the spoken context in a medical examination is then described.) The conditional probability expressed in the correct numerical value is calculated using the chain of transition probabilities: ; in, Numerical expression Length; Numerical expression The word sequence corresponding to the external context, Numerical units in numerical expression The preceding sequence content, together with the preceding content, constitutes the context for calculating conditional probabilities. This is achieved through forced splitting of numerals. This allows the model to learn only The probability of a single character in the expression significantly improves the generalization ability of numerical representation and avoids the sparsity of multi-character numerals. Its effect is manifested in... .

[0035] After processing by the above functions and units, a corpus training set is formed.

[0036] The probabilistic language model training module, based on the training corpus generated by the corpus quality control module, trains the probabilistic language model with the objective of minimizing the negative log-likelihood. Its loss function is: ; in, For the vocabulary participating in the model's probabilistic learning, For vocabulary The word sequence corresponding to the context, For the corpus training set; probability Estimation is performed using a smoothing strategy based on contextual diversity. In this embodiment, the probabilistic language model can adopt an n-gram structure (preferably 3-gram in this embodiment), and combine it with Kneser-Ney smoothing to estimate the conditional probability, thereby improving the estimation stability in the case of sparse data.

[0037] The speech segmentation module receives the real-time audio stream dictated by the doctor. Define a speech activity detection map. By calculating the input length as Real-time audio stream Each frame The probability of belonging to speech Compared with the preset probability threshold The comparison results are used to output a frame-level binary label sequence. In this embodiment, a pre-trained speech activity detection model based on transformers is preferred for implementing the detection mapping. This is to distinguish between "speech" and "silence / noise". It identifies all consecutive non-speech frame intervals, and if their duration exceeds a preset silence threshold... If the specified interval is not found, then discard that interval and output a set of audio segments focused on speech. In this embodiment, the probability threshold The preferred setting is 0.5, the mute threshold. The preferred setting is 12 seconds, in order to reduce the consumption of subsequent steps in processing silent audio segments while ensuring the integrity of the semantic context.

[0038] The acoustic decoding module processes each audio segment output by the speech segmentation module. The system generates several candidate transcribed sequences and their corresponding acoustic scores through acoustic decoding. This embodiment preferably uses a CTC-based acoustic model, which predicts and merges adjacent identical tokens on a frame-by-frame basis of the input audio segment to obtain candidate sequences. Specifically, an acoustic decoding function is defined. The input speech segments are mapped to a set of "candidate sequences – scores": ; in, For the first Candidate sequences, This represents the number of candidate sequences. In this embodiment, a preset number of candidate sequences is preferred. This step provides a diverse hypothesis space for subsequent reordering.

[0039] The rescoring and selection module implements rescoring mapping. The medical domain-extended probabilistic language model, trained using the step-by-step probabilistic language model training module, is used for each candidate sequence output in step S5. Calculate its language model score And, combining the acoustic score and other configuration parameters, a comprehensive score is calculated through weighted fusion: ; in, The probability of PLM predicting this sequence in the medical examination field. , For sequence Length, , , These are pre-set weighting parameters. This embodiment preferably sets them as follows: , , The candidate sequence with the highest overall score was selected as the final transcribed text. This significantly improves the accuracy of recognizing technical terms and precise numerical values ​​without retraining the acoustic model.

[0040] The structured extraction module will re-score and select the final transcribed text from the module's output. Input large language model This embodiment is preferred. This is a pre-trained general-purpose large language model (e.g., the large language model Qwen3-8B). Under template-guided prompting, this large language model provides prompts based on instructions. With preset output template Dual constraints, configured with low-temperature parameters Extract information and output a structured medical clue JSON object. Specifically: ; in, A prompt containing instructions. This is the output template. In this embodiment, the large language model... The temperature parameter is defined as This embodiment The value range is (0, 0.6], preferably... .

[0041] The prompts and templates are for large language models The following constraints and instructions shall be applied: The output style is limited to being concise, objective, and free of subjective speculation; Only process content directly related to medical examination findings, ignoring irrelevant statements; Machine-readable structured objects (such as JSON format) that conform to the interface specifications of the clinical information system are generated strictly according to the template for use in subsequent processes.

[0042] The solution in this embodiment constructs an extended probabilistic language model for the medical examination domain, trained on a non-privacy simulated corpus, to perform joint acoustic-linguistic re-scoring of ASR candidate sequences, effectively correcting homophone misidentification and numerical expression errors. Simultaneously, through corpus quality control mechanisms such as single-character decomposition modeling of numerals and Chinese numerical transcription, the trained language model's ability to model the context of measured values ​​is further optimized. This significantly improves the accuracy of key diagnostic information recognition and system robustness without retraining the acoustic model.

[0043] This embodiment employs a style-guided prompting mechanism to drive a large language model to generate diverse simulated corpora, and combines general corpora with medical dialogue corpora to construct a high-quality training set. Specifically, the simulated corpora utilize a strategy of generating placeholders and replacing them with Gaussian sampling to produce numerical representations. This preserves the authenticity and rationality of numerical values ​​in medical oral presentations while completely avoiding the use of real patient data. This data strategy ensures that PLM training does not require access to any sensitive speech or text, significantly reducing the data acquisition threshold and compliance risks, while simultaneously guaranteeing the model's breadth of language coverage and domain adaptability.

[0044] This embodiment introduces a large language model at the end of the process. Through dual constraints of instruction prompts and output templates, it accurately extracts key elements such as organs, measurements, and morphological descriptions from free-flowing spoken text and outputs machine-readable structured objects that conform to the interface specifications of clinical information systems. This design achieves seamless integration of "voice, text, and structured information," providing directly usable structured input for downstream applications such as automatic electronic medical record entry and report generation.

[0045] In this embodiment, the core processes such as corpus construction, PLM training, ASR rescoring, and structured extraction do not rely on real patient speech or text. Furthermore, key models can be deployed on local edge devices or hospital servers, avoiding the uploading of sensitive data to the cloud. The entire processing pipeline meets regulatory and ethical requirements while also addressing clinical needs for low latency and high reliability, demonstrating strong feasibility and potential for widespread adoption.

[0046] The embodiments described in this specification are merely examples of implementations of the inventive concept and are for illustrative purposes only. The scope of protection of this invention should not be considered limited to the specific forms described in these embodiments; rather, it extends to equivalent technical means conceived by those skilled in the art based on the inventive concept.

Claims

1. A method for extracting key information from real-time speech stream for medical examination, characterized in that, The method comprises the following steps: Step S1, by collecting non-sensitive medical field corpus and public general corpus, and using a prompt control mechanism based on style guidance to drive a large language model to generate diversified simulation corpus, to form a basic corpus library for medical field expansion; Step S2, by introducing a multi-level corpus quality control mechanism, unifying the expression of structure and symbol level, implementing standardization and screening on the basic corpus library constructed in step S1, to form a corpus training set; Step S3, using the corpus training set formed in step S2 to train a probability language model, to build a probability language model for the medical examination field; Step S4, receiving real-time medical examination speech stream, and performing speech detection, marking each frame as "speech" or "silence / noise", discarding long silence segments, to form a set of audio segments focusing on speech; Step S5, performing acoustic decoding on each segment in the set in step S4 to generate a plurality of candidate transcription sequences corresponding to each segment and their acoustic scores; Step S6, based on the probability language model for the medical examination field trained in step S3, combining the acoustic scores in step S5, comprehensively scoring and sorting the candidate transcription sequences to output the overall optimal transcription text; Step S7, using a large language model to extract key information in medical examination from the transcription text output in step S6.

2. A medical examination-oriented real-time speech stream key information extraction method according to claim 1, characterized in that, In the step S1, the base corpus consists of a plurality of subsets: ； wherein, is a general-purpose speech recognition transcription corpus, is a medical question answering and medical dialogue corpus, is a simulated corpus generated by a large language model under multiple different prompt controls; The prompt control mechanism is used to drive a large language model to generate simulated corpus conforming to the speaking rules of medical examinations without relying on privacy-sensitive data The mechanism defines a set of style-oriented prompt function sets to guide the large language model to simulate the natural speaking behavior of doctors in real-time examination process.

3. A method for extracting key information from real-time speech stream for medical examination as claimed in claim 2, wherein, In the prompt control mechanism, a data placeholder strategy is introduced to guide the large language model to use a placeholder instead of specific medical examination measurement data when generating the corpus, and the original corpus set generated by the large language model is wherein each sample contains a placeholder expression, defines a replacement mapping function , which satisfies: ； where m is a simulated medical examination measurement value used to fill in the placeholder, subject to a Gaussian distribution .

4. The method for extracting key information from real-time speech stream for medical examination according to claim 1, characterized in that, In step S2, the corpus quality control mechanism comprises the following operations: Performing a value transcription function on Arabic numerals, Roman numerals, operators, and units in the text converts it into the corresponding Chinese form; Performing a punctuation cleaning function on the text , removing all punctuation marks and special characters, leaving only Chinese characters and capital Latin letters; Extended language model lexicon , obtaining a domain-extended language model lexicon wherein, is a general domain lexicon, is a medical terminology lexicon, is a medical abbreviation lexicon; for each sentence of the text performing a word segmentation function segmenting it into a word sequence wherein, the word sequence the number of words of the word sequence; meanwhile, for each sentence the number of words of the corresponding word sequence if the following conditions are met: or or the sentence is removed; wherein, is a set removal text length threshold, is a set of illegal characters.

5. A medical examination oriented real-time speech stream key information extraction method according to claim 4, characterized in that, The number expression in the medical field extended corpus is split to separate the number into single character sequence, and the number composed of multiple characters in the vocabulary is deleted. A number split mapping function is defined to convert any number expression into the corresponding single character sequence. Meanwhile, all multi-character number words with length greater than one are removed from the language model dictionary .

6. The method for extracting key information from real-time speech stream for medical examination according to claim 1, wherein, In step S3, the probability language model is trained based on the corpus training set formed in step S2, with the goal of minimizing the negative log likelihood.

7. A medical examination oriented real-time speech stream key information extraction method according to claim 1, characterized by, In step S4, the speech detection of the real-time medical examination speech stream comprises: Defining voice activity detection mapping , according to the input stream of medical examination voice stream Each frame Probability of belonging to voice , and the preset probability threshold Comparison, output frame-level binary label sequence ; identify all continuous sequences Satisfy When the duration exceeds the preset silence threshold , split at both ends of the interval and discard the interval to obtain a set of split audio segments focused on voice .

8. The medical examination-oriented real-time speech stream key information extraction method of claim 1, wherein, In step S5, the acoustic decoding process involves decoding each audio segment output in step S4. The acoustic decoder is used to calculate the frame-level token probability, adjacent frames with the same token are merged to generate several candidate transcription sequences, and their corresponding acoustic scores are calculated.

9. A medical examination oriented real-time speech stream key information extraction method according to claim 8, characterized in that, In step S6, a re-scoring mapping function is defined Each sequence in the candidate transcription sequence set output from step S5 is mapped to a composite score The sequence with the highest score is selected as the final transcription text, the composite score is calculated from the acoustic score obtained in step S5, the medical examination field-oriented probabilistic language model trained in step S3, and other parameter configurations ​​ ； wherein, is an acoustic decoding function, is a sequence of medical field PLMs prediction probability ; is a length of the sequence ; , , is a pre-set weight parameter.

10. A medical examination oriented real-time speech stream key information extraction method according to claim 1, characterized by, In step S7, a template-guided prompt engineering is defined to drive the large language model to extract key information in medical examination from the final transcription text output in step S6, and to output a structured medical clue JSON structured object with a low temperature parameter configuration; the template-guided prompt engineering constrains and instructs the large language model as follows: Reasonably correct the expressions that obviously violate medical common sense in the transcription text based on the context; Limit the output style to be concise, objective and without subjective speculation; Only process the content directly related to the findings of medical examination, and ignore irrelevant sentences; Strictly generate a JSON format structured object conforming to the interface specification of the clinical information system according to the preset template.

Images