A medical speech recognition text error correction method and system based on multi-agent cooperation and skill scheduling

Abstract

The application discloses a medical voice recognition text error correction method and system based on multi-agent cooperation and skill scheduling, belongs to the cross technical field of natural language processing and medical informatization, and aims to solve the bottleneck problem existing in the current medical ASR text. The application constructs a four-agent division and cooperation architecture, realizes decoupled closed-loop iterative error correction by relying on a global shared state data unit, avoids invalid retrieval through a two-level tool decision mechanism, obtains medical entity correction candidates by combining pinyin and character shape double-layer hybrid retrieval, completes accurate text repair according to confidence level stratification and priority rules, and realizes result convergence through iterative verification of a supervised agent; meanwhile, a skill grading loading and modular ablation deployment structure is set, and the whole-process localization deployment adapts to the medical data privacy compliance requirements. The application is suitable for the semantic error correction scene of noisy medical transcription text in the automatic generation of electronic medical records, medical dialogue dictation, remote diagnosis and treatment record arrangement, medical retrieval enhancement generation and the like.

Description

Technical Field

[0001] This invention relates to the technical field of the intersection of natural language processing and medical informatics, specifically to a text error correction method, system, electronic device, and computer-readable storage medium for the post-processing stage of an automatic speech recognition (ASR) system in the medical field. Background Technology

[0002] In recent years, applications such as telemedicine, electronic medical record digitization, and physician dictation assistants (AIScribe) have rapidly gained popularity, making Automatic Speech Recognition (ASR) systems a key component in clinical information workflows. Unlike general ASR applications, clinical applications are extremely sensitive to entity-level errors—an error in recognizing a drug name, a disease term, or a diagnostic conclusion can directly lead to medical accidents. Therefore, high-precision asynchronous post-processing of ASR output text is of great significance in reducing the paperwork burden on clinicians and avoiding medical errors.

[0003] Currently, various ASR error correction schemes have been proposed by the industry and academia, but they all have certain shortcomings in medical scenarios: (1) Traditional language model Rescoring and Seq2Seq error correction methods.

[0004] These methods rely on shallow text statistical patterns or end-to-end sequence-to-sequence models to re-evaluate or rewrite ASR candidates. Lacking medical knowledge, they are unable to reliably repair homophone or similar-looking substitution errors such as “fallopian tube → fallopian tube”, “fan bladder → hernia”, and “masic acid → maleic acid”. The error rate increases significantly, especially under low signal-to-noise ratio (SNR) conditions.

[0005] (2) Single-pass direct generation of error correction based on general large language model (LLM).

[0006] These methods use prompting engineering to directly rewrite noisy transcripts by calling general-purpose large models such as GPT-4o, DeepSeek, and Gemini. However, these general-purpose large models lack strict constraints on medical knowledge boundaries and tend to "fabricate" seemingly plausible medical descriptions (i.e., medical illusions) under conditions of low signal-to-noise ratio and incomplete context, which may amplify errors. Furthermore, they have high inference costs, outputting only after each call, and lack a self-verification mechanism.

[0007] (3) Search Enhancement Generation (RAG) error correction scheme based on cloud API.

[0008] These methods typically rely on a combination of commercial cloud models and private medical knowledge bases. However, medical data is subject to strict privacy regulations such as HIPAA and GDPR, and uploading patient medical records to public clouds poses compliance risks. Furthermore, conventional RAG pipelines trigger searches indiscriminately for all entities, introducing a large number of irrelevant candidates and significantly increasing inference latency.

[0009] (4) The ability to call localized model tools on a small and medium scale is limited.

[0010] To meet privacy requirements, the industry tends to deploy small- to medium-sized medical models locally (such as HuatuoGPT-o1-7B, Taiyi, Qwen3-8B, etc.). However, these models do not natively support structured function-calling interfaces, making it difficult to reliably trigger external tools and knowledge bases, thus failing to obtain authoritative medical candidate words as anchors.

[0011] (5) Defects in the evaluation system.

[0012] Existing general ASR metrics such as CER / WER calculate all tokens with equal weight, which cannot reflect the completely different degrees of harm caused by medical entity errors and function word errors in clinical practice; at the same time, there is a lack of Chinese medical ASR evaluation datasets with controllable signal-to-noise ratio, controllable dialect, and medical entity annotations in the industry.

[0013] In summary, the existing technical bottlenecks that this invention aims to solve can be summarized as follows: Bottleneck 1: The error correction paradigm generated directly in a single pass lacks self-verification and is prone to introducing medical illusions.

[0014] Bottleneck 2: Traditional RAG performs indiscriminate retrieval of all entities, introducing redundant candidates and increasing latency.

[0015] Bottleneck 3: Locally deployed small- to medium-sized medical models do not natively support structured tool calls, making it difficult to reliably access external authoritative knowledge bases.

[0016] Bottleneck 4: Existing retrieval paradigms mainly rely on single character similarity matching, which makes it difficult to simultaneously cover two typical error sources: "homophones (ASR acoustic layer error)" and "characters with similar shapes (OCR / input method layer error)".

[0017] Bottleneck 5: The lack of medical ASR evaluation datasets with controllable signal-to-noise ratio makes it difficult to systematically evaluate the robustness of error correction algorithms under different noise environments. Summary of the Invention

[0018] The purpose of this invention is to provide a medical speech recognition text error correction method, system, device, and storage medium based on multi-agent collaboration and skill scheduling, to solve the bottleneck problems existing in the aforementioned background technology. Specifically: (1) Through the four-stage closed-loop multi-agent iterative architecture, the error correction task that was originally difficult to decouple is decomposed into four serial controllable links of "detection-retrieval-repair-verification", which limits the single-point illusion of the large model and significantly improves the final error correction reliability. (2) Through the "AgentSkill" mechanism proposed in this invention and the matching two-stage chain thinking (CoT) simulation tool scheduling strategy, the locally deployed small and medium-sized medical language model can autonomously decide whether to call the external knowledge base without relying on the native function-calling interface, and can review the call decision in the second stage to avoid misuse and omission. (3) By using a two-layer hybrid retrieval algorithm of “pinyin matching + character shape fuzzy matching”, and prioritizing the adoption of pinyin matching candidates when both layers of matching are matched at the same time, the algorithm can specifically handle the two main sources of errors: ASR homophones and OCR / input method character shape errors. (4) The corresponding localized deployment architecture (MCP protocol + local CSV knowledge base + local large model) ensures that no patient data is sent to the outside during the whole process, which meets the requirements of medical privacy regulations such as HIPAA.

[0019] To achieve the above objectives, the present invention provides the following technical solution: This invention proposes a medical speech recognition text error correction method based on multi-agent collaboration and skill scheduling, the method comprising the following steps: Step S1: Construct a shared state data unit for multiple agents, store the original speech recognition transcribed text to be corrected in the original text field and the current text field of the state data unit respectively, and initialize the iteration count parameter to zero; Step S2: Error Detection: The diagnostic agent reads the current text field and, together with the local large model with structured output constraints, identifies suspected erroneous words and their corresponding related information in the text, storing them in the error detection field of the state data unit; if no error is detected, the current text is directly output as the error correction result. Step S3: Level 1 Tool Decision: The skill scheduling agent reads the error information and the preset skill library, combines them to form a prompt message, and then inputs it into the local large model. The model uses reasoning logic to determine whether to call an external processing tool. If it is determined that no tool should be called, the correction candidate content is cleared and the process proceeds to the text correction step. Step S4: Secondary Decision Review: For the tool selected in the primary decision, generate prompt information based on the tool operation specifications and reference examples, and input it into the local large model to perform a secondary verification of the tool call decision; if the call is revoked after verification, record the log and proceed to the text correction step; if the call is retained after verification, perform a hybrid search. Step S5: Hybrid retrieval: Call the knowledge retrieval service to perform a two-layer hybrid retrieval of pinyin matching and character shape fuzzy matching, and store the retrieved structured correction candidate content into the correction candidate field of the status data unit; Step S6: Text Correction Step: The correction agent reads the current text and correction candidate content, replaces and repairs the erroneous content according to the preset reliability stratification rules, matching result priority rules, and combined with the semantic understanding ability of the local large model, and updates the corrected text to the current text domain. Step S7: Iterative Supervision: The supervising agent reads the original text, the updated current text, and the data processed in this round, and uses the local large model to verify the error correction effect; if the verification is successful, or the iteration count reaches the preset iteration limit, the final text is output; otherwise, the iteration count is updated, and the process returns to the error detection step to execute a new round of error correction. Step S8: Output the result: Output the final current text in the state data unit as the error correction completed text.

[0020] Furthermore, the diagnostic agent, skill scheduling agent, correction agent, and supervision agent described in this invention achieve decoupled communication through a globally shared agent state data unit. Each agent only reads and writes its own corresponding field and does not directly transmit intermediate data, thus achieving logical isolation. These four elements constitute a closed-loop iterative chain of diagnostic testing, skills review, text repair, and quality verification. If the verification fails, the iteration is restarted, and the iteration process is terminated by controlling the preset iteration limit. Each intelligent agent has a dedicated division of labor and independent context and prompt word configuration, corresponding to a single function such as error location, tool scheduling, text repair, and quality judgment. The method supports modular switch configuration, allowing selective disabling of corresponding functional modules and switching to various ablation deployment architectures such as complete error correction, skill-free retrieval, unsupervised verification, diagnostic detection, single agent error correction, and pure large model direct correction.

[0021] Furthermore, the tool invocation determination described in this invention adopts a two-level decision-making mechanism; In the first-level decision-making stage, the model outputs structured call decision results based on a concise skill list containing skill descriptions and a list of suspected erroneous words, which can directly skip the retrieval process; In the second-level review stage, the tools selected in the first-level decision are verified a second time by combining complete skill operation specifications and positive and negative reference cases. If the call is revoked, it is judged as a decision reversal and the log is recorded and the search is skipped. Only when both levels of decision retain the call instruction will the mixed search operation be performed.

[0022] Furthermore, the hybrid retrieval described in this invention employs a two-layer retrieval mechanism combining precise pinyin matching and fuzzy character shape matching; The pinyin precise matching layer performs Chinese character to pinyin mapping on each entity in the medical entity database to obtain pinyin strings, and groups entities with the same pinyin into the same index bucket; for suspected erroneous words, the same pinyin mapping is performed to match the index, and corresponding scores are assigned and the pinyin matching type is marked according to whether there is a unique match or multiple entity matches; The character shape fuzzy matching layer calculates the similarity of character sequences using a character sequence similarity algorithm, filters a specified number of similar candidates based on a preset threshold, determines the corresponding score based on the similarity value, and marks the fuzzy matching type. The two-layer search results are integrated into a unified candidate list in descending order of score. When the same erroneous word hits both layers of candidate results, the candidate result corresponding to the pinyin matching is adopted first. The optimal candidate is selected by a weighted calculation method that combines pinyin similarity score, character shape similarity score and preset weight hyperparameters. The weight hyperparameters are adjusted to adapt to error correction scenarios where homophone errors are the main problem in speech recognition.

[0023] Furthermore, the confidence level stratification rule of the present invention is as follows: when the confidence level is greater than or equal to 0.9, the corresponding candidate content is directly adopted; when the confidence level is between 0.6 and 0.9, the candidate content is selected in combination with the context; when the confidence level is less than 0.5, the original text is retained; the priority rule of the matching result is as follows: when the same erroneous word hits both the pinyin matching and character shape matching results, the candidate content corresponding to the pinyin matching is selected first.

[0024] Furthermore, all external tools and knowledge bases in this invention are encapsulated in the form of agent skills and employ a three-layer document hierarchical loading mechanism. The first layer is the skill introduction layer, which includes the skill name and a single-sentence summary. It is loaded only during the first-level tool decision-making stage and is used by the model to quickly identify available tools. The second layer is the skill operation layer, which includes detailed operation specifications for data contracts, triggering logic, search strategies, confidence thresholds, result verification, and input cleaning. The third layer is the skill example layer, which includes positive call cases and negative error cases. The content of the second and third layers is loaded only during the secondary decision review stage, specifically for the tools selected by the first-level decision.

[0025] Furthermore, the four-agent collaborative process of the present invention is modeled using a LangGraph state graph and uses a global Pydantic state object as the only data interaction path for each agent node. The global state object is configured with an original text field, a current text field, an error detection field, a correction candidate field, an iteration count field, and a review result field, which respectively store the original identified text, the intermediate text of the iteration, the suspected error information, the retrieval correction candidate, the iteration round, and the supervision and verification result.

[0026] Based on the same inventive concept, the medical speech recognition text correction method based on multi-agent collaboration and skill scheduling described in this invention can be entirely implemented using computer software. Therefore, correspondingly, this invention also provides a medical speech recognition text correction system based on multi-agent collaboration and skill scheduling.

[0027] Furthermore, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the medical speech recognition text correction method based on multi-agent collaboration and skill scheduling described in any of the preceding descriptions.

[0028] Furthermore, the present invention also provides a computer device, the device including a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes a medical speech recognition text error correction method based on multi-agent collaboration and skill scheduling as described above.

[0029] The present invention achieves the following beneficial effects compared to the prior art: (1) Significantly outperforms the ASR baseline under all signal-to-noise ratio conditions and is superior to local models of similar size: In a pristine, noise-free environment, the precise entity recall rate of this invention reached 98.81% (WhisperASR baseline 88.68%, an improvement of 10.13 percentage points). At 0dB noise, the precise entity recall rate of this invention reaches 98.79%, which is an improvement of 11.46 percentage points compared to WhisperASR's 87.33%. exist At 5dB noise, the precise entity recall rate of this invention reaches 93.11%, which is 11.95 percentage points higher than WhisperASR's 81.16%. In extreme At 10dB noise, the exact entity recall rate of this invention still reaches 80.72% (an improvement of 11.09 percentage points compared to WhisperASR's 69.63%), and the fault-tolerant entity recall rate reaches 81.59%, indicating that the remaining errors are mainly character-level approximation differences. (2) Outperforms several commercially available state-of-the-art large models in the low-to-medium noise range: At 0dB, this invention (98.79%) outperforms DeepSeekV3.2+KB (97.61%), Gemini2.5-flash (95.08%), and end-to-end Qwen3-ASR-1.7B (96.12%). In the downstream RAG retrieval task, at 0dB, the NDCG@3 of this invention is 0.8018 and MAP@3 is 0.7949, which exceed Qwen3-ASR-1.7B (0.7580 / 0.7144) and DeepSeekV3.2+KB (0.7692 / 0.7400), respectively.

[0030] (3) The independent contributions of each innovative module were verified through ablation experiments: After removing the "skills / knowledge base", the precise entity recall rate dropped from 98.79% to 93.85%, a decrease of 4.94 percentage points, proving that the proposed skill scheduling mechanism is the most critical source of performance improvement; After removing the "supervisory agent", the precise entity recall rate dropped to 95.37%, indicating that a closed-loop iterative verification mechanism is indispensable. The performance dropped to 95.22% after degrading to a single-agent ReAct architecture (A4), demonstrating that the multi-agent division of labor architecture described in this invention has structural advantages under strict clinical entity fidelity requirements.

[0031] (4) Meet medical privacy compliance requirements: All components are deployed locally, with no patient data being sent out. It can run completely on the medical institution's intranet or on a standalone workstation, and complies with privacy regulations such as HIPAA and GDPR.

[0032] (5) Project controllability: The ABLATION environment variable supports various ablationable configurations such as Full, w / oSkill, w / oSupervisor, w / oDiagnostician, Single-Agent, and PureLLM, allowing for module-based deployment in scenarios with limited computing power or latency sensitivity. Typically, each utterance takes about 12 to 23 seconds to process on a single RTX4090, and can be embedded into an electronic medical record system as an asynchronous post-processing task.

[0033] The method described in this invention can be applied to scenarios that require semantic-level error correction of noisy medical transcribed text, such as automatic generation of electronic medical records (EMR), medical dialogue transcription, organization of remote medical diagnosis and treatment records, and medical retrieval enhancement generation (RAG). Attached Figure Description

[0034] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0035] Figure 1 This is a schematic diagram of the overall framework of the medical ASR error correction system described in this invention; Figure 2 This is a schematic diagram of the data generation pipeline described in this invention; Figure 3 This is a flowchart of the closed-loop algorithm described in this invention; Figure 4 This is a schematic diagram of the two-layer hybrid retrieval algorithm described in this invention; Figure 5 This is a signal-to-noise ratio robustness curve of the present invention, wherein... Figure 5 (a) in the figure represents the exact entity recall curve. Figure 5 (b) in the figure represents the NDCG@3 curve of the downstream RAG retrieval task; Figure 6 This is a schematic diagram of the ablation experiment comparison of the present invention, wherein... Figure 6 In the figure, (a) represents the exact entity recall rate. Figure 6 In the equation (b), semantic similarity is represented. Figure 6 (c) in the text refers to the downstream retrieval of NDCG@3. Detailed Implementation

[0036] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. The following examples will help those skilled in the art to further understand the present invention, but do not limit the present invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention.

[0037] Implementation Method 1: This implementation method provides a medical speech recognition text error correction method based on multi-agent collaboration and skill scheduling. The method includes the following steps: Step S1: Construct a shared state data unit for multiple agents, store the original speech recognition transcribed text to be corrected in the original text field and the current text field of the state data unit respectively, and initialize the iteration count parameter to zero; Step S2: Error Detection: The diagnostic agent reads the current text field and, together with the local large model with structured output constraints, identifies suspected erroneous words and their corresponding related information in the text, storing them in the error detection field of the state data unit; if no error is detected, the current text is directly output as the error correction result. Step S3: Level 1 Tool Decision: The skill scheduling agent reads the error information and the preset skill library, combines them to form a prompt message, and then inputs it into the local large model. The model uses reasoning logic to determine whether to call an external processing tool. If it is determined that no tool should be called, the correction candidate content is cleared and the process proceeds to the text correction step. Step S4: Secondary Decision Review: For the tool selected in the primary decision, generate prompt information based on the tool operation specifications and reference examples, and input it into the local large model to perform a secondary verification of the tool call decision; if the call is revoked after verification, record the log and proceed to the text correction step; if the call is retained after verification, perform a hybrid search. Step S5: Hybrid retrieval: Call the knowledge retrieval service to perform a two-layer hybrid retrieval of pinyin matching and character shape fuzzy matching, and store the retrieved structured correction candidate content into the correction candidate field of the status data unit; Step S6: Text Correction Step: The correction agent reads the current text and correction candidate content, replaces and repairs the erroneous content according to the preset reliability stratification rules, matching result priority rules, and combined with the semantic understanding ability of the local large model, and updates the corrected text to the current text domain. Step S7: Iterative Supervision: The supervising agent reads the original text, the updated current text, and the data processed in this round, and uses the local large model to verify the error correction effect; if the verification is successful, or the iteration count reaches the preset iteration limit, the final text is output; otherwise, the iteration count is updated, and the process returns to the error detection step to execute a new round of error correction. Step S8: Output the result: Output the final current text in the state data unit as the error correction completed text.

[0038] Furthermore, in one specific embodiment, the above-mentioned medical speech recognition text error correction method is described in detail, and the specific execution steps are as follows: Step S1: Construct the shared agent state S, write the original ASR transcribed text T to be corrected into the "current_text" (current text field) and "original_text" (original text field) fields of the state, and initialize the iteration round "iteration_count" (iteration count parameter) to 0; Step S2: Diagnostic Phase: The Diagnostic Agent reads the current_text from the shared state S, performs semantic error detection on it using the local large model with structured output constraints, identifies all suspected erroneous words and writes them back to the "detected_errors" field of the shared state in the form of {wrong_term, context} tuples; if the detection result is empty, it jumps to step S8 to output the current_text as the final result; Step S3: Skill scheduling first stage (Phase 1 decision): The skill scheduling agent (SkillAgent / ToolDispatcher) reads the current detected_errors and the "SkillHeaders" library, assembles them into decision prompts, and inputs them into the local large model. The model then uses chain-like thinking to autonomously determine whether the current error should call a certain external skill / tool. If the model decides not to call it, it sets correction_candidates to null and jumps to step S6. Step S4: Skill Scheduling Second Stage (Phase 2 Review): For each tool selected in Phase 1, the complete "SkillInstructions" and "ReferenceExamples" are re-entered into the local large model as prompts, allowing the model to perform a second confirmation under the condition of knowing the detailed operation specifications and counterexamples; if the model revoked the call decision in Phase 2, it is considered "Phase 2 decision reversal", recorded in the tool event log and skipped (jump to S6); only when the model still retains the tool call decision in Phase 2 will it proceed to Step S5; Step S5: MCP tool execution phase: The local knowledge retrieval service performs a two-layer hybrid retrieval of "pinyin matching + character shape fuzzy matching", returns structured candidates and writes them to the "correction_candidates" field in the shared state; Step S6: Correction Phase: The correction agent reads current_text and correction_candidates, and performs replacement-style repair on current_text based on the confidence threshold rules (score≥0.9 high confidence is adopted first; 0.6≤score<0.9 must be adopted in conjunction with the context; score<0.5 retains the original text) and the priority rule of "if the same erroneous word is hit in both layers, the candidate with reason='PinyinMatch' is adopted first", combined with the language understanding ability of the local large model, and writes the repair results back to current_text; Step S7: Supervision phase: The supervisor agent reads the original_text and current_text, as well as the detection errors and candidate information of this round, and outputs "OK" or JSON feedback of the problem field from the local large model; if the feedback is "OK" or the current iteration_count has exceeded the preset iteration limit N, then jump to S8; otherwise, return to S2 to enter a new round of iteration. Step S8: Output stage: The final current_text in the shared state is used as the text output after error correction. The independent evaluation module (optional) extracts key medical entities and compares them with the benchmark annotation, outputting indicators such as accurate entity recall, fault-tolerant entity recall and semantic similarity.

[0039] Furthermore, in one specific implementation, the four agents described in this embodiment (diagnostic agent, skill scheduling agent, correction agent, and supervisory agent) communicate decoupledly through a globally shared AgentState (including fields such as original_text, current_text, detected_errors, correction_candidates, iteration_count, and current_analysis). Each agent only consumes / produces a portion of the state fields and does not directly transmit large sections of intermediate results. The overall closed loop is as follows: Diagnostician → SkillDispatcher → Corrector → Supervisor → (If it fails) → Diagnostician (Proceed to the next iteration) Specifically, this closed loop has the following innovative features: The roles are complementary: the diagnostic part is only responsible for "pointing out the error location" and does not perform repair; the correction part is only responsible for "performing the replacement" and does not perform retrieval; the skill scheduling part is only responsible for "acquiring external knowledge" and does not rewrite; and the supervision part is only responsible for "final quality judgment". The prompts within each agent are strictly isolated from the visible context to avoid "cognitive overload" and early retrieval illusions that can occur when a single large model performs multiple roles simultaneously. Iterative convergence: After each iteration, the supervisory agent performs a second review of the current results. If problems are still found, it can trigger a return to the diagnostic phase to start a new round of "detection-retrieval-repair-verification". Convergence is guaranteed by a preset iteration limit N (typical value N=3). Ablation-enabled and configurable: This invention supports disabling modules such as diagnostics, skills, and supervision by configuring switches, thereby forming various ablation configurations such as "Full / w / oSkill / w / oSupervisor / w / oDiagnostician / Single-Agent / PureLLM" for comparison, verification, and deployment customization.

[0040] Furthermore, in one specific implementation, because locally deployed small- to medium-sized medical models (such as HuatuoGPT-o1-7B) do not natively support structured function-calling interfaces, traditional RAG pipelines struggle to reliably drive them to make tool calls. This invention designs a two-stage Chain-of-Thought (CoT) tool scheduling mechanism to overcome this deficiency, as follows: Two-tier decision-making mechanism; Phase 1 decision-making stage: Only a minimal list of available tool descriptions (SkillHeaders) consisting of "skill name + one-sentence summary" is used, and this is combined with a list of suspected erroneous words to be processed to form decision prompts. The local large model is required to output a JSON structure in the form of {"tool_calls":[...]} to express the call decision based on chained thinking. When the model decides "no need to call", the external search is immediately skipped to avoid unnecessary and potentially semantically distorted over-correction of fluent everyday language (see the counterexample in Example 4 where "anger" was mistakenly retrieved as "kidney qi / hernia").

[0041] Phase 2 Review and Execution: For each candidate tool selected in Phase 1, the complete SkillInstructions (detailed operation manual, including data contracts, trigger judgment logic, confidence threshold rules, input cleaning rules, etc.) and ReferenceExamples (set of positive and negative cases) are re-inputted into the model as prompts, requiring the model to perform a secondary confirmation under the condition of knowing the detailed specifications and negative cases. Only if the model still maintains the tool invocation decision in Phase 2 will a search request actually be initiated to the MCP service. If the model retracts the invocation decision in Phase 2, it is considered a "Phase 2 decision reversal," recorded in the tool event log, and the search is skipped.

[0042] The core value of the above two-stage design lies in the fact that by decomposing the tool invocation decision into a chain of judgments in two different context windows of "coarse screening + fine review", the local small and medium-sized model actually "simulates" the structured tool invocation capability that only large models possess; at the same time, it significantly reduces the false invocation rate without sacrificing the tool recall rate, and avoids excessive retrieval of non-medical errors (emotions, everyday language).

[0043] Furthermore, in one specific embodiment, the present invention implements a two-layer hybrid retrieval algorithm of "pinyin matching + character shape fuzzy matching" on the MCP server, specifically targeting two typical errors in medical ASR post-processing: Level 1 (Pinyin Exact Matching, PinyinLayer): Targeting the main homophone / near-homophone errors of ASR. Offline, the grapheme-to-phoneme mapping φ(e) is performed on all entities e in the medical entity library K to obtain their Pinyin strings, and entities with the same Pinyin are grouped into the same index bucket; during query, the same mapping φ(e_err) is performed on the suspected error word e_err, and an exact table lookup is carried out in the Pinyin index. If a unique entity is hit, the score is 0.98; if multiple homophone entities with different characters are hit, they are sorted by surface similarity and assigned a score of 0.95, and the reason is uniformly marked as "PinyinMatch"; Level 2 (Glyph Fuzzy Matching, GlyphLayer): Targeting glyph-similar errors introduced by OCR, input methods or language model decoding (such as "千扰素→干扰素"). The character sequence similarity ratio is calculated based on difflib.SequenceMatcher, and truncated at the server-side hard threshold 0.6 (FUZZY_CUTOFF), only retaining the most similar several (MAX_CANDIDATES_FUZZY) candidates; the score is round(ratio,3), and the reason is marked as "FuzzyMatch"; Comprehensive sorting: The results of the two levels are concatenated in descending order of score to form a unified candidate list for return, and the agent side performs downstream adoption according to the priority rule that "if the same error word is hit in both levels, the candidate with reason='PinyinMatch' is preferentially adopted".

[0044] Formally, for the suspected error word e_err, the optimal candidate k* is given by the following formula: k*=argmax_ { k∈K} λ·S_pho ( φ ( e_err ) ,φ ( k )) + ( 1 λ ) ·S_morph ( e_err,k )] wherein, φ (·) is the Chinese character to Pinyin mapping function; S_pho and S_morph are the Pinyin layer matching and glyph layer fuzzy similarity scoring functions respectively; λ ∈[0,1] is a hyperparameter that controls the relative weight of the acoustic dimension and the morphological dimension. In the medical ASR scenario, the present invention adopts a larger λ(i.e., biased towards the pinyin level) to reflect the statistical pattern that ASR errors are mainly homophone errors.

[0045] Furthermore, in one specific embodiment, during the correction phase, the present invention designs a two-level filtering rule based on confidence thresholds, specifically as follows: After the server performs hard truncation on the glyph candidates at 0.6, this invention applies a secondary filtering rule on the agent side (in the prompt word template of the CorrectorAgent) to divide the confidence interval of the candidates into three levels, as shown in Table 1: Table 1

[0046] Furthermore, in one specific embodiment, the present invention encapsulates each external tool / knowledge base in the form of an "agent skill" and adopts a three-tiered document hierarchical loading structure: The first layer: SkillHeaders—a minimalist “skill name + one-sentence description”, loaded in the Phase 1 decision stage, to allow the model to quickly browse all available tools; The second layer: SkillBody / SkillInstructions—a detailed operation manual, including sub-specifications such as data contracts, trigger judgment logic, search strategies, confidence thresholds, result verification, and input cleaning. It is only loaded for tools selected in Phase 1 during the Phase 2 review stage. The third layer: SkillExamples—a collection of positive and negative examples (few-shot cases), covering correct calling patterns (pinyin hit, character shape hit, batch correction) and negative examples (over-correction, forced adoption with low confidence, fabrication of empty results), which are also loaded in Phase2.

[0047] By employing this three-tiered hierarchical loading method, this invention, without exceeding the limited context window of the local small-to-medium-sized model, enables the system to "only consult the summary when a decision is needed, and then consult the detailed manual and counterexamples when execution is required," which aligns with the habits of human experts reading tool manuals, thereby improving the accuracy of decision-making and execution.

[0048] Furthermore, in one specific embodiment, the present invention designs a shared state and absolutable architecture. The present invention models the four-agent collaborative process in the form of LangGraphStateGraph, and uses a global Pydantic state object, AgentState, as the unique data path between nodes. This state object includes at least the following fields: original_text: Retain the original ASR text for the supervising agent to perform the final quality comparison; current_text: The intermediate error correction text currently being written in the iteration, which is written by the Corrector in each round; detected_errors: A list of suspected errors detected in this round ([{wrong_term,context},...]); correction_candidates: The set of correction candidates returned by the MCP tool in this round ({term:[{word,score,reason},...]}); iteration_count: The number of iterations executed so far, used to force convergence; current_analysis: The Supervisor's conclusion text for this round of review (such as "OK" or issue JSON).

[0049] Furthermore, this invention designs an ablative configuration switch (ABLATION environment variable) that supports the following preset combinations: full, no_skill, no_supervisor, no_diagnostician, and pure_llm. This configurability facilitates both the scientific evaluation of the independent contributions of each module and allows for tailoring based on computing power or latency budgets during actual deployment.

[0050] Furthermore, in one specific implementation, to comply with medical data privacy regulations such as HIPAA, this invention deploys all components on the medical institution's intranet or local workstations, with the specific topology as follows: Local large-scale model inference service: Based on transformers, local small and medium-sized medical models (such as HuatuoGPT-o1-7B) are loaded, and unified langchain-style interfaces such as .invoke / .ainvoke / .with_structured_output / .bind_tools are provided in the LocalLLM wrapper class; Local knowledge retrieval service: The search_medical_entity and batch_search_medical_entities tools are exposed locally using the ModelContextProtocol (MCP) protocol; the backend medical entity database (such as CMeEE) is loaded from the local path specified by the environment variable in CSV format, and the entire retrieval process does not make any requests to the external network; Protocol-independent client: The client starts and connects to the MCPServer via standard stdio. The MCP tool is automatically loaded as a langchainTool object by langchain_mcp_adapters, and then injected into the prompt words of the local model by LocalLLM.bind_tools. Tool event log: The inputs, outputs, and Phase1 / Phase2 decisions for each tool call are recorded in JSONL format to the local log, which facilitates auditing, compliance tracing, and offline analysis.

[0051] In summary, the medical speech recognition text error correction method based on multi-agent collaboration and skill scheduling in this embodiment has the following innovative points: (1) A closed-loop iterative error correction pipeline consisting of four decoupled intelligent agents: diagnosis, skill scheduling, correction, and supervision. It communicates through shared states and the supervising agent decides whether to enter the next round after each iteration. An iteration limit is preset to ensure convergence. (2) A two-stage chain thinking (Phase 1 decision + Phase 2 review) tool scheduling mechanism enables small and medium-sized local medical language models that do not natively support function-calling interfaces to drive external knowledge retrieval stably and with low error call rate. (3) A two-layer hybrid retrieval algorithm of "pinyin matching + character shape fuzzy matching" is proposed, and the downstream priority rule of pinyin matching candidate is adopted first when both layers are hit, so as to specifically handle the two typical error sources of ASR homophone error and character shape similarity error; (4) A three-tiered adoption strategy (score≥0.9 / 0.6≤score<0.9 / score<0.5) and a rule for preserving the original text of empty results, which are used to perform secondary filtering on the agent side for candidates above the server hard threshold; (5) A three-tiered loading structure for agent skills documents, consisting of SkillHeaders / SkillBody / SkillExamples, which matches the limited context window of local small and medium-sized models; (6) An ablationable multi-agent orchestration method based on LangGraphStateGraph+PydanticAgentState, which supports switching multiple configurations through environment variables; (7) A localized service-client deployment architecture based on the MCP protocol, in which all medical data does not leave the local area and meets privacy compliance requirements such as HIPAA.

[0052] Implementation Method Two: Combination Figures 1 to 6This invention further illustrates a medical speech recognition text error correction method based on multi-agent collaboration and skill scheduling. Figure 1 What is shown is the overall framework of the system, such as Figure 1 As shown, the overall framework of the present invention consists of two parts, left and right. Figure 1 The left half shows the pipeline for constructing the noise grading dataset (NoisyMedSum): using Medical-Dialogue-Dataset-Chinese as the original corpus ( Figure 1 Top left), LLM filtering and reconstruction module ( Figure 1 The "LLMFilterRefactoring" module extracts multi-turn doctor-patient dialogues into paragraph-level summaries while retaining medical entity annotations; subsequently, it passes through a dialect injection module ( Figure 1 The "DialectInjectionModule" and the noise injection module ( Figure 1 The "NoiseInjectionModule" ultimately outputs hierarchical ASR transcription pairs. Figure 1 in "GradedASRTranscriptionPairs"). Figure 1 The right half illustrates the multi-agent closed-loop error correction framework: four functional modules—DiagnosticianAgent, SkillAgent, CorrectorAgent, and SupervisorAgent—communicate decoupledly around the central shared agent state (SharedAgentState). Each agent only consumes / produces a subset of fields from the state. The SkillLibrary in the upper right corner is dynamically invoked by the SkillAgent through a three-stage process: "discovering skills—selecting skills—invoking skills." Figure 1 The bidirectional arrows between the agents represent their read-write relationships with the shared state, rather than direct data transfer between agents, reflecting the "state-centric" decoupled communication paradigm proposed in this invention.

[0053] Figure 2 The diagram shows the data generation pipeline of this invention, such as... Figure 2 As shown, the NoisyMedSum dataset constructed in this invention is generated through a three-stage pipeline. The first stage is data acquisition and text distillation (…). Figure 2Left side): Taking the raw MedDialog-CN data as input, the process sequentially performs corpus integration, invalid data deletion, knowledge transfer, and semantic formatting. Post-processing is then performed using LLM combined with Human-Computer Collaboration (HITL) mode, outputting a JSON-structured summary text of the doctor-patient dialogue and its medical entity annotations. The second stage involves acoustic mapping and dialect injection (…). Figure 2 (Central) The JSON text is synthesized into basic speech using the Edge_TTS engine and randomly mapped to four dialect or accent representation spaces according to a preset probability matrix—Wu (Shanghainese), Cantonese, Northeastern Mandarin (Liaoning accent), and Southwestern Mandarin (Sichuanese)—to simulate regional accent differences commonly seen in clinical settings. The third stage involves noise superposition and gradient grading. Figure 2 (Right side): Using the MUSAN corpus as the noise source, samples were taken from its three categories of materials: "Music / Speech / Noise," and the signal-to-noise ratio (SNR) was calculated as SNR = 10·log₂. 10 (P_signal / P_noise) is calculated and then linearly superimposed onto the synthesized speech to obtain the original noise-free, 0dB speech. 5dB The final ASR evaluation sample with a 10dB four-level signal-to-noise ratio gradient. The openSMILE and GenevaMinimalisticAcousticParameterSet in the lower right corner are used to compare the 88-dimensional acoustic features of the synthesized speech with real clinical recordings to verify the acoustic fidelity of the dataset.

[0054] Figure 3 The diagram shows the closed-loop algorithm flow of this invention, as follows: Figure 3 As shown, the SC-MASC closed-loop error correction framework of this invention consists of 8 execution nodes (S1 to S8) and 4 decision nodes (decision 1 to decision 4). The process enters from the top "Start" ellipse, first in S1 ( Figure 3 The light gray rounded rectangle completes the initialization of the shared state S, writes the ASR transcribed text T to be corrected into the S.current_text field, and sets the iteration counter i to zero. Then it enters the S2 diagnostic agent node ( Figure 3 (Light blue rounded rectangle), this agent performs semantic error detection and writes the detection results to the detected_errors field. Decision 1 ( Figure 3 (Medium-gray diamond shape) Check if detected_errors is empty: if empty, then follow... Figure 3 The solid red line on the right leads directly to the S8 output node; if it is not empty, it follows the "Yes" branch to enter the S3 skill scheduling Phase 1 decision node. Figure 3medium light purple rounded rectangle), this node only loads extremely simplified SkillHeaders, and allows the local large model to perform coarse screening based on chain-of-thought to determine whether an external tool needs to be called. Decision 2 processes the decision result of Phase 1: if the model determines "no call required" (that is, a Phase 1 decision reversal occurs), then follow Figure 3 the red solid line path marked with "(avoid over-correction)" to bypass MCP retrieval and directly enter the S6 correction agent, so as to prevent harmful over-correction of smooth daily expressions; if the determination is "call required", then follow the "yes" branch to enter the S4 Phase 2 review node, which loads the complete SkillBody and Examples documents to perform secondary confirmation on the call decision.

[0055] Decision 3 processes the review result of Phase 2: if the review is not passed, it also follows the red solid line to bypass MCP and directly enter S6; if passed, it enters the S5 MCP tool execution node ( Figure 3 medium light turquoise rounded rectangle) to trigger local knowledge retrieval. The S6 correction agent node ( Figure 3 medium light coral rounded rectangle) performs replacement repair on current_text according to the confidence threshold rule, and the result is written back to the shared state. The S7 supervision agent node ( Figure 3 medium light amber rounded rectangle) compares the quality of the text before and after repair. Decision 4 determines the process direction: if the supervision conclusion is "OK" or the number of iteration rounds i has reached the preset upper limit N, follow the "yes" branch to enter S8 to output the final error-corrected text T*; otherwise follow Figure 3 the left red dashed line (marked with "i←i+1") to loop back to the S2 diagnostic agent, and start a new round of iteration. Figure 3 the medium black solid line represents the forward data flow, the red solid line represents the "skip / avoid over-correction" path, and the red dashed line represents the iteration loop, the three types of line styles are respectively marked in the legend.

[0056] Figure 4 shows the two-layer hybrid retrieval algorithm of the present invention, such as Figure 4 as shown, the two-layer hybrid retrieval algorithm proposed by the present invention is presented in a left-right parallel dual-channel structure. The algorithm entrance is the top input node ( Figure 4 top rounded rectangle), which receives the suspected error word e_err (such as "fan qi", "gan rao su", "ma lai suan") as a query. e_err is shunted to two parallel channels at the same time. The left channel is level 1 pinyin exact matching ( Figure 4the large container on the left in the figure), which sequentially includes three embedded submodules: (i) a pinyin mapping sub-block φ(·) that maps "shan qi" to the pinyin string "shanqi"; (ii) a pinyin index exact table lookup sub-block that retrieves the hit set {e∈K:φ(e)=φ(e_err)} from the offline-constructed pinyin index (PINYIN_MAP); (iii) a scoring rule sub-block that assigns score=0.98 to a unique hit and score=0.95 to a polysemous hit, with the uniform annotation of reason="PinyinMatch". This layer specifically handles the main homophonic / near-homophonic errors of the ASR system. The right channel is the layer 2 glyph fuzzy matching ( Figure 4 the large container on the right in the figure), which also includes three submodules: (i) a character sequence similarity sub-block that calls difflib.SequenceMatcher to calculate the ratio value; (ii) a hard threshold truncation sub-block that uses FUZZY_CUTOFF=0.60 as the lower bound to filter candidates and retains at most 3 most similar ones; (iii) a scoring rule sub-block that assigns score=round(ratio,3)∈[0.6,1.0], with the annotation of reason="FuzzyMatch". This layer specifically handles glyph similarity errors introduced by OCR, input methods or language model decoding.

[0057] The two channels converge at the Figure 4 candidate fusion and sorting module below ( Figure 4 the long rectangle in the figure), which merges the two layers of candidates in descending order of score and executes the priority rule of the present invention — "if the same error word is hit in both layers, the candidate with reason=PinyinMatch is preferentially adopted". The formal expression k*=argmax_k[λ·S_pho(φ(e_err),φ(k))+(1 λ)·S_morph(e_err,k)] is directly annotated in Figure 4 the figure. The final output node ( Figure 4 the rounded rectangle at the bottom) returns a structured candidate list [{word: hernia, score:0.98, reason:"PinyinMatch"},...], which is then sent to the correction agent for downstream adoption according to the confidence threshold rule (≥0.9 / 0.6–0.9 / <0.5).

[0058] Figure 5 shows the signal-to-noise ratio robustness curve of the present invention, as shown in Figure 5 , the present invention achieves performance at four levels of signal-to-noise ratio (Original, 0dB, 5dB, Performance was compared with three representative baselines under 10dB conditions, including WhisperASR, Qwen3-ASR-1.7B, DeepSeekV3.2, and the present invention (solid line, marked Ours). Figure 5 (a) Precise entity recall curve: This invention achieves 98.81% under Original conditions and 98.79% at 0dB. At 5dB, it reached 93.11%, in extreme cases. Even at 10 dB, it still reached 80.72%; compared to the Whisper ASR baseline, it improved by 10.13, 11.46, 11.95, and 11.09 percentage points, respectively; from the original to Within the 5dB range, the curve of this invention is significantly higher than all baselines, demonstrating the effective repair capability of multi-agent closed-loop error correction for entity-level errors. Figure 5 (b) NDCG@3 curve for downstream RAG retrieval task: This invention achieves 0.8018 at 0dB, exceeding DeepSeekV3.2's 0.7502 and Qwen3-ASR-1.7B's 0.7580; even at Even under extreme conditions of 10 dB, it maintains a clinically usable level of 0.6552. Figure 5 bottom right corner The “10dB (harsh)” label is used to remind readers that this signal-to-noise ratio corresponds to extreme noise environments such as ambulances or disaster sites. In such environments, all methods are limited by the acoustic bottleneck of the ASR front end, but this invention still maintains the highest error correction performance, demonstrating its robustness in harsh clinical scenarios.

[0059] Figure 6 The above is a comparative ablation experiment of the present invention, such as Figure 6 As shown, to verify the independent contribution of each innovative module of the present invention, a system ablation comparison was performed on six configurations under a signal-to-noise ratio of 0dB. Figure 6 It consists of three parallel subgraphs, each corresponding to one of the three complementary evaluation dimensions: Figure 6 (a) is the exact match recall rate (%). Figure 6 (b) represents semantic similarity. Figure 6(c) NDCG@3 is retrieved downstream. The horizontal axis of each subgraph shows six configurations sequentially: coral bars represent the Full configuration of this invention (simultaneously enabling the diagnostic, skill, and supervision modules); light blue bars (A1, A2, A3) represent the progressive ablation group, which sequentially removes the skill library, supervisory agent, and diagnostic agent; light purple bars (A4, A5) represent the architecture baseline group, where A4 is the degenerate single-agent ReAct framework, and A5 is the pure Huatuo-7B baseline without any agent cooperation; the two groups are separated by a gray vertical dashed line. Observation Figure 6 (a) It can be seen that: removing the skill base (A1) caused the exact entity recall rate to drop from 98.79% to 93.85% (a decrease of 4.94 percentage points), which is the largest drop among all ablation variants, proving that the "skill scheduling mechanism + local knowledge base" proposed in this invention is the most critical source of performance improvement; removing the supervisory agent (A2) caused the recall rate to drop to 95.37%, proving the necessity of the closed-loop iterative verification mechanism; removing the diagnostic agent (A3) caused the smallest drop (96.09%). After degenerating into a single agent ReAct (A4), the recall rate dropped to 95.22%, proving the structural advantage of the multi-agent division of labor architecture described in this invention over the single-agent paradigm. Figure 6 (b) and Figure 6 (c) The consistent relative order in semantic similarity and downstream retrieval metrics further confirms the robustness of the above conclusions.

[0060] Implementation Method 3: This implementation method provides seven specific embodiments to illustrate specific embodiments of the invention. Example 1: End-to-end error correction process for typical homophone errors [Input] ASR transcribed text: "The patient complained of a fan-like sensation in the groin, which was more noticeable when standing."

S1 Initialize Shared State

[0061] [S2 Diagnostic Agent Output] {"has_error":true,"errors":[{"wrong_term":"fanning","context":"The patient complains of fanning in the groin, which is more pronounced when standing."}]} [S3 Skill Scheduling Phase 1] After reading the SkillHeaders, the model considers "fanning" to be a clear medical term with an ASR homophonic error (strongly related to the groin context), and outputs: {"tool_calls":[{"name":"search_medical_entity","args":{"wrong_term":"扇气","context":"..."}}]} [S4 Skill Scheduling Phase 2] After the model re-reads the complete SkillInstructions and ReferenceExamples (including the "over-correction" negative example), it still confirms to call search_medical_entity. The MCP server performs two-layer hybrid retrieval: Pinyin layer: φ("扇气")="shanqi" exactly matches the entity "疝气" in the pinyin index (φ("疝气")="shanqi"), unique match → score=0.98, reason="PinyinMatch".

[0062] Final candidate: [{"word":"疝气","score":0.98,"reason":"PinyinMatch"}].

[0063] [S6 Correction Agent] Perform replacement according to the adoption rule of score≥0.9 combined with the context (groin + aggravation when standing → highly consistent with hernia), and output: "The patient complained of having a hernia in the groin, which is obvious when standing." [S7 Supervision Agent] Review the difference before and after correction and confirm the rationality of the replacement, then output "OK", and the loop terminates.

[0064] [S8 Output] Final error-corrected text: "The patient complained of having a hernia in the groin, which is obvious when standing." Embodiment 2: Drug name glyph error (glyph layer match) [Input] "It is recommended that the patient receive subcutaneous injection of 千扰素 once a week for antiviral treatment." [Diagnosis] Identify wrong_term="千扰素".

[0065] [Phase 1] Decide to call search_medical_entity("千扰素").

[0066] [Phase 2] Review and confirmation.

[0067] [Hybrid Retrieval] Pinyin layer: φ("千扰素")="qianraosu", there is no homophone entry in the pinyin index (the pinyin of "干扰素" is "ganraosu"), no match at the pinyin layer; Glyph layer: "interferon" and "thousand interferon" share 2 / 3 of the characters ("interferon", "succession"), and the first characters "dry" and "thousand" differ by only one stroke. difflibratio≈0.667≥0.6→Return {"word":"interferon","score":0.667,"reason":"FuzzyMatch"}; [Correction] The score is in the middle range of [0.6, 0.9). Considering the context of "subcutaneous injection + antiviral treatment," the adoption is strongly supported. The output should be: "It is recommended that the patient receive subcutaneous injections of interferon once a week for antiviral treatment." Example 3: Batch Retrieval and Processing of Entire Medical Records [Input] "The patient has a history of high snow pressure and has been taking enalapril mariarate for a long time."

Diagnosis

[0068]

Phase 1

[0069] Example 4: A Negative Case—Preventing Over-Modification [Input] "The patient feels angry." (A general emotional description, not a pathological statement) [Diagnosis] "Angry" may be identified as a suspicious word;

Phase 1 Decision Reversal

[0070] [Contrast with the opposite] If a direct Pinyin search is performed for "anger," the tool will return high-score homophone candidates such as "kidney qi" and "hernia," leading to severe semantic distortion. The two-stage scheduling mechanism of this invention moves the "trigger judgment" to Phase 1, avoiding this type of over-correction. This is a key feature that distinguishes this invention from conventional indiscriminate RAG pipelines and is recommended as a key point for protection.

[0071] Example 5: Multi-round convergence of iterative closed loop When the Supervisor determines in round i that there is still a semantic difference between the current_text and the original_text (e.g., the Corrector incorrectly adopts a low-confidence candidate, leading to semantic inversion), the workflow returns to the Diagnostician and enters round i+1 of iteration, until the Supervisor outputs OK or the iteration limit N is reached. This mechanism degenerates into single-pass error correction when the supervisory agent is disabled (A2 ablation), and the precise entity recall rate drops from 98.79% to 95.37%, verifying the necessity of the closed-loop structure described in this invention.

[0072] Example 6: Parameter Values ​​and Optimal Range Based on multiple comparative experiments, the parameter values ​​shown in Table 2 are recommended as the preferred embodiments of the present invention: Table 2

[0073] Example 7: Hardware and Deployment Environment The present invention was evaluated under the following typical conditions: GPU: Single NVIDIA RTX 4090 (24GB VRAM); Local model: HuatuoGPT-o1-7B (torch_dtype=auto, device_map=auto); MCP service: Based on mcp.server.fastmcp.FastMCP, started locally via the stdio protocol; Medical Entities Database: A subset of CMeEE, a CSV file containing at least one "entity" column; Python primarily depends on: langgraph, langchain_mcp_adapters, pydantic, pandas, pypinyin, difflib, and transformers. Typical reasoning time for a single utterance: 12-23 seconds (depending on the number of iterations).

[0074] Implementation Method 4: The medical speech recognition text correction method based on multi-agent collaboration and skill scheduling described above can be fully implemented using computer software. Therefore, this implementation method provides a medical speech recognition text correction system based on multi-agent collaboration and skill scheduling.

[0075] Furthermore, in one specific embodiment, this embodiment provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the medical speech recognition text correction method based on multi-agent collaboration and skill scheduling described above.

[0076] Those skilled in the art will understand that implementing all or part of the processes in the above embodiments can be accomplished by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium can also include combinations of the above types of memory.

[0077] Furthermore, in one specific embodiment, this embodiment provides a computer device including a memory and a processor. The memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes the medical speech recognition text error correction method based on multi-agent collaboration and skill scheduling described above.

[0078] This embodiment provides a computer device. The hardware device in this part is a general model and is not shown in the figure. The system includes a processor and a memory, which can be connected by a bus or other means. The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs and modules, as well as corresponding program instructions / modules. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions and modules stored in the memory, so as to realize the steps in the above method.

[0079] The memory may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, mobile communication networks, and combinations thereof.

[0080] One or more modules are stored in the memory. When the processor executes, it performs the method steps in the embodiments. In this way, the invention objective can be achieved through the method, apparatus and process of the present invention. The specific details of the computer device described above can be understood by referring to the relevant descriptions and effects in the embodiments, and will not be repeated here.

[0081] The above description of the technical solution provided by the present invention through several specific embodiments is intended to highlight the advantages and benefits of the technical solution provided by the present invention. However, the above-described specific embodiments are not intended to limit the present invention. Any reasonable modifications and improvements to the present invention, reasonable combinations of implementation methods and equivalent substitutions based on the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A medical speech recognition text error correction method based on multi-agent collaboration and skill scheduling, characterized in that, The method includes the following steps: Step S1: Construct a shared state data unit for multiple agents, store the original speech recognition transcribed text to be corrected in the original text field and the current text field of the state data unit respectively, and initialize the iteration count parameter to zero; Step S2: Error Detection: The diagnostic agent reads the current text field and, together with the local large model with structured output constraints, identifies suspected erroneous words and their corresponding related information in the text, storing them in the error detection field of the state data unit; if no error is detected, the current text is directly output as the error correction result. Step S3: Level 1 Tool Decision: The skill scheduling agent reads the error information and the preset skill library, combines them to form a prompt message, and then inputs it into the local large model. The model uses reasoning logic to determine whether to call an external processing tool. If it is determined that no tool should be called, the correction candidate content is cleared and the process proceeds to the text correction step. Step S4: Secondary Decision Review: For the tool selected in the primary decision, generate prompt information based on the tool operation specifications and reference examples, and input it into the local large model to perform a secondary verification of the tool call decision; if the call is revoked after verification, record the log and proceed to the text correction step; if the call is retained after verification, perform a hybrid search. Step S5: Hybrid retrieval: Call the knowledge retrieval service to perform a two-layer hybrid retrieval of pinyin matching and character shape fuzzy matching, and store the retrieved structured correction candidate content into the correction candidate field of the status data unit; Step S6: Text Correction Step: The correction agent reads the current text and correction candidate content, replaces and repairs the erroneous content according to the preset reliability stratification rules, matching result priority rules, and combined with the semantic understanding ability of the local large model, and updates the corrected text to the current text domain. Step S7: Iterative Supervision: The supervising agent reads the original text, the updated current text, and the data processed in this round, and uses the local large model to verify the error correction effect; if the verification is successful, or the iteration count reaches the preset iteration limit, the final text is output; Otherwise, update the iteration count and return to the error detection step to perform a new round of error correction; Step S8: Output the result: Output the final current text in the state data unit as the error correction completed text.

2. The method according to claim 1, characterized in that, The diagnostic agent, skill scheduling agent, correction agent, and supervision agent communicate decoupledly through a globally shared agent state data unit. Each agent only reads and writes its own corresponding field and does not directly transmit intermediate data, thus achieving logical isolation. These four elements constitute a closed-loop iterative chain of diagnostic testing, skills review, text repair, and quality verification. If the verification fails, the iteration is restarted, and the iteration process is terminated by controlling the preset iteration limit. Each intelligent agent has a dedicated division of labor and independent context and prompt word configuration, corresponding to a single function such as error location, tool scheduling, text repair, and quality judgment. The method supports modular switch configuration, allowing selective disabling of corresponding functional modules and switching to various ablation deployment architectures such as complete error correction, skill-free retrieval, unsupervised verification, diagnostic detection, single agent error correction, and pure large model direct correction.

3. The method according to claim 1, characterized in that, The tool invocation determination adopts a two-level decision-making mechanism; In the first-level decision-making stage, the model outputs structured call decision results based on a concise skill list containing skill descriptions and a list of suspected erroneous words, which can directly skip the retrieval process; In the second-level review stage, the tools selected in the first-level decision are verified a second time by combining complete skill operation specifications and positive and negative reference cases. If the call is revoked, it is judged as a decision reversal and the log is recorded and the search is skipped. Only when both levels of decision retain the call instruction will the mixed search operation be performed.

4. The method according to claim 1, characterized in that, The hybrid search adopts a two-layer search mechanism of precise pinyin matching and fuzzy character shape matching; The pinyin precise matching layer maps each entity in the medical entity database from Chinese characters to pinyin to obtain pinyin strings, and groups entities with the same pinyin into the same index bucket; After performing the same pinyin mapping on suspected erroneous words, the index is matched, and corresponding scores are assigned and the pinyin matching type is marked according to whether the match is unique or multiple entity matches. The character shape fuzzy matching layer calculates the similarity of character sequences using a character sequence similarity algorithm, filters a specified number of similar candidates based on a preset threshold, determines the corresponding score based on the similarity value, and marks the fuzzy matching type. The two-layer search results are integrated into a unified candidate list in descending order of score. When the same erroneous word hits both layers of candidate results, the candidate result corresponding to the pinyin matching is adopted first. The optimal candidate is selected by a weighted calculation method that combines pinyin similarity score, character shape similarity score and preset weight hyperparameters. The weight hyperparameters are adjusted to adapt to error correction scenarios where homophone errors are the main problem in speech recognition.

5. The method according to claim 1, characterized in that, The confidence level stratification rule is as follows: when the confidence level is greater than or equal to 0.9, the corresponding candidate content is directly adopted; when the confidence level is between 0.6 and 0.9, the candidate content is selected in combination with the context; when the confidence level is less than 0.5, the original text is retained; the matching result priority rule is as follows: when the same incorrect word hits both the pinyin matching and character shape matching results, the candidate content corresponding to the pinyin matching is selected first.

6. The method according to claim 1, characterized in that, All external tools and knowledge bases are encapsulated in the form of agent skills and employ a three-layer hierarchical loading mechanism. The first layer is the skill introduction layer, which includes the skill name and a brief description. This layer is loaded only during the first-level tool decision-making stage and is used by the model to quickly identify available tools. The second layer is the skill operation layer, which includes detailed operational specifications for data contracts, triggering logic, search strategies, confidence thresholds, result verification, and input cleaning. The third layer is the skill example layer, which includes positive invocation cases and negative error cases. The content of the second and third layers is loaded only during the secondary decision review stage for tools selected by the first-level decision.

7. The method according to claim 1, characterized in that, The four-agent collaboration process is modeled using a LangGraph state graph and uses a global Pydantic state object as the only data interaction path for each agent node. The global state object is configured with an original text field, a current text field, an error detection field, a correction candidate field, an iteration count field, and a review result field, which respectively store the original identified text, the intermediate text of the iteration, the suspected error information, the retrieval correction candidate, the iteration round, and the supervision and verification result.

8. A medical speech recognition and text correction system based on multi-agent collaboration and skill scheduling, characterized in that, The system is used to implement the medical speech recognition text error correction method based on multi-agent collaboration and skill scheduling as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the medical speech recognition text correction method based on multi-agent collaboration and skill scheduling as described in any one of claims 1-7.

10. A computer device, characterized in that, The device includes a memory and a processor. The memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes the medical speech recognition text error correction method based on multi-agent collaboration and skill scheduling as described in any one of claims 1-7.

Images