A medical auxiliary diagnosis system based on a medical large model and speech recognition

Abstract

This invention provides a medical auxiliary diagnostic system based on a large medical model and speech recognition, including a doctor's workstation deployed on hospital terminal equipment for receiving and displaying diagnostic reports and draft medical records for doctors to review and confirm before submitting to the hospital information system (HIS); an AI doctor client installed on the same terminal device as the doctor's workstation for real-time acquisition of doctor-patient dialogue audio, displaying system-generated follow-up questions and auxiliary prompts, displaying draft medical records and diagnostic reports, and communicating with the AI ​​server and the hospital data interaction server; and a hospital data interaction server, deployed independently in each hospital, for establishing a WebSocket communication channel to achieve real-time data transmission between the AI ​​doctor client and the doctor's workstation within the hospital, forwarding diagnostic reports, draft medical records, and doctor instructions. This system reduces the workload of doctors in data collection, recording, and analysis, and overcomes the shortcomings of existing technologies that rely entirely on manual labor.

Description

Technical Field

[0001] This invention belongs to the field of AI, and in particular relates to a medical auxiliary diagnostic system based on a large medical model and speech recognition. Background Technology

[0002] In modern medical practice, doctors' workload and stress levels continue to rise. Doctors need to simultaneously perform several core tasks: Doctor-patient communication and consultation; Manually record key patient information; Diagnostic considerations and differential diagnosis; Treatment plan development; After the consultation, the information will be entered into the hospital information system (HIS).

[0003] This multitasking work mode can easily lead to doctors' attention being distracted, which not only reduces work efficiency but may also cause them to miss key symptoms, medical history and other information about patients, indirectly affecting the quality of diagnosis. Summary of the Invention

[0004] The purpose of this embodiment is to provide a medical auxiliary diagnostic system based on a large medical model and speech recognition, which can solve the problem of heavy workload for doctors in data collection, recording, and analysis.

[0005] A medical auxiliary diagnostic system based on a large medical model and speech recognition includes: The doctor workstation is deployed on the hospital's terminal equipment, which is the hospital's existing equipment. It is used to receive and display diagnostic reports and draft medical records for doctors to review and confirm before submitting them to the hospital information system (HIS). The AI ​​doctor client is installed on the same terminal device as the doctor's workstation. It is used to collect audio of doctor-patient dialogue in real time, display follow-up questions and auxiliary prompts generated by the system, display medical record drafts and diagnostic reports, and communicate with the AI ​​server and the hospital data interaction server. Each hospital has its own data interaction server, which is used to establish a WebSocket communication channel to enable real-time data transmission between the AI ​​doctor client and the doctor workstation within the hospital, and to forward diagnostic reports, draft medical records and doctor instructions. The AI ​​server is a core server shared by all hospitals. It establishes a communication connection with the AI ​​doctor client in each hospital to receive audio data and dialogue text data. Through a large medical model, it performs real-time reasoning and analysis to generate auxiliary information such as follow-up questions, draft medical records, and diagnostic reports, and feeds it back to the corresponding AI doctor client. The AI ​​server includes: A multimodal fusion engine is used to jointly encode text data generated by speech recognition with speech feature vectors, and generate a fused semantic vector sequence through a cross-modal attention mechanism; The patient status management service is used to maintain a patient status model during the consultation process. The patient status model includes demographic characteristics, chief complaint, symptom set, medical history, differential diagnosis list and diagnosis probability distribution. It is updated in real time as the conversation progresses using an incremental update strategy. The incremental update strategy involves parsing medical information in new conversation segments and updating the values ​​of the corresponding dimensions in the patient status model.

[0006] Furthermore, the AI ​​doctor client includes: The audio acquisition module is used to acquire audio of doctor-patient dialogue in real time, compress the audio using the Opus encoding format, and upload the audio data to the AI ​​server after segmenting the audio data through a secure and encrypted communication protocol. The WebSocket communication module is used to establish a two-way communication connection with the hospital's data interaction server to realize the real-time transmission of medical records, diagnostic suggestions, and doctor's instructions. The interface display module is used to visually display the intelligent follow-up suggestions, real-time medical record previews, and diagnostic prompts generated by the system. The visual display formats include displaying follow-up suggestions in list and bubble formats, and dynamically displaying structured medical record drafts.

[0007] Furthermore, AI servers also include: The speech recognition service, based on the Whisper model and deployed as a gRPC service, is used to decode the received Opus-encoded audio data into PCM format and then transcribe it into text data in real time, generating speech feature vectors including speech rate, pause duration, and energy intensity. The medical large-scale model reasoning service, based on a large-scale pre-trained model with the Transformer architecture, is used to perform medical entity recognition, semantic understanding, and clinical knowledge reasoning based on the fused semantic vectors, and generate follow-up question suggestions, medical record drafts, and diagnostic reports.

[0008] Furthermore, the medical large-scale model inference service includes: The medical condition information extraction unit is based on the BERT-BiLSTM-CRF model to realize medical entity recognition and on the Relational Attention Network (RAN) to extract entity relations. It is used to identify patient chief complaints, present medical history, past medical history and symptom description information, and to establish a semantic relation graph between entities. The intelligent follow-up question generation unit is used to generate follow-up questions that conform to clinical diagnosis and treatment logic based on the acquired medical information and medical knowledge base; The differential diagnosis analysis unit is used to generate a list of differential diagnoses and their basis in real time based on the patient's symptoms, signs and medical history. The medical record structure generation unit is used to automatically map the identified entities and their relationships to the corresponding fields of the preset structured medical record template, and generate a structured medical record draft. The medical record template is defined using JSON Schema, and the mapping rules are implemented based on a hybrid rule engine and machine learning. The diagnostic and treatment suggestion output unit is used to generate preliminary diagnoses, laboratory test recommendations, and treatment plans by combining the latest medical guidelines.

[0009] Furthermore, the hospital data interaction server runs a WebSocket server program, which enables real-time data transmission between the AI ​​doctor client and the doctor workstation within the hospital through a channel subscription mechanism, either one-to-one or one-to-many. The hospital data interaction server is deployed only on the hospital's intranet and is isolated from the external network.

[0010] Furthermore, the doctor's workstation sends patient registration information to the hospital's data interaction server through the hospital's internal network, and receives medical record reports forwarded by the hospital's data interaction server through a lightweight modification. The report data is automatically populated into the corresponding entry interface of the hospital information system, and the doctor can complete the final submission with one click.

[0011] Furthermore, the patient status management service is implemented using the Go language, and the patient status model is defined as a Protobuf structure; status updates adopt an event sourcing mode, and each status change is recorded as an event for easy tracing and rollback.

[0012] Furthermore, the multimodal fusion engine uses Go language to call Python inference services, and transmits text data and speech feature vectors through gRPC streaming interface. It then jointly encodes the text token sequence and speech feature vectors to generate a fused semantic vector.

[0013] Furthermore, the AI ​​server's large medical model is a deep learning model pre-trained on a large-scale medical corpus, possessing the ability to recognize medical entities, understand semantics, and reason about clinical knowledge. It employs a chain-of-thought prompting process to perform progressive diagnostic reasoning.

[0014] Furthermore, the AI ​​server-side medical model has the ability to dynamically update knowledge, regularly using the latest medical literature, treatment guidelines, expert consensus and other corpora for incremental training or fine-tuning. It supports the coexistence of multiple model versions and can flexibly switch or release new models in a canary release. After the new model is deployed, all real-time inference requests of the AI ​​doctor client will automatically use the latest model without the need for client upgrades or restarts.

[0015] This invention provides a medical auxiliary diagnostic system based on a large medical model and speech recognition, which solves the problem of heavy workload for doctors in data collection, recording, and analysis; and overcomes the shortcomings of existing technologies that rely entirely on manual labor.

[0016] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] Figure 1 : A structural diagram of a medical auxiliary diagnostic system based on a large medical model and speech recognition provided in an embodiment of the present invention; Figure 2: A flowchart of medical auxiliary diagnosis and consultation based on medical big data model and speech recognition provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the present invention will now be described with reference to the accompanying drawings in the embodiments of the present invention.

[0019] In response to the fundamental shortcomings of existing technologies, which rely entirely on manual labor and lack intelligent assistance, this invention aims to construct a deeply intelligent medical auxiliary diagnostic system to achieve full-process AI assistance: deeply integrating AI technology into the entire consultation process, effectively sharing the doctor's work in information collection, recording, and analysis, and significantly reducing the doctor's workload; This embodiment provides a medical auxiliary diagnostic system based on a large medical model and speech recognition (see...). Figure 1 It adopts a distributed architecture design, which specifically includes the following four components: 1. Doctor's workstation: The office computer deployed in each hospital. It is an existing hospital device used to receive and display diagnostic reports and draft medical records for doctors to review and confirm before submitting them to the Hospital Information System (HIS). 2. AI Doctor Client: Installed on the same terminal device as the doctor's workstation, it is used to collect audio of doctor-patient dialogue in real time, display follow-up questions and auxiliary prompts generated by the system, display medical record drafts and diagnostic reports, and communicate with the AI ​​server and data interaction server. 3. Hospital Data Interaction Server: One server is deployed independently in each hospital to establish a WebSocket communication channel, enabling real-time data transmission between the AI ​​doctor client and the doctor's workstation within the hospital, including forwarding diagnostic reports, draft medical records, and doctor instructions; 4. AI Server: This is a core server shared by all hospitals. It establishes communication connections with the AI ​​doctor clients of each hospital to receive audio data and dialogue text data. Through a large medical model, it performs real-time reasoning and analysis to generate auxiliary information such as follow-up questions, draft medical records, and diagnostic reports, and then feeds it back to the corresponding AI doctor clients.

[0020] The system in question includes the following specific operating procedures: 2.1 Initialization Phase: After a doctor sees a patient, both the doctor's workstation and the AI ​​doctor client are activated simultaneously. The AI ​​doctor client automatically establishes a WebSocket connection with the data interaction server deployed within the hospital and joins the corresponding communication channel. The doctor's workstation sends the patient's registration information, including but not limited to basic information such as the patient's name, gender, age, and department, to the data interaction server via the hospital's internal network. The data interaction server forwards this information to the AI ​​doctor client via the WebSocket channel. The AI ​​doctor client parses the information to obtain the patient's basic information (e.g., "male, 65 years old") and uses it as initial context information for subsequent personalized analysis in the large-scale medical model.

[0021] 2.2 Real-time voice acquisition and transmission stage: The AI ​​doctor client activates its local recording function to capture real-time audio of doctor-patient conversations. During the capture process, the AI ​​doctor client segments and compresses the audio data before uploading it to the AI ​​server in real-time via a secure and encrypted communication protocol. The uploaded audio data includes the doctor's questions and the patient's responses.

[0022] 2.3 Real-time speech recognition and medical large-scale model inference stage: After the AI ​​server receives the audio data, the workflow is as follows: Figure 2 First, the built-in speech recognition module transcribes the audio into text data in real time. Then, the text data is input into a large-scale medical model for real-time reasoning and analysis. This large-scale medical model is a deep learning model pre-trained on a large-scale medical corpus, possessing capabilities for medical entity recognition, semantic understanding, and clinical knowledge reasoning. Based on the current dialogue context, the AI ​​server performs the following analysis tasks: Patient information extraction: Identifying key information such as the patient's chief complaint, present illness, past medical history, and symptom description; including the following steps: S1: Receives Opus-encoded audio data uploaded by the AI ​​doctor client, decodes it into PCM format, and then calls the speech recognition service for real-time transcription. The speech recognition service is based on the Whisper large-scale model, deployed as a gRPC service, supports streaming speech recognition, and returns text segments and speech feature vectors (including speech rate, pause duration, energy intensity, etc.) in real time.

[0023] S2: The multimodal fusion layer of the large medical model inputs the text stream and speech feature vectors. This fusion layer is implemented based on the cross-modal attention mechanism, using Go to call Python inference services and transmitting data via the gRPC streaming interface. The fusion layer jointly encodes the text token sequence and speech feature vectors to generate a fused semantic vector sequence.

[0024] / / Example of a multimodal fusion client implemented in Go func (s *AIServer) processAudioStream(stream pb.SpeechRecognition_RecognizeStream) { for { audioChunk, err := stream.Recv() if err == io.EOF { break } / / Call the Whisper service for speech recognition textResult := s.whisperClient.Transcribe(audioChunk.Data) / / Extract speech features acousticFeatures := s.extractAcousticFeatures(audioChunk.Data) / / Calling a large medical model for multimodal fusion inference fusionResult := s.medicalLLMClient.MultimodalInference(textResult, acousticFeatures) / / Return fusion analysis results stream.Send(&pb.AnalysisResponse{Result: fusionResult}) } } S3: The large-scale medical model is based on fused semantic vectors to identify medical entities in dialogues in real time. Entity recognition adopts a BiLSTM-CRF architecture, combined with a medical-specific BERT-BiLSTM-CRF model, to identify entity types such as symptoms, signs, diseases, drugs, and examination items. Entity relation extraction uses a Relational Attention Network (RAN) to build a semantic relation graph between entities.

[0025] S4: Automatically maps identified entities and their relationships to corresponding fields in a pre-defined structured medical record template. The medical record template is defined using JSON Schema, and the mapping rules are implemented using a hybrid approach of a rule engine and machine learning. The mapping service, implemented in Go, converts the entity relationship graph into a standardized JSON structure for medical records.

[0026] go / / Example of structured medical record mapping type MedicalEntity struct { Type string `json:"type"` / / symptom, disease, drug, exam, etc. Text string `json:"text"` / / Entity text Offset int `json:"offset"` / / Position in the dialog } type Relation struct { Subject string `json:"subject"` / / Subject entity ID Object string `json:"object"` / / Object entity ID Predicate string `json:"predicate"` / / Relationship type: has_symptom, located_at, etc. } func MapToEMR(entities []MedicalEntity, relations []Relation) *EMRStructure { emr := &EMRStructure{ ChiefComplaint: extractChiefComplaint(entities), PresentIllness: buildPresentIllness(entities, relations), PastHistory: extractPastHistory(entities), } return emr } The AI ​​server also includes the following execution steps: B1: Initialize the patient state model. The state model is defined as a Protobuf structure, containing the following dimensions: protobuf message PatientState { string patient_id = 1; Demographic demographic = 2; / / Age, gender, etc. string chief_complaint = 3; / / Chief complaint Repeated Symptom Symptoms = 4; / / Symptom List Repeated Disease past_diseases = 5; / / Past history repeated string differential_diagnosis = 6; / / List of differential diagnoses map<string, float> diagnosis_probability = 7; / / Diagnostic probability distribution repeated string information_gaps = 8; / / Information gaps int64 last_update_time = 9; } B2: Upon receiving a new dialogue fragment, the Go-implemented state update service parses the medical information contained within and updates the patient state model using an incremental update strategy. The update logic is based on an event sourcing pattern, where each state change is recorded as an event for easy tracing and rollback.

[0027] func (s *StateService) UpdateState(ctx context.Context, patientIDstring, utterance *Utterance) error { / / Get the current state state, err := s.getState(patientID) if err != nil { return err } / / Analyzing medical information in new discourse entities := s.extractEntities(utterance.Text) / / Incremental update status for _, entity := range entities { switch entity.Type { case "symptom": state.Symptoms = append(state.Symptoms, &Symptom{ Name: entity.Text, OnsetTime: extractOnsetTime(utterance.Text), }) case "disease": state.PastDiseases = append(state.PastDiseases, &Disease{ Name: entity.Text, DiagnosedTime: extractDiagnosedTime(utterance.Text), }) } } / / Save the updated state return s.saveState(patientID, state) } B3: Based on the updated patient state model, the large medical model performs progressive diagnostic reasoning. The reasoning service uses chain-of-thought prompting engineering to guide the model through step-by-step reasoning. The client calls the Python reasoning service via gRPC, passing the current state JSON and receiving the reasoning results.

[0028] # Example of Python code for large-scale medical model inference def progressive_diagnosis(state_json): prompt = f""" Progressive diagnostic reasoning based on the following patient information: Patient information: {state_json} Please deduce the following steps: 1. List the possible diseases corresponding to the current symptoms. 2. Calculate the probability of each disease. 3. Identify the current information gaps (information that needs to be followed up). 4. Generate suggestions for the next follow-up question. """ response = llm.generate(prompt, temperature=0.3) return parse_reasoning_result(response) B4: The inference results are pushed to the AI ​​doctor client in real time via WebSocket. The push service implemented in Go uses a fan-out pattern to push the results to multiple terminal devices of the doctor simultaneously.

[0029] Intelligent follow-up question generation: Based on the acquired information and combined with the medical knowledge base, generate follow-up questions that conform to the logic of clinical diagnosis and treatment, such as "When did the upper abdominal pain begin? Is there a clear cause?"; Differential diagnosis analysis: Based on the patient's symptoms, signs and medical history, a list of differential diagnoses and their basis are generated in real time; Medical record structure generation: Automatically fill the corresponding fields of the standard medical record template with the information extracted from the dialogue to generate a structured medical record draft; Diagnostic and treatment recommendations: Based on the latest medical guidelines, generate preliminary diagnoses, laboratory test recommendations, and treatment plans.

[0030] The above reasoning results are packaged in real time by the AI ​​server and returned to the corresponding AI doctor client via the WebSocket protocol.

[0031] 2.4 Real-time auxiliary information display stage After receiving the inference results returned by the AI ​​server, the AI ​​doctor client displays them to the doctor in a visual manner on the local interface, specifically including: Intelligent follow-up question suggestion area: Displays system-generated follow-up questions in list or bubble format for doctors to refer to or directly click to use; Real-time medical record preview area: Dynamically displays the generated structured medical record drafts, updating in real time as the conversation progresses; Diagnostic prompts area: Displays auxiliary information such as differential diagnosis, preliminary diagnosis, laboratory test recommendations, and treatment plans.

[0032] Doctors can freely choose to use system prompts or follow their personal diagnostic approach based on their own clinical judgment, and the system can identify and update the analysis results in real time.

[0033] 2.5 Medical Record Review and Confirmation Stage After the consultation, the AI ​​doctor client automatically generates a complete structured medical record draft and diagnostic report. The doctor reviews, supplements, or modifies the draft on the AI ​​doctor client interface. Once confirmed, the doctor clicks the "Submit" button. The AI ​​doctor client then sends the final confirmed medical record report to the hospital's data exchange server via a WebSocket channel.

[0034] 2.6 Data synchronization to HIS stage After receiving the medical record report, the hospital's data exchange server forwards it to the doctor's workstation via an internal interface. The doctor's workstation automatically populates the report data into the corresponding entry interface of the hospital information system. The doctor only needs to confirm with one click to complete the final submission, achieving seamless integration with the HIS system.

[0035] III. Specific Implementation of Distributed Communication Link In this embodiment, the core design of the communication link lies in achieving multi-system compatibility through "AI Doctor Client—Data Interaction Server—Doctor Workstation". The specific implementation is as follows: WebSocket service deployment: Each hospital independently deploys a data interaction server, which runs the WebSocket server program and is responsible for the connection management between all AI doctor clients and doctor workstations within the hospital; Channel subscription mechanism: After the AI ​​doctor client and the doctor workstation are started, they both subscribe to the same channel (such as the department channel or the doctor's personal channel) from the data interaction server to achieve one-to-one or one-to-many real-time communication. Data forwarding mechanism: The medical record report generated by the AI ​​doctor client is sent to the data interaction server via WebSocket. The server forwards the data to the corresponding doctor's workstation according to the channel subscription relationship. Security Guarantee: The data interaction server is deployed only within the hospital's intranet and is isolated from the external network; communication between the AI ​​doctor client and the AI ​​server uses encrypted transmission to ensure the security of medical data.

[0036] IV. Knowledge Update Mechanism of Medical Big Model In this embodiment, the knowledge base of the large medical model has dynamic update capabilities. The specific implementation method is as follows: Regularly iteratively train the model: The large medical model deployed on the AI ​​server is regularly (e.g., monthly or quarterly) incrementally trained or fine-tuned using the latest medical literature, treatment guidelines, expert consensus and other corpora to ensure that the model knowledge is up-to-date; Model version management: The AI ​​server supports the coexistence of multiple model versions, and new models can be flexibly switched or released in a canary manner according to actual needs; Real-time knowledge reasoning: After the new model is deployed, all real-time reasoning requests from AI doctor clients will automatically use the latest model without requiring client upgrades or restarts, ensuring that doctors receive the latest knowledge support during consultations.

[0037] This embodiment provides a medical auxiliary diagnostic system based on a large medical model and speech recognition, which solves the problem of heavy workload for doctors in information collection, recording, and analysis; and overcomes the fundamental defects of existing technologies that rely entirely on manual labor and lack intelligent assistance.

[0038] The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A medical auxiliary diagnostic system based on a large medical model and speech recognition, characterized in that, include: The doctor workstation is deployed on the hospital's terminal equipment, which is the hospital's existing equipment. It is used to receive and display diagnostic reports and draft medical records for doctors to review and confirm before submitting them to the hospital information system (HIS). The AI ​​doctor client is installed on the same terminal device as the doctor's workstation. It is used to collect audio of doctor-patient dialogue in real time, display follow-up questions and auxiliary prompts generated by the system, display medical record drafts and diagnostic reports, and communicate with the AI ​​server and the hospital data interaction server. Each hospital has its own data interaction server, which is used to establish a WebSocket communication channel to enable real-time data transmission between the AI ​​doctor client and the doctor workstation within the hospital, and to forward diagnostic reports, draft medical records and doctor instructions. The AI ​​server is a core server shared by all hospitals. It establishes a communication connection with the AI ​​doctor client in each hospital to receive audio data and dialogue text data. Through a large medical model, it performs real-time reasoning and analysis to generate auxiliary information such as follow-up questions, draft medical records, and diagnostic reports, and feeds it back to the corresponding AI doctor client. The AI ​​server includes: A multimodal fusion engine is used to jointly encode text data generated by speech recognition with speech feature vectors, and generate a fused semantic vector sequence through a cross-modal attention mechanism; The patient status management service is used to maintain a patient status model during the consultation process. The patient status model includes demographic characteristics, chief complaint, symptom set, medical history, differential diagnosis list and diagnosis probability distribution. It is updated in real time as the conversation progresses using an incremental update strategy. The incremental update strategy involves parsing medical information in new conversation segments and updating the values ​​of the corresponding dimensions in the patient status model.

2. The medical auxiliary diagnostic system based on medical large model and speech recognition according to claim 1, characterized in that, The AI ​​doctor client includes: The audio acquisition module is used to acquire audio of doctor-patient dialogue in real time, compress the audio using the Opus encoding format, and upload the audio data to the AI ​​server after segmenting the audio data through a secure and encrypted communication protocol. The WebSocket communication module is used to establish a two-way communication connection with the hospital's data interaction server to realize the real-time transmission of medical records, diagnostic suggestions, and doctor's instructions. The interface display module is used to visually display the intelligent follow-up suggestions, real-time medical record previews, and diagnostic prompts generated by the system. The visual display formats include displaying follow-up suggestions in list and bubble formats, and dynamically displaying structured medical record drafts.

3. The medical auxiliary diagnostic system based on a large medical model and speech recognition according to claim 1, characterized in that, The AI ​​server also includes: The speech recognition service, based on the Whisper model and deployed as a gRPC service, is used to decode the received Opus-encoded audio data into PCM format and then transcribe it into text data in real time, generating speech feature vectors including speech rate, pause duration, and energy intensity. The medical large-scale model reasoning service, based on a large-scale pre-trained model with the Transformer architecture, is used to perform medical entity recognition, semantic understanding, and clinical knowledge reasoning based on the fused semantic vectors, and generate follow-up question suggestions, medical record drafts, and diagnostic reports.

4. The medical auxiliary diagnostic system based on medical large model and speech recognition according to claim 3, characterized in that, The medical large-scale model inference service includes: The medical condition information extraction unit is based on the BERT-BiLSTM-CRF model to realize medical entity recognition and on the Relational Attention Network (RAN) to extract entity relations. It is used to identify patient chief complaints, present medical history, past medical history and symptom description information, and to establish a semantic relation graph between entities. The intelligent follow-up question generation unit is used to generate follow-up questions that conform to clinical diagnosis and treatment logic based on the acquired medical information and medical knowledge base; The differential diagnosis analysis unit is used to generate a list of differential diagnoses and their basis in real time based on the patient's symptoms, signs and medical history. The medical record structure generation unit is used to automatically map the identified entities and their relationships to the corresponding fields of the preset structured medical record template, and generate a structured medical record draft. The medical record template is defined using JSON Schema, and the mapping rules are implemented based on a hybrid rule engine and machine learning. The diagnostic and treatment suggestion output unit is used to generate preliminary diagnoses, laboratory test recommendations, and treatment plans by combining the latest medical guidelines.

5. The medical auxiliary diagnostic system based on a large medical model and speech recognition according to claim 1, characterized in that, The hospital data interaction server runs a WebSocket server program, which enables real-time data transmission between the AI ​​doctor client and the doctor workstation within the hospital in a one-to-one or one-to-many manner through a channel subscription mechanism; the hospital data interaction server is deployed only on the hospital intranet and is isolated from the external network.

6. The medical auxiliary diagnostic system based on medical large model and speech recognition according to claim 1, characterized in that, The doctor's workstation sends patient registration information to the hospital's data interaction server through the hospital's internal network. With a lightweight modification, it receives medical record reports forwarded by the hospital's data interaction server, automatically populates the report data into the corresponding entry interface of the hospital information system, and the doctor can complete the final submission with one click.

7. The medical auxiliary diagnostic system based on a large medical model and speech recognition according to claim 1, characterized in that, The patient status management service is implemented in Go language, and the patient status model is defined as a Protobuf structure. Status updates adopt an event sourcing mode, and each status change is recorded as an event for easy tracing and rollback.

8. The medical auxiliary diagnostic system based on medical large model and speech recognition according to claim 1, characterized in that, The multimodal fusion engine uses Go language to call Python inference services, and transmits text data and speech feature vectors through gRPC streaming interface. It then jointly encodes the text token sequence and speech feature vectors to generate a fused semantic vector.

9. The medical auxiliary diagnostic system based on a large medical model and speech recognition according to claim 1, characterized in that, The AI ​​server's medical big model is a deep learning model pre-trained on a large-scale medical corpus, possessing the ability to recognize medical entities, understand semantics, and reason about clinical knowledge. It uses a chain-of-thought prompting process to perform progressive diagnostic reasoning.

10. The medical auxiliary diagnostic system based on medical large model and speech recognition according to claim 1, characterized in that, The AI ​​server-side medical model has dynamic knowledge update capabilities, regularly using the latest medical literature, treatment guidelines, expert consensus and other corpora for incremental training or fine-tuning. It supports the coexistence of multiple model versions and can flexibly switch or release new models in a phased manner. After the new model is deployed, all AI doctor clients automatically use the latest model for real-time inference requests without requiring client upgrades or restarts.

Images