A voice-controlled ultrasound diagnostic system and its usage method
By combining voice control and gesture recognition technology with medical knowledge graphs and intelligent resource scheduling, the complexity and safety issues of traditional ultrasound diagnostic instruments have been resolved, enabling efficient, safe, aseptic operation and remote consultation.
Patent Information
- Application Number
- CN202510589563.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-08
- Publication Date
- 2026-11-10
AI Technical Summary
Traditional ultrasound diagnostic instruments are complex to operate, inefficient, have a high risk of cross-infection, poor environmental adaptability, insufficient intelligence, and delayed fault recovery, making it difficult to meet the needs of aseptic operation, mobile diagnosis and treatment, and remote consultation.
It employs voice control combined with gesture control, patient status perception, and intelligent resource scheduling technology, integrating speech recognition, noise reduction, gesture recognition, dynamic encrypted communication, and fault protection modules, and combining medical knowledge graphs for command verification and resource optimization.
It improves the efficiency and safety of ultrasound examinations, reduces the delay in fault recovery, ensures the continuity and compliance of diagnosis and treatment, adapts to high humidity and high noise environments, and supports aseptic operation and remote consultation.
Smart Images

Figure CN123058768A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical device technology, specifically to a voice-controlled ultrasound diagnostic system and its usage method. Background Technology
[0002] Ultrasound diagnostic instruments, also known as ultrasound instruments, are medical instruments developed based on the principles of ultrasound. They are used in clinical medicine, diagnosis, and healthcare. The working principle of medical ultrasound examination is to emit ultrasound waves into the human body. When the waves encounter interfaces within the body, they are radiated and refracted. They may also be absorbed and attenuated in human tissues. Because the shapes and structures of various human tissues are different, their degrees of reflection, refraction, and absorption of ultrasound waves also differ. Medical personnel identify these differences by observing the waveforms, curves, or image characteristics reflected by the instrument. By combining this with anatomical knowledge and the distinction between normal and pathological changes, they can diagnose whether the examined organ is diseased.
[0003] Traditional ultrasound diagnostic instruments are complex to operate, requiring medical staff to possess certain professional skills. Moreover, ultrasound examinations suffer from the following shortcomings: low operational efficiency: doctors need to frequently touch the equipment, interrupting the examination process; risk of cross-infection: contact-based operation poses significant risks in aseptic settings; poor environmental adaptability: low voice recognition rate under noise interference, and complex commands are prone to misinterpretation; insufficient intelligence: lack of patient status linkage and multimodal interaction capabilities; delayed fault recovery: traditional methods take a long time to switch to backup mode, affecting the continuity of diagnosis and treatment. Summary of the Invention
[0004] The technical problem to be solved by the present invention is to overcome the existing defects and provide a voice-controlled ultrasound diagnostic system and its usage method, which is suitable for aseptic operation scenarios, mobile diagnosis and treatment and remote consultation environments. By integrating voice recognition, gesture control, patient status perception and intelligent resource scheduling technologies, the system significantly improves the efficiency and safety of ultrasound examinations, reduces fault recovery delays, and ensures continuity of diagnosis and treatment, and can effectively solve the problems in the background technology.
[0005] To achieve the above objectives, the present invention proposes: a voice-controlled ultrasound diagnostic system, comprising a voice acquisition device, a voice processing module, a communication module, an ultrasound host control unit, a fault protection module, a TOF sensing module, a patient status perception interface, an intelligent resource scheduling engine, and an operation tracing device;
[0006] The voice acquisition device is used to receive the operator's voice commands in real time and integrates a waterproof microphone array to adapt to high humidity environments;
[0007] The voice processing module is connected to the voice acquisition device, which includes a noise reduction unit and a command recognition unit.
[0008] The communication module uses the BLE 5.0 protocol to establish dynamically encrypted bidirectional communication with the ultrasound host.
[0009] The ultrasound host control unit includes an instruction verification module, an execution feedback module, and a device drive interface;
[0010] The fault protection module is used to monitor abnormalities in the voice system in real time and automatically switch to the traditional operation mode of the ultrasound host.
[0011] The TOF sensing module integrates a gesture recognition algorithm, supporting gesture-based collaborative control in aseptic scenarios.
[0012] The patient status sensing interface is connected to the vital signs monitor and filters non-emergency commands based on physiological parameters.
[0013] The intelligent resource scheduling engine dynamically allocates edge computing and cloud computing resources based on instruction complexity;
[0014] The operation tracing device generates a structured log that conforms to the DICOM-SR standard.
[0015] As a preferred technical solution of the present invention: the noise reduction unit of the speech processing module adopts an adaptive filter combined with RNN noise modeling to suppress 3-5kHz electromagnetic interference. The instruction recognition unit of the speech processing module has a built-in professional medical dictionary, which includes 38 core instructions such as frozen image, measurement mark, gain adjustment and 200+ extended instructions. It supports the parsing of mixed Chinese and English instructions, and the parsing accuracy is ≥98.6%.
[0016] As a preferred technical solution of the present invention: the communication module generates an independent session key for each connection and uses a dynamic encryption protocol to ensure data transmission security. The instruction verification module of the ultrasound host control unit performs semantic logic verification through a medical knowledge graph. The knowledge graph contains three levels of dynamic association relationships and accesses the latest clinical guidelines and equipment parameter database in real time.
[0017] As a preferred technical solution of the present invention: the anomaly detection unit of the fault protection module includes voice recognition accuracy monitoring, response delay monitoring and resource occupation alarm, and its triggering conditions are: five consecutive recognition errors trigger a first-level warning; instruction processing exceeds 800ms to start a second-level protection; and a forced restart is performed when memory usage is >85%.
[0018] As a preferred technical solution of the present invention: the priority logic of the execution feedback module is as follows: emergency commands trigger red LED flashing and high-frequency prompt sound; warning commands are combined with yellow LED and screen pop-up; normal operations use green LED and voice broadcast.
[0019] As a preferred technical solution of the present invention: the TOF sensing module supports the operation of freezing the image by clenching a fist and switching the probe mode by waving a hand. The gesture recognition algorithm adopts a lightweight CNN model with a response latency of <100ms, and the voice command has a higher priority than the gesture control.
[0020] As a preferred technical solution of the present invention: the intelligent resource scheduling engine ensures that the end-to-end latency of key instructions is <200ms through 5G slicing technology, simple instructions are allocated to edge computing units, and complex instructions call cloud GPU clusters for acceleration.
[0021] As a preferred technical solution of the present invention: the patient status sensing interface automatically blocks non-emergency commands when the patient's heart rate is >120 beats / min, and prioritizes the execution of fast storage and alarm commands when a critical value is detected.
[0022] As a preferred technical solution of the present invention: the log generated by the operation tracing device includes the original audio fingerprint, semantic parsing graph and device status snapshot, and supports the DICOM-SR standard to embed ultrasound reports.
[0023] A method for using a voice-controlled ultrasound diagnostic system includes the following steps:
[0024] S1): Acquisition of doctor's voice and gesture signals, with voice commands having higher priority than gesture control;
[0025] S2): Multimodal command processing is performed through the speech processing module, including noise reduction, semantic parsing and dynamic resource allocation;
[0026] S3): The ultrasound host control unit verifies the instruction logic through a medical knowledge graph;
[0027] S4): The ultrasound host control unit executes commands and generates multimodal feedback, including image feedback and execution feedback;
[0028] S5): When the voice system malfunctions, the fault protection module ensures that it switches to the traditional operation mode within 20ms;
[0029] S6): The operation traceability device generates an encrypted operation log and uploads it to the cloud.
[0030] Compared with existing technologies, the advantages of this invention are: seamless integration of voice and gesture commands to meet the needs of aseptic operation; real-time access to clinical databases such as UpToDate via API to support incremental learning and adaptation of new terminology, ensuring dynamic updates of the medical knowledge graph; and the use of a semantic understanding model with a medical knowledge graph as the core algorithm to improve the accuracy and robustness of command recognition; intelligent linkage with patient status, enabling optimization of command execution logic based on vital signs to improve the safety of diagnosis and treatment; waterproof design and anti-electromagnetic interference algorithms to ensure stability in high humidity and high noise environments; integration of voice recognition, gesture control, patient status perception, and intelligent resource scheduling technologies to significantly improve the efficiency and safety of ultrasound examinations; and full traceability of operation logs to meet medical compliance requirements and enhance interpretability. Attached Figure Description
[0031] Figure 1 This is a schematic diagram illustrating the steps of using the present invention;
[0032] Figure 2 This is a schematic diagram of the system architecture of the present invention;
[0033] Figure 3 This is a schematic diagram of the fault protection module switching process of the present invention;
[0034] Figure 4 This is a schematic diagram of the key instruction processing flow of the present invention;
[0035] Figure 5 This is a schematic diagram of the medical knowledge graph updating mechanism of the present invention;
[0036] Figure 6 This is a schematic diagram of the ultrasonic command response timing of the present invention. Detailed Implementation
[0037] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0038] Please see Figure 1-6 The present invention provides the following technical solutions:
[0039] A voice-controlled ultrasound diagnostic system includes a voice acquisition device, a voice processing module, a communication module, an ultrasound host control unit, a fault protection module, a TOF sensing module, a patient status perception interface, an intelligent resource scheduling engine, and an operation traceability device.
[0040] The voice acquisition device is used to receive the operator's voice commands in real time and integrates a waterproof microphone array to adapt to high humidity environments;
[0041] The voice acquisition device uses a wearable waterproof microphone array, such as ear-hook or lapel clip type. The microphone PCB board is coated with a nano-hydrophobic coating and integrates a humidity sensor. When the risk of condensation is detected, the heating and dehumidification module is automatically activated, supporting stable operation in high humidity environments.
[0042] The voice processing module is connected to the voice acquisition device, which includes a noise reduction unit and a command recognition unit.
[0043] The voice processing module uses an ARM Cortex-M7 processor for processing. The noise reduction unit of the voice processing module uses an adaptive filter combined with RNN noise modeling to suppress 3-5kHz electromagnetic interference. The command recognition unit of the voice processing module has a built-in professional medical dictionary, which includes 38 core commands such as frozen image, measurement mark, and gain adjustment, as well as 200+ extended commands. It supports the parsing of mixed Chinese and English commands with a parsing accuracy of ≥98.6%, ensuring that the doctor's commands are consistent with the professional medical dictionary.
[0044] The communication module uses the BLE 5.0 protocol to establish dynamically encrypted bidirectional communication with the ultrasound host;
[0045] Each connection of the communication module generates an independent session key and uses a dynamic encryption protocol to ensure data transmission security. It provides real-time feedback of device status to the voice processing terminal. The acquisition and processing unit pairs with the ultrasound host via Bluetooth 5.0. Bluetooth communication uses the AES-256 encryption algorithm. Each connection generates a unique session key, and the key lifecycle is synchronized with the single check session.
[0046] The ultrasound host control unit includes an instruction verification module, an execution feedback module, and a device driver interface;
[0047] The instruction verification module of the ultrasound host control unit performs semantic logic verification through a medical knowledge graph, such as verifying the compatibility between the probe type and the preset mode. The knowledge graph contains three levels of dynamic associations and is connected to the latest clinical guidelines and equipment parameter library in real time.
[0048] The execution feedback module can generate multimodal prompts including voice broadcasts, LED status lights (red / yellow / green), and screen pop-ups. The priority logic of the execution feedback module is as follows: emergency commands trigger red LED flashing and high-frequency prompts; warning commands are combined with yellow LEDs and screen pop-ups; and regular operations use green LEDs and voice broadcasts.
[0049] The medical knowledge graph of the ultrasound host control unit is built on a semantic understanding model. The semantic understanding model can perform speech-to-text, semantic parsing, and intent mapping. Speech-to-text uses the Wav2Vec 2.0 model, with an overlay of an ultrasound terminology adaptation layer to optimize the recognition of professional vocabulary. Semantic parsing is based on a graph neural network (CNN) that traverses the medical knowledge graph and dynamically corrects the command logic, such as automatically adjusting "depth 25cm" to the maximum probe value of 20cm. Intent mapping optimizes the accuracy of command mapping through a reinforcement learning model and supports contextual association, such as automatically associating "measure the largest nodule" with the current frozen image.
[0050] The medical knowledge graph automatically synchronizes the latest clinical guidelines and equipment parameters daily. New terms are injected into the semantic parsing model (within the core knowledge base) through offline training using an incremental learning engine, without requiring downtime for updates. It adopts a semantic understanding model with a medical knowledge graph as the core algorithm, which improves the accuracy and robustness of instruction recognition.
[0051] The device driver interface is used to connect the mechanical control mechanism of the ultrasound probe, supporting operations such as parameter adjustment and image freezing, and can also be connected to a vital signs monitor;
[0052] The fault protection module is used to monitor abnormalities in the voice system in real time and automatically switch to the traditional operation mode of the ultrasound host.
[0053] The fault protection module's anomaly detection unit includes voice recognition accuracy monitoring, response latency monitoring, and resource usage alarms. Its triggering conditions are: five consecutive recognition errors trigger a level one warning; instruction processing exceeds 800ms and level two protection is activated; and a forced restart is performed when memory usage is >85%.
[0054] The fault protection module's anomaly detection unit can monitor the voice recognition accuracy, response latency (threshold 800ms), and memory usage (threshold 85%) in real time, triggering a graded protection mechanism. When an anomaly is detected, the voice signal is cut off within 20ms and the physical button operation mode is activated. The recovery unit monitors the voice system and automatically switches back to voice control after it returns to normal.
[0055] The TOF sensing module integrates a gesture recognition algorithm, supporting gesture-based collaborative control in aseptic environments.
[0056] The TOF sensor module supports freezing images by clenching a fist and switching probe modes by waving a hand. The gesture recognition algorithm uses a lightweight CNN model with a response latency of <100ms. Voice commands have higher priority than gesture control. The TOF sensor module is integrated into the ultrasound host housing and aligned with the operating area.
[0057] The patient status awareness interface connects to the vital signs monitor and filters non-emergency commands based on physiological parameters.
[0058] The patient status awareness interface automatically blocks non-emergency commands when the patient's heart rate is >120 beats / min, and prioritizes image freeze, rapid data storage and alarm commands when a critical value is detected.
[0059] The patient status sensing interface is a device driver interface. The patient status sensing interface connects to the vital signs monitor via the HL7 protocol to acquire data such as heart rate and blood oxygen in real time.
[0060] The intelligent resource scheduling engine dynamically allocates edge computing and cloud computing resources based on instruction complexity;
[0061] The intelligent resource scheduling engine ensures end-to-end latency of less than 200ms for critical instructions through 5G slicing technology. Simple instructions are allocated to edge computing units, while complex instructions call cloud GPU clusters for acceleration.
[0062] Dynamic computing power allocation: Simple commands such as "freeze image" are processed by the edge computing unit, while complex commands such as "automatically measure tricuspid regurgitation velocity" call the cloud GPU cluster for acceleration. 5G slicing technology allocates dedicated communication channels for critical commands such as "emergency stop", achieving an end-to-end latency of less than 200ms. The edge computing unit uses an ARM Cortex-M7 processor.
[0063] The operation of the traceability device generates a structured log conforming to the DICOM-SR standard;
[0064] The logs generated by the operation traceability device include the original audio fingerprint, semantic parsing graph, and device status snapshot. It supports embedding ultrasound reports using the DICOM-SR standard. The original audio fingerprint is encrypted using SHA-256. The semantic parsing graph is used to visualize the knowledge graph reasoning path. The device status snapshot includes probe parameters, image frame sequences, etc. The operation log is directly embedded in the ultrasound report, supporting audit trails and quality control analysis.
[0065] A method for using a voice-controlled ultrasound diagnostic system includes the following steps:
[0066] S1): Acquisition of doctor's voice and gesture signals, with voice commands having higher priority than gesture control;
[0067] S2): Multimodal command processing is performed through the speech processing module, including noise reduction, semantic parsing and dynamic resource allocation;
[0068] S3): The ultrasound host control unit verifies the instruction logic through a medical knowledge graph;
[0069] S4): The ultrasound host control unit executes commands and generates multimodal feedback, including image feedback and execution feedback;
[0070] S5): When the voice system malfunctions, the fault protection module ensures that it switches to the traditional operation mode within 20ms;
[0071] S6): The operation traceability device generates an encrypted operation log and uploads it to the cloud.
[0072] For example, during an ultrasound examination: the doctor verbally states, "Measure the left ventricular end-diastolic radius," and the microphone transmits the audio stream to the ASR engine for speech-to-text conversion. The system understands the text content semantically and verifies the instruction logic in the medical knowledge graph. Once the verification is successful, the device receives the instruction to activate the apical four-chamber view and perform real-time measurement using a vernier. The measurement results are then displayed on the monitor.
[0073] The system operation principle in this embodiment of the invention is as follows: the voice acquisition module and the TOF sensing module acquire the commands and gestures of medical staff. The voice processing module and the communication module preprocess the acquired information and send it to the ultrasound host control unit. The command is executed by semantic understanding and verification of the command. After execution, multimodal feedback is provided, such as voice, LED flashing and projection screen.
[0074] For example, when the doctor is speaking, the fault protection module’s abnormal detection unit detects an abnormal fault in the voice control mode through pre-set judgment conditions. At this time, the control unit is switched to the traditional operation mode through hardware-level isolation. When the recovery detection unit detects that the voice control mode has returned to normal, the control unit is switched back to the voice control mode.
[0075] The core knowledge base of the medical knowledge graph is automatically synchronized with the latest clinical guidelines and equipment parameters every day. New terms are injected into the semantic parsing model (within the core knowledge base) through offline training such as incremental learning engine, without the need for downtime for updates.
[0076] The patient status awareness interface connects to the vital signs monitor via the HL7 protocol to acquire data such as heart rate and blood oxygen in real time. When the patient's heart rate is >120 beats / min, the patient status awareness interface automatically blocks non-emergency commands and prioritizes image freeze, rapid data storage, and alarm commands when a critical value is detected.
[0077] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A voice-controlled ultrasound diagnostic system, characterized in that: It includes a voice acquisition device, a voice processing module, a communication module, an ultrasound host control unit, a fault protection module, a TOF sensing module, a patient status perception interface, an intelligent resource scheduling engine, and an operation traceability device; The voice acquisition device is used to receive the operator's voice commands in real time and integrates a waterproof microphone array to adapt to high humidity environments; The voice processing module is connected to the voice acquisition device, which includes a noise reduction unit and a command recognition unit. The communication module uses the BLE 5.0 protocol to establish dynamically encrypted bidirectional communication with the ultrasound host. The ultrasound host control unit includes an instruction verification module, an execution feedback module, and a device drive interface; The fault protection module is used to monitor abnormalities in the voice system in real time and automatically switch to the traditional operation mode of the ultrasound host. The TOF sensing module integrates a gesture recognition algorithm, supporting gesture-based collaborative control in aseptic scenarios. The patient status sensing interface is connected to the vital signs monitor and filters non-emergency commands based on physiological parameters. The intelligent resource scheduling engine dynamically allocates edge computing and cloud computing resources based on instruction complexity; The operation tracing device generates a structured log that conforms to the DICOM-SR standard.
2. The voice-controlled ultrasound diagnostic system according to claim 1, characterized in that: The noise reduction unit of the speech processing module uses an adaptive filter combined with RNN noise modeling to suppress 3-5kHz electromagnetic interference. The instruction recognition unit of the speech processing module has a built-in professional medical dictionary, which includes 38 core instructions such as frozen image, measurement mark, and gain adjustment, as well as 200+ extended instructions. It supports the parsing of mixed Chinese and English instructions, with a parsing accuracy of ≥98.6%.
3. The voice-controlled ultrasound diagnostic system according to claim 1, characterized in that: The communication module generates an independent session key for each connection and uses a dynamic encryption protocol to ensure data transmission security. The instruction verification module of the ultrasound host control unit performs semantic logic verification through a medical knowledge graph, which contains three levels of dynamic associations and accesses the latest clinical guidelines and equipment parameter database in real time.
4. The voice-controlled ultrasound diagnostic system according to claim 1, characterized in that: The fault protection module's anomaly detection unit includes voice recognition accuracy monitoring, response latency monitoring, and resource usage alarms. Its triggering conditions are: five consecutive recognition errors trigger a level one warning; instruction processing exceeds 800ms to activate level two protection; and a forced restart is executed when memory usage is >85%.
5. The voice-controlled ultrasound diagnostic system according to claim 1, characterized in that: The priority logic of the execution feedback module is as follows: emergency commands trigger a flashing red LED and a high-frequency alert sound; warning commands are combined with a yellow LED and a pop-up window on the screen. Standard operation uses green LEDs and voice prompts.
6. The voice-controlled ultrasound diagnostic system according to claim 1, characterized in that: The TOF sensing module supports freezing the image by clenching a fist and switching the probe mode by waving a hand. The gesture recognition algorithm uses a lightweight CNN model with a response latency of <100ms, and voice commands have higher priority than gesture control.
7. The voice-controlled ultrasound diagnostic system according to claim 1, characterized in that: The intelligent resource scheduling engine uses 5G slicing technology to ensure that the end-to-end latency of critical instructions is less than 200ms. Simple instructions are allocated to edge computing units, while complex instructions call on cloud GPU clusters for acceleration.
8. The voice-controlled ultrasound diagnostic system according to claim 1, characterized in that: The patient status awareness interface automatically blocks non-emergency commands when the patient's heart rate is >120 beats / min, and prioritizes the execution of rapid storage and alarm commands when a critical value is detected.
9. The voice-controlled ultrasound diagnostic system according to claim 1, characterized in that: The logs generated by the operation tracing device include the original audio fingerprint, semantic parsing graph, and device status snapshot, and support the embedding of ultrasound reports in the DICOM-SR standard.
10. A method of using a voice-controlled ultrasound diagnostic system, characterized in that: Includes the following steps: S1): Acquisition of doctor's voice and gesture signals, with voice commands having higher priority than gesture control; S2): Multimodal command processing is performed through the speech processing module, including noise reduction, semantic parsing and dynamic resource allocation; S3): The ultrasound host control unit verifies the instruction logic through a medical knowledge graph; S4): The ultrasound host control unit executes commands and generates multimodal feedback, including image feedback and execution feedback; S5): When the voice system malfunctions, the fault protection module ensures that it switches to the traditional operation mode within 20ms; S6): The operation traceability device generates an encrypted operation log and uploads it to the cloud.
