Intelligent voice control system and method for handheld ultrasonic equipment
By integrating speech recognition and natural language processing modules on handheld ultrasound equipment, the voice control system is realized, which solves the problem of inconvenience in operation of traditional equipment in special occasions, improves operation convenience and efficiency, and improves medical work efficiency and user experience.
Patent Information
- Application Number
- CN202411043771.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-12-06
AI Technical Summary
Traditional handheld ultrasound equipment is inconvenient to operate manually in special occasions or in emergencies, which affects the accuracy and efficiency of diagnosis, and cannot be intelligently adjusted according to individual differences.
An intelligent voice control system including speech recognition, natural language processing and feedback modules is designed to identify user intentions through voice commands, generate operation commands and control ultrasound equipment to provide real-time feedback.
Greatly simplify operational processes, improve convenience and efficiency, optimize user experience, and significantly improve medical work efficiency and user satisfaction.
Smart Images

Figure CN119091873A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of handheld ultrasonic equipment, and in particular to an intelligent voice control system and method for handheld ultrasonic equipment. Background Art
[0002] With the rapid development of medical technology, handheld ultrasound devices have been widely used in the medical field due to their portability, high-resolution images and instant diagnosis capabilities. From daily examinations to emergency rescue, from operating rooms to wards, handheld ultrasound devices have become an indispensable diagnostic tool for doctors.
[0003] Traditional handheld ultrasound devices mainly rely on manual operation, such as physical buttons, touch screen interfaces or rotary knobs. This operation method can still meet the needs in normal environments, but in some special occasions, such as surgery that requires two-handed operation or emergency situations when time is tight, manual operation is obviously not convenient enough, and may even affect the accuracy and efficiency of diagnosis due to operational errors. In addition, because each doctor's usage habits and operating skills are different, traditional equipment cannot be intelligently adjusted and optimized according to individual differences, thus limiting the full performance of the equipment. Summary of the invention
[0004] In order to solve the above technical problems, the present invention provides an intelligent voice control system and method for a handheld ultrasound device.
[0005] The technical problem solved by the present invention can be achieved by adopting the following technical solutions:
[0006] An intelligent voice control system for a handheld ultrasound device, comprising:
[0007] A voice recognition module is used to receive voice commands, recognize the voice commands, and obtain voice text information;
[0008] A natural language processing module, connected to the speech recognition module, for analyzing the speech text information, determining the user's intention, and generating corresponding operation commands according to the user's intention;
[0009] A command execution module, connected to the natural language processing module, for controlling the handheld ultrasound device to perform corresponding operations according to the operation command;
[0010] A feedback module is connected to the command execution module and is used to output feedback information during and after the execution of the operation, wherein the feedback information includes the operation result after the handheld ultrasound device executes the corresponding operation.
[0011] Preferably, the speech recognition module comprises:
[0012] A signal acquisition unit, comprising a MEMS microphone array arranged in the handheld ultrasonic device, for acquiring the voice command by using the MEMS microphone array;
[0013] A signal processing unit, connected to the signal acquisition unit, for preprocessing the voice command to obtain a preprocessed voice command;
[0014] A speech recognition unit is connected to the signal processing unit and is used to process the preprocessed speech instruction based on a pre-established speech recognition model to obtain the speech text information.
[0015] Preferably, the natural language processing module includes:
[0016] A text parsing unit, used to parse the speech text information based on a pre-established natural language processing model to obtain a parsing result, and determine the user intention according to the parsing result;
[0017] The instruction matching unit is connected to the text parsing unit and is used to match the user intention with a preset operation instruction library, and generate the corresponding operation command according to one or more operation instructions obtained by matching.
[0018] Preferably, it also includes:
[0019] The instruction customization module is used to allow users to customize operation instructions corresponding to different user intentions, and update the operation instruction library according to the customized operation instructions.
[0020] Preferably, it also includes: a control interface group, the control interface group is communicatively connected with the handheld ultrasound device;
[0021] The control interface group includes a plurality of control interfaces, each of which is defined with input parameters and return parameters;
[0022] The command execution module is connected to the control interface group and is used to use the operation command as an input parameter of one or more control interfaces to control the handheld ultrasound device to perform a corresponding operation; and during and after the execution of the operation, the operation result is fed back to the feedback module according to the return parameters of the one or more control interfaces.
[0023] Preferably, the feedback module comprises:
[0024] A voice feedback unit, used to generate and output voice feedback information according to the operation result; and / or
[0025] The visual feedback unit is used to generate and output visual feedback information according to the operation result.
[0026] Preferably, the voice feedback information and the voice instruction have the same language type; or
[0027] The voice feedback information and the voice instruction are in different language types.
[0028] Preferably, the language type is a single language type or a mixture of multiple language types.
[0029] Preferably, it also includes:
[0030] A gesture recognition module is used to perform gesture recognition on the captured multi-angle gesture images to obtain gesture actions;
[0031] a multimodal data fusion module, connected to the gesture recognition module and the natural language processing module respectively, and used to fuse the operation command corresponding to the gesture action with the operation command corresponding to the voice instruction based on the multimodal model to obtain a fused operation command;
[0032] The command execution module is also connected to the multimodal data fusion module, and is used to control the handheld ultrasound device to perform corresponding operations according to the fused operation commands output by the multimodal data fusion module.
[0033] The present invention also provides an intelligent voice control method for a handheld ultrasound device, which is applied to the intelligent voice control system of the handheld ultrasound device as described above, and comprises:
[0034] Receiving a voice command, recognizing the voice command, and obtaining voice text information;
[0035] Analyze the voice and text information to determine the user's intention, and generate corresponding operation commands according to the user's intention;
[0036] Controlling the handheld ultrasound device to perform corresponding operations according to the operation command;
[0037] Feedback information is output during and after the operation is performed, and the feedback information includes an operation result after the handheld ultrasound device performs the corresponding operation.
[0038] The advantages or beneficial effects of the technical solution of the present invention are:
[0039] The present invention recognizes user intentions through voice commands and intelligently controls the handheld ultrasound equipment, which greatly simplifies the operation process and improves the convenience and efficiency of operation. At the same time, the intelligent design fits the user's habits, optimizes the operation experience, and significantly improves medical work efficiency and user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 The structure diagram of the intelligent voice control system in the preferred embodiment of the present invention is as follows:
[0041] Figure 2 This is a structural block diagram of an intelligent voice control system in a preferred embodiment of the present invention;
[0042] Figure 3 The figure is a flow chart of an intelligent voice control method in a preferred embodiment of the present invention. DETAILED DESCRIPTION
[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0044] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.
[0045] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.
[0046] See also Figure 1 In a preferred embodiment of the present invention, based on the above problems existing in the prior art, an intelligent voice control system 100 for a handheld ultrasound device is provided, comprising:
[0047] The speech recognition module 1 is used to receive the speech command, recognize the speech command and obtain the speech text information;
[0048] A natural language processing (NLP) module 2, connected to the speech recognition module 1, is used to parse and process the speech text information, determine the user's intention, and generate corresponding operation commands according to the user's intention;
[0049] The command execution module 3 is connected to the natural language processing module 2 and is used to control the handheld ultrasound device 200 to perform corresponding operations according to the operation command;
[0050] The feedback module 4 is connected to the command execution module 3 and is used to output feedback information during and after the execution of the operation. The feedback information includes the operation result after the handheld ultrasound device 200 executes the corresponding operation.
[0051] Specifically, in view of the problem that the traditional handheld ultrasound equipment needs manual operation, which increases the burden on doctors in emergency situations (especially when sterile operation is required or the doctor's hands cannot be free), in this embodiment, the voice recognition module 1 collects the voice instructions issued by the user through the microphone array and converts the voice instructions into recognizable text information; then, based on the natural language understanding (NLP) technology, the text information from the voice recognition module 1 is further analyzed to identify the true intention that conforms to the user's habits, and a specific operation command is generated accordingly; then, the command execution module 3 is connected to the control unit of the handheld ultrasound equipment 200 through an internal bus or a specific interface, and can receive and analyze the operation command from the natural language processing module 2, convert the command into an instruction format that can be recognized by the device control unit, and send it to the device control unit through the interface for execution. For example, for a command to adjust the gain, the module will convert it into a corresponding electrical signal that controls the gain adjustment circuit of the ultrasound equipment to achieve instant gain adjustment.
[0052] During and after the operation, the feedback module 4 provides intuitive feedback information to the user in real time. The feedback information may include the operation result and operation status of the device, such as the current gain of the device. The real-time feedback mechanism enables the user to instantly understand the operation status and results of the device, thereby improving the overall user experience. The feedback information can be conveyed to the user by displaying text information on the display screen, playing voice prompts on the speaker, or sending vibration signals by the vibrator.
[0053] In the above solution, user intentions are recognized through voice commands, and the handheld ultrasound equipment is intelligently controlled, which greatly simplifies the operation process and improves the convenience and efficiency of operation. At the same time, the intelligent design fits the user's habits, optimizes the operation experience, and significantly improves medical work efficiency and user experience.
[0054] As a preferred embodiment, the speech recognition module 1 includes:
[0055] The signal acquisition unit 11 includes a MEMS microphone array 201 disposed in the handheld ultrasonic device 200, and is used to acquire voice commands using the MEMS microphone array 201;
[0056] The signal processing unit 12 is connected to the signal acquisition unit 11 and is used to pre-process the voice command to obtain the pre-processed voice command;
[0057] The speech recognition unit 13 is connected to the signal processing unit 12 and is used to process the pre-processed speech instruction based on a pre-established speech recognition model to obtain speech text information.
[0058] Specifically, the speech recognition module 1 is responsible for receiving and recognizing the speech instructions issued by the user and converting them into text form for subsequent processing.
[0059] Specifically, the voice signal is collected by a microphone array built into the handheld ultrasound device 200. The microphone array uses a multi-microphone array technology, which can effectively enhance the voice signal processing capability. A high-sensitivity MEMS microphone array can be used to ensure that the user's voice can be accurately captured even in a noisy environment.
[0060] Furthermore, the MEMS microphone array 201 may adopt a circular or linear array layout to improve the sensitivity and noise resistance of voice collection and ensure the accuracy of voice recognition in complex environments.
[0061] Furthermore, the signal acquisition unit 11 uses beamforming technology to collect the user's voice signal in a concentrated manner to reduce the interference of environmental noise. For example, 8 microphones are used to form a circular array, and the distance between each microphone is set according to the actual application to achieve the best beamforming effect.
[0062] Specifically, after the voice commands are collected, the collected voice commands are preprocessed, and the preprocessing includes but is not limited to noise filtering and gain adjustment through a preprocessing circuit; at the same time, a high-performance digital signal processor (DSP) is used to perform real-time signal processing, such as noise suppression, echo cancellation and voice enhancement.
[0063] After completing the signal preprocessing, the preprocessed voice command is input into the pre-established voice recognition model, and the model output result is the converted voice text information.
[0064] Furthermore, the above speech recognition model is pre-trained using a deep learning model architecture. During the training process, a large speech dataset, such as the large-scale English corpus LibriSpeech, is used for model training; data enhancement, such as adding background noise and changing the speech rate, is used to improve the robustness of the model; and then, data from specific fields, such as conversation data in medical scenarios, is used for fine-tuning to improve the recognition accuracy of the model in specific application scenarios.
[0065] Furthermore, features can be regularly extracted from the speech collected during the device operation process to update the model parameters, thereby realizing online updating of the speech recognition model and continuously improving the model recognition performance.
[0066] Furthermore, the speech recognition model can be a multilingual speech recognition model. Multilingual speech recognition is performed using pre-trained deep learning models (such as Transformer, BERT, Wav2Vec, etc.). Each language has an independent model that is trained with a large amount of speech data to ensure high-precision recognition. Through transfer learning and fine-tuning techniques, the model of each language is further optimized to improve its recognition accuracy.
[0067] The user interface provides a language selection function. Users can select their preferred language in the settings, or allow the system to automatically detect and switch languages. It also supports adding or updating language packs through online updates to ensure the timeliness and accuracy of language support. Users can customize language settings according to their personal needs, and the system will optimize according to the user's preferences. The system supports language preferences for multiple users and can automatically identify and switch the user's language configuration.
[0068] Furthermore, the language is detected in real time during the voice input process and dynamically switched when necessary to ensure a seamless user experience. After receiving the voice command, the language detection algorithm is used for automatic detection. Specifically, algorithms such as mixed Gaussian models (GMM), convolutional neural networks (CNN) or recurrent neural networks (RNN) are used to analyze the phonemes, tones, syllable structure and other features of the voice input to automatically detect the language used by the user. Once the user's language is detected, the system automatically switches to the corresponding speech recognition and NLP model. The switching process is seamless and will not interrupt the user's operating experience.
[0069] Furthermore, the system also supports mixed language input (such as Chinese and English), and dynamically selects the appropriate language model for processing based on the context.
[0070] The speech recognition module 1 utilizes local or cloud computing resources to achieve real-time speech recognition and processing, ensuring high accuracy and low latency in speech recognition, so that user commands can be responded to quickly.
[0071] As a preferred embodiment, the natural language processing (NLP) module 2 includes:
[0072] The text parsing unit 21 is used to parse the speech text information based on a pre-established natural language processing model to obtain a parsing result, and determine the user's intention according to the parsing result;
[0073] The instruction matching unit 22 is connected to the text parsing unit 21 and is used to match the user intention with a preset operation instruction library and generate a corresponding operation command according to one or more operation instructions obtained by matching.
[0074] Specifically, in this embodiment, the natural language processing (NLP) module 2 is responsible for parsing and understanding the text instructions output by the speech recognition module, determining the user's intention, and generating corresponding operation commands. The high-precision parsing NLP technology is used to ensure high-precision parsing and understanding of complex sentences.
[0075] In the natural language processing process, first, use the pre-established natural language processing model, such as the speech representation model (Bidirectional Encoder Representation from Transformers, BERT) and the GPT-4 language model, to parse the speech text information, obtain the speech recognition results, and understand the user's intentions; then, match the parsing results and user intentions with the preset operation instruction library to generate corresponding operation commands.
[0076] Furthermore, during the training of the natural language processing model, a large amount of conversation data in the medical field can be collected and annotated for fine-tuning the model. For example, data such as conversations between doctors and patients and medical equipment operation instructions can be collected and the model can be trained through supervised learning methods to better understand medical terms and specific instructions.
[0077] Furthermore, the natural language processing model can be a multilingual natural language processing model. Use a multilingual NLP model (such as Multilingual BERT, XLM-R, etc.) to understand and process the user's voice commands. The model is trained with a multilingual corpus and can accurately understand the semantics of different languages. Through context understanding, the system can maintain the coherence and context understanding of the conversation in a multilingual environment, improving the user experience.
[0078] As a preferred embodiment, the invention further comprises:
[0079] The instruction customization module 5 is used to allow users to customize operation instructions corresponding to different user intentions, and update the operation instruction library according to the customized operation instructions.
[0080] Specifically, in this embodiment, commonly used operation instructions are preset in the operation instruction library, and the operation instruction library also supports a custom setting function, allowing users to customize voice instructions according to personal habits and needs, thereby improving the flexibility and adaptability of the system. Specifically, a custom setting interface is provided on the user interface, allowing users to add or modify custom instructions, and automatically update the instruction library to meet different operation requirements.
[0081] In the process of command matching, semantic similarity algorithms, such as cosine similarity and Euclidean distance, can be used to match the voice commands issued by the user with the preset operation command library. Furthermore, fuzzy matching and synonym recognition can also be supported. For example, the user saying "start scanning" and "start scanning" can be recognized as the same operation command.
[0082] As a preferred embodiment, it further includes: a control interface group 6, the control interface group 6 is communicatively connected with the handheld ultrasound device 200;
[0083] The control interface group 6 includes multiple control interfaces, each of which is defined with input parameters and return parameters;
[0084] The command execution module 3 is connected to the control interface group 6, and is used to use the operation command as an input parameter of one or more control interfaces to control the handheld ultrasound device 200 to perform the corresponding operation; and during and after the operation, the operation result is fed back to the feedback module 4 according to the return parameters of one or more control interfaces.
[0085] Furthermore, the control interface group 6 is a set of predefined standardized application programming interfaces (Application Programming Interface, API). Each control interface is designed for different functions or operations of the handheld ultrasound device 200. For example, a Representational State Transfer (RESTful, referred to as REST) style API interface is designed, including functions such as starting scanning, stopping scanning, adjusting parameters (such as frequency, gain), and saving images. Each API interface should have clear input parameters (such as command type, specific parameter values, etc.) and return parameters (such as operation results, status codes, error messages, etc.) to ensure the accuracy and controllability of the operation.
[0086] Furthermore, the control interface supports WebSocket or Message Queuing Telemetry Transport (MQTT) protocol. Real-time control and feedback mechanism is implemented through WebSocket or MQTT protocol to ensure timely execution of commands and status feedback. For example, after the user issues an instruction to adjust the frequency, the system can send commands to the handheld ultrasound device 200 in real time through the WebSocket protocol and receive status feedback from the device, so as to provide the user with operation status information and confirm whether the operation is successfully completed.
[0087] As a preferred embodiment, the feedback module 4 includes:
[0088] A voice feedback unit 41, configured to generate and output voice feedback information according to the operation result; and / or
[0089] The visual feedback unit 42 is used to generate and output visual feedback information according to the operation result.
[0090] As a preferred implementation, the voice feedback information and the voice instruction have the same language type; or
[0091] The voice feedback information and the voice command have different language types.
[0092] As a preferred implementation, the language type is a single language type or a mixture of multiple language types.
[0093] Specifically, the voice feedback unit 41 generates voice feedback information through voice synthesis technology, and plays the synthesized voice feedback information through the built-in speaker. Specifically, a custom neural network model, such as a spectrum prediction network (Tacotron 2) and a wave network deep neural network model (WaveNet), can be used to realize voice synthesis and improve the naturalness and clarity of voice feedback.
[0094] In order to adapt to the voice preferences of different users, the model parameters such as pitch, speaking speed, volume, etc. can also be adjusted to customize the voice that meets the user's personal preferences.
[0095] In order to meet the diverse needs of users around the world, the feedback module 4 supports speech synthesis in multiple languages and dialects. By pre-training or integrating multilingual models, the system can automatically identify user language preferences and provide speech feedback in Mandarin, Cantonese, English, etc. At the same time, for specific regions or user groups, language options can be further refined, such as dialect version speech feedback, to enhance intimacy and practicality.
[0096] On the display screen of the device, the visual feedback unit 42 can display the operation status and results in real time. For example, when the "start scanning" operation is executed, the display screen will immediately display a real-time preview of the scanned image and simultaneously update the scanning parameters such as frequency, gain, etc., so that the user can intuitively understand the current operation progress and status.
[0097] In addition to the screen display, indicator lights or other visual elements (such as LED light strips) are also used to provide additional operating status feedback. Through the changes in different colors or flashing patterns, users can quickly identify the working status of the device. For example, green light indicates normal operation, while red flashing may indicate an error or warning.
[0098] In order to provide a more comprehensive and intuitive user experience, the feedback module 4 closely combines voice and visual feedback. When the system completes an operation, not only will the voice feedback unit 41 send out a prompt tone of "scanning completed", but the visual feedback unit 42 will also display the words "scanning completed" and the scan result image on the display screen. This multimodal feedback method not only enhances the information transmission effect, but also improves the convenience and satisfaction of user operation.
[0099] As a preferred embodiment, wherein Figure 2 As shown, it also includes:
[0100] A gesture recognition module 7, used to perform gesture recognition on the captured multi-angle gesture images to obtain gesture actions;
[0101] The multimodal data fusion module 8 is connected to the gesture recognition module 7 and the natural language processing module 2 respectively, and is used to fuse the operation command corresponding to the gesture action with the operation command corresponding to the voice command based on the multimodal model to obtain the fused operation command;
[0102] The command execution module 3 is also connected to the multimodal data fusion module 8, and is used to control the handheld ultrasound device 200 to perform corresponding operations according to the fused operation commands output by the multimodal data fusion module.
[0103] Specifically, the system of the present invention not only supports voice control, but also can perform auxiliary control through gesture recognition to achieve more natural human-computer interaction. By integrating high-precision gesture recognition cameras, such as TOF (Time-of-Flight) cameras, RGB-D cameras, etc., the user's hand movements and postures can be accurately captured. The camera can capture gestures from multiple angles to ensure recognition accuracy in different usage environments.
[0104] After capturing multi-angle gesture images, the images captured by the camera are preprocessed by denoising, contrast enhancement and other preprocessing operations. Then, gesture recognition algorithms, such as convolutional neural networks (CNNs) and long short-term memory networks (LSTMs), are used to process the image data captured by the camera in real time and provide instant gesture recognition feedback.
[0105] During the training process of the gesture recognition model, a large number of gesture data sets are collected and annotated, including gesture images at various angles and lighting conditions and corresponding device operation commands. The gesture images include common gesture actions, such as waving, clicking, sliding, etc. The collected data sets are used for model training, verification and testing, and the parameters are continuously optimized to improve the recognition accuracy.
[0106] The system of the present invention can also use a multimodal deep learning model (such as a Transformer-based multimodal model) combined with voice and gesture input to achieve intelligent coordinated control of voice and gesture. The coordinated control strategy can control the main functions of the ultrasound device (such as starting, stopping, switching modes) through voice commands, and perform fine operations (such as adjusting the image magnification and moving the probe position) through gestures. For example, voice start, gesture adjustment: the user starts the function through voice commands (such as "enlarge the image"), and then uses gestures to finely control the degree of magnification (such as a finger pinch gesture). Or gesture confirmation, voice execution: the user selects the operation object through gestures (such as pointing to a button), and then confirms the execution through voice (such as "confirm").
[0107] Voice and gesture input use a timestamp synchronization mechanism to ensure the time synchronization of voice commands and gesture input, ensuring the continuity and naturalness of the interaction.
[0108] Furthermore, the combination of gestures and voice commands can be customized according to the user's personal habits, making the system more personalized and easy to use. The system can continuously learn the user's habits through machine learning technology and optimize the recognition and response of gestures and voice commands.
[0109] In order to ensure the efficiency and compatibility of data exchange and control instruction transmission between modules of the system, standardized interface protocols such as RESTfulAPI or gRPC are adopted. For example, a unified API interface is defined, including functions such as speech recognition result transmission, instruction parsing, command execution and status feedback, to ensure seamless communication between modules.
[0110] During data transmission, SSL / TLS encryption protocol is used to ensure data security and privacy protection. For example, user voice data is encrypted during transmission and storage to prevent data leakage and unauthorized access.
[0111] The system supports real-time monitoring mechanism, collects the operation status data of each module, such as CPU usage, memory usage, network delay and other data, and performs real-time analysis and processing. For example, use monitoring tools (such as Prometheus) to collect and monitor data, and store it in the local database; use data visualization tools (Grafana) to visualize the collected data, and configure alarm rules to automatically trigger alarm notifications when abnormal data is monitored, so that operation and maintenance personnel can handle it in time.
[0112] The system also supports an automatic recovery mechanism. When a module anomaly is detected, the system automatically restarts the module or switches to a backup module to ensure continuous operation of the system. For example, when an abnormality occurs in the voice recognition module, the system automatically restarts the module or switches to a backup voice recognition service to ensure that user operations are not interrupted.
[0113] See also Figure 3 The present invention also provides an intelligent voice control method for a handheld ultrasound device, which is applied to the intelligent voice control system of the handheld ultrasound device as described above, and comprises:
[0114] S100, receiving a voice command, recognizing the voice command, and obtaining voice text information;
[0115] S200, parsing the voice text information, determining the user's intention, and generating a corresponding operation command according to the user's intention;
[0116] S300, controlling the handheld ultrasound device to perform corresponding operations according to the operation command;
[0117] S400, outputting feedback information during and after the execution of the operation, wherein the feedback information includes the operation result after the handheld ultrasound device executes the corresponding operation.
[0118] Specifically, by integrating the handheld ultrasound device with the intelligent voice control system 100, the diagnostic process can be completed quickly and accurately in emergency situations, greatly improving work efficiency and patient satisfaction.
[0119] In scenarios where sterile operation needs to be maintained or the hands cannot move freely, when the doctor enters the emergency room, he only needs to issue a voice command such as "start the ultrasound device", and the device will respond and start immediately. During the scanning process, the device parameters can be adjusted by voice such as "gain increase by 5", achieving fast and accurate parameter adjustment without manual operation of the device, reducing the complexity of operation in emergency situations.
[0120] You can also use natural language to input commands, such as "Please start scanning the right upper abdomen." The NLP module can parse these commands and convert them into operation commands that the device can recognize. In addition, hospitals can also customize the command library according to actual needs, such as "Save the image and send it to the PACS system," further improving the flexibility and personalized service of the user experience. After the NLP module parses the command, the command execution module immediately controls the ultrasound device to perform the corresponding operation, such as stopping the scan and saving the image.
[0121] At the same time, the device is equipped with a feedback mechanism that combines voice and visual feedback. After performing an operation, the device immediately reports the operation status to the doctor, such as the completion of the operation or the results of parameter adjustment, to ensure that the operator has a clear understanding of the device operation.
[0122] In a sterile environment, the equipment can also be controlled contactlessly through a combination of voice and gestures, reducing physical contact and maintaining a clean environment.
[0123] The advantages or beneficial effects of adopting the above technical solution are: the present invention recognizes user intentions through voice commands and intelligently controls the handheld ultrasound equipment, which greatly simplifies the operation process and improves the convenience and efficiency of operation; at the same time, the intelligent design fits the user's habits, optimizes the operation experience, and significantly improves medical work efficiency and user experience.
[0124] The above are only preferred embodiments of the present invention, and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the contents of this specification and illustrations should be included in the protection scope of the present invention.
Claims
1. An intelligent voice control system for a handheld ultrasound device, characterized in that: include: A voice recognition module is used to receive voice commands, recognize the voice commands, and obtain voice text information; A natural language processing module, connected to the speech recognition module, for analyzing the speech text information, determining the user's intention, and generating corresponding operation commands according to the user's intention; A command execution module, connected to the natural language processing module, for controlling the handheld ultrasound device to perform corresponding operations according to the operation command; A feedback module is connected to the command execution module and is used to output feedback information during and after the execution of the operation, wherein the feedback information includes the operation result after the handheld ultrasound device executes the corresponding operation.
2. The intelligent voice control system of the handheld ultrasound device according to claim 1, characterized in that: The speech recognition module comprises: A signal acquisition unit, comprising a MEMS microphone array arranged in the handheld ultrasonic device, for acquiring the voice command by using the MEMS microphone array; A signal processing unit, connected to the signal acquisition unit, for preprocessing the voice command to obtain a preprocessed voice command; A speech recognition unit is connected to the signal processing unit and is used to process the preprocessed speech instruction based on a pre-established speech recognition model to obtain the speech text information.
3. The intelligent voice control system of the handheld ultrasound device according to claim 1, characterized in that: The natural language processing module includes: A text parsing unit, used to parse the speech text information based on a pre-established natural language processing model to obtain a parsing result, and determine the user intention according to the parsing result; The instruction matching unit is connected to the text parsing unit and is used to match the user intention with a preset operation instruction library, and generate the corresponding operation command according to one or more operation instructions obtained by matching.
4. The intelligent voice control system of the handheld ultrasound device according to claim 3, characterized in that: Also includes: The instruction customization module is used to allow users to customize operation instructions corresponding to different user intentions, and update the operation instruction library according to the customized operation instructions.
5. The intelligent voice control system of the handheld ultrasound device according to claim 1, characterized in that: Also includes: A control interface group, the control interface group is communicatively connected with the handheld ultrasound device; The control interface group includes a plurality of control interfaces, each of which is defined with input parameters and return parameters; The command execution module is connected to the control interface group and is used to use the operation command as an input parameter of one or more control interfaces to control the handheld ultrasound device to perform a corresponding operation; And during and after the execution of the operation, the operation result is fed back to the feedback module according to the return parameters of the one or more control interfaces.
6. The intelligent voice control system of the handheld ultrasound device according to claim 1, characterized in that: The feedback module comprises: A voice feedback unit, used to generate and output voice feedback information according to the operation result; and / or The visual feedback unit is used to generate and output visual feedback information according to the operation result.
7. The intelligent voice control system of the handheld ultrasound device according to claim 6, characterized in that: The voice feedback information and the voice instruction have the same language type; or The voice feedback information and the voice instruction are in different language types.
8. The intelligent voice control system of the handheld ultrasound device according to claim 7, characterized in that: The language type is a single language type or a mixture of multiple language types.
9. The intelligent voice control system of the handheld ultrasound device according to claim 1, characterized in that: Also includes: A gesture recognition module is used to perform gesture recognition on the captured multi-angle gesture images to obtain gesture actions; a multimodal data fusion module, connected to the gesture recognition module and the natural language processing module respectively, and used to fuse the operation command corresponding to the gesture action with the operation command corresponding to the voice instruction based on a multimodal model to obtain a fused operation command; The command execution module is also connected to the multimodal data fusion module, and is used to control the handheld ultrasound device to perform corresponding operations according to the fused operation commands output by the multimodal data fusion module.
10. An intelligent voice control method for a handheld ultrasound device, applied to the intelligent voice control system for a handheld ultrasound device as claimed in any one of claims 1 to 9, characterized in that: include: Receiving a voice command, recognizing the voice command, and obtaining voice text information; Analyze the voice and text information to determine the user's intention, and generate corresponding operation commands according to the user's intention; Controlling the handheld ultrasound device to perform corresponding operations according to the operation command; Feedback information is output during and after the operation is performed, and the feedback information includes an operation result after the handheld ultrasound device performs the corresponding operation.
