Driving method and device of medical equipment and medical system

Abstract

The embodiment of the invention provides a driving method and device of medical equipment and a medical system. The method comprises the following steps: receiving a voice instruction from a sound pickup device; inputting the voice instruction into a deep learning neural network, and outputting a driving instruction for driving the medical equipment based on the deep learning neural network; and driving the action of the medical equipment according to the driving instruction. Therefore, through voice control based on AI / ML, driving of the medical equipment can be accurately realized without assistance of multiple persons, the labor cost can be reduced, the efficiency can be improved, and the risk of operation failure can be reduced.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of medical equipment, and in particular, to a driving method and device of a medical equipment and a medical system. BACKGROUND

[0002] Medical equipment includes medical imaging equipment, which is used to scan a patient in a non-invasive manner to obtain a medical image of an anatomical tissue of interest of the patient to assist a doctor in diagnosis. For example, a computed tomography (CT) uses a precisely collimated X-ray beam to perform a cross-sectional scan one after another around a part of a human body together with a highly sensitive detector, and has the characteristics of fast scanning time and clear image, and can be used for the examination of various diseases.

[0003] The medical imaging equipment such as a CT device has a driving assembly or a separate controller arranged on a device gantry, a scanning bed (which can also be referred to as a patient table), or an operating table outside a scanning room. During scanning, a subject to be detected is placed on the scanning bed, and an operator of the medical imaging equipment can control the scanning bed to perform translational, ascending, or descending movement to complete a specific medical imaging by operating the driving assembly.

[0004] For example, in the case of medical imaging equipment-assisted treatment, an assistant doctor or an image technician controls the medical imaging equipment to obtain a medical diagnostic image of a patient by using a remote controller to assist a lead doctor in surgery. For example, in a puncture surgery, the lead doctor needs to observe the medical diagnostic image and is responsible for performing the puncture operation, and the assistant doctor controls the medical equipment based on the instructions of the lead doctor to assist the lead doctor in determining a needle insertion position and checking a position of the needle in real time. SUMMARY

[0005] The inventors have found that in driving the medical equipment for medical diagnosis and treatment, multiple people often need to cooperate. For example, the lead doctor needs the assistance of the assistant doctor, and the assistant doctor controls the remote controller to control the medical equipment, which increases the labor cost. In addition, if the assistant doctor cannot understand the instructions of the lead doctor in time, it is easy to cause the extension of the surgery time, and even operation errors, which further leads to the failure of the surgery. Therefore, how to reduce the labor cost, improve the efficiency, and reduce the risk of surgery failure is a problem to be solved.

[0006] To solve at least one of the above technical problems or other similar problems, embodiments of the present application provide a driving method and device of a medical equipment and a medical system.

[0007] According to an aspect of embodiments of the present application, a driving method of a medical equipment is provided, including:

[0008] receiving a voice instruction from a sound pickup device;

[0009] inputting the voice instruction into a deep learning neural network, and outputting a driving instruction for driving the medical device based on the deep learning neural network;

[0010] driving the medical device according to the driving instruction.

[0011] In some embodiments, outputting a driving instruction for driving the medical device based on the deep learning neural network comprises:

[0012] performing automatic speech recognition (ASR) on the voice instruction based on a first deep learning neural network to perform feature extraction and output system-recognized voice words; and

[0013] performing natural language processing (NLP) on the system-recognized voice words based on a second deep learning neural network to perform semantic analysis and output the driving instruction for driving the medical device.

[0014] In some embodiments, the method further comprises:

[0015] selecting a corresponding driving instruction for the voice instruction according to pre-stored customized information, wherein the customized information comprises a corresponding relationship between voice instructions and driving instructions.

[0016] In some embodiments, the method further comprises:

[0017] performing voiceprint detection on the voice instruction based on pre-stored voiceprint information; and

[0018] outputting the system-recognized voice words in a case where the voice instruction conforms to voiceprint features of an authorized user, and not outputting the system-recognized voice words in a case where the voice instruction does not conform to the voiceprint features of the authorized user.

[0019] In some embodiments, the method further comprises:

[0020] performing on-off detection on the voice instruction based on pre-stored wake-up information / shut-down information in a case where the voice instruction conforms to voiceprint features of an authorized user; and

[0021] turning on driving of the medical device in a case where the voice instruction comprises the wake-up information, and turning off driving of the medical device in a case where the voice instruction comprises the shut-down information.

[0022] In some embodiments, the method further comprises:

[0023] training the deep learning neural network using training samples.

[0024] In some embodiments, the deep learning neural network is trained using training samples, including:

[0025] performing speech command recognition (SCR) on the training samples based on the deep learning neural network to perform feature extraction and output a feature vector;

[0026] determining a difference between the feature vector and a feature vector of another voice command according to a semantic distance; and

[0027] in a case where the difference is greater than or equal to a preset threshold, determining that the training sample is a valid sample.

[0028] In some embodiments, the method further includes:

[0029] selecting a corresponding driving command for the training sample and storing custom information, wherein the custom information includes a corresponding relationship between the voice command and the driving command.

[0030] According to another aspect of embodiments of the present application, a driving device of a medical device is provided, including a processor and a memory, the processor being configured to perform the driving method of the medical device as described above.

[0031] According to yet another aspect of embodiments of the present application, a medical system is provided, including:

[0032] a sound pickup device that receives a voice command from a user;

[0033] a driving device that inputs the voice command into a deep learning neural network and outputs a driving command for driving a medical device based on the deep learning neural network; and

[0034] a medical device that performs an action according to the driving command.

[0035] In some embodiments, the medical system further includes:

[0036] a display device that displays an image captured by the medical device and the voice command recognized by the driving device.

[0037] In some embodiments, the display device further displays historical information of voice commands from the user within a period of time.

[0038] In some embodiments, the sound pickup device includes a wearable microphone fixed to the user or a microphone fixed to the medical device.

[0039] One of the beneficial effects of the embodiments of the present application is that the voice instruction is input into the deep learning neural network, and a driving instruction for driving the medical device is output based on the deep learning neural network. Thus, through the voice control based on AI / ML, the driving of the medical device can be accurately realized without the assistance of multiple people, the labor cost can be reduced, the efficiency can be improved, and the risk of surgical failure can be reduced; and compared with the traditional driving assembly or controller, in the technical solution of the present application, the doctor does not need to manually operate the driving assembly or the controller, and the driving assembly or the controller can also not be arranged in the operating room or the scanning room, accordingly, the doctor faces less spatial obstacles in the operating room or the scanning room, that is, the doctor has higher freedom in moving or operating in the operating room or the scanning room.

[0040] Specific implementations of the embodiments of the present application are disclosed in detail in the following description and accompanying drawings, indicating the ways in which the principles of the embodiments of the present application can be employed. It should be understood that the embodiments of the present application are not limited in scope in terms of the implementations described. In the scope and spirit of the appended claims and their equivalents, the embodiments of the present application include many changes, modifications and equivalents. BRIEF DESCRIPTION OF DRAWINGS

[0041] The accompanying drawings included to provide a further understanding of the embodiments of the present application, constitute a part of the specification and serve to explain the principles of the present application together with the text. Obviously, the drawings in the following description are only some embodiments of the present application, and other embodiments can be obtained by those skilled in the art without creative labor on the basis of these drawings. In the drawings:

[0042] Figure 1 is a schematic diagram of a CT device of an embodiment of the present application;

[0043] Figure 2 is a schematic diagram of a CT imaging system of an embodiment of the present application;

[0044] Figure 3 is a schematic diagram of a CT device and its driving device of an embodiment of the present application;

[0045] Figure 4 is a schematic diagram of a driving method of a medical device of an embodiment of the present application;

[0046] Figure 5 is a schematic diagram of a deep learning neural network based on an embodiment of the present application;

[0047] Figure 6 is an example diagram of generating a driving instruction based on a voice instruction of an embodiment of the present application;

[0048] Figure 7 is an example diagram of customizing a corresponding relationship of an embodiment of the present application;

[0049] Figure 8 This is another schematic diagram of the driving method of the medical device according to an embodiment of this application;

[0050] Figure 9 This is a schematic diagram of deep learning neural network training according to an embodiment of this application;

[0051] Figure 10 This is an example diagram of deep learning neural network training according to an embodiment of this application;

[0052] Figure 11 This is a schematic diagram of the training process confirmed in an embodiment of this application;

[0053] Figure 12 This is a schematic diagram of a medical system according to an embodiment of this application;

[0054] Figure 13 This is an example diagram of the display interface of a display device according to an embodiment of this application. Detailed Implementation

[0055] Referring to the accompanying drawings, the foregoing and other features of the embodiments of this application will become apparent from the following description. Specific embodiments of this application are specifically disclosed in the description and drawings, illustrating partial implementations in which the principles of the embodiments of this application can be adopted. It should be understood that this application is not limited to the described embodiments; rather, the embodiments of this application include all modifications, variations, and equivalents falling within the scope of the appended claims.

[0056] In the embodiments of this application, the terms "first," "second," etc., are used to distinguish different elements by name, but do not indicate the spatial arrangement or chronological order of these elements, and these elements should not be limited by these terms. The term "and / or" includes any one or more of the terms listed in association and all combinations thereof. The terms "comprising," "including," "having," etc., refer to the presence of the stated features, elements, components, or assemblies, but do not exclude the presence or addition of one or more other features, elements, components, or assemblies.

[0057] In the embodiments of this application, the singular forms "a," "the," etc., including the plural forms, should be broadly understood as "a kind" or "a class" rather than limited to the meaning of "an." Furthermore, the term "the" should be understood to include both the singular and plural forms, unless the context explicitly indicates otherwise. Additionally, the term "according to" should be understood as "at least partially based on…," and the term "based on" should be understood as "at least partially based on…," unless the context explicitly indicates otherwise.

[0058] Features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, combined with features in other embodiments, or substituted for features in other embodiments. The term "comprising / including" as used herein means the presence of a feature, integral, step, or component, but does not exclude the presence or addition of one or more other features, integrals, steps, or components.

[0059] The medical devices described in this application include, for example, medical imaging devices. This application is not limited to this and can be applied to any medical device that can be driven to perform various movements. The following description uses medical imaging devices (such as CT equipment) as an example.

[0060] Medical imaging equipment can be used for a variety of medical imaging modalities, including but not limited to CT (computed tomography) imaging equipment, PET (positron emission tomography)-CT, magnetic resonance imaging (MRI) or any other suitable medical imaging equipment.

[0061] A system for acquiring medical image data may include the aforementioned medical imaging equipment, a separate computer connected to the medical imaging equipment, or a computer connected to an internet cloud, which is connected via the internet to the medical imaging equipment or a storage device for storing medical images. The imaging method may be implemented independently or in combination by the aforementioned medical imaging equipment, the computer connected to the medical imaging equipment, and the computer connected to the internet cloud. For example, a system for acquiring medical image data may be a CT imaging system, etc.

[0062] Exemplary examples are described below in conjunction with X-ray computed tomography (CT) imaging equipment. Those skilled in the art will understand that embodiments of this application can also be applied to other medical devices.

[0063] Figure 1 This is a schematic diagram of a CT device according to an embodiment of this application, illustrating the CT device 100. For example... Figure 1 As shown, the CT equipment 100 includes a scanning gantry 101 and a patient table 102 (e.g., a scanning bed); the scanning gantry 101 has an X-ray source 103 that projects an X-ray beam toward a detector assembly or collimator 104 on the opposite side of the scanning gantry 101. The subject 105 can lie supine on the patient table 102 and moves into the scanning gantry opening 106 as the patient table 102 moves; medical imaging data of the subject 105 can be obtained through scanning by the X-ray source 103.

[0064] Figure 2 This is a schematic diagram of a CT imaging system according to an embodiment of this application, illustrating a block diagram of the CT imaging system 200.Figure 2 As shown, the detector assembly 104 includes multiple detector units 104a and a data acquisition system (DAS) 104b. The multiple detector units 104a sense projected X-rays passing through the object being detected 105.

[0065] DAS104b converts the collected information into projection data based on the sensing of detector unit 104a for subsequent processing. During the scan that acquires X-ray projection data, the scanning gantry 101 and the components mounted thereon rotate around the rotation center 101c.

[0066] The rotation of the scanning gantry 101 and the operation of the X-ray source 103 are controlled by the control mechanism 203 of the CT imaging system 200. The control mechanism 203 includes an X-ray controller 203a that provides power and timing signals to the X-ray source 103, and a scanning gantry motor controller 203b that controls the rotational speed and position of the scanning gantry 101. The image reconstruction unit 204 receives projection data from the DAS 104b and performs image reconstruction. The reconstructed image is transmitted as input to the computer 205, which stores the image in a mass storage device 206.

[0067] Computer 205 also receives commands and scanning parameters from the operator via console 207. Console 207 has some form of operator interface 2071, such as a keyboard, mouse, or voice-activated controller. Console 207 may also have input devices such as pedal assembly 2072. Furthermore, console 207 may have other suitable input devices. An associated display 208 allows the operator to view reconstructed images and other data from computer 205. Commands and parameters provided by the operator are used by computer 205 to provide control signals and information to DAS 104b, X-ray controller 203a, and scanning gantry motor controller 203b. Additionally, computer 205 operates patient table motor controller 209, controlling patient table 102 to position the examination object 105 and scanning gantry 101. Specifically, patient table 102 moves the examination object 105 wholly or partially through... Figure 1 The scanning rack opening is 106.

[0068] The above illustrations depict devices and systems for acquiring medical imaging data (or medical images or medical image data) according to embodiments of this application, but this application is not limited thereto. Medical imaging devices may be CT equipment, PET-CT, or any other suitable imaging equipment. Storage devices may be located within the medical imaging device, on a server outside the medical imaging device, in a separate medical image storage system (such as a PACS, Picture Archiving and Communication System), and / or in a remote cloud storage system.

[0069] Furthermore, medical imaging workstations can be located locally on the medical imaging equipment, meaning they are situated close to the equipment, and both can be located in the same scanning room, radiology department, or within the same hospital. Meanwhile, the medical image cloud platform analysis system can be located away from the medical imaging equipment, for example, in the cloud where it communicates with the equipment.

[0070] As an example, after a medical institution completes an imaging scan using medical imaging equipment, the scanned data is stored in a storage device. A medical imaging workstation can directly read the scanned data and perform image processing through its processor. As another example, a medical image cloud platform analysis system can remotely access medical images stored in the storage device to provide "Software as a Service" (SaaS). SaaS can exist between hospitals, between hospitals and imaging centers, or between hospitals and third-party online medical service providers.

[0071] The above illustrations illustrate medical image scanning. The embodiments of this application will be described in detail below with reference to the accompanying drawings. In the following embodiments, a CT scanner is used as an example for explanation; however, the description is equally applicable to other medical devices.

[0072] Figure 3 This is a schematic diagram of a CT device and its driving mechanism according to an embodiment of this application. Taking a CT device as an example, a block diagram of its driving mechanism is shown. Figure 3 As shown, the medical system 300 includes, for example, a voice pickup device 301, a drive device 302 for a CT scanner 303, the CT scanner 303 itself, and a display device 304. The drive device 302 for the CT scanner 303 converts received voice commands into drive commands to control the CT scanner 303 to perform actions and to control the display device 304 to perform displays.

[0073] For example, when a user issues a voice command, the sound pickup device 301 sends the voice command to the drive device 302 of the CT device 303. The drive device 302 of the CT device 303 processes the voice command and outputs a drive command. The CT device 303 performs corresponding actions according to the drive command, and the display device 304 can also display accordingly based on the voice command.

[0074] The sound pickup device 301 includes a wearable microphone attached to a user or a microphone attached to a medical device. For example, the sound pickup device 301 may be located on the user side, such as within a wearable microphone device, or the sound pickup device 301 may be integrated into a medical device 303, or the sound pickup device 301 may exist independently in other locations (e.g., a sound receiving device mounted on a stand).

[0075] The driving device 302 of the medical device 303 may include a processor, memory, and a driving module (driver). This driving module may be located, for example, in software form within the medical device 303, on a server external to the medical device 303, or independently in a remote cloud system. The display device 304 may exist independently (e.g., as shown in the image). Figure 3 (As shown), it can also be combined with medical devices.

[0076] The above illustrations depict some of the structural components of the embodiments of this application. This application is not limited thereto. The driving method of the embodiments of this application will be further illustrated below.

[0077] This application provides a method for driving a medical device, which drives the medical device based on the user's voice commands.

[0078] Figure 4 This is a schematic diagram of a driving method for a medical device according to an embodiment of this application, and the description is given from the perspective of the driving device of the medical device. Figure 4 As shown, the method includes:

[0079] 401, Receive voice commands from the voice pickup device;

[0080] 402, Inputting voice commands into a deep learning neural network, and outputting driving commands for driving medical devices based on the deep learning neural network; and

[0081] 403, drive the medical device to perform actions according to the drive instructions.

[0082] It is worth noting that the above appendix Figure 4The embodiments of this application have only been illustrated schematically, and the application is not limited thereto. For example, some of the above steps can be performed simultaneously or in a specific order, and the execution order between the various operations can be appropriately adjusted. Furthermore, other operations can be added or some operations can be removed. Those skilled in the art can make appropriate modifications based on the above description, and are not limited to the above-described embodiments. Figure 4 The records.

[0083] In this embodiment, the user's voice commands are acquired through a voice pickup device, input into a deep learning neural network, and then output driving commands for the medical device based on the deep learning neural network. Thus, through voice control based on artificial intelligence / machine learning (AI / ML), the medical device can be accurately driven without the assistance of multiple people, reducing labor costs, improving efficiency, and lowering the risk of surgical failure.

[0084] Furthermore, compared to traditional drive components or controllers, the technical solution of this application eliminates the need for doctors to manually operate drive components or controllers, and the operating room or scanning room may also be free from drive components or controllers. Consequently, doctors face fewer spatial obstacles in the operating room or scanning room, meaning they have greater freedom of movement or operation in the operating room or scanning room.

[0085] In some embodiments, the sound pickup device can be any form of sound receiving device, such as a headset, a microphone clipped to a user's clothing, a stand-alone sound receiving device, or a sound receiving module fixed to a medical device, etc., and the embodiments of this application are not intended to limit it.

[0086] In some embodiments, the voice command may be a command preset by the driver of the medical device at the factory, or it may be a valid command set by an authorized user. The language of the voice command may be Chinese, English, Japanese, Korean, etc., or it may be standard Mandarin or a dialect, etc., and the embodiments of this application are not limited thereto.

[0087] For example, voice commands can indicate the direction and / or amplitude of movement of medical devices, such as "move forward one step," "move down," "down," "up," "move 2 centimeters to the left," etc. As another example, voice commands can indicate the status of medical devices, such as "on," "off," etc. Furthermore, voice commands can indicate the function of medical devices, such as "activate voice control function," "deactivate voice control function," "increase lighting brightness," etc. Still another example is that voice commands can indicate the user's status, such as "activate user B," "deactivate user C," etc.

[0088] In some embodiments, the deep learning neural network may use existing open-source AI / ML models, which can be selected according to the needs of accuracy, etc., during specific implementation, and this application does not impose any restrictions on this. For details regarding deep learning neural networks, please refer to relevant technologies.

[0089] Figure 5 This is a schematic diagram of a deep learning neural network based on an embodiment of this application. Figure 5 As shown, the driving instructions for driving medical devices based on the output of a deep learning neural network can include:

[0090] 501, Automatic Speech Recognition (ASR) is performed on voice commands based on a first deep learning neural network to extract features and output the recognized speech words; and

[0091] 502. Based on the second deep learning neural network, the system recognizes speech words and performs natural language processing (NLP) to perform semantic analysis and output driving instructions for driving medical devices.

[0092] Therefore, combining the two AI / ML modules can further improve speech recognition accuracy. For example, it can adapt to the speech of different people in different regions, and can accurately recognize even non-standard or dialectal speech, thus improving the robustness and scalability of speech recognition.

[0093] Figure 6 This is an example diagram of a voice command-based driver instruction generation embodiment of this application.

[0094] like Figure 6 As shown, ASR may include, for example, keyword search, speech information extraction, speech information preprocessing, voice feature extraction, and neural network processing, but is not limited to these. Specifically, for example, based on factory-predefined voice commands and voice commands set by authorized users, keywords in the input voice command are determined; based on the keywords, the user-issued voice command is identified and extracted, and background noise contained in the voice command is removed; preprocessing and feature extraction operations are performed on the extracted voice command.

[0095] For example, preprocessing operations include, but are not limited to, noise reduction, resampling, and channel coordination; feature extraction operations include, but are not limited to, short-time Fourier transform, Mel-frequency cepstral coefficients, linear predictive coding, and perceptual feature-based cepstral coefficients; this application is not limited thereto.

[0096] like Figure 6As shown, the extracted speech features can be input into a deep learning neural network to output the speech words recognized by the system. Deep learning neural networks include, but are not limited to, recurrent neural networks, long short-term memory networks, convolutional neural networks, or Transformer networks.

[0097] like Figure 6 As shown, NLP can include, but is not limited to, word segmentation, part-of-speech tagging, named entity recognition, syntactic analysis, and semantic analysis. Specifically, for example, it determines the word segmentation rules based on the language corresponding to the speech command, and performs word segmentation on the speech words output by the neural network according to these rules; it performs part-of-speech tagging and named entity recognition on each word to assist in syntactic analysis; it uses syntactic analysis based on the language's grammatical rules to determine the hierarchical structure of the sentence and the dependencies between words; and it uses semantic analysis to assist the machine in recognizing speech words.

[0098] Therefore, combining ASR and NLP can further improve the accuracy of speech recognition.

[0099] In some embodiments, a corresponding driving instruction can be selected for a voice command based on pre-stored custom information; wherein the custom information includes the correspondence between voice commands and driving instructions.

[0100] Figure 7 This is an example diagram of a custom mapping relationship according to an embodiment of this application. This mapping relationship can be pre-stored. The corresponding driver instruction can be selected for the voice command based on the pre-stored custom information.

[0101] For example, voice commands are input into a neural network to obtain output results; based on the output results and according to custom relationships, corresponding driving commands can be matched, thereby realizing the correspondence between voice commands and driving commands. Figure 7 As shown, different voice commands may use the same neural network or different neural networks, and this application does not limit them.

[0102] This enables rapid correspondence between voice commands and driver commands, thereby improving the response speed of medical devices. Furthermore, by customizing information, commands and actions can be easily and flexibly bound, further enhancing the robustness and scalability of speech recognition.

[0103] In some embodiments, voiceprint detection can be performed on voice commands based on pre-stored voiceprint information; and if the voice command matches the voiceprint characteristics of an authorized user, the system-recognized voice word is output; if the voice command does not match the voiceprint characteristics of an authorized user, the system-recognized voice word is not output.

[0104] For example, during authorized user use, the received voice command is detected based on pre-stored voiceprint information to determine whether the voice command comes from an authorized user. If there is no voiceprint feature matching the voice command in the pre-stored voiceprint information, the voice command is considered to come from an unauthorized user, and the first deep neural network does not output the system-recognized voice word.

[0105] Taking a surgery as an example, if the surgeon is an authorized user and other personnel such as assistant doctors and nurses are unauthorized users, only the surgeon's voice can be recognized and output as the system's recognized speech word to generate driving commands. The voices of other personnel cannot generate driving commands. Even if other personnel issue valid voice commands (such as "move to the left"), the medical device will not perform the corresponding action.

[0106] This prevents unauthorized users from driving medical devices and improves their resistance to interference.

[0107] For example, taking a surgery as an example, if the voiceprint feature from authorized user A is the first voiceprint feature belonging to an authorized user detected by the medical system, then in this surgery, only the voiceprint of authorized user A will be identified, and other authorized users will not be able to use the voice-driven function in this surgery, thus avoiding the problem of multiple voice commands being confused.

[0108] For example, if it is necessary to add a user who can use the voice-driven function during this surgery, authorized user A can issue a corresponding voice command, such as "activate user B". If authorized user B has input their voiceprint characteristics, the medical system can recognize those characteristics, meaning that authorized user B can also use the voice-driven function during this surgery. As one example, both authorized user A and authorized user B can use voice-driven functions in this situation. As another example, authorized user A is automatically deactivated, and authorized user B takes over the voice-driven function.

[0109] This can further improve the flexibility of voice command recognition.

[0110] In some embodiments, if the voice command matches the voiceprint characteristics of an authorized user, the voice command is switched on / off based on pre-stored wake-up / off information; and if the voice command includes wake-up information, the driver of the medical device is turned on; and if the voice command includes off information, the driver of the medical device is turned off.

[0111] For example, authorized users can customize a "wake-up word" or use the factory-set "wake-up word" to enable the voice-driven function. When an authorized user issues a wake-up command, the medical device's driver receives the wake-up command from the sound pickup device and processes it.

[0112] For example, voiceprint comparison technology can be used to verify whether the wake-up command comes from an authorized user. If so, the corresponding driving command is determined by the output of the neural network. The voice assistant is then woken up based on the driving command to receive subsequent voice commands. Authorized users can also customize a "close word" or use the factory-set "close word" to disable the voice driving function. This application embodiment does not limit this.

[0113] This can further improve the accuracy and security of voice command recognition, and enhance anti-interference performance.

[0114] The embodiments of this application have been illustrated above, but are not limited thereto. Furthermore, the above embodiments can be implemented individually or in combination of two or more embodiments. For example, the embodiments can be combined when implementing a voice-driven medical device.

[0115] Figure 8 This is another schematic diagram of a driving method for a medical device according to an embodiment of this application, and will be described from the perspective of the driving device of the medical device. Figure 8 As shown, the method includes:

[0116] 801, Receive voice commands from the sound pickup device;

[0117] 802, input the voice command into the deep learning neural network;

[0118] 803, perform voiceprint detection on the voice command based on the pre-stored voiceprint information; and execute 804 if the voice command matches the voiceprint characteristics of an authorized user, and execute 801 if the voice command does not match the voiceprint characteristics of an authorized user;

[0119] 804, if the voice command matches the voiceprint characteristics of an authorized user, perform on / off detection on the voice command based on pre-stored wake-up / off information; and if the voice command includes wake-up information, execute 805 to enable the driver for the medical device; and if the voice command includes off information, execute 801 to disable the driver for the medical device.

[0120] 805, Automatic Speech Recognition (ASR) of voice commands based on a first deep learning neural network, in order to extract features and output the recognized speech words of the system;

[0121] 806, based on a second deep learning neural network, performs natural language processing (NLP) on the speech words recognized by the system to perform semantic analysis and output driving instructions for driving medical devices;

[0122] 807, drives the medical device according to the drive instructions.

[0123] It is worth noting that the above appendix Figure 8 The embodiments of this application have only been illustrated schematically, and the application is not limited thereto. For example, some of the above steps can be performed simultaneously or in a specific order, and the execution order between the various operations can be appropriately adjusted (for example, the order of 803 and 804 can be replaced). In addition, other operations can be added or some operations can be removed. Those skilled in the art can make appropriate modifications based on the above content, and are not limited to the above-described embodiments. Figure 8 The records.

[0124] The above illustrative description illustrates the driving mechanism of the embodiments of this application. The following illustrative description illustrates the training mechanism of this application.

[0125] In some embodiments, deep learning neural networks can be trained using training samples.

[0126] For example, offline training can be performed, such as training a deep learning neural network before speech recognition, and then using the trained deep learning neural network to drive actual medical devices. Alternatively, online training can be performed, such as training the model simultaneously during the actual driving process of medical devices. Another example is combining offline and online training.

[0127] Therefore, through model training, the accuracy and reliability of speech recognition can be further improved.

[0128] In some embodiments, training the deep learning neural network using training samples includes: performing speech command recognition (SCR) on the training samples based on the deep learning neural network to extract features and output feature vectors; determining the difference between the feature vectors and feature vectors of other speech commands based on semantic distance; and determining the training samples as valid samples if the difference is greater than or equal to a preset threshold.

[0129] Figure 9 This is a schematic diagram of deep learning neural network training according to an embodiment of this application. Figure 9 As shown, the method includes:

[0130] 901, Receive training samples from authorized users;

[0131] 902, Based on deep learning neural networks, speech command recognition (SCR) is performed on training samples to extract features and output feature vectors;

[0132] 903, based on semantic distance, determine the difference between the feature vector and the feature vector of other speech commands; and

[0133] 904. If the difference is greater than or equal to a preset threshold, the training sample is determined to be a valid sample.

[0134] It is worth noting that the above appendix Figure 9 The embodiments of this application have only been illustrated schematically, and the application is not limited thereto. For example, some of the above steps can be performed simultaneously or in a specific order, and the execution order between the various operations can be appropriately adjusted. Furthermore, other operations can be added or some operations can be removed. Those skilled in the art can make appropriate modifications based on the above description, and are not limited to the above-described embodiments. Figure 9 The records.

[0135] Figure 10 This is an example diagram of deep learning neural network training according to an embodiment of this application.

[0136] like Figure 10 As shown, SCR, for example, includes preprocessing of training samples, feature extraction, and training of deep learning neural networks. Preprocessing operations include, but are not limited to, noise reduction, resampling, and channel coordination; feature extraction operations include, but are not limited to, short-time Fourier transform, Mel-frequency cepstral coefficients, linear predictive coding, and cepstral coefficients based on perceptual features; and deep learning neural networks include, but are not limited to, recurrent neural networks, long short-term memory networks, convolutional neural networks, or Transformer networks.

[0137] For example, semantic distance discrimination includes, but is not limited to, determining the difference between feature vectors and feature vectors of other voice commands based on the minimum decoding distance of built-in labels. For example, if the voice command "move left" has been trained and pre-stored, and if during training the user issues "move left", the difference between the feature vector of "move left" and the feature vector of "move left" is, for example, below a threshold, then this training sample of "move left" can be considered invalid.

[0138] For example, the voice command "move left" has already been trained and stored. If, during training, the user issues the command "move left 2 centimeters", and the difference between the feature vector of "move left 2 centimeters" and the feature vector of "move left" is greater than a threshold, then the training sample of "move left 2 centimeters" can be considered valid.

[0139] Therefore, semantic distance discrimination can be used to evaluate the separation of commands, identify the differences between custom commands and all pre-set commands, and thus determine whether the current custom command is concise and unambiguous, clarify the driving command corresponding to the voice command, and avoid ambiguity and command confusion that could lead to the execution of incorrect commands.

[0140] In some embodiments, the deep learning neural network training method further includes: creating a custom mapping table between voice commands and driving commands. For example, it can be as follows: Figure 7 As shown, based on the authorized user's custom voice commands and the current deep learning neural network, the output of the deep learning network is determined, thereby determining the corresponding driving command.

[0141] Figure 11 This is a schematic diagram showing the model training process in an embodiment of this application.

[0142] For example, such as Figure 11 As shown, authorized users select the function they want to configure, and then input their desired voice command. The voice command recognition system outputs the current deep learning network's output. After semantic distance determination, it is determined whether the voice command is a valid sample. If it is a valid sample, the voice command corresponds to the current function, i.e., the custom voice command corresponds to the current driving command. If it is an invalid sample, the authorized user is prompted to reset.

[0143] This clarifies the correspondence between voice commands and driver commands. When using a medical system, quickly identifying the corresponding driver command for an input voice command improves the response speed of the medical device.

[0144] The above illustrations illustrate the training process, but this application is not limited to them. For specific details regarding model training and semantic distance discrimination, please refer to relevant technologies. Deep learning neural networks adapt to a department by learning the usage habits of a specific group (e.g., authorized users within the same department). Furthermore, deep learning neural networks can also be applied to a region or area, for example, by accessing a hospital's or region's speech database to obtain sufficient training samples, thereby adapting to the region's accent, hospital-specific vocabulary, and grammar.

[0145] This application also provides a driving device for a medical device, including a processor and a memory. The processor is configured to execute the aforementioned driving method for the medical device. For example, the processor is configured to perform the following operations: receiving a voice command from a voice pickup device; inputting the voice command into a deep learning neural network and outputting a driving command for driving the medical device based on the deep learning neural network; and driving the action of the medical device according to the driving command.

[0146] This application also provides a medical system.

[0147] Figure 12 This is a schematic diagram of a medical system according to an embodiment of this application. The medical system includes: a sound pickup device 1201, a medical device 1203, and a driving device 1202 for the medical device 1203. Furthermore, as... Figure 12 As shown, the medical system also includes display devices such as 1204.

[0148] The voice pickup device 1201 receives voice commands from the user; the driving device 1202 inputs the voice commands into a deep learning neural network and outputs driving commands for driving the medical device 1203 based on the deep learning neural network; and the medical device 1203 performs actions according to the driving commands. The display device 1204 displays images acquired by the medical device and / or voice commands recognized by the driving device 1202.

[0149] In some embodiments, the sound pickup device 1201 can be any form of sound receiving device, such as a wearable microphone fixed to the user or a microphone fixed to a medical device, etc., and this application embodiment is not limited thereto. The sound pickup device 1201 can acquire the user's voice commands in real time, and it is connected to the medical device driving device 1202 wirelessly or wiredly. The medical device driving device 1202 processes the voice information from the sound pickup device 101 and outputs driving commands for driving the medical device based on a neural network.

[0150] like Figure 12 As shown, the driving device 1202 of the medical device may include one or more processors (e.g., a central processing unit, CPU) 1202a and one or more memories 1202b; the memories 1202b are coupled to the processors 1202a. The memories 1202b can store various data such as user-defined instructions, voiceprint data, and historical information from user voice commands. The medical device 1203 and the display device 1204 operate based on received driving instructions. The medical device 1203 and the display device 1204 are connected to the driving device 1202 wirelessly or via a wired connection.

[0151] In some embodiments, the display device 1204 also displays historical information of voice commands from the user over a period of time.

[0152] For example, the display device 1204 not only allows users to view images from the CT device 1203, but also displays voice commands recognized by the CT device's drive unit 1202 and historical information of voice commands from authorized users over a period of time.

[0153] Figure 13 This is an example diagram of the display interface of a display device according to an embodiment of this application.

[0154] For example, such as Figure 13The display device 1204 displays images from the CT device 1203, currently input voice commands, and historical information of voice commands (command history). For example, based on the user's currently input voice command "Page down 1", the drive device 1202 controls the display device 1204 to switch the image to the next page.

[0155] Therefore, by displaying the historical information of voice commands, users or other personnel can confirm the voice commands and thus predict the actions of medical devices. Even if the voice commands of authorized users are not clearly heard, the relevant information can still be obtained through the display device.

[0156] The above embodiments are merely illustrative examples of embodiments of this application, but this application is not limited thereto, and appropriate modifications can be made based on the above embodiments. For example, the above embodiments can be used alone, or one or more of the above embodiments can be combined.

[0157] For simplicity, the figures above only illustrate the connection relationships or signal flows between the various components or modules. However, those skilled in the art should understand that various related technologies, such as bus connections, can be used. The various components or modules described above can be implemented using hardware facilities such as processors and memory; the embodiments of this application do not limit this.

[0158] This application also provides a computer-readable program or program product, wherein when the program is executed in an electronic device, the program causes the computer in the electronic device to perform the driving method of the medical device as described in the preceding embodiments.

[0159] This application also provides a storage medium storing a computer-readable program, wherein the computer-readable program causes a computer in an electronic device to execute the driving method of the medical device as described in the preceding embodiments.

[0160] The apparatus and methods described above in this application can be implemented in hardware or in combination with software. This application relates to a computer-readable program that, when executed by a logic component, enables the logic component to implement the apparatus or components described above, or to implement the various methods or steps described above. This application also relates to storage media for storing the above programs, such as hard disks, magnetic disks, optical disks, DVDs, flash memory, etc.

[0161] The methods / apparatus described in conjunction with the embodiments of this application can be directly embodied in hardware, software modules executed by a processor, or a combination of both. For example, one or more and / or combinations of one or more functional block diagrams shown in the figures can correspond to various software modules in a computer program flow, or to various hardware modules. These software modules can correspond to the various steps shown in the figures, respectively. These hardware modules can be implemented, for example, using a field-programmable gate array (FPGA) to embed these software modules.

[0162] The software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. A storage medium can be coupled to the processor, enabling the processor to read information from and write information to the storage medium; or the storage medium can be an integral part of the processor. The processor and storage medium can reside in an ASIC. The software module can be stored in the memory of a mobile terminal or in a memory card that can be inserted into the mobile terminal. For example, if the device (such as a mobile terminal) uses a high-capacity MEGA-SIM card or a high-capacity flash memory device, the software module can be stored in the MEGA-SIM card or the high-capacity flash memory device.

[0163] One or more and / or one or more combinations of functional blocks described in the accompanying drawings can be implemented as a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, or any suitable combination thereof for performing the functions described herein. One or more and / or one or more combinations of functional blocks described in the accompanying drawings can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in communication with a DSP, or any other such configuration.

[0164] The present application has been described above with reference to specific embodiments. However, those skilled in the art should understand that these descriptions are exemplary and not intended to limit the scope of protection of the present application. Those skilled in the art can make various modifications and variations to the present application based on the principles thereof, and these modifications and variations are also within the scope of the present application.

Claims

1. A driving method of a medical device, characterized by, The method comprises: receiving a voice instruction from a sound pickup device; inputting the voice instruction into a deep learning neural network and outputting a driving instruction for driving the medical device based on the deep learning neural network; and driving the medical device according to the driving instruction.

2. The method of claim 1, wherein, Outputting the driving instruction for driving the medical device based on the deep learning neural network comprises: performing automatic speech recognition on the voice instruction based on a first deep learning neural network to perform feature extraction and output system-recognized voice words; and performing natural language processing on the system-recognized voice words based on a second deep learning neural network to perform semantic analysis and output the driving instruction for driving the medical device.

3. The method of claim 2, wherein, The method further comprises: selecting a corresponding driving instruction for the voice instruction according to pre-stored customized information, wherein the customized information comprises a corresponding relationship between voice instructions and driving instructions.

4. The method of claim 2, wherein, The method further comprises: performing voiceprint detection on the voice instruction based on pre-stored voiceprint information; and outputting the system-recognized voice words if the voice instruction conforms to the voiceprint characteristics of an authorized user, and not outputting the system-recognized voice words if the voice instruction does not conform to the voiceprint characteristics of the authorized user.

5. The method of claim 4, wherein, The method further comprises: performing on-off detection on the voice instruction based on pre-stored wake-up information / shut-down information if the voice instruction conforms to the voiceprint characteristics of an authorized user; and turning on the driving of the medical device if the voice instruction comprises the wake-up information, and turning off the driving of the medical device if the voice instruction comprises the shut-down information.

6. The method of claim 1, wherein, The method further comprises: training the deep learning neural network using training samples.

7. The method of claim 6, wherein, Training the deep learning neural network using training samples comprises: performing voice command recognition on the training samples based on the deep learning neural network to perform feature extraction and output feature vectors; determining the difference between the feature vectors and feature vectors of other voice instructions according to semantic distances; and determining that the training sample is a valid sample if the difference is greater than or equal to a preset threshold.

8. The method of claim 6, wherein, The method further comprises: selecting a corresponding driving instruction for the training sample and storing customized information, wherein the customized information comprises a corresponding relationship between voice instructions and driving instructions.

9. A driving device of a medical device, comprising a processor and a memory, wherein the processor is configured to perform the driving method of the medical device according to any one of claims 1 to 8.

10. A medical system, comprising: a sound pickup device configured to receive a voice instruction from a user; a driving device configured to input the voice instruction into a deep learning neural network and output a driving instruction for driving a medical device based on the deep learning neural network; and a medical device configured to perform an action according to the driving instruction. The medical system further comprises:

11. The medical system of claim 10, wherein, a display device configured to display images captured by the medical device and the voice instruction recognized by the driving device. The medical system further comprises: a display device configured to display images captured by the medical device and the voice instruction recognized by the driving device.

12. The medical system of claim 11, wherein, The display device also displays historical information of voice instructions from the user over a period of time.

13. The medical system of claim 10, wherein, The sound pickup device includes a wearable microphone fixed to a user or a microphone fixed to a medical device.

Images