Voice interaction control method and control system for ear-nose-throat operation power equipment

Abstract

The invention discloses a voice interaction control method and a voice interaction control system for ear-nose-throat operation power equipment. The voice interaction control method comprises the following steps: S1, acquiring voice data; s2, recognizing the voice data and analyzing the voice data into a control instruction, wherein the control instruction comprises at least one of an operation mode instruction, a motor rotating speed instruction or a steering control instruction; s3, synthesizing instruction confirmation voice according to the control instruction, and playing and outputting the instruction confirmation voice; and S4, setting operation parameters of the power equipment according to the control instruction. The ear-nose-throat operation power equipment is controlled by adopting a voice interaction control method, a surgeon can regulate and control the operation parameters of the equipment by himself / herself according to actual requirements, nurses in an operating room do not need to be matched with manual setting, misoperation of information transmission is reduced, and the operation efficiency is improved. Fine parameter adjustment can increase the degree of freedom of surgery operation of doctors, the doctors concentrate on completing various high-difficulty surgeries, errors caused by manual setting are reduced, and the safety of the surgeries is greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical devices, and in particular to a voice interactive control method and a control system for ENT surgery power equipment. Background Art

[0002] With the advancement of medical technology and the improvement of patients' requirements for surgical safety, traditional surgery is gradually being replaced by minimally invasive surgery. In the field of ENT, ENT surgical power system is an indispensable device in ENT minimally invasive surgery.

[0003] At present, most of the ENT surgery power systems at home and abroad are manually operated power systems, which require operating room nurses to manually set according to the various surgical needs of the surgeon. The surgeon needs to use the pedal button to control the one-way or two-way drive of the power system to meet the flexible operation during the operation.

[0004] Due to the differences in proficiency of surgical nurses in power system settings, as well as the differences in skills and experience of different doctors, human operation errors are easily caused, which increase machine wear, reduce machine accuracy, and shorten machine life. More importantly, manual settings have errors and uncertainties, which will cause the ENT surgery power system to fail to switch the rotation mode in time when it is working, increasing the risk of surgery on important parts such as the skull base, blood vessels and nerves. Summary of the invention

[0005] The purpose of the present invention is to solve one of the technical problems in the related art to a certain extent at least. In view of this, on the one hand, the present invention provides a voice interactive control method for ENT surgery power equipment, comprising the following steps: S1: collecting voice data; S2: identifying voice data and parsing it into control instructions, wherein the control instructions include at least one of surgical mode instructions, motor speed instructions or steering control instructions; S3: synthesizing instructions according to the control instructions to confirm the voice and output it externally; S4: setting the operating parameters of the power equipment according to the control instructions.

[0006] According to an example of the present invention, step S2 includes: S21: collecting voice analog signals and preprocessing them to obtain voice digital signals; S22: converting the voice digital signals into acoustic features that can be understood by a voice recognition model; S23: inputting the acoustic features into a pre-built voice recognition model to infer text data; S24: matching the text data with a built-in control instruction set of a power device to obtain control instructions.

[0007] According to an example of the present invention, the speech recognition model is constructed by combining a multi-layer convolutional neural network and a connection temporal classification model, and the network structure of the speech recognition model consists of 5 layers of 3*3 convolutional layers, linear layers and connection temporal classification losses.

[0008] According to an example of the present invention, the method further includes: presetting a matching scheme, wherein the matching scheme is a matching data table of surgical mode instructions and motor speed instructions and steering control instructions corresponding to the surgical mode instructions; if the control instruction obtained by parsing in step S2 is a surgical mode instruction, further parsing according to the surgical mode instruction and the matching scheme to obtain the corresponding motor speed instruction and steering control instruction.

[0009] According to an example of the present invention, between step S3 and step S4, the method further includes: acquiring secondary confirmation voice data and parsing it, and if the parsing result includes confirmation execution information, entering step S4.

[0010] On the other hand, the present invention also provides a voice interactive control system for ENT surgery power equipment, characterized in that it includes: a voice acquisition module for collecting voice data; a voice analysis module for identifying voice data and parsing it into control instructions, wherein the control instructions include at least one of surgical mode instructions, motor speed instructions or steering control instructions; a voice playback module for synthesizing instructions according to the control instructions to confirm the voice and output it externally; and a voice setting module for setting the operating parameters of the power equipment according to the control instructions.

[0011] According to an example of the present invention, the speech analysis module includes: an audio preprocessing unit, which is used to collect speech analog signals and perform preprocessing to obtain speech digital signals; a digital signal processing unit, which is used to convert the speech digital signals into acoustic features that can be understood by a speech recognition model; a speech algorithm processing unit, which is used to input the acoustic features into a pre-built speech recognition model to infer text data; and an instruction matching unit, which is used to match the text data with a built-in control instruction set of a power device to obtain a control instruction.

[0012] According to an example of the present invention, the speech recognition model is constructed by combining a multi-layer convolutional neural network and a connection temporal classification model, and the network structure of the speech recognition model consists of 5 layers of 3*3 convolutional layers, linear layers and connection temporal classification losses.

[0013] According to an example of the present invention, the speech analysis module is also used to further analyze and obtain corresponding motor speed instructions and steering control instructions according to the surgical mode instructions and the matching scheme when the control instructions obtained by analysis are surgical mode instructions; the matching scheme is a matching data table of preset surgical mode instructions and the motor speed instructions and steering control instructions corresponding to the surgical mode instructions.

[0014] According to an example of the present invention, it also includes: a secondary confirmation module, which is used to obtain and parse the secondary confirmation voice data, and if the analysis result contains confirmation execution information, the voice setting module is started.

[0015] Compared with the prior art, the technical solution of the present invention has the following advantages: 1. Use voice interaction control to control ENT surgery power equipment. The surgeon can adjust the equipment operating parameters according to actual needs, such as adjusting the typical working mode switch of the power equipment or more specific motor speed and steering control. There is no need for operating room nurses to cooperate with manual settings, which reduces misoperation of information transmission. This contactless voice interaction control can free the hands of doctors and nurses and improve surgical efficiency. Refined parameter adjustment can increase the doctor's freedom of surgical operation, concentrate on completing various difficult operations, reduce errors caused by manual settings, and greatly increase the safety of surgery.

[0016] 2. Aiming at the small vocabulary and short voice command environment of ENT surgery, a lightweight speech recognition model is designed and adapted. The model has a simple structure, small calculation amount, and fast processing speed. Compared with other large network models, it can better capture the local features of short speech.

[0017] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 It is a flow chart of the steps of a voice interactive control method of an ear, nose and throat surgery power device of the present invention.

[0020] Figure 2 It is a schematic diagram of the framework structure of the voice command recognition algorithm of the present invention.

[0021] Figure 3 It is a flow chart of the voice command recognition algorithm of the present invention.

[0022] Figure 4 It is a schematic diagram of a voice interaction process disclosed by the present invention.

[0023] Figure 5 It is a structural block diagram of a voice interactive control system of an ear, nose and throat surgery power device of the present invention. DETAILED DESCRIPTION

[0024] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0025] Example 1 See also Figure 1 As shown, the present invention provides a voice interactive control method for ENT surgery power equipment, the method specifically comprising the following steps: S1: collect voice data; In this embodiment, a microphone array is used to collect the analog signal of the surgeon's control voice, preferably a circular microphone array, which can achieve omnidirectional sound collection. Generally speaking, for the collected voice analog signal, an audio collection front end can be connected to the microphone array for preliminary noise reduction processing.

[0026] S2: recognizing voice data and parsing it into control instructions, where the control instructions include at least one of a surgical mode instruction, a motor speed instruction, or a steering control instruction; like Figure 2 and 3 As shown, the framework structure and flow chart of the voice control command recognition algorithm of the present invention are shown, which specifically include: S21: Collecting voice analog signals and preprocessing them to obtain voice digital signals; The input voice analog signal is preprocessed, including sampling, quantization, silence detection, and denoising, to obtain a voice digital signal.

[0027] S22: Convert the speech digital signal into acoustic features that can be understood by the speech recognition model; The voice digital signal obtained in the above steps is subjected to feature extraction, and the original audio digital signal is converted into acoustic features that can be understood by the speech recognition model. The feature extraction step at least includes pre-emphasis, framing, windowing, fast Fourier transform, spectrum line energy calculation, Mel filtering, and logarithmic operation to obtain Fbank (Filter bank) features, which retain more original voice information and can give full play to the feature extraction capabilities of the neural network.

[0028] S23: Input the acoustic features into the pre-built speech recognition model to infer text data; A network model combining a pre-built multi-layer convolutional neural network and a connection temporal classification model is used as the speech recognition model of the present invention. In the present invention, the network structure of the model is set to be composed of 5 layers of 3*3 convolutional layers, linear layers and connection temporal classification loss (CTC_loss), which can better capture the local features of short speech and has a simple structure. The Fbank features obtained in the above steps are input into the pre-built speech recognition model to infer the speech text data.

[0029] The corpus required for the training of the above-mentioned speech recognition model can be adapted to the ENT surgery scene for precise training. 200 command texts related to the control of the ENT surgery power system are selected and mapped into 80 control instructions. The mapping relationship between the command text and the control instruction is that 1 control instruction can correspond to 1 or more command texts. These command texts were recorded into speech by 10 people, forming 2000 voice samples. In addition to the aforementioned instructions related to the control of motor speed and steering, based on the operating experience of power equipment in ENT surgery, multiple types of surgeries can also be preset, and the default operating parameters are automatically set for each type of surgery, such as the "ear drill grinding drill" type, which corresponds to one of the surgical modes. In this mode, the default setting operating parameters are that the motor rotates clockwise at a speed of 70,000 rpm.

[0030] The results of the speech recognition model trained on the self-made corpus of ENT surgery scenarios are evaluated using the character error rate (CER), as shown in formula (1).

[0031] (1) Because the application scenario of the speech algorithm of the present invention is the voice command control of the power system, the word error rate cannot directly reflect the control result. The present invention defines an evaluation index, the command accuracy rate R, which reflects the probability of correct execution of the control command in the actual application scenario, as shown in formula (2).

[0032] (2) The final test results of the model of the present invention are a word error rate of 5.02% and a command accuracy rate of 92.0%. With a model size of 350kb and a running speed of 5ms, it meets the application scenario of small vocabulary voice command recognition on terminal devices. Experiments show that the voice command recognition algorithm has excellent performance.

[0033] S24: Match the text data with a built-in control instruction set of the power equipment to obtain a control instruction.

[0034] S3: Synthesize the command according to the control command to confirm the voice and output it externally; S4: Set the operating parameters of the power equipment according to the control instructions.

[0035] According to an example of the present invention, the method further includes: presetting a matching scheme, the matching scheme is a matching data table of a surgical mode instruction and a motor speed instruction and a steering control instruction corresponding to the surgical mode instruction, and multiple typical surgical modes can be preset, each surgical mode corresponds to a specific power system motor speed and its motor steering, and this design can simplify the complexity of the voice that the operator needs to output. If the control instruction obtained by parsing in step S2 is a surgical mode instruction, the corresponding motor speed instruction and steering control instruction are further parsed according to the surgical mode instruction and the matching scheme.

[0036] To ensure the accuracy and safety of voice command control, between step S3 and step S4, the method also includes a redundant secondary confirmation step: after the first recognition and external output of the control command, the operator needs to further use voice "yes" or "no" to make a secondary confirmation of the recognized command result to ensure safety and reliability. The voice interaction control system will obtain the secondary confirmation voice data and parse it. If the analysis result contains confirmation execution information (such as "yes", "OK", "confirm", etc.), it will enter step S4 and send a control signal to the main control unit of the power system to automatically set the power system operating parameters.

[0037] Figure 4 The present invention discloses a relatively comprehensive voice interaction process, which specifically includes the following steps: ①The ENT surgeon issues voice control commands.

[0038] ② The voice interaction control system collects the surgeon’s voice signals, recognizes the voice commands, and parses them into a preset surgical mode (also called surgical type), or directly parses them into motor speed and steering control commands.

[0039] ③If the analysis result is a surgical mode, it is converted into corresponding motor speed and steering control instructions according to the preset matching scheme.

[0040] ④ The voice interaction control system synthesizes the command confirmation voice according to the parsed control command, and outputs it externally through the sound playback module.

[0041] ⑤ The surgeon-in-charge confirms the recognized command result by voice-voice “yes” or “no” for a second time.

[0042] ⑥ After the voice interaction control system receives and analyzes the secondary confirmation result "yes", it sends a control signal to the main control unit of the power system to automatically set the power system motor operating parameters.

[0043] Example 2 like Figure 5 As shown, this embodiment discloses a voice interactive control system for an ENT surgery power device, comprising: Voice collection module, used to collect voice data; The voice analysis module is used to identify voice data and analyze it into control instructions, and the control instructions include at least one of surgical mode instructions, motor speed instructions or steering control instructions; the voice analysis module is also used to further analyze and obtain the corresponding motor speed instructions and steering control instructions according to the surgical mode instructions and the matching scheme when the control instructions obtained by analysis are surgical mode instructions; the matching scheme is a matching data table of preset surgical mode instructions and motor speed instructions and steering control instructions corresponding to the surgical mode instructions. The voice control system also includes a secondary confirmation module, which is used to obtain secondary confirmation voice data and analyze it. If the analysis result contains confirmation execution information, the voice setting module is started.

[0044] The voice playback module is used to synthesize the command confirmation voice according to the control command and output it externally; The voice setting module is used to set the operating parameters of the power equipment according to the control instructions.

[0045] The speech analysis module includes: an audio preprocessing unit for collecting and preprocessing speech analog signals to obtain speech digital signals; a digital signal processing unit for converting speech digital signals into acoustic features that can be understood by the speech recognition model; a speech algorithm processing unit for inputting acoustic features into the pre-built speech recognition model to infer text data; and an instruction matching unit for matching text data with the built-in control instruction set of the power equipment to obtain control instructions. The speech recognition model is constructed by combining a multi-layer convolutional neural network and a connection time series classification model. The network structure of the speech recognition model consists of 5 layers of 3*3 convolutional layers, linear layers, and connection time series classification losses.

[0046] It is easy to know that the voice interaction control system of the present invention should also include a voice chip MCU and a storage unit. The above-mentioned digital signal processing unit extracts voice features from the voice digital signal output by the audio preprocessing unit; the voice algorithm processing unit communicates bidirectionally with the voice chip MCU to implement the voice recognition algorithm and the operation instruction parsing algorithm. The voice chip MCU controls the operation of the entire system, executes the voice recognition algorithm, processes the user interface and external communications, and sends the final recognized control instructions to the main control unit of the power equipment through the serial port to set the operating parameters of the power system. The storage unit is used to store voice models, feature parameters, program codes, voice synthesis audio, etc., and communicates bidirectionally with the voice chip MCU. The voice playback module is linked to the voice chip MCU and is used to decode and play the recognized control instruction results to the surgeon for secondary confirmation.

[0047] The contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0048] The voice interactive control method and computer program product of the control system of ENT surgery power equipment provided in the embodiments of the present invention include a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods in the previous method embodiments. The specific implementation can be found in the method embodiments, which will not be repeated here.

[0049] Specifically, the storage medium can be a general storage medium, such as a mobile disk, a hard disk, etc. When the computer program on the storage medium is run, it can execute the above-mentioned voice interaction control method, reduce the complexity of the model, and better characterize the voice information to improve surgical efficiency.

[0050] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0051] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

[0052] For those skilled in the art, various changes and modifications will undoubtedly be obvious after reading the above description. Therefore, the attached claims should be regarded as covering all changes and modifications of the true intent and scope of the present invention. Any and all equivalent ranges and contents within the scope of the claims should be considered to still be within the intent and scope of the present invention.

Claims

1. A voice interactive control method for ENT surgery power equipment, characterized in that: The following steps are involved: S1: collect voice data; S2: recognizing voice data and parsing it into control instructions, wherein the control instructions include at least one of a surgical mode instruction, a motor speed instruction or a steering control instruction; S3: synthesizing the command confirmation voice according to the control command and outputting it externally; S4: Setting the operating parameters of the power equipment according to the control instruction.

2. The voice interaction control method according to claim 1, characterized in that: The step S2 comprises: S21: Collecting voice analog signals and preprocessing them to obtain voice digital signals; S22: Converting the speech digital signal into acoustic features that can be understood by a speech recognition model; S23: Input the acoustic features into a pre-built speech recognition model to infer and obtain text data; S24: Match the text data with a built-in control instruction set of the power equipment to obtain a control instruction.

3. The voice interaction control method according to claim 2, characterized in that: The speech recognition model is constructed by combining a multi-layer convolutional neural network and a connection temporal classification model. The network structure of the speech recognition model consists of 5 layers of 3*3 convolutional layers, a linear layer and a connection temporal classification loss.

4. The voice interaction control method according to claim 1, characterized in that: The method further comprises: A preset matching scheme, wherein the matching scheme is a matching data table of a surgical mode instruction and a motor speed instruction and a steering control instruction corresponding to the surgical mode instruction; If the control instruction obtained by analysis in step S2 is a surgery mode instruction, further analysis is performed according to the surgery mode instruction and the matching scheme to obtain the corresponding motor speed instruction and steering control instruction.

5. The voice interaction control method according to claim 1, characterized in that: Between step S3 and step S4, the method further comprises: The secondary confirmation voice data is obtained and analyzed. If the analysis result contains confirmation execution information, the process proceeds to step S4.

6. A voice interactive control system for ENT surgery power equipment, characterized in that: include: Voice collection module, used to collect voice data; A speech analysis module, used for recognizing speech data and analyzing it into control instructions, wherein the control instructions include at least one of a surgery mode instruction, a motor speed instruction or a steering control instruction; A voice playing module, used to synthesize the command confirmation voice according to the control command and output it externally; The voice setting module is used to set the operating parameters of the power equipment according to the control instructions.

7. The voice interaction control system according to claim 6, characterized in that: The speech analysis module comprises: An audio preprocessing unit, used for collecting voice analog signals and preprocessing them to obtain voice digital signals; A digital signal processing unit, used for converting the speech digital signal into acoustic features that can be understood by a speech recognition model; A speech algorithm processing unit, used for inputting the acoustic features into a pre-built speech recognition model to infer and obtain text data; The instruction matching unit is used to match the text data with the built-in control instruction set of the power equipment to obtain the control instruction.

8. The voice interaction control system according to claim 7, characterized in that: The speech recognition model is constructed by combining a multi-layer convolutional neural network and a connection temporal classification model. The network structure of the speech recognition model consists of 5 layers of 3*3 convolutional layers, a linear layer and a connection temporal classification loss.

9. The voice interaction control system according to claim 6, characterized in that: The speech analysis module is also used to further analyze and obtain corresponding motor speed instructions and steering control instructions according to the surgical mode instructions and the matching scheme when the control instructions obtained by analysis are surgical mode instructions; the matching scheme is a matching data table of preset surgical mode instructions and the motor speed instructions and steering control instructions corresponding to the surgical mode instructions.

10. The voice interaction control system according to claim 6, characterized in that: Also includes: The secondary confirmation module is used to obtain the secondary confirmation voice data and parse it. If the parsing result contains confirmation execution information, the voice setting module is started.

Images