Voice interaction method, device and equipment of laparoscopic surgery robot and medium

Abstract

The embodiment of the invention discloses a voice interaction method, device and equipment of a laparoscopic surgery robot and a medium. The method comprises the following steps: acquiring sound information in a space where a console of the laparoscopic surgery robot is located; performing wake-up word recognition on the sound information to obtain a wake-up word recognition result, and obtaining a voice instruction input by a user based on the wake-up word recognition result; and intention recognition is performed on the voice instruction to obtain a target operation instruction of the user for the target equipment, and the target operation instruction is sent to the target equipment, so that the target equipment executes the target operation instruction. Through the technical scheme of the embodiment of the invention, the target operation instruction which is issued by the user through voice and aims at the target equipment is accurately and conveniently determined, non-manual operation of a doctor can be supported, the requirement of multi-line simultaneous operation of the doctor can be considered, and the operation execution efficiency is improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the technical field of medical equipment, and in particular to a voice interaction method, device, equipment and medium for a laparoscopic surgical robot. Background Art

[0002] With the development of technology, surgical robots have gradually been applied to modern medicine and have become one of the important means to improve surgical accuracy and reduce surgical risks.

[0003] At present, in the actual operation process, doctors can use mechanical buttons and touch screens on the console to realize functions such as armrest raising and lowering, display raising and lowering, and intraoperative surgical operation ratio precision adjustment to meet the human needs of different doctors.

[0004] However, this operation mode can often only support manual operation by doctors and cannot take into account the needs of doctors to operate multiple lines simultaneously, thus reducing the efficiency of surgical execution. Summary of the invention

[0005] The embodiments of the present invention provide a voice interaction method, device, equipment and medium for a laparoscopic surgical robot, so as to accurately and conveniently determine the target operation instructions for the target device issued by the user through voice, which can support the doctor's non-manual operation and take into account the doctor's needs for multi-line simultaneous operation, thereby improving the efficiency of surgical execution.

[0006] In a first aspect, an embodiment of the present invention provides a voice interaction method for a laparoscopic surgical robot, comprising:

[0007] Acquiring sound information in the space where the console of the laparoscopic surgical robot is located;

[0008] Obtaining a wake-up word recognition result by performing wake-up word recognition on the sound information, and acquiring a voice command input by the user based on the wake-up word recognition result;

[0009] The target operation instruction of the user for the target device is obtained by performing intention recognition on the voice instruction, and the target operation instruction is sent to the target device so that the target device executes the target operation instruction.

[0010] Optionally, the method further includes: performing feature extraction on the sound information based on a Mel-cephalogram coefficient algorithm to obtain sound features; and performing wake-up word recognition on the sound features based on a target network model to obtain a wake-up word recognition result.

[0011] Optionally, the method further includes: if it is detected that the wake-up word recognition result is that there is a wake-up word, obtaining a voice command input by the user; if it is detected that the wake-up word recognition result is that there is no wake-up word, continuing to obtain sound information in the space where the console of the laparoscopic surgical robot is located.

[0012] Optionally, the method further includes: performing intent recognition on the voice command based on a bidirectional gated recurrent unit network combined with an attention mechanism model to obtain a target operation instruction of the user for the target device.

[0013] Optionally, the method also includes: inputting the voice instruction into a bidirectional gated recurrent unit network combined with an attention mechanism model, determining each word vector corresponding to the voice instruction, and performing feature extraction on each word vector to obtain word features corresponding to each word vector; determining the word feature weight corresponding to each word feature, and performing intent recognition based on each word feature weight and each word vector to obtain the user's target operation instructions for the target device.

[0014] Optionally, the method further includes: after the target operation instruction is executed, performing speech synthesis according to the target execution result of the target device to obtain feedback voice information, and playing the feedback voice information.

[0015] In a second aspect, an embodiment of the present invention further provides a voice interaction device for a laparoscopic surgical robot, the device comprising:

[0016] A sound information acquisition module, used to acquire sound information in the space where the console of the laparoscopic surgical robot is located;

[0017] A voice command acquisition module, configured to obtain a wake-up word recognition result by performing wake-up word recognition on the sound information, and obtain a voice command input by the user based on the wake-up word recognition result;

[0018] The target operation instruction determination module is used to obtain the user's target operation instruction for the target device by performing intent recognition on the voice instruction, and send the target operation instruction to the target device so that the target device executes the target operation instruction.

[0019] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0020] one or more processors;

[0021] A memory for storing one or more programs;

[0022] When the one or more programs are executed by the one or more processors, the one or more processors implement the voice interaction method of the laparoscopic surgical robot provided in any embodiment of the present invention.

[0023] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a voice interaction method for a laparoscopic surgical robot as provided in any embodiment of the present invention.

[0024] In a fifth aspect, an embodiment of the present invention provides a computer program product, including a computer program, which, when executed by a processor, implements the voice interaction method of a laparoscopic surgical robot as provided in any embodiment of the present invention.

[0025] The technical solution of the embodiment of the present invention obtains the sound information in the space where the console of the laparoscopic surgical robot is located; obtains the wake-up word recognition result by performing wake-up word recognition on the sound information, and obtains the voice command input by the user based on the wake-up word recognition result; accurately and conveniently determines the target operation command for the target device issued by the user through voice by performing intention recognition on the voice command, and sends the target operation command to the target device so that the target device executes the target operation command. It can support non-manual operation of doctors, and can take into account the needs of doctors for multi-line simultaneous operation, thereby improving the efficiency of surgical execution.

[0026] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0028] Figure 1 is a flow chart of a voice interaction method of a laparoscopic surgical robot provided in Embodiment 1 of the present invention;

[0029] Figure 2 This is an example diagram of a sound feature extraction process based on a Mel-cepstral coefficient algorithm involved in the first embodiment of the present invention;

[0030] Figure 3 This is an example diagram of a wake-up word recognition process based on a target network model involved in the first embodiment of the present invention;

[0031] Figure 4 is a flow chart of a voice interaction method of a laparoscopic surgical robot provided in Embodiment 2 of the present invention;

[0032] Figure 5 This is a structural example diagram of a BiGRU-Attention model involved in Embodiment 2 of the present invention;

[0033] Figure 6 It is a structural schematic diagram of a voice interaction device of a laparoscopic surgical robot provided in Embodiment 3 of the present invention;

[0034] Figure 7 It is a schematic diagram of the structure of an electronic device for implementing the voice interaction method of a laparoscopic surgical robot according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0036] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0037] Embodiment 1

[0038] Figure 1 A flowchart of a voice interaction method for a laparoscopic surgical robot is provided for the first embodiment of the present invention. This embodiment can be applied to the case of determining a target operation instruction for a target device issued by a user through voice. The method can be executed by a voice interaction device of a laparoscopic surgical robot. The voice interaction device of the laparoscopic surgical robot can be implemented in the form of hardware and / or software. The voice interaction device of the laparoscopic surgical robot can be configured in an electronic device. Figure 1 As shown, the method includes:

[0039] S110, obtaining sound information in the space where the console of the laparoscopic surgical robot is located.

[0040] The console may refer to a console where a user controls a laparoscopic surgical robot to perform a surgical operation. The console may include a microphone array, a speaker, and a processor. The space may refer to a space where a microphone in the console is located. For example, the space may be an operating room. The spatial range is proportional to the sound collection capability of the microphone array. Sound information may refer to all sounds in the space. For example, sound information may include, but is not limited to, environmental sounds and user speech sounds.

[0041] Specifically, the microphone in the console collects sound information in real time in the space where the console of the laparoscopic surgical robot is located.

[0042] S120: Obtain a wake-up word recognition result by performing wake-up word recognition on the sound information, and obtain a voice command input by the user based on the wake-up word recognition result.

[0043] The wake-up word recognition result may refer to whether the wake-up word is recognized. For example, the wake-up word recognition result may include whether the wake-up word exists or not. The voice command may refer to the operation command said by the user within a period of time after saying the wake-up word.

[0044] Specifically, when the console is in standby mode, it will use the microphone to continuously collect and process various sounds in the surrounding environment at a lower power consumption, and continuously determine whether the user uses the wake-up word to wake up the machine. If there is no wake-up word in the sound, the console continues to remain in standby mode, otherwise it wakes up the console and enters the working state. After the console enters the working state, it will use the microphone to collect the sound made by the user for a period of time, and then identify the user's command intention in this sound and perform the corresponding work. After the command action is completed, the console will return to standby mode.

[0045] Based on the above technical solution, "obtaining a wake-up word recognition result by performing wake-up word recognition on sound information" may include: extracting features of sound information based on a Mel-cephalogram coefficient algorithm to obtain sound features; and performing wake-up word recognition on sound features based on a target network model to obtain a wake-up word recognition result.

[0046] The target network model is composed of several residual network (resnet) architectures. The residual network architecture includes convolution (CONV), batch normalization (Batch Norm), linear rectification function (ReLU), pooling, and full connection (Fully Connect). Batch Norm keeps the input of each layer of the neural network in the same distribution during the deep neural network training process, which can be used to improve the training stability of the model and accelerate the convergence process.

[0047] Specifically, Figure 2 A sound feature extraction process example based on the Mel-frequency cepstral coefficient algorithm is given. Figure 2 , perform preprocessing operations on the input sound information, such as pre-emphasis. The preprocessed information is converted into the time-frequency domain (such as short-time Fourier transform) to obtain the speech feature sequence, and the power spectrum is calculated. The preprocessed information is then filtered and analyzed using the Gamma tone filter group, and the long-time power spectrum of the preprocessed information is calculated, and the asymmetric noise is suppressed using asymmetric and temporary masking filters. The multi-channel time-frequency and power of the preprocessed information are normalized, and then the power is adjusted. The normalized result is subjected to power function nonlinear processing to obtain the sound characteristics. Figure 3 An example diagram of the wake-up word recognition process based on the target network model is given. Figure 3 , the sound features are input into the target network model, and after passing through multiple residual networks, convolution processing, batch normalization, pooling and full connection are performed again to achieve wake-up word recognition and obtain the wake-up word recognition result.

[0048] Based on the above technical solution, "obtaining the voice command input by the user based on the wake-up word recognition result" may include: if the wake-up word recognition result is detected to be the presence of a wake-up word, then obtaining the voice command input by the user; if the wake-up word recognition result is detected to be the absence of a wake-up word, then continuing to obtain the sound information in the space where the console of the laparoscopic surgical robot is located.

[0049] Specifically, if the wake-up word recognition result is detected to be the absence of the wake-up word, the controller continues to remain in standby mode and continues to obtain sound information in the space where the console of the laparoscopic surgical robot is located until the wake-up word recognition result is detected to be the presence of the wake-up word, and then obtains the voice command input by the user.

[0050] S130, obtaining a target operation instruction of the user for the target device by performing intent recognition on the voice instruction, and sending the target operation instruction to the target device so that the target device executes the target operation instruction.

[0051] The target device may refer to a device that the user expects to operate. For example, the target device may include but is not limited to a console, an imaging trolley, and a patient surgical platform. The target operation instruction may refer to the surgical operation that the user expects the target device to perform. For example, the target operation instruction may include but is not limited to account login, department selection, raising armrests, raising monitors, adjusting the master-slave operation ratio, and lens mode selection.

[0052] Specifically, the voice command is preprocessed to remove non-human voices and enhance the human voice. The enhanced human voice is input into a pre-trained semantic recognition model for laparoscopic surgery for word segmentation, feature extraction and similarity comparison, thereby achieving semantic recognition and determining the user's target operation command for the target device based on the output of the model. The target operation command is sent to the target device so that the target device executes the target operation command.

[0053] The technical solution of the embodiment of the present invention obtains the sound information in the space where the console of the laparoscopic surgical robot is located; obtains the wake-up word recognition result by performing wake-up word recognition on the sound information, and obtains the voice command input by the user based on the wake-up word recognition result; accurately and conveniently determines the target operation command issued by the user through voice for the target device by performing intention recognition on the voice command, and sends the target operation command to the target device so that the target device executes the target operation command. It can support non-manual operation of doctors, and can take into account the needs of doctors for multi-line simultaneous operation, thereby improving the efficiency of surgical execution.

[0054] On the basis of the above technical solution, the method further includes: after the target operation instruction is executed, performing speech synthesis according to the target execution result of the target device to obtain feedback voice information, and playing the feedback voice information.

[0055] The feedback voice information may refer to the completion status of the target operation instruction. For example, the feedback voice information may be that the target operation instruction has been completed.

[0056] Specifically, after the target operation instruction is executed, the target device sends the target execution result to the console. The console performs speech synthesis according to the target execution result of the target device to obtain feedback voice information. The feedback voice information is played through the speaker of the console and displayed through the display screen of the console, thereby informing the user that the target operation instruction has been completed. After that, the console can be restored to the standby state and continue to obtain the sound information in the space where the console of the laparoscopic surgical robot is located.

[0057] Embodiment 2

[0058] Figure 4This is a flowchart of a voice interaction method for a laparoscopic surgical robot provided in Embodiment 2 of the present invention. Based on the above embodiments, this embodiment describes in detail the process of determining the target operation instruction of the user for the target device. The explanations of the terms that are the same or corresponding to the above embodiments are not repeated here. Figure 4 As shown, the method includes:

[0059] S410: Acquire sound information in the space where the console of the laparoscopic surgical robot is located.

[0060] S420: Obtain a wake-up word recognition result by performing wake-up word recognition on the sound information, and obtain a voice command input by the user based on the wake-up word recognition result.

[0061] S430, based on the bidirectional gated recurrent unit network combined with the attention mechanism model, the voice command is recognized for its intent to obtain the user's target operation command for the target device.

[0062] Among them, the bidirectional gated recurrent unit network combined with the attention mechanism model can be referred to as the BiGRU-Attention model. The BiGRU-Attention model can be divided into three parts: text vectorization input layer, hidden layer and output layer.

[0063] Specifically, at the input layer, the user's voice command is segmented, and the segmentation results are input into the Glove layer for semantic feature representation, thereby converting the words into word vectors. At the BiGRU layer, feature extraction is performed on the converted word vectors, and Self-attention is used to calculate the different weights in the sample sequence features in the BiGRU model. The sentence vector with word-level feature weights is input into the Softmax layer to obtain the final intent label, realize the intent recognition of the user's command, and obtain the user's target operation instruction for the target device.

[0064] On the basis of the above technical solution, "recognizing the intention of voice commands based on a bidirectional gated recurrent unit network combined with an attention mechanism model to obtain the user's target operation instructions for the target device" may include: inputting the voice command into the bidirectional gated recurrent unit network combined with the attention mechanism model, determining each word vector corresponding to the voice command, and performing feature extraction on each word vector to obtain the word feature corresponding to each word vector; determining the word feature weight corresponding to each word feature, and performing intent recognition based on each word feature weight and each word vector to obtain the user's target operation instructions for the target device.

[0065] Specifically, Figure 5 A structural example diagram of the BiGRU-Attention model is given. Figure 5, the input layer is the text vectorization input layer, which is mainly used for preprocessing voice commands, so as to process voice commands into a sequence vector form that can be directly received and processed by the BiGRU layer. The model reads the voice command and performs data cleaning. The voice command is vectorized into a form of specified length to obtain a word vector. In the hidden layer, the word feature vector output by the BiGRU layer is calculated. The word vector is the input vector of the BiGRU layer. The purpose of the BiGRU layer is mainly to extract the deep text features of the input word vector. BiGRU can be regarded as consisting of two parts: forward GRU and reverse GRU. After feature extraction through the BiGRU layer, the relationship between the context can be more fully learned to obtain word features. Calculate the probability weight that should be assigned to each word feature. This step is mainly to assign corresponding probability weights to different word features, further extract speech features, and highlight the key information of speech. In speech, different words play different roles in speech emotion classification. Place adverbials and time adverbials are of little importance to speech emotion classification. However, adjectives with emotional colors are crucial to speech emotion classification. In order to highlight the importance of different words to the entire speech emotion classification, the attention mechanism layer is introduced into the BiGRU-Attention model. The input of the attention mechanism layer is the output word features in the previous layer that have been activated by the BiGRU neural network layer. The weight coefficient of the attention mechanism layer can be determined by word features, bias coefficients and preset attention matrix. The attention mechanism matrix is ​​obtained by the accumulation of the product of different probability weights assigned by the attention mechanism and each hidden layer state, and the softmax function is used for normalization. In the output layer, the input of the output layer is the output of the previous attention mechanism layer. The softmax function is used to calculate the input of the output layer accordingly to perform text classification and obtain the user's target operation instructions for the target device.

[0066] S440: Send the target operation instruction to the target device, so that the target device executes the target operation instruction.

[0067] The technical solution of the embodiment of the present invention obtains sound information in the space where the console of the laparoscopic surgical robot is located; obtains a wake-up word recognition result by performing wake-up word recognition on the sound information, and obtains the voice command input by the user based on the wake-up word recognition result; performs intent recognition on the voice command based on a bidirectional gated recurrent unit network combined with an attention mechanism model to obtain the user's target operation command for the target device, thereby improving the voice recognition accuracy and the command response speed, and by introducing the attention mechanism, the model can automatically learn the importance weight of each word for emotion prediction, further improving the accuracy of determining the user's target operation command for the target device.

[0068] The following is an embodiment of a voice interaction device for a laparoscopic surgical robot provided in an embodiment of the present invention. The device and the voice interaction method for a laparoscopic surgical robot in the above-mentioned embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the voice interaction device for a laparoscopic surgical robot, reference can be made to the embodiment of the voice interaction method for a laparoscopic surgical robot mentioned above.

[0069] Embodiment 3

[0070] Figure 6 This is a schematic diagram of the structure of a voice interaction device for a laparoscopic surgical robot provided in Embodiment 3 of the present invention. Figure 6 As shown, the device includes: a sound information acquisition module 610, a voice instruction acquisition module 620 and a target operation instruction determination module 630.

[0071] Among them, the sound information acquisition module 610 is used to obtain the sound information in the space where the console of the laparoscopic surgical robot is located; the voice instruction acquisition module 620 is used to obtain the wake-up word recognition result by performing wake-up word recognition on the sound information, and obtain the voice instruction input by the user based on the wake-up word recognition result; the target operation instruction determination module 630 is used to obtain the user's target operation instruction for the target device by performing intention recognition on the voice instruction, and send the target operation instruction to the target device so that the target device executes the target operation instruction.

[0072] The technical solution of the embodiment of the present invention obtains the sound information in the space where the console of the laparoscopic surgical robot is located; obtains the wake-up word recognition result by performing wake-up word recognition on the sound information, and obtains the voice command input by the user based on the wake-up word recognition result; accurately and conveniently determines the target operation command issued by the user through voice for the target device by performing intention recognition on the voice command, and sends the target operation command to the target device so that the target device executes the target operation command. It can support non-manual operation of doctors, and can take into account the needs of doctors for multi-line simultaneous operation, thereby improving the efficiency of surgical execution.

[0073] Based on the above technical solution, the voice command acquisition module 620 is specifically used for: extracting features of sound information based on the Mel-cephalogram coefficient algorithm to obtain sound features; and recognizing wake-up words on the sound features based on the target network model to obtain wake-up word recognition results.

[0074] Based on the above technical solution, the voice command acquisition module 620 is specifically used for: if the wake-up word recognition result is detected to be the presence of a wake-up word, then the voice command input by the user is obtained; if the wake-up word recognition result is detected to be the absence of a wake-up word, then the sound information in the space where the console of the laparoscopic surgical robot is located is continued to be obtained.

[0075] Based on the above technical solution, the target operation instruction determination module 630 may include:

[0076] The target operation instruction determination submodule is used to perform intent recognition on voice instructions based on a bidirectional gated recurrent unit network combined with an attention mechanism model to obtain the user's target operation instructions for the target device.

[0077] Based on the above technical solution, the target operation instruction determination submodule is specifically used to: input the voice instruction into the bidirectional gated recurrent unit network combined with the attention mechanism model, determine each word vector corresponding to the voice instruction, and perform feature extraction on each word vector to obtain the word feature corresponding to each word vector; determine the word feature weight corresponding to each word feature, and perform intent recognition based on each word feature weight and each word vector to obtain the user's target operation instruction for the target device.

[0078] On the basis of the above technical solution, the device also includes: after the target operation instruction is executed, voice synthesis is performed according to the target execution result of the target device to obtain feedback voice information, and the feedback voice information is played.

[0079] The voice interaction device for a laparoscopic surgical robot provided in an embodiment of the present invention can execute the voice interaction method for a laparoscopic surgical robot provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to executing the voice interaction method for a laparoscopic surgical robot.

[0080] It is worth noting that in the above-mentioned embodiment of the voice interaction of the laparoscopic surgical robot, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0081] Embodiment 4

[0082] Figure 7 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0083] like Figure 7As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0084] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0085] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the voice interaction method of the laparoscopic surgical robot.

[0086] In some embodiments, the voice interaction method of the laparoscopic surgical robot can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the voice interaction method of the laparoscopic surgical robot described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the voice interaction method of the laparoscopic surgical robot in any other appropriate manner (for example, by means of firmware).

[0087] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0088] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0089] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0090] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0091] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0092] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0093] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the voice interaction method of a laparoscopic surgical robot as provided in any embodiment of the present application.

[0094] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, and the programming language includes an object-oriented programming language, such as Java, Smalltalk, C++, and also includes a conventional procedural programming language, such as "C" language or similar programming language. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to connect through the Internet). The program product and the voice interaction method of the laparoscopic surgical robot disclosed in each embodiment of the present application belong to the same inventive concept, so it will not be repeated here.

[0095] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0096] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A voice interaction method for a laparoscopic surgical robot, characterized in that: include: Acquiring sound information in the space where the console of the laparoscopic surgical robot is located; Obtaining a wake-up word recognition result by performing wake-up word recognition on the sound information, and acquiring a voice command input by the user based on the wake-up word recognition result; The target operation instruction of the user for the target device is obtained by performing intention recognition on the voice instruction, and the target operation instruction is sent to the target device so that the target device executes the target operation instruction.

2. The method according to claim 1, characterized in that The step of obtaining a wake-up word recognition result by performing wake-up word recognition on the sound information includes: Extracting features of the sound information based on a Mel-cephalogram coefficient algorithm to obtain sound features; The wake-up word recognition is performed on the sound feature based on the target network model to obtain a wake-up word recognition result.

3. The method according to claim 1, characterized in that The acquiring the voice command input by the user based on the wake-up word recognition result includes: If it is detected that the wake-up word recognition result is that there is a wake-up word, obtaining a voice command input by the user; If it is detected that the wake-up word recognition result is that there is no wake-up word, the sound information in the space where the console of the laparoscopic surgical robot is located continues to be obtained.

4. The method according to claim 1, characterized in that: The obtaining of a target operation instruction of the user for the target device by performing intent recognition on the voice instruction includes: Based on a bidirectional gated recurrent unit network combined with an attention mechanism model, the voice command is used to identify the user's target operation instructions for the target device.

5. The method according to claim 4, characterized in that The method of performing intent recognition on the voice command based on the bidirectional gated recurrent unit network combined with the attention mechanism model to obtain the user's target operation instruction for the target device includes: Input the voice command into a bidirectional gated recurrent unit network combined with an attention mechanism model, determine each word vector corresponding to the voice command, and perform feature extraction on each word vector to obtain a word feature corresponding to each word vector; The word feature weight corresponding to each word feature is determined, and the intent is recognized based on each word feature weight and each word vector to obtain the user's target operation instruction for the target device.

6. The method according to claim 1, characterized in that The method further comprises: After the target operation instruction is executed, speech synthesis is performed according to the target execution result of the target device to obtain feedback voice information, and the feedback voice information is played.

7. A voice interaction device for a laparoscopic surgical robot, characterized in that: The device comprises: A sound information acquisition module, used to acquire sound information in the space where the console of the laparoscopic surgical robot is located; A voice command acquisition module, configured to obtain a wake-up word recognition result by performing wake-up word recognition on the sound information, and obtain a voice command input by the user based on the wake-up word recognition result; The target operation instruction determination module is used to obtain the user's target operation instruction for the target device by performing intent recognition on the voice instruction, and send the target operation instruction to the target device so that the target device executes the target operation instruction.

8. An electronic device, characterized in that: The electronic device comprises: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the voice interaction method of the laparoscopic surgical robot as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the voice interaction method of the laparoscopic surgical robot as described in any one of claims 1-6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by the processor, the voice interaction method of the laparoscopic surgical robot is implemented.

Images