Voice interaction methods, devices, equipment and media for laparoscopic surgical robots

Abstract

The embodiment of the application discloses a voice interaction method, device, equipment and medium of a laparoscopic surgery robot. The method comprises the following steps: acquiring sound information in a space where a console of the laparoscopic surgery robot is located; obtaining a wake-up word recognition result by performing wake-up word recognition on the sound information, and acquiring a voice instruction input by a user based on the wake-up word recognition result; obtaining a target operation instruction of the target device by performing intent recognition on the voice instruction, and sending the target operation instruction to the target device, so that the target device executes the target operation instruction. Through the technical scheme of the embodiment of the application, the target operation instruction of the target device issued by the user through the voice can be accurately and conveniently determined, the doctor can be supported to operate non-manually, the needs of the doctor to operate simultaneously in multiple lines can be considered, and the operation efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of medical device technology, and in particular to a voice interaction method, device, equipment and medium for a laparoscopic surgical robot. Background Technology

[0002] With the development of technology, surgical robots have been gradually applied to modern medicine and have become one of the important means to improve surgical precision and reduce surgical risks.

[0003] Currently, in actual operation, doctors can use mechanical buttons and touch screens on the control panel to perform functions such as armrest lifting, display lifting, and adjustment of the precision of surgical operations during surgery, in order to meet the different human factors needs of doctors.

[0004] However, this operating method often only supports manual operation by doctors and cannot meet the needs of doctors to perform multiple operations simultaneously, thus reducing the efficiency of surgical execution. Summary of the Invention

[0005] This invention provides a voice interaction method, device, equipment, and medium for a laparoscopic surgical robot, which can accurately and conveniently determine the target operation instructions issued by the user via voice for the target device. It can support non-manual operation by doctors and also meet the needs of doctors to perform multiple operations simultaneously, thereby improving the efficiency of surgical execution.

[0006] In a first aspect, embodiments of the present invention provide a voice interaction method for a laparoscopic surgical robot, comprising:

[0007] Acquire sound information from the space where the console of the laparoscopic surgical robot is located;

[0008] The wake word recognition result is obtained by performing wake word recognition on the sound information, and the voice command input by the user is obtained based on the wake word recognition result;

[0009] The user's target operation command for the target device is obtained by performing intent recognition on the voice command, and the target operation command is sent to the target device so that the target device executes the target operation command.

[0010] Optionally, the method further includes: extracting sound features from the sound information based on the Mel-Cepstral Coefficient algorithm; and performing wake-up word recognition on the sound features based on the target network model to obtain a wake-up word recognition result.

[0011] Optionally, the method further includes: if the wake word recognition result indicates the presence of a wake word, then acquiring the user-inputted voice command; if the wake word recognition result indicates the absence of a wake word, then continuing to acquire the sound information in the space where the console of the laparoscopic surgical robot is located.

[0012] Optionally, the method further includes: performing intent recognition on the voice command based on a bidirectional gated recurrent unit network combined with an attention mechanism model to obtain the user's target operation command for the target device.

[0013] Optionally, the method further includes: inputting the voice command into a bidirectional gated recurrent unit network combined with an attention mechanism model, determining each word vector corresponding to the voice command, and performing feature extraction on each word vector to obtain word features corresponding to each word vector; determining the word feature weight corresponding to each word feature, and performing intent recognition based on each word feature weight and each word vector to obtain the user's target operation command for the target device.

[0014] Optionally, the method further includes: after the target operation instruction is executed, performing speech synthesis based on the target execution result of the target device to obtain feedback speech information, and playing the feedback speech information.

[0015] Secondly, embodiments of the present invention also provide a voice interaction device for a laparoscopic surgical robot, the device comprising:

[0016] The sound information acquisition module is used to acquire sound information in the space where the console of the laparoscopic surgical robot is located;

[0017] The voice command acquisition module is used to obtain a wake word recognition result by performing wake word recognition on the sound information, and to obtain the voice command input by the user based on the wake word recognition result;

[0018] The target operation instruction determination module is used to obtain the user's target operation instruction for the target device by performing intent recognition on the voice instruction, and send the target operation instruction to the target device so that the target device executes the target operation instruction.

[0019] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0020] One or more processors;

[0021] Memory, used to store one or more programs;

[0022] When the one or more programs are executed by the one or more processors, the one or more processors implement the voice interaction method for laparoscopic surgical robots as provided in any embodiment of the present invention.

[0023] Fourthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the voice interaction method for a laparoscopic surgical robot as provided in any embodiment of the present invention.

[0024] Fifthly, embodiments of the present invention provide a computer program product, including a computer program that, when executed by a processor, implements the voice interaction method for a laparoscopic surgical robot as provided in any embodiment of the present invention.

[0025] The technical solution of this invention acquires sound information from the space where the console of the laparoscopic surgical robot is located; performs wake-up word recognition on the sound information to obtain a wake-up word recognition result, and obtains the user's voice command based on the wake-up word recognition result; accurately and conveniently determines the user's target operation command issued by the voice for the target device by performing intent recognition on the voice command, and sends the target operation command to the target device so that the target device executes the target operation command. This supports non-manual operation by doctors and can accommodate the needs of doctors to perform multi-line simultaneous operations, improving surgical execution efficiency.

[0026] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart of a voice interaction method for a laparoscopic surgical robot provided in Embodiment 1 of the present invention;

[0029] Figure 2 This is an example diagram of a sound feature extraction process based on the Mel-Cepstral Coefficient algorithm according to Embodiment 1 of the present invention;

[0030] Figure 3 This is an example diagram of a wake word recognition process based on a target network model according to Embodiment 1 of the present invention;

[0031] Figure 4 This is a flowchart of a voice interaction method for a laparoscopic surgical robot provided in Embodiment 2 of the present invention;

[0032] Figure 5 This is a structural example diagram of a BiGRU-Attention model according to Embodiment 2 of the present invention;

[0033] Figure 6 This is a schematic diagram of the structure of a voice interaction device for a laparoscopic surgical robot provided in Embodiment 3 of the present invention;

[0034] Figure 7 This is a schematic diagram of the structure of an electronic device that implements the voice interaction method of the laparoscopic surgical robot according to an embodiment of the present invention. Detailed Implementation

[0035] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0036] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0037] Example 1

[0038] Figure 1 This is a flowchart illustrating a voice interaction method for a laparoscopic surgical robot according to Embodiment 1 of the present invention. This embodiment is applicable to situations where a user issues a target operation command for a target device via voice. This method can be executed by the voice interaction device of the laparoscopic surgical robot, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0039] S110. Acquire sound information in the space where the console of the laparoscopic surgical robot is located.

[0040] The console can refer to the control panel used by the user to operate the laparoscopic surgical robot. The console may include a microphone array, speakers, and a processor. The surrounding space can refer to the area where the microphones are located within the console. For example, the space could be an operating room. The spatial range is proportional to the sound acquisition capability of the microphone array. Sound information can refer to all sounds within the space. For example, sound information may include, but is not limited to, ambient sounds and the user's voice.

[0041] Specifically, the microphone in the control console collects sound information in real time from the space where the control console of the laparoscopic surgical robot is located.

[0042] S120. The wake word recognition result is obtained by performing wake word recognition on the sound information, and the voice command input by the user is obtained based on the wake word recognition result.

[0043] The wake word recognition result can refer to whether or not a wake word was recognized. For example, the wake word recognition result can include whether a wake word is present or not. Voice commands can refer to the operation instructions spoken by the user within a certain period after uttering the wake word.

[0044] Specifically, when the console is in standby mode, it continuously collects and processes various sounds from the surrounding environment using a microphone at low power consumption, constantly determining whether the user has used a wake-up word to wake the machine. If no wake-up word is present in the sound, the console remains in standby mode; otherwise, it is woken up and enters working mode. Once in working mode, the console uses the microphone to collect the user's voice over a period of time, then identifies the user's intended command from this sound and performs the corresponding action. After the command action is completed, the console returns to standby mode.

[0045] Based on the above technical solution, "obtaining wake word recognition results by recognizing wake words from sound information" can include: extracting sound features from sound information based on the Mel-Cepstral Coefficient algorithm; and recognizing wake words from sound features based on a target network model to obtain wake word recognition results.

[0046] The target network model is composed of several residual network (ResNet) architectures. These residual network architectures include convolutional vectors (CONV), batch normalization (BN), rectified linear function (ReLU), pooling, and fully connected layers. BN ensures that the input to each layer of the deep neural network maintains the same distribution during training, which can improve the model's training stability and accelerate the convergence process.

[0047] Specifically, Figure 2 A flowchart illustrating a sound feature extraction process based on the Mel-Cepstral Coefficients algorithm is provided. See also... Figure 2 The input audio information undergoes preprocessing operations, such as pre-emphasis. The preprocessed information is then transformed in the time-frequency domain (e.g., short-time Fourier transform) to obtain a speech feature sequence, and the power spectrum is calculated. A gamma tone filter bank is then used to filter and analyze the preprocessed information, and the long-time power spectrum is calculated, using asymmetric and temporary masking filters to suppress asymmetric noise. The multi-channel time-frequency and power data of the preprocessed information are normalized, and the power is then adjusted. Finally, the normalization result is processed using a power function nonlinearity to obtain the audio features. Figure 3 A flowchart illustrating a wake-word recognition process based on a target network model is provided. See also... Figure 3 The sound features are input into the target network model, and after passing through multiple residual networks, they are subjected to convolution, batch normalization, pooling, and full connection to achieve wake word recognition and obtain the wake word recognition result.

[0048] Based on the above technical solution, "obtaining user-inputted voice commands based on wake word recognition results" can include: if the wake word recognition result indicates the presence of a wake word, then obtaining the user-inputted voice commands; if the wake word recognition result indicates the absence of a wake word, then continuing to obtain the sound information in the space where the console of the laparoscopic surgical robot is located.

[0049] Specifically, if the wake word recognition result indicates that no wake word exists, the controller continues to remain in standby mode and continues to acquire sound information in the space where the console of the laparoscopic surgical robot is located until the wake word recognition result indicates that a wake word exists, at which point the controller acquires the voice command input by the user.

[0050] S130: Obtain the user's target operation command for the target device by performing intent recognition on the voice command, and send the target operation command to the target device so that the target device executes the target operation command.

[0051] The target device can refer to the device that the user expects to operate. For example, the target device can include, but is not limited to, a control console, an imaging trolley, and a patient surgical platform. The target operation command can refer to the surgical operation that the user expects the target device to perform. For example, the target operation command can include, but is not limited to, account login, department selection, raising the armrests, raising the monitor, adjusting the master-slave operation ratio, and lens mode selection.

[0052] Specifically, the voice commands are preprocessed to remove non-human voices and enhance human voices. The enhanced human voice is then input into a pre-trained semantic recognition model for laparoscopic surgery for word segmentation, feature extraction, and similarity comparison, thereby achieving semantic recognition and determining the user's target operation command for the target device based on the model's output. The target operation command is then sent to the target device to execute the target operation command.

[0053] The technical solution of this invention acquires sound information from the space where the console of the laparoscopic surgical robot is located; obtains a wake-up word recognition result by performing wake-up word recognition on the sound information, and acquires the user's voice command based on the wake-up word recognition result; accurately and conveniently determines the user's target operation command issued by voice for the target device by performing intent recognition on the voice command, and sends the target operation command to the target device so that the target device executes the target operation command. This supports non-manual operation by doctors and can accommodate the needs of doctors to perform multi-line simultaneous operations, improving surgical execution efficiency.

[0054] Based on the above technical solution, the method further includes: after the target operation instruction is executed, performing speech synthesis based on the target execution result of the target device to obtain feedback speech information, and playing the feedback speech information.

[0055] The feedback voice information can refer to the completion status of the target operation command. For example, the feedback voice information could indicate that the target operation command has been completed.

[0056] Specifically, after the target operation command is executed, the target device sends the execution result to the console. The console performs speech synthesis based on the target device's execution result to obtain feedback voice information. The feedback voice information is played through the console's speakers and displayed on the console's screen, thereby informing the user that the target operation command has been completed. Afterward, the console can return to standby mode and continue to acquire sound information from the space where the laparoscopic surgical robot's console is located.

[0057] Example 2

[0058] Figure 4This is a flowchart of a voice interaction method for a laparoscopic surgical robot according to Embodiment 2 of the present invention. Based on the above embodiments, this embodiment describes in detail the process of determining the user's target operation command for the target device. Explanations of terms that are the same as or corresponding to those in the above embodiments are not repeated here. Figure 4 As shown, the method includes:

[0059] S410. Acquire sound information in the space where the console of the laparoscopic surgical robot is located.

[0060] S420: Obtain wake word recognition result by performing wake word recognition on the sound information, and obtain the user's voice command based on the wake word recognition result.

[0061] S430 uses a bidirectional gated recurrent unit network combined with an attention mechanism model to perform intent recognition on voice commands to obtain the user's target operation command for the target device.

[0062] Among them, the bidirectional gated recurrent unit network combined with the attention mechanism model can refer to the BiGRU-Attention model. The BiGRU-Attention model can be divided into three parts: a text vectorized input layer, a hidden layer, and an output layer.

[0063] Specifically, at the input layer, the user's voice commands are segmented into words, and the segmentation results are input into the GloVe layer for semantic feature representation, thereby converting words into word vectors. In the BiGRU layer, features are extracted from the converted word vectors, and Self-attention is used to calculate different weights in the sample sequence features within the BiGRU model. The sentence vectors, containing word-level feature weights, are then input into the Softmax layer to obtain the final intent label, enabling the recognition of the user's intent and determining the user's target operation command for the target device.

[0064] Based on the above technical solution, "using a bidirectional gated recurrent unit network combined with an attention mechanism model to perform intent recognition on voice commands to obtain the user's target operation command for the target device" can include: inputting the voice command into the bidirectional gated recurrent unit network combined with an attention mechanism model, determining each word vector corresponding to the voice command, and extracting features from each word vector to obtain the word features corresponding to each word vector; determining the word feature weights corresponding to each word feature, and performing intent recognition based on each word feature weight and each word vector to obtain the user's target operation command for the target device.

[0065] Specifically, Figure 5 A structural example diagram of a BiGRU-Attention model is provided. See [link / reference] Figure 5The input layer, or text vectorization input layer, primarily preprocesses speech commands, transforming them into sequence vectors that the BiGRU layer can directly receive and process. The model reads the speech commands and performs data cleaning. The speech commands are vectorized into a specified length, yielding word vectors. In the hidden layer, the word feature vectors output by the BiGRU layer are calculated. These word vectors are the input vectors of the BiGRU layer. The purpose of the BiGRU layer is primarily to extract deep textual features from the input word vectors. BiGRU can be viewed as consisting of a forward GRU and a backward GRU. After feature extraction through the BiGRU layer, the relationships between contexts can be learned more fully, thus obtaining word features. The probability weights to be assigned to each word feature are calculated. This step mainly assigns corresponding probability weights to different word features, further extracting speech features and highlighting key speech information. In speech, different words play different roles in speech sentiment classification. Adverbs of place and time are of minimal importance for speech sentiment classification, while adjectives with emotional connotations are crucial. To highlight the importance of different words in the overall speech sentiment classification, an attention mechanism layer is introduced in the BiGRU-Attention model. The input to the attention mechanism layer is the output word features from the previous layer, processed by the BiGRU neural network. The weights of the attention mechanism layer are determined by the word features, bias coefficients, and a pre-defined attention matrix. The attention mechanism matrix is ​​obtained by summing the products of the different probability weights assigned by the attention mechanism and the hidden state, and then normalizing using the softmax function. In the output layer, the input is the output of the previous attention mechanism layer. By using the softmax function to perform corresponding calculations on the input of the output layer, text classification is performed to obtain the user's target operation command for the target device.

[0066] S440. Send the target operation command to the target device so that the target device executes the target operation command.

[0067] The technical solution of this invention acquires sound information in the space where the console of the laparoscopic surgical robot is located; obtains wake word recognition results by performing wake word recognition on the sound information, and obtains the user's voice command based on the wake word recognition results; and obtains the user's target operation command for the target device by performing intent recognition on the voice command based on a bidirectional gated recurrent unit network combined with an attention mechanism model, thereby improving the accuracy of voice recognition and the speed of command response. Furthermore, by introducing an attention mechanism, the model can automatically learn the importance weight of each word for emotion prediction, further improving the accuracy of determining the user's target operation command for the target device.

[0068] The following are embodiments of the voice interaction device for a laparoscopic surgical robot provided in this invention. This device and the voice interaction method for the laparoscopic surgical robot in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the voice interaction device for the laparoscopic surgical robot, please refer to the embodiments of the voice interaction method for the laparoscopic surgical robot described above.

[0069] Example 3

[0070] Figure 6 This is a schematic diagram of the voice interaction device for a laparoscopic surgical robot provided in Embodiment 3 of the present invention. Figure 6 As shown, the device includes: a sound information acquisition module 610, a voice command acquisition module 620, and a target operation command determination module 630.

[0071] The system includes a sound information acquisition module 610, which acquires sound information in the space where the console of the laparoscopic surgical robot is located; a voice command acquisition module 620, which obtains a wake-up word recognition result by performing wake-up word recognition on the sound information, and acquires the voice command input by the user based on the wake-up word recognition result; and a target operation command determination module 630, which obtains the user's target operation command for the target device by performing intent recognition on the voice command, and sends the target operation command to the target device so that the target device executes the target operation command.

[0072] The technical solution of this invention acquires sound information from the space where the console of the laparoscopic surgical robot is located; obtains a wake-up word recognition result by performing wake-up word recognition on the sound information, and acquires the user's voice command based on the wake-up word recognition result; accurately and conveniently determines the user's target operation command issued by voice for the target device by performing intent recognition on the voice command, and sends the target operation command to the target device so that the target device executes the target operation command. This supports non-manual operation by doctors and can accommodate the needs of doctors to perform multi-line simultaneous operations, improving surgical execution efficiency.

[0073] Based on the above technical solution, the voice command acquisition module 620 is specifically used to: extract sound features from sound information based on the Mel-Cepstral Coefficient algorithm; and perform wake word recognition on the sound features based on the target network model to obtain the wake word recognition result.

[0074] Based on the above technical solution, the voice command acquisition module 620 is specifically used to: if the wake word recognition result is that a wake word exists, then acquire the voice command input by the user; if the wake word recognition result is that no wake word exists, then continue to acquire the sound information in the space where the console of the laparoscopic surgical robot is located.

[0075] Based on the above technical solution, the target operation instruction determination module 630 may include:

[0076] The target operation instruction determination submodule is used to perform intent recognition on voice commands based on a bidirectional gated recurrent unit network combined with an attention mechanism model to obtain the user's target operation instructions for the target device.

[0077] Based on the above technical solution, the target operation instruction determination submodule is specifically used to: input the voice instruction into the bidirectional gated recurrent unit network combined with the attention mechanism model, determine each word vector corresponding to the voice instruction, and extract features from each word vector to obtain the word features corresponding to each word vector; determine the word feature weights corresponding to each word feature, and perform intent recognition based on each word feature weight and each word vector to obtain the user's target operation instruction for the target device.

[0078] Based on the above technical solution, the device further includes: after the target operation command is executed, performing speech synthesis based on the target execution result of the target device to obtain feedback speech information, and playing the feedback speech information.

[0079] The voice interaction device for the laparoscopic surgical robot provided in this embodiment of the invention can execute the voice interaction method for the laparoscopic surgical robot provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the voice interaction method for the laparoscopic surgical robot.

[0080] It is worth noting that in the above-described embodiment of the voice interaction of the laparoscopic surgical robot, the various units and modules are divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0081] Example 4

[0082] Figure 7 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0083] like Figure 7As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0084] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0085] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the voice interaction method for a laparoscopic surgical robot.

[0086] In some embodiments, the voice interaction method for the laparoscopic surgical robot can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the voice interaction method for the laparoscopic surgical robot described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the voice interaction method for the laparoscopic surgical robot by any other suitable means (e.g., by means of firmware).

[0087] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0088] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0089] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0090] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0091] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0092] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0093] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the voice interaction method for a laparoscopic surgical robot as provided in any embodiment of this application.

[0094] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider). This program product belongs to the same inventive concept as the voice interaction method for the laparoscopic surgical robot disclosed in the embodiments of this application, and therefore will not be described further here.

[0095] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and no limitation is imposed herein.

[0096] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A voice interaction method of a laparoscopic surgery robot, characterized by, The method comprises: obtaining sound information in a space where a console of a laparoscopic surgery robot is located; obtaining a voice instruction input by a user based on a wake-up word recognition result obtained by performing wake-up word recognition on the sound information; obtaining a target operation instruction of the user for a target device by performing intent recognition on the voice instruction, and sending the target operation instruction to the target device to enable the target device to execute the target operation instruction; the target device comprises at least one of a console, an image trolley, and a patient surgery platform; and the target operation instruction comprises at least one of account login, department selection, raising a handrail, raising a display, adjusting a master-slave operation ratio, and lens mode selection; wherein the obtaining of the target operation instruction of the user for the target device by performing intent recognition on the voice instruction comprises: obtaining the target operation instruction of the user for the target device by performing intent recognition on the voice instruction based on a bidirectional gated recurrent unit network combined with an attention mechanism model; wherein the obtaining of the target operation instruction of the user for the target device by performing intent recognition on the voice instruction based on the bidirectional gated recurrent unit network combined with the attention mechanism model comprises: inputting the voice instruction into the bidirectional gated recurrent unit network combined with the attention mechanism model, determining each word vector corresponding to the voice instruction, and performing feature extraction on each word vector to obtain a word feature corresponding to each word vector; determining a word feature weight corresponding to each word feature, and performing intent recognition on each word feature weight and each word vector to obtain the target operation instruction of the user for the target device.

2. The method of claim 1, wherein, The obtaining of the wake-up word recognition result by performing wake-up word recognition on the sound information comprises: performing feature extraction on the sound information based on a mel-frequency cepstral coefficient algorithm to obtain sound features; performing wake-up word recognition on the sound features based on a target network model to obtain the wake-up word recognition result.

3. The method of claim 1, wherein, The obtaining of the voice instruction input by the user based on the wake-up word recognition result comprises: if it is detected that the wake-up word recognition result is that a wake-up word exists, obtaining the voice instruction input by the user; if it is detected that the wake-up word recognition result is that a wake-up word does not exist, continuing to obtain the sound information in the space where the console of the laparoscopic surgery robot is located.

4. The method of claim 1, wherein, The method further comprises: after the target operation instruction is executed, performing voice synthesis based on a target execution result of the target device to obtain feedback voice information, and playing the feedback voice information.

5. A voice interaction device for a laparoscopic surgical robot, the device comprising: The device comprises: a sound information acquisition module configured to obtain sound information in a space where a console of a laparoscopic surgery robot is located; a voice instruction acquisition module configured to obtain a voice instruction input by a user based on a wake-up word recognition result obtained by performing wake-up word recognition on the sound information; The target operation instruction determination module is configured to obtain a target operation instruction of the user for a target device through intent recognition on the voice instruction, and send the target operation instruction to the target device, so that the target device executes the target operation instruction; the target device includes at least one of a console, an image trolley, and a patient surgery platform; the target operation instruction includes at least one of account login, department selection, raising a handrail, raising a display, adjusting a master-slave operation ratio, and lens mode selection; The target operation instruction determination module includes a target operation instruction determination submodule configured to obtain a target operation instruction of the user for a target device based on a bidirectional gated recurrent unit network combined with an attention mechanism model through intent recognition on the voice instruction. The target operation instruction determination submodule is specifically configured to input the voice instruction into the bidirectional gated recurrent unit network combined with the attention mechanism model, determine each word vector corresponding to the voice instruction, and perform feature extraction on each word vector to obtain a word feature corresponding to each word vector; determine a word feature weight corresponding to each word feature, and perform intent recognition based on each word feature weight and each word vector to obtain a target operation instruction of the user for a target device.

6. An electronic device, comprising: The electronic device includes: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the voice interaction method of the laparoscopic surgery robot as claimed in any one of claims 1-4.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that The program is executed by the processor to implement the voice interaction method of the laparoscopic surgery robot as claimed in any one of claims 1-4.

8. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the voice interaction method of the laparoscopic surgery robot as claimed in any one of claims 1-4.

Images