Puncture robot speech control system and control method, and puncture robot system

Abstract

Disclosed in the present invention are a puncture robot speech control system and control method, and a puncture robot system. The method comprises: acquiring a speech signal; converting the acquired speech signal into text information by using an NLP technology; extracting a voiceprint feature of the acquired speech signal, performing matching to obtain a speech signal conforming to a preset target voiceprint, and performing matching to obtain, from the converted text information, text information corresponding to the speech signal conforming to the preset target voiceprint, so as to generate potential instruction text information; when the potential instruction text information pertains to a pre-stored preset instruction set, generating a voice on the basis of the potential instruction text information for feedback; and when the speech signal acquired again is a preset acknowledgment reply signal, generating, on the basis of the potential instruction text information, a control instruction for planning a motion of the puncture robot. According to the present invention, on the basis of secondary acknowledgment after the voiceprint recognition and feedback, the speech control of the puncture robot is strictly executed, and the risks of bacterial infection during an operation are reduced.

Description

Puncture robot voice control system, control method and puncture robot system Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a puncture robot voice control system, a control method, and a puncture robot system. Background Art

[0002] A medical puncture robot is a robot that can perform punctures during medical procedures. By controlling the movement of its robotic arm, it can precisely locate the area requiring puncture and perform the puncture. This robot can reduce errors during surgery and improve surgical accuracy and safety.

[0003] Puncture robots typically present two major operational inconveniences: First, contact between the doctor and the robot can compromise sterility requirements and create an infection risk. Second, to prevent accidental robot operation from harming the patient or doctor, current surgical robots generally incorporate some form of safety feature. For example, the doctor must press a pedal before the robotic arm can move. This introduces significant operational inconvenience. During surgery, the doctor must control the robotic arm using foot pedals, joysticks, and touchscreens. This method of operation not only compromises the sterile environment during surgery and creates an infection risk, but also increases the number of steps required by the doctor, which in turn increases intraoperative risk. Therefore, a more convenient and efficient interactive operation method is needed between the doctor and the puncture robot to further improve surgical safety.

[0004] The disclosure of the above background technology content is only used to assist in understanding the inventive concept and technical solution of the present invention. It does not necessarily belong to the prior art of this patent application, nor does it necessarily provide technical guidance. In the absence of clear evidence that the above content has been disclosed before the filing date of this patent application, the above background technology should not be used to evaluate the novelty and creativity of this application. Summary of the Invention

[0005] The purpose of the present invention is to provide an improved voice control technical solution for a puncture robot, which strictly implements the voice control of the puncture robot based on secondary confirmation after voiceprint recognition and feedback.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] A puncture robot voice control system, including the following modules:

[0008] A voice acquisition module configured to acquire voice signals;

[0009] A speech recognition module is configured to convert the speech signal collected by the speech collection module into text information using NLP technology;

[0010] a processing module configured to extract voiceprint features of the voice signal collected by the voice collection module, match the voice signal with a preset target voiceprint therefrom, and match text information corresponding to the voice signal with the preset target voiceprint from the text information converted by the voice recognition module to generate potential instruction text information;

[0011] The system also includes a feedback module and a motion control module;

[0012] The processing module pre-stores a preset instruction set, and when the potential instruction text information belongs to the preset instruction set, the potential instruction text information is sent to the feedback module;

[0013] The feedback module is configured to generate text or voice for feedback based on the received potential instruction text information;

[0014] If the voice signal collected again by the voice collection module is a preset confirmation reply signal, the processing module sends the potential instruction text information or the motion instruction generated therefrom to the motion control module;

[0015] The motion control module is configured to generate a control instruction for planning the motion of the puncture robot according to the potential instruction text information or the motion instruction.

[0016] Furthermore, based on any one of the above technical solutions or a combination of multiple technical solutions, the processing module generates the potential instruction text information in the following manner:

[0017] The processing module extracts keywords from the text information corresponding to the voice signal that meets the preset target voiceprint, classifies them using a neural network, and generates the potential instruction text information accordingly;

[0018] Alternatively, the processing module uses the text information corresponding to the voice signal that meets the preset target voiceprint as the potential instruction text information.

[0019] Further, based on any one of the above technical solutions or a combination of multiple technical solutions, after the feedback module feeds back information, the voice collection module collects the voice signal again, and the voice recognition module converts it into text information;

[0020] The processing module determines whether the voiceprint feature of the re-collected voice signal meets the preset target voiceprint or whether it is the same voiceprint;

[0021] If the voice signal collected again meets the preset target voiceprint or is the same voiceprint, and its corresponding text information belongs to the preset confirmation reply instruction set, then the voice signal collected again is determined to be the preset confirmation reply signal.

[0022] Furthermore, based on any one of the above technical solutions or a combination of multiple technical solutions, before the processing module extracts the voiceprint features, it also includes:

[0023] It is determined whether the text information converted by the speech recognition module belongs to a preset stop instruction set. If so, the processing module skips the step of extracting voiceprint features and directly sends a stop movement instruction to the motion control module.

[0024] Furthermore, based on any one of the above technical solutions or a combination of multiple technical solutions, after the processing module matches the text information corresponding to the voice signal that meets the preset target voiceprint and before generating the potential instruction text information, it also includes:

[0025] Determine whether the text information corresponding to the voice signal that meets the preset target voiceprint belongs to a preset stop instruction set. If so, the processing module skips the step of generating potential instruction text information and directly sends a stop movement instruction to the motion control module.

[0026] Furthermore, based on any one of the technical solutions or a combination of multiple technical solutions described above, after the processing module sends an instruction to stop the movement to the motion control module, the feedback module feeds back an emergency stop notification message.

[0027] Furthermore, based on any one of the above technical solutions or a combination of multiple technical solutions, the puncture robot voice control system further includes a voiceprint registration module, which is configured to pre-record one or more target voiceprints;

[0028] The processing module compares the extracted voiceprint features with the target voiceprint entered by the voiceprint registration module. If the similarity between the two reaches a preset ratio threshold, the voice signal is determined to be a voice signal that meets the preset target voiceprint.

[0029] Furthermore, based on any one of the above technical solutions or a combination of multiple technical solutions, the motion control module generates a control instruction for planning the motion of the puncture robot, including:

[0030] The motion control module plans a motion trajectory for the puncture robot according to the potential instruction text information or the motion instruction.

[0031] Furthermore, based on any one of the technical solutions or a combination of multiple technical solutions described above, if the voiceprint characteristics of the voice signal collected by the voice collection module do not meet the preset target voiceprint, or the potential instruction text information does not belong to the preset instruction set, subsequent processing operations on the signal or information are stopped.

[0032] Furthermore, based on any one of the technical solutions or a combination of multiple technical solutions mentioned above, the puncture robot voice control system also includes a voice wake-up module, which is configured to wake up the voice control system in response to the voice of a preset wake-up phrase when the voice control system is not started.

[0033] According to another aspect of the present invention, the present invention provides a voice-controlled puncture robot system, comprising a puncture robot and the puncture robot voice control system as described above;

[0034] The puncture robot includes a robotic arm, a puncture end arranged on the robotic arm, and a driving mechanism for driving the robotic arm to move. The driving mechanism is electrically connected to a motion control module of the puncture robot voice control system.

[0035] Furthermore, based on any one of the technical solutions or a combination of multiple technical solutions described above, the puncture robot system also includes an optical positioning device, which is configured to determine the relative position relationship between the puncture end and the patient by optically tracking the reflection of infrared light by a ball.

[0036] Furthermore, based on any one of the above technical solutions or a combination of multiple technical solutions, the puncture robot and / or the puncture robot voice control system is equipped with an emergency stop button;

[0037] If the emergency stop button is operated, the puncture robot stops moving.

[0038] According to another aspect of the present invention, a method for voice control of a puncture robot is provided, comprising the following steps:

[0039] Collect voice signals;

[0040] Use NLP technology to convert the collected voice signals into text information;

[0041] Extracting voiceprint features from the collected voice signal, matching a voice signal that meets a preset target voiceprint, and matching text information corresponding to the voice signal that meets the preset target voiceprint from the converted text information to generate potential instruction text information;

[0042] When the potential instruction text information belongs to a pre-stored preset instruction set, generating text or voice for feedback according to the potential instruction text information;

[0043] If the voice signal collected again is a preset confirmation reply signal, a control instruction for planning the movement of the puncture robot is generated according to the potential instruction text information.

[0044] Furthermore, based on any one of the above technical solutions or a combination of multiple technical solutions, before extracting the voiceprint features, the method further includes:

[0045] Determine whether the converted text information belongs to a preset stop instruction set. If so, skip the step of extracting the voiceprint feature and directly send a stop movement instruction;

[0046] Alternatively, after matching the text information corresponding to the voice signal that meets the preset target voiceprint and before generating the potential instruction text information, it also includes: determining whether the text information corresponding to the voice signal that meets the preset target voiceprint belongs to a preset stop instruction set. If so, skipping the step of generating the potential instruction text information and directly sending the instruction to stop the movement to the motion control module.

[0047] Furthermore, based on any one of the above technical solutions or a combination of multiple technical solutions, before generating the control instructions for planning the movement of the puncture robot according to the potential instruction text information, the method includes:

[0048] Convert the collected voice signal into text information;

[0049] Determine whether the voiceprint characteristics of the re-collected voice signal meet the preset target voiceprint or whether they are the same voiceprint;

[0050] If the voice signal collected again meets the preset target voiceprint or is the same voiceprint, and its corresponding text information belongs to the preset confirmation reply instruction set, then the voice signal collected again is determined to be the preset confirmation reply signal.

[0051] Furthermore, according to any one of the above technical solutions or a combination of multiple technical solutions, before collecting the voice signal, the method further includes:

[0052] Pre-record one or more target voiceprints;

[0053] The voice signal that meets the preset target voiceprint is matched in the following way: the extracted voiceprint features are compared with the pre-recorded target voiceprint. If the similarity between the two reaches a preset ratio threshold, the voice signal is determined to be a voice signal that meets the preset target voiceprint.

[0054] Furthermore, based on any one of the technical solutions or a combination of multiple technical solutions described above, the OpenAI open source Whisper speech recognition model is used to convert speech signals into text information, and the Asyncio library provided by the Python language is used to asynchronously process the task of collecting speech signals and other tasks.

[0055] Furthermore, based on any one of the above technical solutions or a combination of multiple technical solutions, before collecting the voice signal, the method further includes: recording and storing the text corresponding to the wake-up phrase; and extracting the target voiceprint therefrom and storing it in association with the text of the corresponding wake-up phrase;

[0056] After collecting the first voice signal, it is converted into text information and the voiceprint features are extracted; the target voiceprint and the text of the associated wake-up phrase are searched. If the converted text information matches the text of the wake-up phrase and the extracted voiceprint features match the target voiceprint, the voice control of the puncture robot is started.

[0057] Furthermore, according to any one of the above technical solutions or a combination of multiple technical solutions, before collecting the voice signal, the method further includes: pre-recording the puncture path of the puncture site and determining the corresponding path starting point;

[0058] If the potential instruction text information is information related to performing puncture, after collecting the confirmation reply signal again, the motion control module generates a control instruction to move the puncture end of the puncture robot to the starting point of the path and / or move along the puncture path.

[0059] According to yet another aspect of the present invention, the present invention provides a computer-readable storage medium for storing program instructions, wherein the program instructions are configured to be executed to perform the steps of the above method.

[0060] According to another aspect of the present invention, the present invention provides an electronic device comprising a processor and a memory, wherein the memory is used to store program instructions, and the processor is configured to run the program instructions, characterized in that the program instructions are run to perform the steps of the method described above.

[0061] The beneficial effects brought about by the technical solution provided by the present invention are as follows:

[0062] a. Strictly implement voice control of the puncture robot based on secondary confirmation after voiceprint recognition and feedback;

[0063] b. The voice wake-up and voice control of the puncture robot effectively reduces the risk of bacterial infection during surgery;

[0064] c. Voice control replaces traditional auxiliary methods such as foot switches and control handles. It is more convenient and efficient, simplifies surgical steps, and improves surgical safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0066] FIG1 is a block diagram of a voice control system for a puncture robot according to an exemplary embodiment of the present invention;

[0067] FIG2 is a flow chart of a voice control method for a puncture robot provided by an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0068] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0069] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0070] In one embodiment of the present invention, a voice control system for a puncture robot is provided. Referring to FIG1 , the system includes the following modules:

[0071] a voiceprint registration module configured to pre-record one or more target voiceprints;

[0072] A voice acquisition module configured to acquire voice signals; specifically, the voice acquisition module includes a voice input device, such as a microphone or headphones with a sound collector, for collecting the user's voice signals;

[0073] A speech recognition module is configured to convert the speech signal collected by the speech collection module into text information using NLP technology;

[0074] Specifically, the speech recognition module is based on deep neural network speech recognition technology and NLP (Natural Language Processing) technology, and can convert speech signals into text information. In this embodiment, the NLP technology can be OpenAI's open source Whisper speech recognition model, which is trained on a large multilingual dataset and can convert sounds in multiple languages ​​into text.

[0075] The system further includes a processing module, a feedback module, and a motion control module. The processing module is configured to extract voiceprint features of the voice signal collected by the voice collection module, match the voice signal with a preset target voiceprint, and match text information corresponding to the voice signal with the preset target voiceprint from the text information converted by the voice recognition module to generate potential instruction text information.

[0076] Specifically, the processing module compares the extracted voiceprint features with the target voiceprint entered by the voiceprint registration module. If the similarity between the two reaches a preset ratio threshold, the voice signal is determined to be a voice signal that meets the preset target voiceprint, and the corresponding text information is obtained.

[0077] In this embodiment, a doctor's target voiceprint A can be selected from the multiple target voiceprints pre-recorded in the voiceprint registration module as the target voiceprint for this voice control; the extracted voiceprint features can also be compared one by one with the multiple pre-recorded target voiceprints for matching; and the target voiceprint can also be stored using cache technology, and this cached target voiceprint can only be used as the target voiceprint corresponding to the current voice control.

[0078] The voice acquisition module may also collect voice signals and converted text information of non-target voiceprints. Since the voiceprint does not match, this part will not be subsequently recognized (it will not be recognized as command information), but can be temporarily stored as a record during the operation.

[0079] The processing module can directly use the text information corresponding to the voice signal that meets the preset target voiceprint as the potential instruction text information, or extract part of the content as keywords, classify it using a neural network, and generate corresponding potential instruction text information accordingly.

[0080] The processing module pre-stores a preset instruction set. When the potential instruction text information belongs to the preset instruction set, the potential instruction text information is sent to the feedback module. There are two definitions of potential instruction text information belonging to the instruction set: one is that the potential instruction text information is the same as an instruction in the preset instruction set; the other is to use a pre-trained AI model for semantic classification. Its advantage is that it does not have to stick to the input instruction must be exactly the same as the preset instruction. For example, a semantic classification model of the BERT (Bidirectional Encoder Representations from Transformers) model type can be trained. Assuming there are 10 preset robot-executable operations, the BERT model can directly classify the input text information into one of the 11 categories of action 1 to action 10 plus "non-executable action". For example, if one of the operation types is "robot arm move up", then input text such as "move the robot arm up", "move the robot arm up", "move the robot arm up", etc. can all be accurately classified as "robot arm move up".

[0081] The feedback module is configured to feed back the received potential instruction text information to the user;

[0082] In this embodiment, the feedback module may be a speaker that broadcasts the potential instruction text information to the user in the form of voice, or the feedback module may be a display screen that plays the potential instruction text information to the user in the form of text.

[0083] The user then needs to respond to this feedback information by emitting a voice signal again, and their voice signal is collected again. For example, the feedback information given to the user is "Please confirm whether to initialize the puncture", "Please confirm whether to start the puncture", or "Please confirm whether to withdraw the needle"; the user then responds with affirmative confirmation signals such as "Yes", "Confirm", "Start execution", "Yes", "Correct", "Yes", etc., or negative response signals such as "Error", "No", "No".

[0084] If the voice signal collected again is a negative reply signal, the processing of the current potential instruction text information is terminated, and the next voice signal is waited for to regenerate new potential instruction text information; if the voice signal collected again is a preset confirmation reply signal, the processing module sends the potential instruction text information to the motion control module; and then the motion control module generates a control instruction for planning the movement of the puncture robot according to the potential instruction text information.

[0085] In another embodiment, if the voice signal collected again is a preset confirmation reply signal, the processing module generates a motion instruction based on the potential instruction text information and sends the motion instruction to the motion control module. Specifically, the motion control module plans the motion trajectory of the puncture robot based on the motion instruction. In this embodiment, before collecting the voice signal, the puncture path of the puncture site is pre-planned and recorded, and the corresponding path starting point is determined;

[0086] If the potential instruction text information is information related to performing puncture, after the feedback "Do you want to start puncture, please confirm", after collecting the confirmation reply signal "Confirm" again, the motion control module generates a control instruction to move the puncture end of the puncture robot to the starting point of the path and / or move along the puncture path.

[0087] In one embodiment of the present invention, after the feedback module feeds back information, the voice collection module collects the voice signal again, and the voice recognition module converts it into text information;

[0088] The processing module determines whether the voiceprint feature of the re-collected voice signal meets the preset target voiceprint or whether it is the same voiceprint;

[0089] If the re-collected voice signal matches the preset target voiceprint or is the same voiceprint, and the corresponding text information belongs to the preset confirmation reply instruction set, the re-collected voice signal is determined to be the preset confirmation reply signal. This ensures that only when there is a positive confirmation reply with the target voiceprint will the motion control module generate a control instruction to drive the puncture robot to move, preventing other people from mistakenly confirming and causing the robot to malfunction.

[0090] The Asyncio library provided by the Python language is used to asynchronously process the task of collecting voice signals and other tasks (such as extracting voiceprints, converting to text, comparing voiceprints, comparing preset instruction sets, and providing feedback). Due to the use of asynchronous processing, the voice signal collection process will not be blocked while other tasks are being processed.

[0091] In one embodiment of the present invention, a solution is proposed for responding to situations requiring an emergency stop during a puncture: after each voice signal is collected, the corresponding converted text information is determined to determine whether it belongs to a preset set of stop instructions (such as stop and terminate). If so, the processing module skips the steps of extracting voiceprint features (i.e., generating potential instruction text information, feedback, and reconfirmation) and directly sends a stop instruction to the motion control module. As can be seen from the above, the emergency stop instruction is not limited to voices with the target voiceprint. Therefore, when anyone on site (including the patient) expresses a stop instruction by voice, the system can respond promptly, thereby reducing the patient's intraoperative risk.

[0092] In another embodiment, another emergency stop puncture solution is provided to prevent unrelated personnel at the surgical site from randomly shouting to stop: that is, voiceprint matching is still required, and only when the voiceprint matches and corresponds to a preset stop instruction set can the stop movement instruction be sent to the motion control module. Specifically, after the processing module matches the text information corresponding to the voice signal that meets the preset target voiceprint and before generating the potential instruction text information, it is determined whether the text information corresponding to the voice signal that meets the preset target voiceprint belongs to the preset stop instruction set. If so, the processing module skips the step of generating the potential instruction text information (i.e., it will not perform feedback or reconfirmation operations) and directly sends the stop movement instruction to the motion control module.

[0093] After the processing module sends a stop motion instruction to the motion control module, the feedback module feeds back an emergency stop notification message, so that the doctor can clearly know that the puncture robot is currently in a stopped state.

[0094] In one embodiment of the present invention, the voice control system is also provided with an emergency stop button, which can be a virtual button on the touch screen or a physical button on the hardware device; when the emergency stop button is operated, a stop movement instruction is directly sent to the motion control module to prevent the puncture robot from moving.

[0095] After eliminating the interference of the emergency stop factor, the robot can continue to be controlled by voice containing the target voiceprint. During the entire voice control process, the voice containing the target voiceprint needs to be used as the effective voice to generate control instructions for driving the robot. If the voiceprint characteristics of the voice signal collected by the voice acquisition module do not meet the preset target voiceprint, the subsequent processing of the signal or information will be stopped.

[0096] To "possibly generate control instructions to drive the robot," the following conditions must be met simultaneously:

[0097] First, the voice control system has been started; for example, it can be started using a mechanical switch or a touch screen switch. In one embodiment of the present invention, the voice control system also includes a voice wake-up module: before collecting the voice signal, the text corresponding to the wake-up sentence is pre-recorded and stored; and the target voiceprint is extracted and stored in association with the text of the corresponding wake-up sentence; different users with permission can set the same or different wake-up sentences.

[0098] When the voice control system is not activated, after the first voice signal is collected, it is converted into text information and the voiceprint features are extracted. The target voiceprint and the text of the associated wake-up phrase are searched. If the converted text information matches the text of the wake-up phrase and the extracted voiceprint features match the target voiceprint, that is, the authorized identity and the set wake-up phrase match, then the voice control of the puncture robot is activated. This operation of activating the voice control system further effectively avoids compromising the sterile environment of the surgery.

[0099] Second, the voiceprint characteristics of the collected voice signal meet the preset target voiceprint;

[0100] Third, extracting potential instruction text information generated by keywords / words from the text converted from the collected voice signal, where the potential instruction text information belongs to a preset instruction set;

[0101] Fourth, use the feedback mechanism to obtain secondary confirmation from users;

[0102] If all four of the above conditions are met, control instructions for driving the robot are generated to reduce the probability of malfunction of the puncture robot.

[0103] In one embodiment of the present invention, a voice-controlled puncture robot system is provided, comprising a puncture robot and the voice-controlled puncture robot system as described above;

[0104] The puncture robot includes a robotic arm, a puncture end arranged on the robotic arm, and a driving mechanism for driving the robotic arm to move. The driving mechanism is electrically connected to a motion control module of the puncture robot voice control system.

[0105] Specifically, the puncture robot system also includes an optical positioning device, which is configured to determine the relative position relationship between the puncture end and the patient by reflecting infrared light from an optical tracking ball (set in advance on the skin at the target puncture site of the patient), thereby enabling the puncture robot to drive the puncture end to move to the target puncture position.

[0106] In one embodiment, the puncture robot is provided with an emergency stop button. When the emergency stop button is operated, the puncture robot will not move regardless of the instruction sent by the voice control system at this time. Accordingly, a reset button can also be provided. By operating the reset button or restarting the system of the puncture robot, the last emergency stop operation can be ended.

[0107] In one embodiment of the present invention, a method for voice control of a puncture robot is provided. Referring to FIG2 , the method comprises the following steps:

[0108] Turn on voice control, for example, by using a mechanical switch or a touchscreen switch to activate the voice control system;

[0109] Collect voice signals and convert them into text information using NLP technology;

[0110] Extracting voiceprint features from the collected voice signals, matching voice signals that meet the preset target voiceprint, matching text information corresponding to the voice signals that meet the preset target voiceprint from the converted text information, and extracting keywords from the text to generate potential instruction text information;

[0111] When the potential instruction text information belongs to a pre-stored preset instruction set, the potential instruction text information is fed back in the form of text or voice;

[0112] If the voice signal collected again is a preset confirmation reply signal, a control instruction for planning the movement of the puncture robot is generated according to the potential instruction text information.

[0113] As shown in Figure 2, if a voice signal that meets the preset target voiceprint is not matched, or the potential instruction text information does not belong to the pre-stored preset instruction set, or the voice signal collected again is not the preset confirmation reply signal, or the collected voice signal belongs to the preset stop instruction set, the voice signal will not be processed and the next voice signal will be collected.

[0114] The embodiment of the puncture robot voice control method and the above-mentioned puncture robot voice control system embodiment have the same concept, and the entire content of the system embodiment is incorporated into this method embodiment by introducing the full text.

[0115] An embodiment of the present invention further provides a computer-readable storage medium for storing program instructions, wherein the program instructions are configured to be run to execute the steps of the above method.

[0116] An embodiment of the present invention further provides an electronic device, including a processor and a memory, wherein the memory is used to store program instructions, and the processor is configured to run the program instructions, and the program instructions are run to perform the steps of the above method.

[0117] Voice interaction commands enable doctors to control the puncture robot better and more conveniently. Compared with manual operation, voice commands can be transmitted to the puncture robot faster and more efficiently, and the robot can respond to the corresponding commands faster, further improving intraoperative accuracy and reducing harm to patients. With the rapid development of current artificial intelligence technology, especially with the increase in deep learning models and training data, the accuracy of voice recognition has been substantially improved. The present invention is a voice control method for a puncture robot, which replaces the current method of stepping on a foot pedal or operating a handle or touch screen, avoids destroying the sterile environment of the operation, simplifies the process of doctors controlling the robot, improves operational efficiency and accuracy, reduces patients' intraoperative risks, and provides a better interactive experience, which can further improve doctors' user satisfaction and trust.

[0118] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0119] The above is only a specific implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A voice control system for a puncture robot, characterized in that: Includes the following modules: A voice collection module, which is configured to collect voice signals; A speech recognition module, which is configured to convert the speech signal collected by the speech collection module into text information using NLP technology; a processing module configured to extract voiceprint features of the voice signal collected by the voice collection module, match the voice signal that meets the preset target voiceprint, and match the text information corresponding to the voice signal that meets the preset target voiceprint from the text information converted by the voice recognition module to generate potential instruction text information; The system also includes a feedback module and a motion control module; The processing module pre-stores a preset instruction set, and when the potential instruction text information belongs to the preset instruction set, the potential instruction text information is sent to the feedback module; The feedback module is configured to generate text or voice for feedback according to the received potential instruction text information; If the voice signal collected again by the voice collection module is a preset confirmation reply signal, the processing module sends the potential instruction text information or the motion instruction generated therefrom to the motion control module; The motion control module is configured to generate a control instruction for planning the motion of the puncture robot according to the potential instruction text information or the motion instruction.

2. The puncture robot voice control system according to claim 1, characterized in that: The processing module generates the potential instruction text information in the following manner: The processing module extracts keywords from the text information corresponding to the voice signal that meets the preset target voiceprint, classifies them using a neural network, and generates the potential instruction text information accordingly; Alternatively, the processing module uses the text information corresponding to the voice signal that meets the preset target voiceprint as the potential instruction text information.

3. The puncture robot voice control system according to claim 1, characterized in that: After the feedback module feeds back information, the voice collection module collects the voice signal again, and the voice recognition module converts it into text information; The processing module determines whether the voiceprint feature of the voice signal collected again meets the preset target voiceprint or whether it is the same voiceprint; If the voice signal collected again meets the preset target voiceprint or is the same voiceprint, and its corresponding text information belongs to the preset confirmation reply instruction set, it is determined that the voice signal collected again is the preset confirmation reply signal.

4. The puncture robot voice control system according to claim 1, characterized in that: Before the processing module extracts the voiceprint feature, it also includes: It is determined whether the text information converted by the speech recognition module belongs to a preset stop instruction set. If so, the processing module skips the step of extracting voiceprint features and directly sends a stop movement instruction to the motion control module.

5. The puncture robot voice control system according to claim 1, characterized in that: After the processing module matches the text information corresponding to the voice signal that meets the preset target voiceprint and before generating the potential instruction text information, it also includes: Determine whether the text information corresponding to the voice signal that meets the preset target voiceprint belongs to a preset stop instruction set. If so, the processing module skips the step of generating potential instruction text information and directly sends a stop movement instruction to the motion control module.

6. The puncture robot voice control system according to claim 4 or 5, characterized in that: After the processing module sends a command to stop the motion to the motion control module, the feedback module feeds back an emergency stop notification message.

7. The puncture robot voice control system according to claim 1, characterized in that: Also included is a voiceprint registration module, which is configured to pre-record one or more target voiceprints; The processing module compares the extracted voiceprint features with the target voiceprint entered by the voiceprint registration module. If the similarity between the two reaches a preset ratio threshold, the voice signal is determined to be a voice signal that meets the preset target voiceprint.

8. The puncture robot voice control system according to claim 1, characterized in that: The motion control module generates control instructions for planning the motion of the puncture robot, including: The motion control module plans a motion trajectory for the puncture robot according to the potential instruction text information or the motion instruction.

9. The puncture robot voice control system according to claim 1, characterized in that: If the voiceprint feature of the voice signal collected by the voice collection module does not conform to the preset target voiceprint, or the potential instruction text information does not belong to the preset instruction set, then subsequent processing operations on the signal or information are stopped.

10. The puncture robot voice control system according to any one of claims 1 to 5 and 7 to 9, characterized in that: It also includes a voice wake-up module, which is configured to wake up the voice control system in response to using the voice of a preset wake-up phrase when the voice control system is not started.

11. A voice-controlled puncture robot system, characterized in that: It comprises a puncture robot and a puncture robot voice control system as claimed in any one of claims 1 to 10; The puncture robot includes a mechanical arm, a puncture end arranged on the mechanical arm, and a driving mechanism for driving the mechanical arm to move. The driving mechanism is electrically connected to a motion control module of the voice control system of the puncture robot.

12. The puncture robot system according to claim 11, characterized in that: It also includes an optical positioning device, which is configured to determine the relative position relationship between the puncture end and the patient by optically tracking the reflection of infrared light by a small ball.

13. The puncture robot system according to claim 11, characterized in that: The puncture robot and / or the puncture robot voice control system is equipped with an emergency stop button; If the emergency stop button is operated, the puncture robot stops moving.

14. A voice control method for a puncture robot, characterized in that: The following steps are involved: Collecting voice signals; Use NLP technology to convert the collected voice signals into text information; Extracting voiceprint features of the collected voice signals, matching voice signals that meet the preset target voiceprints, and matching text information corresponding to the voice signals that meet the preset target voiceprints from the converted text information to generate potential instruction text information; When the potential instruction text information belongs to a pre-stored preset instruction set, generating text or voice for feedback according to the potential instruction text information; If the voice signal collected again is a preset confirmation reply signal, a control instruction for planning the movement of the puncture robot is generated according to the potential instruction text information.

15. The voice control method of the puncture robot according to claim 14, characterized in that: Before extracting the voiceprint feature, the method further includes: determining whether the converted text information belongs to a preset stop instruction set, and if so, skipping the step of extracting the voiceprint feature and directly sending a stop movement instruction; Alternatively, after matching the text information corresponding to the voice signal that meets the preset target voiceprint and before generating the potential instruction text information, it also includes: determining whether the text information corresponding to the voice signal that meets the preset target voiceprint belongs to a preset stop instruction set, and if so, skipping the step of generating the potential instruction text information and directly sending an instruction to stop the movement to the motion control module.

16. The voice control method of the puncture robot according to claim 14, characterized in that: Before generating the control instruction for planning the motion of the puncture robot according to the potential instruction text information, the following steps are included: Convert the collected voice signal into text information; Determine whether the voiceprint feature of the voice signal collected again meets the preset target voiceprint or whether it is the same voiceprint; If the voice signal collected again meets the preset target voiceprint or is the same voiceprint, and its corresponding text information belongs to the preset confirmation reply instruction set, it is determined that the voice signal collected again is the preset confirmation reply signal.

17. The voice control method of the puncture robot according to claim 14, characterized in that: Before collecting the voice signal, it also includes: Pre-record one or more target voiceprints; The voice signal that meets the preset target voiceprint is matched in the following manner: the extracted voiceprint feature is compared with the pre-recorded target voiceprint. If the similarity between the two reaches a preset ratio threshold, the voice signal is determined to be a voice signal that meets the preset target voiceprint.

18. The voice control method of the puncture robot according to claim 14, characterized in that: The open source Whisper speech recognition model of OpenAI is used to convert speech signals into text information, and the Asyncio library provided by the Python language is used to asynchronously process the tasks of collecting speech signals and other tasks.

19. The voice control method of the puncture robot according to claim 14, characterized in that: Before collecting the voice signal, it also includes: recording and storing the text corresponding to the wake-up word; and extracting the target voiceprint and storing it in association with the text of the corresponding wake-up word; After the first voice signal is collected, it is converted into text information and the voiceprint features therein are extracted; the target voiceprint and the text of the wake-up phrase associated with it are searched, and if the converted text information matches the text of the wake-up phrase and the extracted voiceprint features match the target voiceprint, the voice control of the puncture robot is started.

20. The voice control method of the puncture robot according to claim 14, characterized in that: Before collecting the voice signal, the method further includes: pre-recording the puncture path of the puncture site and determining the corresponding path starting point; If the potential instruction text information is information related to performing puncture, then after collecting the confirmation reply signal again, the motion control module generates a control instruction to move the puncture end of the puncture robot to the starting point of the path and / or move along the puncture path.

21. A computer-readable storage medium for storing program instructions, characterized in that: The program instructions are configured to be executed to perform the steps of the method according to any one of claims 14 to 20.

22. An electronic device comprising a processor and a memory, wherein: The memory is used to store program instructions, and the processor is configured to execute the program instructions, wherein the program instructions are executed to perform the steps of the method according to any one of claims 14 to 20.

Images