A throat swab sampling robot control system and method

Abstract

The application discloses a throat swab sampling robot control system and method, the system comprises a main control module, a hardware control module, a visual control module and a comprehensive control module which are sequentially connected in communication; the main control module is used for receiving the visual tracking message sent by the visual control module, sending the prompt and running state message to the comprehensive control module, and interacting with the hardware control module to execute the instruction information; the hardware control module is used for interacting with the visual control module to sample the information of the mechanism, and delivering the voice playing information to the comprehensive control module; the visual control module is used for delivering the voice playing or text image prompt information to the comprehensive control module, and receiving the visual tracking activation message sent by the comprehensive control module, the flexibility and the expandability of the throat swab sampling robot are increased, and various uncertain problems possibly occurring in the sampling process can be intelligently coped with.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, in particular to a throat swab sampling robot control system and method. BACKGROUND

[0002] Throat swab sampling is a routine test operation in modern clinical medicine, and a throat swab sampling robot can replace humans to fully autonomously complete test consumable access, personnel information collection, verification, target sampling site identification and positioning, throat swab scraping sampling, sampling effectiveness discrimination, sample collection, disinfection and sterilization, and fault handling.

[0003] In the prior art, a throat swab sampling robot generally adopts a hardware control mechanical structure mode, but the hardware control programming of the existing throat swab sampling robot is relatively complex, has insufficient flexibility and expandability, and the functions are relatively insufficient, and various uncertain problems that may occur in the sampling process cannot be well coped with. SUMMARY

[0004] Therefore, the present application provides a throat swab sampling robot control system and method to solve the problem of insufficient flexibility and expandability of the throat swab sampling robot with a hardware control mechanical structure in the prior art, which cannot well cope with various uncertain problems that may occur in the sampling process.

[0005] The present application provides a throat swab sampling robot control system, which comprises a main control module, a hardware control module, a visual control module and a comprehensive control module connected in sequence;

[0006] The main control module is used for receiving a visual tracking message of the visual control module, and publishing a visual tracking state of the throat swab sampling robot in real time according to the visual tracking message, while sending a text prompt message or a running state message to the comprehensive control module, and controlling the comprehensive control module to display the text prompt message or the running state message on a user interface of the throat swab sampling robot in real time;

[0007] The hardware control module is used for information interaction with the visual control module, publishing a motion instruction of a sampling mechanism of the throat swab sampling robot in real time according to information provided by the visual control module, and delivering voice playing information to the comprehensive control module to control the comprehensive control module to control the throat swab sampling robot to play voice;

[0008] The visual control module is used for monitoring a throat of a patient and tracking a throat position, and delivering voice playing information or text image prompt information to the comprehensive control module, and when receiving a visual tracking activation message sent by the comprehensive control module, controlling the throat swab sampling robot to start a visual servo tracking function;

[0009] The comprehensive control module is configured to receive the sampling information transmitted by the main control module, the hardware control module and the visual control module, and display the sampling information on the user interface of the throat swab sampling robot in real time, and send the visual tracking activation message to the visual control module to control the visual tracking function of the robot system.

[0010] Optionally, the main control module is configured to receive the visual tracking message sent by the visual control module, and publish the current visual tracking state of the throat swab sampling robot in real time after receiving the visual tracking message sent by the visual control module; the main control module is configured to send a text prompt message or a running state message to the comprehensive control module, and the comprehensive control module displays the text prompt message or the running state message on the throat swab sampling robot in real time after receiving the text prompt message or the running state message; and the main control module is configured to interact with the hardware control module to execute the instruction information, and control the hardware control module by using the instruction information.

[0011] The hardware control module is configured to interact with the visual control module to transmit the information of the sampling mechanism, and publish the motion instruction of the sampling mechanism in real time according to the direction of the target point deviation; the hardware control module is configured to transmit the voice playing information to the comprehensive control module, and the comprehensive control module controls the throat swab sampling robot to play the voice in real time after receiving the voice playing information.

[0012] The visual control module is configured to transmit the voice playing information or the text image prompt information to the comprehensive control module, and the comprehensive control module controls the throat swab sampling robot to play the voice or display the text image in real time after receiving the voice playing information or the text image prompt information; the visual control module is configured to receive the visual tracking activation message sent by the comprehensive control module, and start the visual servo throat tracking function of the throat swab sampling robot after receiving the visual tracking activation message sent by the comprehensive control module.

[0013] The comprehensive control module is configured to receive the text prompt message or the running state message sent by the main control module, and display the text prompt message or the running state message on the throat swab sampling robot; the comprehensive control module is configured to receive the voice playing information transmitted by the hardware control module, and display the voice playing information on the throat swab sampling robot; the comprehensive control module is configured to receive the voice playing information or the text image prompt information transmitted by the visual control module, and display the voice playing information or the text image prompt information on the throat swab sampling robot; and the comprehensive control module is configured to send the visual tracking activation message to the visual control module.

[0014] Optionally, the main control module comprises a monitoring window node and a main state machine node, the monitoring window node is in communication connection with the main state machine node, and the monitoring window node and the main state machine node interact with each other through the running state message and the manual intervention service;

[0015] The hardware control module comprises a hardware driving node and a servo sampling node, and the hardware driving node is in communication connection with the servo sampling node;

[0016] The main state machine node is in communication connection with the servo sampling node and is used for sending a sampling start message to the servo sampling node, and the monitoring window node is in communication connection with the hardware driving node and is used for interacting with the hardware driving node through the execution instruction information.

[0017] Optionally, the main control module further comprises a password verification node, and the password verification node is used for verifying the authority when a medical staff logs in.

[0018] Optionally, the hardware driving node comprises a CAN communication driving node, a gantry driving node and a mobile mask driving node, a 3-PRS driving node, a gripper driving node and a motion control node;

[0019] The CAN communication driving node is used for completing the bidirectional CAN communication between the upper computer and the lower computer.

[0020] The gantry driving node and the mobile mask driving node are used for controlling the motor movement of the gantry and the mobile mask of the sampling robot.

[0021] The 3-PRS driving node is used for controlling the movement of the 3-PRS sampling hand of the throat swab sampling robot.

[0022] The gripper driving node is used for controlling the opening and closing of the swab gripper of the sampling robot.

[0023] The motion control node is used for the multi-mechanism cooperative motion control and the mechanism modular operation.

[0024] The servo sampling node is used for jointly controlling the gantry, the mobile mask and the sampling hand of the 3-PRS to realize the force servo scraping sampling.

[0025] Optionally, the visual control module comprises a face tracking node, and the face tracking node is in communication connection with the hardware driving node and the main state machine node respectively.

[0026] The face tracking node is used for sending a visual tracking end message to the main state machine node and interacting with the hardware driving node through the position information and the rapid motion information of the sampling mechanism.

[0027] Optionally, the comprehensive control module comprises a voice interaction node, a sampled person window node and an identity verification node, the identity verification node is respectively communicatively connected to the voice interaction node and the sampled person window node, the identity verification node is configured to send a voice playing message to the voice interaction node, and send a sampled person information message to the sampled person window node;

[0028] The voice interaction node is respectively communicatively connected to the face tracking node and the servo sampling node, and the voice interaction node is configured to receive the voice playing message from the face tracking node and the servo sampling node;

[0029] The sampled person window node is respectively communicatively connected to the face tracking node and the main state machine node, and the sampled person window node is configured to receive the text prompt message and the running state message from the face tracking node and the main state machine node;

[0030] The identity verification node is communicatively connected to the face tracking node, and the identity verification node is configured to send a visual tracking activation message to the face tracking node.

[0031] Optionally, the main control module, the hardware control module, the visual control module and the comprehensive control module communicate and interact through a first communication mode and a second communication mode of a ROS1 system;

[0032] The first communication mode is one-way information publishing and message receiving, and the information publisher and the information receiver are independent of and do not interfere with each other, and the second communication mode is that a service requester sends a request to a service provider, and the service provider responds and feeds back.

[0033] Optionally, the running state message is a current running state of the throat swab sampling robot published by the main control module in real time, and the manual intervention service is that a medical staff inputs a configuration parameter and an artificial intervention instruction in an interactive window.

[0034] Optionally, the position information of the sampling mechanism includes end position information of a gantry of the sampling mechanism and a moving mask of the sampling mechanism, and the interaction of the rapid motion information of the sampling mechanism includes receiving a relative motion instruction of the sampling mechanism, and driving the sampling mechanism to complete rapid following during visual servo throat tracking.

[0035] The application also provides a throat swab sampling robot control method based on the above-mentioned throat swab sampling robot control system, comprising:

[0036] The main control module receives the visual tracking message of the visual control module, and publishes the visual tracking state of the throat swab sampling robot in real time according to the visual tracking message, while sending a text prompt message or a running state message to the integrated control module, and controlling the integrated control module to display the text prompt message or the running state message on the user interface of the throat swab sampling robot in real time.

[0037] The hardware control module interacts with the visual control module, publishes the motion instruction of the sampling mechanism of the throat swab sampling robot in real time according to the information provided by the visual control module, and transmits voice playing information to the integrated control module to control the integrated control module to control the throat swab sampling robot to play voice.

[0038] The visual control module monitors the throat of the patient and tracks the throat position, and transmits voice playing information or text image prompt information to the integrated control module, and controls the throat swab sampling robot to start the visual servo tracking function after receiving the visual tracking activation message sent by the integrated control module.

[0039] The integrated control module receives the sampling information transmitted by the main control module, the hardware control module and the visual control module, and displays it on the user interface of the throat swab sampling robot in real time, and sends the visual tracking activation message to the visual control module to control the visual tracking function of the robot system.

[0040] The beneficial effects of the present application are: different from the prior art, the present application is provided with the main control module, the hardware control module, the visual control module and the comprehensive control module which are sequentially communicated and connected, so that the software control system is divided into a plurality of independent "function modules" according to the function modules, each function module contains a plurality of program nodes which can be independently operated, the nodes are communicated with each other through messages and services, the overall operation is realized, and the main control module is used for receiving the visual tracking message sent by the visual control module, sending the prompt and running state message to the comprehensive control module, and performing the instruction information interaction with the hardware control module, so as to realize the function of the main control module; secondly, the hardware control module of the present application is used for the information interaction of the sampling mechanism with the visual control module, and the voice playing information is transmitted to the comprehensive control module; thirdly, the visual control module of the present application is used for transmitting the voice playing or text image prompt information to the comprehensive control module, and receiving the visual tracking activation message sent by the comprehensive control module, using the modular development of the ROS1 system module, based on communication, flexible and expandable characteristics, the robot software control system uses the ROS1 operating system to realize the overall operation control of the robot, has a certain expandability, is convenient for adding new functions and equipment in the later period, also increases the flexibility and expandability of the throat swab sampling robot, and can intelligently cope with various uncertain problems that may occur in the sampling process.

[0041] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, but not limiting the present application. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0043] Figure 1 is a module principle schematic diagram of the throat swab sampling robot control system of the present application;

[0044] Figure 2 is a running schematic diagram of the preferred embodiment of the throat swab sampling robot control system of the present application;

[0045] Figure 3 is a flow chart of the preferred embodiment of the throat swab sampling robot control method of the present application. DETAILED DESCRIPTION

[0046] For those skilled in the art to better understand the technical solutions of the present application, the control system and method of the pharyngeal swab sampling robot provided by the present application will be further described in detail below in combination with the drawings and specific embodiments. It can be understood that the described embodiments are only part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0047] The terms "first", "second", and the like in the present application are used to distinguish different objects, not to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or equipment including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or equipment.

[0048] The present application provides a control system and method of a pharyngeal swab sampling robot to solve the problem of insufficient flexibility and scalability of the pharyngeal swab sampling robot controlled by hardware mechanical structure in the prior art, which cannot well cope with various uncertain problems that may occur in the sampling process.

[0049] Please refer to Figures 1 to 2 , Figure 1 is a module principle diagram of the control system of the pharyngeal swab sampling robot of the present application; Figure 2 is a node diagram of a preferred embodiment of the control system of the pharyngeal swab sampling robot of the present application.

[0050] The present application provides a control system of a pharyngeal swab sampling robot, wherein, as Figure 1 and Figure 2As shown, the throat swab sampling robot control system comprises a main control module 51, a hardware control module 52, a visual control module 53 and a comprehensive control module 54 connected in sequence, wherein the main control module 51 is used for receiving the visual tracking message sent by the visual control module 53, and the main control module 51 publishes the current visual tracking state of the throat swab sampling robot in real time after receiving the visual tracking message sent by the visual control module 53; the main control module 51 is used for sending a text prompt message or a running state message to the comprehensive control module 54, and the comprehensive control module 54 displays in real time on the throat swab sampling robot after receiving the text prompt message or the running state message; and the main control module 51 is used for executing instruction information interaction with the hardware control module 52, and using the execution instruction information to control the hardware control module 52; the hardware control module 52 is used for information interaction of the sampling mechanism with the visual control module 53, and publishes the sampling mechanism motion instruction in real time according to the direction of the target point deviation; the hardware control module 52 is used for transmitting voice playing information to the comprehensive control module 54, and the comprehensive control module 54 controls the throat swab sampling robot to play voice in real time after receiving the voice playing information; the visual control module 53 is used for transmitting voice playing information or text image prompt information to the comprehensive control module 54, and the comprehensive control module 54 controls the throat swab sampling robot to play voice or text image prompt in real time after receiving the voice playing information or the text image prompt information; the visual control module 53 is used for receiving the visual tracking activation message sent by the comprehensive control module 54, and the visual control module 53 starts the visual servo throat tracking function of the throat swab sampling robot after receiving the visual tracking activation message sent by the comprehensive control module 54; the comprehensive control module 54 is used for receiving the text prompt message or the running state message sent by the main control module 51, and displaying on the throat swab sampling robot; the comprehensive control module 54 is used for receiving the voice playing information transmitted by the hardware control module 52, and displaying on the throat swab sampling robot; the comprehensive control module 54 is used for receiving the voice playing information or the text image prompt information transmitted by the visual control module 53, and displaying on the throat swab sampling robot; and the comprehensive control module 54 is used for sending the visual tracking activation message to the visual control module 53.

[0051] In the above embodiment, the application divides the software control system into a plurality of independent function modules according to function modules by setting the main control module, the hardware control module, the visual control module and the comprehensive control module connected in sequence, each function module contains a plurality of program nodes which can be independently run, each node communicates with each other through messages and services to realize the overall operation, and the main control module is used for receiving the visual tracking message sent by the visual control module, sending the prompt and running state message to the comprehensive control module, and interacting with the hardware control module to realize the function of the main control module; secondly, the hardware control module of the application is used for information interaction of the sampling mechanism with the visual control module, and the voice playing information is transmitted to the comprehensive control module; thirdly, the visual control module of the application is used for transmitting voice playing or text image prompt information to the comprehensive control module, and receiving the visual tracking activation message sent by the comprehensive control module, using the modular development of the ROS1 system module, based on communication, flexible and expandable characteristics, the robot software control system uses the ROS1 operating system to realize the overall operation control of the robot, has certain expandability, is convenient for adding new functions and equipment in later period, also increases the flexibility and expandability of the throat swab sampling robot, and can intelligently cope with various uncertain problems that may occur in the sampling process.

[0052] Further, the main control module includes a monitoring window node and a main state machine node, the monitoring window node is in communication connection with the main state machine node, the monitoring window node and the main state machine node interact with each other in running state message and manual intervention service, the hardware control module includes a hardware drive node and a servo sampling node, the hardware drive node is in communication connection with the servo sampling node, the hardware drive node and the servo sampling node interact with each other in sampling mechanism fast motion message, sampling mechanism position message and force feedback condition message, the main state machine node is in communication connection with the servo sampling node, and is used for sending a sampling start message to the servo sampling node, and the monitoring window node is in communication connection with the hardware drive node, and is used for interacting with the hardware drive node in execution instruction information.

[0053] The main state machine node is used for completing the whole process control of the sampling robot and fault monitoring and processing, the monitoring window node is used for running control of the medical staff interaction window, manual intervention of the main state machine running, parameter configuration, fault, log feedback and other functions, and also provides a lock screen function.

[0054] Further, the main control module further includes a password verification node, the password verification node is used for verifying the authority when the medical staff logs in.

[0055] The medical staff password verification service includes inputting the medical staff ID, name, password, feeding back whether the user has machine operation permission, and whether the password matches, the manual intervention service is inputting the configuration parameters and manual intervention instructions for the medical staff interaction window to feed back the state machine response result, and the running state message is the current running state published by the main control module in real time.

[0056] Further, the hardware driving nodes include a CAN communication driving node, a gantry driving node and a mobile mask driving node, a 3-PRS driving node, a clamping jaw driving node, and a motion control node.

[0057] The CAN communication driving node is used to complete the bidirectional CAN communication between the upper computer and the lower computer.

[0058] The gantry driving node and the mobile mask driving node are used to control the motor movement of the gantry and the mobile mask of the sampling robot.

[0059] The 3-PRS driving node is used to control the movement of the 3-PRS sampling hand of the throat swab sampling robot.

[0060] The clamping jaw driving node is used to control the opening and closing of the steering engine of the swab clamping jaw of the sampling robot.

[0061] The motion control node is used to complete the complex multi-mechanism cooperative motion control and the realization of mechanism modular operation.

[0062] The servo sampling node is used to jointly control the gantry, the mobile mask, and the sampling hand of the 3-PRS to realize force servo scraping sampling.

[0063] Further, the hardware driving nodes include a force sensor driving node, a seat control node, a UPS monitoring node, and other device driving nodes, wherein the force sensor driving node is used to interact with the force sensor; the seat control node is used to control the liftable seat; the UPS monitoring node is used to monitor the working state of the uninterruptible power supply (UPS); and the other device driving nodes are used to control the mechanisms including the mobile door plate, the test tube storage, the swab storage, and the communication with the sensors such as the photoelectric door.

[0064] Further, the vision control module includes a face tracking node, which is respectively communicatively connected to the hardware driving nodes and the main state machine node.

[0065] The face tracking node is used to send a vision tracking end message to the main state machine node, and to interact with the hardware driving nodes for position information and rapid motion information of the sampling mechanism. The face tracking node can also be used to complete face recognition, image acquisition, mouth opening detection, throat three-dimensional positioning, and the like.

[0066] Further, the comprehensive control module comprises a voice interaction node, a sampled person window node and an identity verification node, the identity verification node is communicatively connected to the voice interaction node and the sampled person window node respectively, the identity verification node is configured to send a voice playing message to the voice interaction node, and send a sampled person information message to the sampled person window node;

[0067] The voice interaction node is communicatively connected to the face tracking node and the servo sampling node respectively, and the voice interaction node is configured to receive the voice playing message from the face tracking node and the servo sampling node;

[0068] The sampled person window node is communicatively connected to the face tracking node and the main state machine node respectively, and the sampled person window node is configured to receive the text prompt message and the running state message from the face tracking node and the main state machine node;

[0069] The identity verification node is communicatively connected to the face tracking node, and the identity verification node is configured to send a visual tracking activation message to the face tracking node.

[0070] The identity verification node can also obtain and verify the identity information of the sampled person by reading an identity card or face recognition, the voice interaction node can generate a voice prompt during the sampling stage, and the sampled person window node can display prompt text, sampled person information, image guidance / prompt information and the like on a visual interface facing the sampled person.

[0071] Further, the main control module, the hardware control module, the visual control module and the comprehensive control module communicate and interact with each other through a first communication mode and a second communication mode of the ROS1 system;

[0072] The first communication mode is in the form of "message", that is, one-sided information publishing and message receiving, and the information publisher and the information receiver are independent of and do not interfere with each other, and are also asynchronous, and the second communication mode is in the form of "service", that is, the service requester sends a request to the service provider, and the service provider responds and feeds back, which is synchronous.

[0073] Further, the running state message is the current running state of the throat swab sampling robot published by the main control module in real time, and the manual intervention service is that a medical staff inputs a configuration parameter and an artificial intervention instruction in the interactive window.

[0074] Further, the position information of the sampling mechanism includes end position information of a gantry of the sampling mechanism and a movable mask of the sampling mechanism, and the interaction of the rapid motion information of the sampling mechanism includes receiving a relative motion instruction of the sampling mechanism and driving the sampling mechanism to complete rapid following during visual servo throat tracking.

[0075] Among them, the execution instruction service is specifically a motor / device control instruction in the form of a string, and feedback is whether the execution is successful or not. The execution script service is specifically a motor / device control script file name in the form of a string, and feedback is whether the execution is successful or not. The force feedback situation message is specifically to publish the reading of the force feedback sensor. The sampling mechanism position message is specifically to publish the end position information of the sampling mechanism (including the gantry and the moving mask). The sampling mechanism rapid motion message is specifically to receive the relative motion instruction of the sampling mechanism and drive the sampling mechanism to complete the corresponding action (used for the rapid following of the sampling mechanism when visual servoing throat tracking). The sampling start message is specifically to receive the message, and the sampling mechanism starts to perform the sampling operation. The sampling end message is specifically to publish the sampling success / failure after the sampling ends.

[0076] Further, when the visual tracking activation message is listened to, the visual servoing throat tracking is started. When the visual throat servoing tracking ends, the visual tracking end message is published. The voice playing message and the text prompt message are published according to the direction of the target point deviation while the visual throat servoing tracking is performed.

[0077] Among them, the visual tracking activation message is specifically to start the visual servoing throat tracking when the message is listened to. The visual tracking end message is specifically to publish the message when the visual throat servoing tracking ends. The voice playing message is specifically to publish the voice prompt message and the text prompt message according to the direction of the target point deviation while the visual throat servoing tracking is performed. The sampling mechanism rapid motion message is specifically to publish the sampling mechanism motion instruction according to the direction of the target point deviation in real time during the visual servoing throat tracking. The image message is specifically to publish the image of the throat part of the person to be sampled after highlighting the image during the visual servoing throat tracking, so as to provide visual guidance in the form of image for the person to be sampled.

[0078] Among them, in the comprehensive control module, the voice playing message is specifically to receive the voice content in the form of a string and play the corresponding audio. The text prompt message is specifically to receive the text prompt content in the form of a string and display it on the window of the person to be sampled. The person to be sampled information message is specifically to publish the information of the person to be sampled obtained by scanning the code or face recognition. The image message is specifically to receive the image message and display it on the interface in real time.

[0079] In some real-time examples, the throat swab sampling robot control system of the present application can also be based on the ROS1 system (i.e. Robot Operating System (Robot Operating System)), according to the characteristics of the ROS1 system, the control system as a whole is designed according to function, each simple function unit (node) can be independently and in parallel in the form of thread in the system, the nodes communicate with each other in the form of message and service, and necessary information is transmitted. Or based on other systems, all belong to the protection scope of the present application, which will not be repeated here.

[0080] Please refer to Figure 3 , Figure 3 is a flowchart of a preferred embodiment of the throat swab sampling robot control method of the present application.

[0081] The present application also proposes a throat swab sampling robot control method based on the throat swab sampling robot control system of the above-mentioned embodiment, the method comprising:

[0082] Step S100: the main control module receives the visual tracking message of the visual control module, and publishes the visual tracking state of the throat swab sampling robot in real time according to the visual tracking message, at the same time sends a text prompt message or a running state message to the comprehensive control module, and controls the comprehensive control module to display the text prompt message or the running state message on the user interface of the throat swab sampling robot in real time;

[0083] Step S200: the hardware control module and the visual control module interact with each other, and publish the motion instruction of the sampling mechanism of the throat swab sampling robot in real time according to the information provided by the visual control module, and deliver voice playing information to the comprehensive control module, and control the comprehensive control module to control the throat swab sampling robot to play voice;

[0084] Step S300: the visual control module monitors the throat of the patient and tracks the throat position, and delivers voice playing information or text image prompt information to the comprehensive control module, and when receiving the visual tracking activation message sent by the comprehensive control module, controls the throat swab sampling robot to start the visual servo tracking function;

[0085] Step S400: the comprehensive control module receives the sampling information delivered by the main control module, the hardware control module and the visual control module, and displays it on the user interface of the throat swab sampling robot in real time, and sends the visual tracking activation message to the visual control module, and controls the visual tracking function of the robot system.

[0086] To sum up, the application sets the main control module, the hardware control module, the visual control module and the comprehensive control module connected in turn, so that the software control system is divided into multiple independent function modules according to function modules, each function module contains several program nodes that can run independently, each node communicates with each other through messages and services to realize overall operation, and the main control module is used for receiving the visual tracking message sent by the visual control module, sending prompt and running state message to the comprehensive control module, and interacting with the hardware control module to realize the function of the main control module. Secondly, the hardware control module of the application is used for information interaction with the visual control module of the sampling mechanism, and the voice playing information is transmitted to the comprehensive control module. Thirdly, the visual control module of the application is used for transmitting voice playing or text image prompt information to the comprehensive control module, and receiving the visual tracking activation message sent by the comprehensive control module. The ROS1 system module development, communication-based and flexible extensible characteristics are used, the robot software control system uses the ROS1 operating system to realize the overall operation control of the robot, has certain extensibility, is convenient for adding new functions and equipment in the later period, and also increases the flexibility and extensibility of the throat swab sampling robot, and can intelligently cope with various uncertain problems that may occur in the sampling process.

[0087] It should be noted that the various optional embodiments introduced in the embodiments of the application can be combined with each other to realize, or can be realized alone, and the embodiments of the application do not limit this.

[0088] In the description of the application, it should be understood that the terms "upper", "lower", "left", "right", etc. indicate the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, and a particular orientation configuration and operation. Therefore, it cannot be understood as a limitation on the application. In addition, "first", "second" is only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features limited by "first", "second" can explicitly or implicitly include one or more features. In the description of the application, unless otherwise specified, the meaning of "multiple" is two or more.

[0089] In the description of the application, it should be noted that, unless otherwise specified and limited, the terms "mounting", "connecting", "connecting" and the like should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the application can be understood according to the specific circumstances.

[0090] The foregoing embodiments are described in reference to the drawings, which are not necessarily drawn to scale, and in which like numerals refer to like components, where appropriate. The foregoing embodiments are described in sufficient detail to enable those skilled in the art to make and use the application, and to implement and use alternative forms of the application. The foregoing embodiments are therefore not to be construed as limiting the application but merely as being illustrative. The herein disclosed subject matter is to be provided in the context of what is considered to be the most practical and preferred embodiment, and those skilled in the art will recognize that the protection afforded to the application, 1s not limited to those embodiments actually disclosed. It is therefore desired that what is claimed be interpreted as broadly as it will fairly be interpreted consistent with the specification. In the drawings, the size and relative sizes of components can be exaggerated for clarity. The use of the same reference numbers in different figures indicates similar or identical components.

[0091] The foregoing embodiments are described in reference to the drawings, which are not necessarily drawn to scale, and in which like numerals refer to like components, where appropriate. The foregoing embodiments are described in sufficient detail to enable those skilled in the art to make and use the application, and to implement and use alternative forms of the application. The foregoing embodiments are therefore not to be construed as limiting the application but merely as being illustrative. The herein disclosed subject matter is to be provided in the context of what is considered to be the most practical and preferred embodiment, and those skilled in the art will recognize that the protection afforded to the application, 1s not limited to those embodiments actually disclosed. It is therefore desired that what is claimed be interpreted as broadly as it will fairly be interpreted consistent with the specification. In the drawings, the size and relative sizes of components can be exaggerated for clarity. The use of the same reference numbers in different figures indicates similar or identical components.

Claims

1. A control system for a throat swab sampling robot, characterized in that, It includes a main control module, a hardware control module, a vision control module, and an integrated control module that are connected in sequence via communication. The main control module is used to receive visual tracking messages from the visual control module, and publish the visual tracking status of the pharyngeal swab sampling robot in real time according to the visual tracking messages. At the same time, it sends text prompt messages or running status messages to the integrated control module, and controls the integrated control module to display the text prompt messages or running status messages on the user interface of the pharyngeal swab sampling robot in real time. The main control module includes a monitoring window node and a main state machine node. The monitoring window node is communicatively connected to the main state machine node, and the monitoring window node and the main state machine node interact with each other to exchange the running status messages and manual intervention services. The hardware control module includes a hardware driver node and a servo sampling node, and the hardware driver node is communicatively connected to the servo sampling node. The master state machine node is communicatively connected to the servo sampling node and is used to send a sampling start message to the servo sampling node. The monitoring window node is communicatively connected to the hardware driver node and is used to interact with the hardware driver node to exchange execution instruction information. The hardware driving nodes include a CAN communication driving node, a gantry driving node and a moving mask driving node, a 3-PRS driving node, a gripper driving node and a motion control node. The CAN communication driver node is used to complete bidirectional CAN communication between the host computer and the slave computer. The gantry drive node and the moving mask drive node are used to control the motor movement of the gantry and moving mask of the sampling robot. The 3-PRS drive node is used to control the movement of the 3-PRS sampling hand of the pharyngeal swab sampling robot; The gripper drive node is used to control the opening and closing of the swab gripper of the sampling robot; The motion control node is used for multi-mechanism collaborative motion control and modular operation of mechanisms; The servo sampling node is used to jointly control the gantry, the moving mask, and the sampling hand of the 3-PRS to achieve force servo scraping sampling; The hardware control module is used to interact with the vision control module, and to issue motion commands for the sampling mechanism of the pharyngeal swab sampling robot in real time based on the information provided by the vision control module. It also transmits voice playback information to the integrated control module and controls the integrated control module to control the pharyngeal swab sampling robot to play voice. The visual control module is used to monitor the patient's throat and track its position, and transmit voice playback information or text image prompts to the integrated control module. When it receives the visual tracking activation message sent by the integrated control module, it controls the throat swab sampling robot to start the visual servo tracking function. The integrated control module is used to receive sampling information transmitted by the main control module, the hardware control module and the vision control module, display it in real time on the user interface of the throat swab sampling robot, and send the vision tracking activation message to the vision control module to control the vision tracking function of the robot system.

2. The throat swab sampling robot control system according to claim 1, characterized in that, The main control module also includes a password verification node, which is used to verify permissions when medical personnel log in.

3. The throat swab sampling robot control system according to claim 1, characterized in that, The vision control module includes a face tracking node, which is communicatively connected to the hardware driver node and the main state machine node. The face tracking node is used to send a visual tracking end message to the main state machine node and to interact with the hardware driver node to obtain the position information and rapid motion information of the sampling mechanism.

4. The throat swab sampling robot control system according to claim 3, characterized in that, The integrated control module includes a voice interaction node, a sampled user window node, and an identity verification node. The identity verification node is communicatively connected to the voice interaction node and the sampled user window node, respectively. The identity verification node is used to send voice playback messages to the voice interaction node and to send sampled user information messages to the sampled user window node. The voice interaction node is communicatively connected to the face tracking node and the servo sampling node, respectively, and the voice interaction node is used to receive voice playback messages from the face tracking node and the servo sampling node; The sampled user window node is communicatively connected to the face tracking node and the main state machine node, respectively. The sampled user window node is used to receive text prompt messages and running status messages from the face tracking node and the main state machine node. The identity verification node is communicatively connected to the face tracking node, and the identity verification node is used to send a visual tracking activation message to the face tracking node.

5. The pharyngeal swab sampling robot control system according to claim 1, characterized in that, The main control module, the hardware control module, the vision control module, and the integrated control module communicate and interact with each other through the first and second communication methods of the ROS1 system. The first communication method involves one-way information publishing and message receiving, with the information publisher and the information receiver being independent of each other and not interfering with each other. The second communication method involves the service requester sending a request to the service provider, which then responds and provides feedback.

6. The pharyngeal swab sampling robot control system according to claim 1, characterized in that, The running status message is the current running status of the pharyngeal swab sampling robot published in real time by the main control module, and the manual intervention service is for medical personnel to input configuration parameters and manual intervention commands in the interactive window.

7. The throat swab sampling robot control system according to claim 4, characterized in that, The position information of the sampling mechanism includes the position information of the gantry of the sampling mechanism and the end position information of the moving mask of the sampling mechanism. The interaction of the rapid motion information of the sampling mechanism includes receiving the relative motion command of the sampling mechanism and driving the sampling mechanism to complete the rapid following when performing visual servo throat tracking.

8. A method for controlling a pharyngeal swab sampling robot based on the pharyngeal swab sampling robot control system according to any one of claims 1-7, characterized in that, include: The main control module receives the visual tracking message from the visual control module, and publishes the visual tracking status of the pharyngeal swab sampling robot in real time according to the visual tracking message. At the same time, it sends a text prompt message or running status message to the integrated control module, and controls the integrated control module to display the text prompt message or running status message on the user interface of the pharyngeal swab sampling robot in real time. The hardware control module interacts with the vision control module, and releases motion commands for the sampling mechanism of the pharyngeal swab sampling robot in real time based on the information provided by the vision control module. It also transmits voice playback information to the integrated control module, and controls the integrated control module to control the pharyngeal swab sampling robot to play voice messages. The visual control module monitors the patient's throat and tracks its position, and transmits voice playback information or text image prompts to the integrated control module. When it receives the visual tracking activation message sent by the integrated control module, it controls the throat swab sampling robot to start the visual servo tracking function. The integrated control module receives sampling information from the main control module, the hardware control module, and the vision control module, displays it in real time on the user interface of the throat swab sampling robot, and sends the vision tracking activation message to the vision control module to control the vision tracking function of the robot system.

Images