Channel expansion control method, device and system

Abstract

The present application relates to a channel expansion control method, device and system, and the method is used for a remote control device to control a local puncture device to expand a puncture channel. The method includes: determining a puncture path for puncturing a target target in an ultrasound angiography image showing a target target; controlling the local puncture device to puncture along the puncture path according to the position information of the puncture path in the ultrasound angiography image; when the local puncture device completes the puncture, sending a channel expansion instruction to the local puncture device to control the local puncture device to expand the puncture channel; wherein the local puncture device includes a first robotic arm and a second robotic arm, and when the local puncture device receives a channel expansion instruction, the first robotic arm is controlled to grab a first tool for fixing the channel, and the second robotic arm is controlled to grab a second tool for expanding the channel. The use of the method can improve the convenience of expanding the puncture channel.

Description

Technical Field

[0001] The present application relates to the field of medical technology, and in particular to a channel expansion control method, device and system. Background Art

[0002] With the continuous development of computer technology, medical technology has also ushered in a major breakthrough. Contrast ultrasound-guided puncture lithotripsy has begun to be used in percutaneous renal lithotripsy. In the current clinical contrast ultrasound-guided puncture lithotripsy method, the process of expanding the puncture channel formed by puncture is inconvenient. Summary of the invention

[0003] Based on this, it is necessary to provide a channel expansion control method, device and system to address the above technical problems, so as to improve the convenience of expanding the puncture channel.

[0004] In a first aspect, the present application provides a channel expansion control method, the method being used for a remote control device to control a local puncture device to expand a puncture channel, the puncture channel being a channel formed by puncture; the method comprising:

[0005] Determining a puncture path for puncturing the target point in an ultrasound contrast image showing the target point;

[0006] According to the position information of the puncture path in the ultrasound angiography, controlling the local puncture device to perform puncture along the puncture path;

[0007] When the local puncture device completes the puncture, sending a channel expansion instruction to the local puncture device to control the local puncture device to expand the puncture channel;

[0008] Wherein, the local puncture device includes a first robotic arm and a second robotic arm. When the local puncture device receives the channel expansion instruction, the first robotic arm is controlled to grab the first tool for fixing the channel, and the second robotic arm is controlled to grab the second tool for expanding the channel.

[0009] The local puncture device refers to a device located in the local area where the puncture is performed and used for puncture and lithotripsy; the remote control device refers to a device that is located at a different location from the local puncture device and is used to remotely control the local puncture device for puncture and lithotripsy. After the remote control device determines the puncture path for puncturing the target target in the ultrasound angiography image showing the target target, it controls the local puncture device to puncture according to the puncture path according to the position information of the puncture path in the ultrasound angiography image. Furthermore, when the local puncture device completes the puncture, the remote control device sends a channel expansion instruction to the local puncture device to control the local puncture device to expand the puncture channel. Specifically, the local puncture device includes a first mechanical arm and a second mechanical arm. When the local puncture device receives the channel expansion instruction sent by the remote control device, based on the channel expansion instruction, under the control of the remote control device, the first mechanical arm included in it is automatically controlled to grab the first tool of the fixed channel, and the second mechanical arm included in it is controlled to grab the second tool of the expanded channel. By controlling the joint operation of the first mechanical arm and the second mechanical arm, the expansion and fixation of the puncture channel formed by the puncture is achieved. By adopting the above-mentioned channel expansion control method, the local puncture device is controlled by a remote control device located in a different place to automatically expand the puncture channel, thereby realizing remote control of the expansion of the puncture channel and intelligent expansion of the puncture channel. It is not necessary for medical personnel to expand the puncture channel in the local area where the puncture is performed, such as in the operating room, which can improve the convenience of expanding the puncture channel.

[0010] In one embodiment, the target point is located in a target tissue containing a calculus; the method further comprises:

[0011] When the local puncture device completes the expansion of the puncture channel, sending a lithotripsy preparation instruction to the local puncture device to control the local puncture device to place a lithotripsy device into the puncture channel;

[0012] If it is determined through the information displayed on the target image that the stone is within the stone crushing range of the stone crushing device, a stone crushing instruction is sent to the local puncture device to control the local puncture device to trigger the stone crushing device to crush the stone;

[0013] The target image is at least one of the following: a first image acquired by the lithotripsy device, a second image acquired by an endoscope introduced into the target tissue, and the ultrasound angiography image.

[0014] In one embodiment, the method further comprises:

[0015] When the local puncture device and the lithotripsy device have completed lithotripsy and the drainage device has been introduced into the target tissue, a drainage instruction is sent to the local puncture device to control the local puncture device to trigger the drainage device to drain water, so that the gravel formed after the stone is crushed is discharged from the puncture channel.

[0016] In one embodiment, when the target image is the ultrasound angiography image, before sending the lithotripsy instruction to the local puncture device, the method further includes:

[0017] Determine the pixel value range of the pixel points corresponding to the stone in the target image;

[0018] After sending a drainage instruction to the local puncture device to control the local puncture device to trigger the drainage device to drain water so that the crushed stones formed after the stones are crushed are discharged from the puncture channel, the method further includes:

[0019] In the case where there is a first target point in the target image whose pixel value belongs to the pixel value range, determining that there is residual gravel in the target tissue;

[0020] A stone removal instruction is sent to the local puncture device to control the local puncture device to place the stone removal device into the puncture channel and remove the residual gravel through the stone removal device.

[0021] In one embodiment, the sending of a stone removal instruction to the local puncture device to control the local puncture device to place the stone removal device into the puncture channel and remove the residual gravel through the stone removal device includes:

[0022] Sending the stone removal instruction to the local puncture device to control the local puncture device to place the stone removal device into the puncture channel along the central axis of the puncture channel, with the starting point of the stone removal device being placed into the puncture channel flush with the end point of the puncture channel;

[0023] According to the position information of the central axis and the residual gravel in the target image, determining the rotation angle and the depth of the stone removal device in the actual space, wherein the depth of the stone removal device refers to the distance the stone removal device needs to move forward after rotation;

[0024] The angle to be rotated and the depth to be removed are sent to the local puncture device to control the local puncture device to carry the stone removal device to move with the angle to be rotated and the depth to be removed as the target, and control the local puncture device to remove the residual gravel through the stone removal device.

[0025] In one embodiment, an ultrasound probe for photographing the ultrasound angiogram is fixed on the local puncture device, and the local puncture device is used to grab a puncture needle for puncture; and controlling the local puncture device to perform puncture along the puncture path according to the position information of the puncture path in the ultrasound angiogram includes:

[0026] Determine the first pixel coordinates of the puncture starting point of the puncture path in the ultrasound contrast image, the second pixel coordinates of the puncture end point of the puncture path in the ultrasound contrast image, and the vertex pixel coordinates of the second target point at the top of the ultrasound contrast image, wherein the second target point corresponds to the target position in the ultrasound probe;

[0027] Using the conversion relationship between the pixel coordinate system of the ultrasound angiogram and the spatial coordinate system of the actual space, the first pixel coordinate is converted into a first spatial coordinate, the second pixel coordinate is converted into a second spatial coordinate, and the vertex pixel coordinate is converted into a vertex spatial coordinate;

[0028] Determine the tip space coordinates and the end space coordinates of the puncture needle based on the vertex space coordinates by using the positional relationship between the target position in the ultrasound probe and the puncture needle grasped by the local puncture device;

[0029] The spatial coordinate information is sent to the local puncture device to control the local puncture device to perform puncture according to the spatial coordinate information, wherein the spatial coordinate information includes the first spatial coordinate, the second spatial coordinate, the tip spatial coordinate, and the end spatial coordinate.

[0030] In one embodiment, sending the spatial coordinate information to the local puncture device to control the local puncture device to perform puncture according to the spatial coordinate information includes:

[0031] Calculating, according to the spatial coordinate information, a puncture angle between the puncture needle and the puncture path and a depth to be punctured by the puncture needle in the actual space;

[0032] The puncture angle and the depth to be punctured are sent to the local puncture device to control the local puncture device to perform puncture according to the puncture angle and the depth to be punctured.

[0033] In one embodiment, the local puncture device comprises a third mechanical arm, on which an ultrasound probe is fixed; before determining a puncture path for puncturing the target point in an ultrasound angiography image showing the target point, the method further comprises:

[0034] Acquiring an ultrasonic image taken by the ultrasonic probe;

[0035] Sending the posture data to the local puncture device to control the third mechanical arm fixed with the ultrasound probe to move with the posture data as a target, and when the third mechanical arm moves with the posture data as a target, the target point of the object is displayed in the ultrasound image;

[0036] In the case where the subject is intravenously injected with an ultrasound contrast agent, the ultrasound contrast image is obtained based on the ultrasound image, and the ultrasound contrast image is an image after contrast development of the ultrasound image.

[0037] In a second aspect, the present application further provides a channel expansion control device, which is used for a remote control device to control a local puncture device to expand a puncture channel, wherein the puncture channel refers to a channel formed by puncture; the device comprises:

[0038] A path determination module, used to determine a puncture path for puncturing the target point in an ultrasound angiography image showing the target point;

[0039] A puncture control module, used for controlling the local puncture device to perform puncture along the puncture path according to the position information of the puncture path in the ultrasound angiography image;

[0040] A channel expansion module, used for sending a channel expansion instruction to the local puncture device when the local puncture device completes the puncture, so as to control the local puncture device to expand the puncture channel;

[0041] Wherein, the local puncture device includes a first robotic arm and a second robotic arm. When the local puncture device receives the channel expansion instruction, the first robotic arm is controlled to grab the first tool for fixing the channel, and the second robotic arm is controlled to grab the second tool for expanding the channel.

[0042] In a third aspect, the present application also provides a channel expansion control system, the system is used to expand a puncture channel, the puncture channel refers to a channel formed by puncture; the system includes a remote control device and a local puncture device connected to each other;

[0043] The remote control device is used to determine a puncture path for puncturing the target target in an ultrasound angiography image showing the target target; control the local puncture device to perform puncture along the puncture path according to the position information of the puncture path in the ultrasound angiography image; and send a channel expansion instruction to the local puncture device when the local puncture device completes the puncture, so as to control the local puncture device to expand the puncture channel;

[0044] The local puncture device is used to expand the puncture channel under the control of the remote control device;

[0045] Wherein, the local puncture device includes a first robotic arm and a second robotic arm. When the local puncture device receives the channel expansion instruction, the first robotic arm is controlled to grab the first tool for fixing the channel, and the second robotic arm is controlled to grab the second tool for expanding the channel.

[0046] In a fourth aspect, the present application also provides a remote control device, comprising: a memory and a processor, wherein the memory stores program instructions; when the program instructions are executed by the processor, the processor executes the method shown in the first aspect or any embodiment of the first aspect.

[0047] In a fifth aspect, the present application further provides a computer-readable storage medium, in which a computer program is stored; when the computer program runs on one or more processors, the method shown in the first aspect or any embodiment of the first aspect is executed.

[0048] In a sixth aspect, the present application further provides a computer program product, comprising a computer program or instructions; when the computer program or instructions are run on a computer, the computer is enabled to execute the method as shown in the first aspect or any embodiment of the first aspect.

[0049] It can be understood that the channel expansion control device provided in the second aspect, the channel expansion control system provided in the third aspect, the remote control device provided in the fourth aspect, the computer readable storage medium provided in the fifth aspect, and the computer program product provided in the sixth aspect can be used to execute the channel expansion control method shown in the first aspect of the present application or any embodiment of the first aspect. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the channel expansion control method, which will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments are introduced as follows.

[0051] Figure 1 A schematic diagram of a local puncture device provided in an embodiment of the present application;

[0052] Figure 2 A schematic diagram of a remote control device provided in an embodiment of the present application;

[0053] Figure 3 One of the schematic diagrams of the display interface of the second display provided in the embodiment of the present application;

[0054] Figure 4 A second schematic diagram of a display interface of a second display provided in an embodiment of the present application;

[0055] Figure 5 A schematic diagram of a flow chart of a channel expansion control method provided in an embodiment of the present application;

[0056] Figure 6 A schematic diagram of a second robotic arm provided in an embodiment of the present application;

[0057] Figure 7 A schematic diagram of a first robotic arm provided in an embodiment of the present application;

[0058] Figure 8 A schematic diagram of placing a lithotripsy device into a puncture channel provided in an embodiment of the present application;

[0059] Fig. 9 A schematic diagram of the discharge of gravel from the puncture channel provided in an embodiment of the present application;

[0060] Fig.10 One of the schematic diagrams of placing a stone removal device into a puncture channel and removing residual gravel through the stone removal device provided in an embodiment of the present application;

[0061] Fig.11 A second schematic diagram of placing a stone removal device into a puncture channel and removing residual gravel through the stone removal device provided in an embodiment of the present application;

[0062] Fig.12 A third schematic diagram of placing a stone removal device into a puncture channel and removing residual gravel through the stone removal device provided in an embodiment of the present application;

[0063] Fig.13 Ultrasound angiography provided in the embodiment of the present application;

[0064] Fig.14 A schematic diagram of an ultrasound probe provided in an embodiment of the present application;

[0065] Fig.15 A schematic diagram of a partial structure of a local puncture device provided in an embodiment of the present application;

[0066] Fig.16 A schematic diagram of the structure of a channel expansion control device provided in an embodiment of the present application;

[0067] Fig.17 A schematic diagram of the structure of a channel expansion control system provided in an embodiment of the present application;

[0068] Fig.18 A schematic diagram of the structure of the remote control device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0069] In order to facilitate understanding of the embodiments of the present application, the embodiments of the present application will be described more fully below with reference to the relevant drawings. The preferred embodiments of the embodiments of the present application are given in the drawings. However, the embodiments of the present application can be implemented in many different forms and are not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the embodiments of the present application more thorough and comprehensive.

[0070] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the technical field of the embodiments of the present application. The terms used herein in the description of the embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the embodiments of the present application.

[0071] The terms "first", "second", etc. used in this document are used to distinguish different objects rather than to describe a specific order. For the purpose of distinguishing different objects, the "first robotic arm" can also be described as the "second robotic arm", and the "second robotic arm" can be described as the "first robotic arm". When used, the singular expressions "one", "a", "said", "above", "the" and "this" are intended to include plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term "includes / comprising" specifies the presence of stated features, wholes, steps, operations, parts or their fusions, but does not exclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, parts or their fusions.

[0072] With the continuous development of computer technology, medical technology has also ushered in a major breakthrough. Contrast ultrasound-guided puncture and lithotripsy has begun to be used in percutaneous renal puncture and lithotripsy. In the current clinical contrast ultrasound-guided puncture and lithotripsy method, the corresponding medical personnel are usually required to operate at the local operation site, such as the medical personnel manually expanding the puncture channel formed by the puncture based on experience, or expanding with the assistance of a machine. If the corresponding medical personnel cannot return to the local operation site in time, and there is no medical personnel who can perform expansion control at the local operation site, it will make it difficult to expand the puncture channel. In the clinical contrast ultrasound-guided puncture and lithotripsy method, there is inconvenience in the expansion process of the puncture channel. Based on this, the embodiment of the present application provides a channel expansion control method, which controls the local puncture device to automatically expand the puncture channel through a remote control device located in a different place, realizes remote control of the expansion puncture channel, and intelligent expansion of the puncture channel. There is no need for medical personnel to expand the puncture channel at the local location where the puncture is performed, such as in the operating room, which can improve the convenience of expanding the puncture channel.

[0073] The channel expansion control method provided in the embodiment of the present application is used for a remote control device to control a local puncture device to expand a puncture channel. The puncture channel refers to a channel formed by puncture; a local puncture device refers to a device located in the local area where the puncture is performed and used for puncture and lithotripsy; a remote control device refers to a device that is located at a different location from the local puncture device and is used to remotely control the local puncture device for puncture and lithotripsy. For example, the local puncture device is located in the operating room of Hospital A, and the remote control device is located in Hospital B. Medical personnel in other places can control the local puncture device in the operating room of Hospital A through a remote control device in Hospital B to automatically expand the puncture channel, thereby improving the convenience of expanding the puncture channel.

[0074] Before introducing the specific scheme of the channel expansion control method in the embodiment of the present application, the relevant information of the remote control device and the local puncture device involved in the channel expansion control method is introduced as follows.

[0075] Figure 1 Schematic diagram of the local puncture device provided in the embodiment of the present application. Figure 1 As shown, the local puncture device at least includes a mechanical arm, which has at least one of the following uses: fixing the ultrasound probe, grabbing the puncture needle, grabbing the tool for fixing the puncture channel and expanding the puncture channel, grabbing the lithotripsy device, grabbing the stone removal device. The local puncture device may also include a first display, which is used to display information such as ultrasound images for local assistants to view. Figure 2 A schematic diagram of a remote control device provided in an embodiment of the present application. Figure 2 As shown, the remote control device includes at least a force feedback device and at least one second display. The force feedback device is used to control the movement of the mechanical arm in the local puncture device, and the second display in the remote control device is used to display information such as ultrasound images for viewing by medical personnel in a remote location who control the remote control device. Figure 2The remote control device shown includes three second displays. When the remote control device includes three second displays, the three second displays of the remote control device can display the following image information: a CT three-dimensional model image generated by a local computed tomography (CT) scan of the target tissue, a soft endoscope image collected by an endoscope introduced into the object, a local surgical scene image collected by a local surgical shooting device, an ultrasound image collected by a local surgical ultrasound probe, and an interactive interface for the remote control device to control the local puncture device. Among them, medical personnel in other places can understand the overall situation of the target tissue of the object, such as the kidney, and the target target points and stones in the target tissue based on the CT three-dimensional model image; can view the specific situation of the target tissue and the target target points and stones in the target tissue based on the soft endoscope image; can understand the local actual situation of the surgery based on the local surgical scene image; can touch the remote control device based on the ultrasound image and the interactive interface, so that the remote control device controls the local puncture device to perform puncture, expansion of the puncture channel, lithotripsy, etc. For more detailed uses of each image information, please refer to the specific introduction in the following embodiments. Each of the above image information can be displayed in the three second displays separately or in a combination of two. Specifically, among the three second displays of the remote control device, the first one can be used to display the CT three-dimensional model image and the soft mirror image. The display interface of this second display is as follows: Figure 3 As shown, Figure 3 One of the schematic diagrams of the display interface of the second display provided in the embodiment of the present application; the second one can be used to display the local surgical scene diagram; the third one can be used to display the ultrasound image and the interactive interface. The display interface of the second display is as follows Figure 4 As shown, Figure 4 This is a second schematic diagram of the display interface of the second display provided in an embodiment of the present application.

[0076] It should be noted that in the embodiment of the present application, the number of second displays of the remote control device can be set according to actual needs, and the image information displayed in the second display can also be set according to actual needs. For example, in the process of implementing the channel expansion control method, if other image information is needed to assist, or displaying each image information separately in the second display is more conducive to the implementation of the method, the number of second displays can continue to increase; if combining and displaying each image information is more conducive to the implementation of the method, the number of second displays can be reduced accordingly; if it is not necessary to view the CT three-dimensional model diagram, the local surgical scene diagram, etc. in a different place, there is no need to display the CT three-dimensional model diagram, the local surgical scene diagram, etc. in the second display. The embodiment of the present application only takes the example that the remote control device includes three second displays, and the three second displays display the above-mentioned image information, and does not constitute a limitation on the structure of the remote control device and the role of each second display.

[0077] Information transmission can be performed between the remote control device and the local puncture device through the cloud server. Specifically, the remote control device and the local puncture device are respectively connected to the cloud server, and then the remote control device and the local puncture device communicate with the cloud server through the network communication protocol respectively. The cloud server transfers the received data to realize the communication between the remote control device and the local puncture device in different places. The network communication protocol is such as the Transmission Control Protocol / Internet Protocol (TCP / IP). Based on this, the remote control device obtains the CT three-dimensional model image, soft mirror image, and local surgical scene image in real time in any of the following ways. First, the corresponding acquisition device of the above-mentioned image information is connected to the cloud server, and communicates with the cloud server through a network communication protocol such as TCP / IP. The cloud server transfers the received data to realize the communication between the corresponding acquisition device and the remote control device. Second, the corresponding acquisition device of the above-mentioned image information is connected to the local puncture device, and the corresponding data is sent to the local puncture device, and then the communication between the remote control device and the local puncture device is realized through the cloud server, so as to realize the communication between the remote control device and the corresponding acquisition device. It should be understood that the remote control device can communicate with different acquisition devices in different ways, such as the remote control device obtains the CT three-dimensional model image in real time through one way, and obtains the soft mirror image in real time through the second way. In an embodiment of the present application, the local ultrasound probe of the operation is connected to the local puncture device, and the local puncture device obtains the ultrasound image from the ultrasound probe and displays the ultrasound image in the first display of the local puncture device. On this basis, the remote control device also remotely controls the local puncture device through remote control software such as Sunflower, and displays the ultrasound image displayed in the local puncture device in its own second display through the remote control software. In addition, the second display of the remote control device displays an interactive interface for controlling the local puncture device, and the remote control device responds to the touch operation of the controls in the interactive interface, and remotely controls the local puncture device based on the remote control software.

[0078] Figure 5 The following is a flow chart of the channel expansion control method provided in the embodiment of the present application. Figure 5 As shown, the channel expansion control method includes the following steps S101 to S103.

[0079] S101, determining a puncture path for puncturing the target point in an ultrasound angiography image showing the target point.

[0080] Among them, when the local ultrasound probe of the operation collects the target target showing the object, the local puncture device obtains the ultrasound image from the ultrasound probe, and then the remote control device displays the ultrasound image displayed in the local puncture device on its own second display through the remote control software. The remote control device obtains the ultrasound image showing the target target of the object in real time. Taking the application of the channel expansion control method in this embodiment to percutaneous renal lithotripsy as an example, in percutaneous renal lithotripsy, when the object is injected intravenously with ultrasound contrast agent, the remote control device obtains the ultrasound image after ultrasound image contrast development in real time.

[0081] The remote control device determines the puncture path for puncturing the target target in the ultrasound angiography image showing the target target. Exemplarily, the remote control device segments the ultrasound angiography image to obtain and mark the vascular area in the ultrasound angiography image; the puncture path for puncturing the target target is determined in the ultrasound angiography image, and the puncture path does not pass through the vascular area. Specifically, the remote control device can segment the ultrasound angiography image in real time based on the image segmentation model to obtain the vascular area in the ultrasound angiography image. Then, the vascular area is marked in time in the ultrasound angiography image so that the vascular area is retained and displayed. It can be realized that the remote control device segments the ultrasound angiography image in real time, and by comparing the vascular areas segmented at different times, it is determined that the vascular area segmented at the time when the vascular development is clearest is the required vascular area. And the vascular area segmented at the time when the vascular development is clearest is marked and retained and displayed. Among them, the image segmentation model is a model obtained by supervised training with references through labeled data, and the labeled data is the segmented vascular area in the ultrasound angiography image. The remote control device can also segment the ultrasound image based on the touch and manual selection of the ultrasound image by the medical staff in another place, and obtain and mark the vascular area in the ultrasound image. Further, the remote control device determines the puncture path for puncturing the target target in the ultrasound image, and the puncture path does not pass through the vascular area. Exemplarily, the remote control device marks the puncture safety area in the ultrasound image, and the puncture safety area does not include the vascular area; the puncture path for puncturing the target target is determined in the puncture safety area. Among them, the puncture path is the path between the needle entry point on the skin of the object and the target target of the object. When determining the puncture path, the remote control device can make the length of the puncture path as the shortest as the goal, determine the needle entry point and puncture path on the skin of the object, and obtain a safe puncture path with the shallowest puncture depth. The remote control device plans a safe puncture path in the ultrasound image, and the puncture path can be continuously displayed in the ultrasound image.

[0082] For example, Figure 4In the interactive interface shown, there is a "Show Auxiliary Image" control. When the remote medical staff touches to turn on this control, the remote medical staff or the remote control device can mark the vascular area, puncture path, puncture safety area and other information in the ultrasound angiography. After marking, the remote control device can automatically identify the location information of the marked information, such as pixel coordinates.

[0083] It should be understood that during the process of the remote control device segmenting the ultrasound angiogram, marking the vascular area and determining the puncture path, medical personnel in other places can view the CT three-dimensional model image to understand the overall situation of the target tissue and the target target in the target tissue, so as to assist in marking the vascular area and determining the puncture path.

[0084] In the current clinical contrast ultrasound guided puncture lithotripsy method, medical personnel with rich puncture experience are usually required to perform puncture at the local operation site. When medical personnel judge the vascular area in the ultrasound angiography based on puncture experience, it is difficult for medical personnel to clearly and accurately judge the vascular area because the overall clarity of vascular development is low and the moment when the vascular development is clearest is fleeting. And because the duration of vascular development is short, if the vascular area is not distinguished in time and the puncture path is not planned in the current time, it is necessary to wait for the next ultrasound angiography or the medical personnel perform puncture based on experience. That is, in the target clinical contrast ultrasound guided puncture lithotripsy method, there is still a deficiency that it is difficult to accurately determine the vascular area and the puncture path, which easily leads to puncture difficulties and low puncture accuracy. In the above example of this embodiment, the remote control device can also accurately determine the vascular area and make the vascular area retain and display, and plan the puncture path based on the vascular area, which is conducive to reducing the difficulty of puncture, increasing the puncture accuracy, and reducing the risk of puncture.

[0085] S102, controlling the local puncture device to perform puncture along the puncture path according to the position information of the puncture path in the ultrasound angiography.

[0086] The remote control device plans a safe puncture path in the ultrasound angiography image, and can determine the position information of the puncture path in the ultrasound angiography image based on the pixel coordinate system corresponding to the ultrasound angiography image. Furthermore, based on the conversion relationship between the pixel coordinate system corresponding to the ultrasound angiography image and the spatial coordinate system in the local actual space of the operation, the position information of the puncture path in the actual space can be obtained. Furthermore, based on the position information of the puncture path in the actual space, the remote control device can establish communication with the local puncture device via the cloud server, and / or respond to the touch operation of the corresponding control in the interactive interface, based on the remote control software, remotely control the local puncture device to perform puncture according to the puncture path.

[0087] In this embodiment, the channel expansion control method can also enable the remote control device to remotely control the local puncture device to automatically puncture according to the puncture path, eliminating the need for medical personnel to perform puncture locally, which can also improve the convenience of puncture.

[0088] S103, when the local puncture device completes the puncture, sending a channel expansion instruction to the local puncture device to control the local puncture device to expand the puncture channel.

[0089] The local puncture device includes a first robotic arm and a second robotic arm. When the local puncture device receives a channel expansion instruction, the first robotic arm is controlled to grab a first tool for fixing the channel, and the second robotic arm is controlled to grab a second tool for expanding the channel.

[0090] When the local puncture device completes the puncture, the remote control device sends a channel expansion instruction to the local puncture device via the cloud server to control the local puncture device to expand the puncture channel. Specifically, the local puncture device includes a first robotic arm and a second robotic arm. When the local puncture device receives the channel expansion instruction sent by the remote control device, based on the channel expansion instruction, under the control of the remote control device, the first robotic arm it contains automatically controls the first tool that grabs the fixed channel and controls the second robotic arm it contains to grab the second tool that expands the channel. By controlling the first robotic arm and the second robotic arm to operate jointly, the expansion and fixation of the puncture channel formed by the puncture is achieved. Among them, the first tool is such as a guide wire, and the second tool is such as a dilator. The first robotic arm and the second robotic arm are both flexible robotic arms, and this embodiment does not limit the specific types of the first robotic arm and the second robotic arm. Optionally, since the process of grabbing and expanding the puncture channel requires a high degree of flexibility of the robotic arm, the second robotic arm can be a serpentine multi-joint robotic arm. A serpentine multi-joint robotic arm with strong flexibility can refer to Figure 6 As shown, Figure 6 Schematic diagram of the second robotic arm provided in the embodiment of the present application. The first robotic arm may be a general cooperative robotic arm, such as Figure 7 As shown, Figure 7 A schematic diagram of a first robotic arm provided in an embodiment of the present application.

[0091] Exemplarily, the specific process of the local puncture device automatically controlling the first mechanical arm and the second mechanical arm to operate jointly under the control of the remote control device is as follows. First, the serpentine multi-joint mechanical arm is controlled to grasp the first dilator of the expansion channel and put it on the periphery of the puncture needle in the puncture channel, and the universal cooperative mechanical arm is controlled to take the puncture needle out of the puncture channel. Secondly, the universal cooperative mechanical arm is controlled to grasp the first guide wire of the fixed channel and insert it into the first dilator in the puncture channel. The first guide wire is thicker than the puncture needle, and the serpentine multi-joint mechanical arm is controlled to take the first dilator out of the puncture channel. Thirdly, the serpentine multi-joint mechanical arm is controlled to grasp the second dilator of the expansion channel and put it on the periphery of the first guide wire in the puncture channel. The diameter of the second dilator is greater than the diameter of the first dilator, and the universal cooperative mechanical arm is controlled to take the first guide wire out of the puncture channel. The thickness of the guide wire inserted into the puncture channel gradually increases, and the diameter of the dilator inserted into the puncture channel also gradually increases. The local puncture device controls the serpentine multi-joint mechanical arm and the universal cooperative mechanical arm to gradually expand the puncture channel until the diameter of the puncture channel meets the preset requirements.

[0092] Exemplarily, the specific process of the local puncture device controlling the movement of the robot arm to grab the tool includes any of the following. First, the location information of the tool is preset in the local puncture device. Based on the location information of the tool and the current location information and posture information of the corresponding robot arm, the local puncture device controls the movement of the corresponding robot arm to grab the tool. Second, in response to the touch operation of the "Scan Fixed Point" control in the interactive interface, the remote control device has the function of controlling the movement of the robot arm of the local puncture device, and automatically obtains the current location information and posture information of the corresponding robot arm. The remote control device can remotely control the local puncture device based on the remote control software. The interactive interface of the remote control device is as follows: Figure 4 As shown. Furthermore, the force feedback device in the remote control device generates the position information and posture information of the corresponding robot arm target based on the current position information and posture information of the corresponding robot arm and the position information of the tool. Furthermore, the remote control device sends the position information and posture information of the corresponding robot arm target to the local puncture device via the cloud server to control the corresponding robot arm of the local puncture device to move to the position information and posture information of the target to achieve the grasping of the tool. Optionally, when the local puncture device controls the movement of the robot arm, the movement of the robot arm can be specifically controlled by the navigation and positioning device in the local puncture device.

[0093] By adopting the channel expansion control method in this embodiment, firstly, the local puncture device is controlled by a remote control device located in a different place to automatically expand the puncture channel, thereby realizing remote control of the expansion of the puncture channel and intelligent expansion of the puncture channel. There is no need for medical personnel to expand the puncture channel locally during the puncture operation, which can improve the convenience of expanding the puncture channel. Secondly, the remote control device controls the local puncture device to automatically puncture according to the puncture path according to the position information of the puncture path in the ultrasound angiography image, and there is no need for medical personnel to perform puncture locally during the operation, which can improve the convenience of puncture. Thirdly, when the remote control device segments the ultrasound angiography image, obtains and marks the vascular area in the ultrasound angiography image; marks the puncture safety area in the ultrasound angiography image, and the puncture safety area does not include the vascular area; and determines the puncture path for puncturing the target target in the puncture safety area, then this embodiment also has the following beneficial effects. The remote control device can also accurately determine the vascular area and retain the vascular area for display, as well as separate the safe puncture area and plan the puncture path based on the vascular area, which is conducive to reducing the difficulty of puncture, increasing the accuracy of puncture, reducing the risk of puncture, and making the puncture safer and more stable. Finally, the channel expansion control method in this embodiment is used to perform puncture lithotripsy under contrast ultrasound, from the determination of the vascular area to the expansion of the puncture channel, which can achieve full intelligent mechanization.

[0094] In one embodiment, the target target point is located in a target tissue containing stones. In the above step S103, when the local puncture device completes the puncture, after sending a channel expansion instruction to the local puncture device to control the local puncture device to expand the puncture channel, the channel expansion control method further includes the following steps S201 to S202.

[0095] S201, when the local puncture device completes the expansion of the puncture channel, a lithotripsy preparation instruction is sent to the local puncture device to control the local puncture device to place the lithotripsy device into the puncture channel.

[0096] Among them, taking the application of the channel expansion control method in this embodiment to percutaneous renal lithotripsy as an example, the target tissue is the kidney, and the target point is located in the kidney containing stones. Exemplarily, the calyx region of the kidney contains stones, and the target point is located in the calyx region. Since the puncture channel is obtained based on the puncture path, and the puncture path is determined based on the target point, in order to target the stones based on the puncture channel, the distance between the stones in the target tissue and the target point should be as small as possible. Optionally, the target point is a stone.

[0097] When the local puncture device completes the expansion of the puncture channel, the remote control device sends a lithotripsy preparation instruction to the local puncture device via the cloud server to control the local puncture device to place the lithotripsy device into the puncture channel. Specifically, the local puncture device needs to control the movement of the mechanical arm to grab the lithotripsy device, and then place the lithotripsy device into the expanded puncture channel. The specific process of the local puncture device controlling the movement of the mechanical arm to grab the lithotripsy device can be referred to the description of the specific process of the local puncture device controlling the movement of the mechanical arm to grab the tool in the aforementioned embodiment, which will not be repeated in this embodiment. Figure 8 A schematic diagram of placing a lithotripsy device into a puncture channel provided in an embodiment of the present application. Figure 8 As shown, the lithotripsy device enters the target tissue through a puncture channel through the subject's skin and is directed toward the target site.

[0098] S202: If it is determined through the information displayed on the target image that the stone is within the stone crushing range of the lithotripsy device, a lithotripsy instruction is sent to the local puncture device to control the local puncture device to trigger the lithotripsy device to crush the stone.

[0099] The target image is at least one of the following: a first image acquired by a lithotripsy device, a second image acquired by an endoscope introduced into the target tissue, and an ultrasound angiography image.

[0100] Exemplarily, the lithotripsy device includes a camera. When the lithotripsy device is placed in the puncture channel, the remote control device acquires the first image captured by the lithotripsy device in real time. Medical personnel in other places view the target tissue and the stones in the target tissue through the remote control device to determine whether the stones are located within the lithotripsy range of the lithotripsy device. The remote control device acquires the first image in real time in any of the following ways. First, the lithotripsy device is connected to a cloud server and communicates with the cloud server through a network communication protocol such as TCP / IP. The cloud server transfers the received first image to realize communication between the lithotripsy device and the remote control device. Second, the lithotripsy device is connected to a local puncture device, and the first image is sent to the local puncture device, and then the communication between the remote control device and the local puncture device is realized through the cloud server to realize communication between the remote control device and the lithotripsy device. The lithotripsy device is such as a laser device. Exemplarily, the local assistant at the surgical site introduces the endoscope into the target tissue of the object, such as Figure 8As shown, a second image is taken, and the second image is a soft endoscope image. The second image shows the specific situation of the target tissue and the stones in the target tissue. The remote control device acquires the second image collected by the endoscope in real time, and the medical staff in another place view the target tissue and the stones in the target tissue through the remote control device to determine whether the stones are within the stone crushing range of the lithotripsy device. Exemplarily, the medical staff in another place view the ultrasound angiography image through the remote control device to view the target tissue and the stones in the target tissue to determine whether the stones are within the stone crushing range of the lithotripsy device. It should be understood that the medical staff in another place can view one or more images of the first image, the second image and the ultrasound angiography image through the remote control device to determine whether the stones are within the stone crushing range of the lithotripsy device.

[0101] Among them, the medical staff at other places can judge whether the stone is within the stone crushing range of the stone crushing device through the information displayed by the target image, and then the remote control device sends a stone crushing instruction to the local puncture device in response to the touch operation of the medical staff at other places. The remote control device can also judge whether the stone is within the stone crushing range of the stone crushing device through the information displayed by the target image; if the information displayed by the target image determines that the stone is within the stone crushing range of the stone crushing device, a stone crushing instruction is sent to the local puncture device to control the local puncture device to trigger the stone crushing device to crush the stone. Specifically, the remote control device converts the stone crushing range of the stone crushing device in the actual space into the position range in the target image according to the conversion relationship between the spatial coordinate system in the actual space and the pixel coordinate system in the target image. Among them, the stone crushing range includes the angle range of stone crushing and the distance range of stone crushing at each angle. For example, in the actual space, with the laser device emitting the laser hole and the extension line of the emission pipe as the reference, the angle range of the stone that can be crushed is plus or minus 10 degrees from the extension line, the farthest distance that can be crushed at 0 degrees is 20 mm, and the farthest distance that can be crushed at 10 degrees is 5 mm. Furthermore, the remote control device marks the position range in the target image. In the case where the medical staff in another place judges whether the stone is within the stone crushing range of the stone crushing device, the medical staff in another place can judge whether the stone is within the position range based on the stone, the stone crushing device and the position range of the stone crushing device displayed in the target image, so as to judge whether the stone is within the stone crushing range of the stone crushing device. In the case where the stone is within the position range, it is determined that the stone is within the stone crushing range of the stone crushing device. In the case where the remote control device judges whether the stone is within the stone crushing range of the stone crushing device, the remote control device judges whether the pixel coordinates of the stone in the target image are within the position range of the stone crushing device, so as to judge whether the stone is within the stone crushing range of the stone crushing device. When the pixel coordinates of the stone are within the position range, it is determined that the stone is within the lithotripsy range of the lithotripsy device.

[0102] The remote control device sends a lithotripsy instruction to the local puncture device via the cloud server to control the local puncture device to trigger the lithotripsy device to crush the stone. Specifically, the local puncture device triggers the lithotripsy device to crush the stone by the local puncture device controlling the mechanical arm to trigger the launch button of the lithotripsy device to cause the lithotripsy device to crush the stone. Exemplarily, when the lithotripsy device establishes communication with the remote control device via the cloud server, the remote control device may also send a lithotripsy instruction to the lithotripsy device to control the lithotripsy device to crush the stone. Figure 8 The figure also shows the state of the stone being broken.

[0103] For example, in step S202, if the information displayed by the target image determines that the stone is within the stone crushing range of the stone crushing device, a stone crushing instruction is sent to the local puncture device to control the local puncture device to trigger the stone crushing device to crush the stone. If the information displayed by the target image determines that the degree of stone crushing does not meet the preset degree, a stone crushing instruction is sent to the local puncture device to control the local puncture device to trigger the stone crushing device to crush the stone. That is, it is also possible to determine whether the stone crushing is completed through the target image. If the stone crushing is not completed, the remote control device continues to send the stone crushing instruction to the local puncture device to control the local puncture device to continue to trigger the stone crushing device to crush the stone.

[0104] In this embodiment, the local puncture device is controlled by a remote control device to control the lithotripsy device to break up the stones. This can also help medical personnel to control lithotripsy remotely without the need for medical personnel to control it locally during surgery, thereby improving the convenience of lithotripsy.

[0105] In one embodiment, in the above step S202, if it is determined through the information displayed in the target image that the stone is within the stone crushing range of the stone crushing device, a stone crushing instruction is sent to the local puncture device to control the local puncture device to trigger the stone crushing device to crush the stone, and the channel expansion control method further includes the following step S301:

[0106] When the local puncture device and lithotripsy device have completed lithotripsy and the drainage device has been introduced into the target tissue, a drainage instruction is sent to the local puncture device to control the local puncture device to trigger the drainage device to drain water, so that the gravel formed after the stone is crushed is discharged from the puncture channel.

[0107] Drainage equipment, such as equipment with a drainage pipe. During the puncture lithotripsy, in order to allow the crushed stones to be discharged from the subject's body, the drainage equipment is usually introduced into the target tissue. For example, the drainage pipe is introduced into the target tissue. When the local puncture equipment and lithotripsy equipment have completed the lithotripsy, and the drainage equipment has been introduced into the target tissue, the remote control device sends a drainage instruction to the local puncture equipment via the cloud server to control the local puncture equipment to trigger the drainage equipment to drain water. Fig. 9Schematic diagram of the discharge of gravel from the puncture channel provided in the embodiment of the present application. Fig. 9 As shown, under the impact of the water discharged by the drainage device, the gravel in the target tissue can be discharged from the puncture channel together with the water flow.

[0108] Exemplarily, after step S301, if it is determined that there are still residual gravel in the target tissue through the information displayed by the preset image, a drainage instruction is sent to the local puncture device to control the local puncture device to trigger the drainage device to drain water, so that the gravel formed after the stone is crushed is discharged from the puncture channel. That is, it can be determined whether all the gravel is discharged from the puncture channel through the preset image. In the case where there are still residual gravel in the target tissue, the remote control device continues to send a drainage instruction to the local puncture device to control the local puncture device to continue to trigger the drainage device to drain water. Among them, the preset image can be at least one of an ultrasound angiography image and a soft mirror image collected by an endoscope. In the case where the drainage device includes a camera, the preset image can also be an image collected by the drainage device. Similar to the way of obtaining the first image collected by the lithotripsy device, the remote control device can obtain the image collected by the drainage device through the cloud server and the local control device, and can also directly obtain the image collected by the drainage device through the cloud server. Medical personnel in other places can view the above one or more images through the remote control device to determine whether there are residual gravel in the target tissue. In the case where there are still residual lithotripsy in the target tissue, the remote control device responds to the touch operation of the remote medical staff and sends a drainage instruction to the local puncture device.

[0109] Exemplarily, when the drainage device establishes communication with the remote control device via the cloud server, the remote control device may also send a drainage instruction to the drainage device to control the drainage device to drain water.

[0110] In this embodiment, the local puncture device is controlled by the remote control device to control the drainage device to drain the water, so that the gravel is discharged from the puncture channel. It can also help medical staff to control the drainage and stone removal remotely, without the need for medical staff to control it locally during surgery, which can improve the convenience of drainage and stone removal.

[0111] In one embodiment, the channel expansion control method further includes the following steps S401 to S408. Among them, steps S401 to S404 and S406 correspond one-to-one to steps S101 to S103, S201, and S301 in the embodiment of the present application. The specific discussion of steps S401 to S404 and S406 in this embodiment can refer to the specific discussion of steps S101 to S103, S201, and S301 in the embodiment of the present application, and will not be repeated in this embodiment.

[0112] S401, determining a puncture path for puncturing the target point in an ultrasound angiography image showing the target point.

[0113] S402, controlling the local puncture device to perform puncture along the puncture path according to the position information of the puncture path in the ultrasound angiography.

[0114] S403, when the local puncture device completes the puncture, sending a channel expansion instruction to the local puncture device to control the local puncture device to expand the puncture channel.

[0115] The local puncture device includes a first mechanical arm and a second mechanical arm. When the local puncture device receives a channel expansion instruction, the first mechanical arm is controlled to grab a first tool for fixing the channel, and the second mechanical arm is controlled to grab a second tool for expanding the channel. The target point is located in the target tissue containing the stone.

[0116] S404, when the local puncture device completes the expansion of the puncture channel, a lithotripsy preparation instruction is sent to the local puncture device to control the local puncture device to place the lithotripsy device into the puncture channel.

[0117] S405, if the stone is determined to be within the stone crushing range of the stone crushing device through the information displayed by the target image, the pixel value range of the pixel point corresponding to the stone in the target image is determined, and a stone crushing instruction is sent to the local puncture device to control the local puncture device to trigger the stone crushing device to crush the stone. The target image is an ultrasound contrast image.

[0118] Regarding the specific description of "if it is determined through the information displayed by the target image that the stone is within the stone crushing range of the lithotripsy device, a stone crushing instruction is sent to the local puncture device to control the local puncture device to trigger the lithotripsy device to crush the stone", please refer to the specific description of step S202 in the aforementioned embodiment, which will not be repeated in this embodiment.

[0119] If the information displayed in the target image determines that the stone is within the stone crushing range of the lithotripsy device, the remote control device will not only send a stone crushing instruction to the local puncture device, but also determine the pixel value range of the pixel points corresponding to the stone in the target image. It can be understood that the stone corresponds to multiple pixels, and the pixel values ​​of each pixel point are different. Therefore, the remote control device determines the pixel value range of the pixel points corresponding to the stone. Pixel points whose pixel values ​​fall within the pixel value range can be considered as stones.

[0120] S406, when the local puncture device and lithotripsy device have completed lithotripsy and the drainage device has been introduced into the target tissue, a drainage instruction is sent to the local puncture device to control the local puncture device to trigger the drainage device to drain water, so that the gravel formed after the stone is crushed is discharged from the puncture channel.

[0121] S407: When there is a first target point in the target image whose pixel value falls within the pixel value range, determine that there is residual gravel in the target tissue.

[0122] After the drainage device drains water and the crushed stones are discharged from the puncture channel, if the pixel value of the first target point in the target image still belongs to the pixel value range, it indicates that the stone still exists in the target tissue. That is, the remote control device can determine that there are still residual stones in the target tissue.

[0123] S408, sending a stone removal instruction to the local puncture device to control the local puncture device to place the stone removal device into the puncture channel and remove the remaining gravel through the stone removal device.

[0124] When it is determined that there are still residual gravel in the target tissue, the remote control device sends a stone removal instruction to the local puncture device via the cloud server to control the local puncture device to place the stone removal device into the puncture channel and remove the residual gravel through the stone removal device. The stone removal device is such as stone removal forceps. Specifically, the local puncture device needs to control the movement of the mechanical arm to grab the stone removal device, and then place the stone removal device into the expanded puncture channel, and remove the residual gravel through the stone removal device. The specific process of the local puncture device controlling the movement of the mechanical arm to grab the stone removal device can be referred to the description of the specific process of the local puncture device controlling the movement of the mechanical arm to grab the tool in the aforementioned embodiment, which will not be repeated in this embodiment. Fig.10 One of the schematic diagrams of placing the stone removal device into the puncture channel and removing the residual gravel through the stone removal device provided in the embodiment of the present application. Fig.10 As shown, the local puncture device places the stone removal device into the puncture channel and removes the remaining stones through the stone removal device.

[0125] Exemplarily, when it is determined that there are still residual gravel in the target tissue, the remote control device sends a drainage instruction to the local puncture device to control the local puncture device to continue to trigger the drainage device to drain water, so that the residual gravel is discharged from the puncture channel.

[0126] Adopting the channel expansion control method in this embodiment, first, the local puncture device is controlled by a remote control device located in a different place to automatically puncture, automatically expand the puncture channel, crush, drain, and remove stones according to the puncture path, thereby realizing remote control of puncture and crushing and stone removal, and intelligent puncture and crushing and stone removal. There is no need for medical personnel to control the local operation of the puncture, allowing medical personnel to perform remote control operations in a different place throughout the whole process, which can improve the convenience of puncture and crushing. Secondly, when the remote control device segments the ultrasound angiography image, obtains and marks the vascular area in the ultrasound angiography image; marks the puncture safety area in the ultrasound angiography image, and the puncture safety area does not include the vascular area; determines the puncture path for puncturing the target target in the puncture safety area, then this embodiment also has the following beneficial effects. The remote control device can also accurately determine the vascular area and make the vascular area retained and displayed, as well as separate the puncture safety area and plan the puncture path based on the vascular area, which is conducive to reducing the difficulty of puncture, increasing the accuracy of puncture, reducing the risk of puncture, and making the puncture safer and more stable. Finally, the channel expansion control method in this embodiment is used to perform puncture lithotripsy under contrast ultrasound. From the determination of the vascular area to the puncture lithotripsy and stone removal, the remote control device controls the local puncture device to operate automatically, thereby realizing intelligent mechanization of the entire surgical process.

[0127] In one of the embodiments, the above step S408, sending a stone removal instruction to the local puncture device to control the local puncture device to place the stone removal device into the puncture channel and remove the residual gravel through the stone removal device, includes the following steps S501 to S503.

[0128] S501, sending a stone removal instruction to the local puncture device to control the local puncture device to place the stone removal device into the puncture channel along the central axis of the puncture channel. The starting point of the stone removal device into the puncture channel is flush with the end point of the puncture channel.

[0129] Fig.11 The second schematic diagram of placing the stone removal device into the puncture channel and removing the residual gravel through the stone removal device provided in the embodiment of the present application. Fig.11 As shown, after receiving the stone removal instruction, the local puncture device responds to the stone removal instruction and controls the mechanical arm to grab the stone removal device and place it in the puncture channel along the central axis of the puncture channel. The starting point of the central axis is set as A and the end point is set as B. The local puncture device places the stone removal device in the puncture channel in the direction from A to B. Among them, the starting point of the stone removal device placed in the puncture channel is flush with the end point of the puncture channel.

[0130] S502, according to the position information of the central axis and the residual gravel in the target image, determine the rotation angle and the depth of the stone removal device in the actual space. The depth of the stone removal refers to the distance that the stone removal device needs to move forward after rotation.

[0131] Since the residual gravel is not necessarily on the extension line of the central axis of the puncture channel, after the stone removal device is placed into the puncture channel along the central axis, it needs to be rotated slightly to align with the residual gravel, and it also needs to move forward after the rotation to penetrate into the target tissue to contact the residual gravel. Based on this, after the remote control device controls the local puncture device to place the stone removal device into the puncture channel along the central axis of the puncture channel, it also determines the rotation angle and stone removal depth of the stone removal device in the actual space according to the position information of the central axis and the residual gravel in the target image.

[0132] Specifically, determine the pixel coordinates of the starting point A, the end point B and the center point C of the residual gravel of the central axis in the target image. Then, according to the conversion relationship between the pixel coordinate system of the target image and the spatial coordinate system in the actual space, determine the spatial coordinates of the three points A, B and C in the actual space. Furthermore, based on the spatial coordinates of the three points A, B and C, determine the size of the angle a between the line segment AB and the line segment AC in the actual space, as well as the length of the line segment AC. It can be understood that in order to be able to align with the residual gravel, the size of the angle a is the angle that the stone removal equipment needs to rotate. That is, the angle a is the angle to be rotated. Fig.12 The third schematic diagram of placing the stone removal device into the puncture channel and removing the residual gravel through the stone removal device is provided in the embodiment of the present application. Fig.12 As shown, when the stone removal device is rotated with A as the rotation point and angle a as the angle to be rotated, the stone removal device rotates to the position of line segment AD, wherein the length of line segment AD is equal to the length of the central axis AB. When the length of line segment AC is determined and the length of the central axis AB of the puncture channel is known, the length of line segment DC can be determined. It can be understood that in order to penetrate into the target tissue and contact the residual gravel, the length of DC is the distance that the stone removal device needs to move forward after rotation, that is, the length of DC is the depth to be removed.

[0133] It should be noted that the puncture channel is the channel formed by puncture, and the puncture path is planned based on the position of the target point, and the distance between the target point and the stone is usually small. Therefore, the stone is usually located around the extension line of the central axis of the puncture channel. The gravel formed after the stone is crushed is usually also located around the extension line of the central axis of the puncture channel. When the puncture channel is fixed, in actual space, the angle a formed by the center point C of the residual gravel and the starting point A and end point B of the central axis is usually small. It is allowed for the stone removal equipment to rotate at an angle a in the puncture channel.

[0134] S503, sending the desired rotation angle and the desired stone removal depth to the local puncture device to control the local puncture device to carry the stone removal device to move with the desired rotation angle and the desired stone removal depth as the target, and controlling the local puncture device to remove the remaining gravel through the stone removal device.

[0135] After determining the angle to be rotated and the depth to be removed, the remote control device sends the angle to be rotated and the depth to be removed to the local puncture device via the cloud server to control the local puncture device to carry the stone removal device to move with the angle to be rotated and the depth to be removed as the target. The local puncture device controls the mechanical arm to carry the stone removal device to move, so that the stone removal device rotates with A as the rotation point and the angle to be rotated. And the rotated stone removal device moves forward to the depth to be removed to contact the residual gravel in the target tissue. Then, the local puncture device controls the mechanical arm to remove the residual gravel through the stone removal device.

[0136] In this embodiment, a remote control device located in a different place controls the local puncture device to automatically remove stones, thereby realizing remote control of stone removal and intelligent stone removal. Medical personnel are not required to perform local control during the puncture and lithotripsy operation, allowing medical personnel to perform remote control of the operation in a different place throughout the entire process, which can improve the convenience of stone removal. In addition, after determining the rotation angle and stone removal depth of the stone removal device, the remote control device controls the local puncture device to intelligently remove stones, which is conducive to the rapid and accurate removal of residual gravel, and can improve the accuracy and efficiency of the puncture and lithotripsy operation.

[0137] In one embodiment, an ultrasound probe for taking ultrasound contrast images is fixed on the local puncture device, and the local puncture device is used to grab the puncture needle for puncture. The above step S102, according to the position information of the puncture path in the ultrasound contrast image, controls the local puncture device to perform puncture along the puncture path, including the following steps S601 to S604.

[0138] S601, determining the first pixel coordinates of the puncture starting point of the puncture path in the ultrasound contrast image, the second pixel coordinates of the puncture end point of the puncture path in the ultrasound contrast image, and the vertex pixel coordinates of the second target point at the top in the ultrasound contrast image. The second target point corresponds to the target position in the ultrasound probe.

[0139] Among them, the puncture path is the path between the needle insertion point on the object's skin and the object's target point. Based on this, the puncture starting point corresponds to the needle insertion point on the object's skin, and the puncture end point corresponds to the target point. The remote control device determines the position information of the puncture path in the ultrasound contrast image, specifically, determines the first pixel coordinates of the puncture starting point in the ultrasound contrast image and the second pixel coordinates of the puncture end point in the ultrasound contrast image. In addition, the remote control device also determines the vertex pixel coordinates of the second target point at the top of the ultrasound contrast image. The second target point can be understood as the tip point of the object's body information displayed in the ultrasound contrast image. Fig.13 This is an ultrasound angiography image provided in the embodiment of the present application. Fig.13 As shown, Fig.13 The tip point shown refers to the second topmost target point in the ultrasound contrast image. Fig.14Schematic diagram of an ultrasound probe provided in an embodiment of the present application. Based on the principle of ultrasound probe collecting ultrasound images, the target position of the ultrasound probe corresponding to the second target point in the ultrasound angiography is as follows: Fig.14 shown. Fig.14 The actual position corresponding to the tip point shown refers to the target position of the above-mentioned ultrasound probe.

[0140] It should be noted that the origin, x-axis, and y-axis of the pixel coordinate system in the ultrasound angiography can usually be determined according to actual needs. Fig.13 Take the x-axis and y-axis shown in the figure as an example. The horizontal rightward direction is the positive direction of the x-axis, and the vertical upward direction is the positive direction of the y-axis. The intersection of the x-axis and the y-axis is the origin of the pixel coordinate system. When the above pixel coordinate system is set, a search algorithm is used to search for all pixel points in the ultrasound trace image. The pixel point with the largest y is the tip point. Optionally, to reduce the amount of searching, only the query can be performed. Fig.13 All pixels within the dashed box shown are used to more quickly determine the tip point.

[0141] S602, using the conversion relationship between the pixel coordinate system of the ultrasound angiography image and the spatial coordinate system of the actual space, convert the first pixel coordinate into the first spatial coordinate, convert the second pixel coordinate into the second spatial coordinate, and convert the vertex pixel coordinate into the vertex spatial coordinate.

[0142] The conversion relationship between the pixel coordinate system of the ultrasound image and the spatial coordinate system of the actual space can be preset in the remote control device, and the ultrasound image spatial calibration can also be performed by the remote control device. The ultrasound image spatial calibration specifically includes first determining the pixel coordinates of several calibration points in the ultrasound image, and then determining the conversion relationship between the pixel coordinate system and the spatial coordinate system when the spatial coordinates of several calibration points in the ultrasound image corresponding to the actual space are known.

[0143] By utilizing the conversion relationship between the pixel coordinate system of the ultrasound angiography and the spatial coordinate system of the actual space, the remote control device can convert the first pixel coordinate into the first spatial coordinate, the second pixel coordinate into the second spatial coordinate, and the vertex pixel coordinate into the vertex spatial coordinate. Fig.15 A schematic diagram of a partial structure of a local puncture device provided in an embodiment of the present application. That is, the remote control device can determine in the actual space, such as Fig.15 The first spatial coordinates of the needle insertion point, the second spatial coordinates of the target point, and Fig.14 The actual position corresponding to the tip point shown, that is, the target position of the ultrasound probe, is the vertex spatial coordinate. It can be understood that the settings of the origin, x-axis, and y-axis in the spatial coordinate system should correspond to the settings of the origin, x-axis, and y-axis in the pixel coordinate system. Fig.13In the case of the x-axis and y-axis of the pixel coordinate system shown in FIG. Fig.15 The x-axis and y-axis of the spatial coordinate system shown have the positive direction of the x-axis pointing horizontally to the right and the positive direction of the y-axis pointing vertically upward.

[0144] S603, using the positional relationship between the target position in the ultrasound probe and the puncture needle captured by the local puncture device, based on the vertex spatial coordinates, determine the tip spatial coordinates and the end spatial coordinates of the puncture needle.

[0145] Specifically, according to the structural characteristics of the local puncture device, the positional relationship between the target position in the ultrasound probe and the puncture needle grasped by the local puncture device can be known. The remote control device can obtain the positional relationship between the target position in the ultrasound probe and the puncture needle grasped by the local puncture device from the local puncture device via the cloud server. Fig.15 As shown in FIG. 1 , the robot arm is set to grab the launch device for launching the puncture needle. The length of the puncture needle is consistent with the length of the launch device, and the puncture needle is located at the central axis of the launch device. Based on this, the tip and end of the puncture needle are as shown in FIG. Fig.15 The positional relationship between the target position and the puncture needle includes the distance between the target position and the tip, the distance between the target position and the end, the angle between the line segment formed by the target position and the end and the puncture needle, the angle between the line segment formed by the target position and the tip and the puncture needle, the length of the puncture needle, etc.

[0146] Furthermore, the remote control device uses the above positional relationship to determine the spatial coordinates of the tip of the puncture needle and the spatial coordinates of the end of the puncture needle in the actual space when the vertex spatial coordinates of the target position are known. The above tip spatial coordinates refer to the spatial coordinates of the tip; the above end spatial coordinates refer to the spatial coordinates of the end.

[0147] S604, sending the spatial coordinate information to the local puncture device to control the local puncture device to perform puncture according to the spatial coordinate information. The spatial coordinate information includes the first spatial coordinate, the second spatial coordinate, the tip spatial coordinate, and the end spatial coordinate.

[0148] Furthermore, the remote control device will be Fig.15The spatial coordinate information such as the first spatial coordinate of the needle insertion point, the second spatial coordinate of the target point, the spatial coordinate of the tip of the puncture needle and the spatial coordinate of the end are sent to the local puncture device. When the local puncture device receives the spatial coordinate information, it locates the puncture channel and automatically punctures the needle based on the spatial coordinate information. Among them, puncture channel positioning refers to calculating the puncture angle between the puncture needle and the puncture path and the puncture depth to be punctured of the puncture needle. The depth to be punctured refers to the depth that the puncture needle needs to move forward from the needle insertion point to contact the target target. The depth to be punctured can also be understood as the distance between the needle insertion point and the target target.

[0149] Optionally, the remote control device sends spatial coordinate information to the local puncture device, and in response to the touch operation of the remote medical staff on the "Locate puncture channel" control and the "Automatic puncture needle insertion" control in the "Locate puncture" column in the interactive interface, remotely controls the local puncture device to locate the puncture channel and automatically puncture the needle based on the spatial coordinate information. Figure 4 shown.

[0150] Exemplarily, the specific process of the local puncture device positioning the puncture channel based on the spatial coordinate information is as follows. The local puncture device automatically calculates the puncture angle formed between the puncture path and the puncture needle according to the spatial coordinates of the needle entry point, the target point, the tip and the end of the puncture needle in the actual space. According to the spatial coordinates of the needle entry point and the target point in the actual space, the depth of the puncture needle to be punctured is automatically calculated. Then, the local puncture device automatically punctures and inserts the needle according to the puncture angle and the depth to be punctured. The specific process of automatic puncture and insertion of the needle by the local puncture device is as follows. First, the local puncture device takes the puncture angle as the target, controls the mechanical arm to carry the puncture needle to rotate, so that the angle between the puncture needle and the puncture path is 0 degrees, and the tip of the puncture needle is close to the puncture path, and the end of the puncture needle is away from the puncture path. Secondly, the local puncture device controls the mechanical arm to carry the puncture needle to translate, so that the tip of the puncture needle coincides with the needle entry point, and the angle between the puncture needle and the puncture path is maintained at 0 degrees. Finally, the local puncture device controls the robotic arm to move the puncture needle forward based on the desired puncture depth to achieve needle insertion and puncture.

[0151] Optionally, a control button is added to the local puncture device and / or the remote control device. When the local puncture device controls the mechanical arm to move the puncture needle forward to insert the needle with the puncture depth as the target, the local assistant and / or the remote medical staff can touch the control button to stop the needle insertion. This design is conducive to ensuring the safety of needle insertion.

[0152] Optionally, a warning area is provided in the interactive interface. Figure 4As shown, in the process of controlling the local puncture device to automatically puncture and insert the needle, if the puncture needle deviates from the puncture path during the insertion process, the warning area displays text information such as "Safety Tip: Puncture Needle Deviates" to prompt medical personnel in other places to intervene. In the case where a control button for stopping the needle insertion is set in the remote control device, medical personnel in other places can control the stop of the needle insertion by touching the control button. In addition, the warning area can also display information such as the puncture depth of the puncture needle for reference by medical personnel in other places, such as displaying "Puncture Depth: 100.5mm".

[0153] In this embodiment, after the remote control device plans a safe puncture path in the ultrasound angiography, it determines the position information of the puncture path in the ultrasound angiography based on the pixel coordinate system corresponding to the ultrasound angiography. Furthermore, according to the position information of the puncture path in the ultrasound angiography, the local puncture device is controlled to locate the puncture channel and automatically puncture the needle according to the puncture path. The channel expansion control method in this embodiment can realize that the remote control device controls the local puncture device to perform automatic puncture remotely, without the need for medical personnel to perform puncture locally during surgery, which can improve the convenience of puncture. In addition, the fully intelligent mechanized calculation of data such as position information, puncture angle, depth to be punctured, and the realization of automatic machine puncture are conducive to reducing the difficulty of puncture, increasing puncture accuracy, and reducing puncture risks.

[0154] In one embodiment, the above step S604, sending the spatial coordinate information to the local puncture device to control the local puncture device to perform puncture according to the spatial coordinate information, includes the following steps S701 to S702.

[0155] S701, calculating the puncture angle between the puncture needle and the puncture path and the puncture depth to be punctured of the puncture needle in the actual space according to the spatial coordinate information.

[0156] After determining the spatial coordinate information, the remote control device automatically calculates the puncture angle formed between the puncture path and the puncture needle based on the spatial coordinates of the needle insertion point, target point, tip and end of the puncture needle in actual space; and automatically calculates the puncture depth of the puncture needle based on the spatial coordinates of the needle insertion point and the target point in actual space.

[0157] S702: Send the puncture angle and the depth to be punctured to the local puncture device to control the local puncture device to perform puncture according to the puncture angle and the depth to be punctured.

[0158] Then, the remote control device sends the puncture angle and the depth to be punctured to the local puncture device via the cloud server. When the local puncture device receives the puncture angle and the depth to be punctured, it automatically punctures and inserts the needle based on the puncture angle and the depth to be punctured. The specific process of the local puncture device automatically puncturing and inserting the needle can be referred to the description in the above embodiment, and will not be repeated in this embodiment.

[0159] Optionally, the remote control device sends the puncture angle and the depth to be punctured to the local puncture device, and in response to the touch operation of the remote medical staff on the "Automatic puncture and needle insertion" control in the "positioning puncture" column in the interactive interface, remotely controls the local puncture device to automatically puncture the needle based on the puncture angle and the depth to be punctured. Figure 4 As shown. Optionally, a control button is added to the local puncture device and / or the remote control device. Optionally, a warning area is provided in the interactive interface. For the specific description of adding a control button and providing a warning area in the interactive interface, reference can be made to the description in the aforementioned embodiment, which will not be repeated in this embodiment.

[0160] In this embodiment, after determining the spatial coordinates of the insertion point, target point, tip and end of the puncture needle in the actual space, the remote control device further determines the puncture angle between the puncture needle and the puncture path and the depth of the puncture needle to be punctured in the actual space. Then, according to the puncture angle and the depth to be punctured, the local puncture device is controlled to automatically puncture and insert the needle. The channel expansion control method in this embodiment enables the remote control device to remotely control the local puncture device for automatic puncture, without the need for medical personnel to perform puncture locally, which can improve the convenience of puncture. In addition, the fully intelligent mechanized calculation of data such as position information, puncture angle, depth to be punctured, and the realization of automatic machine puncture are conducive to reducing the difficulty of puncture, increasing puncture accuracy, and reducing puncture risks.

[0161] In one embodiment, the local puncture device includes a third mechanical arm, and an ultrasound probe is fixed on the third mechanical arm. In the above step S101, before determining the puncture path for puncturing the target target in the ultrasound angiography image showing the target target, the channel expansion control method also includes the following steps S801 to S803.

[0162] S801, acquiring an ultrasound image captured by an ultrasound probe.

[0163] Before determining the puncture path in the ultrasound contrast image, the remote control device needs to first obtain the ultrasound contrast image showing the target point. Before obtaining the ultrasound contrast image, it is necessary to first obtain an ultrasound image showing the target point.

[0164] Specifically, the ultrasound probe fixed on the third mechanical arm of the local puncture device takes real-time photos of the target tissue of the object to collect ultrasound images, and transmits the ultrasound images to the local puncture device. The local puncture device displays the ultrasound images on the first display. The remote control sensor remotely controls the local puncture device through the remote control software, and displays the ultrasound images displayed in the local puncture device on its own second display through the remote control software. Based on this, the remote control software obtains the ultrasound images taken by the ultrasound probe in real time.

[0165] S802, sending the posture data to the local puncture device to control the third mechanical arm fixed with the ultrasound probe to move with the posture data as the target. When the third mechanical arm moves with the posture data as the target, the target point of the object is displayed in the ultrasound image.

[0166] Furthermore, the medical staff in another place view the ultrasound image and the CT three-dimensional model diagram. When the endoscope has been introduced into the target tissue of the object, the medical staff in another place can also view the soft endoscope image taken by the endoscope. The medical staff in another place determine the positional relationship between the target target and the ultrasound probe through the information in the object body displayed by the ultrasound image, the overall situation of the target tissue and the target target in the target tissue displayed by the CT three-dimensional model diagram, and the specific situation of the target tissue and the target target in the target tissue displayed by the soft endoscope diagram. Furthermore, based on the positional relationship between the target target and the ultrasound probe, the medical staff in another place manipulates the force feedback device, and the force feedback device generates posture data, which is sent to the local puncture device via the cloud server. The local puncture device controls the third mechanical arm fixed with the ultrasound probe to move with the posture data as the target. In this way, the movement of the third mechanical arm fixed with the ultrasound probe is controlled. When the third mechanical arm moves with the posture data as the target, the target target in the target tissue of the object is displayed in the ultrasound image. It should be noted that the off-site medical staff manipulates the force feedback device to make the third robotic arm, which is fixed with the ultrasound probe, move with the posture data as the target until the target point of the object is displayed in the ultrasound image. This is a continuous control process.

[0167] Based on this, under the control of the medical staff in other places and the remote control device, the local puncture device controls the ultrasound probe to capture the ultrasound image showing the target target, and the remote control device also obtains the ultrasound image showing the target target in real time. The third mechanical arm of the local puncture device maintains the current state, so that the ultrasound probe maintains the current state to continuously display the currently captured ultrasound image showing the target target. The currently captured ultrasound image can be used to determine the puncture path, etc.

[0168] It can be understood that if the target point of the object is displayed in the ultrasound image, the ultrasound probe is maintained in the current state to continuously display the currently captured ultrasound image. The ultrasound image obtained at this time can be used to determine the puncture path, etc. In order to determine the puncture path more clearly and efficiently, and to make the puncture process safer, medical personnel in other places can continue to manipulate the force feedback device to control the third mechanical arm to carry the ultrasound probe to continue to move. Until the ultrasound image collected by the ultrasound probe is clear, and in addition to showing the target point, the ultrasound image also shows less information. At this time, the third mechanical arm and ultrasound probe of the local puncture device are maintained in the current state to continuously display the currently captured ultrasound image. The currently captured ultrasound image shows less information and the image is clear, indicating that the ultrasound image presents the best puncture surface of the target tissue. Using this best puncture surface to determine the puncture path and puncture the target point, there are more puncture paths to choose from, and it can avoid the interference of too many blood vessels and organ tissues on the puncture, which is conducive to improving the safety of the puncture.

[0169] S803, when the subject is intravenously injected with ultrasound contrast agent, obtain an ultrasound contrast image based on the ultrasound image. The ultrasound contrast image is an image after ultrasound image contrast development.

[0170] Furthermore, when the subject is intravenously injected with ultrasound contrast agent, based on the characteristics of ultrasound image contrast development, the remote control device obtains an ultrasound contrast image showing the target point in real time.

[0171] For example, Figure 4 In the interactive interface shown, there is an "image processing" column. The remote medical staff touches the "image processing" column to display the controls of the "image processing" column on the interactive interface. The remote medical staff can adjust the display effects of ultrasound images and ultrasound contrast images by touching the controls of the "image processing" column. Display effects such as brightness and contrast.

[0172] In this embodiment, the movement of the third mechanical arm fixed with the ultrasonic probe is controlled by a remote control device so that the ultrasonic probe can collect an ultrasonic image showing the target target. Then, the third mechanical arm and the ultrasonic probe are maintained in the current state to continuously display the currently captured ultrasonic image showing the target target. The currently captured ultrasonic image can be used to determine the puncture path, etc. Based on this, under the control of the machine, the ultrasonic probe and the ultrasonic image are more stable, which is conducive to improving the safety of the puncture. This process realizes the determination of the puncture surface and the fixation of the ultrasonic probe by the machine, without the need for local medical staff to determine the puncture surface and manually fix the ultrasonic probe based on experience, which is conducive to improving the accuracy of puncture, reducing the difficulty of puncture, and improving the convenience of surgery.

[0173] The embodiment of the present application also provides a channel expansion control device. Fig.16This is a schematic diagram of the structure of the channel expansion control device provided in the embodiment of the present application. Fig.16 As shown, the channel expansion control device 1600 is used to remotely control the local puncture device to expand the puncture channel, and the puncture channel refers to the channel formed by puncture. The channel expansion control device 1600 includes a path determination module 1601, a puncture control module 1602 and a channel expansion module 1603. The path determination module 1601 is used to determine the puncture path for puncturing the target target in the ultrasound angiography image showing the target target. The puncture control module 1602 is used to control the local puncture device to puncture according to the puncture path according to the position information of the puncture path in the ultrasound angiography image. The channel expansion module 1603 is used to send a channel expansion instruction to the local puncture device when the local puncture device completes the puncture, so as to control the local puncture device to expand the puncture channel. Among them, the local puncture device includes a first mechanical arm and a second mechanical arm. When the local puncture device receives the channel expansion instruction, the first mechanical arm is controlled to grab the first tool for fixing the channel, and the second mechanical arm is controlled to grab the second tool for expanding the channel.

[0174] In one embodiment, the target target point is located in the target tissue containing the stone. The above-mentioned channel expansion control device 1600 also includes a lithotripsy control module, which is used to send a lithotripsy preparation instruction to the local puncture device when the local puncture device completes the expansion of the puncture channel, so as to control the local puncture device to place the lithotripsy device into the puncture channel; if the stone is determined to be within the lithotripsy range of the lithotripsy device through the information displayed by the target image, a lithotripsy instruction is sent to the local puncture device to control the local puncture device to trigger the lithotripsy device to crush the stone; wherein the target image is at least one of the following: a first image collected by the lithotripsy device, a second image collected by the endoscope introduced into the target tissue, and an ultrasound angiogram.

[0175] In one embodiment, the channel expansion control device 1600 further includes a drainage control module, which is used to send a drainage instruction to the local puncture device when the local puncture device and the lithotripsy device have completed lithotripsy and the drainage device has been introduced into the target tissue, so as to control the local puncture device to trigger the drainage device to drain water, so that the gravel formed after the stone is crushed is discharged from the puncture channel.

[0176] In one embodiment, when the target image is an ultrasound angiogram, the lithotripsy control module is further used to determine the pixel value range of the pixel points corresponding to the stones in the target image. The channel expansion control device 1600 also includes a lithotripsy control module, which is used to determine the presence of residual lithotripsy in the target tissue when the pixel value of the first target point in the target image belongs to the pixel value range; send a lithotripsy instruction to the local puncture device to control the local puncture device to place the lithotripsy device into the puncture channel and remove the residual lithotripsy through the lithotripsy device.

[0177] In one of the embodiments, the stone removal control module is also used to send a stone removal instruction to the local puncture device to control the local puncture device to place the stone removal device into the puncture channel along the central axis of the puncture channel, and the starting point of the stone removal device into the puncture channel is flush with the end point of the puncture channel; according to the position information of the central axis and the residual gravel in the target image, the rotation angle and the stone removal depth of the stone removal device in the actual space are determined, and the stone removal depth refers to the distance the stone removal device needs to move forward after rotation; the rotation angle and the stone removal depth are sent to the local puncture device to control the local puncture device to carry the stone removal device to move with the rotation angle and the stone removal depth as the target, and control the local puncture device to remove the residual gravel through the stone removal device.

[0178] In one embodiment, an ultrasonic probe for taking ultrasonic contrast images is fixed on the local puncture device, and the local puncture device is used to grab the puncture needle for puncture. The puncture control module 1602 is also used to determine the first pixel coordinates of the puncture starting point of the puncture path in the ultrasonic contrast image, the second pixel coordinates of the puncture end point of the puncture path in the ultrasonic contrast image, and the vertex pixel coordinates of the second target point at the top of the ultrasonic contrast image, and the second target point corresponds to the target position in the ultrasonic probe; using the conversion relationship between the pixel coordinate system of the ultrasonic contrast image and the spatial coordinate system of the actual space, the first pixel coordinate is converted into the first spatial coordinate, the second pixel coordinate is converted into the second spatial coordinate, and the vertex pixel coordinate is converted into the vertex spatial coordinate; using the positional relationship between the target position in the ultrasonic probe and the puncture needle grabbed by the local puncture device, based on the vertex spatial coordinates, determine the tip spatial coordinates and the end spatial coordinates of the puncture needle; send spatial coordinate information to the local puncture device to control the local puncture device to perform puncture according to the spatial coordinate information, and the spatial coordinate information includes the first spatial coordinate, the second spatial coordinate, the tip spatial coordinate, and the end spatial coordinate.

[0179] In one embodiment, the puncture control module 1602 is further used to calculate the puncture angle between the puncture needle and the puncture path and the puncture depth of the puncture needle in the actual space according to the spatial coordinate information; and send the puncture angle and the puncture depth to be punctured to the local puncture device to control the local puncture device to perform puncture according to the puncture angle and the puncture depth to be punctured.

[0180] In one embodiment, the local puncture device includes a third mechanical arm, on which an ultrasonic probe is fixed. The channel expansion control device 1600 also includes a contrast image acquisition module, which is used to acquire an ultrasonic image taken by the ultrasonic probe; send posture data to the local puncture device to control the third mechanical arm fixed with the ultrasonic probe to move with the posture data as the target, and when the third mechanical arm moves with the posture data as the target, the target point of the object is displayed in the ultrasonic image; when the object is injected with an ultrasonic contrast agent intravenously, an ultrasonic contrast image is obtained based on the ultrasonic image, and the ultrasonic contrast image is an image after the ultrasonic image is contrasted and developed.

[0181] For the explanation of each term, please refer to the relevant description in the aforementioned channel expansion control method embodiment, which will not be elaborated here.

[0182] It should be noted that the specific execution process of the channel expansion control device 1600 can be found in Figures 1 to 15 The specific description of the illustrated embodiment will not be repeated here.

[0183] Each module in the channel expansion control device 1600 can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a remote control device in the form of hardware, or can be stored in a memory in the remote control device in the form of software, so that the processor can call and execute operations corresponding to each module. The remote control device is a computer device.

[0184] The embodiment of the present application also provides a channel expansion control system. Fig.17 This is a schematic diagram of the structure of the channel expansion control system provided in the embodiment of the present application. Fig.17 As shown, the channel expansion control system 1700 is used to expand the puncture channel, and the puncture channel refers to the channel formed by puncture. The channel expansion control system 1700 includes a remote control device 1701 and a local puncture device 1702 connected to each other. Optionally, the remote control device 1701 and the local puncture device 1702 are connected to each other through a cloud server.

[0185] The remote control device 1701 is used to determine a puncture path for puncturing the target target in the ultrasound contrast image showing the target target; according to the position information of the puncture path in the ultrasound contrast image, control the local puncture device 1702 to puncture along the puncture path; when the local puncture device 1702 completes the puncture, send a channel expansion instruction to the local puncture device 1702 to control the local puncture device 1702 to expand the puncture channel;

[0186] The local puncture device 1702 is used to expand the puncture channel under the control of the remote control device 1701;

[0187] The local puncture device 1702 includes a first robotic arm and a second robotic arm. When the local puncture device 1702 receives a channel expansion instruction, the first robotic arm is controlled to grab a first tool for fixing the channel, and the second robotic arm is controlled to grab a second tool for expanding the channel.

[0188] The channel expansion control system 1700 is used to execute the channel expansion control method in the aforementioned embodiment through the remote control device 1701 to control the local puncture device 1702 to automatically expand the puncture channel, etc. For the specific discussion of the channel expansion control system 1700 in this embodiment, reference can be made to the relevant discussion on the channel expansion control method in the aforementioned embodiment, which will not be repeated in this embodiment.

[0189] The channel expansion control system 1700 in this embodiment has at least the following beneficial effects. First, the local puncture device 1702 is controlled by the remote control device 1701 located in a different place to automatically expand the puncture channel, thereby realizing remote control of the expansion puncture channel and intelligent expansion of the puncture channel. There is no need for medical personnel to expand the puncture channel locally during the puncture operation, which can improve the convenience of expanding the puncture channel. Secondly, the remote control device 1701 controls the local puncture device 1702 to automatically puncture according to the puncture path according to the position information of the puncture path in the ultrasound angiography, and there is no need for medical personnel to perform puncture locally during the operation, which can improve the convenience of puncture. Finally, the channel expansion control system 1700 in this embodiment is used to perform puncture lithotripsy under contrast ultrasound, and full intelligent mechanization can be achieved from the determination of the puncture path to the expansion of the puncture channel. For other beneficial effects of the channel expansion control system 1700 in this embodiment, reference can be made to the beneficial effects of the channel expansion control method described in the aforementioned embodiment, which will not be repeated in this embodiment.

[0190] The present application also provides a remote control device. Fig.18 This is a schematic diagram of the structure of the remote control device provided in the embodiment of the present application. Fig.18 As shown, the remote control device 1800 may include a processor 1801, a communication interface 1802, a communication bus 1803, and a memory 1804. Specifically, the remote control device 1800 may include:

[0191] At least one processor 1801, such as a CPU, at least one communication interface 1802, a memory 1804, and at least one communication bus 1803. The communication bus 1803 is used to realize the connection and communication between these components. The communication interface 1802 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface or a Bluetooth interface, etc.). The memory 1804 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 1804 may optionally also be at least one storage device located away from the aforementioned processor 1801. Fig.18 As shown, the memory 1804 as a computer storage medium may include an operating system and program instructions.

[0192] Exemplarily, the processor 1801 may be used to implement the above Fig.16 The steps or methods performed by the path determination module 1601, the puncture control module 1602 and the channel expansion module 1603.

[0193] It is understandable that the above method is only an example, and the processor 1801 and other modules in the above remote control device 1800 may also cooperate to perform the above method. Fig.16 The steps or methods executed by the path determination module 1601, the puncture control module 1602 and the channel expansion module 1603 are not limited in this document.

[0194] exist Fig.18 In the remote control device 1800 shown, the processor 1801 can be used to load the program instructions stored in the memory 1804 and specifically perform the following operations:

[0195] Determining a puncture path for puncturing the target point in the ultrasound contrast image showing the target point;

[0196] According to the position information of the puncture path in the ultrasound angiography, the local puncture device is controlled to perform puncture along the puncture path;

[0197] When the local puncture device completes the puncture, a channel expansion instruction is sent to the local puncture device to control the local puncture device to expand the puncture channel;

[0198] The local puncture device includes a first robotic arm and a second robotic arm. When the local puncture device receives a channel expansion instruction, the first robotic arm is controlled to grab a first tool for fixing the channel, and the second robotic arm is controlled to grab a second tool for expanding the channel.

[0199] For the explanation of each term, please refer to the relevant description in the aforementioned channel expansion control method embodiment, which will not be elaborated here.

[0200] It should be noted that the specific implementation process can be found in Figures 1 to 15 The specific description of the illustrated embodiment will not be repeated here.

[0201] The present application also provides a computer-readable storage medium, which can store a plurality of instructions, and the instructions are suitable for being loaded and executed by a processor as described above. Figures 1 to 15 The method steps of the embodiment shown in the figure can be found in the specific execution process. Figures 1 to 15 The specific description of the illustrated embodiment will not be repeated here.

[0202] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0203] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any fusion thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0204] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0205] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0206] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A channel expansion control device, characterized in that: The device is used for remotely controlling a local puncture device to expand a puncture channel, wherein the puncture channel refers to a channel formed by puncture; the device comprises: A path determination module, used to determine a puncture path for puncturing the target point in an ultrasound angiography image showing the target point; A puncture control module, used for controlling the local puncture device to perform puncture along the puncture path according to the position information of the puncture path in the ultrasound angiography image; A channel expansion module, used for sending a channel expansion instruction to the local puncture device when the local puncture device completes the puncture, so as to control the local puncture device to expand the puncture channel; Wherein, the local puncture device comprises a first mechanical arm and a second mechanical arm, and when the local puncture device receives the channel expansion instruction, the first mechanical arm is controlled to grasp the first tool for fixing the channel, and the second mechanical arm is controlled to grasp the second tool for expanding the channel; The local puncture device is fixed with an ultrasonic probe for photographing the ultrasonic angiogram, and the local puncture device is used to grab a puncture needle for puncture; the puncture control module is also used to determine the first pixel coordinates of the puncture start point of the puncture path in the ultrasonic angiogram, the second pixel coordinates of the puncture end point of the puncture path in the ultrasonic angiogram, and the vertex pixel coordinates of the second target point at the top of the ultrasonic angiogram, wherein the second target point corresponds to the target position in the ultrasonic probe; using the conversion relationship between the pixel coordinate system of the ultrasonic angiogram and the spatial coordinate system of the actual space, the first pixel coordinate is converted into the first spatial coordinate, the second pixel coordinate is converted into the second spatial coordinate, and the vertex pixel coordinate is converted into the vertex spatial coordinate; using the positional relationship between the target position in the ultrasonic probe and the puncture needle grabbed by the local puncture device, based on the vertex spatial coordinates, the tip spatial coordinates and the terminal spatial coordinates of the puncture needle are determined; sending spatial coordinate information to the local puncture device to control the local puncture device to perform puncture according to the spatial coordinate information, wherein the spatial coordinate information includes the first spatial coordinate, the second spatial coordinate, the tip spatial coordinate, and the terminal spatial coordinate.

2. The device according to claim 1, characterized in that The target point is located in a target tissue containing stones; the device further comprises: a lithotripsy control module, configured to send a lithotripsy preparation instruction to the local puncture device when the local puncture device completes the expansion of the puncture channel, so as to control the local puncture device to place the lithotripsy device into the puncture channel; and if it is determined through the information displayed on the target image that the stone is within the lithotripsy range of the lithotripsy device, send a lithotripsy instruction to the local puncture device, so as to control the local puncture device to trigger the lithotripsy device to crush the stone; Wherein, the target image is at least one of the following: a first image acquired by the lithotripsy device, a second image acquired by an endoscope introduced into the target tissue, and the ultrasound angiography image.

3. The device according to claim 2, characterized in that The device also includes: The drainage control module is used to send a drainage instruction to the local puncture device when the local puncture device and the lithotripsy device have completed lithotripsy and the drainage device has been introduced into the target tissue, so as to control the local puncture device to trigger the drainage device to drain water, so that the gravel formed after the stone is crushed is discharged from the puncture channel.

4. The device according to claim 3, characterized in that In the case where the target image is the ultrasound angiography image, the lithotripsy control module is further configured to determine the pixel value range of the pixel points corresponding to the stone in the target image if it is determined through information displayed by the target image that the stone is within the lithotripsy range of the lithotripsy device, and send a lithotripsy instruction to the local puncture device to control the local puncture device to trigger the lithotripsy device to crush the stone; the device further comprises: A stone removal control module is used to determine the presence of residual gravel in the target tissue when the pixel value of the first target point in the target image falls within the pixel value range; and send a stone removal instruction to the local puncture device to control the local puncture device to place the stone removal device into the puncture channel and remove the residual gravel through the stone removal device.

5. The device according to claim 4, characterized in that The stone removal control module is also used to send the stone removal instruction to the local puncture device to control the local puncture device to place the stone removal device into the puncture channel along the central axis of the puncture channel, and the starting point of the stone removal device into the puncture channel is flush with the end point of the puncture channel; according to the position information of the central axis and the residual gravel in the target image, the rotation angle and the stone removal depth of the stone removal device in the actual space are determined, and the stone removal depth refers to the distance that the stone removal device needs to move forward after rotation; The angle to be rotated and the depth to be removed are sent to the local puncture device to control the local puncture device to carry the stone removal device to move with the angle to be rotated and the depth to be removed as the target, and control the local puncture device to remove the residual gravel through the stone removal device.

6. The device according to claim 1, characterized in that The puncture control module is also used to calculate the puncture angle between the puncture needle and the puncture path and the puncture depth to be punctured of the puncture needle in the actual space according to the spatial coordinate information; and send the puncture angle and the puncture depth to be punctured to the local puncture device to control the local puncture device to perform puncture according to the puncture angle and the puncture depth to be punctured.

7. The device according to any one of claims 1 to 6, characterized in that The local puncture device comprises a third mechanical arm, on which an ultrasonic probe is fixed; the device also comprises: A contrast image acquisition module is used to acquire the ultrasound image taken by the ultrasound probe; send posture data to the local puncture device to control the third mechanical arm fixed with the ultrasound probe to move with the posture data as the target, when the third mechanical arm moves with the posture data as the target, the target point of the object is displayed in the ultrasound image; when the object is intravenously injected with ultrasound contrast agent, the ultrasound contrast image is obtained based on the ultrasound image, and the ultrasound contrast image is the image after contrast development of the ultrasound image.

8. A channel expansion control system, characterized in that: The system is used to expand a puncture channel, wherein the puncture channel refers to a channel formed by puncture; the system includes a remote control device and a local puncture device connected to each other; The remote control device is used to determine a puncture path for puncturing the target target in an ultrasound angiography image showing the target target; control the local puncture device to perform puncture along the puncture path according to the position information of the puncture path in the ultrasound angiography image; and send a channel expansion instruction to the local puncture device when the local puncture device completes the puncture, so as to control the local puncture device to expand the puncture channel; The local puncture device is used to expand the puncture channel under the control of the remote control device; Wherein, the local puncture device comprises a first mechanical arm and a second mechanical arm, and when the local puncture device receives the channel expansion instruction, the first mechanical arm is controlled to grasp the first tool for fixing the channel, and the second mechanical arm is controlled to grasp the second tool for expanding the channel; The local puncture device is fixed with an ultrasonic probe for photographing the ultrasonic angiogram, and the local puncture device is used to grab a puncture needle for puncture; the puncture control module is also used to determine the first pixel coordinates of the puncture start point of the puncture path in the ultrasonic angiogram, the second pixel coordinates of the puncture end point of the puncture path in the ultrasonic angiogram, and the vertex pixel coordinates of the second target point at the top of the ultrasonic angiogram, wherein the second target point corresponds to the target position in the ultrasonic probe; using the conversion relationship between the pixel coordinate system of the ultrasonic angiogram and the spatial coordinate system of the actual space, the first pixel coordinate is converted into the first spatial coordinate, the second pixel coordinate is converted into the second spatial coordinate, and the vertex pixel coordinate is converted into the vertex spatial coordinate; using the positional relationship between the target position in the ultrasonic probe and the puncture needle grabbed by the local puncture device, based on the vertex spatial coordinates, the tip spatial coordinates and the terminal spatial coordinates of the puncture needle are determined; sending spatial coordinate information to the local puncture device to control the local puncture device to perform puncture according to the spatial coordinate information, wherein the spatial coordinate information includes the first spatial coordinate, the second spatial coordinate, the tip spatial coordinate, and the terminal spatial coordinate.

Images