A method, system, medium, and electronic device for controlling the insertion speed of a puncture needle.
By using real-time ultrasound image segmentation and deformation response estimation models, the needle insertion speed of the puncture needle is automatically adjusted, solving the problem of unstable needle insertion speed during anesthesia puncture, achieving smoother and more accurate needle insertion control, and improving the success rate and safety of the operation.
Patent Information
- Application Number
- CN202311377937.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-23
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2043-10-23
AI Technical Summary
During anesthesia and puncture, the speed of needle insertion is difficult to control precisely, leading to instability in the needle insertion process and affecting the success rate of the surgery and patient safety.
By receiving ultrasound images in real time and inputting them into a pre-trained ultrasound image segmentation model, combined with a puncture needle deformation response estimation model, the needle insertion speed is automatically adjusted, and a servo motor control system is used to achieve smooth and accurate needle insertion.
This improves the stability of the needle insertion process, avoids tissue damage, and enhances the success rate and safety of the surgery.
Smart Images

Figure CN117414186B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of mechanical control technology, and in particular to a method, system, medium and electronic equipment for controlling the insertion speed of a puncture needle. Background Technology
[0002] Anesthesia puncture devices are widely used in clinical practice as medical instruments for surgical anesthesia. Currently, all anesthesia puncture procedures performed in hospitals are performed by experienced anesthesiologists. Ultrasound-guided local anesthesia puncture is a procedure performed under the guidance of ultrasound imaging. This procedure uses ultrasound imaging technology to observe the patient's internal structures in real time, guiding the physician to accurately perform local anesthesia and puncture. As needed, the physician uses a puncture needle or catheter to enter the patient's skin and tissues under local anesthesia.
[0003] In related technologies, accurately locating and controlling the insertion speed of the puncture needle during anesthesia puncture is crucial for the success of the surgery and the safety of the patient. However, due to the complexity and diversity of human tissues, as well as the slight deformation of the puncture needle that may occur during puncture, the accuracy and smoothness of the insertion speed can be affected, thereby reducing the stability of the puncture process. Summary of the Invention
[0004] This application provides a method, system, medium, and electronic device for controlling the insertion speed of a puncture needle. To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general description, nor is it intended to identify key / important components or describe the scope of protection of these embodiments. Its sole purpose is to present some concepts in a simple form as a prelude to the detailed description that follows.
[0005] In a first aspect, embodiments of this application provide a method for controlling the insertion speed of a puncture needle, the method comprising:
[0006] The real-time received ultrasound image to be segmented is input into a pre-trained ultrasound image segmentation model, which outputs the segmentation result corresponding to the ultrasound image to be segmented.
[0007] Based on the pre-established puncture needle deformation response estimation model, predict the deformation response of the puncture needle of the current puncture device;
[0008] Based on the segmentation results and deformation response, determine the speed adjustment factor of the puncture needle;
[0009] The speed adjustment factor controls the forward or backward movement of the puncture needle.
[0010] Optionally, before inputting the real-time received ultrasound image to be segmented into a pre-trained ultrasound image segmentation model, the method further includes:
[0011] Acquire a preset number of ultrasound images to obtain multiple sample ultrasound images;
[0012] Tissue type labels are assigned to the tissue category of each pixel in the ultrasound image sample to obtain the ultrasound image dataset;
[0013] Data preprocessing is performed on the ultrasound image dataset to obtain model training samples;
[0014] Construct an ultrasound image segmentation model and the corresponding pixel-level cross-entropy loss function for the ultrasound image segmentation model;
[0015] The ultrasound image segmentation model is trained using model training samples and a pixel-level cross-entropy loss function to minimize the similarity between the model's predictions and tissue type labels, thus generating a pre-trained ultrasound image segmentation model.
[0016] Optionally, output the segmentation results corresponding to the ultrasound image to be segmented, including:
[0017] Output the probability that each pixel in the ultrasound image to be segmented belongs to a different tissue category;
[0018] The probability of each pixel in the ultrasound image to be segmented belonging to a different tissue category is used as the segmentation result of the ultrasound image to be segmented.
[0019] The pixel-level cross-entropy loss function is:
[0020]
[0021] Among them, L pixel This represents pixel-level cross-entropy loss, where H is the image height (number of rows), W is the image width (number of columns), and y... ij It is the index of the organization category to which pixel (i,j) belongs in the actual label, and its value ranges from 0 to K-1; p ij c It is the probability that the model predicts class c at position (i,j).
[0022] Optionally, based on a pre-established puncture needle deformation response estimation model, the deformation response of the puncture needle of the current puncture device is predicted, including:
[0023] Obtain the material and geometric parameters of the puncture needle;
[0024] Fit the direction and magnitude of the force exerted on the needle tip during the puncture process;
[0025] The direction and magnitude of the force on the needle tip are decomposed into mechanical components;
[0026] The deformation response is calculated based on the material and geometric parameters and mechanical components of the puncture needle, combined with a pre-established puncture needle deformation response estimation model.
[0027] Optionally, based on the segmentation results and deformation response, a velocity adjustment factor for the puncture needle is determined, including:
[0028] Based on the segmentation results, determine the puncture path;
[0029] Get the current insertion angle of the puncture needle;
[0030] Determine the target needle insertion angle based on the puncture path;
[0031] Based on the current needle insertion angle, the target needle insertion angle, and the needle deformation response, the needle insertion speed adjustment factor is calculated; where,
[0032] Needle insertion speed adjustment factor
[0033]
[0034] Here, α and β are control parameters used to adjust the influence of the needle insertion angle and deformation response on the speed. These parameters can be adjusted based on actual experimental and simulation data; the current needle insertion angle is θ, and the target needle insertion angle is θ₀. target The deformation response of the puncture needle is w(x).
[0035] Optionally, based on the segmentation results, the puncture path is determined, including:
[0036] The segmentation results are visualized on the ultrasound image to be segmented, and the muscle, artery, vein, nerve, and fascia tissue regions are displayed with different colors or markers to obtain the visualization results.
[0037] Receive puncture point selection instructions based on the visualization results;
[0038] The puncture path is generated based on the puncture point selected by the puncture point selection command.
[0039] Optionally, the forward or backward movement of the puncture needle can be controlled according to a speed adjustment factor, including:
[0040] Obtain the system parameters and actual mechanical structure parameters of the servo motor control system on the puncture robot used to control the insertion of the puncture needle;
[0041] Based on the system parameters of the servo motor control system and the actual mechanical structure parameters, the needle feed speed adjustment factor is converted into a speed control signal;
[0042] The speed control signal controls the puncture needle to move forward or backward.
[0043] Secondly, embodiments of this application provide a needle insertion speed control system for a puncture needle, the system comprising:
[0044] Ultrasonic probe, puncture needle, puncture frame, and main unit connected to the ultrasonic probe;
[0045] An ultrasonic probe is used to acquire ultrasonic images to be segmented in real time.
[0046] The control system inputs the real-time received ultrasound images to be segmented into a pre-trained ultrasound image segmentation model and outputs the segmentation results corresponding to the ultrasound images to be segmented; based on the pre-established puncture needle deformation response estimation model, it predicts the deformation response of the puncture needle of the current puncture device; based on the segmentation results and deformation response, it determines the speed adjustment factor of the puncture needle; based on the speed adjustment factor, it controls the puncture needle to move forward or backward.
[0047] Fourthly, embodiments of this application provide an electronic device that may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and executed by the above-described method steps.
[0048] The technical solutions provided in this application embodiment may include the following beneficial effects:
[0049] In this embodiment, the needle insertion speed control system first inputs the real-time received ultrasound image to be segmented into a pre-trained ultrasound image segmentation model, and outputs the segmentation result corresponding to the ultrasound image to be segmented; then, based on a pre-established needle deformation response estimation model, it predicts the deformation response of the needle in the current puncture device; secondly, based on the segmentation result and the deformation response, it determines the needle speed adjustment factor; finally, based on the speed adjustment factor, it controls the needle to advance or retreat. Since this application predicts the possible deformation response of the needle based on a pre-established needle deformation response estimation model, it can automatically adjust the needle insertion speed based on this deformation response and the image segmentation result output by the pre-trained ultrasound image segmentation model, making the needle insertion speed smoother and more accurate during the insertion process, avoiding excessive pressure and tissue damage, thereby improving the stability of the needle insertion process.
[0050] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description
[0051] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.
[0052] Figure 1 This is a flowchart illustrating a method for controlling the insertion speed of a puncture needle according to an embodiment of this application.
[0053] Figure 2 This is a schematic diagram of the network architecture of a model provided in an embodiment of this application;
[0054] Figure 3 This is a schematic diagram of a tissue structure visualized by processing ultrasound images, as provided in this application.
[0055] Figure 4 This application provides a schematic diagram of the needle insertion path formed after selecting the needle insertion point on a visualized image;
[0056] Figure 5 This is a schematic diagram of the structure of a needle insertion speed control system for a puncture needle provided in an embodiment of this application;
[0057] Figure 6 This is a schematic diagram of the structure of a drive mechanism and an actuator provided in an embodiment of this application;
[0058] Figure 7 This is a schematic diagram of the structure of an angle motor in a puncture drive motor provided in an embodiment of this application;
[0059] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0060] The following description and accompanying drawings fully illustrate specific embodiments of this application to enable those skilled in the art to practice them.
[0061] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.
[0062] In the following description, when referring to the accompanying drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of systems and methods consistent with some aspects of this application as detailed in the appended claims.
[0063] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.
[0064] To address the issue of smooth needle insertion, this application provides a method, system, medium, and electronic device for controlling the insertion speed of a puncture needle, thereby resolving the problems mentioned above. In the technical solution provided by this application, since the possible deformation response of the puncture needle is predicted based on a pre-established puncture needle deformation response estimation model, the insertion speed can be automatically adjusted based on this deformation response and the image segmentation results output by a pre-trained ultrasound image segmentation model. This makes the insertion speed smoother and more accurate, avoiding excessive pressure and tissue damage, thus improving the stability of the insertion process. Exemplary embodiments are described in detail below.
[0065] The following will be combined with the appendix Figure 1 -Appendix Figure 4 This application provides a detailed description of the needle insertion speed control method for puncture needles according to embodiments. This method can be implemented using a computer program and can run on a puncture needle insertion speed control system based on the von Neumann architecture. This computer program can be integrated into an application or run as a standalone utility application.
[0066] Please see Figure 1 This is a flowchart illustrating a method for controlling the insertion speed of a puncture needle, as provided in this application embodiment. Figure 1 As shown, the method in this application embodiment may include the following steps:
[0067] S101, input the real-time received ultrasound image to be segmented into the pre-trained ultrasound image segmentation model, and output the segmentation result corresponding to the ultrasound image to be segmented;
[0068] The ultrasound image to be segmented is acquired in real time by the ultrasound probe of the ultrasound equipment, and the pre-trained ultrasound image segmentation model is a mathematical model that can determine different tissues or structures in the image.
[0069] In this embodiment, when generating a pre-trained ultrasound image segmentation model, a preset number of ultrasound images are first collected to obtain multiple sample ultrasound images; then, tissue type labels are set for the tissue category to which pixels in each sample ultrasound image belong, resulting in an ultrasound image dataset; next, the ultrasound image dataset is preprocessed to obtain model training samples; then, an ultrasound image segmentation model and a pixel-level cross-entropy loss function corresponding to the ultrasound image segmentation model are constructed; finally, the ultrasound image segmentation model is trained using the model training samples and the pixel-level cross-entropy loss function to minimize the similarity between the model's prediction results and the tissue type labels, thereby generating a pre-trained ultrasound image segmentation model.
[0070] Specifically, tissue type labels include the tissue category to which each pixel belongs, such as muscle, artery, vein, nerve, fascia, etc. Data preprocessing includes image normalization, cropping, and enhancement to improve the model's robustness and generalization ability.
[0071] Specifically, when constructing an ultrasound image segmentation model, a pre-trained DeepLabV3 model can be used, or a new model can be trained from scratch. The main structure of the model includes a backbone network with dilated convolutions and ASPP (Atrous Spatial Pyramid Pooling). A schematic diagram of the model's network structure is shown below. Figure 2 As shown.
[0072] Dilated convolution is a special type of convolutional operation in convolutional neural networks (CNNs). Compared to traditional convolution operations, dilated convolution introduces an adjustable dilation rate (or stride) into the convolution kernel, allowing it to view the image over a larger receptive field. Traditional convolution operations use a fixed stride to slide the image through the kernel, typically 1, meaning it moves pixel by pixel. This type of convolution only focuses on pixels in local regions and may be insensitive to larger images or broader contextual information. Dilated convolution achieves different effects by adjusting the dilation rate. When the dilation rate is 1, dilated convolution becomes a traditional convolution; when the dilation rate is greater than 1, the kernel takes on a spatially dilated form, meaning there are gaps between adjacent pixels of the kernel. As a result, the kernel views the image over a larger receptive field, thus better capturing the contextual information in the image.
[0073] ASPP concatenates or adds the output feature maps of dilated convolutions with different dilation rates, thereby fusing contextual information at different scales. This allows the model to perform semantic segmentation of images at multiple scales and has better perception capabilities for targets of different sizes. Furthermore, the model is adjusted to suit the input and output dimensions for ultrasound image segmentation.
[0074] Specifically, the loss function used is the pixel-level cross-entropy loss function, which measures the difference between the model's predicted segmentation result and the true label. For ultrasound image tissue segmentation, assuming there is an ultrasound image of size H×W, where each pixel belongs to one of K tissue categories (such as muscle, artery, vein, nerve, fascia, etc.), then the pixel-level cross-entropy loss function is:
[0075]
[0076] Among them, L pixel This represents pixel-level cross-entropy loss, where H is the image height (number of rows), W is the image width (number of columns), and y... ij It is the index of the organization category to which pixel (i,j) belongs in the actual label, and its value ranges from 0 to K-1; p ij c It is the probability that the model predicts class c at position (i,j). Minimizing the pixel-level cross-entropy loss function can make the model's prediction result as close as possible to the true label, thereby achieving accurate tissue segmentation of ultrasound images.
[0077] Specifically, during training, the model continuously adjusts its weights and biases through an optimization algorithm (stochastic gradient descent) to make its predictions as close as possible to the true labels.
[0078] In this embodiment of the application, when outputting the segmentation result corresponding to the ultrasound image to be segmented, the probability of each pixel in the ultrasound image to be segmented belonging to different tissue categories can be output; then, the probability of each pixel in the ultrasound image to be segmented belonging to different tissue categories is used as the segmentation result corresponding to the ultrasound image to be segmented. A schematic diagram of the process from the ultrasound image to the segmentation result is shown below. Figure 3 As shown.
[0079] Furthermore, the tissue category to which a pixel belongs can be determined based on the highest probability. Post-processing of the segmentation results, such as removing small noise regions and connecting scattered regions of the same tissue, can yield smoother and more accurate segmentation results.
[0080] S102, Based on the pre-established puncture needle deformation response estimation model, predict the deformation response of the puncture needle of the current puncture device;
[0081] In this embodiment of the application, when predicting the deformation response of the puncture needle of the current puncture device according to the pre-established puncture needle deformation response estimation model, the material parameters and geometric parameters of the puncture needle are first obtained; then the direction and magnitude of the force on the needle tip during the puncture process are fitted; next, the direction and magnitude of the force on the needle tip are decomposed into mechanical components; finally, the deformation response is calculated according to the material parameters and geometric parameters of the puncture needle, the mechanical components, and the pre-established puncture needle deformation response estimation model.
[0082] In one possible implementation, a pre-established model for estimating the deformation response of the puncture needle is obtained through the following process: Determining material properties and geometry: Understanding the elastic modulus E, shear modulus G, density ρ, and other material parameters of the material used for the puncture needle. Determining the geometry of the puncture needle, including the shape and size of the cross-section, the length L, diameter d, and other key geometric parameters. Force analysis: The external forces acting on the puncture needle during the puncture process originate from tissue resistance, gravity, friction, etc. Considering the direction and magnitude of the forces acting on the needle tip during the puncture process, and decomposing them into appropriate mechanical components. A beam bending model is used to describe the deformation behavior of the puncture needle. In the beam bending model, it is assumed that the deformation response w of the puncture needle can vary along the x-axis. The longitudinal position of the puncture needle after deformation can then be represented by y(x). For the puncture needle, the equation of the beam bending model is:
[0083]
[0084] in, The curvature of the puncture needle is represented by the second derivative of the puncture needle deformation response w(x) with respect to x. M(x) is the bending moment of the puncture needle at position x, which is the moment related to the external force acting on the puncture needle. The bending moment M(x) can be calculated from the external load acting on the puncture needle and the geometry of the puncture needle cross-section. The elastic modulus E of the material used for the puncture needle and the length L of the puncture needle are also given. The established beam bending equation is solved to obtain the deformation w(x) of the puncture needle. Model verification and optimization: The established deformation response model is verified using experimental or simulation data to ensure that the model matches the actual situation. If there are discrepancies between the model and the actual situation, the parameters of the model are adjusted according to the experimental data to improve the accuracy and applicability of the model.
[0085] S103, Determine the speed adjustment factor of the puncture needle based on the segmentation results and deformation response;
[0086] In this embodiment of the application, when determining the speed adjustment factor of the puncture needle based on the segmentation result and deformation response, the puncture path is first determined based on the segmentation result; then, the current insertion angle of the puncture needle is obtained; then, the target insertion angle is determined based on the puncture path; finally, the insertion speed adjustment factor of the puncture needle is calculated based on the current insertion angle, the target insertion angle, and the deformation response of the puncture needle; wherein,
[0087] Needle insertion speed adjustment factor
[0088]
[0089] Here, α and β are control parameters used to adjust the influence of the needle insertion angle and deformation response on the speed. These parameters can be adjusted based on actual experimental and simulation data; the current needle insertion angle is θ, and the target needle insertion angle is θ₀. target The deformation response of the puncture needle is w(x).
[0090] Specifically, when determining the puncture path based on the segmentation results, the segmentation results are first visualized on the ultrasound image to be segmented, displaying muscle, artery, vein, nerve, and fascia tissue regions with different colors or markers to obtain a visualization result. Then, a puncture point selection instruction based on the visualization result is received. Finally, the puncture path is generated according to the puncture point indicated by the puncture point selection instruction. This application, through ultrasound-guided technology, allows doctors to view ultrasound images in real time, accurately locate the puncture needle and target tissue, and assist doctors in performing precise puncture operations. At the same time, image segmentation technology identifies different tissue structures and provides doctors with a visual interface to help them select the optimal puncture path, avoid important tissue structures, and improve the safety and accuracy of puncture.
[0091] For example, the segmentation results can be visualized on the image, using different colors or markers to show different tissue regions such as muscles, arteries, veins, nerves, and fascia. Doctors use the mouse to click on the image to select puncture points. These puncture points are located in the target tissue area where the doctor wants to puncture. Doctors can use the mouse to connect the needle insertion points, forming a puncture path. Doctors can adjust the path as needed. If the doctor is not satisfied with the puncture path, they can modify it by reselecting puncture points or adjusting the order of connections. After the doctor is satisfied with the puncture path, they can confirm the selected path. The system needs to remind the doctor that the selected path must ensure that the needle tip will not damage important tissue structures to avoid accidents. Only after the doctor confirms again can it be set as the puncture path. The puncture path is, for example... Figure 4 As shown.
[0092] S104 controls the forward or backward movement of the puncture needle based on the speed adjustment factor.
[0093] In this embodiment of the application, when controlling the puncture needle to move forward or backward according to the speed adjustment factor, the system parameters and actual mechanical structure parameters of the servo motor control system on the puncture robot used to control the needle insertion are first obtained; then, the needle insertion speed adjustment factor is converted into a speed control signal according to the system parameters of the servo motor control system and the actual mechanical structure parameters; finally, the puncture needle is controlled to move forward or backward according to the speed control signal.
[0094] Specifically, based on the speed adjustment factor, the speed control signal of the servo motor is adjusted to ensure the needle insertion speed is appropriate for forward or backward movement. The needle insertion speed adjustment factor (speed_factor) is a real number reflecting the influence of deformation response and insertion angle on speed during needle insertion. The needle insertion speed adjustment factor is then converted into an appropriate speed control signal. This requires adaptation based on the specific requirements of the servo motor control system and the actual mechanical structure. Based on the speed control signal, the servo motor speed is adjusted, and a PID controller is used to achieve closed-loop speed control. The PID controller is a commonly used control algorithm that adjusts the servo motor control signal based on the difference between the target speed and the actual speed, gradually bringing the actual speed closer to the target speed.
[0095] It should be noted that in practical applications, the maximum needle insertion speed should be limited to avoid tissue damage caused by excessively fast puncture speed.
[0096] In the embodiments of this application, accurate positioning and control, as well as adaptive needle insertion speed adjustment, can significantly improve the success rate of surgery and reduce potential problems and complications during surgery.
[0097] In this embodiment, the needle insertion speed control system first inputs the real-time received ultrasound image to be segmented into a pre-trained ultrasound image segmentation model, and outputs the segmentation result corresponding to the ultrasound image to be segmented; then, based on a pre-established needle deformation response estimation model, it predicts the deformation response of the needle in the current puncture device; secondly, based on the segmentation result and the deformation response, it determines the needle speed adjustment factor; finally, based on the speed adjustment factor, it controls the needle to advance or retreat. Since this application predicts the possible deformation response of the needle based on a pre-established needle deformation response estimation model, it can automatically adjust the needle insertion speed based on this deformation response and the image segmentation result output by the pre-trained ultrasound image segmentation model, making the needle insertion speed smoother and more accurate during the insertion process, avoiding excessive pressure and tissue damage, thereby improving the stability of the needle insertion process.
[0098] The following are system embodiments of this application, which can be used to execute the method embodiments of this application. For details not disclosed in the system embodiments of this application, please refer to the method embodiments of this application.
[0099] Please see Figure 5 This illustration shows a schematic diagram of the needle insertion speed control system of a puncture needle provided in an exemplary embodiment of this application. The needle insertion speed control system can be implemented as all or part of an electronic device through software, hardware, or a combination of both. System 1 includes an ultrasonic probe, a puncture needle, a puncture frame, and a main unit connected to the ultrasonic probe;
[0100] An ultrasonic probe is used to acquire ultrasonic images to be segmented in real time.
[0101] The control system inputs the real-time received ultrasound images to be segmented into a pre-trained ultrasound image segmentation model and outputs the segmentation results corresponding to the ultrasound images to be segmented; based on the pre-established puncture needle deformation response estimation model, it predicts the deformation response of the puncture needle of the current puncture device; based on the segmentation results and deformation response, it determines the speed adjustment factor of the puncture needle; based on the speed adjustment factor, it controls the puncture needle to move forward or backward.
[0102] Specifically, before inputting the real-time received ultrasound images to be segmented into the pre-trained ultrasound image segmentation model, the process includes: acquiring a preset number of ultrasound images to obtain multiple sample ultrasound images; setting tissue type labels for the tissue categories to which pixels in each sample ultrasound image belong to, thus obtaining an ultrasound image dataset; performing data preprocessing on the ultrasound image dataset to obtain model training samples; constructing an ultrasound image segmentation model and the corresponding pixel-level cross-entropy loss function; and training the ultrasound image segmentation model using the model training samples and the pixel-level cross-entropy loss function to minimize the similarity between the model's prediction results and the tissue type labels, thereby generating the pre-trained ultrasound image segmentation model.
[0103] Specifically, when outputting the segmentation result corresponding to the ultrasound image to be segmented, the probability of each pixel in the ultrasound image to be segmented belonging to different tissue categories is output; the probability of each pixel in the ultrasound image to be segmented belonging to different tissue categories is used as the segmentation result corresponding to the ultrasound image to be segmented.
[0104] The pixel-level cross-entropy loss function is:
[0105]
[0106] Among them, L pixel This represents pixel-level cross-entropy loss, where H is the image height (number of rows), W is the image width (number of columns), and y... ij It is the index of the organization category to which pixel (i,j) belongs in the actual label, and its value ranges from 0 to K-1; p ij c It is the probability that the model predicts class c at position (i,j).
[0107] Specifically, when predicting the deformation response of the puncture needle of the current puncture device based on the pre-established puncture needle deformation response estimation model, the material parameters and geometric parameters of the puncture needle are obtained; the direction and magnitude of the force on the needle tip during puncture are fitted; the direction and magnitude of the force on the needle tip are decomposed into mechanical components; and the deformation response is calculated based on the material parameters, geometric parameters, and mechanical components of the puncture needle, combined with the pre-established puncture needle deformation response estimation model.
[0108] Specifically, when determining the speed adjustment factor of the puncture needle based on the segmentation results and deformation response, the puncture path is determined according to the segmentation results; the current insertion angle of the puncture needle is obtained; the target insertion angle is determined based on the puncture path; and the insertion speed adjustment factor of the puncture needle is calculated based on the current insertion angle, the target insertion angle, and the deformation response of the puncture needle.
[0109] Needle insertion speed adjustment factor
[0110]
[0111] Here, α and β are control parameters used to adjust the influence of the needle insertion angle and deformation response on the speed. These parameters can be adjusted based on actual experimental and simulation data; the current needle insertion angle is θ, and the target needle insertion angle is θ₀. target The deformation response of the puncture needle is w(x).
[0112] Specifically, when determining the puncture path based on the segmentation results, the segmentation results are visualized on the ultrasound image to be segmented, and the muscle, artery, vein, nerve, and fascia tissue regions are displayed with different colors or markers to obtain the visualization results; the puncture point selection instruction based on the visualization results is received; and the puncture path is generated according to the puncture point indicated by the puncture point selection instruction.
[0113] Specifically, when controlling the puncture needle to move forward or backward according to the speed adjustment factor, the system parameters and actual mechanical structure parameters of the servo motor control system on the puncture robot used to control the needle insertion are obtained; the needle insertion speed adjustment factor is converted into a speed control signal according to the system parameters of the servo motor control system and the actual mechanical structure parameters; and the puncture needle is controlled to move forward or backward according to the speed control signal.
[0114] Furthermore, for example Figure 6 As shown, the puncture frame integrates a position motor, a position drive mechanism, an angle adjustment mechanism, a puncture drive motor, and a puncture linear drive mechanism. The control system is connected to the ultrasound probe, the position drive mechanism, the angle adjustment mechanism, and the puncture linear drive mechanism, and outputs control commands.
[0115] Specifically, the angle adjustment mechanism employs a crank-slider mechanism to achieve angle adjustment. A servo motor drives a gear set, which in turn drives a lead screw to move a nut slider linearly, thus driving a connecting rod. This causes the arc-shaped slider at the end of the connecting rod to move along an arc track, achieving puncture angle adjustment. The lead screw and nut mechanism uses a linear guide for guidance, and a full-stroke patch sensor enables closed-loop control, providing real-time position information to the control system, which then calculates and converts it into angle information.
[0116] Specifically, the position drive mechanism uses a linear servo drive mechanism to drive the aforementioned angle adjustment mechanism, and uses a linear guide rail for guidance, so as to realize the linear position adjustment of the puncture needle (the linear distance between the puncture needle and the ultrasound probe), and to provide real-time feedback of position information to the control system.
[0117] Specifically, the linear drive mechanism for puncture uses a lead screw and nut transmission to meet the functional requirements of radial puncture. A servo motor drives a gear set, which in turn drives the puncture needle radially via the lead screw. A linear guide rail provides guidance to achieve the puncture function. A full-stroke patch sensor enables closed-loop control, providing real-time position information to the control system.
[0118] Specifically, the angle motor in the puncture drive motor, for example... Figure 7 As shown.
[0119] It should be noted that the needle insertion speed control system for the puncture needle provided in the above embodiments is only an example of the division of the functional modules described above when executing the needle insertion speed control method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the needle insertion speed control system for the puncture needle provided in the above embodiments and the needle insertion speed control method embodiments belong to the same concept, and its implementation process is detailed in the method embodiments, which will not be repeated here.
[0120] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0121] In this embodiment, the needle insertion speed control system first inputs the real-time received ultrasound image to be segmented into a pre-trained ultrasound image segmentation model, and outputs the segmentation result corresponding to the ultrasound image to be segmented; then, based on a pre-established needle deformation response estimation model, it predicts the deformation response of the needle in the current puncture device; secondly, based on the segmentation result and the deformation response, it determines the needle speed adjustment factor; finally, based on the speed adjustment factor, it controls the needle to advance or retreat. Since this application predicts the possible deformation response of the needle based on a pre-established needle deformation response estimation model, it can automatically adjust the needle insertion speed based on this deformation response and the image segmentation result output by the pre-trained ultrasound image segmentation model, making the needle insertion speed smoother and more accurate during the insertion process, avoiding excessive pressure and tissue damage, thereby improving the stability of the needle insertion process.
[0122] This application also provides a computer-readable medium having program instructions stored thereon, which, when executed by a processor, implement the needle insertion speed control method for the puncture needle provided in the above-described method embodiments.
[0123] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the needle insertion speed control method of the puncture needle in the above-described method embodiments.
[0124] Please see Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 8 As shown, the electronic device 1000 may include: at least one processor 1001, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002.
[0125] The communication bus 1002 is used to realize the connection and communication between these components.
[0126] The user interface 1003 may include a display screen and a camera. Optionally, the user interface 1003 may also include a standard wired interface and a wireless interface.
[0127] The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).
[0128] The processor 1001 may include one or more processing cores. The processor 1001 connects to various parts within the electronic device 1000 using various interfaces and lines. It executes various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1005, and by calling data stored in the memory 1005. Optionally, the processor 1001 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 1001 may integrate one or more of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed on the screen; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip, without being integrated into the processor 1001.
[0129] The memory 1005 may include random access memory (RAM) or read-only memory. Optionally, the memory 1005 may include a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 1005 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 1005 may also be at least one storage system located remotely from the aforementioned processor 1001. Figure 8 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for controlling the insertion speed of the puncture needle.
[0130] exist Figure 8In the illustrated electronic device 1000, the user interface 1003 is mainly used to provide an input interface for the user and to acquire user input data; while the processor 1001 can be used to call the needle insertion speed control application stored in the memory 1005 and specifically perform the following operations:
[0131] The real-time received ultrasound image to be segmented is input into a pre-trained ultrasound image segmentation model, which outputs the segmentation result corresponding to the ultrasound image to be segmented.
[0132] Based on the pre-established puncture needle deformation response estimation model, predict the deformation response of the puncture needle of the current puncture device;
[0133] Based on the segmentation results and deformation response, determine the speed adjustment factor of the puncture needle;
[0134] The speed adjustment factor controls the forward or backward movement of the puncture needle.
[0135] In one embodiment, before the processor 1001 executes the input of the ultrasound image to be segmented received in real time into a pre-trained ultrasound image segmentation model, it also performs the following operations:
[0136] Acquire a preset number of ultrasound images to obtain multiple sample ultrasound images;
[0137] Tissue type labels are assigned to the tissue category of each pixel in the ultrasound image sample to obtain the ultrasound image dataset;
[0138] Data preprocessing is performed on the ultrasound image dataset to obtain model training samples;
[0139] Construct an ultrasound image segmentation model and the corresponding pixel-level cross-entropy loss function for the ultrasound image segmentation model;
[0140] The ultrasound image segmentation model is trained using model training samples and a pixel-level cross-entropy loss function to minimize the similarity between the model's predictions and tissue type labels, thus generating a pre-trained ultrasound image segmentation model.
[0141] In one embodiment, when the processor 1001 executes the segmentation result corresponding to the ultrasound image to be segmented, it specifically performs the following operations:
[0142] Output the probability that each pixel in the ultrasound image to be segmented belongs to a different tissue category;
[0143] The probability of each pixel in the ultrasound image to be segmented belonging to a different tissue category is used as the segmentation result of the ultrasound image to be segmented.
[0144] The pixel-level cross-entropy loss function is:
[0145]
[0146] Among them, L pixel This represents pixel-level cross-entropy loss, where H is the image height (number of rows), W is the image width (number of columns), and y... ij It is the index of the organization category to which pixel (i,j) belongs in the actual label, and its value ranges from 0 to K-1; p ij c It is the probability that the model predicts class c at position (i,j).
[0147] In one embodiment, when the processor 1001 executes the operation of predicting the deformation response of the puncture needle of the current puncture device based on a pre-established puncture needle deformation response estimation model, it specifically performs the following operations:
[0148] Obtain the material and geometric parameters of the puncture needle;
[0149] Fit the direction and magnitude of the force exerted on the needle tip during the puncture process;
[0150] The direction and magnitude of the force on the needle tip are decomposed into mechanical components;
[0151] The deformation response is calculated based on the material and geometric parameters and mechanical components of the puncture needle, combined with a pre-established puncture needle deformation response estimation model.
[0152] In one embodiment, when the processor 1001 determines the speed adjustment factor of the puncture needle based on the segmentation result and the deformation response, it specifically performs the following operations:
[0153] Based on the segmentation results, determine the puncture path;
[0154] Get the current insertion angle of the puncture needle;
[0155] Determine the target needle insertion angle based on the puncture path;
[0156] Based on the current needle insertion angle, the target needle insertion angle, and the needle deformation response, the needle insertion speed adjustment factor is calculated; where,
[0157] Needle insertion speed adjustment factor
[0158]
[0159] Here, α and β are control parameters used to adjust the influence of the needle insertion angle and deformation response on the speed. These parameters can be adjusted based on actual experimental and simulation data; the current needle insertion angle is θ, and the target needle insertion angle is θ₀. target The deformation response of the puncture needle is w(x).
[0160] In one embodiment, when the processor 1001 determines the puncture path based on the segmentation result, it specifically performs the following operations:
[0161] The segmentation results are visualized on the ultrasound image to be segmented, and the muscle, artery, vein, nerve, and fascia tissue regions are displayed with different colors or markers to obtain the visualization results.
[0162] Receive puncture point selection instructions based on the visualization results;
[0163] The puncture path is generated based on the puncture point selected by the puncture point selection command.
[0164] In one embodiment, when the processor 1001 controls the forward or backward movement of the puncture needle according to the speed adjustment factor, it specifically performs the following operations:
[0165] Obtain the system parameters and actual mechanical structure parameters of the servo motor control system on the puncture robot used to control the insertion of the puncture needle;
[0166] Based on the system parameters of the servo motor control system and the actual mechanical structure parameters, the needle feed speed adjustment factor is converted into a speed control signal;
[0167] The speed control signal controls the puncture needle to move forward or backward.
[0168] In this embodiment, the needle insertion speed control system first inputs the real-time received ultrasound image to be segmented into a pre-trained ultrasound image segmentation model, and outputs the segmentation result corresponding to the ultrasound image to be segmented; then, based on a pre-established needle deformation response estimation model, it predicts the deformation response of the needle in the current puncture device; secondly, based on the segmentation result and the deformation response, it determines the needle speed adjustment factor; finally, based on the speed adjustment factor, it controls the needle to advance or retreat. Since this application predicts the possible deformation response of the needle based on a pre-established needle deformation response estimation model, it can automatically adjust the needle insertion speed based on this deformation response and the image segmentation result output by the pre-trained ultrasound image segmentation model, making the needle insertion speed smoother and more accurate during the insertion process, avoiding excessive pressure and tissue damage, thereby improving the stability of the needle insertion process.
[0169] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program for controlling the needle insertion speed can be stored in a computer-readable storage medium. When executed, the program can include the processes of the embodiments of the above methods. The storage medium for the program for controlling the needle insertion speed can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.
[0170] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.
Claims
1. A needle insertion speed control system for a puncture needle, characterized in that, The system includes: An ultrasound probe, a puncture needle, a puncture frame, and a main unit connected to the ultrasound probe; The ultrasonic probe is used to acquire the ultrasonic image to be segmented in real time; The control system inputs the real-time received ultrasound image to be segmented into a pre-trained ultrasound image segmentation model and outputs the segmentation result corresponding to the ultrasound image to be segmented; predicts the deformation response of the puncture needle of the current puncture device according to a pre-established puncture needle deformation response estimation model; determines the speed adjustment factor of the puncture needle according to the segmentation result and the deformation response; and controls the puncture needle to move forward or backward according to the speed adjustment factor; the step of determining the speed adjustment factor of the puncture needle according to the segmentation result and the deformation response includes: Based on the segmentation results, the puncture path is determined; the current insertion angle of the puncture needle is obtained; based on the puncture path, the target insertion angle is determined; based on the current insertion angle, the target insertion angle, and the deformation response of the puncture needle, the insertion speed adjustment factor of the puncture needle is calculated; wherein, The needle insertion speed adjustment factor of the puncture needle Here, α and β are control parameters used to adjust the influence of the needle insertion angle and deformation response on the speed. These parameters can be adjusted based on actual experimental and simulation data; the current needle insertion angle is... The target needle insertion angle is The puncture needle deformation response is w ( x ).
2. The system according to claim 1, characterized in that, Before inputting the real-time received ultrasound image to be segmented into the pre-trained ultrasound image segmentation model, the process further includes: Acquire a preset number of ultrasound images to obtain multiple sample ultrasound images; Tissue type labels are assigned to the tissue category of each pixel in the ultrasound image sample to obtain the ultrasound image dataset; The ultrasound image dataset is preprocessed to obtain model training samples; Construct an ultrasound image segmentation model and the corresponding pixel-level cross-entropy loss function for the ultrasound image segmentation model; The ultrasound image segmentation model is trained using the model training samples and the pixel-level cross-entropy loss function to minimize the similarity between the model's prediction results and the tissue type labels, thereby generating a pre-trained ultrasound image segmentation model.
3. The system according to claim 1, characterized in that, The output of the segmentation result corresponding to the ultrasound image to be segmented includes: Output the probability that each pixel in the ultrasound image to be segmented belongs to a different tissue category; The probability of each pixel in the ultrasound image to be segmented belonging to a different tissue category is used as the segmentation result corresponding to the ultrasound image to be segmented. The pixel-level cross-entropy loss function is: ; in, This represents the pixel-level cross-entropy loss, where H is the image height and W is the image width. It is the index of the organization category to which pixel (i,j) belongs in the real label, and its value ranges from 0 to K-1; It is the probability that the model predicts class c at position (i,j).
4. The system according to claim 1, characterized in that, The step of predicting the deformation response of the puncture needle of the current puncture device based on a pre-established puncture needle deformation response estimation model includes: Obtain the material parameters and geometric parameters of the puncture needle; Fit the direction and magnitude of the force exerted on the needle tip during the puncture process; The direction and magnitude of the force on the needle tip are decomposed into mechanical components; The deformation response is calculated based on the material and geometric parameters and mechanical components of the puncture needle, combined with a pre-established puncture needle deformation response estimation model.
5. The system according to claim 1, characterized in that, Determining the puncture path based on the segmentation result includes: The segmentation results are visualized on the ultrasound image to be segmented, and the muscle, artery, vein, nerve, and fascia tissue regions are displayed with different colors or markers to obtain the visualization results. Receive a puncture point selection instruction based on the visualization results; A puncture path is generated based on the puncture point selected by the puncture point selection instruction.
6. The system according to claim 1, characterized in that, The step of controlling the forward or backward movement of the puncture needle according to the speed adjustment factor includes: Obtain the system parameters and actual mechanical structure parameters of the servo motor control system on the puncture robot used to control the insertion of the puncture needle; Based on the system parameters of the servo motor control system and the actual mechanical structure parameters, the needle feed speed adjustment factor is converted into a speed control signal; The speed control signal controls the puncture needle to move forward or backward.
Citation Information
Patent Citations
Mechanical and bending deformation modeling method for flexible needle penetration into soft tissue
CN107808698A
Venipuncture robot control method based on ultrasonic image Kalman filtering
CN112807087A
