A method of puncture visualization based on ultrasound imaging

Abstract

This application provides a puncture visualization method based on ultrasound imaging, comprising: acquiring a low-frequency ultrasound image and a real-time ultrasound image at the current ultrasound frequency, wherein the ultrasound frequency of each stage includes the low-frequency ultrasound frequency and the real-time ultrasound frequency of that stage, and the low-frequency ultrasound image and the real-time ultrasound image of the same stage have the same size and a one-to-one pixel correspondence; determining the needle tip position based on the real-time ultrasound image of the current stage; fusing the real-time ultrasound image of the current stage with the low-frequency ultrasound image to obtain a fused ultrasound image of the current stage; determining the tissue type on the current puncture path based on the fused ultrasound image of the current stage; and determining the ultrasound frequency of the next stage based on the needle tip position and the tissue type on the current puncture path. This method realizes intelligent control of the puncture process, reduces the doctor's operational burden, and helps the doctor complete the puncture operation more quickly and accurately.

Description

Technical Field

[0001] This application relates to the field of ultrasound-guided puncture technology, and more specifically, to a puncture visualization method based on ultrasound imaging. Background Technology

[0002] Ultrasound imaging technology is a common and important imaging technique in clinical practice. It has become the second most commonly used diagnostic imaging method in medicine (after traditional X-rays) and is a crucial diagnostic tool for many medical institutions.

[0003] Ultrasound-guided puncture is widely used in various medical fields, such as the diagnosis and treatment of tumors, pancreatic diseases, thyroid diseases, and vascular punctures. This technology uses ultrasound imaging to display the position and path of the puncture needle in real time, guiding doctors to accurately insert the needle into the target area, thereby improving the accuracy and safety of the puncture.

[0004] However, existing ultrasound-guided puncture techniques also have some drawbacks. Some are limited by inherent defects in the technique itself, while others require a high level of skill from the physician; otherwise, the quality of the ultrasound image may be significantly affected. For example, different ultrasound frequencies produce different imaging effects on different tissues because different tissue types have different absorption and reflectivity rates, leading to differences in imaging. Low-frequency ultrasound is more suitable for penetrating deep tissues, thus focusing on overall imaging, while high-frequency ultrasound (different tissue types and depths are suitable for different ultrasound frequencies) focuses more on clear imaging of local areas.

[0005] Currently, when using ultrasound-guided puncture technology, medical staff usually need to manually change the ultrasound frequency to guide the puncture. This method requires a high level of skill from the operator and has relatively low guidance efficiency.

[0006] Currently, there are also studies combining artificial intelligence technology for ultrasound-guided punctures. However, their main research focus is on collecting a large amount of ultrasound-guided puncture operation data and combining it with expert-annotated data to develop intelligent ultrasound frequency control strategies throughout the entire process. This research approach does not have high applicability to different puncture types and individual differences. Operators usually still need to rely on their own experience to adjust the ultrasound frequency locally during the procedure in order to obtain reliable ultrasound-guided images for puncture guidance.

[0007] Therefore, how to provide an ultrasound-guided puncture technique with intelligent adjustment of ultrasound frequency to provide puncture guidance is a technical problem that needs to be solved in this field. Summary of the Invention

[0008] The purpose of this application is to provide a puncture visualization method based on ultrasound imaging, so as to achieve ultrasound visualization guidance for puncture by intelligently and in real time adjusting the ultrasound frequency.

[0009] To achieve the above objectives, the embodiments of this application are implemented in the following manner:

[0010] In a first aspect, embodiments of this application provide a puncture visualization method based on ultrasound imaging, comprising: acquiring a low-frequency ultrasound image and a real-time ultrasound image at the current ultrasound frequency, wherein the ultrasound frequency of each stage includes the low-frequency ultrasound frequency and the real-time ultrasound frequency of that stage, and the low-frequency ultrasound image and the real-time ultrasound image of the same stage have the same size and correspond one-to-one pixels; determining the needle tip position based on the real-time ultrasound image of the current stage, and fusing the real-time ultrasound image of the current stage with the low-frequency ultrasound image to obtain a fused ultrasound image of the current stage; determining the tissue type on the current puncture path based on the fused ultrasound image of the current stage; and determining the ultrasound frequency of the next stage based on the needle tip position and the tissue type on the current puncture path, for controlling the ultrasound emission frequency of the ultrasound probe in the next stage.

[0011] In conjunction with the first aspect, in the first possible implementation of the first aspect, determining the needle tip position based on the real-time ultrasound image of the current stage includes: preprocessing the real-time ultrasound image of the current stage and inputting it into a preset needle tip position recognition model, identifying the needle tip position in the real-time ultrasound image of the current stage through the needle tip position recognition model, and outputting the needle tip position.

[0012] In conjunction with the first aspect, in a second possible implementation of the first aspect, the real-time ultrasound image of the current stage is fused with the low-frequency ultrasound image to obtain a fused ultrasound image of the current stage, including: pixel registration of the real-time ultrasound image of the current stage with the low-frequency ultrasound image; and pixel fusion of the real-time ultrasound image of the current stage and the low-frequency ultrasound image based on the needle tip position to obtain a fused ultrasound image of the current stage.

[0013] In conjunction with the second possible implementation of the first aspect, in the third possible implementation of the first aspect, pixel fusion is performed on the real-time ultrasound image and the low-frequency ultrasound image of the current stage based on the needle tip position to obtain the fused ultrasound image of the current stage, including: determining the weight index corresponding to each pixel based on the pixel coordinates of the needle tip position in the real-time ultrasound image of the current stage; and performing pixel fusion on the real-time ultrasound image and the low-frequency ultrasound image of the current stage based on the weight index of each pixel to obtain the fused ultrasound image of the current stage.

[0014] In conjunction with the third possible implementation of the first aspect, in the fourth possible implementation of the first aspect, the weight index corresponding to each pixel is determined based on the pixel coordinates of the needle tip position in the real-time ultrasound image at the current stage, including: determining the anti-attenuation parameter based on the real-time ultrasound frequency at the current stage; and calculating the weight index corresponding to each pixel by attenuating outward from the pixel coordinates of the needle tip position in the real-time ultrasound image at the current stage in combination with the anti-attenuation parameter.

[0015] In conjunction with the fourth possible implementation of the first aspect, in the fifth possible implementation of the first aspect, the weight index corresponding to each pixel is calculated by taking the pixel coordinates of the needle tip position in the real-time ultrasound image of the current stage as the center and combining the anti-attenuation parameter for outward attenuation. This includes: taking the pixel coordinates of the needle tip position in the real-time ultrasound image of the current stage as the center, denoted as p (0,0) Let p′ be the pixel coordinates of the corresponding needle tip position in the low-frequency ultrasound image of the current stage. (0,0) Convert the pixel coordinates of each pixel in the real-time ultrasound image to p (0,0) Let p be the plane coordinate of the origin. (i,j) ; Convert the pixel coordinates of each pixel in the low-frequency ultrasound image to p′ (0,0) The plane coordinates of the origin are denoted as p′. (i,j) The weight index corresponding to each pixel in a real-time ultrasound image is calculated using the following formula:

[0016]

[0017] in, For pixel p (i,j) The corresponding weighting exponent, ρ, is the anti-decay parameter, |d(p (i,j) ,p (0,0) | is for pixel p (i,j) With the origin p (0,0) The absolute value of the distance between them, d max The maximum distance between two pixels in a real-time ultrasound image is given. The weight index for each pixel in a low-frequency ultrasound image is calculated using the following formula:

[0018]

[0019] in, For pixel p′ (i,j) The corresponding weighting index.

[0020] In conjunction with the fifth possible implementation of the first aspect, in the sixth possible implementation of the first aspect, pixel fusion is performed on the real-time ultrasound image and the low-frequency ultrasound image of the current stage based on the weight index of each pixel, including: calculating pixel p using the following formula. (i,j)pixel value g (i,j) :

[0021]

[0022] Among them, f (i,j) The merged pixel value, i.e., pixel p″ in the merged image. (i,j) pixel value, g (i,j) For pixel p (i,j) The pixel value, g′ (i,j) For pixel p′ (i,j) The pixel value.

[0023] In conjunction with the first aspect, in the seventh possible implementation of the first aspect, the tissue type on the current puncture path is determined based on the fused ultrasound image at the current stage, including: preprocessing the fused ultrasound image at the current stage and inputting it into a preset tissue type recognition model, identifying the tissue type in the fused ultrasound image at the current stage through the tissue type recognition model, and determining the tissue type in the fused ultrasound image; determining and outputting the tissue type on the current puncture path based on the needle tip position and puncture target point.

[0024] In conjunction with the first aspect, in the eighth possible implementation of the first aspect, the ultrasound frequency of the next stage is determined based on the needle tip position and the tissue type on the current puncture path, including: calculating the interval distance between the needle tip position and the nearest target tissue type on the current puncture path; determining whether the interval distance is lower than a threshold; if so, determining the real-time ultrasound frequency of the next stage based on the target tissue type, and combining it with the low-frequency ultrasound frequency to determine the ultrasound frequency of the next stage; if not, determining the real-time ultrasound frequency of the next stage based on the current tissue type where the needle tip position is located, and combining it with the low-frequency ultrasound frequency to determine the ultrasound frequency of the next stage.

[0025] In the ninth possible implementation of the first aspect, in combination with the first possible implementation of the first aspect, or in combination with the seventh possible implementation of the first aspect, the needle tip location recognition model is trained using a key point detection network, and the tissue type recognition model is trained using a lightweight convolutional neural network model.

[0026] Beneficial effects:

[0027] 1. By acquiring low-frequency ultrasound images and real-time ultrasound images at the current ultrasound frequency (each stage's ultrasound frequency includes both low-frequency and real-time ultrasound frequencies for that stage, with images of the same size and corresponding pixels), the needle tip position is determined based on the real-time ultrasound image. The real-time and low-frequency ultrasound images are then fused to obtain a fused ultrasound image for the current stage. This fused ultrasound image serves as a visual guide for puncture, providing a reference for medical personnel. Based on this image, the tissue type along the current puncture path is determined, and the ultrasound frequency for the next stage is determined based on the needle tip position and the tissue type along the current puncture path. This allows for control of the ultrasound probe's emission frequency in the next stage. This provides a continuous stream of fused images as a visual reference during puncture guidance. The real-time acquisition of low-frequency and real-time ultrasound images (usually high-frequency or mid-frequency) ensures real-time updates of information during the puncture process. Real-time identification and positioning of the needle tip improves the accuracy and safety of the puncture. The good penetration of low-frequency ultrasound images (e.g., 2-5 MHz) helps to observe deep tissues, while the high resolution of real-time ultrasound images (e.g., 5-10 MHz, or 10 MHz and above) clearly displays superficial tissues or fine structures. The fused image combines the advantages of both, providing more comprehensive tissue information. Using the needle tip location as a reference, it provides fine structural information about the area adjacent to the needle tip (including the tissue type of the area, and usually also the next or more tissue types along the current puncture path), providing clear ultrasound images for puncture guidance. Pixel-level fusion ensures the consistency and accuracy of information from both low-frequency and real-time ultrasound images in the fused ultrasound image, reducing image artifacts and interference caused by frequency switching. Based on the fused ultrasound image, the (partial or complete) tissue type along the current puncture path can be determined in real time (the entire puncture guidance process can provide information on all tissue types through information from each stage), providing physicians with richer puncture reference information. The ultrasound frequency is automatically adjusted for the next stage based on the needle tip position and tissue type, enabling intelligent control of the puncture process, reducing the doctor's workload, and helping the doctor complete the puncture operation more quickly and accurately.

[0028] 2. In the visualization process of ultrasound-guided puncture, a pixel fusion scheme was designed. During the fusion process, the needle tip position was considered, and an anti-attenuation parameter was introduced. Different weights were assigned to different pixels, achieving intelligent fusion of real-time ultrasound images and low-frequency ultrasound images. Centered on the needle tip position, the weight exponent of each pixel attenuates outwards, ensuring that the needle tip and its surrounding area receive higher weight in the fused image. This highlights the critical puncture area, providing a more reliable visualization scheme for puncture guidance. This ensures that information about the needle tip and its surrounding area is preserved and highlighted to the greatest extent in the fused image. This is crucial for ultrasound-guided puncture, helping doctors to see the relationship between the needle tip and surrounding tissue more clearly, thereby reducing puncture difficulty and improving accuracy and safety.

[0029] 3. Introducing anti-attenuation parameters allows for adjustment of the attenuation rate and degree according to actual needs (e.g., tissue type, ultrasound frequency), balancing the clarity and contrast of different areas in the image. Real-time ultrasound images have high resolution, clearly displaying tissue details, while low-frequency ultrasound images have better penetration into deep tissues, displaying them and suitable for overall reference. Through pixel-level fusion with assigned weights, the advantages of both ultrasound images are combined, providing more comprehensive and accurate tissue information, thus achieving better puncture guidance. Simultaneously, this pixel-level fusion method, by accurately calculating the weight index of each pixel and attenuating it according to its distance from the needle tip, ensures the continuity and consistency of the fused ultrasound image, reducing artifacts and interference caused by frequency switching or image superposition.

[0030] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0031] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 A flowchart of a puncture visualization method based on ultrasound imaging provided in this application embodiment.

[0033] Figure 2 A schematic diagram illustrating the process of fusing low-frequency ultrasound images and real-time ultrasound images to obtain a fused ultrasound image. Detailed Implementation

[0034] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0035] The ultrasound imaging-based puncture visualization method provided in this embodiment can run on an electronic device, which can be an ultrasound diagnostic instrument or a smart terminal (such as a computer, but requiring a GPU with higher performance than a typical home computer; NVIDIA Tesla or A100 series GPUs can be used to improve deep learning processing performance, and the number of GPUs can be increased to compress the overall processing time of the artificial intelligence model and fused ultrasound images involved in this solution to within 20 milliseconds). This solution is implemented by communicating with the ultrasound diagnostic instrument. This embodiment uses the communication between a smart terminal and an ultrasound diagnostic instrument as an example to illustrate this solution, but it is not limited to this example.

[0036] Please see Figure 1 , Figure 1 This is a flowchart illustrating a puncture visualization method based on ultrasound imaging, provided in an embodiment of this application. The puncture visualization method based on ultrasound imaging may include steps S10, S20, S30, and S40.

[0037] First, before the smart terminal runs step S10, some basic information about the puncture can be obtained, such as the puncture type (thoracentesis, paracentesis, joint puncture, intracranial puncture, spinal cord puncture, etc.), target point (the specific structure or abnormal site to be punctured), skin marker point (the position where the needle is inserted through the skin during puncture), puncture depth (the distance between the skin marker point and the target point), and puncture angle (the angle between the puncture needle and the skin plane where the skin marker point is located during puncture). Based primarily on the puncture type, the initial low-frequency ultrasound frequency (e.g., 4MHz) and real-time ultrasound frequency (e.g., 12MHz) are determined as the initial ultrasound frequency. The intelligent terminal can acquire low-frequency ultrasound images (the ultrasound image formed by the ultrasound diagnostic instrument based on the received signal after emitting an initial low-frequency ultrasound signal) and real-time ultrasound images (the ultrasound image formed by the ultrasound diagnostic instrument based on the received signal after emitting an initial real-time ultrasound signal) in this state (usually including multiple stages), and fuse them (the fusion process will be described in detail later; the difference is that the fusion process here is centered on the skin marker point, while the fusion process in the later text is centered on the needle tip position) to obtain a fused ultrasound image for the operator (medical staff) to refer to and operate the puncture needle for puncture. When the puncture needle has not penetrated into the skin, since the needle tip position cannot be detected, when determining the ultrasound frequency for the next stage, it is only necessary to emit an ultrasound signal according to the initial ultrasound frequency (including the initial low-frequency ultrasound frequency and the real-time ultrasound frequency).

[0038] Furthermore, in this embodiment, for each stage (which can be the display time of one frame of fused ultrasound image) both before and after needle insertion (before the needle is inserted into the skin), to ensure real-time performance, taking 50 frames as an example, each frame of fused ultrasound image corresponds to one stage, which is 0.02 seconds, or 20 milliseconds (of course, it can also be 30 frames, in which case the corresponding time would be 33 milliseconds, reducing the hardware requirements for the smart terminal). Current ultrasound diagnostic instruments can achieve pulse repetition frequency (PRF, representing the number of ultrasound pulses emitted per second) of thousands, while this solution only needs to emit a set of ultrasound signals of the corresponding frequency every 20 milliseconds. This set of ultrasound signals includes a low-frequency ultrasound signal and a real-time ultrasound signal. The transmission interval between the low-frequency and real-time ultrasound signals in the same set can be 1 millisecond (or 5 milliseconds, to avoid signal interference). The transmission interval of 20 milliseconds does not mean transmission every 20 milliseconds, but rather that each 20 millisecond period constitutes a phase. Assuming the transmission interval between the low-frequency ultrasonic signal and the real-time ultrasonic signal in the same phase is 5 milliseconds, then the low-frequency ultrasonic signal in this phase is transmitted at millisecond 0 (the start of this phase), and the real-time ultrasonic signal in this phase is transmitted at millisecond 5. The transmission interval between the low-frequency ultrasonic signal of the next phase and the real-time ultrasonic signal of the current phase is 15 milliseconds. For ease of explanation, transmission duration and other factors are not considered here; this is simply to facilitate understanding of the scheme.

[0039] Based on this, once the puncture needle is inserted into the skin at the puncture angle from the skin marker, the smart terminal can begin running step S10.

[0040] Step S10: Acquire low-frequency ultrasound images and real-time ultrasound images at the current ultrasound frequency. The ultrasound frequency of each stage includes the low-frequency ultrasound frequency and the real-time ultrasound frequency of that stage. The low-frequency ultrasound images and real-time ultrasound images of the same stage have the same size and correspond to each other in pixels.

[0041] In this embodiment, the smart terminal can acquire low-frequency ultrasound images and real-time ultrasound images at the current ultrasound frequency from the ultrasound diagnostic instrument. The ultrasound frequency of each stage includes the low-frequency ultrasound frequency and the real-time ultrasound frequency of that stage. The low-frequency ultrasound images and real-time ultrasound images of the same stage have the same size and correspond to each other in pixels.

[0042] After obtaining the low-frequency ultrasound image and real-time ultrasound image of the current stage, the smart terminal can run step S20.

[0043] Step S20: Determine the needle tip position based on the real-time ultrasound image of the current stage, and fuse the real-time ultrasound image of the current stage with the low-frequency ultrasound image to obtain the fused ultrasound image of the current stage.

[0044] In this embodiment, the smart terminal can determine the needle tip position based on the real-time ultrasound image at the current stage.

[0045] For example, the smart terminal can preprocess the real-time ultrasound image of the current stage and input it into a preset needle tip position recognition model. The needle tip position in the real-time ultrasound image of the current stage is identified by the needle tip position recognition model, and the needle tip position is output.

[0046] The needle tip location recognition model in this embodiment is trained using a keypoint detection network (PANet). First, real-time ultrasound images containing the needle tip are collected (to improve recognition speed, the real-time ultrasound image can be cropped based on the previously detected needle tip location; otherwise, it is cropped based on skin markers, which greatly reduces image size and improves recognition speed). The needle tip location is then labeled to form a training dataset (which can be split into training and testing sets). A keypoint detection network model architecture is built using a deep learning framework (e.g., Tensorflow), and initial model parameters, loss functions, and other relevant parameters are set. The training dataset is then imported for training, ultimately resulting in a trained needle tip location recognition model. Model training is a mature technology and will not be elaborated upon here. The trained needle tip location recognition model is lightweight and has high accuracy, with the time to identify the needle tip location controlled within milliseconds (6-8 milliseconds).

[0047] At the same time, the smart terminal can fuse the real-time ultrasound image of the current stage with the low-frequency ultrasound image to obtain the fused ultrasound image of the current stage.

[0048] In this embodiment, the smart terminal can perform pixel registration between the real-time ultrasound image and the low-frequency ultrasound image of the current stage. Since the low-frequency ultrasound image and the real-time ultrasound image of the same stage are the same size and have a one-to-one pixel correspondence, this registration takes almost no time. Then, based on the needle tip position, the real-time ultrasound image and the low-frequency ultrasound image of the current stage can be pixel-fused to obtain the fused ultrasound image of the current stage.

[0049] For example, the smart terminal can determine the weight index corresponding to each pixel based on the pixel coordinates of the needle tip position in the real-time ultrasound image at the current stage.

[0050] Specifically, the intelligent terminal can determine the anti-attenuation parameter based on the real-time ultrasound frequency of the current stage. This can be quickly determined by looking up a table, where each frequency range of real-time ultrasound corresponds to an anti-attenuation parameter, making the table-based determination of this parameter highly efficient. Then, using the pixel coordinates of the needle tip position in the real-time ultrasound image of the current stage as the center, and combining the anti-attenuation parameter to attenuate outwards, the weight index corresponding to each pixel can be calculated.

[0051] For example, the smart terminal can use the pixel coordinates of the needle tip position in the real-time ultrasound image at the current stage as the center, denoted as p. (0,0) Let p′ be the pixel coordinates of the corresponding needle tip position in the low-frequency ultrasound image of the current stage. (0,0) Simultaneously, the pixel coordinates of each pixel in the real-time ultrasound image are converted to p... (0,0) Let p be the plane coordinate of the origin. (i,j) ; Convert the pixel coordinates of each pixel in the low-frequency ultrasound image to p′ (0,0) The plane coordinates of the origin are denoted as p′. (i,j) Then, the smart terminal can use the following formula to calculate the weight index corresponding to each pixel in the real-time ultrasound image:

[0052]

[0053] in, For pixel p (i,j) The corresponding weighting exponent, ρ, is the anti-decay parameter, |d(p (i,j) ,p (0,0) | is for pixel p (i,j) With the origin p (0,0) The absolute value of the distance between them, d max This represents the maximum distance between two pixels in a real-time ultrasound image.

[0054] Furthermore, the smart terminal can use the following formula to calculate the weight index corresponding to each pixel in the low-frequency ultrasound image:

[0055]

[0056] in, For pixel p′ (i,j) The corresponding weighting index.

[0057] After determining the weight index of each pixel, the smart terminal can perform pixel fusion of the real-time ultrasound image and the low-frequency ultrasound image at the current stage based on the weight index of each pixel to obtain the fused ultrasound image at the current stage.

[0058] Specifically, smart terminals can use the following formula to calculate pixel p. (i,j) pixel value g(i,j) :

[0059]

[0060] Among them, f (i,j) The merged pixel value, i.e., pixel p″ in the merged image. (i,j) pixel value, g (i,j) For pixel p (i,j) The pixel value, g′ (i,j) For pixel p′ (i,j) The pixel value.

[0061] This allows for efficient and accurate fusion of current-stage low-frequency ultrasound images and real-time ultrasound images, resulting in a fused ultrasound image for the current stage (e.g., Figure 2 (As shown). After obtaining the fused ultrasound image of the current stage, the smart terminal can transmit the fused ultrasound image of the current stage to the ultrasound diagnostic instrument for display, so as to realize visualized puncture guidance.

[0062] Furthermore, after obtaining the fused ultrasound image at the current stage, the smart terminal can also run step S30.

[0063] Step S30: Based on the fused ultrasound images of the current stage, determine the tissue type along the current puncture path.

[0064] In this embodiment, the smart terminal can determine the tissue type along the current puncture path based on the fused ultrasound images at the current stage. This embodiment also uses an artificial intelligence model to efficiently accomplish this task.

[0065] For example, the smart terminal can preprocess the fused ultrasound image of the current stage and input it into a preset tissue type recognition model. The tissue type recognition model can then identify the tissue type in the fused ultrasound image of the current stage and determine the tissue type in the fused ultrasound image.

[0066] Tissue type recognition models can be trained using lightweight convolutional neural network models. The process involves collecting fused ultrasound images, labeling them with tissue types to create a training dataset (which can be split into training and test sets), building a Fast R-CNN architecture using a deep learning framework (such as Tensorflow), setting initial model parameters, loss functions, and other relevant parameters, importing the training dataset, and training the model to obtain the trained tissue type recognition model. The training process for tissue type recognition models is a mature technology and will not be elaborated upon here. The trained tissue type recognition model is lightweight, has high accuracy, and the time for tissue type recognition can be controlled within milliseconds (4-6 milliseconds).

[0067] After identifying the tissue type in the fused ultrasound image, the tissue type on the current puncture path (e.g., the straight line connecting the two) can be further determined and output based on the needle tip position and puncture target point.

[0068] After obtaining the tissue type along the current puncture path, the smart terminal can proceed to step S40.

[0069] Step S40: Based on the needle tip position and the tissue type along the current puncture path, determine the ultrasound frequency for the next stage, which is used to control the ultrasound transmission frequency of the ultrasound probe in the next stage.

[0070] In this embodiment, the smart terminal can determine the ultrasound frequency of the next stage based on the needle tip position and the tissue type on the current puncture path, and the ultrasound frequency of the next stage can be used as the ultrasound emission frequency of the ultrasound probe in the next stage.

[0071] For example, the smart terminal can calculate the distance between the needle tip position and the nearest target tissue type on the current puncture path. It then determines whether the distance is below a threshold (e.g., 10 mm).

[0072] If the interval distance is less than a threshold (e.g., 10 mm), the smart terminal can determine the real-time ultrasound frequency for the next stage based on the target tissue type (here, the target tissue type refers to the tissue type adjacent to the current tissue type on the current puncture path, while the current tissue type is the tissue type where the needle tip is located), and combine it with the low-frequency ultrasound frequency (which can be the initial low-frequency ultrasound frequency) to determine the ultrasound frequency for the next stage.

[0073] If the interval distance is not less than the threshold (e.g., 10 mm), the smart terminal can determine the real-time ultrasound frequency of the next stage based on the current tissue type where the needle tip is located, and combine it with the low-frequency ultrasound frequency (which can be the initial low-frequency ultrasound frequency) to determine the ultrasound frequency of the next stage.

[0074] This allows for the determination of the ultrasound frequency in the next stage, enabling intelligent control of the ultrasound frequency at each stage of the entire puncture process.

[0075] In summary, this application provides a puncture visualization method based on ultrasound imaging. It acquires low-frequency ultrasound images and real-time ultrasound images at the current ultrasound frequency (each stage's ultrasound frequency includes both low-frequency and real-time ultrasound frequencies) (low-frequency and real-time ultrasound images at the same stage have the same size and corresponding pixels). The needle tip position is determined based on the real-time ultrasound image of the current stage, and the real-time and low-frequency ultrasound images are fused to obtain a fused ultrasound image for the current stage. This fused ultrasound image can serve as a visually guiding ultrasound image for medical personnel, providing a reference for puncture. Furthermore, based on the fused ultrasound image of the current stage, the tissue type along the current puncture path is determined. Based on the needle tip position and the tissue type along the current puncture path, the ultrasound frequency for the next stage is determined, used to control the ultrasound probe's emission frequency in the next stage. This provides a continuous stream of fused images as a visual reference during puncture guidance, and the real-time acquisition of low-frequency and real-time ultrasound images (usually high-frequency or mid-frequency) ensures real-time updates of information during the puncture process. Real-time identification and positioning of the needle tip improves the accuracy and safety of puncture. The good penetration of low-frequency ultrasound images (e.g., 2-5 MHz) helps observe deep tissues, while the high resolution of real-time ultrasound images (e.g., 5-10 MHz, or 10 MHz and above) clearly displays superficial tissues or fine structures. The fused image combines the advantages of both, providing more comprehensive tissue information. Using the needle tip position as a reference, it provides fine structural information about the area adjacent to the needle tip (including the tissue type of the area, and usually also the next or more tissue types along the current puncture path), providing clear ultrasound images for puncture guidance. Pixel-level fusion ensures the consistency and accuracy of information from both low-frequency and real-time ultrasound images in the fused ultrasound image, reducing image artifacts and interference caused by frequency switching. Based on the fused ultrasound image, the (partial or complete) tissue types along the current puncture path can be determined in real time (the entire puncture guidance process can provide information on all tissue types through information from each stage), providing physicians with richer puncture reference information. The ultrasound frequency is automatically adjusted for the next stage based on the needle tip position and tissue type, enabling intelligent control of the puncture process, reducing the doctor's workload, and helping the doctor complete the puncture operation more quickly and accurately.

[0076] In the visualization process of ultrasound-guided puncture, a pixel fusion scheme was designed. During the fusion process, the needle tip position was considered, and an anti-attenuation parameter was introduced, assigning different weights to different pixels. This achieved intelligent fusion of real-time ultrasound images and low-frequency ultrasound images. Centered on the needle tip position, the weight exponent of each pixel attenuates outwards, ensuring that the needle tip and its surrounding area receive higher weight in the fused image. This highlights the critical puncture area, providing a more reliable visualization solution for puncture guidance. This ensures that information about the needle tip and its surrounding area is preserved and highlighted to the greatest extent possible in the fused image. This is crucial for ultrasound-guided puncture, helping doctors to see the relationship between the needle tip and surrounding tissue more clearly, thereby reducing puncture difficulty and improving accuracy and safety.

[0077] Introducing anti-attenuation parameters allows for adjustment of the attenuation rate and degree based on actual needs (e.g., tissue type, ultrasound frequency) to balance the clarity and contrast of different regions in the image. Real-time ultrasound images have high resolution, clearly displaying tissue details, while low-frequency ultrasound images have better penetration into deep tissues, displaying them and serving as a holistic reference. Through pixel-level fusion with assigned weights, the advantages of both ultrasound images are combined, providing more comprehensive and accurate tissue information, thus improving puncture guidance. Simultaneously, this pixel-level fusion method, by precisely calculating the weight index of each pixel and attenuating it according to its distance from the needle tip, ensures the continuity and consistency of the fused ultrasound image, reducing artifacts and interference caused by frequency switching or image overlay.

[0078] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An electronic device, characterized in that, The electronic device is an ultrasound diagnostic instrument or a smart terminal communicating with an ultrasound diagnostic instrument, used to implement a puncture visualization method based on ultrasound imaging. The puncture visualization method based on ultrasound imaging includes: Acquire low-frequency ultrasound images and real-time ultrasound images at the current ultrasound frequency. The ultrasound frequency of each stage includes the low-frequency ultrasound frequency and the real-time ultrasound frequency of that stage. The low-frequency ultrasound images and real-time ultrasound images of the same stage have the same size and correspond to each other in pixels. The needle tip position is determined based on the real-time ultrasound image of the current stage, and the real-time ultrasound image of the current stage is fused with the low-frequency ultrasound image to obtain the fused ultrasound image of the current stage. Based on the current fused ultrasound images, the tissue type along the current puncture path is determined; Based on the needle tip position and the tissue type along the current puncture path, the ultrasound frequency for the next stage is determined, which is used to control the ultrasound emission frequency of the ultrasound probe in the next stage. The current real-time ultrasound image is fused with the low-frequency ultrasound image to obtain the current fused ultrasound image, including: The real-time ultrasound image and the low-frequency ultrasound image of the current stage are pixel-registered; based on the needle tip position, the real-time ultrasound image and the low-frequency ultrasound image of the current stage are pixel-fused to obtain the fused ultrasound image of the current stage. Based on the needle tip position, pixel fusion is performed on the current real-time ultrasound image and the low-frequency ultrasound image to obtain the current fused ultrasound image, including: Based on the pixel coordinates of the needle tip position in the real-time ultrasound image of the current stage, the weight index corresponding to each pixel is determined; based on the weight index of each pixel, the real-time ultrasound image and the low-frequency ultrasound image of the current stage are fused pixel by pixel to obtain the fused ultrasound image of the current stage. Based on the pixel coordinates of the needle tip position in the real-time ultrasound image at the current stage, the weight index corresponding to each pixel is determined, including: Based on the current real-time ultrasound frequency, the anti-attenuation parameter is determined; taking the pixel coordinates of the needle tip position in the current real-time ultrasound image as the center, and combining the anti-attenuation parameter to attenuate outwards, the weight index corresponding to each pixel is calculated. Based on the needle tip location and the tissue type along the current puncture path, the ultrasound frequency for the next stage is determined, including: Calculate the distance between the needle tip position and the nearest target tissue type on the current puncture path; determine if the distance is below a threshold; if so, determine the real-time ultrasound frequency for the next stage based on the target tissue type, and combine it with the low-frequency ultrasound frequency to determine the ultrasound frequency for the next stage; if not, determine the real-time ultrasound frequency for the next stage based on the current tissue type where the needle tip is located, and combine it with the low-frequency ultrasound frequency to determine the ultrasound frequency for the next stage.

2. The electronic device according to claim 1, characterized in that, The needle tip position is determined based on the current real-time ultrasound images, including: After preprocessing the real-time ultrasound image of the current stage, it is input into the preset needle tip position recognition model. The needle tip position in the real-time ultrasound image of the current stage is identified by the needle tip position recognition model, and the needle tip position is output.

3. The electronic device according to claim 1, characterized in that, Centered on the pixel coordinates of the needle tip position in the real-time ultrasound image at the current stage, and considering the attenuation resistance parameter, the weight index corresponding to each pixel is calculated outwards, including: Centered on the pixel coordinates of the needle tip position in the real-time ultrasound image at the current stage, let it be denoted as... The pixel coordinates corresponding to the needle tip position in the current low-frequency ultrasound image are marked as... ; The pixel coordinates of each pixel in the real-time ultrasound image are converted to... The plane coordinates of the origin are denoted as . ; The pixel coordinates of each pixel in the low-frequency ultrasound image are converted to... The plane coordinates of the origin are denoted as . ; The weight index for each pixel in a real-time ultrasound image is calculated using the following formula: ， in, For pixels The corresponding weight index For anti-attenuation parameters, For pixels and the origin The absolute value of the distance between them This represents the maximum distance between two pixels in a real-time ultrasound image. The weight index for each pixel in a low-frequency ultrasound image is calculated using the following formula: ， in, For pixels The corresponding weighting index.

4. The electronic device according to claim 3, characterized in that, Based on the weight index of each pixel, the real-time ultrasound image and the low-frequency ultrasound image of the current stage are fused pixel by pixel, including: Pixels are calculated using the following formula. pixel values : ， in, The merged pixel values, i.e., the pixels in the merged image. pixel values, For pixels pixel values, For pixels The pixel value.

5. The electronic device according to claim 1, characterized in that, Based on the current fused ultrasound images, the tissue types along the current puncture path are determined, including: After preprocessing the fused ultrasound image of the current stage, it is input into the preset tissue type recognition model. The tissue type recognition model identifies the tissue type in the fused ultrasound image of the current stage and determines the tissue type in the fused ultrasound image. Based on the needle tip position and puncture target point, the tissue type along the current puncture path is determined and output.

6. The electronic device according to claim 2 or 5, characterized in that, The needle tip location recognition model was trained using a key point detection network, while the tissue type recognition model was trained using a lightweight convolutional neural network model.

Images