Puncture visualization method based on ultrasonic imaging

Abstract

The invention provides a puncture visualization method based on ultrasonic imaging, and the method comprises the steps: obtaining a low-frequency ultrasonic image and a real-time ultrasonic image under the ultrasonic frequency of a current stage, and the ultrasonic frequency of each stage comprises the low-frequency ultrasonic frequency and the real-time ultrasonic frequency of the stage; the low-frequency ultrasonic image and the real-time ultrasonic image at the same stage have the same size and pixels are in one-to-one correspondence; determining the position of a needle point based on the real-time ultrasonic image of the current stage, and fusing the real-time ultrasonic image of the current stage with the low-frequency ultrasonic image to obtain a fused ultrasonic image of the current stage; determining the tissue type on the current puncture path based on the fusion ultrasonic image at the current stage; and determining the ultrasonic frequency of the next stage based on the position of the needle tip and the tissue type on the current puncture path. By means of the mode, intelligent control over the puncture process is achieved, the operation burden of doctors is relieved, and the doctors can complete puncture operation more quickly and more accurately.

Description

Technical Field

[0001] The present application relates to the technical field of ultrasound-guided puncture, and in particular to a puncture visualization method based on ultrasound imaging. Background Art

[0002] Ultrasound imaging technology is a common and important imaging technology in clinical practice. It has become the second most commonly used diagnostic imaging method in medicine (second only to traditional X-rays) and is a vital diagnostic tool in many medical institutions.

[0003] The use of ultrasound to achieve visual guidance of puncture is currently widely used in various medical fields, such as the diagnosis and treatment of tumors, pancreatic diseases, thyroid diseases, and vascular puncture. This type of technology uses ultrasound imaging to display the position and path of the puncture needle in real time, guiding the doctor to accurately insert the puncture needle into the target area, thereby improving the accuracy and safety of the puncture.

[0004] However, the existing ultrasound-guided puncture technology also has some drawbacks. Some are limited by some defects of the ultrasound-guided puncture technology itself, while others require doctors to have a high level of operation, otherwise it may seriously affect the quality of ultrasound images. For example, due to different ultrasound frequencies, different tissues have different imaging effects. The reason is that different tissue types have different absorption and reflectivity of ultrasound frequencies, which leads to differences in imaging. Low-frequency ultrasound is more suitable for penetrating deep tissues, so imaging focuses on overall imaging, while high-frequency ultrasound (different tissue types and depths are suitable for different ultrasound frequencies) focuses more on clear imaging of local areas.

[0005] When medical staff currently use ultrasound-guided puncture technology to guide puncture, they usually need the operator to manually change the ultrasound frequency to guide the puncture. This method places high demands on the operator and has a relatively low guidance efficiency.

[0006] Currently, there are also people combining artificial intelligence technology with ultrasound puncture guidance, but their main research direction is to collect a large amount of ultrasound puncture guidance operation data and combine it with expert annotation data to develop an intelligent ultrasound frequency control strategy for the entire process. The technical solutions in this research direction are not very adaptable to different puncture types and individual differences. Operators usually still need to rely on their own experience to adjust the ultrasound frequency during the local process in order to obtain reliable ultrasound guidance images for puncture guidance.

[0007] Therefore, how to provide an ultrasound-guided puncture technology that can intelligently adjust the ultrasound frequency to provide puncture guidance is a technical problem that needs to be solved in this field. Summary of the invention

[0008] The purpose of the embodiments of the present application is to provide a puncture visualization method based on ultrasound imaging, so as to achieve ultrasound visualization guidance of puncture by intelligently and real-time adjustment of ultrasound frequency.

[0009] In order to achieve the above purpose, the embodiments of the present application are implemented in the following manner:

[0010] In a first aspect, an embodiment of the present application provides a puncture visualization method based on ultrasonic imaging, comprising: obtaining a low-frequency ultrasonic image and a real-time ultrasonic image at the ultrasonic frequency of the current stage, wherein the ultrasonic frequency of each stage includes the low-frequency ultrasonic frequency and the real-time ultrasonic frequency of the stage, and the low-frequency ultrasonic image and the real-time ultrasonic image of the same stage have the same size and one-to-one pixel correspondence; determining the needle tip position based on the real-time ultrasonic image of the current stage, and fusing the real-time ultrasonic image of the current stage with the low-frequency ultrasonic image to obtain a fused ultrasonic image of the current stage; determining the tissue type on the current puncture path based on the fused ultrasonic image of the current stage; determining the ultrasonic frequency of the next stage based on the needle tip position and the tissue type on the current puncture path, which is used to control the ultrasonic emission frequency of the ultrasonic probe in the next stage.

[0011] In combination with the first aspect, in a first possible implementation of the first aspect, the needle tip position is determined based on the real-time ultrasound image of the current stage, including: preprocessing the real-time ultrasound image of the current stage, inputting it into a preset needle tip position recognition model, identifying the needle tip position in the real-time ultrasound image of the current stage through the needle tip position recognition model, and outputting the needle tip position.

[0012] In combination with the first aspect, in a second possible implementation of the first aspect, the real-time ultrasound image of the current stage is fused with the low-frequency ultrasound image to obtain a fused ultrasound image of the current stage, including: pixel-aligning the real-time ultrasound image of the current stage with the low-frequency ultrasound image; and based on the needle tip position, pixel-fusing the real-time ultrasound image of the current stage and the low-frequency ultrasound image to obtain a fused ultrasound image of the current stage.

[0013] In combination with the second possible implementation method of the first aspect, in a third possible implementation method of the first aspect, based on the needle tip position, the real-time ultrasound image and the low-frequency ultrasound image of the current stage are pixel-fused to obtain a fused ultrasound image of the current stage, including: determining the weight index corresponding to each pixel based on the pixel coordinates of the needle tip position in the real-time ultrasound image of the current stage; based on the weight index of each pixel, performing pixel-fusion on the real-time ultrasound image and the low-frequency ultrasound image of the current stage to obtain a fused ultrasound image of the current stage.

[0014] In combination with the third possible implementation method of the first aspect, in a fourth possible implementation method of the first aspect, a weight index corresponding to each pixel is determined based on the pixel coordinates of the needle tip position in the real-time ultrasound image at the current stage, including: determining an anti-attenuation parameter based on the real-time ultrasound frequency at the current stage; taking the pixel coordinates of the needle tip position in the real-time ultrasound image at the current stage as the center, and attenuating outward in combination with the anti-attenuation parameter, to calculate the weight index corresponding to each pixel.

[0015] In combination with the fourth possible implementation manner of the first aspect, in a fifth possible implementation manner of the first aspect, taking the pixel coordinates of the needle tip position in the real-time ultrasound image at the current stage as the center, combining the anti-attenuation parameter to decay outward, and calculating the weight index corresponding to each pixel, including: taking the pixel coordinates of the needle tip position in the real-time ultrasound image at the current stage as the center, denoted as p (0,0) , the pixel coordinates corresponding to the needle tip position in the current low-frequency ultrasound image are marked as p′ (0,0) ; Convert the pixel coordinates of each pixel in the real-time ultrasound image into p (0,0) is the plane coordinate of the origin, denoted by p (i,j) ; Convert the pixel coordinates of each pixel in the low-frequency ultrasound image to p′ (0,0) is the plane coordinate of the origin, denoted by p′ (i,j) ; The weight index corresponding to each pixel in the real-time ultrasound image is calculated using the following formula:

[0016]

[0017] in, is pixel p (i,j) The corresponding weight index, ρ is the anti-attenuation parameter, |d(p (i,j) ,p (0,0) )| is pixel p (i,j) With the origin p (0,0) The absolute value of the distance between max is the maximum value of the distance between two pixels in the real-time ultrasound image; the weight index corresponding to each pixel in the low-frequency ultrasound image is calculated using the following formula:

[0018]

[0019] in, is pixel p′ (i,j) The corresponding weight index.

[0020] In combination with the fifth possible implementation manner of the first aspect, in a sixth possible implementation manner of the first aspect, based on the weight index of each pixel, the real-time ultrasound image and the low-frequency ultrasound image at the current stage are pixel-fused, including: using the following formula to calculate the pixel p (i,j)The pixel value g (i,j) :

[0021]

[0022] Among them, f (i,j) is the pixel value after fusion, that is, the pixel p″ in the fused image (i,j) The pixel value, g (i,j) is pixel p (i,j) The pixel value, g′ (i,j) is pixel p′ (i,j) The pixel value of .

[0023] In combination with the first aspect, in a seventh possible implementation of the first aspect, based on the fused ultrasound image at the current stage, determining the tissue type on the current puncture path, including: preprocessing the fused ultrasound image at the current stage, inputting it into a preset tissue type recognition model, identifying the tissue type in the fused ultrasound image at the current stage through the tissue type recognition model, and determining the tissue type in the fused ultrasound image; based on the needle tip position and the puncture target, determining the tissue type on the current puncture path and outputting it.

[0024] In combination with the first aspect, in an eighth possible implementation of the first aspect, the ultrasonic frequency of the next stage is determined based on the needle tip position and the tissue type on the current puncture path, including: calculating the interval distance between the needle tip position and the nearest target tissue type on the current puncture path; judging whether the interval distance is lower than a threshold; if so, determining the real-time ultrasonic frequency of the next stage based on the target tissue type, and determining the ultrasonic frequency of the next stage in combination with the low-frequency ultrasonic frequency; if not, determining the real-time ultrasonic frequency of the next stage based on the current tissue type at the needle tip position, and determining the ultrasonic frequency of the next stage in combination with the low-frequency ultrasonic frequency.

[0025] In combination with the first possible implementation of the first aspect, or in combination with the seventh possible implementation of the first aspect, in a ninth possible implementation of the first aspect, the needle tip position recognition model is obtained by training a key point detection network, and the tissue type recognition model is obtained by training a lightweight convolutional neural network model.

[0026] Beneficial effects:

[0027] 1. By obtaining the low-frequency ultrasound image and real-time ultrasound image (the low-frequency ultrasound image and real-time ultrasound image of the same stage have the same size and one-to-one pixel correspondence) under the ultrasound frequency of the current stage (the ultrasound frequency of each stage includes the low-frequency ultrasound frequency and the real-time ultrasound frequency of the stage); the needle tip position is determined based on the real-time ultrasound image of the current stage, and the real-time ultrasound image of the current stage is fused with the low-frequency ultrasound image to obtain the fused ultrasound image of the current stage. The fused ultrasound image of the current stage can be used as an ultrasound image for visually guiding puncture, providing a reference for puncture for medical staff. Based on the fused ultrasound image of the current stage, the tissue type on the current puncture path is determined, and then the ultrasound frequency of the next stage is determined based on the needle tip position and the tissue type on the current puncture path, which is used to control the ultrasound emission frequency of the ultrasound probe in the next stage. In this way, a continuous fused image can be provided as a visual reference in the puncture guidance operation, and the low-frequency ultrasound image and real-time ultrasound image (usually high frequency or medium frequency) obtained in real time ensure the real-time update of information during the puncture process. The real-time identification and positioning of the needle tip position improves the accuracy and safety of the puncture. The good penetration of low-frequency ultrasound images (such as 2-5MHz) helps to observe deep tissues, while the high resolution of real-time ultrasound images (such as medium-frequency ultrasound, such as 5-10MHz, and high-frequency ultrasound, such as 10MHz or above) can clearly display superficial tissues or fine structures. The fused image combines the advantages of both and provides more comprehensive tissue information. With the needle tip position as a reference, the fine structure of the area adjacent to the needle tip position (including the tissue type of the part where it is located, and usually also including the next tissue type or more tissue types on the current puncture path) can be provided, providing a clear ultrasound image for puncture guidance. The pixel-level fusion ensures the consistency and accuracy of the information from the low-frequency ultrasound image and the real-time ultrasound image in the fused ultrasound image, and reduces the image artifacts and interference caused by frequency switching. Based on the fused ultrasound image, the (partial or all) tissue type on the current puncture path can be determined in real time (the entire puncture guidance process can provide all tissue types through information at each stage), providing doctors with richer puncture reference information. The ultrasonic frequency of the next stage is automatically adjusted according to the needle tip position and tissue type, realizing intelligent control of the puncture process, reducing the doctor's operating burden and helping the doctor to complete the puncture operation more quickly and accurately.

[0028] 2. In the visualization process of ultrasound-guided puncture, a pixel fusion scheme is designed. The needle tip position is considered in the fusion process and anti-attenuation parameters are introduced. Different weights are assigned to different pixels, realizing the intelligent fusion of real-time ultrasound images and low-frequency ultrasound images. With the needle tip position as the center, the weight index of each pixel decays outward, which can ensure that the needle tip and its surrounding area receive higher weights in the fused image, thereby highlighting the key areas of puncture and providing a more reliable visualization scheme for puncture guidance. This ensures that the information of the needle tip and its surrounding area in the fused image is retained and highlighted to the greatest extent. This is crucial for ultrasound-guided puncture, helping doctors to see the relationship between the needle tip and surrounding tissues more clearly, thereby reducing the difficulty of puncture and improving the accuracy and safety of puncture.

[0029] 3. The introduction of anti-attenuation parameters allows the speed and degree of attenuation to be adjusted according to actual needs (such as tissue type, ultrasound frequency, etc.) to balance the clarity and contrast of different areas in the image. Real-time ultrasound images have higher resolution and can clearly display the details of tissue types, while low-frequency ultrasound images have better penetration into deep tissues, can display deep tissues, and are suitable for overall reference. Through pixel-level fusion after weighting, the advantages of the two ultrasound images are combined to provide more comprehensive and accurate tissue information, thereby playing a better role in puncture guidance. At the same time, this pixel-level fusion method ensures the continuity and consistency of the fused ultrasound image by accurately calculating the weight index of each pixel and attenuating it according to its distance from the needle tip position, reducing artifacts and interference caused by frequency switching or image superposition.

[0030] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 A flowchart of a puncture visualization method based on ultrasound imaging is provided in an embodiment of the present application.

[0033] Figure 2 This is a schematic diagram of obtaining a fused ultrasound image by fusing a low-frequency ultrasound image with a real-time ultrasound image. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0035] The puncture visualization method based on ultrasound imaging provided in this embodiment can be run on an electronic device, which can be an ultrasound diagnostic instrument or an intelligent terminal (such as a computer, but the GPU performance is required to be higher than that of an ordinary home computer. NVIDIA's Tesla or A100 series GPU can be used to improve the processing performance of deep learning, and the number of GPUs can be increased so that the overall processing time of the artificial intelligence model and fused ultrasound image involved in this solution can be compressed to within 20 milliseconds). By communicating with the ultrasound diagnostic instrument, this solution is implemented. This embodiment takes the communication between the intelligent terminal and the ultrasound diagnostic instrument to cooperate in implementing this solution as an example for explanation, but it is not limited to this.

[0036] See also Figure 1 , Figure 1 A flowchart of a puncture visualization method based on ultrasound imaging provided in an embodiment of the present application. The puncture visualization method based on ultrasound imaging may include step S10, step S20, step S30, and step S40.

[0037] First, before the intelligent terminal runs step S10, some basic information of the puncture can be obtained, such as the puncture type (thoracic puncture, abdominal puncture, joint cavity puncture, intracranial puncture, spinal puncture, etc.), target (specific structure or abnormal part to be punctured), skin marker (the position of insertion from the skin during puncture), puncture depth (the distance between the skin marker and the target) and puncture angle (the angle between the puncture needle and the skin plane where the skin marker is located during puncture), etc., mainly based on the puncture type to determine the initial low-frequency ultrasonic frequency (such as 4Mhz) and real-time ultrasonic frequency (such as 12Mhz) as the initial ultrasonic frequency The intelligent terminal can obtain the low-frequency ultrasonic image (the ultrasonic image formed based on the received signal after the ultrasonic diagnostic instrument transmits the initial low-frequency ultrasonic signal) and the real-time ultrasonic image (the ultrasonic image formed based on the received signal after the ultrasonic diagnostic instrument transmits the initial real-time ultrasonic signal) of this state (usually including multiple stages), and perform fusion (the fusion process will be described in detail later, the difference is that the fusion process here is centered on the skin marking point, while the fusion process in the following text is centered on the needle tip position) to obtain a fused ultrasonic image for the operator (medical staff) to refer to and operate the puncture needle for puncture. When the puncture needle has not penetrated the skin, the needle tip position cannot be detected. Therefore, when determining the ultrasonic frequency of the next stage, it is only necessary to transmit the ultrasonic signal according to the initial ultrasonic frequency (including the initial low-frequency ultrasonic frequency and the real-time ultrasonic frequency).

[0038] In addition, in this embodiment, whether it is each stage before needle insertion (before the puncture needle is inserted into the skin) or after needle insertion (after the puncture needle is inserted into the skin) (a stage can be the display time of a frame of fused ultrasound image), in order to ensure real-time performance, taking 50 frames as an example, each frame of fused ultrasound image corresponds to a stage, that is, 0.02 seconds, that is, 20 milliseconds (of course, 30 frames can also be used, and the corresponding time can be 33 milliseconds, reducing the hardware requirements for the smart terminal). The current ultrasound diagnostic instrument has a pulse repetition frequency (Pulse Repetition Frequency, PRF, indicating the number of times the ultrasound diagnostic instrument transmits ultrasonic pulses per second) of up to thousands, and this solution only needs to transmit a group of ultrasound signals of the ultrasound frequency of this stage every 20 milliseconds, and a group of ultrasound signals of ultrasound frequency includes an ultrasound signal of a low-frequency ultrasound frequency and an ultrasound signal of a real-time ultrasound frequency. The transmission interval between the ultrasound signal of the low-frequency ultrasound frequency and the ultrasound signal of the real-time ultrasound frequency in the same group can be 1 millisecond (or 5 milliseconds to avoid signal interference). The transmission of a group every 20 milliseconds does not mean that the transmission is every 20 milliseconds, but that the transmission interval between the ultrasonic signal of the low-frequency ultrasonic frequency and the ultrasonic signal of the real-time ultrasonic frequency in the same group is 5 milliseconds, then the ultrasonic signal of the low-frequency ultrasonic frequency in this stage is at the 0th millisecond (that is, the beginning of this stage), and the ultrasonic signal of the real-time ultrasonic frequency in this stage is at the 5th millisecond; and the transmission interval between the ultrasonic signal of the low-frequency ultrasonic frequency in the next stage and the ultrasonic signal of the real-time ultrasonic frequency in this stage is 15 milliseconds. For the sake of convenience of explanation, factors such as the transmission duration are not considered here, just to facilitate the understanding of this solution.

[0039] Based on this, the smart terminal can start running step S10 until the puncture needle pierces the skin from the skin marking point according to the puncture angle.

[0040] Step S10: Acquire a low-frequency ultrasonic image and a real-time ultrasonic image at the ultrasonic frequency of the current stage, wherein the ultrasonic frequency of each stage includes the low-frequency ultrasonic frequency and the real-time ultrasonic frequency of the stage, and the low-frequency ultrasonic image and the real-time ultrasonic image of the same stage have the same size and one-to-one pixel correspondence.

[0041] In this embodiment, the intelligent terminal can obtain low-frequency ultrasonic images and real-time ultrasonic images at the ultrasonic frequency of the current stage from the ultrasonic diagnostic instrument. The ultrasonic frequency of each stage includes the low-frequency ultrasonic frequency and real-time ultrasonic frequency of that stage. The low-frequency ultrasonic image and real-time ultrasonic image of the same stage have the same size and one-to-one pixel correspondence.

[0042] After obtaining the low-frequency ultrasound image and the real-time ultrasound image of the current stage, the smart terminal may execute step S20.

[0043] Step S20: determining the needle tip position based on the real-time ultrasound image at the current stage, and fusing the real-time ultrasound image at the current stage with the low-frequency ultrasound image to obtain a fused ultrasound image at the current stage.

[0044] In this embodiment, the intelligent terminal can determine the needle tip position based on the real-time ultrasound image at the current stage.

[0045] Exemplarily, the intelligent terminal may preprocess the real-time ultrasound image of the current stage and input it into a preset needle tip position recognition model, and use the needle tip position recognition model to recognize the needle tip position in the real-time ultrasound image of the current stage, thereby outputting the needle tip position.

[0046] The needle tip position recognition model in this embodiment is obtained by training with a key point detection network (PANet). First, a real-time ultrasound image containing the needle tip of the puncture needle is collected (in order to improve the recognition speed, the real-time ultrasound image can be cut based on the needle tip position detected last time, and if there is no real-time ultrasound image, the real-time ultrasound image can be cut based on the skin marker point, which can greatly reduce the image size and improve the recognition speed), and the needle tip position is annotated to form a training data set (which can be divided into a training set and a test set), and a key point detection network model architecture is built using a deep learning framework (such as Tensorflow), and the initial model parameters, loss function and other related parameters are set, and the training data set is imported for training, and finally a trained needle tip position recognition model is obtained. Model training is a mature technology at present, and will not be described in detail here. The needle tip position recognition model obtained by training is lightweight and has a high accuracy rate. The time used to identify the needle tip position can be controlled at the millisecond level (6-8 milliseconds).

[0047] At the same time, the intelligent terminal can fuse the real-time ultrasound image of the current stage with the low-frequency ultrasound image to obtain the fused ultrasound image of the current stage.

[0048] In this embodiment, the intelligent terminal can perform pixel registration of the real-time ultrasound image at the current stage with the low-frequency ultrasound image. Since the low-frequency ultrasound image and the real-time ultrasound image at the same stage have the same size and one-to-one pixel correspondence, this registration takes almost no time. Then, based on the needle tip position, the real-time ultrasound image and the low-frequency ultrasound image at the current stage can be pixel-fused to obtain a fused ultrasound image at the current stage.

[0049] Exemplarily, the intelligent terminal may determine the weight index corresponding to each pixel based on the pixel coordinates of the needle tip position in the real-time ultrasound image at the current stage.

[0050] Specifically, the intelligent terminal can determine the anti-attenuation parameter based on the real-time ultrasound frequency at the current stage. This can be quickly determined by looking up a table. The real-time ultrasound frequency in each frequency range corresponds to an anti-attenuation parameter. The anti-attenuation parameter can be determined very efficiently by looking up a table. Then, the pixel coordinates of the needle tip position in the real-time ultrasound image at the current stage can be used as the center, and the anti-attenuation parameter can be combined to attenuate outward to calculate the weight index corresponding to each pixel.

[0051] For example, the intelligent terminal can take the pixel coordinates of the needle tip position in the real-time ultrasound image at the current stage as the center, denoted as p (0,0) , the pixel coordinates corresponding to the needle tip position in the current low-frequency ultrasound image are marked as p′ (0,0) At the same time, the pixel coordinates of each pixel in the real-time ultrasound image are converted to p (0,0) is the plane coordinate of the origin, denoted by p (i,j) ; Convert the pixel coordinates of each pixel in the low-frequency ultrasound image to p′ (0,0) is the plane coordinate of the origin, denoted by p′ (i,j) Then, the intelligent terminal can use the following formula to calculate the weight index corresponding to each pixel in the real-time ultrasound image:

[0052]

[0053] in, is pixel p (i,j) The corresponding weight index, ρ is the anti-attenuation parameter, |d(p (i,j) ,p (0,0) )| is pixel p (i,j) With the origin p (0,0) The absolute value of the distance between max It is the maximum value of the distance between two pixels in the real-time ultrasound image.

[0054] And, the smart terminal can use the following formula to calculate the weight index corresponding to each pixel in the low-frequency ultrasound image:

[0055]

[0056] in, is pixel p′ (i,j) The corresponding weight index.

[0057] After determining the weight index of each pixel, the intelligent terminal can perform pixel fusion on the real-time ultrasound image and the low-frequency ultrasound image at the current stage based on the weight index of each pixel to obtain the fused ultrasound image at the current stage.

[0058] Specifically, the smart terminal can use the following formula to calculate the pixel p (i,j) The pixel value g(i,j) :

[0059]

[0060] Among them, f (i,j) is the pixel value after fusion, that is, the pixel p″ in the fused image (i,j) The pixel value, g (i,j) is pixel p (i,j) The pixel value, g′ (i,j) is pixel p′ (i,j) The pixel value of .

[0061] This can achieve efficient and accurate fusion of the current low-frequency ultrasound image and the real-time ultrasound image, and obtain the current fused ultrasound image (such as Figure 2 After obtaining the fused ultrasound image at the current stage, the intelligent terminal can transmit the fused ultrasound image at the current stage to the ultrasound diagnostic instrument for display, so as to realize visual puncture guidance.

[0062] Furthermore, after obtaining the fused ultrasound image of the current stage, the intelligent terminal may further execute step S30.

[0063] Step S30: Based on the fused ultrasound image at the current stage, determine the tissue type on the current puncture path.

[0064] In this embodiment, the intelligent terminal can determine the tissue type on the current puncture path based on the fused ultrasound image at the current stage. In this embodiment, an artificial intelligence model is also used to efficiently complete this task.

[0065] Exemplarily, the intelligent terminal may pre-process the fused ultrasound image at the current stage and input it into a preset tissue type recognition model, identify the tissue type in the fused ultrasound image at the current stage through the tissue type recognition model, and determine the tissue type in the fused ultrasound image.

[0066] The tissue type recognition model can be trained using a lightweight convolutional neural network model. It also collects fused ultrasound images and annotates tissue types to form a training data set (which can be divided into a training set and a test set). The Fast R-CNN architecture is built using a deep learning framework (such as Tensorflow), and the initial model parameters, loss function and other related parameters are set. The training data set is imported for training, and finally a trained tissue type recognition model is obtained. The training process of the tissue type recognition model is also a mature technology at present, and will not be elaborated here. The trained tissue type recognition model is lightweight and has a high accuracy rate. The time required to identify the tissue type can also be controlled in milliseconds (4-6 milliseconds).

[0067] After the tissue type in the fused ultrasound image is identified, the tissue type on the current puncture path (eg, a straight line connecting the two) can be further determined based on the needle tip position and the puncture target point and output.

[0068] After obtaining the tissue type on the current puncture path, the intelligent terminal may execute step S40.

[0069] Step S40: Based on the needle tip position and the tissue type on the current puncture path, the ultrasonic frequency of the next stage is determined to control the ultrasonic emission frequency of the ultrasonic probe in the next stage.

[0070] In this embodiment, the intelligent terminal can determine the ultrasonic frequency of the next stage based on the needle tip position and the tissue type on the current puncture path, and the ultrasonic frequency of the next stage can be used as the ultrasonic emission frequency of the ultrasonic probe in the next stage.

[0071] For example, the intelligent terminal can calculate the distance between the needle tip position and the nearest target tissue type on the current puncture path, and then determine whether the distance is lower than a threshold value (e.g., 10 mm):

[0072] If the interval distance is lower than the threshold (for example, 10 mm), the intelligent terminal can determine the real-time ultrasonic frequency of the next stage based on the target tissue type (the target tissue type here refers to the tissue type adjacent to the current tissue type on the current puncture path, and the current tissue type is the tissue type where the needle tip is located) and combine it with the low-frequency ultrasonic frequency (which can be the initial low-frequency ultrasonic frequency) to determine the ultrasonic frequency of the next stage.

[0073] If the interval distance is not less than the threshold (for example, 10 mm), the intelligent terminal can determine the real-time ultrasonic frequency of the next stage based on the current tissue type at the needle tip position, and determine the ultrasonic frequency of the next stage in combination with the low-frequency ultrasonic frequency (which can be the initial low-frequency ultrasonic frequency).

[0074] In this way, the ultrasonic frequency of the next stage can be determined, and the ultrasonic frequency of each stage in the entire puncture process can be intelligently controlled.

[0075] In summary, the embodiment of the present application provides a puncture visualization method based on ultrasonic imaging, by obtaining a low-frequency ultrasonic image and a real-time ultrasonic image (the low-frequency ultrasonic image and the real-time ultrasonic image of the same stage have the same size and one-to-one pixel correspondence) under the ultrasonic frequency of the current stage (the ultrasonic frequency of each stage includes the low-frequency ultrasonic frequency and the real-time ultrasonic frequency of the stage); the needle tip position is determined based on the real-time ultrasonic image of the current stage, and the real-time ultrasonic image of the current stage is fused with the low-frequency ultrasonic image to obtain the fused ultrasonic image of the current stage. The fused ultrasonic image of the current stage can be used as an ultrasonic image for visually guiding puncture, providing a reference for puncture for medical staff. Based on the fused ultrasonic image of the current stage, the tissue type on the current puncture path is determined, and then the ultrasonic frequency of the next stage is determined based on the needle tip position and the tissue type on the current puncture path, which is used to control the ultrasonic emission frequency of the ultrasonic probe in the next stage. In this way, a continuous fused image can be provided as a visual reference in the puncture guidance operation, and the low-frequency ultrasonic image and the real-time ultrasonic image (usually high frequency or medium frequency) acquired in real time ensure the real-time update of information during the puncture process. Real-time identification and positioning of the needle tip position improves the accuracy and safety of puncture. The good penetration of low-frequency ultrasound images (such as 2-5MHz) helps to observe deep tissues, while the high resolution of real-time ultrasound images (such as 5-10MHz, high-frequency ultrasound such as 10MHz or above) can clearly display superficial tissues or fine structures. The fused image combines the advantages of both and provides more comprehensive tissue information. With the needle tip position as a reference, the fine structure of the area adjacent to the needle tip position (including the tissue type of the part where it is located, and usually also including the next tissue type or more tissue types on the current puncture path) can be provided, providing a clear ultrasound image for puncture guidance. The pixel-level fusion ensures the consistency and accuracy of the information from the low-frequency ultrasound image and the real-time ultrasound image in the fused ultrasound image, and reduces the image artifacts and interference caused by frequency switching. Based on the fused ultrasound image, the (partial or all) tissue type on the current puncture path can be determined in real time (the entire puncture guidance process can provide all tissue types through information at each stage), providing doctors with richer puncture reference information. The ultrasonic frequency of the next stage is automatically adjusted according to the needle tip position and tissue type, realizing intelligent control of the puncture process, reducing the doctor's operating burden and helping the doctor to complete the puncture operation more quickly and accurately.

[0076] In the visualization process of ultrasound-guided puncture, a pixel fusion scheme is designed. The needle tip position is considered in the fusion process and anti-attenuation parameters are introduced. Different weights are assigned to different pixels, realizing the intelligent fusion of real-time ultrasound images and low-frequency ultrasound images. With the needle tip position as the center, the weight index of each pixel decays outward, which can ensure that the needle tip and its surrounding area receive higher weights in the fused image, thereby highlighting the key areas of puncture and providing a more reliable visualization scheme for puncture guidance. This ensures that the information of the needle tip and its surrounding area in the fused image is retained and highlighted to the greatest extent. This is crucial for ultrasound-guided puncture, helping doctors to see the relationship between the needle tip and surrounding tissues more clearly, thereby reducing the difficulty of puncture and improving the accuracy and safety of puncture.

[0077] The introduction of anti-attenuation parameters allows the speed and degree of attenuation to be adjusted according to actual needs (such as tissue type, ultrasound frequency, etc.) to balance the clarity and contrast of different areas in the image. Real-time ultrasound images have higher resolution and can clearly display the details of tissue types, while low-frequency ultrasound images have better penetration into deep tissues, can display deep tissues, and are suitable for overall reference. Through pixel-level fusion after weighting, the advantages of the two ultrasound images are combined to provide more comprehensive and accurate tissue information, thereby playing a better role in puncture guidance. At the same time, this pixel-level fusion method ensures the continuity and consistency of the fused ultrasound image by accurately calculating the weight index of each pixel and attenuating it according to its distance from the needle tip position, reducing artifacts and interference caused by frequency switching or image superposition.

[0078] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A puncture visualization method based on ultrasound imaging, characterized in that: include: Acquire a low-frequency ultrasonic image and a real-time ultrasonic image at the ultrasonic frequency of the current stage, wherein the ultrasonic frequency of each stage includes the low-frequency ultrasonic frequency and the real-time ultrasonic frequency of the stage, and the low-frequency ultrasonic image and the real-time ultrasonic image of the same stage have the same size and one-to-one pixel correspondence; The needle tip position is determined based on the real-time ultrasound image of the current stage, and the real-time ultrasound image of the current stage is fused with the low-frequency ultrasound image to obtain a fused ultrasound image of the current stage; Based on the fused ultrasound image at the current stage, determine the tissue type on the current puncture path; Based on the needle tip position and the tissue type on the current puncture path, the ultrasonic frequency of the next stage is determined to control the ultrasonic emission frequency of the ultrasonic probe in the next stage.

2. The puncture visualization method based on ultrasound imaging according to claim 1, characterized in that: The needle tip position is determined based on the real-time ultrasound image at the current stage, including: After preprocessing, the real-time ultrasound image of the current stage is input into a preset needle tip position recognition model, and the needle tip position in the real-time ultrasound image of the current stage is recognized by the needle tip position recognition model, and the needle tip position is output.

3. The puncture visualization method based on ultrasound imaging according to claim 1, characterized in that: The real-time ultrasound image of the current stage is fused with the low-frequency ultrasound image to obtain the fused ultrasound image of the current stage, including: Perform pixel registration of the real-time ultrasound image at the current stage with the low-frequency ultrasound image; Based on the needle tip position, the real-time ultrasound image and the low-frequency ultrasound image at the current stage are pixel-fused to obtain a fused ultrasound image at the current stage.

4. The puncture visualization method based on ultrasound imaging according to claim 3, characterized in that: Based on the needle tip position, the real-time ultrasound image and the low-frequency ultrasound image at the current stage are pixel-fused to obtain the fused ultrasound image at the current stage, including: Determine a weight index corresponding to each pixel based on the pixel coordinates of the needle tip position in the real-time ultrasound image at the current stage; Based on the weight index of each pixel, the real-time ultrasound image and the low-frequency ultrasound image at the current stage are pixel-fused to obtain the fused ultrasound image at the current stage.

5. The puncture visualization method based on ultrasound imaging according to claim 4, characterized in that: Based on the pixel coordinates of the needle tip position in the real-time ultrasound image at the current stage, the weight index corresponding to each pixel is determined, including: Based on the real-time ultrasonic frequency at the current stage, anti-attenuation parameters are determined; The pixel coordinates of the needle tip position in the real-time ultrasound image at the current stage are taken as the center, and the weight index corresponding to each pixel is calculated by combining the anti-attenuation parameters to decay outward.

6. The puncture visualization method based on ultrasound imaging according to claim 5, characterized in that: Taking the pixel coordinates of the needle tip position in the real-time ultrasound image at the current stage as the center, combined with the anti-attenuation parameter attenuation outward, the weight index corresponding to each pixel is calculated, including: The pixel coordinates of the needle tip position in the real-time ultrasound image at the current stage are taken as the center, denoted as p (0,0) , the pixel coordinates corresponding to the needle tip position in the current low-frequency ultrasound image are marked as p′ (0,0) ; The pixel coordinates of each pixel in the real-time ultrasound image are converted into (0,0) is the plane coordinate of the origin, denoted by p (i,j) ; The pixel coordinates of each pixel in the low-frequency ultrasound image are converted to p′ (0,0) is the plane coordinate of the origin, denoted by p′ (i,j) ; The weight index corresponding to each pixel in the real-time ultrasound image is calculated using the following formula: in, is pixel p (i,j) The corresponding weight index, ρ is the anti-attenuation parameter, |d(p (i,j) ,p (0,0) )| is pixel p (i,j) With the origin p (0,0) The absolute value of the distance between max is the maximum value of the distance between two pixels in the real-time ultrasound image; The weight index corresponding to each pixel in the low-frequency ultrasound image is calculated using the following formula: in, is pixel p′ (i,j) The corresponding weight index.

7. The puncture visualization method based on ultrasound imaging according to claim 6, characterized in that: Based on the weight index of each pixel, the real-time ultrasound image and the low-frequency ultrasound image at the current stage are pixel-fused, including: The pixel p is calculated using the following formula (i,j) The pixel value g (i,j) : Among them, f (i,j) is the pixel value after fusion, that is, the pixel p″ in the fused image (i,j) The pixel value, g (i,j) is pixel p (i,j) The pixel value, g′ (i,j) is pixel p′ (i,j) The pixel value of .

8. The puncture visualization method based on ultrasound imaging according to claim 1, characterized in that: Based on the fused ultrasound image at the current stage, the tissue type on the current puncture path is determined, including: After preprocessing the fused ultrasound image at the current stage, the image is input into a preset tissue type recognition model, and the tissue type in the fused ultrasound image at the current stage is recognized by the tissue type recognition model to determine the tissue type in the fused ultrasound image; Based on the needle tip position and puncture target, the tissue type on the current puncture path is determined and output.

9. The puncture visualization method based on ultrasound imaging according to claim 1, characterized in that: Based on the needle tip position and the tissue type on the current puncture path, the ultrasound frequency for the next stage is determined, including: Calculate the interval distance between the needle tip position and the nearest target tissue type on the current puncture path; Determine whether the interval distance is lower than a threshold; If so, determining the real-time ultrasound frequency of the next stage based on the target tissue type, and combining the low-frequency ultrasound frequency to determine the ultrasound frequency of the next stage; If not, the real-time ultrasound frequency of the next stage is determined based on the current tissue type where the needle tip is located, and the ultrasound frequency of the next stage is determined in combination with the low-frequency ultrasound frequency.

10. The puncture visualization method based on ultrasound imaging according to claim 2 or 8, characterized in that: The needle tip position recognition model is trained using a key point detection network, and the tissue type recognition model is trained using a lightweight convolutional neural network model.

Images