A puncture positioning method and device based on ultrasonic imaging

Abstract

The present invention discloses a puncture positioning method and device based on ultrasonic imaging, which relates to the field of ultrasonic imaging technology; the ultrasonic image and photoacoustic image collected synchronously when the puncture needle moves are subjected to feature extraction to obtain a first feature and a second feature; the first feature and the second feature are aligned and fused to obtain a target feature, and the target feature is annotated on the ultrasonic image to obtain a target image; a target three-dimensional space is constructed according to all target images when the puncture needle moves, and the position of the puncture needle in the target three-dimensional space is identified to obtain the puncture positioning position. By simultaneously acquiring ultrasonic images and photoacoustic images and performing feature fusion, a target image containing more information can be generated, thereby improving the accuracy and reliability of positioning, and by constructing a three-dimensional space, the motion trajectory and position of the puncture needle can be more intuitively displayed, thereby improving the efficiency of puncture positioning.

Description

Technical Field

[0001] The present invention belongs to the technical field of ultrasonic imaging, and in particular relates to a puncture positioning method and device based on ultrasonic imaging. Background Art

[0002] Ultrasound (US), as a non-invasive, non-ionizing radiation imaging technology, is increasingly used in modern clinical medicine, especially in intraoperative operations. Ultrasound imaging can not only provide doctors with a clear view of anatomical structures, but also guide operations in real time, greatly improving the accuracy and safety of clinical diagnosis and treatment. Under the guidance of ultrasound imaging, clinicians can perform a variety of interventional operations, including surgical ablation (such as tumor ablation), tissue biopsy, regional anesthesia, drug delivery and therapeutic injections, etc. These operations rely on the real-time imaging function of ultrasound, allowing doctors to accurately locate target tissues or lesions in a minimally invasive manner, reducing patient pain and postoperative complications.

[0003] Patent CN115530878A discloses a puncture section positioning method based on ultrasonic imaging and an ultrasonic imaging system method, which generates multiple frames of first ultrasonic images by controlling a first transducer array to transmit a first ultrasonic wave to the prostate, receiving the echo signal of the first ultrasonic wave, and determining at least three frames of target ultrasonic images evenly distributed therein according to the similarity between the multiple frames of first ultrasonic images; using at least two frames of the at least three frames of target ultrasonic images as ultrasonic images to be matched; matching the real-time ultrasonic image of the prostate with each frame of the ultrasonic image to be matched, so as to determine a second ultrasonic image matching each frame of the ultrasonic image to be matched in the real-time ultrasonic image, and using the longitudinal section corresponding to the second ultrasonic image as the target puncture section to be punctured, thereby guiding the puncture of the target puncture section and ensuring uniform puncture of the prostate. However, during puncture positioning, the smooth surface of the puncture needle will produce strong mirror reflection, resulting in the inability of the ultrasonic wave to effectively return to the probe, making the needle blurred in the image, especially at the critical moment of puncture positioning, reducing the efficiency of puncture positioning. Summary of the invention

[0004] The purpose of the present invention is to solve the problem that the smooth surface of the puncture needle will produce strong mirror reflection, resulting in the inability of ultrasonic waves to effectively return to the probe, making the needle blurred in the image, especially at the critical moment of puncture positioning, thereby reducing the efficiency of puncture positioning, and propose a puncture positioning method and device based on ultrasonic imaging.

[0005] In a first aspect of the present invention, a puncture positioning method based on ultrasound imaging is first proposed, the method comprising:

[0006] The ultrasound probe and the photoacoustic probe are used to synchronously collect ultrasound images and photoacoustic images when the puncture needle moves to obtain an ultrasound image set and a photoacoustic image set;

[0007] Feature extraction is performed on the ultrasonic image and the photoacoustic image to obtain a first feature and a second feature respectively; the ultrasonic image is any ultrasonic image in the ultrasonic image set; the photoacoustic image is a photoacoustic image acquired at the same time as the ultrasonic image;

[0008] Aligning the first feature and the second feature and fusing them to obtain a target feature, and marking the target feature on the ultrasound image to obtain a target image;

[0009] A target three-dimensional space is constructed according to all target images when the puncture needle moves, and the position of the puncture needle in the target three-dimensional space is identified to obtain the puncture positioning position.

[0010] Optionally, extracting features from the ultrasound image and the photoacoustic image to obtain the first feature and the second feature includes:

[0011] Performing image preprocessing on the ultrasonic image and the photoacoustic image to obtain a first ultrasonic image and a first photoacoustic image respectively;

[0012] Performing image segmentation on the first ultrasonic image and the first photoacoustic image respectively by using an edge detection algorithm to obtain a second ultrasonic image and a second photoacoustic image;

[0013] Extracting geometric features from the second ultrasonic image and the second photoacoustic image respectively to obtain a first geometric feature and a second geometric feature;

[0014] Substituting the second ultrasound image and the second photoacoustic image into a convolutional neural network model respectively to obtain a first depth feature and a second depth feature;

[0015] The first geometric feature and the first depth feature are recorded as first features; the second geometric feature and the second depth feature are recorded as second features.

[0016] Optionally, aligning the first feature and the second feature and performing spatial fusion to obtain a target feature includes:

[0017] Acquire a first geometric feature and a second geometric feature in the first feature and the second feature, and respectively extract the needle tip position of the puncture needle in the first geometric feature and the second geometric feature to obtain a first puncture position and a second puncture position;

[0018] Calculating the Euclidean distance between the first puncture position and the second puncture position to obtain a target distance, and if the target distance is greater than a preset distance, performing a similarity transformation on the first puncture position and the second puncture position until the target distance is less than or equal to the preset distance, and geometrically aligning the ultrasound image and the photoacoustic image according to the adjusted distance to obtain a first aligned ultrasound image and a first aligned photoacoustic image;

[0019] Performing local binary processing on the first aligned ultrasound image to obtain a first texture feature, and extracting the texture in the first aligned photoacoustic image through a gray level co-occurrence matrix to obtain a second texture feature;

[0020] Mapping the first texture feature and the second texture feature to the same feature space through principal component analysis to perform spatial alignment to obtain a second aligned ultrasound image and a second aligned photoacoustic image;

[0021] An initial target feature is obtained by weighted fusion of the first feature and the second feature corresponding to the second aligned ultrasound image and the second aligned photoacoustic image.

[0022] Optionally, constructing a target three-dimensional space based on all target images when the puncture needle moves includes:

[0023] Performing inter-frame alignment on all target images to obtain an aligned image set, and mapping each aligned image in the aligned image set to a three-dimensional space in sequence according to a time sequence to obtain an initial three-dimensional space;

[0024] The target three-dimensional space is obtained by filling the unmapped voxels in the initial three-dimensional space through cubic interpolation.

[0025] Optionally, mapping each aligned image to a three-dimensional space in sequence according to the time series to obtain an initial three-dimensional space includes:

[0026] Each aligned image is projected into a three-dimensional coordinate system using the geometric information of the ultrasound device, and the pixels of the two-dimensional image are mapped into voxels in the three-dimensional space;

[0027] The spatial positions of the puncture needle at different time points are gradually accumulated to obtain the initial three-dimensional space.

[0028] In a second aspect of the present invention, a puncture positioning device based on ultrasound imaging is provided, comprising:

[0029] An image acquisition module is used to synchronously acquire ultrasound images and photoacoustic images of the puncture needle when it moves through an ultrasound probe and a photoacoustic probe to obtain an ultrasound image set and a photoacoustic image set;

[0030] A feature extraction module, used to extract features from an ultrasonic image and a photoacoustic image to obtain a first feature and a second feature respectively; the ultrasonic image is any ultrasonic image in the ultrasonic image set; and the photoacoustic image is a photoacoustic image acquired at the same time as the ultrasonic image;

[0031] a feature fusion module, configured to align and fuse the first feature and the second feature to obtain a target feature, and mark the target feature on the ultrasound image to obtain a target image;

[0032] The three-dimensional construction module is used to construct a target three-dimensional space according to all target images when the puncture needle moves, and identify the position of the puncture needle in the target three-dimensional space to obtain the puncture positioning position.

[0033] Optionally, the feature extraction module includes:

[0034] An image preprocessing module, used to perform image preprocessing on the ultrasonic image and the photoacoustic image to obtain a first ultrasonic image and a first photoacoustic image respectively;

[0035] An image segmentation module, configured to perform image segmentation on the first ultrasonic image and the first photoacoustic image respectively by using an edge detection algorithm to obtain a second ultrasonic image and a second photoacoustic image;

[0036] A geometric feature extraction module, used to extract geometric features from the second ultrasonic image and the second photoacoustic image to obtain a first geometric feature and a second geometric feature;

[0037] A depth feature extraction module, used for respectively substituting the second ultrasound image and the second photoacoustic image into a convolutional neural network model to obtain a first depth feature and a second depth feature;

[0038] A feature determination module is used to record the first geometric feature and the first depth feature as a first feature; and record the second geometric feature and the second depth feature as a second feature.

[0039] Optionally, the feature fusion module includes:

[0040] a puncture position determination module, used for acquiring a first geometric feature and a second geometric feature in the first feature and the second feature, and respectively extracting the needle tip position of the puncture needle in the first geometric feature and the second geometric feature to obtain a first puncture position and a second puncture position;

[0041] a geometric alignment module, configured to calculate the Euclidean distance between the first puncture position and the second puncture position to obtain a target distance, and if the target distance is greater than a preset distance, perform a similarity transformation on the first puncture position and the second puncture position until the target distance is less than or equal to the preset distance, and geometrically align the ultrasound image and the photoacoustic image according to the adjusted distance to obtain a first aligned ultrasound image and a first aligned photoacoustic image;

[0042] A texture feature extraction module, configured to perform local binary processing on the first aligned ultrasound image to obtain a first texture feature, and extract the texture in the first aligned photoacoustic image through a gray level co-occurrence matrix to obtain a second texture feature;

[0043] A spatial alignment module, used for mapping the first texture feature and the second texture feature to the same feature space through principal component analysis to perform spatial alignment to obtain a second aligned ultrasound image and a second aligned photoacoustic image;

[0044] A weighted fusion module is used to perform weighted fusion on the first feature and the second feature corresponding to the second aligned ultrasound image and the second aligned photoacoustic image to obtain an initial target feature.

[0045] Optionally, the three-dimensional building module includes:

[0046] An image mapping module is used to perform inter-frame alignment on all target images to obtain an aligned image set, and for the aligned images in the aligned image set, sequentially map each aligned image to a three-dimensional space in a time sequence to obtain an initial three-dimensional space;

[0047] The voxel filling module is used to fill the unmapped voxels in the initial three-dimensional space by cubic interpolation to obtain the target three-dimensional space.

[0048] Optionally, the image mapping module includes:

[0049] An image projection module, used to project each aligned image into a three-dimensional coordinate system by using geometric information of the ultrasound device, and map the pixels of the two-dimensional image into voxels in the three-dimensional space;

[0050] The position accumulation module is used to gradually accumulate the spatial positions of the puncture needle at different time points to obtain the initial three-dimensional space.

[0051] Beneficial effects of the present invention:

[0052] The present invention proposes a puncture positioning method based on ultrasonic imaging, wherein an ultrasonic image and a photoacoustic image when a puncture needle moves are synchronously collected by an ultrasonic probe and a photoacoustic probe to obtain an ultrasonic image set and a photoacoustic image set; the ultrasonic image and the photoacoustic image are respectively feature extracted to obtain a first feature and a second feature; the ultrasonic image is any ultrasonic image in the ultrasonic image set; the photoacoustic image is a photoacoustic image collected at the same time as the ultrasonic image; the first feature and the second feature are aligned and fused to obtain a target feature, and the target feature is annotated on the ultrasonic image to obtain a target image; a target three-dimensional space is constructed according to all target images when the puncture needle moves, and the position of the puncture needle in the target three-dimensional space is identified to obtain the puncture positioning position. By simultaneously acquiring ultrasonic images and photoacoustic images, and extracting and fusing images taken at the same time using two imaging technologies, a target image containing more information can be generated, thereby improving the accuracy and reliability of positioning, and by analyzing all target images when the puncture needle moves, the needle tip is clear in the image, and by constructing a three-dimensional space, the motion trajectory and position of the puncture needle tip can be more intuitively displayed, thereby improving the efficiency of puncture positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The present invention will be further described below in conjunction with the accompanying drawings.

[0054] Figure 1 A flowchart of a puncture positioning method based on ultrasound imaging is provided for an embodiment of the present invention;

[0055] Figure 2 A flowchart of another puncture positioning method based on ultrasound imaging is provided for an embodiment of the present invention;

[0056] Figure 3 An embodiment of the present invention provides a schematic structural diagram of a puncture positioning device based on ultrasonic imaging. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there can be three relationships. For example, A and B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the present invention, the description of "first", "second", etc. is only used for descriptive purposes, and cannot be understood as indicating or implying its relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" can explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0058] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work shall fall within the scope of protection of the present invention.

[0059] The embodiment of the present invention provides a puncture positioning method based on ultrasonic imaging. Figure 1 , Figure 1 A flowchart of a puncture positioning method based on ultrasonic imaging provided by an embodiment of the present invention. The method comprises the following steps:

[0060] S101, synchronously collecting ultrasound images and photoacoustic images of the puncture needle when it moves by using an ultrasound probe and a photoacoustic probe to obtain an ultrasound image set and a photoacoustic image set;

[0061] S102, extracting features from the ultrasound image and the photoacoustic image to obtain a first feature and a second feature;

[0062] S103, aligning the first feature and the second feature, fusing them to obtain a target feature, and marking the target feature on the ultrasound image to obtain a target image;

[0063] S104, constructing a target three-dimensional space according to all target images when the puncture needle moves, identifying the position of the puncture needle in the target three-dimensional space to obtain a puncture positioning position.

[0064] The ultrasonic image is any ultrasonic image in the ultrasonic image set; and the photoacoustic image is a photoacoustic image collected at the same time as the ultrasonic image.

[0065] A puncture positioning method based on ultrasound imaging provided by an embodiment of the present invention simultaneously acquires ultrasound images and photoacoustic images, and uses the two imaging technologies to extract and fuse features of images taken at the same time to generate a target image containing more information, thereby improving the accuracy and reliability of positioning. By analyzing all target images when the puncture needle moves, the needle tip is clear in the image, and by constructing a three-dimensional space, the motion trajectory and position of the puncture needle tip can be more intuitively displayed, thereby improving the efficiency of puncture positioning.

[0066] In one implementation, by marking the target features on the ultrasound image, the structure of the puncture needle and surrounding tissues is clearly identified on the target image, which can improve the visualization of the puncture needle in the image and make the positioning more accurate; the target features of the puncture needle position in the three-dimensional space are identified to obtain specific location information.

[0067] In one implementation, the model of the ultrasound probe may be GE Healthcare 12L, Philips L12-3, Siemens 14L5, Mindray L12-4s; the model of the photoacoustic probe may be LZ250, MX550D, MSOT AcuityEcho, MX201, etc.

[0068] In one implementation, ultrasound images can provide information about tissue structure, but the puncture needle is often difficult to display clearly due to mirror reflection; photoacoustic images can enhance the visualization of the puncture needle because it has a better effect on improving the contrast of the needle through the photoacoustic effect; combining the two can significantly improve the clarity and visibility of the needle in imaging, helping doctors to locate the needle more accurately.

[0069] In one implementation, the movement of the puncture needle is monitored in real time and a three-dimensional space is constructed based on all target images, which can accurately track the dynamic position of the needle in the body, accurately grasp the position and path of the needle in complex anatomical structures, adjust the operation strategy in real time, reduce errors and improve the success rate of the operation.

[0070] In one implementation, the planning and adjustment of the puncture path can be optimized by using the target three-dimensional space construction. This method can not only optimize the puncture path before the operation, but also adjust the path in real time during the operation to ensure that the needle can accurately reach the target position according to the predetermined path.

[0071] In one embodiment, see Figure 2 , step S102 specifically includes:

[0072] S1021, performing image preprocessing on the ultrasound image and the photoacoustic image to obtain a first ultrasound image and a first photoacoustic image respectively;

[0073] S1022, performing image segmentation on the first ultrasonic image and the first photoacoustic image respectively by using an edge detection algorithm to obtain a second ultrasonic image and a second photoacoustic image;

[0074] S1023, extracting geometric features from the second ultrasonic image and the second photoacoustic image respectively to obtain a first geometric feature and a second geometric feature;

[0075] S1024, respectively substituting the second ultrasound image and the second photoacoustic image into the convolutional neural network model to obtain a first depth feature and a second depth feature;

[0076] S1025, record the first geometric feature and the first depth feature as the first feature; and record the second geometric feature and the second depth feature as the second feature.

[0077] In one implementation, the ultrasound image is preprocessed by first performing Gaussian filtering to remove speckle noise in the image, and then using contrast enhancement technology to improve the contrast between the puncture needle and the background tissue to make it more prominent in the image; the photoacoustic image is preprocessed by first using Wiener filtering to remove background noise in the image, and then applying histogram equalization to enhance the image contrast to ensure that the puncture needle and the surrounding tissue are presented more clearly.

[0078] In one implementation, an edge detection algorithm is used to identify important boundaries and contours in the image, making the boundary between the puncture needle and surrounding tissue more obvious; geometric feature extraction helps capture structural information such as the shape, size, and boundaries of the target area in the image; the deep features extracted by the convolutional neural network (CNN) can extract complex nonlinear features from local details and global patterns. The combined use of geometric features and deep features helps to enhance the model's feature extraction of the puncture needle.

[0079] In one implementation, the second ultrasound image and the second photoacoustic image both separate the puncture needle area from the background, and only retain the image of the puncture needle area; the geometric feature extraction is to extract the length, width, shape, position information, etc. of the puncture needle; the convolutional neural network model is a CNN model with three 3x3 and five 5x5 convolutional layers.

[0080] In one embodiment, aligning the first feature and the second feature and performing spatial fusion to obtain the target feature includes:

[0081] Acquire the first geometric feature and the second geometric feature in the first feature and the second feature, and respectively extract the needle tip position of the puncture needle in the first geometric feature and the second geometric feature to obtain the first puncture position and the second puncture position;

[0082] Calculating the Euclidean distance between the first puncture position and the second puncture position to obtain a target distance, and if the target distance is greater than a preset distance, performing a similarity transformation on the first puncture position and the second puncture position until the target distance is less than or equal to the preset distance, and geometrically aligning the ultrasound image and the photoacoustic image according to the adjusted distance to obtain a first aligned ultrasound image and a first aligned photoacoustic image;

[0083] Performing local binary processing on the first aligned ultrasound image to obtain a first texture feature, and extracting the texture in the first aligned photoacoustic image through a gray level co-occurrence matrix to obtain a second texture feature;

[0084] Mapping the first texture feature and the second texture feature to the same feature space through principal component analysis to perform spatial alignment to obtain a second aligned ultrasound image and a second aligned photoacoustic image;

[0085] The first feature and the second feature corresponding to the second aligned ultrasonic image and the second aligned photoacoustic image are weightedly fused to obtain an initial target feature.

[0086] In one implementation, the preset distance is determined by a technician, and the positions of the needle tip of the puncture needle in the ultrasound image and the photoacoustic image are extracted respectively, and geometric alignment is performed through similarity transformation, so as to ensure that the two modal images are accurately matched in space; by calculating the Euclidean distance between the positions of the puncture needle tip in the two modal images, setting a predetermined alignment threshold, and adjusting the positions of the two images based on similarity transformation translation, scaling, rotation, etc., to ensure alignment accuracy, multimodal images can be compared and analyzed in the same space, and texture features in ultrasound and photoacoustic images are extracted respectively through local binary processing and grayscale co-occurrence matrix, rich detail information in the image can be captured, which improves the image fusion effect and the accuracy of the overall analysis results.

[0087] In one implementation, the texture features of ultrasound and photoacoustic images are mapped to the same feature space through principal component analysis, achieving unification and alignment of feature dimensions, effectively reducing data dimensions, retaining key information, and improving image processing speed and efficiency.

[0088] In one implementation, by weighted fusion of the features of the aligned ultrasound image and photoacoustic image, the information of the two modalities can be comprehensively utilized, and useful features from different image sources can be combined to obtain more comprehensive image information, which helps to improve the robustness and accuracy of the diagnostic model.

[0089] In one implementation, local binary processing is used to enhance the contrast of local areas of an image, making the texture information more prominent; the gray-level co-occurrence matrix can calculate the texture statistics of an image and extract subtle changes in the image. The combination of the two can better capture the microstructural differences in ultrasound and photoacoustic images.

[0090] In one implementation, geometric alignment and feature alignment, combined with weighted fusion, ultimately result in a more reliable and accurate initial target feature. This high-quality image fusion can provide doctors with richer diagnostic information and help them make more informed medical decisions, especially in interventional treatment, tumor localization, and tissue analysis.

[0091] In one embodiment, constructing a target three-dimensional space based on all target images when the puncture needle moves includes:

[0092] Perform inter-frame alignment on all target images to obtain an aligned image set. For the aligned images in the aligned image set, map each aligned image to a three-dimensional space in sequence according to a time sequence to obtain an initial three-dimensional space.

[0093] The target three-dimensional space is obtained by filling the unmapped voxels in the initial three-dimensional space through cubic interpolation.

[0094] In one implementation, inter-frame alignment ensures that the spatial position of each time series image is consistent in the same reference system, thereby eliminating image misalignment problems caused by motion, posture changes or other factors. Inter-frame alignment can be based on feature points, optical flow, global transformation, deep learning, etc. Taking feature points as an example, by detecting feature points in each frame of the image, and then matching the positional relationship between these feature points, the displacement, rotation or scaling transformation matrix between frames is calculated, thereby achieving image alignment.

[0095] In one implementation, by mapping the aligned images of the time series into three-dimensional space, a three-dimensional structure containing complete spatial information can be generated. Compared with a single-frame image, the three-dimensional structure can display the spatial form more intuitively and comprehensively, helping doctors to better see the location of the puncture needle.

[0096] In one implementation, cubic interpolation is a high-precision interpolation method that can generate a smoother and more continuous three-dimensional model by interpolating and filling unmapped voxels in the initial three-dimensional space, avoiding spatial faults and data loss caused by insufficient image frames or gaps, and improving the fineness and accuracy of three-dimensional reconstruction. The generation of three-dimensional medical images often encounters incomplete voxel data. Filling blank areas with cubic interpolation can ensure the integrity and smoothness of the three-dimensional model, which is particularly suitable for detailed analysis of tissue edges or complex structures.

[0097] In one embodiment, mapping each aligned image to a three-dimensional space in sequence according to a time sequence to obtain an initial three-dimensional space includes:

[0098] Each aligned image is projected into a three-dimensional coordinate system using the geometric information of the ultrasound device, and the pixels of the two-dimensional image are mapped into voxels in the three-dimensional space;

[0099] The spatial positions of the puncture needle at different time points are gradually accumulated to obtain the initial three-dimensional space.

[0100] In one implementation, the geometric information of the ultrasound device includes the probe position, angle and scanning range. The starting point of the three-dimensional coordinate system is the position of the ultrasound probe, the Z axis is along the scanning direction of the probe, the X axis and the Y axis correspond to the horizontal and vertical directions of the image, respectively. For a pixel point of a two-dimensional image, its position on the image plane is calculated, and then it is mapped to a voxel position in three-dimensional space. The specific mapping is, for example, the image resolution is 0.5 mm / pixel, the physical size of the image is 100 mmx100 mm, the image plane is in the Z=0 plane, and the image is directly in front of the probe. The two-dimensional coordinates of the pixel (i, j) are (x2D, y2D), and its three-dimensional coordinates are (x3D, y3D, z3D), x3D=x2D×0.5 mm, y3D=y2D×0.5 mm, and z3D is the depth information at the pixel (i, j).

[0101] In one implementation, the spatial position of the puncture needle at different time points is gradually accumulated to obtain the initial three-dimensional space. Specifically, the three-dimensional coordinate points of the puncture needle at all time points are gradually accumulated. The coordinate points of each time point are added to an overall data set to form a complete puncture needle motion trajectory. The accumulated puncture needle coordinate points are used to generate a three-dimensional space model through point cloud data.

[0102] Based on the same inventive concept, the present invention also provides a puncture positioning device based on ultrasound imaging. Figure 3 , Figure 3 A schematic structural diagram of a puncture positioning device based on ultrasonic imaging provided by an embodiment of the present invention includes:

[0103] An image acquisition module is used to synchronously acquire ultrasound images and photoacoustic images of the puncture needle when it moves through an ultrasound probe and a photoacoustic probe to obtain an ultrasound image set and a photoacoustic image set;

[0104] A feature extraction module, used to extract features from an ultrasonic image and a photoacoustic image to obtain a first feature and a second feature respectively; the ultrasonic image is any ultrasonic image in the ultrasonic image set; and the photoacoustic image is a photoacoustic image acquired at the same time as the ultrasonic image;

[0105] A feature fusion module, used for aligning the first feature and the second feature and fusing them to obtain a target feature, and marking the target feature on the ultrasound image to obtain a target image;

[0106] The three-dimensional construction module is used to construct a target three-dimensional space according to all target images when the puncture needle moves, and identify the position of the puncture needle in the target three-dimensional space to obtain the puncture positioning position.

[0107] A puncture positioning device based on ultrasound imaging provided by an embodiment of the present invention simultaneously acquires ultrasound images and photoacoustic images, and uses the two imaging technologies to extract and fuse features of images taken at the same time to generate a target image containing more information, thereby improving the accuracy and reliability of positioning. By analyzing all target images when the puncture needle moves, the needle tip is clear in the image, and by constructing a three-dimensional space, the motion trajectory and position of the puncture needle tip can be more intuitively displayed, thereby improving the efficiency of puncture positioning.

[0108] In one embodiment, the feature extraction module includes:

[0109] An image preprocessing module, used to perform image preprocessing on the ultrasonic image and the photoacoustic image to obtain a first ultrasonic image and a first photoacoustic image respectively;

[0110] An image segmentation module, used to perform image segmentation on the first ultrasonic image and the first photoacoustic image respectively by using an edge detection algorithm to obtain a second ultrasonic image and a second photoacoustic image;

[0111] A geometric feature extraction module, used to extract geometric features from the second ultrasonic image and the second photoacoustic image to obtain a first geometric feature and a second geometric feature;

[0112] A depth feature extraction module, used to substitute the second ultrasound image and the second photoacoustic image into the convolutional neural network model to obtain a first depth feature and a second depth feature;

[0113] The feature determination module is used to record the first geometric feature and the first depth feature as the first feature; and record the second geometric feature and the second depth feature as the second feature.

[0114] In one embodiment, the feature fusion module includes:

[0115] A puncture position determination module is used to obtain the first geometric feature and the second geometric feature in the first feature and the second feature, and respectively extract the needle tip position of the puncture needle in the first geometric feature and the second geometric feature to obtain the first puncture position and the second puncture position;

[0116] a geometric alignment module, configured to calculate the Euclidean distance between the first puncture position and the second puncture position to obtain a target distance, and if the target distance is greater than a preset distance, perform a similarity transformation on the first puncture position and the second puncture position until the target distance is less than or equal to the preset distance, and geometrically align the ultrasound image and the photoacoustic image according to the adjusted distance to obtain a first aligned ultrasound image and a first aligned photoacoustic image;

[0117] A texture feature extraction module, used for performing local binary processing on the first aligned ultrasound image to obtain a first texture feature, and extracting the texture in the first aligned photoacoustic image through a gray level co-occurrence matrix to obtain a second texture feature;

[0118] A spatial alignment module, used for mapping the first texture feature and the second texture feature to the same feature space through principal component analysis to perform spatial alignment to obtain a second aligned ultrasound image and a second aligned photoacoustic image;

[0119] The weighted fusion module is used to perform weighted fusion on the first feature and the second feature corresponding to the second aligned ultrasonic image and the second aligned photoacoustic image to obtain an initial target feature.

[0120] In one embodiment, the three-dimensional building block comprises:

[0121] An image mapping module is used to perform inter-frame alignment on all target images to obtain an aligned image set, and for aligned images in the aligned image set, sequentially map each aligned image to a three-dimensional space in a time sequence to obtain an initial three-dimensional space;

[0122] The voxel filling module is used to fill the unmapped voxels in the initial three-dimensional space through cubic interpolation to obtain the target three-dimensional space.

[0123] In one embodiment, the image mapping module includes:

[0124] An image projection module, used to project each aligned image into a three-dimensional coordinate system by using geometric information of the ultrasound device, and map the pixels of the two-dimensional image into voxels in the three-dimensional space;

[0125] The position accumulation module is used to gradually accumulate the spatial positions of the puncture needle at different time points to obtain the initial three-dimensional space.

[0126] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A puncture positioning device based on ultrasonic imaging, characterized in that: The device comprises: An image acquisition module is used to synchronously acquire ultrasound images and photoacoustic images of the puncture needle when it moves through an ultrasound probe and a photoacoustic probe to obtain an ultrasound image set and a photoacoustic image set; A feature extraction module, used to extract features from an ultrasonic image and a photoacoustic image to obtain a first feature and a second feature respectively; the ultrasonic image is any ultrasonic image in the ultrasonic image set; and the photoacoustic image is a photoacoustic image acquired at the same time as the ultrasonic image; a feature fusion module, configured to align and fuse the first feature and the second feature to obtain a target feature, and mark the target feature on the ultrasound image to obtain a target image; A three-dimensional construction module is used to construct a target three-dimensional space according to all target images when the puncture needle moves, and identify the position of the puncture needle in the target three-dimensional space to obtain a puncture positioning position; The feature extraction module comprises: An image preprocessing module, used to perform image preprocessing on the ultrasonic image and the photoacoustic image to obtain a first ultrasonic image and a first photoacoustic image respectively; An image segmentation module, configured to perform image segmentation on the first ultrasonic image and the first photoacoustic image respectively by using an edge detection algorithm to obtain a second ultrasonic image and a second photoacoustic image; A geometric feature extraction module, used to extract geometric features from the second ultrasonic image and the second photoacoustic image to obtain a first geometric feature and a second geometric feature; A depth feature extraction module, used for respectively substituting the second ultrasound image and the second photoacoustic image into a convolutional neural network model to obtain a first depth feature and a second depth feature; A feature determination module, configured to record the first geometric feature and the first depth feature as a first feature; and record the second geometric feature and the second depth feature as a second feature; The feature fusion module includes: a puncture position determination module, used for acquiring a first geometric feature and a second geometric feature in the first feature and the second feature, and respectively extracting the needle tip position of the puncture needle in the first geometric feature and the second geometric feature to obtain a first puncture position and a second puncture position; a geometric alignment module, configured to calculate the Euclidean distance between the first puncture position and the second puncture position to obtain a target distance, and if the target distance is greater than a preset distance, perform a similarity transformation on the first puncture position and the second puncture position until the target distance is less than or equal to the preset distance, and geometrically align the ultrasound image and the photoacoustic image according to the adjusted distance to obtain a first aligned ultrasound image and a first aligned photoacoustic image; A texture feature extraction module, configured to perform local binary processing on the first aligned ultrasound image to obtain a first texture feature, and extract the texture in the first aligned photoacoustic image through a gray level co-occurrence matrix to obtain a second texture feature; A spatial alignment module, used for mapping the first texture feature and the second texture feature to the same feature space through principal component analysis to perform spatial alignment to obtain a second aligned ultrasound image and a second aligned photoacoustic image; A weighted fusion module is used to perform weighted fusion on the first feature and the second feature corresponding to the second aligned ultrasound image and the second aligned photoacoustic image to obtain an initial target feature.

2. The puncture positioning device based on ultrasonic imaging according to claim 1, characterized in that: The three-dimensional building block comprises: An image mapping module is used to perform inter-frame alignment on all target images to obtain an aligned image set, and for the aligned images in the aligned image set, sequentially map each aligned image to a three-dimensional space in a time sequence to obtain an initial three-dimensional space; The voxel filling module is used to fill the unmapped voxels in the initial three-dimensional space by cubic interpolation to obtain the target three-dimensional space.

3. The puncture positioning device based on ultrasonic imaging according to claim 2, characterized in that: The image mapping module comprises: An image projection module, used to project each aligned image into a three-dimensional coordinate system by using geometric information of the ultrasound device, and map the pixels of the two-dimensional image into voxels in the three-dimensional space; The position accumulation module is used to gradually accumulate the spatial positions of the puncture needle at different time points to obtain the initial three-dimensional space.

Images