Marker extraction method and radio-intervention system

Abstract

The application relates to a marker extraction method and a radio interventional system, which comprises the following steps: acquiring a to-be-segmented image, the to-be-segmented image being a medical image containing a marker, the marker being an infrared marker ball; determining an initial reference point of the marker in the to-be-segmented image, performing layering on the to-be-segmented image based on the initial reference point, and obtaining a plurality of to-be-processed images; performing multi-scale target segmentation on each to-be-processed image in a wavelet domain, and obtaining a plurality of initial segmentation images of different scales; mapping initial segmentation results of the initial segmentation images of different scales to the to-be-processed images of the original scale; performing target segmentation based on the to-be-processed images of the original scale which have mapped the initial segmentation results, and obtaining target segmentation results; and fitting the target segmentation results to obtain a marker. The method can improve segmentation precision.

Description

Technical Field

[0001] This application relates to the field of intelligent medical technology, and in particular to a marker extraction method and a radiological intervention system. Background Technology

[0002] Traditional methods for extracting markers from CT images are generally based on thresholding and connected component segmentation. This traditional method first divides the CT data into foreground and background by thresholding, then extracts connected components and sets rules to extract target connected components, and uses the extracted target connected components or the center of the target connected components as the final extraction result.

[0003] However, the objects to be segmented are mostly solid spheres made of special materials. The infrared optical markers are composed of a reflective coating and plastic components. Since the reflective coating is thin, directly segmenting the optical markers may result in voids, breaks, or unevenness in the segmentation results, which cannot be handled by traditional extraction methods. If the markers are not directly segmented but are indirectly segmented through special materials or specially designed components, it will increase hardware costs and usage complexity while introducing human errors such as processing, assembly, calculation, and registration, thus reducing the segmentation accuracy. Summary of the Invention

[0004] Therefore, it is necessary to provide a marker extraction method and a radiological intervention system that can improve segmentation accuracy in response to the above-mentioned technical problems.

[0005] In a first aspect, this application provides a marker extraction method, the method comprising:

[0006] Obtain an image to be segmented, wherein the image to be segmented is a medical image including markers, and the markers are infrared marker spheres;

[0007] Determine the initial reference points of the markers in the image to be segmented, and perform layer cutting on the image to be segmented based on the initial reference points to obtain several images to be processed.

[0008] Multi-scale target segmentation is performed on each image to be processed in the wavelet domain to obtain several initial segmentation images of different scales.

[0009] The initial segmentation results of the initial segmented images at different scales are mapped to the image to be processed at the original scale.

[0010] Based on the original scale of the image to be processed, which is mapped to the initial segmentation result, target segmentation is performed to obtain the target segmentation results;

[0011] The markers are obtained by fitting the segmentation results of each target.

[0012] In one embodiment, determining the initial reference point for the marker in the image to be segmented includes:

[0013] Obtain the initial reference points of the markers in the input image to be segmented; or

[0014] The image to be segmented is thresholded based on the target pixel value of the marker. Based on the threshold segmentation result, connected component analysis is performed to obtain connected components that meet the characteristics of the marker as candidate connected components, and target points in the candidate connected components are extracted as initial reference points.

[0015] In one embodiment, the step of slicing the image to be segmented based on the initial reference point to obtain several images to be processed includes:

[0016] Based on the initial reference point, the image to be segmented is obliquely or tangently cut to obtain several images to be processed, wherein at least three images to be processed are mutually perpendicular. The tangent is a layer cut along the coordinate system plane, and the oblique cut is a layer cut not along the coordinate system plane.

[0017] In one embodiment, the step of performing multi-scale target segmentation on each image to be processed in the wavelet domain to obtain several initial segmented images of different scales includes:

[0018] For each image to be processed, wavelet coefficients for each pixel of the image to be processed at different scales are obtained based on wavelet transform.

[0019] By analyzing the wavelet coefficients of each pixel in the image to be processed at different scales using a Gaussian mixture model, initial segmented images at different scales are obtained.

[0020] In one embodiment, obtaining the wavelet coefficients of each pixel of the image to be processed at different scales based on wavelet transform includes:

[0021] Obtain the determined wavelet basis functions;

[0022] The wavelet basis function is convolved with the image to be processed to obtain the current convolution result, and the wavelet coefficients of each pixel in the image to be processed are determined based on the current convolution result.

[0023] The current convolution result is scaled to obtain the image to be processed at the next scale. The wavelet basis function is then convolved with the image to be processed at the next scale to obtain a new current convolution result. Based on the new current convolution result, the wavelet coefficients of each pixel in the image to be processed at the next scale are determined. If the number of scale transformations has not reached the threshold, the step of scaling the current convolution result to obtain the image to be processed at the next scale continues until the number of scale transformations reaches the threshold, thus obtaining the wavelet coefficients of each pixel in the image to be processed at different scales.

[0024] In one embodiment, the step of analyzing the wavelet coefficients of each pixel of the image to be processed at different scales using a Gaussian mixture model to obtain initial segmented images at different scales includes:

[0025] For each scale of the image to be processed, the wavelet coefficients are updated based on the spatial location of the pixels, and the pixels are grouped based on the updated wavelet coefficients to obtain the target group of pixels.

[0026] By using multiple Gaussian mixture models, the wavelet coefficients of each pixel in the target group are fitted to obtain the probability that each wavelet coefficient belongs to a Gaussian mixture model.

[0027] Based on the probability corresponding to the Gaussian mixture model that meets the mean square error requirement, the label of the pixel corresponding to the wavelet coefficient is determined.

[0028] An initial segmented image of the corresponding scale is obtained based on the labels of the pixels.

[0029] In one embodiment, mapping the initial segmentation results of initial segmented images at different scales to the image to be processed at the original scale includes:

[0030] Based on the labels of each pixel in the initial segmented images at different scales, the corresponding pixels in the image to be processed at the original scale are mapped.

[0031] When the labels after pixel mapping are not the same, the label of the corresponding pixel in the initial segmentation image at the target scale is used as the label of the corresponding pixel in the image to be processed at the original scale. The target scale is the smallest scale among different scales.

[0032] In one embodiment, the process of performing target segmentation based on the original scale of the image to be processed, mapped to each of the initial segmentation results, to obtain target segmentation results includes:

[0033] Using a Markov random field model, the initial segmentation results of each pixel in the image to be processed at the original scale are used as the initial state. Based on the similarity of the neighboring pixels of each pixel, the pixels are iterated and evolved to obtain the target segmentation result.

[0034] In one embodiment, the process of fitting the markers based on the segmentation results of each target includes:

[0035] The segmentation results of each target are discretized to obtain multiple sets of pixel points;

[0036] Based on the spatial positional relationship of the multiple sets of pixels in the image to be segmented, a three-dimensional reconstruction is performed to obtain multiple sets of spatial points, where each set of spatial points corresponds to a scale.

[0037] Based on the multiple sets of spatial points, a model is fitted to obtain the markers.

[0038] In one embodiment, the step of performing model fitting based on the multiple sets of spatial points to obtain markers includes:

[0039] Multiple sets of spatial points are weighted and merged to obtain a target set of spatial points, wherein the weight of each set of spatial points decreases as the scale increases.

[0040] A set of initial points is randomly selected from the target space point set, and an initial model of markers is generated based on the initial points;

[0041] Substitute the unselected points from the target space point set into the initial model of the marker, and calculate the first deviation between the unselected points and the initial model of the marker;

[0042] Count the number of points where the first deviation falls within the deviation range;

[0043] When the number of points within the deviation range of the first deviation is greater than or equal to the number threshold, a model is fitted based on the points within the deviation range of the first deviation and the initial point to obtain the current marker model;

[0044] When the deviations of the points within the first deviation range and the initial point from the current marker model meet the deviation requirements, the current marker model is used as the fitted marker.

[0045] When the deviation between the points within the first deviation range and the initial point and the current marker model does not meet the deviation requirements, or when the number of points within the first deviation range is less than the number threshold, the step of randomly selecting a set of initial points from the target space point set and generating an initial marker model based on the initial points continues until the marker is obtained.

[0046] In one embodiment, the step of performing model fitting based on the multiple sets of spatial points to obtain markers includes:

[0047] Each set of spatial points is fitted to obtain the corresponding marker fitting model;

[0048] Based on the fitting model of each marker, the marker is obtained.

[0049] In one embodiment, after fitting the markers based on the segmentation results of each target, the process includes:

[0050] The target reference point is determined based on the fitted markers;

[0051] When the positional deviation between the initial reference point and the target reference point is less than or equal to the deviation threshold, the fitted marker and the target reference point will be output.

[0052] When the positional deviation between the initial reference point and the target reference point is greater than the deviation threshold, the target reference point is used as the initial reference point, and the process of slicing the image to be segmented based on the initial reference point to obtain several images to be processed continues until the positional deviation between the initial reference point and the target reference point is less than or equal to the deviation threshold. Then, the fitted markers and the target reference point are output.

[0053] Secondly, this application also provides an interventional radiology system, the system comprising:

[0054] A patient target is applied to the surface of the patient's body. The patient target includes several markers, which are infrared marker balls. Each infrared marker ball is at a different distance from the center of the patient target.

[0055] An image acquisition unit acquires an image to be segmented, including the infrared marker sphere.

[0056] A processor, which is programmed to perform:

[0057] Determine the initial reference points of the markers in the image to be segmented, and perform layer cutting on the image to be segmented based on the initial reference points to obtain several images to be processed.

[0058] Multi-scale target segmentation is performed on each image to be processed in the wavelet domain to obtain several initial segmentation images of different scales.

[0059] The initial segmentation results of the initial segmented images at different scales are mapped to the image to be processed at the original scale.

[0060] Based on the original scale of the image to be processed, which is mapped to the initial segmentation result, target segmentation is performed to obtain the target segmentation results;

[0061] The markers are obtained by fitting the segmentation results of each target.

[0062] In one embodiment, the infrared marker ball includes an infrared reflective coating and a sphere, the infrared reflective coating being applied to the surface of the sphere.

[0063] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method in any of the above embodiments.

[0064] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the methods in any of the above embodiments.

[0065] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the methods in any of the above embodiments.

[0066] The aforementioned marker extraction method, interventional radiology system, computer equipment, storage medium, and computer program product, after acquiring the image to be segmented, determine the initial reference point of the marker in the image to be segmented. Based on the initial reference point, the image to be segmented is sliced ​​to obtain several images to be processed. Multi-scale target segmentation is performed on each image to be processed in the wavelet domain to obtain several initial segmented images of different scales, thereby highlighting the target features of the marker while maintaining its integrity. Subsequently, the initial segmentation results of the initial segmented images of different scales are mapped to the image to be processed at the original scale. Based on each image to be processed at the original scale mapped with the initial segmentation results, target segmentation is performed to obtain each target segmentation result. Based on each target segmentation result, the marker is obtained by fitting. This directly extracts infrared optical markers, thereby reducing errors caused by assembly, design, coordinate transformation, and other factors. Extracting target images from multiple scales improves the accuracy of the scheme. Furthermore, processing in the wavelet domain and time domain eliminates interference and environmental factors, further improving segmentation accuracy. Attached Figure Description

[0067] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0068] Figure 1 This is a diagram illustrating the application environment of the marker extraction method in one embodiment;

[0069] Figure 2 This is a schematic diagram of a patient target in one embodiment;

[0070] Figure 3 A schematic diagram of an infrared optical marker in one embodiment;

[0071] Figure 4 This is a schematic diagram of a spatial registration method in one embodiment;

[0072] Figure 5 This is a flowchart illustrating a marker extraction method in one embodiment;

[0073] Figure 6 This is a schematic diagram of tangent in one embodiment;

[0074] Figure 7 This is a schematic diagram of a bevel cut in one embodiment;

[0075] Figure 8 A flowchart of the initial reference point determination steps in one embodiment;

[0076] Figure 9 This is a flowchart of the coarse segmentation step in one embodiment;

[0077] Figure 10 This is a schematic diagram of wavelet coefficient weights in one embodiment;

[0078] Figure 11 This is a schematic diagram of pixel information in one embodiment;

[0079] Figure 12 A schematic diagram of a wavelet system fitted by a Gaussian mixture model in one embodiment;

[0080] Figure 13 This is a schematic diagram of the coarse segmentation result in one embodiment;

[0081] Figure 14 A flowchart of the model fitting steps in one embodiment;

[0082] Figure 15 This is a schematic diagram of model fitting in one embodiment;

[0083] Figure 16 This is a comparison diagram of the markers obtained by the conventional method and the method of this application. Detailed Implementation

[0084] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0085] The marker extraction method provided in this application can be applied to, for example... Figure 1 The surgical application scenario is shown. Within this scenario, the entire navigation and positioning system includes a patient target, medical imaging equipment, a robotic arm and its control carriage, a hospital bed, a main control carriage, an optical tracking and positioning device, a display, and a processor storing the computer program corresponding to the marker extraction method.

[0086] In this embodiment, an optical tracking and positioning device can acquire medical images containing patient markers. By recognizing the medical images, the patient markers can be extracted for spatial registration and other purposes.

[0087] Among them, combined Figure 2 and Figure 3 As shown, Figure 2 This is a schematic diagram of a patient target in one embodiment. Figure 3 This is a schematic diagram of an infrared optical marker in one embodiment. In this embodiment, the patient target includes multiple markers, which are infrared marker spheres. These infrared marker spheres can be connected by flexible or non-flexible materials, or they can be used individually without connection. The patient target includes infrared marker spheres and a connecting device, wherein the connecting device is used to connect the infrared marker spheres to the human body, and the connection method can be adhesive, adsorption, or other methods. The infrared marker spheres consist of an infrared reflective coating that can be visualized under CT and a sphere body. In other embodiments, the markers can have other shapes, which are not specifically limited here.

[0088] In this embodiment, combined with Figure 4 As shown, Figure 4 This is a schematic diagram of a spatial registration method in one embodiment. In this embodiment, the processor acquires a medical image collected by a medical imaging device, i.e., the image to be segmented in this application. The infrared optical tracking and positioning instrument acquires the pose of the infrared marker ball in the positioning instrument space. The processor processes the image to be segmented using a marker extraction method to extract the pose of the infrared marker ball in the image space. Thus, based on the pose of the infrared marker ball in the positioning instrument space and the pose of the infrared marker ball in the image space, registration between the image space and the positioning instrument space is achieved. The following section of this application focuses on extracting markers from the image to be segmented. Based on the extracted markers, the pose of the markers in the image space can be obtained.

[0089] In one exemplary embodiment, such as Figure 5As shown, a marker extraction method is provided, which can be applied to... Figure 1 Taking the processor in the example, the explanation includes steps 502 to 512. Wherein:

[0090] S502: Obtain the image to be segmented. The image to be segmented is a medical image including markers, and the markers are infrared marker balls.

[0091] The image to be segmented is a medical image, which can be acquired using medical imaging equipment and is a three-dimensional medical image. The patient target or a separate infrared marker ball is fixed to the surgical area, and then medical images of the patient and the infrared marker ball are acquired using medical imaging equipment to obtain the image to be segmented. In one optional embodiment, the medical image is a CT image acquired using a CT (Computed Tomography) scanner.

[0092] S504: Determine the initial reference point of the marker in the image to be segmented, and perform layer cutting on the image to be segmented based on the initial reference point to obtain several images to be processed.

[0093] The initial reference point is a point on the marker, which can optionally be the initial center point of the marker, such as an initial center point determined manually or automatically. The manual method can be based on the result of human-computer interaction, while the automatic method is based on the initial center point determined by image processing or other methods. In addition, in this application, the initial reference point is selected as the initial center point. In other embodiments, it can also be other points on the marker, which are not specifically limited here.

[0094] Layer slicing refers to dividing an image to be segmented into multiple images to be processed. In one optional embodiment, the image to be segmented is sliced ​​based on an initial reference point to obtain several images to be processed. This includes: oblique or tangential slicing based on the initial reference point to obtain several images to be processed, wherein at least three images to be processed are mutually perpendicular. Tangential slicing is slicing along the coordinate system plane, while oblique slicing is slicing outside the coordinate system plane.

[0095] For convenience, combined Figure 6 and Figure 7 As shown, where Figure 6 This is a schematic diagram of tangent in one embodiment. Figure 7 This is a schematic diagram of a slanted cut in one embodiment. Assume the coordinate system is as follows: Figure 6 and Figure 7As shown, tangent is obtained by cutting along the three planes XOY, YOZ, and ZOX, while oblique is obtained by rotating the tangent image. This application does not restrict the cutting direction; both tangent and oblique cutting are acceptable, and the oblique cutting direction is not restricted, as long as the three planes are perpendicular. This application does not restrict the upper limit of the number of cutting layers, but restricts the lower limit of the number of cutting layers to be greater than 3, such as 4, 5, and 6, etc., rather than requiring it to be a multiple of 3, as long as there are three mutually perpendicular images to be processed.

[0096] S506: Perform multi-scale target segmentation on each image to be processed in the wavelet domain to obtain several initial segmented images of different scales.

[0097] Multi-scale target segmentation refers to performing multi-scale transformation on each image to be processed. Specifically, multi-scale transformation is performed on the image to be processed in the wavelet domain. For example, a wavelet basis is first selected, and then the wavelet basis is convolved with the image to be processed to determine the wavelet coefficients of each pixel at the original scale. Then, the scale transformation of the convolved image is performed, such as downsampling, and the scale-transformed convolved image is convolved with the wavelet basis again to determine the wavelet coefficients of each pixel at that scale. The scale transformation is continued until the image to be processed at the corresponding scale is obtained. Then, coarse segmentation is performed based on the image to be processed at each scale to obtain the initial segmented image. For example, a Gaussian mixture model (number of models > 3) is used to fit the wavelet coefficients of each group based on maximum likelihood estimation to obtain the probability that different components at the same scale belong to a specific Gaussian model, and the corresponding parts are divided into targets or non-targets according to the probability.

[0098] In this embodiment, a Gaussian mixture model is used to segment multi-scale image features in the wavelet domain. Since there are step features in the part where the coating is in contact with the air, a wavelet basis with obvious step characteristics is used to perform multi-scale analysis of the image in the wavelet domain to obtain complete target features. Based on the characteristics that the coating pixel values ​​are not unique and conform to the probability distribution model, a Gaussian mixture model is used to fit the wavelet coefficients. The image is softly segmented by the probability that the wavelet coefficients belong to a specific Gaussian model, making the segmentation results more continuous and thus suitable for the extraction of infrared optical markers.

[0099] S508: Map the initial segmentation results of initial segmentation images at different scales to the image to be processed at the original scale.

[0100] Each original-scale image to be processed corresponds to multiple initial segmentation images of different scales. The initial segmentation results of the initial segmentation images of different scales are inversely scaled and mapped to the original-scale image to be processed.

[0101] S510: Based on the original scale images to be processed that have been mapped with the initial segmentation results, perform target segmentation to obtain the segmentation results for each target.

[0102] Here, the target segmentation is fine segmentation. In one optional embodiment, target segmentation is performed based on the original-scale image to be processed, which is mapped with each initial segmentation result, to obtain each target segmentation result. This includes: using a Markov random field model, taking the initial segmentation result of each pixel in the original-scale image to be processed as the initial state, and iterating through each pixel based on the similarity of the pixel's neighboring pixels to obtain the target segmentation result.

[0103] In this embodiment, Markov random fields are used for estimation based on maximum a posteriori probability. The Gaussian mixture model segmentation result is used as the initial state, that is, the original scale image to be processed is used as the initial state, which maps the initial segmentation result. The process is iterated based on the similarity between voxels in the neighborhood of the same scale until the maximum a posteriori probability meets the stopping condition, thereby obtaining a complete and continuous segmentation result, that is, the target segmentation result.

[0104] S512: Obtain markers by fitting the segmentation results of each target.

[0105] In this process, the target segmentation results are converted into point clouds, and model fitting is performed on the point clouds to obtain a sphere model (because the marker is a sphere). In this embodiment, the segmentation results of at least three mutually perpendicular directions are converted into point cloud data, and the point cloud is used for model fitting. This avoids the impact of incomplete spherical surfaces and uneven coating at the base connection on the extraction of the final target center, thus obtaining an accurate sphere center.

[0106] In one optional embodiment, after fitting markers based on each target segmentation result, the process includes: determining target reference points based on the fitted markers; outputting the fitted markers and target reference points when the positional deviation between the initial reference point and the target reference point is less than or equal to a deviation threshold; and using the target reference point as the initial reference point and continuing to slice the image to be segmented based on the initial reference point to obtain several images to be processed, until the positional deviation between the initial reference point and the target reference point is less than or equal to the deviation threshold, at which point the fitted markers and target reference points are output.

[0107] Specifically, in this embodiment, the corresponding center feature point is extracted based on the sphere model. If the offset between the feature point and the initial reference point meets the deviation threshold, the process ends; otherwise, the center feature point is used as the new initial reference point, and the process continues to perform the step of slicing the image to be segmented based on the initial reference point to obtain several images to be processed, until the offset between the feature point and the initial reference point meets the deviation threshold.

[0108] The aforementioned marker extraction method, after acquiring the image to be segmented, determines the initial reference points of the markers in the image to be segmented. Based on the initial reference points, the image to be segmented is sliced ​​to obtain several images to be processed. Multi-scale target segmentation is then performed on each image to be processed in the wavelet domain, resulting in several initial segmentation images at different scales. This highlights the target features of the markers while maintaining their integrity. Subsequently, the initial segmentation results of the initial segmentation images at different scales are mapped to the original scale images to be processed. Based on each original scale image to be processed mapped with the initial segmentation results, target segmentation is performed to obtain each target segmentation result. The markers are then fitted based on each target segmentation result. This method directly extracts infrared optical markers, thereby reducing errors caused by factors such as assembly, design, and coordinate transformation. Extracting target images from multiple scales improves the accuracy of the scheme. Furthermore, processing in the wavelet and time domains eliminates interference and environmental factors, further improving segmentation accuracy.

[0109] In one optional embodiment, determining the initial reference point of the marker in the image to be segmented includes: obtaining the initial reference point of the marker in the input image to be segmented; or performing threshold segmentation on the image to be segmented based on the target pixel value of the marker, performing connected component analysis based on the threshold segmentation result, obtaining connected components that meet the characteristics of the marker as candidate connected components, and extracting the target points in the candidate connected components as initial reference points.

[0110] In this embodiment, the initial reference point can be obtained in different ways. The first way is to obtain the initial reference point of the marker in the input image to be segmented through human-computer interaction. The second way is to obtain the initial reference point by performing image processing by a computer.

[0111] Combination Figure 8 As shown, image processing can determine the initial reference point based on thresholding and connected component analysis. Specifically, since the pixel values ​​of the markers are fixed, thresholding can be performed on the image to be segmented based on the target pixel values ​​corresponding to the markers to determine the pixels belonging to and not belonging to the markers. Then, connected component analysis is performed based on the thresholding results, and the connected components are filtered, for example, based on marker features. Optionally, connected components can be filtered based on roundness and pixel count to obtain candidate connected components. Target points in the candidate connected components, such as the center point of the candidate connected component, are extracted as the initial reference point.

[0112] In the above embodiments, the initial reference point can be determined based on different methods.

[0113] In one embodiment, multi-scale target segmentation is performed on each image to be processed in the wavelet domain to obtain several initial segmentation images of different scales, including: for each image to be processed, obtaining the wavelet coefficients of each pixel of the image to be processed at different scales based on wavelet transform; and analyzing the wavelet coefficients of each pixel of the image to be processed at different scales through a Gaussian mixture model to obtain initial segmentation images at different scales.

[0114] In this embodiment, images of different scales are extracted for each image to be processed, and wavelet coefficients corresponding to each pixel of the image at each scale are extracted by wavelet transform. In this way, the initial reference points of the markers in the image to be segmented are then obtained by using a Gaussian mixture model. For example, the wavelet transform results are fitted with a Gaussian mixture model, and the pixels are marked based on the Gaussian mixture model.

[0115] In one embodiment, combined with Figure 9 As shown, Figure 9 This is a flowchart of a coarse segmentation step in one embodiment. In this embodiment, the step includes a wavelet coefficient extraction step and a Gaussian mixture model fitting step. The wavelet coefficient extraction step involves obtaining the wavelet coefficients of each pixel in the image to be processed at different scales based on wavelet transform. The Gaussian mixture model fitting step involves analyzing the wavelet coefficients of each pixel in the image to be processed at different scales using a Gaussian mixture model to obtain initial segmented images at different scales. For ease of explanation, these two steps are presented in a single flowchart. This coarse segmentation step specifically includes:

[0116] S902: Obtain the determined wavelet basis functions.

[0117] The wavelet basis function is predetermined and is a wavelet basis function with obvious step characteristics, such as the DB6 wavelet basis function. In other embodiments, other wavelet basis functions can be selected, and no specific limitation is made here.

[0118] S904: Convolve the wavelet basis function with the image to be processed to obtain the current convolution result, and determine the wavelet coefficient of each pixel in the image to be processed based on the current convolution result.

[0119] The wavelet basis function is convolved with the image to be processed to achieve the transformation from the time domain to the wavelet domain, and the current convolution result is obtained. The wavelet coefficient of each pixel is the product of the pixel and the corresponding value of the wavelet basis function and then summed.

[0120] S906: Perform a scale transformation on the current convolution result to obtain the image to be processed at the next scale, and convolve the wavelet basis function with the image to be processed at the next scale to obtain a new current convolution result. Based on the new current convolution result, determine the wavelet coefficients of each pixel in the image to be processed at the next scale. If the number of scale transformations has not reached the threshold, continue to perform the step of performing a scale transformation on the current convolution result to obtain the image to be processed at the next scale until the number of scale transformations reaches the threshold, and obtain the wavelet coefficients of each pixel in the image to be processed at different scales.

[0121] After determining the wavelet coefficients, the current convolution result is scaled, for example, downsampled, to obtain the image to be processed at the next scale. Step S904 is repeated until the number of scale transformations reaches a threshold, such as 3 or 6 times. In other embodiments, the threshold can be other values, determined by considering computational resources and the accuracy of the output. After multi-scale and wavelet transformations, each image to be processed corresponds to several multi-scale images, and each pixel in each multi-scale image corresponds to a wavelet coefficient.

[0122] S908: For each scale of the image to be processed, the wavelet coefficients are updated based on the spatial location of the pixels, and the pixels are grouped based on the updated wavelet coefficients to obtain the target group of pixels.

[0123] Among them, combined Figure 10 As shown, Figure 10 This is a schematic diagram of wavelet coefficient weights in one embodiment. For each scale of the image to be processed, the wavelet coefficients are updated based on the spatial location of the pixels. The target area of ​​the marker is located near the initial reference point and, due to its continuity, must contain multiple high-frequency pixels in the neighborhood. Therefore, based on this characteristic, the wavelet coefficients of the current pixel are weighted using the distance d between the current pixel and the initial reference point and the number of high-frequency pixels n in the neighborhood. The weight is n / d.

[0124] After updating the wavelet coefficients, the pixels are grouped based on the wavelet coefficients to obtain the target group of pixels. Specifically, this is combined with... Figure 11 As shown, Figure 11 This is a schematic diagram of pixel information in one embodiment. In this embodiment, the target region is represented as high-frequency information in the image, but the coefficient values ​​corresponding to the same boundary are not consistent (the wavelet coefficients of the pixel are represented by the length of the line segment in the diagram), such as... Figure 11As shown, this application divides pixels into four parts according to wavelet coefficients in the horizontal and vertical directions: LL (low-low), HL (high-low), LH (low-high), and HH (high-high). HH includes high-frequency components such as targets and noise. Although some targets may still be retained in HL and LH due to various environmental factors, since the information provided by the multi-scale is far more than the information lost by not processing here, this application only performs statistics based on wavelet coefficients on the HH part, i.e., the target group.

[0125] S910: Using multiple Gaussian mixture models, the wavelet coefficients of each pixel in the target group are fitted to obtain the probability that each wavelet coefficient belongs to the Gaussian mixture model.

[0126] S912: Determine the pixel label corresponding to the wavelet coefficient based on the probability corresponding to the Gaussian mixture model that meets the mean square error requirement.

[0127] Combination Figure 12 As shown, Figure 12 This is a schematic diagram of fitting a wavelet system to a Gaussian mixture model in one embodiment. In this embodiment, the dark black curve represents the distribution of wavelet coefficients, and multiple light gray curves represent Gaussian distributions. At least three Gaussian distributions, i.e., three Gaussian mixture models, are selected to fit the wavelet coefficient distribution curves. Assuming that Gaussian distribution 1 is the Gaussian distribution conformed to by the target object, the probability of a position belonging to the target can be determined based on the probability of different wavelet coefficients belonging to Gaussian distribution 1. Here, a dual-threshold method is used to classify whether a position belongs to the target: values ​​less than or equal to the first threshold and values ​​greater than or equal to the second threshold are determined to belong to the target; values ​​greater than the first threshold or less than the second threshold are determined not to belong to the target. Figure 12 The part between the thick and thin black lines can be removed by the dual threshold method, which can remove other tissues and high-frequency noise, thus obtaining the wavelet coefficient range representing the target part.

[0128] The probability of the Gaussian mixture model that meets the mean square error requirement is chosen here because the appearance of the marker is continuous, so the model with the smallest mean square error is selected.

[0129] S914: Pixel-based labels yield the initial segmented image at the corresponding scale.

[0130] After labeling each pixel at each scale, the initial segmentation image at that scale is obtained.

[0131] In the above embodiments, infrared optical markers are directly extracted, thereby reducing errors caused by factors such as assembly, design, and coordinate transformation. Segmentation is performed in non-grayscale space, making it applicable to various external factors such as different image devices and different scanning methods. An optical sphere segmentation method is provided to extract target images from multiple scales, thereby improving the accuracy of the solution. The entire solution achieves high-precision extraction of infrared optical markers without the need for special materials or special designs or additional cost expenditures.

[0132] In one embodiment, mapping the initial segmentation results of initial segmentation images at different scales to the image to be processed at the original scale includes: mapping the labels of each pixel in the initial segmentation images at different scales to the corresponding pixels in the image to be processed at the original scale; when the labels of the mapped pixels are different, the labels of the corresponding pixels in the initial segmentation images at the target scale are used as the labels of the corresponding pixels in the image to be processed at the original scale, and the target scale is the smallest scale among the different scales.

[0133] Specifically, in combination Figure 13 As shown, Figure 13 This is a schematic diagram of the coarse segmentation result in one embodiment. In this embodiment, the initial segmentation results of the initial segmentation images at different scales are mapped to the image to be processed at the original scale. This can determine the label of each pixel in the image to be processed at the original scale. Due to the inverse scale transformation, there may be a pixel corresponding to multiple scale labels, and the labels may be inconsistent. To solve this problem, when labels conflict, the label of the smallest scale is used as the label of the pixel.

[0134] Combination Figure 13 Assuming a pixel at scale n-2 is identified as the target region, then according to the downsampling principle, the four corresponding regions at scale n-1 are also labeled as target regions. This process is carried out step by step from small scale to large scale, ultimately yielding a labeling result obtained through multi-scale voting. When mixing labeling results from different scales, since the step signal preserved in the small-scale image is stronger in the original scale, the labels represented by the small scale are prioritized during mixing. Labels can be assigned to different scales, such as scale 6 corresponding to label 1, scale 5 corresponding to label 2, and so on, thus flexibly and effectively preserving high-frequency information across multiple scales.

[0135] In one optional embodiment, the marker is obtained by fitting the segmentation results of each target, including: discretizing the segmentation results of each target to obtain multiple sets of pixel points; performing three-dimensional reconstruction based on the spatial positional relationship of the multiple sets of pixel points in the image to be segmented to obtain multiple sets of spatial points, wherein each set of spatial points corresponds to a scale; and performing model fitting based on the multiple sets of spatial points to obtain the marker.

[0136] The target segmentation result is based on the original scale of the image to be processed, which is mapped with labels of different scales. Since there are multiple original scale images to be processed, the multiple original scale images to be processed, which are mapped with labels of different scales, are discretized, that is, point cloudified, to obtain multiple sets of pixel points. Each set of pixel points corresponds to a label, which includes labels of different scales.

[0137] By performing 3D reconstruction based on the spatial positional relationship of multiple sets of pixels in the image to be segmented, the reconstructed target can be obtained. Since the point sets are labeled and include different labels, multiple sets of spatial point sets with different labels can be obtained, each set corresponding to a scale.

[0138] In this way, subsequent model fitting of these multiple sets of spatial points can yield markers.

[0139] In one alternative embodiment, combined with Figure 14 As shown, based on model fitting of multiple sets of spatial points, markers are obtained, including:

[0140] S1402: Multiple sets of spatial points are weighted and merged to obtain the target set of spatial points, wherein the weight of each set of spatial points decreases as the scale increases.

[0141] The process involves weighting and merging spatial point sets at different scales to obtain the target spatial point set. During the weighting process, the weight of each point at a different scale decreases as the scale increases.

[0142] S1404: Randomly select a set of initial points from the target space point set, and generate an initial model of markers based on the initial points.

[0143] After determining the target space point set, a set of initial points is randomly selected to generate the initial model of the marker. That is, it is assumed that the set of initial points belongs to the same sphere, so the initial model of the marker is obtained by directly fitting the sphere.

[0144] S1406: Substitute the unselected points from the target space point set into the initial model of the marker, and calculate the first deviation between the unselected points and the initial model of the marker.

[0145] Substituting the unselected points from the target space point set into the initial model of the markers yields the first deviation between the unselected points and the initial model of the markers. This means determining whether the unselected points conform to the initial model of the markers. If they do, there is no deviation; otherwise, the first deviation between the unselected points and the initial model of the markers is calculated.

[0146] S1408: Count the number of points where the first deviation is within the deviation range.

[0147] S1410: When the number of points within the first deviation range is greater than or equal to the number threshold, the model is fitted based on the points within the first deviation range and the initial point to obtain the current marker model.

[0148] The deviation range is predetermined. The number of points with the first deviation within the deviation range is counted. If there are enough points that fit the model, the model can be fitted based on the points with the first deviation within the deviation range and the initial point to obtain the current marker model.

[0149] S1412: When the deviation of the first deviation point within the deviation range and the initial point from the current marker model meets the deviation requirements, the current marker model is used as the fitted marker.

[0150] S1414: When the deviation of the points within the first deviation range and the initial point from the current marker model does not meet the deviation requirements, or when the number of points within the first deviation range is less than the number threshold, continue to execute the step of randomly selecting a set of initial points from the target space point set and generating an initial marker model based on the initial points, until the marker is obtained.

[0151] The process involves calculating the deviation between the points within the first deviation range, the initial point, and the current marker model. If the deviation requirement is met, the current marker model is used as the fitted marker. If the requirement is not met, or if the number of points within the first deviation range is less than a threshold, a set of initial points is randomly selected from the target space point set. The process continues until the marker is obtained, based on the step of generating an initial marker model from the initial points.

[0152] In another embodiment, the marker is obtained by fitting a model based on multiple sets of spatial points, including: fitting each set of spatial points to obtain a corresponding marker fitting model; and obtaining the marker based on each marker fitting model.

[0153] Specifically, in combination Figure 15 As shown, Figure 15 This is a schematic diagram of model fitting in one embodiment. It is assumed that the infrared marker sphere is segmented at three different scales and labeled as label1, label2, and label3 when transformed back to the original scale. The three labels are fused, and the probability of random point selection is weighted according to the labels. The fitting is performed based on the weighted result. For example, each set of spatial points is fitted to obtain the corresponding marker fitting model. Then, the fitting models of each marker are weighted to obtain the marker. Optionally, the three labels are solved using the least squares method to obtain the sphere center features corresponding to different labels. The sphere center features are weighted and fitted according to the labels to obtain the average value to obtain the final feature.

[0154] Combination Figure 16 As shown, Figure 16 This is a comparison diagram of the markers obtained by the traditional method and the method of this application. The segmentation results in this application are more complete and continuous, and have better performance.

[0155] This application directly extracts infrared optical markers, thereby reducing errors caused by factors such as assembly, design, and coordinate transformation; it segments in a non-grayscale space, making it applicable to various external factors such as different image devices and different scanning methods; it provides an optical sphere segmentation method to extract target images from multiple scales, thus improving the accuracy of the solution; it also provides an optical sphere segmentation method that solves the problems of spherical defects and inhomogeneity through model fitting, making the solution applicable to infrared marker spheres; the entire solution achieves high-precision extraction of infrared marker spheres without the need for special materials or special designs or additional cost.

[0156] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0157] Based on the same inventive concept, this application also provides a marker extraction device for implementing the marker extraction method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more marker extraction device embodiments provided below can be found in the limitations of the marker extraction method described above, and will not be repeated here.

[0158] like Figures 1 to 3 As shown, the present invention also provides a radiological interventional system, the system comprising:

[0159] The patient target is applied to the patient's body surface. The patient target consists of several markers, which are infrared marker balls. The distance from each infrared marker ball to the center of the patient target is different.

[0160] The image acquisition unit acquires the image to be segmented, including the infrared marker ball.

[0161] The processor, which is programmed to execute:

[0162] An initial reference point for the marker in the image to be segmented is determined. Based on the initial reference point, the image to be segmented is sliced ​​to obtain several images to be processed. Multi-scale target segmentation is performed on each image to be processed in the wavelet domain to obtain several initial segmentation images at different scales. The initial segmentation results of the initial segmentation images at different scales are mapped to the original scale images to be processed. Based on each original scale image to be processed mapped with the initial segmentation results, target segmentation is performed to obtain each target segmentation result. The marker is obtained by fitting based on each target segmentation result.

[0163] In one alternative embodiment, the infrared marker ball includes an infrared reflective coating and a sphere, the infrared reflective coating being applied to the surface of the sphere.

[0164] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0165] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for extracting markers, characterized in that, The method includes: Obtain an image to be segmented, wherein the image to be segmented is a medical image including markers, and the markers are infrared marker spheres; Determine the initial reference points of the markers in the image to be segmented, and perform layer cutting on the image to be segmented based on the initial reference points to obtain several images to be processed. Multi-scale target segmentation is performed on each image to be processed in the wavelet domain to obtain several initial segmented images at different scales. This includes: for each image to be processed, obtaining wavelet coefficients for each pixel at different scales based on wavelet transform; updating the wavelet coefficients based on the spatial location of the pixels for each scale of the image to be processed; grouping the pixels based on the updated wavelet coefficients to obtain target group pixels; fitting the wavelet coefficients of each pixel in the target group pixels using multiple Gaussian mixture models to obtain the probability that each wavelet coefficient belongs to a Gaussian mixture model; determining the label of the pixel corresponding to the wavelet coefficient based on the probability corresponding to the Gaussian mixture model that meets the mean square error requirement; and obtaining the initial segmented image at the corresponding scale based on the pixel label. The initial segmentation results of the initial segmented images at different scales are mapped to the image to be processed at the original scale. Based on the original scale of the image to be processed, which is mapped to the initial segmentation result, target segmentation is performed to obtain the target segmentation results; The markers are obtained by fitting the segmentation results of each target.

2. The method according to claim 1, characterized in that, Determining the initial reference points for markers in the image to be segmented includes: Obtain the initial reference points of the markers in the input image to be segmented; or The image to be segmented is thresholded based on the target pixel value of the marker. Based on the threshold segmentation result, connected component analysis is performed to obtain connected components that meet the characteristics of the marker as candidate connected components, and target points in the candidate connected components are extracted as initial reference points.

3. The method according to claim 1, characterized in that, The process of slicing the image to be segmented based on the initial reference point yields several images to be processed, including: Based on the initial reference point, the image to be segmented is obliquely or tangently cut to obtain several images to be processed, wherein at least three images to be processed are mutually perpendicular. The tangent is a layer cut along the coordinate system plane, and the oblique cut is a layer cut not along the coordinate system plane.

4. The method according to claim 1, characterized in that, The step of obtaining the wavelet coefficients of each pixel of the image to be processed at different scales based on wavelet transform includes: Obtain the determined wavelet basis functions; The wavelet basis function is convolved with the image to be processed to obtain the current convolution result, and the wavelet coefficients of each pixel in the image to be processed are determined based on the current convolution result. The current convolution result is scaled to obtain the image to be processed at the next scale. The wavelet basis function is then convolved with the image to be processed at the next scale to obtain a new current convolution result. Based on the new current convolution result, the wavelet coefficients of each pixel in the image to be processed at the next scale are determined. If the number of scale transformations has not reached the threshold, the step of scaling the current convolution result to obtain the image to be processed at the next scale continues until the number of scale transformations reaches the threshold, thus obtaining the wavelet coefficients of each pixel in the image to be processed at different scales.

5. The method according to claim 1, characterized in that, The step of mapping the initial segmentation results of initial segmented images at different scales to the image to be processed at the original scale includes: Based on the labels of each pixel in the initial segmented images at different scales, the corresponding pixels in the image to be processed at the original scale are mapped. When the labels after pixel mapping are not the same, the label of the corresponding pixel in the initial segmentation image at the target scale is used as the label of the corresponding pixel in the image to be processed at the original scale. The target scale is the smallest scale among different scales.

6. The method according to claim 1, characterized in that, The process involves segmenting the target image based on its original scale, which is mapped to the initial segmentation result, to obtain target segmentation results, including: Using a Markov random field model, the initial segmentation results of each pixel in the image to be processed at the original scale are used as the initial state. Based on the similarity of the neighboring pixels of each pixel, the pixels are iterated and evolved to obtain the target segmentation result.

7. The method according to claim 1, characterized in that, The process of fitting the markers based on the segmentation results of each target includes: The segmentation results of each target are discretized to obtain multiple sets of pixel points; Based on the spatial positional relationship of the multiple sets of pixels in the image to be segmented, a three-dimensional reconstruction is performed to obtain multiple sets of spatial points, where each set of spatial points corresponds to a scale. Based on the multiple sets of spatial points, a model is fitted to obtain the markers.

8. The method according to claim 1, characterized in that, After obtaining the markers by fitting based on the segmentation results of each target, the process includes: The target reference point is determined based on the fitted markers; When the positional deviation between the initial reference point and the target reference point is less than or equal to the deviation threshold, the fitted marker and the target reference point will be output. When the positional deviation between the initial reference point and the target reference point is greater than the deviation threshold, the target reference point is used as the initial reference point, and the process of slicing the image to be segmented based on the initial reference point to obtain several images to be processed continues until the positional deviation between the initial reference point and the target reference point is less than or equal to the deviation threshold. Then, the fitted markers and the target reference point are output.

9. A radiological interventional system, characterized in that, The system includes: A patient target is applied to the surface of the patient's body. The patient target includes several markers, which are infrared marker balls. Each infrared marker ball is at a different distance from the center of the patient target. An image acquisition unit acquires an image to be segmented, including the infrared marker sphere. A processor, which is programmed to perform: Determine the initial reference points of the markers in the image to be segmented, and perform layer cutting on the image to be segmented based on the initial reference points to obtain several images to be processed. Multi-scale target segmentation is performed on each image to be processed in the wavelet domain to obtain several initial segmented images at different scales. This includes: for each image to be processed, obtaining wavelet coefficients for each pixel at different scales based on wavelet transform; updating the wavelet coefficients based on the spatial location of the pixels for each scale of the image to be processed; grouping the pixels based on the updated wavelet coefficients to obtain target group pixels; fitting the wavelet coefficients of each pixel in the target group pixels using multiple Gaussian mixture models to obtain the probability that each wavelet coefficient belongs to a Gaussian mixture model; determining the label of the pixel corresponding to the wavelet coefficient based on the probability corresponding to the Gaussian mixture model that meets the mean square error requirement; and obtaining the initial segmented image at the corresponding scale based on the pixel label. The initial segmentation results of the initial segmented images at different scales are mapped to the image to be processed at the original scale. Based on the original scale of the image to be processed, which is mapped to the initial segmentation result, target segmentation is performed to obtain the target segmentation results; The markers are obtained by fitting the segmentation results of each target.

10. The system according to claim 9, characterized in that, The infrared marker sphere comprises an infrared reflective coating and a sphere, wherein the infrared reflective coating is applied to the surface of the sphere.

Images