Feature and intensity based x-ray and three-dimensional scan model registration method and system

Abstract

The application discloses a feature and intensity-based X-ray and three-dimensional scanning model registration method and system, which comprises the following steps: acquiring a spine X-ray orthographic image and a spine three-dimensional scanning model of a patient; preliminarily registering the spine X-ray orthographic image and the spine three-dimensional scanning model based on the correlation between the first feature point position of the spine X-ray orthographic image and the second feature point position of the spine three-dimensional scanning model; establishing a structural similarity index of the spine X-ray orthographic image and the spine three-dimensional scanning model based on a rigid transformation parameter; obtaining an optimal rigid transformation parameter based on the optimal structural similarity index; and intensity registering the spine X-ray orthographic image and the spine three-dimensional scanning model based on the optimal rigid transformation parameter. The application ensures the intensity registration of the spine X-ray orthographic image and the spine three-dimensional scanning model by preliminarily registering and intensity registering the spine X-ray orthographic image and the spine three-dimensional scanning model.

Description

Technical Field

[0001] This invention relates to the field of intelligent diagnosis and treatment technology, specifically to a method and system for registering X-ray and three-dimensional scanning models based on features and intensity. Background Technology

[0002] Scoliosis braces are an important non-surgical treatment for scoliosis. Their design relies on precise registration between the patient's spinal X-ray image and a three-dimensional skin mesh model. The accuracy of this registration directly affects the positioning of the force application and the corrective effect of the brace. Currently, commonly used registration methods mainly rely on manual operation or a single registration algorithm, which suffers from low registration efficiency and unstable accuracy. In clinical practice, factors such as changes in patient posture and differences in image quality often lead to errors in the registration process, making it difficult to meet the combined requirements of speed and high precision in brace design. These problems, to some extent, limit the efficiency and design quality of orthopedic braces in clinical application. Summary of the Invention

[0003] To address the shortcomings of existing technologies, the present invention aims to provide a method and system for registering X-ray and three-dimensional scanning models based on features and intensity. By performing preliminary registration and intensity registration on spinal X-ray anteroposterior images and spinal three-dimensional scanning models, the intensity registration of spinal X-ray anteroposterior images and spinal three-dimensional scanning models is ensured.

[0004] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0005] According to a first aspect of this application, a method for registering X-ray and 3D scanning models based on features and intensity is provided, comprising:

[0006] Acquire an anteroposterior X-ray image of the patient's spine, the anteroposterior X-ray image of the spine covering the lower cervical spine to the pelvic region;

[0007] Obtain a three-dimensional scan model of the patient's spine, the three-dimensional scan model of the spine including the trunk region;

[0008] The first feature point position of the spinal X-ray anteroposterior image and the second feature point position of the spinal 3D scanning model mapped to the spinal X-ray anteroposterior image are obtained. A two-dimensional affine transformation matrix is ​​constructed based on the correlation between the first and second feature point positions. The spinal X-ray anteroposterior image and the spinal 3D scanning model are initially registered based on the two-dimensional affine transformation matrix.

[0009] A structural similarity index is established between the spinal X-ray anteroposterior image and the spinal 3D scan model based on rigid transformation parameters. The structural similarity index is iterated using a constrained optimization algorithm until the optimal structural similarity index is obtained. The optimal rigid transformation parameters are then obtained based on the optimal structural similarity index. Intensity registration is performed between the spinal X-ray anteroposterior image and the spinal 3D scan model based on the optimal rigid transformation parameters.

[0010] In some embodiments of this application, based on the foregoing scheme, the acquisition of a three-dimensional scan model of the patient's spine, the three-dimensional scan model of the spine including the trunk region, includes:

[0011] Obtain an initial 3D scanning model, and then correct the initial 3D scanning model to obtain a standard 3D scanning model;

[0012] The standard 3D scanning model was cut along the plane of the upper edge of the clavicle and the fold plane of the root of the thigh to remove the head and lower limb regions, thus obtaining the 3D scanning model of the spine.

[0013] In some embodiments of this application, based on the foregoing scheme, the step of obtaining an initial three-dimensional scanning model and correcting the initial three-dimensional scanning model to obtain a standard three-dimensional scanning model includes:

[0014] Obtain the third left anterior superior iliac spine point, the third right anterior superior iliac spine point, and the pubic symphysis point of the initial three-dimensional scanning model, and construct a pelvic reference plane P based on the third left anterior superior iliac spine point, the third right anterior superior iliac spine point, and the pubic symphysis point;

[0015] Align the pelvic reference plane with the XY plane of the world coordinate system to perform rigid rotational transformation correction on the initial three-dimensional scanning model, thereby obtaining a standard three-dimensional scanning model.

[0016] In some embodiments of this application, based on the aforementioned scheme, the first feature point location includes the first annular cartilage point, the first left anterior superior iliac spine point, and the first right anterior superior iliac spine point, and the second feature point location includes the second annular cartilage point, the second left anterior superior iliac spine point, and the second right anterior superior iliac spine point.

[0017] In some embodiments of this application, based on the aforementioned scheme, the steps of obtaining the first feature point position of the spinal X-ray anteroposterior image and the second feature point position of the spinal three-dimensional scanning model mapped to the spinal X-ray anteroposterior image, constructing a two-dimensional affine transformation matrix based on the correlation between the first and second feature point positions, and performing preliminary registration of the spinal X-ray anteroposterior image and the spinal three-dimensional scanning model based on the two-dimensional affine transformation matrix include:

[0018] Mark the first feature point on the anteroposterior X-ray image of the spine and obtain the location of the first feature point;

[0019] Mark the third feature point on the three-dimensional scanning model of the spine, and map the third feature point on the three-dimensional scanning model of the spine to the two-dimensional image plane of the spinal X-ray anteroposterior image through orthogonal projection to obtain the second feature point of the three-dimensional scanning model of the spine mapped to the spinal X-ray anteroposterior image, and obtain the position of the second feature point.

[0020] A two-dimensional affine transformation matrix is ​​used to perform a two-dimensional affine transformation on the anteroposterior X-ray image of the spine, so as to achieve preliminary registration between the anteroposterior X-ray image and the three-dimensional scanning model of the spine.

[0021] In some embodiments of this application, based on the foregoing scheme, the correlation includes a scale and an offset, the scale includes a horizontal scaling scale and a vertical scaling scale, and a two-dimensional affine transformation matrix is ​​constructed based on the horizontal scaling scale, the vertical scaling scale, and the translation vector.

[0022] In some embodiments of this application, based on the aforementioned scheme, the step of establishing a structural similarity index between a spinal X-ray anteroposterior image and a spinal 3D scan model based on rigid transformation parameters, iterating the structural similarity index using a constrained optimization algorithm until an optimal structural similarity index is obtained, obtaining the optimal rigid transformation parameters based on the optimal structural similarity index, and performing intensity registration between the spinal X-ray anteroposterior image and the spinal 3D scan model based on the optimal rigid transformation parameters includes:

[0023] The three-dimensional scanning model of the spine is orthogonally projected onto a two-dimensional plane to generate a binary mask image with the same resolution as the anteroposterior X-ray image of the spine.

[0024] A structural similarity index between binary mask images and three-dimensional spinal scan models is established based on rigid transformation parameters;

[0025] The constrained optimization algorithm iterates the structural similarity index based on a convergence threshold and / or the maximum number of iterations;

[0026] The algorithm is considered to have converged when the gain of the structural similarity index in continuous iterations is lower than the convergence threshold of the structural similarity index or the maximum number of iterations is reached, and the optimal structural similarity index is output.

[0027] The optimal rigid transformation parameters are obtained based on the optimal structural similarity index. The optimal rigid transformation parameters are then used to perform a three-dimensional rigid transformation on the three-dimensional spinal scanning model, thereby achieving intensity registration between the spinal X-ray anteroposterior image and the three-dimensional spinal scanning model.

[0028] According to a second aspect of this application, a feature- and intensity-based X-ray and three-dimensional scan model registration system is provided, the system comprising:

[0029] The first acquisition module is used to acquire an anteroposterior X-ray image of the patient's spine, the anteroposterior X-ray image of the spine covering the lower cervical spine to the pelvic region;

[0030] The second acquisition module is used to acquire a three-dimensional scan model of the patient's spine, the three-dimensional scan model of the spine including the trunk region;

[0031] The first registration module is used to obtain the position of the first feature point of the spinal X-ray anteroposterior image and the position of the second feature point of the spinal three-dimensional scanning model mapped to the spinal X-ray anteroposterior image. Based on the correlation between the positions of the first and second feature points, a two-dimensional affine transformation matrix is ​​constructed, and the spinal X-ray anteroposterior image and the spinal three-dimensional scanning model are initially registered based on the two-dimensional affine transformation matrix.

[0032] The second registration module is used to establish a structural similarity index between the spinal X-ray anteroposterior image and the spinal 3D scan model based on rigid transformation parameters, iterate the structural similarity index based on a constrained optimization algorithm until the optimal structural similarity index is obtained, obtain the optimal rigid transformation parameters based on the optimal structural similarity index, and perform intensity registration between the spinal X-ray anteroposterior image and the spinal 3D scan model based on the optimal rigid transformation parameters.

[0033] According to a third aspect of this application, a computer-readable storage medium is provided that stores a computer program thereon, the computer program including executable instructions that, when executed by a processor, implement the method described above.

[0034] According to a fourth aspect of this application, an electronic device is provided, comprising:

[0035] One or more processors;

[0036] A memory for storing executable instructions of the processor, which, when executed by the one or more processors, cause the one or more processors to implement the method described above.

[0037] The beneficial effects of this application are as follows:

[0038] This application provides a method and system for registering X-ray and 3D scanning models based on features and intensity. The method obtains the positions of first feature points in a spinal X-ray anteroposterior image and the positions of second feature points mapped from the 3D scanning model to the spinal X-ray anteroposterior image. A two-dimensional affine transformation matrix is ​​constructed based on the correlation between the first and second feature point positions. Preliminary registration of the spinal X-ray anteroposterior image and the spinal 3D scanning model is performed based on the two-dimensional affine transformation matrix. Then, a structural similarity index is established between the spinal X-ray anteroposterior image and the spinal 3D scanning model based on rigid transformation parameters. The optimal structural similarity index is obtained iteratively. The optimal rigid transformation parameters are obtained based on the optimal structural similarity index. Intensity registration of the spinal X-ray anteroposterior image and the spinal 3D scanning model is performed based on the optimal rigid transformation parameters. This application significantly improves the automation level of the registration process, reduces reliance on human experience, and enhances the standardization and consistency of the registration operation. Combining initial feature point registration with intensity-driven fine registration, it significantly improves registration speed while ensuring accuracy, meeting the clinical needs for rapid brace design and enhancing system practicality and design efficiency.

[0039] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0040] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and are intended to explain the invention, but do not constitute an undue limitation thereof. In the drawings:

[0041] Figure 1 This is a flowchart of a method for registering X-ray and 3D scanning models based on features and intensity, according to the present invention.

[0042] Figure 2 This is a schematic diagram of the preprocessed and annotated three-dimensional scanning model of the spine in this invention;

[0043] Figure 3 This is a schematic diagram after preliminary registration in this invention;

[0044] Figure 4 This is a schematic diagram of the preprocessed and annotated spinal X-ray anteroposterior image in this invention;

[0045] Figure 5 This is a schematic diagram after strength registration in this invention;

[0046] Figure 6 This is a schematic diagram of an X-ray and three-dimensional scanning model registration system based on features and intensity according to the present invention;

[0047] Figure 7 This is a schematic diagram of an electronic device according to the present invention. Detailed Implementation

[0048] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0049] It should be understood that the terms "comprising" and other similar expressions in the specification, claims, and accompanying drawings of this invention are intended to cover a non-exclusive inclusion, such as a process, method, system, or apparatus that includes a series of steps or units and is not limited to the listed steps or units. Furthermore, "first" and "second" are used to distinguish different objects and are not intended to describe a specific order.

[0050] Please see Figure 1 The diagram shows a flowchart of a feature- and intensity-based X-ray and 3D scanning model registration method according to the present invention.

[0051] According to a first aspect of this application, this embodiment provides a method for registering X-ray and three-dimensional scanning models based on features and intensity, comprising:

[0052] Step S1: Obtain an anteroposterior X-ray image of the patient's spine, the anteroposterior X-ray image of the spine covering the lower cervical spine to the pelvic region.

[0053] In some embodiments of this example, since the design process of the scoliosis correction brace relies on the precise registration between the patient's spinal X-ray anteroposterior image and the three-dimensional scanning model of the spine, the spinal X-ray anteroposterior image covers the lower cervical spine to the pelvic region to ensure complete coverage of the spine, thereby realizing the design of the scoliosis correction brace.

[0054] Step S2: Obtain a three-dimensional scan model of the patient's spine, which includes the trunk region.

[0055] In some embodiments of this example, the acquisition of a three-dimensional scan model of the patient's spine, the three-dimensional scan model of the spine including the trunk region, includes:

[0056] Obtain an initial 3D scanning model, and then correct the initial 3D scanning model to obtain a standard 3D scanning model;

[0057] The standard 3D scanning model was cut along the plane of the upper edge of the clavicle and the fold plane of the root of the thigh to remove the head and lower limb regions, thus obtaining the 3D scanning model of the spine.

[0058] In some embodiments of this example, obtaining an initial three-dimensional scanning model and correcting the initial three-dimensional scanning model to obtain a standard three-dimensional scanning model includes:

[0059] Obtain the third left anterior superior iliac spine point, the third right anterior superior iliac spine point, and the pubic symphysis point of the initial three-dimensional scanning model, and construct a pelvic reference plane P based on the third left anterior superior iliac spine point, the third right anterior superior iliac spine point, and the pubic symphysis point;

[0060] Align the pelvic reference plane with the XY plane of the world coordinate system to perform rigid rotational transformation correction on the initial three-dimensional scanning model, thereby obtaining a standard three-dimensional scanning model.

[0061] In some embodiments of this example, the step of cutting the standard three-dimensional scan model along the plane of the upper edge of the clavicle and the fold plane of the groin to remove the head and lower limb regions and obtain a three-dimensional scan model of the spine includes:

[0062] The standard 3D scan model is cut along the plane above the clavicle to remove the head region. The plane above the clavicle is positioned so that its normal vector is parallel to the direction of gravity. Then, the standard 3D scan model is cut along the fold plane at the root of the thigh to remove the lower limbs, retaining only the torso region. Subsequently, a mesh simplification algorithm is applied to compress the mesh count of the spinal 3D scan model to 20%–40% of the original model, improving subsequent processing efficiency while preserving geometric features.

[0063] In this embodiment, the mesh simplification algorithm includes an edge shrinkage algorithm, a vertex clustering algorithm, and a quadratic error metric simplification algorithm, which are not limited in this embodiment. The edge shrinkage algorithm reduces the number of faces in the mesh by merging the two endpoints of an edge and combining associated triangles. The vertex clustering algorithm divides the 3D space into regular mesh cells, merges all vertices within each mesh cell into a representative vertex, and then retriangulates the representative vertex within each mesh cell to generate a simplified mesh. The quadratic error metric simplification algorithm quantifies the geometric error after vertex movement and prioritizes the operation with the smallest error when selecting shrinkage edges, thereby minimizing shape distortion while reducing the number of mesh faces.

[0064] In this way, a corrected and simplified three-dimensional scan model of the spine can be obtained, such as Figure 2 The image shown is a schematic diagram of the preprocessed and annotated three-dimensional scanning model of the spine in this invention.

[0065] Step S3: Obtain the position of the first feature point in the spinal X-ray anteroposterior image and the position of the second feature point mapped from the three-dimensional scanning model of the spine to the spinal X-ray anteroposterior image. Construct a two-dimensional affine transformation matrix based on the correlation between the positions of the first and second feature points. Perform preliminary registration on the spinal X-ray anteroposterior image based on the two-dimensional affine transformation matrix to obtain the registered spinal X-ray anteroposterior image.

[0066] In some embodiments of this example, the feature points include three key anatomical feature points: the cricoid cartilage (CRIC), the left anterior superior iliac spine (EPI1), and the right anterior superior iliac spine (EPI2). The locations of the first feature points include the first cricoid cartilage point, the first left anterior superior iliac spine point, and the first right anterior superior iliac spine point. The locations of the second feature points include the second cricoid cartilage point, the second left anterior superior iliac spine point, and the second right anterior superior iliac spine point.

[0067] In this embodiment, the cricoid cartilage (CRIC) is defined as the only complete ring-shaped cartilage in the larynx, located below the thyroid cartilage (approximately at the level of the C6 vertebral body), and is commonly used for the positioning of endotracheal tubes.

[0068] In the preferred embodiment, the anterior superior iliac spine (EPI) is defined as the bony prominence on the anterior superior side of the ilium (part of the pelvis), palpable on the body surface, located on the lower sides of the abdomen. EPI1 is the left anterior superior iliac spine, and EPI2 is the right anterior superior iliac spine.

[0069] In some embodiments of this example, the correlation includes the horizontal scaling ratio, the vertical scaling ratio, and the translation vector, and a two-dimensional affine transformation matrix is ​​constructed based on the horizontal scaling ratio, the vertical scaling ratio, and the translation vector of the first feature point position and the second feature point position.

[0070] Specifically, the process involves obtaining the positions of first feature points in a spinal X-ray anteroposterior image and the positions of second feature points mapped from a three-dimensional spinal scanning model to the spinal X-ray anteroposterior image. A two-dimensional affine transformation matrix is ​​constructed based on the correlation between the first and second feature point positions. Preliminary registration of the spinal X-ray anteroposterior image is then performed based on this matrix to obtain the registered spinal X-ray anteroposterior image. The process also includes:

[0071] Mark the first feature point on the anteroposterior X-ray image of the spine and obtain the location of the first feature point;

[0072] Mark the third feature point on the three-dimensional scanning model of the spine, and map the third feature point on the three-dimensional scanning model of the spine to the two-dimensional image plane of the spinal X-ray anteroposterior image through orthogonal projection to obtain the second feature point of the three-dimensional scanning model of the spine mapped to the spinal X-ray anteroposterior image, and obtain the position of the second feature point.

[0073] Obtain the ratio and offset between the position of the second feature point and the position of the first feature point. The ratio includes the horizontal scaling ratio and the vertical scaling ratio. The formulas for calculating the horizontal scaling ratio and the vertical scaling ratio are as follows:

[0074] ,

[0075] In the formula, This is the horizontal scaling ratio. This is the vertical scaling ratio. Indicates the horizontal direction. Indicates the vertical direction. This indicates the left anterior superior iliac spine in the X-ray image. This indicates the right anterior superior iliac spine in the X-ray image. This indicates the cricoid cartilage in an X-ray image. This represents the left anterior superior iliac spine in a 3D model. This represents the right anterior superior iliac spine in a 3D model. Represents a three-dimensional model of cricoid cartilage. The coordinates of the left anterior superior iliac spine in the horizontal direction on the X-ray image are given. The coordinates of the right anterior superior iliac spine in the transverse direction on the X-ray image are given. The coordinates of the left anterior superior iliac spine projected from the 3D model onto the 2D plane in the lateral direction. The coordinates of the right anterior superior iliac spine projected from the 3D model onto the 2D plane in the lateral direction. The coordinates of the left anterior superior iliac spine in the longitudinal direction in the X-ray image are given. The coordinates of the cricoid cartilage in the longitudinal direction in the X-ray image are given. The coordinates of the left anterior superior iliac spine projected from the 3D model onto the 2D plane in the longitudinal direction are given. The coordinates of the cricoid cartilage in the three-dimensional model projected onto the two-dimensional plane in the longitudinal direction. This represents the Euclidean distance between two points.

[0076] The translation vector is obtained by calculating the difference in geometric center positions between the second feature point and the first feature point. ,in, The difference between the geometric center positions of the second feature point and the first feature point is the x-coordinate position. It represents the difference in geometric center position between the ordinate position of the second feature point and the ordinate position of the first feature point.

[0077] Construct a two-dimensional affine transformation matrix based on the horizontal scaling ratio, the vertical scaling ratio, and the translation vector. The calculation formula is as follows:

[0078] .

[0079] A two-dimensional affine transformation matrix is ​​used to perform a two-dimensional affine transformation on the anteroposterior X-ray image of the spine, so as to achieve preliminary registration between the anteroposterior X-ray image and the three-dimensional scanning model of the spine.

[0080] In this way, the initial spatial alignment of the spinal X-ray anteroposterior image and the spinal 3D scan model is achieved, such as... Figure 3 The diagram shown is a schematic diagram after preliminary registration in this invention.

[0081] Step S4: Establish a structural similarity index between the spinal X-ray anteroposterior image and the spinal 3D scanning model based on the rigid transformation parameters. Iterate the structural similarity index based on the convergence threshold and / or the maximum number of iterations until the optimal structural similarity index is obtained. Obtain the optimal rigid transformation parameters based on the optimal structural similarity index. Perform intensity registration on the spinal 3D scanning model image based on the optimal rigid transformation parameters to obtain the registered spinal X-ray anteroposterior image.

[0082] In some embodiments of this example, before establishing the structural similarity index between the spinal X-ray anteroposterior image and the spinal 3D scan model based on rigid transformation parameters, preprocessing of the spinal X-ray anteroposterior image is also included. This preprocessing aims to standardize the input spinal X-ray anteroposterior image, enhance its structural information, reduce noise interference, and provide a stable basis for subsequent intensity registration. Specifically, this includes:

[0083] Grayscale conversion: A weighted average method is used to convert color spinal X-ray images to single-channel grayscale images, thus completing data standardization;

[0084] Noise Removal: A median filter with a kernel size of 3×3 is applied to suppress noise, eliminating noise interference while preserving the edges of anatomical structures;

[0085] Opening morphological processing: Opening morphological processing is performed using a circular kernel with a radius of 3 pixels to achieve contour smoothing and separation of fine connections;

[0086] Contour extraction and maximum contour retention: The contour of the spinal X-ray anteroposterior image is extracted using the Canny edge detection algorithm, and the maximum connected region (trunk region) is retained using an area thresholding mechanism.

[0087] Filling holes: A flooding fill algorithm is used to close the holes inside the contour, ensuring the spatial continuity of the anatomical area;

[0088] Image scaling: Bilinear interpolation algorithm is applied to scale the spinal X-ray anteroposterior image proportionally, simplifying subsequent similarity calculations.

[0089] Specifically, the input RGB format spinal X-ray image (anteroposterior view) is converted to grayscale using a weighted average method to convert the three-channel image into a single-channel grayscale image. The conversion formula is as follows:

[0090]

[0091] In the formula, , , x-axis and ordinate The pixels represent the pixel values ​​of the red channel, green channel, and blue channel, respectively. These are the pixel values ​​of a single-channel grayscale image.

[0092] After generating a single-channel grayscale image, median filtering is used for noise suppression. Specifically, for each pixel, median filtering is applied to its neighborhood. The median grayscale value of the pixels within the region replaces the pixel value, suppressing high-frequency noise and preserving edge information. The filtering formula is expressed as:

[0093]

[0094] In the formula, x-axis and ordinate The grayscale value of the pixel. This represents the operation of sorting all pixel values ​​within a neighborhood and then removing the median value. For a single-channel grayscale image on the horizontal axis and ordinate The pixel value of the pixel, Represents the x-coordinate and ordinate neighborhood of a pixel area.

[0095] Subsequently, morphological opening operations were applied to smooth the edges of the spinal X-ray anteroposterior image. The opening operation consists of two steps: erosion and dilation, and is performed using a circular structuring element with a radius of 3 pixels. Specifically, it is expressed as follows:

[0096]

[0097] In the formula, This represents the erosion operation. This represents the expansion operation. It is a structural element.

[0098] After morphological processing, the Canny edge detection algorithm was used to extract structural edges from the anteroposterior X-ray image of the spine. First, the horizontal coordinate of the anteroposterior X-ray image of the spine was calculated. and ordinate The gradient of pixels in the horizontal and vertical directions , And determine the edge strength and direction:

[0099] ,

[0100] In the formula, x-axis and ordinate The edge intensity of the pixel, x-axis and ordinate The direction of the edge intensity of the pixel.

[0101] A binary edge image is generated using a double thresholding method and non-maximum suppression. The system then calculates the area of ​​connected components for all edges and retains the largest connected component as the torso region contour. To enhance the spatial closure of the contour, a flooding fill algorithm is used to fill holes within closed edges, ensuring regional continuity.

[0102] Finally, the processed spinal X-ray images (anteroposterior view) underwent scale-unifying. Bilinear interpolation was used to scale the images proportionally to a preset resolution to ensure consistent image size during subsequent registration. (The horizontal axis of the spinal X-ray image is shown in the original text.) and ordinate The pixel value of a pixel is calculated as the weighted average of the pixel values ​​of its four surrounding pixels, which are respectively... , , , .

[0103] The horizontal axis of the spinal X-ray anteroposterior image and ordinate The expression for the pixel value of a pixel is:

[0104]

[0105] In the formula, These are the weighting coefficients. In the anteroposterior view of a spinal X-ray image, the x-axis represents the x-axis. and ordinate pixels Adjacent pixel positions, Represents the x-coordinate and ordinate The pixel value corresponding to the pixel point.

[0106] Thus, through the above image preprocessing procedure, a clear-boundary, structurally complete, and size-standardized spinal X-ray anteroposterior image is obtained, providing an input basis for subsequent intensity registration. Figure 4The image shown is a schematic diagram of the preprocessed and annotated spinal X-ray anteroposterior image in this invention.

[0107] In some embodiments of this example, step S4 includes:

[0108] The preprocessed 3D spinal scan model is orthogonally projected onto a 2D plane to generate a binary mask image with the same resolution as the spinal X-ray anteroposterior image.

[0109] The structural similarity index (SSIM) was used to evaluate the similarity between the projection mask and the preprocessed spinal X-ray anteroposterior image, and the brightness, contrast and structural feature matching degree were comprehensively quantified.

[0110] Iterative optimization of rigid transformation parameters using constrained optimization algorithms, in translational degrees of freedom and rotational degrees of freedom Search for the maximum similarity solution in the constructed six-dimensional space;

[0111] When the similarity gain of consecutive iterations is lower than The algorithm is considered to have converged when the maximum number of iterations is reached, and the globally optimal transformation parameters are output.

[0112] The optimal transformation matrix is ​​applied to the three-dimensional mesh model to achieve intensity registration between the spinal X-ray anteroposterior image and the spinal three-dimensional scan model.

[0113] This step, based on the initial feature registration, further optimizes the spatial correspondence between the spinal X-ray anteroposterior image and the spinal 3D scan model through image intensity matching to improve registration accuracy.

[0114] Specifically, an orthogonal projection is performed on the 3D scanning model of the spine to generate a binary mask image with the same resolution as the anteroposterior X-ray image of the spine. Let the point set of the 3D scanning model of the spine be... The three-dimensional rigid transformation process is defined as follows:

[0115]

[0116]

[0117] In the formula, These are the model points before the transformation. For the transformed model points, The rotation matrix is ​​constructed sequentially from Euler angles about the X, Y, and Z axes. Let Euler angles be about the X-axis. Let Euler angles be about the Y-axis. Let the Euler angles be about the Z-axis. Let be the rotation vector about the X-axis. Let be the rotation vector about the Y-axis. Let be the rotation vector about the Z-axis. It is a three-dimensional translation vector. , , These represent the translation amounts along the X, Y, and Z axes, respectively, with N being the total number of model points.

[0118] Transformed model points Projected onto a two-dimensional plane:

[0119]

[0120] In the formula, Indicates projection. Let be the projection point.

[0121] All projection points form a binary mask image. Binary mask image Compared with preprocessed spinal X-ray anteroposterior images Jointly participate in similarity evaluation.

[0122] The similarity assessment uses the Structural Similarity Index (SSIM) as the objective function for registration optimization. , The similarity between them is defined as follows:

[0123]

[0124] In the formula, , These are all local regional averages. , Both are variances. For covariance, , This is the stability constant. The SSIM value for the entire graph is the average of the results from the sliding window.

[0125] In this embodiment, the binary mask image is... As Preprocessed spinal X-ray anteroposterior image As The structural similarity index (SSIM) is used to analyze the binary mask image. Compared with preprocessed spinal X-ray anteroposterior images Jointly participate in similarity evaluation.

[0126] In this embodiment, to optimize the SSIM value, constraint optimization is performed in the six-dimensional rigid transformation parameter space, and the objective function is defined as:

[0127]

[0128] The optimization process utilizes the COBYLA algorithm, which performs derivative-free constraint optimization through linear approximation. Stiff transformation parameters. Construct a six-dimensional rigid transformation parameter space, with the rigid transformation parameters as the optimization variables. Optimize and iterate until any of the following termination conditions are met:

[0129] or

[0130] In the formula, This represents the convergence threshold of the structural similarity index. This represents the maximum number of iterations.

[0131] After optimization, the system applies the optimal rigidity transformation parameters to the 3D spinal scanning model, achieving precise spatial alignment between the spinal X-ray anteroposterior image and the model. Practical verification shows that this registration method exhibits good robustness and high accuracy under various image quality conditions.

[0132] like Figure 5 The diagram shown is a schematic diagram after strength registration in this invention.

[0133] According to the second aspect of this application, such as Figure 6 As shown, this embodiment provides a feature- and intensity-based X-ray and 3D scanning model registration system, the system comprising:

[0134] The first acquisition module is used to acquire an anteroposterior X-ray image of the patient's spine, the anteroposterior X-ray image of the spine covering the lower cervical spine to the pelvic region;

[0135] The second acquisition module is used to acquire a three-dimensional scan model of the patient's spine, the three-dimensional scan model of the spine including the trunk region;

[0136] The first registration module is used to obtain the position of the first feature point of the spinal X-ray anteroposterior image and the position of the second feature point of the spinal three-dimensional scanning model mapped to the spinal X-ray anteroposterior image. Based on the correlation between the positions of the first and second feature points, a two-dimensional affine transformation matrix is ​​constructed, and the spinal X-ray anteroposterior image and the spinal three-dimensional scanning model are initially registered based on the two-dimensional affine transformation matrix.

[0137] The second registration module is used to establish a structural similarity index between the spinal X-ray anteroposterior image and the spinal 3D scan model based on rigid transformation parameters, iterate the structural similarity index based on a constrained optimization algorithm until the optimal structural similarity index is obtained, obtain the optimal rigid transformation parameters based on the optimal structural similarity index, and perform intensity registration between the spinal X-ray anteroposterior image and the spinal 3D scan model based on the optimal rigid transformation parameters.

[0138] Specifically, this embodiment corresponds one-to-one with the above method embodiments. The functions of each module have been described in detail in the corresponding method embodiments, so they will not be repeated here.

[0139] According to a third aspect of this application, this embodiment provides a computer-readable storage medium having a computer program stored thereon, the computer program including executable instructions that, when executed by a processor, implement the method described above.

[0140] The present invention can implement all or part of the processes in the above methods, or it can be accomplished by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or system capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0141] According to a fourth aspect of this application, an electronic device is provided, such as... Figure 7 As shown, it includes:

[0142] One or more processors;

[0143] Memory is used to store executable instructions for the processor, which, when executed by one or more processors, cause one or more processors to implement the methods described above.

[0144] Electronic devices are manifested in the form of general-purpose computing devices. Components of an electronic device may include, but are not limited to: at least one processor, at least one memory, and a bus connecting different system components (including memory and processor).

[0145] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of a computer system, connecting all parts of the computer system through various interfaces and lines.

[0146] Memory can be used to store computer programs and / or modules. The processor implements various functions of the computer system by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory. Memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system and at least one application program required for a function (e.g., sound playback, image playback, etc.); the data storage area can store data created based on the use of the mobile phone (e.g., audio data, video data, etc.). Furthermore, memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, SmartMedia Cards (SMC), Secure Digital (SD) cards, Flash Cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.

[0147] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, servers, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and memory) containing computer-usable program code.

[0148] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), servers, and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A system that specifies functions in one or more boxes.

[0149] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction set implemented in a process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0150] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0151] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0152] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0153] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for registering X-ray and 3D scanning models based on features and intensity, characterized in that, include: Obtain an anteroposterior X-ray image of the patient's spine, the anteroposterior X-ray image of the spine covering the lower cervical spine to the pelvic region; Obtain a three-dimensional scan model of the patient's spine, the three-dimensional scan model of the spine including the trunk region; The positions of the first feature points in the anteroposterior X-ray image of the spine and the positions of the second feature points mapped from the 3D spinal scanning model to the anteroposterior X-ray image of the spine are obtained. A two-dimensional affine transformation matrix is ​​constructed based on the correlation between the positions of the first and second feature points, wherein the correlation includes scaling and offset. Preliminary registration of the anteroposterior X-ray image of the spine and the 3D spinal scanning model is performed based on the two-dimensional affine transformation matrix, specifically as follows: Obtain the ratio and offset between the position of the second feature point and the position of the first feature point. The ratio includes the horizontal scaling ratio and the vertical scaling ratio. The formulas for calculating the horizontal scaling ratio and the vertical scaling ratio are as follows: ， In the formula, This is the horizontal scaling ratio. This is the vertical scaling ratio. Indicates the horizontal direction. Indicates the vertical direction. This indicates the left anterior superior iliac spine in the X-ray image. This indicates the right anterior superior iliac spine in the X-ray image. This indicates the cricoid cartilage in an X-ray image. This represents the left anterior superior iliac spine in a 3D model. This represents the right anterior superior iliac spine in a 3D model. Represents a three-dimensional model of cricoid cartilage. The coordinates of the left anterior superior iliac spine in the horizontal direction on the X-ray image are given. The coordinates of the right anterior superior iliac spine in the transverse direction on the X-ray image are given. The coordinates of the left anterior superior iliac spine projected from the 3D model onto the 2D plane in the lateral direction. The coordinates of the right anterior superior iliac spine projected from the 3D model onto the 2D plane in the lateral direction. The coordinates of the left anterior superior iliac spine in the longitudinal direction in the X-ray image are given. The coordinates of the cricoid cartilage in the longitudinal direction in the X-ray image are given. The coordinates of the left anterior superior iliac spine projected from the 3D model onto the 2D plane in the longitudinal direction are given. The coordinates of the cricoid cartilage in the three-dimensional model projected onto the two-dimensional plane in the longitudinal direction. This represents the Euclidean distance between two points; The translation vector is obtained by calculating the difference in geometric center positions between the second feature point and the first feature point. ,in, The difference between the geometric center positions of the second feature point and the first feature point is the x-coordinate position. The difference between the geometric center positions of the second feature point and the first feature point is the ordinate position. Construct a two-dimensional affine transformation matrix based on the horizontal scaling ratio, the vertical scaling ratio, and the translation vector. The calculation formula is as follows: A two-dimensional affine transformation matrix is ​​used to perform a two-dimensional affine transformation on the anteroposterior X-ray image of the spine, so as to achieve preliminary registration between the anteroposterior X-ray image and the three-dimensional scanning model of the spine. A structural similarity index is established between the spinal X-ray anteroposterior image and the spinal 3D scan model based on rigid transformation parameters. The structural similarity index is iterated using a constrained optimization algorithm until the optimal structural similarity index is obtained. The optimal rigid transformation parameters are then obtained based on the optimal structural similarity index. Intensity registration is performed between the spinal X-ray anteroposterior image and the spinal 3D scan model based on the optimal rigid transformation parameters. Specifically: The three-dimensional scanning model of the spine is orthogonally projected onto a two-dimensional plane to generate a binary mask image with the same resolution as the anteroposterior X-ray image of the spine. A structural similarity index between binary mask images and spinal X-ray anteroposterior images is established based on rigid transformation parameters; Constraint optimization is performed in a six-dimensional rigid transformation parameter space. The structural similarity index is iterated based on the convergence threshold and / or the maximum number of iterations. The optimization process is carried out by calling the COBYLA algorithm and performing derivative-free constraint optimization through linear approximation. The algorithm is considered to have converged when the gain of the structural similarity index in continuous iterations is lower than the convergence threshold of the structural similarity index or the maximum number of iterations is reached, and the optimal structural similarity index is output. The optimal rigid transformation parameters are obtained based on the optimal structural similarity index. The optimal rigid transformation parameters are then used to perform a three-dimensional rigid transformation on the three-dimensional spinal scanning model, thereby achieving intensity registration between the spinal X-ray anteroposterior image and the three-dimensional spinal scanning model.

2. The method according to claim 1, characterized in that, The process involves acquiring a three-dimensional scan model of the patient's spine, which includes the trunk region and comprises: Obtain an initial 3D scanning model, and then correct the initial 3D scanning model to obtain a standard 3D scanning model; The standard 3D scanning model was cut along the plane of the upper edge of the clavicle and the fold plane of the root of the thigh to remove the head and lower limb regions, thus obtaining the 3D scanning model of the spine.

3. The method according to claim 2, characterized in that, The process of obtaining an initial 3D scanning model and correcting the initial 3D scanning model to obtain a standard 3D scanning model includes: Obtain the third left anterior superior iliac spine point, the third right anterior superior iliac spine point, and the pubic symphysis point of the initial three-dimensional scanning model, and construct a pelvic reference plane P based on the third left anterior superior iliac spine point, the third right anterior superior iliac spine point, and the pubic symphysis point; Align the pelvic reference plane with the XY plane of the world coordinate system to perform rigid rotational transformation correction on the initial three-dimensional scanning model, thereby obtaining a standard three-dimensional scanning model.

4. The method according to claim 1, characterized in that: The first feature point location includes the first annular cartilage point, the first left anterior superior iliac spine point, and the first right anterior superior iliac spine point; the second feature point location includes the second annular cartilage point, the second left anterior superior iliac spine point, and the second right anterior superior iliac spine point.

5. The method according to claim 1, characterized in that, The process involves acquiring the positions of the first feature points in the anteroposterior X-ray image of the spine and the second feature points mapped from the 3D spinal scanning model to the anteroposterior X-ray image of the spine. A two-dimensional affine transformation matrix is ​​constructed based on the correlation between the first and second feature point positions. Preliminary registration of the anteroposterior X-ray image of the spine and the 3D spinal scanning model is then performed based on this two-dimensional affine transformation matrix, including: Mark the first feature point on the anteroposterior X-ray image of the spine and obtain the location of the first feature point; Mark the third feature point on the three-dimensional scanning model of the spine, and map the third feature point on the three-dimensional scanning model of the spine to the two-dimensional image plane of the spinal X-ray anteroposterior image through orthogonal projection to obtain the second feature point of the three-dimensional scanning model of the spine mapped to the spinal X-ray anteroposterior image, and obtain the position of the second feature point. A two-dimensional affine transformation matrix is ​​used to perform a two-dimensional affine transformation on the anteroposterior X-ray image of the spine, so as to achieve preliminary registration between the anteroposterior X-ray image and the three-dimensional scanning model of the spine.

6. A feature- and intensity-based X-ray and three-dimensional scanning model registration system, applied to the feature- and intensity-based X-ray and three-dimensional scanning model registration method according to any one of claims 1-5, characterized in that, The system includes: The first acquisition module is used to acquire an anteroposterior X-ray image of the patient's spine, the anteroposterior X-ray image of the spine covering the lower cervical spine to the pelvic region; The second acquisition module is used to acquire a three-dimensional scan model of the patient's spine, the three-dimensional scan model of the spine including the trunk region; The first registration module is used to obtain the position of the first feature point of the spinal X-ray anteroposterior image and the position of the second feature point of the spinal three-dimensional scanning model mapped to the spinal X-ray anteroposterior image. Based on the correlation between the positions of the first and second feature points, a two-dimensional affine transformation matrix is ​​constructed, and the spinal X-ray anteroposterior image and the spinal three-dimensional scanning model are initially registered based on the two-dimensional affine transformation matrix. The second registration module is used to establish a structural similarity index between the spinal X-ray anteroposterior image and the spinal 3D scan model based on rigid transformation parameters, iterate the structural similarity index based on a constrained optimization algorithm until the optimal structural similarity index is obtained, obtain the optimal rigid transformation parameters based on the optimal structural similarity index, and perform intensity registration between the spinal X-ray anteroposterior image and the spinal 3D scan model based on the optimal rigid transformation parameters.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program includes executable instructions that, when executed by a processor, implement the method of any one of claims 1-5.

8. An electronic device, characterized in that, include: One or more processors; A memory for storing executable instructions of the processor, which, when executed by the one or more processors, cause the one or more processors to perform the method according to any one of claims 1-5.

Images