Method, apparatus and device for evaluating balance state based on spinal-pelvic alignment

Abstract

The present application provides a method, device and equipment for evaluating the balance state based on the spinal-pelvic force line, which relates to the field of medical technology and can more accurately evaluate the body balance state of a patient. The method includes: obtaining M groups of first feature points, where the first feature points are feature points extracted based on two adjacent target images, the target images are two-dimensional images of the spine-pelvis of a target patient, and M is an integer greater than 1; splicing multiple target images according to the M groups of first feature points to obtain a first spliced image; performing three-dimensional reconstruction on the first spliced image through a trained three-dimensional image reconstruction network to obtain a first three-dimensional image; determining three-dimensional morphological parameters of the spine-pelvis of the target patient according to the first three-dimensional image; determining the spinal-pelvic force line according to the three-dimensional morphological parameters; and evaluating the body balance state of the target patient according to the spinal-pelvic force line.

Description

Technical Field

[0001] This application relates to the field of medical technology, and particularly to a method, device and equipment for evaluating the balance state based on the spinal-pelvic alignment. Background Art

[0002] The spine and pelvis are interrelated in terms of morphology and alignment, enabling a person to maintain a stable posture and movement with the least energy consumption. The change of the spinal-pelvic alignment is one of the common pathological changes in patients with lumbar spine diseases and low back pain. The spinal-pelvic alignment parameters play an important reference role in evaluating the patient's body balance state, surgical effect, postoperative recovery, and progression of skeletal deformities, etc., and are important reference parameters for the treatment of diseases such as scoliosis, ankylosing spondylitis, spinal fractures, pelvic fractures, lower limb deformities and fractures.

[0003] Currently, clinically, parameter measurement is carried out based on the lateral view of the spine to obtain the spinal-pelvic alignment parameters of sagittal plane balance. However, since the lateral view of the spine is a two-dimensional sagittal plane balance parameter, it has limitations in dimension and cannot accurately evaluate the spinal-pelvic alignment, thus unable to accurately evaluate the patient's body balance state, which is not conducive to the doctor's preoperative planning and postoperative evaluation. Summary of the Invention

[0004] The embodiments of this application provide a method, device and equipment for evaluating the balance state based on the spinal-pelvic alignment, which can more accurately evaluate the patient's body balance state and is conducive to the doctor's better preoperative planning and postoperative evaluation.

[0005] In the first aspect, this application provides a method for evaluating the balance state based on the spinal-pelvic alignment, including: obtaining M groups of first feature points, where the first feature points are feature points extracted based on two adjacent target images, the target images are two-dimensional images of the spine and pelvis of a target patient, and M is an integer greater than 1; splicing multiple target images according to the M groups of first feature points to obtain a first spliced image; performing three-dimensional reconstruction on the first spliced image through a trained three-dimensional image reconstruction network to obtain a first three-dimensional image; determining the three-dimensional morphological parameters of the spine and pelvis of the target patient according to the first three-dimensional image; determining the spinal-pelvic alignment according to the three-dimensional morphological parameters; and evaluating the body balance state of the target patient according to the spinal-pelvic alignment.

[0006] In the embodiments of the present application, multiple target images are stitched by using multiple sets of feature points extracted from two adjacent target images, and the first stitched image obtained by stitching is three-dimensionally reconstructed by a trained neural network model to obtain a first three-dimensional image with high matching degree and accuracy. Then, the three-dimensional morphological parameters of the spine-pelvis of the target patient are determined based on the first three-dimensional image, so as to obtain a more accurate spine-pelvis alignment line, improve the accuracy of the evaluation of the patient's body balance state, and facilitate the doctor to better perform preoperative planning and postoperative evaluation.

[0007] In a second aspect, the present application provides a balance state evaluation device based on the spine-pelvis alignment line, including:

[0008] A first feature point acquisition unit, configured to acquire M sets of first feature points, where the first feature points are the feature points extracted from a set of target images, the target images are two-dimensional images of the spine-pelvis of the target patient, each set of the target images are two adjacent target images, and M is an integer greater than 1;

[0009] An image stitching unit, configured to stitch multiple target images according to the M sets of first feature points to obtain a first stitched image;

[0010] A first three-dimensional reconstruction unit, configured to three-dimensionally reconstruct the first stitched image by a trained three-dimensional image reconstruction network to obtain a first three-dimensional image;

[0011] A three-dimensional morphological parameter determination unit, configured to determine the three-dimensional morphological parameters of the spine-pelvis of the target patient according to the first three-dimensional image;

[0012] A spine-pelvis alignment line determination unit, configured to determine the spine-pelvis alignment line according to the three-dimensional morphological parameters;

[0013] A balance state evaluation unit, configured to evaluate the body balance state of the target patient according to the spine-pelvis alignment line.

[0014] In a third aspect, the present application provides a balance state evaluation device based on the spine-pelvis alignment line, including a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method described in the first aspect or any optional implementation manner of the first aspect is implemented.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program, where when the computer program is executed by a processor, the method described in the first aspect or any optional implementation manner of the first aspect is implemented.

[0016] Fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on an evaluation device for the balance state based on the spinal-pelvic alignment line, it causes the evaluation device for the balance state based on the spinal-pelvic alignment line to execute the steps of the method for evaluating the balance state based on the spinal-pelvic alignment line described in the above first aspect.

[0017] It can be understood that the beneficial effects of the above second aspect to fifth aspect can be referred to the relevant descriptions in the above first aspect, and will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0019] Figure 1 is a schematic flowchart of a method for evaluating the balance state based on the spinal-pelvic alignment line provided by an embodiment of the present application;

[0020] Figure 2 is a schematic flowchart of a method for splicing a target image provided by an embodiment of the present application;

[0021] Figure 3 is a schematic flowchart of a three-dimensional reconstruction method provided by an embodiment of the present application;

[0022] Figure 4 is a schematic diagram of the architecture of a three-dimensional image reconstruction network model provided by an embodiment of the present application;

[0023] Figure 5 is a schematic flowchart of a method for determining three-dimensional morphological parameters provided by an embodiment of the present application;

[0024] Figure 6 is a schematic diagram of the structure of an evaluation device for the balance state based on the spinal-pelvic alignment line provided by an embodiment of the present application;

[0025] Figure 7 is a schematic diagram of the structure of an evaluation device for the balance state based on the spinal-pelvic alignment line provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] In the following description, specific details such as specific system architectures, technologies, etc. are presented for purposes of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from obscuring the description of the present application.

[0027] It should be understood that the term “and / or” used in the specification and appended claims of the present application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations. Additionally, in the description of the specification and appended claims of the present application, the terms “first,” “second,” “third,” etc. are only used for distinguishing descriptions and cannot be construed as indicating or implying relative importance.

[0028] It should also be understood that references to “one embodiment” or “some embodiments” etc. described in the specification of the present application mean that specific features, structures, or characteristics described in connection with that embodiment are included in one or more embodiments of the present application. Thus, statements such as “in one embodiment,” “in some embodiments,” “in other some embodiments,” “in still other embodiments,” etc. that appear in different places in this specification are not necessarily all referring to the same embodiment, but mean “one or more but not all embodiments,” unless otherwise specifically emphasized in another way. The terms “comprising,” “including,” “having,” and their variants all mean “including but not limited to,” unless otherwise specifically emphasized in another way.

[0029] In recent years, deep learning has been widely applied in the field of medical imaging. For example, semi-supervised learning is used to reconstruct high-dose volumetric images from low-dose volumetric images. Based on this, a deep convolutional neural network model generates three-dimensional images from single X-ray animal skull images. Its network structure adopts an end-to-end structure and is compared with the structures of other neural network models, demonstrating that this deep convolutional neural network model has better three-dimensional reconstruction effects. Another two-dimensional to three-dimensional network model proposes the idea of converting two-dimensional feature information into a spatial tensor for three-dimensional deconvolution. However, this two-dimensional to three-dimensional network model contains a large number of parameters that need to be updated, resulting in low computational efficiency. Three-dimensional reconstruction technologies based on machine learning have also carried out a number of studies in fields such as dentistry, spine, and chest. However, in practical applications, they often perform poorly on different datasets and cannot accurately evaluate the body balance status of patients with diseases such as scoliosis.

[0030] In the prior art, when taking X-ray images, operators usually focus the field of view on the parts or organs of concern. For example, when obtaining X-ray images of the lateral spine, a set of X-ray films taken generally move along a fixed direction. The X-ray images can be offset in the horizontal or vertical axis, and there is no fixed rule for rows and columns, resulting in the inability to accurately locate the landmark points, and thus the inability to accurately evaluate the patient's body balance state.

[0031] Based on this, the embodiments of the present application provide a method for evaluating the balance state based on the spinal-pelvic force line. By stitching multiple target images according to multiple sets of feature points extracted from adjacent two target images, and performing three-dimensional reconstruction on the first stitched image obtained by stitching through a trained neural network model to quickly obtain a first three-dimensional image with higher matching degree and accuracy, and then determining the three-dimensional morphological parameters of the spine-pelvis of the target patient according to the first three-dimensional image, so as to obtain a more accurate spinal-pelvic force line, improve the accuracy of evaluating the patient's body balance state, and facilitate doctors to better perform preoperative planning and postoperative evaluation.

[0032] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of a method for evaluating the balance state based on the spinal-pelvic force line provided by the embodiments of the present application, and is described in detail as follows:

[0033] Step S101, obtain M groups of first feature points. The above first feature points are feature points extracted from adjacent two target images, and the above target images are two-dimensional images of the spine-pelvis of the target patient, and M is an integer greater than 1.

[0034] In the embodiments of the present application, the X-ray images taken before or after the patient's surgery are the above target images, and the target images contain image data of bone tissues such as the patient's spine and pelvis.

[0035] In some embodiments of the present application, through the Speeded-Up Robust Features (SURF) algorithm, a set of feature points is extracted through steps such as integral image, scale space establishment, spatial extreme point detection, and SURF feature descriptor construction. The above steps are repeated for adjacent two target images to complete the extraction of feature points of adjacent two target images in multiple target images, and multiple groups of feature points are obtained.

[0036] Since each group of feature points is a feature point extracted from adjacent two target images, this group of feature points is the matching feature points in the adjacent two target images, that is, this feature point exists in both of the adjacent two target images. Therefore, each group of feature points is also the image matching point of the adjacent two target images.

[0037] Step S102: Stitch multiple target images based on the M groups of the above first feature points to obtain a first stitched image.

[0038] In the embodiments of the present application, image stitching is based on feature matching between two images. When the number of feature points is too small, the matching degree of the finally stitched image is low, thereby reducing the accuracy of the evaluation of the body balance state of the target patient. When the number of feature points is too large, the computational amount will increase, resulting in low evaluation efficiency. To balance accuracy and efficiency, a predetermined number of feature points can be selected to achieve a balance between the accuracy and efficiency of the balance state evaluation.

[0039] Stitching multiple target images can more comprehensively obtain the relevant image data of bone tissues such as the spine and pelvis contained in the target images, thereby providing a basis for accurately evaluating the body balance state of the patient.

[0040] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of a method for stitching target images provided by the embodiments of the present application, and is described in detail as follows:

[0041] Step S201: Normalize the point coordinates of the M groups of the above first feature points.

[0042] In the embodiments of the present application, the point coordinates of each group of feature points are normalized through a normalization formula to obtain M groups of first feature point coordinates.

[0043] To improve the accuracy of image stitching, before normalizing the point coordinates of the M groups of first feature points, the optimal matching pairs are obtained through the Random Sample Consensus (RANSAC) algorithm, and the mis-matched feature points are removed.

[0044] Step S202: Extract N groups of the above first feature points from the M groups of the above first feature points after normalization processing according to a predetermined value N, where N < M.

[0045] Step S203: Determine the first feature points that meet the preset conditions from the N groups of the above first feature points to obtain T groups of the above first feature points, where T < N < M.

[0046] In an embodiment of the present application, from the M groups of first feature points after normalization processing, any predetermined number N of groups, such as 8 groups of first feature points, are randomly selected. Then, based on the least squares method, the fundamental matrix is calculated according to these N groups of feature points. Then, taking the Sampson distance as the judgment basis, the matching degree of all the first feature points in the M groups of first feature points except the extracted N groups of feature points is calculated to find the correct matching feature points. The fundamental matrix corresponding to the largest number of inliers is selected, and then taking the Sampson distance as the judgment basis, the final inliers that meet the preset conditions are solved, which are the above-mentioned T groups of first feature points.

[0047] It should be noted that the preset condition referred to here is a distance condition, specifically referring to that the distance is less than a predetermined threshold. That is, when the distance between two feature points is less than the predetermined threshold, these two feature points are considered to be actual matching points.

[0048] Step S204: Stitch the target images corresponding to the T groups of the above first feature points to obtain the above first stitched image.

[0049] In an embodiment of the present application, the target images corresponding to the T groups of feature points that meet the preset conditions are stitched to obtain a stitched image with a higher matching degree.

[0050] Step S103: Perform three-dimensional reconstruction on the above first stitched image through a trained three-dimensional image reconstruction network to obtain a first three-dimensional image.

[0051] In an embodiment of the present application, the three-dimensional image reconstruction network includes an encoding network, a 2D-3D data conversion module, and a decoding network. Among them, the encoding network includes a two-dimensional convolutional module and a residual module, and the decoding network includes an image upsampling module and a three-dimensional convolutional module.

[0052] Please refer to Figure 3 and Figure 4 , Figure 3 is a schematic flowchart of a three-dimensional reconstruction method provided by an embodiment of the present application, Figure 4 is a schematic architecture diagram of a three-dimensional image reconstruction network model provided by an embodiment of the present application. Combining Figure 4 , for Figure 3 the three-dimensional reconstruction method shown is described in detail as follows:

[0053] Step S301: Extract the two-dimensional image features of the above first stitched image through the above encoding network.

[0054] In an embodiment of the present application, the three-dimensional image reconstruction network model is mainly used to reconstruct a two-dimensional X-ray image into a three-dimensional image, such as three-dimensional CT image data.

[0055] Such as Figure 4As shown, in the above three-dimensional image reconstruction network model, the encoding network consists of a two-dimensional convolutional module and several, for example, 4 residual modules. The first spliced image is input into the encoding network, and image feature extraction is performed by the two-dimensional convolutional module and several residual modules to obtain the two-dimensional image features of the first spliced image. Among them, the residual module consists of a two-dimensional convolutional layer, a batch normalization layer, and an activation layer.

[0056] It should be noted that the above residual module is a basic residual network model, such as ResNet34, which uses the MSE value between the output and the label as the loss and uses the Adam optimizer to optimize the network. Other optimizers, such as gradient descent, SGD, etc., can also be used.

[0057] It should also be noted that the image size of the first spliced image input into the encoding network and the image size of the first three-dimensional image output by the decoding network can be selected according to specific configurations. The larger the input and output image sizes, the higher the resolution, but at the same time, the number of parameters of the three-dimensional reconstruction network model increases, the computing cost increases, and the efficiency decreases.

[0058] In an embodiment of the present application, the encoding network consists of a two-dimensional convolutional module and four residual modules. The output size and kernel size of each module are shown in Table 1:

[0059]

[0060] Table 1

[0061] Step S302, convert the above two-dimensional image features into three-dimensional image features through the above 2D-3D data conversion module.

[0062] In the embodiment of the present application, the essence of the 2D-3D data conversion module is to reorganize the image features and redefine the dimension of the image, that is, the dimension of the image is converted from two-dimensional to three-dimensional.

[0063] In an embodiment of the present application, change the resolution of the two-dimensional image in the depth direction according to a preset resolution, and then convert the two-dimensional image features into three-dimensional image features through the 2D-3D data conversion module with the changed resolution in the depth direction. By adopting a resolution different from the x direction and the y direction in the depth direction, that is, the z direction, the computing amount of the 2D-3D data conversion module is reduced, and the conversion efficiency of the 2D-3D data conversion module is improved.

[0064] Step S303, decode and reconstruct the above three-dimensional image features through the above decoding network to obtain the above first three-dimensional image.

[0065] In the embodiment of the present application, the decoding network is used to decode and reconstruct the three-dimensional image features converted by the 2D-3D data conversion module to obtain a three-dimensional image.

[0066] As Figure 4 shown, the decoding network consists of several image upsampling modules and a three-dimensional convolutional module. After multiple image upsamplings, the image data output by the last image upsampling module is input into the three-dimensional convolutional module for reconstruction, and a three-dimensional image reconstructed based on multiple target images is obtained.

[0067] In an embodiment of the present application, the decoding network consists of 4 image upsampling modules and a three-dimensional convolutional module. The image upsampling module includes a 3D conversion module (abbreviated as 3D conversion in Table 2) and a simplified Inception network (abbreviated as Simple Inception in Table 2). The output size, kernel size, and depth of each module are shown in Table 2:

[0068]

[0069] Table 2

[0070] Step S104, determine the three-dimensional morphological parameters of the spine-pelvis of the target patient according to the above first three-dimensional image.

[0071] In the embodiment of the present application, after obtaining the first three-dimensional image through the three-dimensional image reconstruction network, it is necessary to determine the three-dimensional morphological parameters of the spine-pelvis of the target patient based on this first three-dimensional image. The three-dimensional morphological parameters referred to here include, but are not limited to, the center point of the spine, the center point of the vertebra, the distance between each segment of the vertebrae, the posture of the vertebrae, the three-dimensional curve of the spine, etc.

[0072] Since the first three-dimensional image also includes other tissues besides bone tissue, it is necessary to preprocess this three-dimensional image to obtain three-dimensional image data containing only bone tissue, that is, the second three-dimensional image.

[0073] Please refer to Figure 5 , Figure 5 which is a schematic flowchart of a method for determining three-dimensional morphological parameters provided by an embodiment of the present application, and is described in detail as follows:

[0074] Step S501, use a Gaussian smoothing function and a level set function to perform a rough segmentation process on the above first three-dimensional image, and generate an initial contour map of the bone tissue in a predetermined area of the bone tissue in the above first three-dimensional image.

[0075] Step S502, use a regularization energy function and a level set evolution function, and use the region histograms inside and outside the level set contour to perform a fine segmentation process on the above initial contour map to obtain a binary mask image of the bone tissue.

[0076] Step S503: Perform three-dimensional reconstruction on the above binary mask image to obtain a second three-dimensional image, which is the three-dimensional image data of the bone tissue.

[0077] In the embodiment of the present application, the first three-dimensional image is smoothed by a Gaussian smoothing function to remove noises such as burrs on the image surface. Then, a level set function is used to generate a uniform and smooth initial contour near the boundary of the target region of the bone tissue, completing the rough segmentation process of the first three-dimensional image. Then, the level set function is initialized, and a regularization energy function and the level set function are used to solve the weak boundary leakage problem by using the region histograms inside and outside the level set contour, thereby obtaining a bone tissue segmentation image with higher accuracy.

[0078] In the embodiment of the present application, the above binary mask image is preprocessed by a Gaussian smoothing function to remove the image noise of the binary mask image; then, through an image affine transformation algorithm, the binary mask image after removing the image noise is subjected to pit filling to obtain a second three-dimensional image, that is, a bone tissue segmentation image.

[0079] Step S504: Extract the second feature points in the above second three-dimensional image.

[0080] In the embodiment of the present application, after obtaining the second three-dimensional image, it is exported to a geometric file format that can extract patches, such as the STL file format, and then the vertices of the surface triangular patches are extracted from the STL file as the feature points in the second three-dimensional image.

[0081] Then, the feature points in the second three-dimensional image are extracted through a feature point recognition network. The feature points referred to here, that is, the second feature points, are the feature recognition points of bone tissues such as the skull, spine, pelvis, and femur. The feature point recognition network is built based on the convolutional neural network model CNN. The difference between the feature point recognition networks for different parts lies in the different numbers of outputs, and the number of outputs is 3*P, where P is the number of feature points for different parts.

[0082] When extracting the second feature points in the second three-dimensional image, first focus on specific parts of the head, spinal vertebrae, pelvis, and legs, and then input the image data corresponding to the focused parts into the feature recognition point networks corresponding to each part, and output the three-dimensional feature point coordinates of each part respectively, completing the extraction of the second feature points.

[0083] Step S505: Determine the three-dimensional morphological parameters of the spine-pelvis of the target patient according to the above second feature points.

[0084] In the embodiment of the present application, based on the second feature points corresponding to different parts of each bone tissue, the three-dimensional morphological parameters of the spine-pelvis of the target patient are determined through reference feature points and rotation matrices.

[0085] The second feature point and the reference feature point can be regarded as a system of two sets of vectors plus an axial rotation angle. According to matrix operations, the rotation matrix R can be calculated. Then, by using the rotation matrix array equation based on Euler angles, three spatial Euler angles can be solved. That is, through the corresponding second feature points, reference feature points of each part, and the rotation matrix determined by the second feature points and the reference feature points, the three-dimensional Euler rotation angles of each part of the bone tissue can be determined, thereby determining the three-dimensional morphological parameters of the spine-pelvis of the target patient.

[0086] In the embodiments of the present application, the three-dimensional morphological parameters of the spine-pelvis of the target patient can be quickly and effectively determined through the level set function and the regularization energy function, and have a high accuracy rate, providing a good judgment basis for evaluating the body balance state of the patient and improving the evaluation accuracy rate.

[0087] Step S105: Determine the spine-pelvis force line according to the above three-dimensional morphological parameters.

[0088] In the embodiments of the present application, the spine-pelvis force line determined according to the three-dimensional morphological parameters is a three-dimensional force line, which is the connection of the three-dimensional spatial positions of each segment of the spine, such as the center position of the vertebral body. In addition, it includes the pose states of characteristic positions, such as the spatial angles of the sacrum, pelvis, and vertebral body, that is, the three-dimensional inclination angles. Based on the three-dimensional morphological parameters, the spine-pelvis force line can be obtained through geometric measurement.

[0089] Step S106: Evaluate the body balance state of the above target patient according to the above spine-pelvis force line.

[0090] In the embodiments of the present application, the spine-pelvis force line is an important indicator for evaluating the body balance state of the patient. After determining a more comprehensive and accurate spine-pelvis force line, it can help doctors accurately evaluate the body balance state of the patient, such as evaluating the spine-pelvis balance state of adolescent idiopathic scoliosis.

[0091] In the embodiments of the present application, multiple target images are stitched by using feature points extracted from adjacent two target images, and the first stitched image obtained by stitching is three-dimensionally reconstructed through a trained neural network model to quickly obtain a first three-dimensional image with a higher matching degree and accuracy. Then, the three-dimensional morphological parameters of the spine-pelvis of the target patient are determined according to the first three-dimensional image, thereby obtaining a more accurate spine-pelvis force line, improving the evaluation accuracy of the body balance state of the patient, and facilitating doctors to better perform preoperative planning and postoperative evaluation.

[0092] It should be understood that the magnitudes of the sequence numbers of the above steps in the embodiments do not mean the order of execution. The order of execution of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0093] Based on the balance state evaluation method of the spinal-pelvic alignment provided in the above embodiments, the embodiments of the present application further provide an apparatus embodiment for implementing the above method embodiments.

[0094] Please refer to Figure 6 , Figure 6 which is a schematic diagram of the balance state evaluation apparatus of the spinal-pelvic alignment provided by the embodiments of the present application. Each unit included is used to execute Figure 1 the respective steps in the corresponding embodiments. For specific details, please refer to Figure 1 the relevant descriptions in the corresponding embodiments. For the sake of convenience of description, only the parts related to this embodiment are shown. Refer to Figure 6 , the balance state evaluation apparatus 6 of the spinal-pelvic alignment includes:

[0095] A first feature point acquisition unit 61, configured to acquire M groups of first feature points, where the first feature points are feature points extracted based on a set of target images, the target images are two-dimensional images of the spine-pelvis of a target patient, each group of the target images are two adjacent target images, and M is an integer greater than 1;

[0096] An image stitching unit 62, configured to stitch multiple target images according to the M groups of first feature points to obtain a first stitched image;

[0097] A first 3D reconstruction unit 63, configured to perform 3D reconstruction on the first stitched image through a trained 3D image reconstruction network to obtain a first 3D image;

[0098] A 3D morphological parameter determination unit 64, configured to determine the 3D morphological parameters of the spine-pelvis of the target patient according to the first 3D image;

[0099] A spine-pelvis alignment determination unit 65, configured to determine the spine-pelvis alignment according to the 3D morphological parameters;

[0100] A balance state evaluation unit 66, configured to evaluate the body balance state of the target patient according to the spine-pelvis alignment.

[0101] In some embodiments of the present application, the image stitching unit 62 includes:

[0102] A normalization processing subunit, configured to perform normalization processing on the point coordinates of the M groups of first feature points;

[0103] A feature point extraction subunit, configured to extract N groups of the first feature points from the M groups of first feature points after normalization processing according to a predetermined value N, where N < M;

[0104] A feature point confirmation subunit, configured to determine the first feature points that meet a preset condition from N groups of the above-mentioned first feature points, and obtain T groups of the above-mentioned first feature points, where T < N < M;

[0105] An image stitching subunit, configured to stitch the target images corresponding to T groups of the above-mentioned first feature points to obtain the above-mentioned first stitched image.

[0106] In the embodiments of the application, the above-mentioned three-dimensional image reconstruction network includes an encoding network, a 2D-3D data conversion module, and a decoding network. The above-mentioned encoding network includes a two-dimensional convolution module and a residual module, and the above-mentioned residual module is composed of a two-dimensional convolution layer, a batch normalization layer, and an activation layer.

[0107] In some embodiments of the present application, the first three-dimensional reconstruction unit 63 includes:

[0108] A two-dimensional image feature extraction subunit, configured to extract two-dimensional image features of the above-mentioned first stitched image through the above-mentioned encoding network;

[0109] An image dimension conversion subunit, configured to convert the above-mentioned two-dimensional image features into three-dimensional image features through the above-mentioned 2D-3D data conversion module;

[0110] A first three-dimensional reconstruction subunit, configured to decode and reconstruct the above-mentioned three-dimensional image features through the above-mentioned decoding network to obtain the above-mentioned first three-dimensional image.

[0111] In some other embodiments of the present application, the image dimension conversion subunit is further specifically configured to:

[0112] Change the resolution of the above-mentioned two-dimensional image in the depth direction according to a preset resolution;

[0113] Convert the above-mentioned two-dimensional image features into the above-mentioned three-dimensional image features through the above-mentioned 2D-3D data conversion module with the changed resolution in the depth direction.

[0114] In some other embodiments of the present application, the three-dimensional shape parameter determination unit 64 includes:

[0115] An image rough segmentation subunit, configured to perform rough segmentation processing on the above-mentioned first three-dimensional image by using a Gaussian smoothing function and a level set function, and generate an initial contour map of the bone tissue in a predetermined area of the bone tissue in the above-mentioned first three-dimensional image;

[0116] An image fine segmentation subunit, configured to perform fine segmentation processing on the above-mentioned initial contour map by using a regularization energy function and a level set evolution function, and using the region histograms inside and outside the level set contour, to obtain a binary mask image of the bone tissue;

[0117] A second 3D reconstruction subunit, configured to perform 3D reconstruction on the above binary mask image to obtain a second 3D image, where the second 3D image is 3D imaging data of the above bone tissue;

[0118] A second feature point extraction subunit, configured to extract second feature points in the above second 3D image;

[0119] A 3D morphological parameter determination subunit, configured to determine 3D morphological parameters of the spine-pelvis of the above target patient according to the above second feature points.

[0120] In some embodiments of the present application, the second 3D reconstruction subunit includes:

[0121] An image noise removal subunit, configured to preprocess the above binary mask image through a Gaussian smoothing function to remove image noise in the above binary mask image;

[0122] An image filling subunit, configured to perform pit filling on the above binary mask image after removing image noise through an image affine transformation algorithm to obtain the above second 3D image.

[0123] The 3D morphological parameter determination unit 64 is specifically further configured to:

[0124] Based on the second feature points corresponding to different parts of each bone tissue, determine the 3D morphological parameters of the spine-pelvis of the above target patient through a reference feature point and a rotation matrix.

[0125] It should be noted that for the information interaction, execution process, etc. between the above modules, since they are based on the same concept as the method embodiments of the present application, their specific functions and the technical effects brought are specifically described in the method embodiment part, and will not be elaborated here.

[0126] Figure 7 is a schematic diagram of an equilibrium state evaluation device based on the spine-pelvis force line provided by an embodiment of the present application. As Figure 7 shown, the equilibrium state evaluation device 7 based on the spine-pelvis force line in this embodiment includes: a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70, such as a voice recognition program. When the processor 70 executes the computer program 72, it implements the steps in the above method embodiments of each equilibrium state evaluation method based on the spine-pelvis force line, such as Figure 1 the steps 101-106 shown. Or, when the processor 70 executes the computer program 72, it implements the functions of each module / unit in the above device embodiments, such as Figure 6 the functions of the units 61-66 shown.

[0127] Exemplarily, the computer program 72 can be divided into one or more modules / units, and one or more modules / units are stored in the memory 71 and executed by the processor 70 to complete the present application. One or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program 72 in the balance state evaluation device 7 based on the spinal-pelvic alignment. For example, the computer program 72 can be divided into a first feature point acquisition unit 61, an image stitching unit 62, a first three-dimensional reconstruction unit 63, a three-dimensional morphological parameter determination unit 64, a spinal-pelvic alignment determination unit 65, and a balance state evaluation unit 66. For the specific functions of each unit, please refer to Figure 1 the relevant descriptions in the corresponding embodiments, which will not be elaborated here.

[0128] The balance state evaluation device based on the spinal-pelvic alignment may include, but is not limited to, the processor 70 and the memory 71. Those skilled in the art can understand that Figure 7 merely being examples of the balance state evaluation device 7 based on the spinal-pelvic alignment, which do not constitute a limitation on the balance state evaluation device 7 based on the spinal-pelvic alignment. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the balance state evaluation device based on the spinal-pelvic alignment may also include input / output devices, network access devices, buses, etc.

[0129] The so-called processor 70 may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0130] The memory 71 can be an internal storage unit of the balance state evaluation device 7 based on the spinal-pelvic alignment line, such as the hard disk or memory of the balance state evaluation device 7 based on the spinal-pelvic alignment line. The memory 71 can also be an external storage device of the balance state evaluation device 7 based on the spinal-pelvic alignment line, such as a plug-in hard disk equipped on the balance state evaluation device 7 based on the spinal-pelvic alignment line, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the memory 71 can also include both the internal storage unit of the balance state evaluation device 7 based on the spinal-pelvic alignment line and the external storage device. The memory 71 is used to store computer programs and other programs and data required by the balance state evaluation device based on the spinal-pelvic alignment line. The memory 71 can also be used to temporarily store the data that has been output or will be output.

[0131] An embodiment of the present application also provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the above-mentioned method for evaluating the balance state based on the spinal-pelvic alignment line can be implemented.

[0132] An embodiment of the present application provides a computer program product, and when the computer program product runs on the balance state evaluation device based on the spinal-pelvic alignment line, the balance state evaluation device based on the spinal-pelvic alignment line can be made to implement the above-mentioned method for evaluating the balance state based on the spinal-pelvic alignment line when executed.

[0133] Those skilled in the art can clearly understand that, for the convenience and conciseness of description, only the above-mentioned division of each functional unit and module is used as an example for illustration. In actual applications, the above-mentioned functions can be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiment and will not be described in detail here.

[0134] In the above embodiments, the descriptions of each embodiment have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0135] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application.

[0136] The above-described embodiments are only used to illustrate the technical solutions of this application, rather than to limit them; although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of each embodiment of this application, and should all be included in the protection scope of this application.

Claims

1. A method for evaluating the balance state based on the spinal-pelvic alignment line, characterized in that The method includes: Obtaining M groups of first feature points, where the first feature points are feature points extracted based on two adjacent target images, and the target images are two-dimensional images of the spine-pelvis of a target patient, and M is an integer greater than 1; Stitching multiple target images according to the M groups of first feature points to obtain a first stitched image; Performing three-dimensional reconstruction on the first stitched image through a trained three-dimensional image reconstruction network to obtain a first three-dimensional image; Determining three-dimensional morphological parameters of the spine-pelvis of the target patient according to the first three-dimensional image; Determining the spine-pelvis force line according to the three-dimensional morphological parameters; Evaluating the body balance state of the target patient according to the spine-pelvis force line; The determining the three-dimensional morphological parameters of the spine-pelvis of the target patient according to the first three-dimensional image includes: Performing rough segmentation processing on the first three-dimensional image by using a Gaussian smoothing function and a level set function, and generating an initial contour map of the bone tissue in a predetermined area of the bone tissue in the first three-dimensional image; Performing fine segmentation processing on the initial contour map by using a regularization energy function and a level set evolution function, and using the region histograms inside and outside the level set contour to obtain a binary mask image of the bone tissue; Performing three-dimensional reconstruction on the binary mask image to obtain a second three-dimensional image, where the second three-dimensional image is three-dimensional image data of the bone tissue; Extracting second feature points in the second three-dimensional image; Determining the three-dimensional morphological parameters of the spine-pelvis of the target patient according to the second feature points.

2. The method for evaluating the balance state based on the spinal-pelvic alignment according to claim 1, wherein The stitching multiple target images according to the M groups of first feature points to obtain a first stitched image includes: Normalizing the point coordinates of the M groups of first feature points; Extracting N groups of first feature points from the M groups of first feature points after normalization processing according to a predetermined value N, where N < M; Determining the first feature points that meet the preset conditions from the N groups of first feature points to obtain T groups of first feature points, where T < N < M; Stitching the target images corresponding to the T groups of first feature points to obtain the first stitched image.

3. The method for evaluating the balance state based on the spinal-pelvic alignment according to claim 1 or 2, characterized in that, The three-dimensional image reconstruction network includes an encoding network, a 2D-3D data conversion module, and a decoding network; the performing three-dimensional reconstruction on the first stitched image through a trained three-dimensional image reconstruction network to obtain a first three-dimensional image includes: Extracting two-dimensional image features of the first stitched image through the encoding network; the encoding network includes a two-dimensional convolution module and a residual module, and the residual module is composed of a two-dimensional convolutional layer, a batch normalization layer, and an activation layer; Converting the two-dimensional image features into three-dimensional image features through the 2D-3D data conversion module; Performing decoding and reconstruction on the three-dimensional image features through the decoding network to obtain the first three-dimensional image.

4. The method for evaluating the balance state based on the spinal-pelvic alignment according to claim 3, wherein In the converting the two-dimensional image features into three-dimensional image features through the 2D-3D data conversion module, it includes: Changing the resolution of the two-dimensional image in the depth direction according to a preset resolution; The two-dimensional image features are converted into the three-dimensional image features by the 2D-3D data conversion module with the resolution changed in the depth direction.

5. The method for evaluating the balance state based on the spinal-pelvic alignment according to claim 1, wherein The three-dimensional reconstruction of the binary mask image to obtain the second three-dimensional image includes: Preprocessing the binary mask image through a Gaussian smoothing function to remove the image noise of the binary mask image; Performing pit filling on the binary mask image after removing the image noise through an image affine transformation algorithm to obtain the second three-dimensional image.

6. The method for evaluating the balance state based on the spinal-pelvic alignment according to claim 1, wherein The determination of the three-dimensional morphological parameters of the spine-pelvis of the target patient according to the second feature points includes: Based on the second feature points corresponding to different parts of each bone tissue, determining the three-dimensional morphological parameters of the spine-pelvis of the target patient through the reference feature points and the rotation matrix.

7. An equilibrium state evaluation device based on the spinal-pelvic alignment, characterized in that, Applied to the method for evaluating the balance state based on the spine-pelvis force line according to any one of claims 1 to 6, the device includes: A first feature point acquisition unit, configured to acquire M groups of first feature points, where the first feature points are feature points extracted based on a group of target images, the target images are two-dimensional images of the spine-pelvis of a target patient, each group of the target images are two adjacent target images, and M is an integer greater than 1; An image stitching unit, configured to stitch multiple target images according to the M groups of first feature points to obtain a first stitched image; A first three-dimensional reconstruction unit, configured to perform three-dimensional reconstruction on the first stitched image through a trained three-dimensional image reconstruction network to obtain a first three-dimensional image; A three-dimensional morphological parameter determination unit, configured to determine the three-dimensional morphological parameters of the spine-pelvis of the target patient according to the first three-dimensional image; A spine-pelvis force line determination unit, configured to determine the spine-pelvis force line according to the three-dimensional morphological parameters; A balance state evaluation unit, configured to evaluate the body balance state of the target patient according to the spine-pelvis force line.

8. A balance state evaluation device based on the spinal-pelvic alignment line, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for evaluating the balance state based on the spine-pelvis force line according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium storing a computer program, characterized in that, The computer program, when executed by the processor, implements the method for evaluating the balance state based on the spine-pelvis force line according to any one of claims 1 to 6.

Images