A three-dimensional spinal reconstruction method and system

Abstract

The present invention relates to a method and system for three-dimensional reconstruction of the spine, belonging to the technical field of image processing. In the three-dimensional reconstruction method of the spine, after processing the depth image by using the light and shadow information of the color image to obtain the processed depth image, the dorsal spinous process line is obtained based on the color image and the processed depth image. Then, the local symmetry information of the dorsal spinous process line is extracted. After that, based on the prior knowledge of the human spine, a prior model of the human spine is constructed. Based on the prior model of the human spine, a constraint function is constructed by using the local symmetry information of the dorsal spinous process line. Finally, after determining the characteristic parameters based on the constraint function, the three-dimensional model of the spine is accurately obtained according to the characteristic parameters, thereby achieving the purpose of improving the accuracy of spine reconstruction. Moreover, the present invention abandons the processing means based on X-ray films or CT data, which can significantly reduce costs, and at the same time, the use of image processing can also simplify the operation process of three-dimensional spine reconstruction.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a three-dimensional spinal reconstruction method and system. Background Art

[0002] Scoliosis is one of the diseases that seriously endanger the health of humans, especially children and adolescents. In the general population, the prevalence of idiopathic scoliosis varies widely. Most literature believes that the proportion of adolescent idiopathic scoliosis with a Cobb angle exceeding 10° is 2% to 3%. The traditional methods for identifying spinal deformities are to directly measure the angle of the scoliosis curve on an X-ray film, or to use a scoliosis measurer for measurement, or to perform the Adam test. However, these traditional methods cannot reflect the complete three-dimensional condition of the human spine. Moreover, the accuracy of these measurements is not high and is even related to the proficiency of the operator. Therefore, the establishment and application of a 3D spinal model are very important.

[0003] Previously, most of the 3D spinal model reconstruction methods focusing on the diagnosis or classification of scoliosis were based on X-ray films or CT data. For the methods based on X-ray films, generally two X-ray films with different perspectives are required. For such methods that do not use a spinal statistical model, generally matching points are found on the X-ray films from two perspectives, and the spinal model is reconstructed using a stereo matching method. For the methods that use a spinal statistical model, key points are detected on the X-ray films from two perspectives, and then the key points of the spinal statistical model are used to constrain and optimize the parameters of the statistical model. These methods can reconstruct a three-dimensional spinal model with high accuracy. However, since they require X-ray films as input, they have high costs, complex operations, and radiation damage to the human body. Therefore, they are not suitable for the screening of scoliosis and the detection of the spinal state during the treatment process. Summary of the Invention

[0004] To solve the above problems existing in the prior art, the present invention provides a three-dimensional spinal reconstruction method and system.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A three-dimensional spinal reconstruction method, comprising:

[0007] Obtaining a color image and a depth image of the human back;

[0008] Processing the depth image using the light and shadow information of the color image to obtain a processed depth image;

[0009] Obtaining a dorsal spinous process line based on the color image and the processed depth image;

[0010] Extracting local symmetry information of the dorsal spinous process line;

[0011] Based on the prior knowledge of the human spine, a prior model of the human spine is constructed; the prior model of the human spine includes a spine shape statistical model, a spine posture model, and an intervertebral disc mechanics model;

[0012] Based on the prior model of the human spine, a constraint function is constructed using the local symmetry information of the dorsal spinous process line; the constraint function includes: the distance constraint from the end point of the spinous process to the dorsal spinous process line, the local symmetry constraint of the human back surface, the constraint of the mechanical properties of the intervertebral disc, and the parameter regularization constraint of the spine shape statistical model; the constraint of the mechanical properties of the intervertebral disc includes: the constraint of the axial compression / tension property and the constraint of the torsional property about the axis;

[0013] Determine the characteristic parameters based on the constraint function;

[0014] Obtain the three-dimensional model of the spine according to the characteristic parameters.

[0015] Preferably, the extraction of the local symmetry information of the dorsal spinous process line specifically includes:

[0016] In the 2D image space, draw a perpendicular line to the dorsal spinous process line through each pixel point on the dorsal spinous process line;

[0017] Taking each pixel point on the dorsal spinous process line in the 2D image space as the center point, intercept a line segment on the perpendicular line according to preset conditions to obtain a perpendicular line segment;

[0018] Extend one pixel at each end of each perpendicular line segment to obtain a strip;

[0019] Taking each pixel point on the dorsal spinous process line in the 2D image space as the center point, select a rectangular area at each quarter of the perpendicular line segment to obtain a first point set;

[0020] Back-project the dorsal spinous process line and the strip in the 2D image space through the camera matrix to obtain the dorsal spinous process line in the 3D image space and the strip in the 3D image space;

[0021] Construct an index tree of the dorsal spinous process line in the 3D image space and an index tree of the strip in the 3D image space;

[0022] Taking the index corresponding to each pixel point on the dorsal spinous process line in the 3D image space as the key and the index tree of the strip corresponding to each pixel point on the dorsal spinous process line in the 3D image space as the value, construct a dictionary;

[0023] Obtain the end point of the spinous process;

[0024] Determine the pixel point closest to the end point of the spinous process on the index tree of the dorsal spinous process line in the 3D image space, and obtain the index of this pixel point;

[0025] Obtain the value corresponding to the index in the dictionary based on the index to obtain the strip of the spinous process end point;

[0026] Perform mirror symmetry on the pixel points in the first point set regarding the spinous process end point, the center point of the vertebral body, and the center point of the intervertebral disc to obtain a second point set;

[0027] Determine the pixel point closest to the pixel points in the second point set on the index tree of the dorsal spinous process line in the 3D image space to obtain a third point set;

[0028] Perform distance minimization constraint on the pixel points in the third point set to obtain the local symmetry information of the dorsal spinous process line.

[0029] Preferably, performing distance minimization constraint on the pixel points in the third point set to obtain the local symmetry information of the dorsal spinous process line specifically includes:

[0030] Adopt the formula Perform distance minimization constraint on the pixel points in the third point set to obtain the local symmetry information of the dorsal spinous process line;

[0031] where Q′ is the pixel point in the second point set, Q″ is the pixel point in the third point set, and L symm is the local symmetry constraint value.

[0032] Preferably, the preset condition is: the length of the intercepted line segment does not exceed one-half of the range of the dorsal region in the image, and the length of the intercepted line segment decreases from the lumbar vertebra to the cervical vertebra.

[0033] Preferably, the formula for the decreasing length of the intercepted line segment is:

[0034]

[0035] where W is half of the length of the intercepted line segment, W 0 is half of the length of the first intercepted line segment, N is the total number of pixels of the dorsal spinous process line, and i is the i-th pixel on the dorsal spinous process line, with i increasing from the lumbar vertebra to the cervical vertebra.

[0036] Preferably, the spinal shape statistical model is:

[0037]

[0038] where T k is the spinal shape statistical model of the k-th vertebral body, is the mean vertebral body shape of the k-th vertebral body, B k is the spinal shape parameter matrix of the k-th vertebral body, and β is the spinal shape parameter.

[0039] Preferably, the spinal posture model is:

[0040]

[0041] J k =(M k-1 +M k ) / 2;

[0042] Wherein, M k is the center point of the k-th vertebral body, is the first auxiliary point located on the surface of the k-th vertebral body, is the second auxiliary point located on the surface of the k-th vertebral body, J k is the center point of the intervertebral disc of the k-th vertebral body, M k-1 is the center point of the (k - 1)-th vertebral body.

[0043] Preferably, the intervertebral disc mechanical model is:

[0044]

[0045]

[0046] Wherein, is the axial compression / tension characteristic of the k-th vertebral body, n is the number of deformations, is the axial deformation amount of the k-th vertebral body during the j-th deformation, is the torsional characteristic about the axis of the k-th vertebral body, is the torsional angle of the k-th vertebral body, and λ is a constant factor.

[0047] Preferably, the constraint function is:

[0048]

[0049] Wherein, L is the constraint function, is the distance constraint from the end point of the spinous process of the spinal model to the k-th vertebral body on the dorsal spinous process line, is the local symmetry constraint of the human back surface of the k-th vertebral body, is the axial compression / tension characteristic of the k-th vertebral body, is the torsional characteristic about the axis of the k-th vertebral body, α 1 、α 2 and α 3 are all weight coefficients, and L R is the parameter regularization constraint of the spinal shape statistical model.

[0050] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0051] The three-dimensional spinal reconstruction method provided by the present invention is based on the color image and depth image information of the human back surface, and combines the prior models of the human spine, including the spinal shape statistical model, the spinal posture model, and the intervertebral disc mechanical model. Without using X-ray film information as input, a three-dimensional model of the human spine with high precision can be reconstructed, solving the problems of inconvenience, high cost, and radiation damage caused by X-ray film shooting, and can be applied to the screening of scoliosis and the spinal monitoring during the treatment of scoliosis.

[0052] Corresponding to the above-provided three-dimensional spinal reconstruction method, the present invention also provides a three-dimensional spinal reconstruction system, including:

[0053] An image acquisition module for acquiring the color image and depth image of the human back;

[0054] An image processing module for processing the depth image by using the light and shadow information of the color image to obtain a processed depth image;

[0055] A dorsal spinous process line determination module for obtaining the dorsal spinous process line based on the color image and the processed depth image;

[0056] A local symmetry information extraction module for extracting the local symmetry information of the dorsal spinous process line;

[0057] A prior model construction module of the human spine for constructing a prior model of the human spine based on the prior knowledge of the human spine; the prior model of the human spine includes a spinal shape statistical model, a spinal posture model, and an intervertebral disc mechanical model;

[0058] A constraint function construction module for constructing a constraint function based on the prior model of the human spine and using the local symmetry information of the dorsal spinous process line; the constraint function includes: the distance constraint from the end point of the spinous process to the dorsal spinous process line, the local symmetry constraint of the human back surface, the intervertebral disc mechanical property constraint, and the parameter regularization constraint of the spinal shape statistical model; the intervertebral disc mechanical property constraint includes: the constraint of the axial compression / tension property and the constraint of the torsional property around the axis;

[0059] A characteristic parameter determination module for determining characteristic parameters based on the constraint function;

[0060] A three-dimensional spinal model construction module for obtaining a three-dimensional model of the spine according to the characteristic parameters.

[0061] Since the technical effects achieved by the three-dimensional spinal reconstruction system provided by the present invention are the same as those achieved by the above-provided three-dimensional spinal reconstruction method, they will not be elaborated here. Description of the Drawings

[0062] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0063] Figure 1 It is a flowchart of the three-dimensional spine reconstruction method provided by the present invention;

[0064] Figure 2 It is an image processing block diagram of the three-dimensional spine reconstruction method provided by the embodiments of the present invention;

[0065] Figure 3 It is a flowchart for extracting local symmetry information of the dorsal spinous process line provided by the embodiments of the present invention;

[0066] Figure 4 It is a schematic structural diagram of the three-dimensional spine reconstruction system provided by the present invention. Detailed implementation manners

[0067] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0068] The purpose of the present invention is to provide a three-dimensional spine reconstruction method and system with low cost and simple operation, so as to improve the accuracy of spine reconstruction.

[0069] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the drawings and specific implementation manners.

[0070] As Figure 1 shown, the three-dimensional spine reconstruction method provided by the present invention includes:

[0071] Step 100: Obtain a color image and a depth image of the human back. In the present invention, mainly the color images including the positions of spinous processes with fluorescence markings and those without fluorescence markings are obtained.

[0072] Step 101: Process the depth image using the light and shadow information of the color image to obtain a processed depth image. In this step, mainly the light and shadow information of the color image without markings is used to process the depth image to obtain a refined depth image. For this specific processing process, reference can be made to the public content of the invention document named "A Method and System for Detecting Spinous Process Lines on the Human Back".

[0073] Step 102: Obtain the dorsal spinous process line based on the color image and the processed depth image.

[0074] Step 103: Extract the local symmetry information of the dorsal spinous process line. The implementation process of this step can be as follows:

[0075] Step 103-0: In the 2D image space, draw a perpendicular line to the dorsal spinous process line through each pixel point on the dorsal spinous process line.

[0076] Step 103-1: Taking each pixel point on the dorsal spinous process line in the 2D image space as the center point, intercept line segments on the perpendicular lines according to preset conditions to obtain perpendicular line segments. Among them, the preset conditions are: the length of the intercepted line segment does not exceed one-half of the range of the dorsal region in the image, and in the direction from the lumbar vertebra to the cervical vertebra, the length of the line segment intercepted decreases. The formula for the decreasing length of the intercepted line segment is:

[0077]

[0078] where, W is half of the length of the intercepted line segment, W 0 is half of the length of the first intercepted line segment, N is the total number of pixels of the dorsal spinous process line, i is the i-th pixel on the dorsal spinous process line, and i increases from the lumbar vertebra to the cervical vertebra.

[0079] Step 103-2: Extend one pixel at each end of each perpendicular line segment to obtain a strip.

[0080] Step 103-3: Taking each pixel point on the dorsal spinous process line in the 2D image space as the center point, select a rectangular area at each quarter of the perpendicular line segment to obtain a first point set.

[0081] Step 103-4: Back-project the dorsal spinous process line and the strip in the 2D image space through the camera matrix to obtain the dorsal spinous process line in the 3D image space and the strip in the 3D image space.

[0082] Step 103-5: Construct an index tree for the dorsal spinous process line in the 3D image space and an index tree for the strip in the 3D image space.

[0083] Step 103-6: Using the index corresponding to each pixel point on the dorsal spinous process line in the 3D image space as the key and the index tree of the strip corresponding to each pixel point on the dorsal spinous process line in the 3D image space as the value, construct a dictionary.

[0084] Step 103-7: Obtain the end point of the spinous process.

[0085] Step 103-8: Determine the pixel point on the index tree of the dorsal spinous process line in the 3D image space that is closest to the end point of the spinous process, and obtain the index of this pixel point.

[0086] Step 103-9: Obtain the value corresponding to the index in the dictionary based on the index to obtain the strip of the spinous process end point.

[0087] Step 103-10: Make the pixel points in the first point set that are related to the spinous process end point, the vertebral body center point, and the intervertebral disc center point mirror-symmetrical to obtain the second point set.

[0088] Step 103-11: Determine the pixel point closest to the pixel points in the second point set on the index tree of the dorsal spinous process line in the 3D image space to obtain the third point set.

[0089] Step 103-12: Apply distance minimization constraints to the pixel points in the third point set to obtain the local symmetry information of the dorsal spinous process line. Specifically, use the formula Apply distance minimization constraints to the pixel points in the third point set to obtain the local symmetry information of the dorsal spinous process line.

[0090] where Q′ is the pixel point in the second point set, Q″ is the pixel point in the third point set, and L symm is the local symmetry constraint value.

[0091] Step 104: Based on the prior knowledge of the human spine, construct a prior model of the human spine. The prior model of the human spine includes a spine shape statistical model, a spine posture model, and an intervertebral disc mechanics model. Among them, the spine shape statistical model is:

[0092]

[0093] where T k is the spine shape statistical model of the k-th vertebral body, is the mean vertebral body shape of the k-th vertebral body, B k is the spine shape parameter matrix of the k-th vertebral body, and β is the spine shape parameter.

[0094] The spine posture model is:

[0095]

[0096] J k =(M k-1 +M k ) / 2 (4)

[0097] where M k is the center point of the k-th vertebral body, is the first auxiliary point located on the surface of the k-th vertebral body, is the second auxiliary point located on the surface of the k-th vertebral body, J k is the intervertebral disc center point of the k-th vertebral body, and M k-1 is the center point of the (k-1)-th vertebral body.

[0098] The mechanical model of the intervertebral disc is as follows:

[0099]

[0100]

[0101] Among them, is the axial compression / tension characteristic of the k-th vertebral body, n is the number of deformations, is the axial deformation amount of the k-th vertebral body at the j-th deformation, is the torsional characteristic about the axis of the k-th vertebral body, is the torsional angle of the k-th vertebral body, and λ is a constant factor.

[0102] Step 105: Based on the prior model of the human spine, construct a constraint function by using the local symmetry information of the dorsal spinous process line. The constraint function includes: the distance constraint from the end point of the spinous process to the dorsal spinous process line, the local symmetry constraint of the human back surface, the mechanical property constraint of the intervertebral disc, and the parameter regularization constraint of the spinal shape statistical model. The mechanical property constraint of the intervertebral disc includes: the constraint of the axial compression / tension characteristic and the constraint of the torsional characteristic about the axis. Among them, the constraint function is:

[0103]

[0104] Among them, L is the constraint function, which is essentially a function of the spinal posture parameter θ and the spinal vertebral body shape parameter β, is the distance constraint from the end point of the spinous process of the spinal model to the k-th vertebral body on the dorsal spinous process line, is the local symmetry constraint of the human back surface of the k-th vertebral body, is the axial compression / tension characteristic of the k-th vertebral body, is the torsional characteristic about the axis of the k-th vertebral body, α 1 、α 2 and α 3 are all weight coefficients, and L R is the parameter regularization constraint of the spinal shape statistical model. By optimizing the spinal posture parameter θ and the spinal vertebral body shape parameter β, the minimized constraint function L can be obtained.

[0105] Step 106: Determine the characteristic parameters based on the constraint function.

[0106] Step 107: Obtain the three-dimensional model of the spine according to the characteristic parameters.

[0107] The following gives an embodiment to illustrate the specific implementation process of the above spine three-dimensional reconstruction method provided by the present invention.

[0108] In this embodiment, the image acquisition device adopted consists of a depth camera, a color camera, several (at least two) ultraviolet lamps, and a computer. The depth camera, the color camera, and the ultraviolet lamps are jointly controlled by the computer acquisition program. Among them, the KinectV2 is used for the depth camera and the color camera, the irradiation direction of the ultraviolet lamp is the shooting direction of the KinectV2 camera, and the irradiation area (human back area) of the ultraviolet lamp is larger than the shooting area (human back area) of the KinectV2 camera.

[0109] The image acquisition device acquires the back color image and depth image without fluorescence marking when the ultraviolet lamp is turned off, and acquires the back color image and depth image with the spinous process position marked by fluorescence when the ultraviolet lamp is turned on. Among them, the position of the spinous process marked by fluorescence is manually marked on the human back with a fluorescent pen before data acquisition.

[0110] The two acquisition processes with fluorescence marking and without fluorescence marking are controlled by the computer acquisition program and completed in a very short time, so that it can be considered that the human body posture does not change during the two acquisition processes.

[0111] Based on the color image and depth image acquired by the image acquisition device, the above-mentioned spinal three-dimensional reconstruction method provided by the present invention is converted into a computer software program and implanted into the computer of the image acquisition device to realize the three-dimensional reconstruction of the spine.

[0112] Based on this, as Figure 2 shown, the process of data processing in the computer is as follows:

[0113] Step 1: Acquire the color and depth images of the human back through the acquisition device.

[0114] Step 2: Process the depth image with the light and shadow information of the color image without marking to obtain a refined depth image.

[0115] Step 3: Detect the back spinous process line based on the back color and depth images.

[0116] Step 4: Extract and construct the information and data structure of the local symmetry of the back surface based on the depth image and the back spinous process line. The specific implementation process of this step is as Figure 3 shown, including:

[0117] Step 301: In the 2D image space, based on the back spinous process line, draw a perpendicular line to the spinous process line through each pixel point of the back spinous process line, and take each pixel point on the spinous process line as the center point, and intercept the perpendicular line with a shorter length as the perpendicular segment.

[0118] Step 302: In the 2D image space, extend each perpendicular segment up and down by one pixel to obtain a strip perpendicular to the spinous process line.

[0119] Step 303: In the 2D image space, taking each pixel point on the spinous process line as a starting point, select a set of points Q of a 3x3 rectangular area centered at about one-fourth of the length of the perpendicular line segment on both sides.

[0120] Step 304: Back-project the dorsal spinous process line and the strip perpendicular to the spinous process line in the 2D image space through the camera matrix to obtain the dorsal spinous process line and the strip perpendicular to the spinous process line in the 3D space.

[0121] Step 305: Construct kd-trees for the dorsal spinous process line and the strip perpendicular to the spinous process line in the 3D space respectively, and construct a dictionary (dict) with the index of each point on the dorsal spinous process line as the key and the corresponding kd-tree of the strip as the value.

[0122] Based on the dictionary, when in use, first find the nearest point on the kd-tree of the spinous process line with the spinous process end point of the model, obtain its index, then find the corresponding strip perpendicular to the spinous process line in the dictionary through this index, mirror the points on its Q set symmetrically to the other side with respect to the plane formed by the spinous process end point, the vertebral body center, and the intervertebral disc center point to obtain the Q' set, and finally find the nearest point Q'' set of the Q' set on the strip kd-tree and perform a distance minimization constraint with the points of the Q' set.

[0123] The local symmetry of the human back surface means that in the local areas on both sides around the dorsal spinous process line, it satisfies the mirror symmetry relationship with respect to the plane formed by the spinous process end point of the spinal vertebra, the vertebral body center, and the intervertebral disc center point. The dorsal spinous process line is a curve connected by the dorsal spinous process end points.

[0124] Taking each pixel point on the spinous process line as the center point, intercept the perpendicular line with a shorter length as the perpendicular line segment. The shorter length means that the strip does not exceed half of the dorsal area in the image, and from the lumbar region to the cervical region, the length decreases, and the decreasing formula is shown in formula (1).

[0125] The set of points Q of the 3x3 rectangular area should satisfy the following property: The points on this set, with the plane formed by the spinous process end point of the spinal vertebra, the vertebral body center, and the intervertebral disc center point as the symmetry plane, are mirror-symmetrical to the other side to obtain the point set Q'. In theory, all points on the Q' set can find points at the same position on the strip.

[0126] Mirror the points on its Q set symmetrically to the other side with respect to the plane formed by the spinous process end point, the vertebral body center, and the intervertebral disc center point to obtain the Q' set, and finally find the nearest point Q'' set of the Q' set on the strip kd-tree and perform a distance constraint with the points of the Q' set. The formula is as follows:

[0127] Mirror symmetry: Point \(P\) is a point on the symmetry plane. is the normal vector of the symmetry plane.

[0128] Distance constraint: \(L\) symm represents the local symmetry constraint.

[0129] Step 5: Based on the prior knowledge of the human spine, construct a prior model of the human spine, including a spine shape statistical model, a spine posture model, and an intervertebral disc mechanics model.

[0130] Step 6: Based on the prior model of the human spine, use the dorsal spinous process line and local symmetry to construct a constraint function between the human spine model and the human back information, and a constraint function of the mechanical properties of the human spine model itself. Construct a prior model of the human spine, including a spine shape statistical model, a spine posture model, and an intervertebral disc mechanics model. Among them: the spine shape statistical model is a parametric model of the spine vertebrae, which expresses the spine vertebrae shape as a mean model and principal components, and the formula is shown in the above formula (2); the spine posture model is a key point model, see formulas (3) and (4). The intervertebral disc mechanics model is used to simulate the mechanical properties when the intervertebral disc deforms, mainly including axial compression / tension properties and torsional properties around the axis. In the present invention, the two properties are respectively modeled as several non-linear axial springs connected to the edges of adjacent vertebrae and a torsional spring, and the specific expression forms are shown in formulas (5) and (6), where \(n\) can be equivalently regarded as the number of springs. In this embodiment, \(k\in[1,17]\), \(k = 1\) represents the fifth lumbar vertebra \(L5\), \(k = 17\) represents the first thoracic vertebra \(T1\), and the spine and the spine model altogether include 17 vertebrae of the first to fifth lumbar vertebrae \(L1 - L5\) and the first to twelfth thoracic vertebrae \(T1 - T12\).

[0131] Among them, the posture is determined by the chain motion of each vertebra rotating around the center point of its corresponding intervertebral disc, and its mathematical formula expression is:

[0132]

[0133] Among them, \(P\) k represents the point on the \(k\)-th vertebra, and the subscript 0 represents the initial state. represents the center point of the intervertebral disc corresponding to the \(k\)-th vertebra in the initial state, \(\theta\) represents the rotation angle, represents the rotation axis, represents the exponential mapping from \(so(3)\) to \(SO(3)\), and the 17 \(\theta\)s of the 17 vertebrae and together constitute the spine posture parameters.

[0134] Step 7: Construct a constraint function between the human spine model and the human back information, and a constraint function of the mechanical properties of the human spine model itself, including:

[0135] The distance constraint from the end point of the spinous process of the constructed spinal model to the dorsal spinous process line is as follows: S k is the end point of the spinous process of the spinal model, and S′ k is the point on the dorsal spinous process line that is closest to S k among them.

[0136] The local symmetry constraint for the constructed human back surface is as follows:

[0137] The mechanical property constraint of the constructed intervertebral disc is the same as the intervertebral disc mechanical model provided above.

[0138] The parameter regularization constraint for constructing the statistical model of spinal shape is: L R = ||β||;

[0139] Step 8: Based on the above-constructed constraint function, minimize the total constraint through an optimization algorithm to obtain the characteristic parameters of the statistical model, and reconstruct the three-dimensional spinal model. Among them, the constraint expression refers to the above formula (7). By optimizing the spinal pose parameter θ and the spinal shape parameter β, the total constraint function can be minimized.

[0140] In addition, corresponding to the above-provided three-dimensional spinal reconstruction method, the present invention also provides a three-dimensional spinal reconstruction system, as Figure 4 shown. This system includes:

[0141] An image acquisition module 400 for acquiring a color image and a depth image of the human back.

[0142] An image processing module 401 for processing the depth image using the light and shadow information of the color image to obtain a processed depth image.

[0143] A dorsal spinous process line determination module 402 for obtaining the dorsal spinous process line based on the color image and the processed depth image.

[0144] A local symmetry information extraction module 403 for extracting the local symmetry information of the dorsal spinous process line.

[0145] A human spinal prior model construction module 404 for constructing a human spinal prior model based on human spinal prior knowledge. The human spinal prior model includes a spinal shape statistical model, a spinal pose model, and an intervertebral disc mechanical model.

[0146] The constraint function construction module 405 is configured to construct a constraint function based on the prior model of the human spine and by using the local symmetry information of the dorsal spinous process line. The constraint function includes: the distance constraint from the end point of the spinous process to the dorsal spinous process line, the local symmetry constraint of the human back surface, the mechanical property constraint of the intervertebral disc, and the parameter regularization constraint of the spinal shape statistical model. The mechanical property constraint of the intervertebral disc includes: the constraint of the axial compression / tension property and the constraint of the torsional property about the axis.

[0147] The characteristic parameter determination module 406 is configured to determine characteristic parameters based on the constraint function.

[0148] The three-dimensional spinal model construction module 407 is configured to obtain a three-dimensional model of the spine according to the characteristic parameters.

[0149] In the present specification, the various embodiments are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other. For the system disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and reference can be made to the description in the method part for the relevant parts.

[0150] In this article, specific examples are used to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation on the present invention.

Claims

1. A three-dimensional spinal reconstruction method, characterized in that, it includes: Obtain the color image and depth image of the human back; Process the depth image using the light and shadow information of the color image to obtain the processed depth image; Obtain the dorsal spinous process line based on the color image and the processed depth image; Extract the local symmetry information of the dorsal spinous process line; Based on the prior knowledge of the human spine, construct a prior model of the human spine; the prior model of the human spine includes a spine shape statistical model, a spine posture model, and an intervertebral disc mechanics model; among them, the spine shape statistical model is: where, T k is the statistical model of the spinal shape of the k-th vertebral body, is the mean vertebral body shape of the k-th vertebral body, B k is the spinal shape parameter matrix of the k-th vertebral body, and β is the spinal shape parameter; The spine posture model is: J k = (M k-1 + M k ) / 2; where M k is the center point of the k-th vertebral body, is the first auxiliary point located on the surface of the k-th vertebral body, is the second auxiliary point located on the surface of the k-th vertebral body, J k is the center point of the intervertebral disc of the k-th vertebral body, M k-1 is the center point of the (k - 1)-th vertebral body; The intervertebral disc mechanics model is: In the formula, is the axial compression / tension characteristic of the k-th vertebral body, n is the number of deformation times, is the axial deformation amount of the k-th vertebral body at the j-th deformation, is the torsional characteristic about the axis of the k-th vertebral body, is the torsional angle of the k-th vertebral body, and λ is a constant factor; Based on the prior model of the human spine, use the local symmetry information of the dorsal spinous process line to construct a constraint function; the constraint function includes: the distance constraint from the spinous process end point to the dorsal spinous process line, the local symmetry constraint of the human back surface, the intervertebral disc mechanics property constraint, and the parameter regularization constraint of the spine shape statistical model; the intervertebral disc mechanics property constraint includes: the constraint of the axial compression / tension property and the constraint of the torsional property around the axis; among them, the constraint function is: Wherein, L is a constraint function, is the distance constraint from the end point of the spinous process of the spinal model to the k-th vertebral body on the dorsal spinous process line, is the local symmetry constraint of the human back surface of the k-th vertebral body, is the axial compression / tension characteristic of the k-th vertebral body, is the torsional characteristic about the axis of the k-th vertebral body, α 1 、α 2 and α 3 are all weight coefficients, and L R is the parameter regularization constraint of the spinal shape statistical model; Determine the characteristic parameters based on the constraint function; Obtain the three-dimensional model of the spine according to the characteristic parameters.

2. The three-dimensional spinal reconstruction method according to claim 1, characterized in that, The extraction of the local symmetry information of the dorsal spinous process line specifically includes: In the 2D image space, draw a perpendicular line to the dorsal spinous process line through each pixel point of the dorsal spinous process line; Taking each pixel point on the dorsal spinous process line in the 2D image space as the center point, intercept line segments on the perpendicular line according to preset conditions to obtain perpendicular line segments; Extend each end of each perpendicular line segment by one pixel to obtain a strip; Taking each pixel point on the dorsal spinous process line in the 2D image space as the center point, select a rectangular area at each quarter of the perpendicular line segment to obtain the first point set; Back-project the dorsal spinous process line and the strip in the 2D image space through the camera matrix to obtain the dorsal spinous process line in the 3D image space and the strip in the 3D image space; Construct an index tree of the dorsal spinous process line in the 3D image space and an index tree of the strip in the 3D image space; Construct a dictionary with the index corresponding to each pixel point on the dorsal spinous process line in the 3D image space as the key and the index tree of the strip corresponding to each pixel point on the dorsal spinous process line in the 3D image space as the value; Obtain the spinous process end point; Determine the pixel point closest to the spinous process end point on the index tree of the dorsal spinous process line in the 3D image space and obtain the index of this pixel point; Obtain the strip of the spinous process end point based on the index in the dictionary; Mirror-symmetric the pixel points in the first point set regarding the spinous process end point, the vertebral body center point, and the intervertebral disc center point to obtain the second point set; Determine the pixel points closest to the pixel points in the second point set on the index tree of the dorsal spinous process line in the 3D image space to obtain the third point set; Perform distance minimization constraint on the pixel points in the third point set to obtain the local symmetry information of the dorsal spinous process line.

3. The three-dimensional spinal reconstruction method according to claim 2, characterized in that the performing distance minimization constraint on the pixel points in the third point set to obtain the local symmetry information of the dorsal spinous process line specifically includes: Using the formula to perform distance minimization constraint on the pixel points in the third point set to obtain the local symmetry information of the dorsal spinous process line; where Q' is the pixel point in the second point set, Q'' is the pixel point in the third point set, and L symm is the local symmetry constraint value.

4. The three-dimensional spinal reconstruction method according to claim 2, characterized in that the preset condition is: the length of the intercepted line segment does not exceed one-half of the range of the dorsal region in the image, and in the direction from the lumbar vertebra to the cervical vertebra, the length of the intercepted line segment decreases.

5. The three-dimensional spinal reconstruction method according to claim 4, characterized in that the formula for the decreasing length of the intercepted line segment is: where W is half of the length of the intercepted line segment, and W 0 is half of the length of the first intercepted line segment, N is the total number of pixels of the dorsal spinous process line, and i is the i-th pixel on the dorsal spinous process line, with i increasing from the lumbar vertebra to the cervical vertebra direction.

6. A three-dimensional spinal reconstruction system, characterized in that the three-dimensional spinal reconstruction system is applied to the three-dimensional spinal reconstruction method according to any one of claims 1-5; the three-dimensional spinal reconstruction system includes: an image acquisition module for acquiring a color image and a depth image of the human back; an image processing module for processing the depth image by using the light and shadow information of the color image to obtain a processed depth image; a dorsal spinous process line determination module for obtaining a dorsal spinous process line based on the color image and the processed depth image; a local symmetry information extraction module for extracting the local symmetry information of the dorsal spinous process line; a human spinal prior model construction module for constructing a human spinal prior model based on human spinal prior knowledge; the human spinal prior model includes a spinal shape statistical model, a spinal posture model, and an intervertebral disc mechanics model; a constraint function construction module for constructing a constraint function based on the human spinal prior model by using the local symmetry information of the dorsal spinous process line; the constraint function includes: the distance constraint from the spinous process end point to the dorsal spinous process line, the local symmetry constraint of the human back surface, the intervertebral disc mechanics property constraint, and the parameter regularization constraint of the spinal shape statistical model; the intervertebral disc mechanics property constraint includes: the constraint of the axial compression / tension property and the constraint of the torsional property about the axis; a characteristic parameter determination module for determining characteristic parameters based on the constraint function; a three-dimensional spinal model construction module for obtaining a three-dimensional model of the spine according to the characteristic parameters.

Images