Spinal malformation auxiliary observation method and system

Abstract

The invention relates to the technical field of spinal deformity auxiliary observation, and discloses a spinal deformity auxiliary observation method, which comprises the following steps: S1, fixedly erecting a plurality of cameras in an evaluation site, calibrating the cameras, and collecting a multi-view video image of an evaluated person; s2, background filtering processing is carried out on the multi-view image, and a pure human body image is extracted; and S3, generating a 3D human body shape model by using the pure human body images under each view angle. Through a multi-camera dynamic capture technology, the back and spine forms of an assessed person can be observed in all directions in the dynamic processes of walking, stooping and the like, compared with traditional static image assessment, the dynamic observation mode can provide more comprehensive and three-dimensional data, doctors are helped to accurately assess the changes of the spine forms, and the assessment efficiency is improved. Therefore, the accuracy and comprehensiveness of diagnosis are improved, a doctor can better understand the spine structure of a patient through multi-angle and multi-action capture, and more effective support is provided for diagnosis and treatment of spine malformation.

Description

Technical Field

[0001] The present invention relates to the technical field of spinal deformity auxiliary observation, and in particular to a spinal deformity auxiliary observation method and system. Background Art

[0002] Spinal morphological abnormalities such as scoliosis and lordosis are currently mainly evaluated by obtaining static spinal skeletal images through X-ray irradiation, and professional doctors observe and evaluate possible pathological abnormalities. Since the radioactivity of X-rays has a certain impact on human health and is not convenient for large-scale screening, some evaluation methods have emerged in recent years by taking or scanning static images, such as CN113951874B, CN109785297B, etc., which can facilitate doctors to perform screening without affecting the health of the person being evaluated. However, such methods can only be analyzed and evaluated based on static images. The observation angle and the singleness of the subject's movements may affect the doctor's comprehensive evaluation and accurate judgment. There is an urgent need for a method and system that can help doctors observe the back and spinal morphology of the person being evaluated in all directions during dynamic processes such as walking and bending. Summary of the invention

[0003] In view of the deficiencies of the prior art, the present invention provides a spinal deformity auxiliary observation method and system, which solves the problems mentioned in the background technology.

[0004] To achieve the above objectives, the present invention is implemented through the following technical solutions: a spinal deformity auxiliary observation method, comprising the following steps: S1. Fixedly set up multiple cameras at the assessment site, calibrate the cameras, and collect multi-view video images of the person being assessed; S2, performing background filtering on multi-view images to extract pure human body images; S3, generating a 3D human body shape model using pure human body images at various viewing angles; S4, predicting a 3D human spine model based on a 3D human body shape model; S5. Render and display the 3D human body shape model and the 3D human spine model.

[0005] Preferably, in step S1, the number of the multiple cameras is not less than 4, arranged in a ring or semi-ring array, and the field of view of each camera covers the center of the evaluation site.

[0006] Preferably, the step S1 further comprises: calibrating the camera involves calibration of camera internal parameters and external parameters, the internal parameters include focal length, principal point coordinates and pixel size, and the external parameters include the position and orientation of the camera. The calibration process adopts a standard chessboard, and finely adjusts the calibration result in combination with an optimization algorithm, and its function expression is: ; in, It is The parameter vector of the step, including the camera internal and external parameters; is the step size, which controls the distance to advance in the search direction; It is The gradient of the step, which represents the objective function in the parameter vector The gradient at is the search direction of the previous step; is the correction factor to ensure the search direction Same as the previous search direction Conjugation.

[0007] Preferably, in step S2, the background filtering process includes a background difference algorithm: collecting a background image of the site without anyone, subtracting it from the evaluation image, and combining morphological filtering to remove noise and fill holes to obtain a human body image outside the background; its main steps are: collecting a background image of the site without anyone ; Then evaluate the image Subtract the background difference image , ; The background difference image The pixel value and threshold Compare and get a rough human mask ; Mask for human body Morphological filtering is performed to obtain a fine human mask, thereby obtaining the human image outside the background.

[0008] Preferably, step S2 further comprises: a semantic segmentation algorithm based on deep learning: using a trained semantic segmentation model to perform pixel-level segmentation on the evaluation image to separate the human body from the background, thereby obtaining a pure human body image, and its function expression is: ,in is the input image data, is a function representing a deep learning model, with parameters , It is the segmentation result output by the model, which is a matrix of the same size as the input image. Each pixel contains a category label to indicate the category to which the pixel belongs (human body, background).

[0009] Preferably, in step S3, each viewing angle refers to a different imaging angle at which a plurality of cameras fixedly mounted at the evaluation site photograph the evaluation object located at the evaluation site.

[0010] Preferably, the method of obtaining the 3D human body shape model in step S3 includes one or more of the following methods: Multi-view stereo matching plus dense reconstruction: Reconstruct 3D point clouds from human body images from different perspectives through feature matching and triangulation principles; Reconstruct a dense 3D human surface model from a sparse point cloud using algorithms such as PMVS (Patch-based Multi-view Stereo) or CPCR (Colored Point Cloud Registration); Furthermore, the above process can be expressed as: For the feature matching and triangulation phase: ; in, is the calculated 3D point cloud data; are feature points extracted from different perspectives; It is a triangulation function used to calculate 3D point cloud; For the dense reconstruction phase: ; in, It is a dense 3D human body surface model; It is the sparse 3D point cloud data obtained in the previous stage; It is a dense reconstruction function, which is used to reconstruct a complete 3D human body surface model based on sparse 3D point cloud data; Deep learning multi-view 3D reconstruction algorithm: Including PIFu (Pixel-aligned Implicit function), PIFuHD, SMPL (Skinned Multi-Person Linear Model); The main steps include data acquisition, sparse reconstruction, depth map estimation, dense reconstruction, and depth optimization iteration.

[0011] Preferably, in step S4, predicting the 3D human spine model by using the 3D human body shape model comprises the following steps: Data collection and preprocessing: Collect human CT data covering the entire back and spine region; Extracting back epidermal voxels and spine voxels from CT data, or converting an anatomical 3D model into back epidermal voxels and spine voxels; The back epidermal voxels are used as input data and the spine voxels are used as expected output data, and the data are divided into training set, validation set and test set; Network model selection and training: Choose 3D CNN and 3D Unet, which are network architectures specifically designed to process 3D data; Define the number of network layers and width, and use IoU and Dice coefficient as loss functions; Use the training set data to train the neural network model; Here are the function expressions for these steps: Data preprocessing stage: ; in, It is human CT data; are the back epidermal voxel and spine voxel data of the training set; It is the back epidermal voxel and spine voxel data of the validation set; It is the back epidermal voxel and spine voxel data of the test set; It is a data preprocessing function, which is used to extract back epidermal voxels and spine voxels from CT data and divide the data set; Network model training phase: ; in, It is a trained neural network model; are the back epidermal voxel and spine voxel data of the training set; Including the number of network layers and width parameters; It is a training function used to train the neural network model, using IoU and Dice coefficient as loss function.

[0012] Preferably, in step S4, further, after the 3D human spine model training is completed, in the inference stage, the back epidermis part is separated from the input 3D human body shape model, and converted into back epidermis voxels through the marching cube algorithm, and then the back epidermis voxels are input into the 3D human spine model to obtain the output of the spine voxels, and finally the spine voxels are converted into the 3D human spine model through the isosurface extraction algorithm. The following are expressions of the marching cube algorithm and the isosurface extraction algorithm: Marching cube algorithm: ; in, is the input voxel data, representing the dorsal epidermis; is the value of the isosurface; It is a polygonal mesh model generated by the marching cube algorithm; Isosurface extraction algorithm: ; in, It is the spinal voxel data predicted by the 3D human spine model; It is a 3D human spine model generated by isosurface extraction algorithm.

[0013] The spinal deformity auxiliary observation system includes the following modules: A video acquisition module is used to obtain a video of the evaluation object in a static or moving state in the evaluation site; A human body pure image extraction module is used to run the method for background filtering of multi-view images in step S2 to extract pure human body images from videos in a static or moving state; A 3D human body shape model synthesis module is used to run the 3D human body shape model synthesis algorithm in step S3 to synthesize a 3D human body shape model from pure human body images from multiple perspectives; A 3D human spine model prediction module is used to run the deep neural network in step S4 to predict the 3D human spine model from the 3D human body shape model; 3D human body shape model and 3D human spine model rendering and display module; used to render 3D human body shape model and 3D human spine model, visualize 3D human body shape model and 3D human spine model on the interactive interface, so as to facilitate doctors to playback, watch and evaluate, and extract pure human body images from static or moving videos.

[0014] The present invention provides a spinal deformity auxiliary observation method and system, which has the following beneficial effects: 1. The present invention uses multi-camera dynamic capture technology to achieve all-round observation of the back and spinal morphology of the person being evaluated during dynamic processes such as walking and bending. Compared with traditional static image evaluation, this dynamic observation method can provide more comprehensive and three-dimensional data, helping doctors to accurately evaluate changes in spinal morphology, thereby improving the accuracy and comprehensiveness of diagnosis. By using multi-angle and multi-action capture, doctors can better understand the patient's spinal structure and provide more effective support for the diagnosis and treatment of spinal deformities.

[0015] 2. The present invention adopts 3D human body shape model synthesis and spinal model prediction technology, which can convert data captured from multiple perspectives into a three-dimensional 3D model, providing doctors with a more intuitive observation tool. The three-dimensional display method can help doctors better understand the patient's spinal structure and promote the diagnosis and treatment of spinal deformities.

[0016] 3. The present invention can extract pure human body images through background filtering processing and deep learning algorithms, achieve accurate display of the back and spine of the person being evaluated, avoid the problem of background interference in traditional images, allow doctors to focus on observing the human body structure, and help discover potential spinal deformities or abnormalities.

[0017] 4. The systematic design of the present invention, including modules such as video acquisition, image processing, model synthesis and rendering display, provides doctors with a comprehensive auxiliary observation tool. By integrating multiple technical means, the system can realize the complete process from dynamic data to 3D model, providing a more comprehensive and convenient solution for the diagnosis and observation of spinal deformities. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a schematic diagram of the process of the present invention; Figure 2 It is a system flow chart of the present invention. DETAILED DESCRIPTION

[0019] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0020] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0021] The present invention is further described in detail below in conjunction with the accompanying drawings: For examples, please see the attached Figure 1-2 The embodiment of the present invention provides a spinal deformity auxiliary observation method, comprising the following steps: S1. Fixedly set up multiple cameras at the assessment site, calibrate the cameras, and collect multi-view video images of the person being assessed. The number of multiple cameras shall not be less than 4, arranged in a circular or semi-circular array, and the field of view of each camera shall cover the center of the assessment site; S2, performing background filtering on multi-view images to extract pure human body images; S3, generating a 3D human body shape model using pure human body images at various viewing angles, where various viewing angles refer to different imaging angles at which multiple cameras fixed at the evaluation site shoot the evaluation object at the evaluation site; S4, predicting a 3D human spine model based on a 3D human body shape model; S5. Render and display the 3D human body shape model and the 3D human spine model.

[0022] Step S1 further includes: calibrating the camera involves calibration of the camera's internal and external parameters. The internal parameters include focal length, principal point coordinates, and pixel size. The external parameters include the position and orientation of the camera. The calibration process uses a standard chessboard and combines an optimization algorithm to fine-tune the calibration result. The function expression is: , in, It is The parameter vector of the step, including the camera internal and external parameters; is the step size, which controls the distance to advance in the search direction; It is The gradient of the step, which represents the objective function in the parameter vector The gradient at is the search direction of the previous step; is the correction factor to ensure the search direction Same as the previous search direction Conjugation.

[0023] In step S2, the background filtering process includes a background difference algorithm: collecting the background image of the site without people, subtracting it from the evaluation image, and combining morphological filtering to remove noise and fill holes to obtain the human body image outside the background; its main steps are: collecting the background image of the site without people ; Then evaluate the image Subtract the background difference image , ; The background difference image The pixel value and threshold Compare and get a rough human mask ; Mask for human body Morphological filtering is performed to obtain a fine human mask, thereby obtaining the human image outside the background.

[0024] Step S2 further includes: a semantic segmentation algorithm based on deep learning: using a trained semantic segmentation model to perform pixel-level segmentation on the evaluation image to separate the human body from the background, thereby obtaining a pure human body image, and its function expression is: ,in is the input image data, is a function representing a deep learning model, with parameters , It is the segmentation result output by the model, which is a matrix of the same size as the input image. Each pixel contains a category label to indicate the category to which the pixel belongs (human body, background).

[0025] The method of obtaining the 3D human body shape model in step S3 includes one or more of the following methods: Multi-view stereo matching plus dense reconstruction: Reconstruct 3D point clouds from human body images from different perspectives through feature matching and triangulation principles; Reconstruct a dense 3D human surface model from a sparse point cloud using algorithms such as PMVS (Patch-based Multi-view Stereo) or CPCR (Colored Point Cloud Registration); Furthermore, the above process can be expressed as: For the feature matching and triangulation phase: ; in, is the calculated 3D point cloud data; are feature points extracted from different perspectives; It is a triangulation function used to calculate 3D point cloud; For the dense reconstruction phase: ; in, It is a dense 3D human body surface model; It is the sparse 3D point cloud data obtained in the previous stage; It is a dense reconstruction function, which is used to reconstruct a complete 3D human body surface model based on sparse 3D point cloud data; Deep learning multi-view 3D reconstruction algorithms: including PIFu (Pixel-aligned Implicit function), PIFuHD, and SMPL (Skinned Multi-Person Linear Model); The main steps include data acquisition, sparse reconstruction, depth map estimation, dense reconstruction, and depth optimization iteration.

[0026] In step S4, predicting the 3D human spine model by using the 3D human body shape model includes the following steps: Data collection and preprocessing: Collect human CT data covering the entire back and spine region; Extracting back epidermal voxels and spine voxels from CT data, or converting an anatomical 3D model into back epidermal voxels and spine voxels; The back epidermal voxels are used as input data and the spine voxels are used as expected output data, and the data are divided into training set, validation set and test set; Network model selection and training: Choose 3D CNN and 3D Unet, which are network architectures specifically designed to process 3D data; Define the number of network layers and width, and use IoU and Dice coefficient as loss functions; Use the training set data to train the neural network model; Here are the function expressions for these steps: Data preprocessing stage: ; in, It is human CT data; are the back epidermal voxel and spine voxel data of the training set; It is the back epidermal voxel and spine voxel data of the validation set; It is the back epidermal voxel and spine voxel data of the test set; It is a data preprocessing function, which is used to extract back epidermal voxels and spine voxels from CT data and divide the data set; Network model training phase: ; in, It is a trained neural network model; are the back epidermal voxel and spine voxel data of the training set; Including the number of network layers and width parameters; It is a training function used to train the neural network model, using IoU and Dice coefficient as loss function.

[0027] In step S4, further, after the 3D human spine model training is completed, in the inference stage, the back epidermis part is separated from the input 3D human body shape model, and converted into back epidermis voxels through the marching cube algorithm, and then the back epidermis voxels are input into the 3D human spine model to obtain the output of the spine voxels, and finally the spine voxels are converted into the 3D human spine model through the isosurface extraction algorithm. The following are the expressions of the marching cube algorithm and the isosurface extraction algorithm: Marching cube algorithm: ; in, is the input voxel data, representing the dorsal epidermis; is the value of the isosurface; It is a polygonal mesh model generated by the marching cube algorithm; Isosurface extraction algorithm: ;

[0028] in, It is the spinal voxel data predicted by the 3D human spine model; It is a 3D human spine model generated by isosurface extraction algorithm.

[0029] In step S5, the 3D human body shape model and the 3D human spine model are rendered and displayed, including presenting the dynamic process of the person being evaluated through a 3D engine and displaying it through a rendered video, and supporting switching of any observation angle during video playback, as well as adjusting the transparency of the 3D human body shape model or the 3D human spine model during video playback.

[0030] The spinal deformity auxiliary observation system includes the following modules: A video acquisition module is used to obtain a video of the evaluation object in a static or moving state in the evaluation site; A human body pure image extraction module is used to run the method for background filtering of multi-view images in step S2 to extract pure human body images from videos in a static or moving state; A 3D human body shape model synthesis module is used to run the 3D human body shape model synthesis algorithm in step S3 to synthesize a 3D human body shape model from pure human body images from multiple perspectives; A 3D human spine model prediction module is used to run the deep neural network in step S4 to predict the 3D human spine model from the 3D human body shape model; 3D human body shape model and 3D human spine model rendering and display module; used to render 3D human body shape model and 3D human spine model, visualize 3D human body shape model and 3D human spine model on the interactive interface, so as to facilitate doctors to playback, watch and evaluate, and extract pure human body images from static or moving videos.

[0031] The following are specific embodiments of the present invention:

[0032] Embodiment 1: step: S1. Six cameras are fixed in a circular arrangement on a 4m×2m wall at the assessment site, ensuring that the field of view of each camera covers the center of the assessment site; S2. Calibrate the camera's intrinsic parameters (focal length, principal point coordinates, and pixel size) and extrinsic parameters (position and orientation) using the Zhang Zhengyou calibration method; S3, background filtering of multi-view video images, using a semantic segmentation algorithm based on deep learning; S4, obtaining a 3D human body shape model through pure human body images at various viewing angles; S5, predicting 3D human spine model, using a pre-trained deep neural network model; S6. Render and display the 3D human body model and the 3D human spine model.

[0033] Summary: This embodiment achieves comprehensive observation and assessment of the spinal condition of the person being assessed through multi-camera shooting, background filtering, 3D human body model synthesis and spinal model prediction.

[0034] Embodiment 2: step: S1. 8 cameras are fixedly installed at the assessment site, covering every corner and center of the assessment site; S2, using background difference algorithm to perform background filtering on multi-view images; S3, using multi-view stereo matching and dense reconstruction method to obtain a 3D human body shape model; S4, predicting 3D human spine model using deep learning-based multi-view 3D reconstruction algorithm; S5. Render and display the 3D human body model and the 3D human spine model.

[0035] Summary: This embodiment improves the accuracy and comprehensiveness of spinal conditions through more cameras, different background filtering algorithms and reconstruction methods.

[0036] Embodiment 3: step: S1. 10 cameras are fixedly installed at the assessment site, covering every corner and central area of ​​the assessment site; S2, using the semantic segmentation algorithm based on deep learning in the background filtering processing method; S3, using a deep learning-based multi-view 3D reconstruction algorithm to obtain a 3D human body shape model; S4, predict 3D human spine model, using algorithms such as SMPL (Skinned Multi-Person Linear Model); S5. Use the 3D engine to render and display the dynamic process of the evaluation object, supporting multi-angle observation and transparency adjustment.

[0037] Summary: This embodiment provides a more intuitive and comprehensive spinal observation and assessment method through more cameras, deep learning algorithms and 3D engine technology.

[0038] Embodiment 4: step: S1. Five cameras are fixed at the assessment site to cover the main areas of the assessment site; S2, use background subtraction algorithm and deep learning multi-view 3D reconstruction algorithm to process images; S3, obtaining a 3D human body shape model through multi-view stereo matching and dense reconstruction method; S4, predicting 3D human spine model using a custom deep neural network model; S5. Render and display the 3D human body model and the 3D human spine model.

[0039] Summary: This embodiment combines different algorithms and a small number of cameras to achieve observation and assessment of spinal conditions.

[0040] Embodiment 5: step: S1. 7 cameras are fixedly installed at the assessment site to cover all angles of the assessment site; S2, using the background difference algorithm in the background filtering processing method; S3, using Poisson surface reconstruction technology to smooth and refine the 3D human body surface model; S4. Predict 3D human spine model using deep neural network model; S5. Render and display the 3D human body model and the 3D human spine model through a 3D engine.

[0041] Summary: This embodiment provides a more accurate and intuitive way to observe the spine through different camera arrangements, background filtering algorithms and reconstruction techniques.

[0042] Summary: The above five embodiments combine different camera arrangements, background filtering processing, 3D reconstruction technology and prediction methods to provide doctors with comprehensive and intuitive support for observing and evaluating the spinal condition of the person being evaluated.

[0043] Test Experiment Design: Purpose of the experiment: To evaluate the effectiveness of the system in supporting the observation and playback of the spinal deformity of the person being evaluated from any viewing angle.

[0044] Experimental parameter settings: Experimental environment: simulated spinal deformity observation system; Observation angle: front, side, back; State: still, moving; Transparency: 0.3, 0.5, 0.7.

[0045] Comparative test: Experiment 1: Effects of different viewing angles on spinal deformity assessment Experimental design: The doctor observes the spinal deformity of the person being evaluated from different angles; Observation angles include front, side, back and other angles; Doctors score spinal deformities based on the degree of curvature and the degree of deformation.

[0046] Observation angle Score 1 (degree of curvature) Rating 2 (deformation) Overall Rating front 8.5 7.8 8.2 side 7.9 8.3 8.1 back 8.2 8.0 8.1 Experiment 2: Comparison of spinal deformity observations in static and dynamic states Experimental design: Compare the observed effects of spinal deformity of the assessed subjects in static state and in motion state; Observe the details and changes of spinal deformity in static state and the effects in motion; Doctors scored and compared the spinal deformity in the two conditions.

[0047] state Rating 1 (Detailed Observation) Rating 2 (Change Observation) Overall Rating Static state 8.4 8.0 8.2 Movement status 7.8 7.5 7.7 Experiment 3: Observation and comparison of the spine model at different transparencies Experimental design: Observe the effect of the spine model at different levels of transparency; The transparency is set to different values ​​such as 0.3, 0.5, and 0.7; Doctors scored and compared the effects of observation under different transparencies.

[0048] transparency Rating 1 (Clarity) Rating 2 (Detailed observation) Overall Rating 0.3 8.2 7.9 8.0 0.5 8.0 8.1 8.1 0.7 7.8 8.0 7.9 Summary: Based on the above experimental results, the system has shown good functions in supporting the observation and playback of spinal deformities of the person being evaluated at rest or in motion from any perspective. Doctors can use the multi-angle observation, static and dynamic comparison, and transparency adjustment functions provided by the system to comprehensively and intuitively evaluate the spinal condition of the person being evaluated, providing more accurate support for diagnosis and treatment.

[0049] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for assisting observation of spinal deformity, characterized in that: The following steps are involved: S1. Fixedly set up multiple cameras at the assessment site, calibrate the cameras, and collect multi-view video images of the person being assessed; S2, performing background filtering on multi-view images to extract pure human body images; S3, generating a 3D human body shape model using pure human body images at various viewing angles; S4, predicting a 3D human spine model based on a 3D human body shape model; S5. Render and display the 3D human body shape model and the 3D human spine model.

2. The spinal deformity auxiliary observation method according to claim 1, characterized in that: In step S1, the number of the multiple cameras is not less than 4, arranged in a ring or semi-ring array, and the field of view of each camera covers the center of the evaluation site.

3. The spinal deformity auxiliary observation method according to claim 1, characterized in that: The step S1 further includes: calibrating the camera involves calibration of camera internal parameters and external parameters, the internal parameters include focal length, principal point coordinates and pixel size, and the external parameters include the position and orientation of the camera. The calibration process uses a standard chessboard and combines the optimization algorithm to fine-tune the calibration result. The function expression is: ; in, It is The parameter vector of the step, including the camera internal and external parameters; is the step size, which controls the distance to advance in the search direction; It is The gradient of the step, which represents the objective function in the parameter vector The gradient at is the search direction of the previous step; is the correction factor to ensure the search direction Same as the previous search direction Conjugation.

4. The spinal deformity auxiliary observation method according to claim 1, characterized in that: In step S2, the background filtering process includes a background difference algorithm: collecting a background image of the site without people, subtracting it from the evaluation image, and combining morphological filtering to remove noise and fill holes to obtain a human body image outside the background; The main steps are: collecting background images of the site without anyone around ; Then evaluate the image Subtract the background difference image , ; Background difference image The pixel value and threshold Compare and get a rough human body mask ; Mask for human body Morphological filtering is performed to obtain a fine human mask, thereby obtaining the human image outside the background.

5. The spinal deformity auxiliary observation method according to claim 1, characterized in that: The step S2 further includes: a semantic segmentation algorithm based on deep learning: using a trained semantic segmentation model to perform pixel-level segmentation on the evaluation image to separate the human body from the background, thereby obtaining a pure human body image, and its function expression is: ,in is the input image data, is a function representing a deep learning model, with parameters , It is the segmentation result output by the model, which is a matrix of the same size as the input image. Each pixel contains a category label to indicate the category to which the pixel belongs (human body, background).

6. The spinal deformity auxiliary observation method according to claim 1, characterized in that: In step S3, each viewing angle refers to a different imaging angle at which a plurality of cameras fixedly mounted at the evaluation site photograph the evaluation object located at the evaluation site.

7. The spinal deformity auxiliary observation method according to claim 1, characterized in that: The method of obtaining the 3D human body shape model in step S3 includes one or more of the following methods: Multi-view stereo matching plus dense reconstruction: Reconstruct 3D point clouds from human body images from different perspectives through feature matching and triangulation principles; Reconstruct a dense 3D human surface model from a sparse point cloud using algorithms such as PMVS (Patch-based Multi-view Stereo) or CPCR (Colored Point Cloud Registration); Furthermore, the above process can be expressed as: For the feature matching and triangulation phase: ; in, is the calculated 3D point cloud data; are feature points extracted from different perspectives; It is a triangulation function used to calculate 3D point cloud; For the dense reconstruction phase: ; in, It is a dense 3D human body surface model; It is the sparse 3D point cloud data obtained in the previous stage; It is a dense reconstruction function, which is used to reconstruct a complete 3D human body surface model based on sparse 3D point cloud data; Deep learning multi-view 3D reconstruction algorithm: Including PIFu (Pixel-aligned Implicit function), PIFuHD, SMPL (Skinned Multi-Person Linear Model); The main steps include data acquisition, sparse reconstruction, depth map estimation, dense reconstruction, and depth optimization iteration.

8. The spinal deformity auxiliary observation method according to claim 1, characterized in that: In the step S4, predicting the 3D human spine model by using the 3D human body shape model comprises the following steps: Data collection and preprocessing: Collect human CT data covering the entire back and spine region; Extracting back epidermal voxels and spine voxels from CT data, or converting an anatomical 3D model into back epidermal voxels and spine voxels; The back epidermal voxels are used as input data and the spine voxels are used as expected output data, and the data are divided into training set, validation set and test set; Network model selection and training: Choose 3D CNN and 3D Unet, which are network architectures specifically designed to process 3D data; Define the number of network layers and width, and use IoU and Dice coefficient as loss functions; Use the training set data to train the neural network model; Here are the function expressions for these steps: Data preprocessing stage: ; in, It is human CT data; are the back epidermal voxel and spine voxel data of the training set; It is the back epidermal voxel and spine voxel data of the validation set; It is the back epidermal voxel and spine voxel data of the test set; It is a data preprocessing function, which is used to extract back epidermal voxels and spine voxels from CT data and divide the data set; Network model training phase: ; in, It is a trained neural network model; are the back epidermal voxel and spine voxel data of the training set; Including the number of network layers and width parameters; It is a training function used to train the neural network model, using IoU and Dice coefficient as loss function.

9. The spinal deformity auxiliary observation method and system according to claim 1, characterized in that: In the step S4, further, after the 3D human spine model training is completed, in the inference stage, the back epidermis part is separated from the input 3D human body shape model, and converted into back epidermis voxels through the marching cube algorithm, and then the back epidermis voxels are input into the 3D human spine model to obtain the output of the spine voxels, and finally the spine voxels are converted into the 3D human spine model through the isosurface extraction algorithm. The following are the expressions of the marching cube algorithm and the isosurface extraction algorithm: Marching cube algorithm: ; in, is the input voxel data, representing the dorsal epidermis; is the value of the isosurface; It is a polygonal mesh model generated by the marching cube algorithm; Isosurface extraction algorithm: ; in, It is the spinal voxel data predicted by the 3D human spine model; It is a 3D human spine model generated by isosurface extraction algorithm.

10. A spinal deformity auxiliary observation system, according to the spinal deformity auxiliary observation method according to any one of claims 1 to 10, characterized in that: Includes the following modules: A video acquisition module is used to obtain a video of the evaluation object in a static or moving state in the evaluation site; A human body pure image extraction module is used to run the method for background filtering of multi-view images in step S2 to extract pure human body images from videos in a static or moving state; A 3D human body shape model synthesis module is used to run the 3D human body shape model synthesis algorithm in step S3 to synthesize a 3D human body shape model from pure human body images from multiple perspectives; A 3D human spine model prediction module is used to run the deep neural network in step S4 to predict the 3D human spine model from the 3D human body shape model; 3D human body shape model and 3D human spine model rendering display module; It is used to render 3D human body shape model and 3D human spine model, visualize the 3D human body shape model and 3D human spine model on the interactive interface, which is convenient for doctors to playback, watch and evaluate, and extract pure human body images from static or moving videos.

Images